Rust: What the fuck is a borrow checker and why is it yelling at me?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »

Vanadium: Jan 8, 2005

Don't think there's a great way. In C you'd just take an out-pointer and then just return a flag, but that's not gonna look particularly nice in either Rust or C#.

# ? Mar 13, 2016 23:36

Adbot: ADBOT LOVES YOU

# ? Apr 25, 2024 08:55

VikingofRock: Aug 24, 2008

gonadic io posted:

Is this a reasonable way to pass an optional value over a FFI boundary? Passing a (nullable) pointer and then having the C# code calling another function to free it seems like lots of pain.
code:
#[repr(C)]
pub struct BadOption<T : Sized> {
	is_some : bool,
	value : T
}
e: here's the context.

That seems pretty reasonable to me--in fact, I think that that's how an Option would be laid out if you put #[repr(C)] on it. (The piece I'm not sure about is whether it would use a bool as the discriminant.)

# ? Mar 14, 2016 02:15

gonadic io: Feb 16, 2011; >>=

I have a weird and pretty gross function that I can't help but think "surely there must be a better way!" about :

code:

pub enum SVO {
    Voxel { voxel_type: i32,
            external_id: Cell<Option<u32>> },
    Octants ([Box<SVO>; 8]) 
}

impl SVO {
    pub fn new_octants(octant_types: [i32; 8]) -> SVO { ... }

    fn subdivide_voxel<D>(&mut self, deregister_voxel: &D) where D : Fn(u32) {
        match *self {
            SVO::Voxel{ ref external_id, .. } => {
                for voxel_id in external_id.get() {
                    deregister_voxel(voxel_id)
                };
            },
            _ => panic!("subdivide_voxel called on a non-voxel!")
        }


        match *self {
            SVO::Voxel { voxel_type, .. } => {
                *self = SVO::new_octants([voxel_type, voxel_type, voxel_type, voxel_type,
                                          voxel_type, voxel_type, voxel_type, voxel_type])
            },
            _ => panic!("subdivide_voxel called on a non-voxel!")
        }
    }
}

This currently works (or at least compiles), and the reason that is has to be two match statements is that one value is captured by ref and one by value. However external_id can't be in scope when we match on voxel_type because we overwrite the contents of *self. Hence one match to capture external_id and then another to match voxel_type (but not to capture external_id).

As ever, the context is here.

# ? Mar 19, 2016 01:31

syncathetic: Oct 21, 2010

Why not this?

code:


    fn subdivide_voxel<D>(&mut self, deregister_voxel: &D) where D : Fn(u32) {
        *self = match self {
            &mut SVO::Voxel{ ref external_id, voxel_type } => {
                for voxel_id in external_id.get() {
                    deregister_voxel(voxel_id)
                };
                SVO::new_octants([voxel_type, voxel_type, voxel_type, voxel_type,
                                  voxel_type, voxel_type, voxel_type, voxel_type])
            },
            _ => panic!("subdivide_voxel called on a non-voxel!")
        }

# ? Mar 19, 2016 05:41

gonadic io: Feb 16, 2011; >>=

Perfect, thanks.

e: is there a better way to represent once-assignable nullable variables than Cell<Option<_>>? The type allows for re-assignation but I will never do that. I suppose I could always just put a wrapper around it.

gonadic io fucked around with this message at 11:26 on Mar 19, 2016

# ? Mar 19, 2016 10:52

VikingofRock: Aug 24, 2008

gonadic io posted:

Perfect, thanks.

e: is there a better way to represent once-assignable nullable variables than Cell<Option<_>>? The type allows for re-assignation but I will never do that. I suppose I could always just put a wrapper around it.

Well if you really want to get fancy, you could do something like what hyper (an http library) does for Responses, and verify this at type level at compile time. Here's an example of what that might look like for this case.

edit: To be a bit more explicit (and maybe save you some reading), the idea is to make a Variable type, which is parameterized over a type that can be either Unassigned or Assigned. Then you implement an assign() function for Variable<Unassigned>, which consumes the Variable<Unassigned> and returns a Variable<Assigned>. Because of this consuming, you can guarantee that a Variable can be assigned at most once, and because there is no assign() function for Variable<Assigned>, you can guarantee that you can't re-assign a variable once it has been assigned.

VikingofRock fucked around with this message at 07:04 on Mar 21, 2016

# ? Mar 21, 2016 06:49

VikingofRock: Aug 24, 2008

Actually now that I think about it a little more, you probably want this to work at run-time, not compile time. You can either use the above approach and Any to do run-time type checking, or you could just write a wrapper around an Option (which is a lot simpler, but loses the ability to do compile-time checking). I guess it depends on your use case (I haven't really read through your code).

# ? Mar 22, 2016 00:39

Linear Zoetrope: Nov 28, 2011; A hero must cook

VikingofRock posted:

Well if you really want to get fancy, you could do something like what hyper (an http library) does for Responses, and verify this at type level at compile time. Here's an example of what that might look like for this case.

edit: To be a bit more explicit (and maybe save you some reading), the idea is to make a Variable type, which is parameterized over a type that can be either Unassigned or Assigned. Then you implement an assign() function for Variable<Unassigned>, which consumes the Variable<Unassigned> and returns a Variable<Assigned>. Because of this consuming, you can guarantee that a Variable can be assigned at most once, and because there is no assign() function for Variable<Assigned>, you can guarantee that you can't re-assign a variable once it has been assigned.

Though the immediately obvious problem that jumps out at me with this scheme is it makes loops problematic if you need to assign in those.

code:

let mut x = item::<Unassigned>::new(stuff, Unassigned)

loop {
    do_stuff();
    if some_condition() {
        x = item::<Assigned>::new(stuff, Assigned(thing));
        break;
    }
}

Won't work for typing reasons.

# ? Mar 22, 2016 02:21

VikingofRock: Aug 24, 2008

Jsor posted:

Though the immediately obvious problem that jumps out at me with this scheme is it makes loops problematic if you need to assign in those.
code:
let mut x = item::<Unassigned>::new(stuff, Unassigned)

loop {
    do_stuff();
    if some_condition() {
        x = item::<Assigned>::new(stuff, Assigned(thing));
        break;
    }
}
Won't work for typing reasons.

It's not just in loops--you can't modify x in place because when you assign you change the type. So when you modify it you need to do something like what I did in my linked example code on like 42. That will work in loops (so long as you only assign once), but yeah it's pretty awkward. So maybe writing a wrapper is nicer anyways. It is a neat trick though--I was blown away when I saw it in hyper.

edit: slightly better example of loops: http://is.gd/6i8ELg

VikingofRock fucked around with this message at 04:19 on Mar 22, 2016

# ? Mar 22, 2016 04:14

Linear Zoetrope: Nov 28, 2011; A hero must cook

Oh yeah, you can do that. But I mentioned loops specifically because for most other constructs you can use a let rebind pattern:

code:

let x = Item::<Unassigned>::new(stuff, Unassigned)

let x = if conditional {
   x.assign(whatever)
} else {
   x.assign(something_else)
};

For instance.

I don't know if it's a good pattern, but I use it semi-frequently to create an "x prime" variable where you're deriving a value from some earlier value that's never going to be used again.

Linear Zoetrope fucked around with this message at 04:37 on Mar 22, 2016

# ? Mar 22, 2016 04:33

VikingofRock: Aug 24, 2008

Jsor posted:

Oh yeah, you can do that. But I mentioned loops specifically because for most other constructs you can use a let rebind pattern:
code:
let x = Item::<Unassigned>::new(stuff, Unassigned)

let x = if conditional {
   x.assign(whatever)
} else {
   x.assign(something_else)
};
For instance.

I don't know if it's a good pattern, but I use it semi-frequently to create an "x prime" variable where you're deriving a value from some earlier value that's never going to be used again.

I often do something similar to "freeze" a variable. It's useful when there is some non-trivial initialization to a variable, but then the variable will not be mutated after the initialization. Like this:

code:

let mut v = Vec::new();
for i in 0..10 {
    v.push(i);
}
let v = v;
// ... do stuff with immutable v ...

# ? Mar 22, 2016 05:05

gonadic io: Feb 16, 2011; >>=

Hi all, thanks loads for your previous help. I'm having another fight with the borrow checker again though.

In short, what I want to do is this:

code:

fn with_vec(vec: &mut Vec<u8>) {
    let octant_indices = octants.iter().map(|octant| {
        vec.add(???); // some operation that requires vec to be &mut
        vec.len() as u64 - 1
    });

    vec.add(???); // some operation that also requires vec to be &mut
}

And I'm getting the error "cannot borrow `*vec` as mutable because previous closure requires unique access [E0501]". I understand why I'm getting the error, as there are lots of mutable references to the same vec in scope and Mr B Checker is not a fan of that. I don't understand how to get around this though - the closure absolutely needs mutable access to the full vector and I absolutely need to be able to use it mutably afterwards.

Full context is here.

gonadic io fucked around with this message at 23:47 on Mar 22, 2016

# ? Mar 22, 2016 23:29

syncathetic: Oct 21, 2010

I think your problem comes from Iterator::map being lazy. If the the code in your post could be executed, the code inside the closure would never run. If you want to append something to vec after the closure is evaluated and still use its result, you need to do something like:

code:

fn with_vec(vec: &mut Vec<u8>) {
    let octant_indices = Vec::from_iter(octants.iter().map(|octant| {
        vec.add(???);
        vec.len() as u64 - 1
    }));

    vec.add(???);

    for octant_index in octant_indices {
        //....
    }
}

# ? Mar 23, 2016 00:15

gonadic io: Feb 16, 2011; >>=

Yep, that was exactly it thanks.

So even though the closure is lexically out of scope, because the iterator hasn't been evaluated yet the compiler still considers it to be in scope for the purposes of borrowing?

E: I think I get it now - the iterator borrows the reference because the closure does, and the iterator is still in scope (regardless of how much it was evaluated) which is causing the problem.

gonadic io fucked around with this message at 10:24 on Mar 23, 2016

# ? Mar 23, 2016 09:39

VikingofRock: Aug 24, 2008

Yup, you got it (in your edit).

# ? Mar 23, 2016 18:35

gonadic io: Feb 16, 2011; >>=

Are people happy for me to continue asking this stuff here? I don't want to dominate the thread, I guess I could go to the terrible programmers yospos thread or the rust IRC instead.

Speaking of, I'm running into a situation with the FFI that I'm pretty sure is down to me using it incorrectly. My Rust code looks like:

code:

#[no_mangle]
pub extern "stdcall" fn svo_create
    (voxel_type: i32,
     register_voxel_extern: extern "stdcall" fn(Vec3<f32>, i32, i32) -> u32,
     deregister_voxel_extern: extern "stdcall" fn(u32)
    ) -> *mut ExternalSVO<'static> {

    ...

    println!("test deregister start");
    deregister_voxel_extern(0);
    println!("test deregister end");

    let deregister_voxel = &|external_id| {
        println!("rust callback deregister_voxel {}", external_id);
        deregister_voxel_extern(external_id);
        println!("rust callback deregister_voxel done");
    };

    ...

    let external_svo = ExternalSVO {
        register_voxel: register_voxel,
        deregister_voxel: deregister_voxel,
        svo: SVO::new_voxel(voxel_data, uid)
    };
    unsafe { transmute(Box::new(external_svo)) }
}

And the idea is for those callbacks to be able to be called given a pointer to this object. The callback itself (in this function) is fine, and the logs contain
test deregister start from unity deregistering 0 test deregister end

However in other FFI functions, i.e.

code:

#[no_mangle]
pub extern "stdcall" fn svo_set_block(svo_ptr: *mut ExternalSVO, index_ptr: *const u8,
        index_len: usize, new_voxel_type: i32) {
    let mut svo_ref: &mut ExternalSVO = unsafe { &mut *svo_ptr };
    let index: &[u8] = unsafe { slice::from_raw_parts(index_ptr, index_len) };
    println!("about to deregister in other function");
    (svo_ref.deregister_voxel)(0);
    println!("deregistered in other function");
    let voxel_data = VoxelData::new(new_voxel_type);
    svo_ref.svo.set_block(&svo_ref.deregister_voxel, &svo_ref.register_voxel, index, voxel_data);
}

When this gets called the log looks like:
about to deregister in other function rust callback deregister_voxel 0 <end of log>

So from what I can see the callback works fine when called initially, but when the address is called again later it crashes with
Thread 0 Crashed:: CrBrowserMain Dispatch queue: com.apple.main-thread 0 ??? 0x00007fff5fbfc860 0 + 140734799792224 1 libcarved_rust.bundle 0x000000011eeef77c svo_set_block + 28

I must say that debugging FFI is really quite frustrating because Unity just crashes with no message, the "send to Apple?" window contains the message above and then the Unity log contains the tracing statements.

So how should I keep these functions around and allow them to be called just given the pointer to the ExternalSVO.

Full context (including the definition of ExternalSVO) is on my github.

gonadic io fucked around with this message at 20:40 on Mar 27, 2016

# ? Mar 27, 2016 20:23

Linear Zoetrope: Nov 28, 2011; A hero must cook

I think the Box is getting dropped as soon as svo_create exits, meaning the memory is freed. This is why it's working the first time -- there's really no guarantee how many times it will work, but you're probably getting lucky and then suddenly the memory gets corrupted and everything is terrible. You probably also need to do something like:

code:

let external_svo = [...];
let mut svo_box = Box::new(external_svo);
unsafe { 
    mem::forget(svo_box);
    mem::transmute(svo_box)
}

Then you have to remember to explicitly call a destruction function later to manually destroy the block using... uh... I'll be honest I never use forget so I'm not sure, something like:

code:

#[no_mangle]
pub extern "stdcall" fn svo_destroy(svo: *mut ExternalSVO<'static>) {
    let svo: Box<ExternalSVO<'static>> = unsafe { mem::transmute(svo) };
    drop(svo);
}

probably works.

Linear Zoetrope fucked around with this message at 20:51 on Mar 27, 2016

# ? Mar 27, 2016 20:42

Vanadium: Jan 8, 2005

Edit: The box doesn't get dropped because transmute takes it by value, and then internally forgets it.

deregister_voxel is a reference to a closure that closes over the deregister_voxel_extern pointer on the stack, borrowing it. So the closure reference and also the closure itself are basically broken once you return from svo_create. I assume you hide the lifetime problems from the borrow checker by transmuting to something with 'static lifetime.

If you want to use closures here, they should capture all their state by move (move |foo| { ... }), and then you probably gotta box them into a Box<Fn(...) -> ...>. Alternatively maybe you can store all the relevant state in that struct you return, including those function pointers you take, and just impl methods on the struct instead of putting closures in? I dunno.

I'm happy to see Rust chatter somewhere on CoC even if I can't answer questions very well. :shobon:

# ? Mar 27, 2016 20:47

Linear Zoetrope: Nov 28, 2011; A hero must cook

Oh yeah, it's definitely the closure, I didn't even notice he was closing over anything, I thought he was using the &Fn as a straight up alias for extern fn.

# ? Mar 27, 2016 21:00

VikingofRock: Aug 24, 2008

gonadic io posted:

Are people happy for me to continue asking this stuff here? I don't want to dominate the thread, I guess I could go to the terrible programmers yospos thread or the rust IRC instead.

Definitely keep posting here, I'm learning a lot from thinking about the issues that you are presenting and people's responses to them. Also the Rust IRC is a fantastic resource. If you post a question here and no one answers it, you should ask IRC (and then post their response here so we can learn from it too).

# ? Mar 27, 2016 22:18

gonadic io: Feb 16, 2011; >>=

I moved the closure to the internal function like so:

code:

#[repr(C)]
pub struct ExternalSVO {
    pub register_voxel_extern: extern "stdcall" fn(Vec3<f32>, i32, VoxelData) -> u32,
    pub deregister_voxel_extern: extern "stdcall" fn(u32),
    pub svo: SVO
}

#[no_mangle]
pub extern "stdcall" fn svo_set_block(svo_ptr: *mut ExternalSVO, index_ptr: *const u8, index_len: usize, new_voxel_type: i32) {
    let &mut ExternalSVO { ref mut svo, register_voxel_extern, deregister_voxel_extern } = unsafe { &mut *svo_ptr };
    let index: &[u8] = unsafe { slice::from_raw_parts(index_ptr, index_len) };
    let voxel_data = VoxelData::new(new_voxel_type);

    let r = &|vec, depth, data| register_voxel_extern(vec, depth, data);
    let d = &|id| deregister_voxel_extern(id);
    svo.set_block(d, r, index, voxel_data);
}

And it's all good now, as the closure hasn't been dropped when set_block is called.

I tried a few times without the intermediate (seemingly no-op) closure there by casting the extern "stdcall" fn to both &Fn(...) -> u32 and Box<Fn(...) -> u32> and then passing them directly to set_blocks which expects a &Fn but neither worked.

I got either
carved_rust.rs:43:13: 43:35 error: the trait `core::ops::Fn<(nalgebra::structs::vec::Vec3<f32>, i32, svo::VoxelData)>` is not implemented for the type `extern "stdcall" fn(nalgebra::structs::vec::Vec3<f32>, i32, svo::VoxelData) -> u32` [E0277]
or
carved_rust.rs:45:9: 45:44 error: the trait `core::ops::Fn<(nalgebra::structs::vec::Vec3<f32>, i32, svo::VoxelData)>` is not implemented for the type `Box<core::ops::Fn(nalgebra::structs::vec::Vec3<f32>, i32, svo::VoxelData) -> u32>` [E0277]

Is the Fn trait implemented for extern "stdcall" fn? Or is it something to do with the missing "-> u32"?

# ? Mar 27, 2016 22:56

Vanadium: Jan 8, 2005

I'm surprised that the Fn traits aren't implemented or non-Rust ABI fns, but it looks like that's it.

The missing -> u32 is baffling, that frankly looks like a bug in the error message. Edit: Dude says it's because it's an associated type. Ugh.

(fwiw people tend to pass closures by value when possible and I don't recall having seen &|...|)

Vanadium fucked around with this message at 00:36 on Mar 28, 2016

# ? Mar 28, 2016 00:30

taqueso: Mar 8, 2004

VikingofRock posted:

Definitely keep posting here, I'm learning a lot from thinking about the issues that you are presenting and people's responses to them. Also the Rust IRC is a fantastic resource. If you post a question here and no one answers it, you should ask IRC (and then post their response here so we can learn from it too).

I definitely like having some activity in this thread, and these are great little discussions/examples to look at.

# ? Mar 28, 2016 00:45

gonadic io: Feb 16, 2011; >>=

Vanadium posted:

(fwiw people tend to pass closures by value when possible and I don't recall having seen &|...|)

The reason I'm doing this is that inside set_blocks they get cloned and passed around a whole bunch and so references are needed. I have at least made it so that the public function takes them by value.

Honestly I'm hoping that the compiler will optimise away passing the references to every invocation of set_voxel_from. The way I'd do this in Haskell would be to define the sub-function as a closure inside the top-level function which just uses the register and deregister functions in scope but apparently in Rust you can't define recursive closures or fns inside another fn.

code:

pub trait Register: Fn(Vec3<f32>, i32, VoxelData) -> u32 {} // No trait aliases :(
impl<R: Fn(Vec3<f32>, i32, VoxelData) -> u32> Register for R {}

pub trait Deregister: Fn(u32) {}
impl<D: Fn(u32)> Deregister for D {}

impl SVO {
    pub fn set_block<R: Register, D: Deregister>(&mut self, register_voxel: R, deregister_voxel: D,
                           index: &[u8], new_voxel_data: VoxelData) {
        self.set_voxel_from(&register_voxel, &deregister_voxel, index, new_voxel_data, zero(), 0);
    }

    fn set_voxel_from<R: Register, D: Deregister>(&mut self, register_voxel: &R, deregister_voxel: &D,
    	                    index: &[u8], new_voxel_data: VoxelData, origin: Vec3<f32>, depth: i32) {
        ...
    }
}

e: I guess I could just make a little private SetBlockEnvironment struct and pass references to that to the sub-function? That'd certainly be cleaner I suppose.

gonadic io fucked around with this message at 11:00 on Mar 28, 2016

# ? Mar 28, 2016 09:55

taqueso: Mar 8, 2004

I'm trying to install rust nightly on a new VM. The download from static.rust-lang.org is going incredibly slowly - about 0.1% per five minutes.

Is there a mirror? If so, how do I specify the mirror for the rustup script?

e: Just tried again and it is fixed now.

taqueso fucked around with this message at 06:00 on Apr 3, 2016

# ? Apr 2, 2016 21:07

gonadic io: Feb 16, 2011; >>=

I have lot of fixed sized arrays. Functions that return [T; 8] etc.

Working with them is a real pain since neither FromIterator nor ToIterator are implemented for them. I feel that things like map and collect should be available but it's pretty cumbersome converting between slices in order to use these functions.

Is there a recommended way for working with arrays rather than slices or should I not bother?

e: for a concrete example I have

code:

impl SubImage {
    fn split_threshold(&self) -> Option<[SubImage; 2]>
    fn quads(&self) -> Option<[SubImage; 4]>
}

and I want to implement

code:

impl SubImage {
    pub fn octs(&self) -> Option<[SubImage; 8]>
}

and the order that quads/split_threshold are called in on a SubImage doesn't matter.

I tried at first to use some flat_maps, but couldn't figure that out and instead am now trying to do it explicitly with pattern matching but still haven't managed yet. Is there a nice way to do this? Should I just use slices and forgo the compile-time guarantees?

gonadic io fucked around with this message at 20:18 on Apr 4, 2016

# ? Apr 4, 2016 20:13

gonadic io: Feb 16, 2011; >>=

Presented without comment:

code:

let sum = (self.x_0 .. self.x_n).map(|x| {
    ...
}).fold(0u32, |x, y| x+y); // Dear Rust, gently caress you.

Yes I could use nightly and a feature gate, but I quite like my code to not break between releases.

# ? Apr 4, 2016 20:46

VikingofRock: Aug 24, 2008

gonadic io posted:

I have lot of fixed sized arrays. Functions that return [T; 8] etc.

Working with them is a real pain since neither FromIterator nor ToIterator are implemented for them. I feel that things like map and collect should be available but it's pretty cumbersome converting between slices in order to use these functions.

Is there a recommended way for working with arrays rather than slices or should I not bother?

e: for a concrete example I have
code:
impl SubImage {
    fn split_threshold(&self) -> Option<[SubImage; 2]>
    fn quads(&self) -> Option<[SubImage; 4]>
}
and I want to implement
code:
impl SubImage {
    pub fn octs(&self) -> Option<[SubImage; 8]>
}
and the order that quads/split_threshold are called in on a SubImage doesn't matter.

I tried at first to use some flat_maps, but couldn't figure that out and instead am now trying to do it explicitly with pattern matching but still haven't managed yet. Is there a nice way to do this? Should I just use slices and forgo the compile-time guarantees?

In my experience, working with arrays in Rust totally sucks and will likely continue to suck until they add dependent typing (which is planned, but which seems somewhat far-off). For now, I'd just use slices and save yourself the headache.

# ? Apr 4, 2016 22:22

Linear Zoetrope: Nov 28, 2011; A hero must cook

VikingofRock posted:

In my experience, working with arrays in Rust totally sucks and will likely continue to suck until they add dependent typing (which is planned, but which seems somewhat far-off). For now, I'd just use slices and save yourself the headache.

Wait, what? I thought Dependent Typing was a No Go for Rust. I mentioned it a few times in discussions on the issue tracker, and people mentioned they abandoned the idea because it wasn't a "good fit". Which is a shame because I'd really love it. Especially because first-class dependent typing isn't much work away from a full-on theorem prover which means you can prove some pretty strong guarantees for your implementations. (Unless Dependent Typing is a consequence of HKTs? I feel like people wouldn't have vehemently denied plans for dependent types if that were the case though, given that HKTs are very much in the works, though they don't think they can get it done in 2016).

Anyway, I have some questions about traits. Or rather, a few trait requirements:

1. Why does Fn require FnMut require FnOnce? Having FnOnce be the base type almost seems backwards to me, especially since this is roughly akin to Take By Immutable Reference -> Take By Mutable Reference -> Take By Move. It feels like taking by reference should be the "least onerous" to implement. But regardless it feels like all these traits should be disjoint, rather than extensions of each other. (Not that you generally implement the Fn traits yourself anyway).
2. Is there any case where you'd have Sync but not Send? It feels like it would have to be a pretty drat wonky scenario, like some of the OS X Cocoa weirdness where certain calls HAVE to be made on the main thread, but sync wouldn't fix that problem.

Linear Zoetrope fucked around with this message at 08:40 on Apr 5, 2016

# ? Apr 5, 2016 08:38

Vanadium: Jan 8, 2005

Jsor posted:

1. Why does Fn require FnMut require FnOnce? Having FnOnce be the base type almost seems backwards to me, especially since this is roughly akin to Take By Immutable Reference -> Take By Mutable Reference -> Take By Move. It feels like taking by reference should be the "least onerous" to implement. But regardless it feels like all these traits should be disjoint, rather than extensions of each other. (Not that you generally implement the Fn traits yourself anyway).

I dunno, this makes sense to me? FnOnce leaves the most freedom to the implementer (can consume environment), FnMut slightly less (can only mutate, not consume) and Fn the least (can only look, not touch). If you have a function that can work by immutable reference, it'll also work if you let it mutate the environment, but not the other way around, so FnMut requiring Fn would make zero sense because then a function can only mutate its environment if it can also work without mutating it.

I'm not sure about the Sync/Send thing, I always get confused there.

# ? Apr 5, 2016 18:54

Arcsech: Aug 5, 2008

Vanadium posted:

I'm not sure about the Sync/Send thing, I always get confused there.

On this note, if anyone has a link to a good explanation of Send & Sync besides the official docs, could you post it?

Like I get it in theory (Send means it's safe to transfer between threads, Sync means it's safe to access from multiple threads), but I haven't yet quite wrapped my head around it in practice. Like, I see the Arc<Mutex<ActualThing>> pattern a fair bit, but I'd like somewhere that does a deeper dive on an/a few examples.

My interpretation of that is that Mutex makes it Sync, and Arc makes it Send, but I'm still not confident I'd know how to use it myself (especially Arc). And I'm coming up on a point in the project I'm working on where that would be super helpful.

# ? Apr 5, 2016 19:01

VikingofRock: Aug 24, 2008

Jsor posted:

Wait, what? I thought Dependent Typing was a No Go for Rust. I mentioned it a few times in discussions on the issue tracker, and people mentioned they abandoned the idea because it wasn't a "good fit". Which is a shame because I'd really love it. Especially because first-class dependent typing isn't much work away from a full-on theorem prover which means you can prove some pretty strong guarantees for your implementations. (Unless Dependent Typing is a consequence of HKTs? I feel like people wouldn't have vehemently denied plans for dependent types if that were the case though, given that HKTs are very much in the works, though they don't think they can get it done in 2016).

Okay so I looked into this a little more, and it seems like full-on dependent typing is pretty controversial, but there is at least a push for type-level numerics (which would make arrays much nicer to work with). Here is the most recent issue about it AFAIK.

# ? Apr 5, 2016 19:12

Linear Zoetrope: Nov 28, 2011; A hero must cook

Arcsech posted:

On this note, if anyone has a link to a good explanation of Send & Sync besides the official docs, could you post it?

Like I get it in theory (Send means it's safe to transfer between threads, Sync means it's safe to access from multiple threads), but I haven't yet quite wrapped my head around it in practice. Like, I see the Arc<Mutex<ActualThing>> pattern a fair bit, but I'd like somewhere that does a deeper dive on an/a few examples.

My interpretation of that is that Mutex makes it Sync, and Arc makes it Send, but I'm still not confident I'd know how to use it myself (especially Arc). And I'm coming up on a point in the project I'm working on where that would be super helpful.

Mutexes are both Send and Sync. If you're doing a fork/join where your data will never go out of scope, you can just create a Mutex and send a reference to each thread. The Arc<Mutex<T>> pattern is so you can send a reference to a bunch of threads, return, and continue doing other stuff in the thread that spawned the other threads.

# ? Apr 7, 2016 05:12

gonadic io: Feb 16, 2011; >>=

gonadic io posted:

code:

impl SubImage {
    fn split_threshold(&self) -> Option<[SubImage; 2]>
    fn quads(&self) -> Option<[SubImage; 4]>
}

I had a chance to try and combine these two functions tonight and am still fighting the borrow checker. SubImage implements Copy and Clone, so why I'm getting reference not living long enough errors even when I tried to clone() everything is beyond me. I get that quads goes out of scope at the end of the closure so any references to it would be invalid, but I'm copying all of the SubImages right? Making it a move closure doesn't help either.

Honestly I feel that this entire thing could be done as a few flat_maps but I've yet to get it to compile even 1) doing a bunch of manual indexing and 2) pretending that split_threshold can never fail.

code:

impl SubImage {
	pub fn octs(&self) -> Option<[SubImage; 8]> {
		self.quads().and_then(|quads| {
			let octs01 = quads[0].split_threshold().unwrap();
			let octs23 = quads[1].split_threshold().unwrap();
			let octs45 = quads[2].split_threshold().unwrap();
			let octs67 = quads[3].split_threshold().unwrap();
			Some([octs01[0], octs01[1],
			      octs23[0], octs23[1],
			      octs45[0], octs45[1],
			      octs67[0], octs67[1]])
		})
	}
}

error: `quads[..]` does not live long enough
reference must be valid for the anonymous lifetime #1 defined on the block at 63:45... (the entire function octs)
...but borrowed value is only valid for the scope of parameters for function at 64:37 (the closure)

# ? Apr 7, 2016 23:00

Linear Zoetrope: Nov 28, 2011; A hero must cook

It has something to do with your implementation of split_threshold, quads, or the definition of SubImage, because the trivial version works on the Playground:

code:

#[derive(Copy,Clone)]
struct SubImage;

impl SubImage {
    fn split_threshold(&self) -> Option<[SubImage; 2]> {
        Some([SubImage, SubImage])
    }
    fn quads(&self) -> Option<[SubImage; 4]> {
        Some([SubImage, SubImage, SubImage, SubImage])
    }

    pub fn octs(&self) -> Option<[SubImage; 8]> {
        self.quads().and_then(|quads| {
            let octs01 = quads[0].split_threshold().unwrap();
            let octs23 = quads[1].split_threshold().unwrap();
            let octs45 = quads[2].split_threshold().unwrap();
            let octs67 = quads[3].split_threshold().unwrap();
            Some([octs01[0], octs01[1], octs23[0], octs23[1], octs45[0], octs45[1], octs67[0],
                  octs67[1]])
        })
    }
}

https://play.rust-lang.org/?gist=26d3c276f2539c1f47fec1ccc77b3971&version=stable&backtrace=0

# ? Apr 7, 2016 23:16

gonadic io: Feb 16, 2011; >>=

I see now, it's because SubImage contains a reference to a byte buffer.

code:

struct SubImage<'a> {image: & 'a [u8] }

(as well as a bunch of u32s tracking the start and end of the sub images)

I tried adding 'a lifetimes everywhere to all the references and SubImage but I guess that was being done implicitly by the compiler anyway.

e: when I next get internet I'll push to github so you can see the whole code

e2: https://github.com/djmcgill/carved/blob/master/carved_rust/src/svo/generator/height_map.rs

gonadic io fucked around with this message at 09:44 on Apr 8, 2016

# ? Apr 8, 2016 08:44

Linear Zoetrope: Nov 28, 2011; A hero must cook

Here's the problem:

code:

fn split_threshold(&self) -> Option<[SubImage;2]>;

What you want:

code:

fn split_threshold(&self) -> Option<[SubImage<'a>;2]>

BUT NOT:

code:

fn split_threshold(&'a self) -> Option<[SubImage<'a>;2]>

// or

fn split_threshold<'b>(&'b self) -> Option<[SubImage<'b>;2]>

Basically, what you wrote was filling in the <'b> version due to the lifetime elision rules. You were saying "I am asserting my borrow of self will last as long as my the sub image I return." This is impossible since quads doesn't live that long, so that borrow also can't live that long. You need to give them different lifetimes. What you really wanted to say is "whatever I return, the lifetime of the reference to the underlying data I was given needs to last at least as long, but really, how long this function's borrow of me lasts is irrelevant."

I'm probably explaining this very poorly because I don't 100% grasp the intricacies myself.

E: You probably want to make this change for the other functions too, but they don't cause this specific function to fail.

Linear Zoetrope fucked around with this message at 10:27 on Apr 8, 2016

# ? Apr 8, 2016 10:23

syncathetic: Oct 21, 2010

I was able to get it to compile making the following changes:

code:

diff --git a/carved_rust/src/svo/generator/height_map.rs b/carved_rust/src/svo/generator/height_map.rs
index 000291a..f3d2bd9 100644
--- a/carved_rust/src/svo/generator/height_map.rs
+++ b/carved_rust/src/svo/generator/height_map.rs
@@ -34,7 +34,7 @@ impl<'a> SubImage<'a> {
 		self.y_n - self.y_0
 	}
 
-	fn rect(&self, x_0: u32, x_n: u32, y_0: u32, y_n: u32) -> SubImage {
+	fn rect(&self, x_0: u32, x_n: u32, y_0: u32, y_n: u32) -> SubImage<'a> {
 		SubImage { x_0: x_0, x_n: x_n, y_0: y_0, y_n: y_n, .. *self }
 	}
 
@@ -51,7 +51,7 @@ impl<'a> SubImage<'a> {
 		Some([ll, lr, rl, rr])
 	}
 
-	fn split_threshold(&'a self) -> Option<[SubImage<'a>; 2]> {
+	fn split_threshold(&self) -> Option<[SubImage<'a>; 2]> {
 		let half_range = (self.b_n - self.b_0) / 2;
 		if half_range == 0 { return None; }
 
@@ -113,4 +113,4 @@ impl SVO {
 			//SVO::Octants([])
 		}
 	}
-}
\ No newline at end of file
+}

I can't really explain why that works, though. I'm pretty sure doing &'a self is generally counter-productive. Most of the time, 'a needs to outlive self to make sense.

# ? Apr 8, 2016 10:24

gonadic io: Feb 16, 2011; >>=

That makes perfect sense, thanks all. I'll sort it out tonight.

Of course, given how much of a pain using map with arrays is I might just stick with my manual indexing. That or write a map_8 function.

# ? Apr 8, 2016 11:31

Adbot: ADBOT LOVES YOU

# ? Apr 25, 2024 08:55

gonadic io: Feb 16, 2011; >>=

I own a [T; 8] and would like to access the Ts out of it without copying (this is important). Then I'd like to call a FnOnce(T) -> T on each one and package them back up in an array.

I managed to cobble together a working version thanks to SO:

code:

impl SVO<Registered> {
    fn deregister<D: Deregister>(self, deregister_voxel: &D) -> SVO<Unregistered> {
        match self {
            SVO::Voxel { registration: Registered { external_id }, data } => {
                deregister_voxel(external_id);
                SVO::Voxel { data: data, registration: Unregistered }
            },
            SVO::Octants (mut octants) => {
                let new0 = unsafe { mem::replace(&mut octants[0], mem::uninitialized()).deregister(deregister_voxel) };
                let new1 = unsafe { mem::replace(&mut octants[1], mem::uninitialized()).deregister(deregister_voxel) };
                let new2 = unsafe { mem::replace(&mut octants[2], mem::uninitialized()).deregister(deregister_voxel) };
                let new3 = unsafe { mem::replace(&mut octants[3], mem::uninitialized()).deregister(deregister_voxel) };
                let new4 = unsafe { mem::replace(&mut octants[4], mem::uninitialized()).deregister(deregister_voxel) };
                let new5 = unsafe { mem::replace(&mut octants[5], mem::uninitialized()).deregister(deregister_voxel) };
                let new6 = unsafe { mem::replace(&mut octants[6], mem::uninitialized()).deregister(deregister_voxel) };
                let new7 = unsafe { mem::replace(&mut octants[7], mem::uninitialized()).deregister(deregister_voxel) };
                SVO::Octants([
                    Box::new(new0), Box::new(new1), Box::new(new2), Box::new(new3),
                    Box::new(new4), Box::new(new5), Box::new(new6), Box::new(new7)
                ])
            }
        }
    }
}

Which is functioning, if ugly. Note that SVO does NOT implement Drop so I'm fine (I think) just messing with their memory.

However I have the function:

code:

impl<R: RegistrationTrait> SVO<R> {
    pub fn new_octants<F: Fn(u8) -> SVO<R>>(make_octant: F) -> SVO<R> { ... }
}

which I'd like to use to make this much nicer, but when I try to package up the repeated code inside the closure like so:

code:

let fun: &Fn(u8) -> SVO<_> = &|ix: u8| unsafe { mem::replace(&mut octants[ix as usize], mem::uninitialized()).deregister(deregister_voxel) };
SVO::new_octants(fun)

(the trait object is just to convince Rust to give a useful type error, otherwise I just get "Fn(u8) is not implemented for [closure]"). Note too that this has to (I'm pretty sure) be a Fn and not a FnOnce as it'll get called on each of the 8 octants in new_octants.

I get the error
error: cannot borrow data mutably in a captured outer variable in an `Fn` closure [E0387].

Is there any way I can 1) bypass needing a mutable pointer to an outer variable in the closure, or 2) have a mutable pointer to an outer variable in the closure?

Full context is here: https://github.com/djmcgill/carved/blob/master/carved_rust/src/svo/mod.rs

gonadic io fucked around with this message at 19:44 on Apr 20, 2016

# ? Apr 20, 2016 19:41

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »