Rust: What the fuck is a borrow checker and why is it yelling at me?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »

Sagacity: May 2, 2003; Hopefully my epitaph will be funnier than my custom title.

That's one of the most annoying things about the language for me, currently. I understand why the restriction is there but as a regular developer it ends up feeling like such an arbitrary restriction around a very common use case.

# ? Mar 14, 2020 23:17

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 18:52

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

gonadic io posted:

Yes, wanting to mutably borrow only one field of a struct and then pass the rest of the struct somewhere else is a common scenario but is unfortunately impossible* to reuse the struct itself. The easiest way is to make all the grouped fields in a second struct so you have like
code:
struct MainStruct {
  field1: Butt,
  otherFields: SecondaryStruct,
}

struct SecondaryStruct {
  field2: Butt,
  ...
  field99: Butt
}
and then you can have methods take SecondaryStruct - that way you don't need to destructure every field out and pass them around individually. It's a bit of a rough corner in Rust IMO.

*: well you can start messing around with Rc and Arc and Cell and Refcell etc but that's not what I'm talking about

I find this dance usually ends up with me having better encapsulation than I otherwise would have, so it's not all bad.

# ? Mar 15, 2020 00:33

drainpipe: May 17, 2004; AAHHHHHHH!!!!

I'm running into a weird error. My basic code is this:

code:

loop {
	tokio::select! {
		msg1 = self.rx.recv() => {
			// do stuff
		},
		msg2 = self.stream.read() => {
			// do stuff 
		}
	}
}

self.rx is a broadcast channel and self.stream is a wrapper around a tokio TcpStream that reads formatted messages (it first reads a byte x and then reads a number of bytes equal to the value of x). The problem I'm getting is that I'll sometimes randomly get garbled messages coming from stream.read() (the length byte is way too big). I'm thinking that if rx.recv() finishes first, it can trigger the loop to begin again half way through the first stream.read(). Then the subsequent call to stream.read() is made which will try to read the stream in the middle of the first read attempt. Does this sound reasonable?

# ? Mar 23, 2020 19:58

gonadic io: Feb 16, 2011; >>=

I don't believe stream.read can be reset half way through like that - since my understanding of how select! works is it calls poll on its futures and then if one returns Ok(value) then it executes that branch. If rx.recv returns a value then steam.read.poll is never called and any incoming bytes stay in the buffer. If stream.read.poll is called, starting the read, then in the next iteration recv finishes quickly, then stream.read.poll will either finish its future or still not be ready.

As for fixing this problem, could you get a minimal working example? There's too little context to make too much sense of it for me.

# ? Mar 23, 2020 21:48

drainpipe: May 17, 2004; AAHHHHHHH!!!!

Here is the streamlined version of the read method.

code:

async fn read(&mut self) -> Option<Message> {
    let len = self.s.read_u32().await.ok()? as usize;

    let mut buf = vec![0; len];
    self.s.read_exact(&mut buf).await.ok()?;
    Message::deserialize(buf)
}

I'm guessing that if rx.recv() returns in between the read_u32 and read_exact finishing, then that can throw a wrench into things. The stream has already read the length byte, but when the read method reenters after the loop, it'll try to read another length byte but instead get a byte from the payload of a previous message.

It seems select does not work well with functions that have serial side effects.

edit: One fix I'm considering is having a background task constantly read from the stream and push the result into a queue which the read method would pop from.

drainpipe fucked around with this message at 23:14 on Mar 23, 2020

# ? Mar 23, 2020 23:09

gonadic io: Feb 16, 2011; >>=

Consider something like https://docs.rs/tokio-util/0.3.1/tokio_util/codec/struct.Framed.html

# ? Mar 23, 2020 23:37

drainpipe: May 17, 2004; AAHHHHHHH!!!!

Oh great, thanks! It seems like that and the LengthDelimitedCodec are exactly what I'm looking for.

# ? Mar 24, 2020 13:07

Minus Pants: Jul 18, 2004

I'm writing an app that does some basic network performance tests, and I'd like to be able to format some types from std in my print calls (e.g. std::net::SocketAddr). I stumbled around trying to implement the Display trait, which doesn't work because it's an external type. What's the idiomatic way to handle this? Make a wrapper type? Formatter function?

Edit: It looks like I can't use a type alias - same error about an external crate. Do I need to make a type with a SockAddr field and getters/setters? That seems a bit ugly.

Minus Pants fucked around with this message at 03:15 on Apr 17, 2020

# ? Apr 17, 2020 03:11

Threep: Apr 1, 2006; It's kind of a long story.

Minus Pants posted:

I'm writing an app that does some basic network performance tests, and I'd like to be able to format some types from std in my print calls (e.g. std::net::SocketAddr). I stumbled around trying to implement the Display trait, which doesn't work because it's an external type. What's the idiomatic way to handle this? Make a wrapper type? Formatter function?

Edit: It looks like I can't use a type alias - same error about an external crate. Do I need to make a type with a SockAddr field and getters/setters? That seems a bit ugly.

I usually make thin wrappers that only implement Display and just create them in the print call:

code:

println!("{}", SockAddrFormatter(&addr));

# ? Apr 17, 2020 03:29

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

But SocketAddr implements Display already?

# ? Apr 17, 2020 05:42

Minus Pants: Jul 18, 2004

Ralith posted:

But SocketAddr implements Display already?

Sorry, I should have said I want to override the default implementation.

Wrapper approach will work, I just wasn't sure if that's the Rust way to do it.

# ? Apr 17, 2020 06:14

gonadic io: Feb 16, 2011; >>=

Do you actually need the trait Display specifically? I often just make a write_foo method which, as a standalone function that just takes an SocketAddr as a parameter, doesn't care about orphan rules.

# ? Apr 17, 2020 10:12

Minus Pants: Jul 18, 2004

Yeah that's true, and simpler. I need to get out of inherit-and-override thinking.

# ? Apr 18, 2020 07:32

gonadic io: Feb 16, 2011; >>=

To make a Display wrapper is probably the "better" approach so I would still use it e.g. in a big codebase with lots of coworkers who might accidentally use the display impl. And then there's stuff like Eq, Ord, and Hash that you usually do just have to make wrappers for. And as for (De)Serialise...

# ? Apr 18, 2020 13:08

drainpipe: May 17, 2004; AAHHHHHHH!!!!

I realized I don't know what a lot of the trait sections in rustdocs are as well as I thought I did. This is stuff like "Blanket implementation" and "Implementation on Foreign Types". Here's what I thought they meant:

Trait implementations: Implementing foreign traits for a new struct/trait
Auto-trait implementations: Foreign traits that are implemented by default for any new trait
Implementors: Implementations of a new trait for foreign structs (like stuff from the standard library)
Implementations on Foreign Types: Implementations of a new trait for foreign traits.
Blanket implementations: ???

According to https://doc.rust-lang.org/book/ch10-02-traits.html#implementing-a-trait-on-a-type, blanket implementations seem to be what I thought Implementations on Foreign Types are. But then what is the difference between the two?

Also, if that is true, what is going on in the blanket implementations section of https://docs.rs/warp/0.2.2/warp/filters/multipart/struct.Part.html where everything is just stuff from the standard library (even when you click the source)? Why are those listed in these rustdocs?

# ? May 7, 2020 12:45

Mr. Glass: May 1, 2009

blanket implementations are generic implementations of the form

code:

impl<T> Frobber for T {
...
}

if you wrote it like this, though, you wouldn't be able to implement Frobber for any other type, since that would be a conflicting implementation. you also wouldn't actually be able to do anything in that impl, since you don't know anything about the T you're implementing the trait for. thankfull, it's a little more flexible than that, since you can put trait bounds on the blanket impl:

code:

impl<T: Debug> Frobber for T {
...
}

this is saying "I know how to frob anything that implements Debug; everyone else is on their own." a classic example of this in the standard library is the blanket implementation of From<T> for U where T: Into<U> (or the other way around, i can't remember which one it is). the reason that they're listed in the rustdocs is to give you a full picture of all the traits implemented by the type you're looking at, both explicitly or implicitly.

an implementation on a foreign type is just where you as the library author implement your trait on a type defined outside of your crate:

code:

impl Frobber for String {
...
}

it's true that blanket implementations can indeed be implementations on foreign types, but since they are generic (and implicit) i guess they decided to give them a different name.

# ? May 7, 2020 15:45

drainpipe: May 17, 2004; AAHHHHHHH!!!!

Ok, that seems kinda minutiae-y, but understandable. But still, is there a reason why the Blanket implementation section of https://docs.rs/warp/0.2.2/warp/filters/multipart/struct.Part.html has stuff like

code:

impl<T> Any for T
where
    T: 'static + ?Sized,

code:

impl<T> Borrow<T> for T
where
    T: ?Sized,

when none of those traits are related to the traits defined in the crate?

# ? May 7, 2020 18:29

Mr. Glass: May 1, 2009

i assume they are included so you don't have to remember the trait bounds required to receive the blanket impl or cross reference the type with the standard library traits yourself. it makes sense to me that you would want to see a summary of all of the functionality available for a particular type on the same page, regardless of whether that functionality is provided by the crate containing the type or by the standard library.

# ? May 7, 2020 18:46

Dominoes: Sep 20, 2007

I have a diff trait question: How do you find trait bounds in lib docs? Usually the docs and compiler messages are clear, but I'm striking out on both counts. Example: here. At the top, it defines bounds on I2C, but not I2CInterface

# ? May 7, 2020 19:09

Mr. Glass: May 1, 2009

that's because I2cInterface is a struct, not a generic type: https://docs.rs/crate/ads1x1x/0.2.0/source/src/interface.rs

not sure why it doesn't show up in the rustdoc.

# ? May 7, 2020 21:22

Dominoes: Sep 20, 2007

Thank you; that sorts it out. Of note, some of the items in <> in that page are in the docs (and are linked), while most aren't. I'd assumed it not being in the docs meant it was a trait.

# ? May 7, 2020 21:56

Stinky_Pete: Aug 16, 2015; Stinkier than your average bear; Lipstick Apathy

Hello, I started learning Rust because this game/graphics engine called Amethyst uses it. I've been going through Rust by Example and I'm totally in love, but I was just wondering, who is responsible for Cargo staying up? Is it Mozilla?

# ? May 19, 2020 05:49

xtal: Jan 9, 2011; by Fluffdaddy

Stinky_Pete posted:

Hello, I started learning Rust because this game/graphics engine called Amethyst uses it. I've been going through Rust by Example and I'm totally in love, but I was just wondering, who is responsible for Cargo staying up? Is it Mozilla?

I don't think so, but Mozilla probably helps. There is a whole Cargo org and team, and I don't think Mozilla is a majority sponsor.

# ? May 19, 2020 06:22

Bongo Bill: Jan 17, 2012

Stinky_Pete posted:

Hello, I started learning Rust because this game/graphics engine called Amethyst uses it. I've been going through Rust by Example and I'm totally in love, but I was just wondering, who is responsible for Cargo staying up? Is it Mozilla?

Bet you could email them to find out.

# ? May 19, 2020 07:52

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

Mozilla Legal still makes the calls on DMCA/trademark/etc stuff, at least, and are the contact of record for privacy and data use notices.

# ? May 19, 2020 10:05

drainpipe: May 17, 2004; AAHHHHHHH!!!!

I have a question about kinda self-referential structs. I have a struct that generates objects, but I'd also like for it to store some references to the objects it generates. The following is a non-working example. It generates some Strings that it owns in a Vec. But I'd also like for it to have a Vec of references to some of the Strins that it has generated.

code:

struct Foo<'a> {
    pub objs: Vec<String>,
    pub refs: Vec<&'a String>,
}

impl<'a> Foo<'a> {
    pub fn makenew(&mut self) -> &String {
        let n = self.objs.len();
        self.objs.push(format!("{}", n));
        self.objs.iter().rev().peekable().peek().unwrap()
    }
    
    pub fn push(&mut self, s: &'a String) {
        self.refs.push(s);
    }
}

fn main() {
    let mut x: Foo<'static> = Foo { objs: vec![], refs: vec![] };
    let z = x.makenew();
    x.push(z);
}

This doesn't work as x is mutably borrowing z while z is borrowing x (I guess). Here's the borrow checker.

code:

error[E0597]: `x` does not live long enough
  --> src/main.rs:20:13
   |
19 |     let mut x: Foo<'static> = Foo { objs: vec![], refs: vec![] };
   |                ------------ type annotation requires that `x` is borrowed for `'static`
20 |     let z = x.makenew();
   |             ^ borrowed value does not live long enough
21 |     x.push(z);
22 | }
   | - `x` dropped here while still borrowed

error[E0499]: cannot borrow `x` as mutable more than once at a time
  --> src/main.rs:21:5
   |
19 |     let mut x: Foo<'static> = Foo { objs: vec![], refs: vec![] };
   |                ------------ type annotation requires that `x` is borrowed for `'static`
20 |     let z = x.makenew();
   |             - first mutable borrow occurs here
21 |     x.push(z);
   |     ^ second mutable borrow occurs here

error: aborting due to 2 previous errors

I realize I could probably make the refs a Vec of indices, but this wouldn't generalize well with the application I had in mind, and I was just wondering if something like this with references is at all possible.

Also, if this is possible, what should the lifetime 'a be? It seems like it should be the lifetime of Foo itself, but I don't know how to write that. I punted with 'static.

# ? May 25, 2020 19:35

gonadic io: Feb 16, 2011; >>=

Just as a quick answer, self-referential structs are only possible with the newish Pin infrastructure: https://doc.rust-lang.org/std/pin/index.html
I don't really know how it works but if you try and fail I could take a look.

It is a bit complicated though, so maybe indices is just an easier quicker bet. Also another way to do similar stuff is to not own the objs yourself but to store that somewhere else using e.g. a central memory arena or something. I believe this is the approach that e.g. the Specs entity container system (used by the Amethyst game engine) does to allow you to push and pull data while having lots of references to them in different places.

# ? May 25, 2020 19:55

drainpipe: May 17, 2004; AAHHHHHHH!!!!

Ok, I'll have a look at pin later. Moving the generation into its own structure makes a lot of sense, since there's no reason it has to be in in reference holder. It also clears up what 'a should be. I'll try both.

# ? May 25, 2020 20:15

drainpipe: May 17, 2004; AAHHHHHHH!!!!

I ended up splitting up the generation and reference storage since that made more sense. Also, I realized that the makenew method above really sucked since you can't do it twice while holding onto the references since it's a mutable borrow.

I have a quick question. I have a struct containing a reference, and I'd like to use strict pointer equality to implement equality for the struct. However, it's not working. Here's the code.

code:

use std::ptr;

pub enum Foo<'a> {
    Bar(&'a String)
}

fn main() {
    let a = String::from("blah");
    let b = Foo::Bar(&a);
    let c = Foo::Bar(&a);
    
    match (&b, &c) {
        (Foo::Bar(p1), Foo::Bar(p2)) => {
            println!("{}", ptr::eq(p1,p2))
        }
    }
}

This prints false. If I match on (b, c) or even ((&b).clone(), (&c).clone()) then it prints true. Is there a reason referencing them breaks equality? I know pointers can be moved between instructions, but I don't see why this would break equality.

edit: edited away extraneous detail
edit2:

drainpipe fucked around with this message at 20:07 on May 27, 2020

# ? May 27, 2020 19:50

gonadic io: Feb 16, 2011; >>=

code:

use std::ptr;

pub enum Foo<'a> {
    Bar(&'a String)
}

fn main() {
    let a = String::from("blah");
    let b = Foo::Bar(&a);
    let c = Foo::Bar(&a);
    
    match (&b, &c) {
        (Foo::Bar(p1), Foo::Bar(p2)) => {
            println!("1 {}", ptr::eq(*p1,*p2))
        }
    }
    
    match (b, c) {
        (Foo::Bar(p1), Foo::Bar(p2)) => {
            println!("2 {}", ptr::eq(p1,p2))
        }
    }
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0a920fc80c23dc47935e3a96358857ae

This prints true and true. The answer is that what you're actually matching on is (Foo::Bar(ref p1), Foo::Bar(ref p2)) i.e. references to p1 and p2, which are references to a. You can't move p1 and p2 directly into your ptr::eq when you're matching on the references of b and c (but explicitly dereferencing the outer pointer copies them in).

Rust used to not insert refs into matches for you, then people got a bunch of errors and threw ref into odd places until it worked. So now rust does implicitly insert them in for you leading to situations like this.

e: you can make things even clearer by changing your printlns to e.g.

code:

println!("1 {:?},{:?} {}", *p1 as *const _, *p2 as *const _, ptr::eq(*p1,*p2))

gonadic io fucked around with this message at 20:10 on May 27, 2020

# ? May 27, 2020 20:06

drainpipe: May 17, 2004; AAHHHHHHH!!!!

Ah, ok. That makes sense. Thanks!

# ? May 27, 2020 20:08

gonadic io: Feb 16, 2011; >>=

I would really recommend avoid using raw pointers for your memory arena thing. Better to use std::rc::Rc imo, way WAY less chance of your unsafe blocks being incorrect and invoking UB. (because there won't be unsafe blocks)

gonadic io fucked around with this message at 20:15 on May 27, 2020

# ? May 27, 2020 20:11

drainpipe: May 17, 2004; AAHHHHHHH!!!!

Yeah, I realized the Rc thing, and I just switched to an owned version of the generator and store since the reference version also had some awkwardness to it.

This is a completely separate part. I'm writing a toy compiler and this is for tracking identifiers. The reference is to the node on the AST that bound the identifier. This should allow me to differentiate between two identifiers of the same name.

# ? May 27, 2020 20:18

Minus Pants: Jul 18, 2004

For a fairly low level networking app, would you stick to the standard library, or use something like Socket2 or some other crate? I'm still wrapping my head around the Rust ecosystem.

I'm looking in that direction since it seems like std::net is missing some functionality. E.g. a way to enumerate u32 network interfaces for join_multicast_v6() or v4 bind methods that don't require IpAddrs.

# ? May 30, 2020 04:38

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

You can also just fill in the gaps with libc, if you don't mind complicating portability/if you want more system specific stuff.

# ? May 30, 2020 16:58

Dominoes: Sep 20, 2007

Used a lifetime for the first time today. It's always been a feature I've never needed to use, in my range of Rust projects. The case was trying to format strs using byte buffers in a no_std / no allocator environment. The resulting &'static str needed to be tied to the buffer arg.

# ? May 31, 2020 00:49

Siguy: Sep 15, 2010; 10.0 10.0 10.0 10.0 10.0

Anecdote: I�ve been learning Rust by using it for a little side project that involves parsing some text files and converting them into other formats. After a day bashing my head against the borrow checker trying to give myself the most efficient string performance with the fewest copies, I decided to just make copies whenever convenient and get the drat thing done.

I suppose I learned less than I would�ve by conquering lifetimes and Copy on Write but the final result processes a typical file in 2 milliseconds, so I realized I was putting myself through hell for no reason.

Something about �learning Rust� made me think I had to find the most efficient possible solution but the language is already really drat efficient even if you do things the sloppy, naive way.

# ? Dec 29, 2020 07:43

necrotic: Aug 2, 2005; I owe my brother big time for this!

That's a solid allegory for programming in general: make it work first, then worry about performance.

# ? Dec 29, 2020 07:56

gonadic io: Feb 16, 2011; >>=

It's a common tale. People, myself included, often feel like they're "doing it wrong" but as you say, it's rarely actually needed.

Having said that, god I love nom's zero copy stuff. Turning a flat file into a complex AST, all the strings only references into the original one. Luckily my files are small enough I can just read them into memory and don't have to figure out the streaming api.

# ? Dec 29, 2020 16:57

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 18:52

Gaukler: Oct 9, 2012

gonadic io posted:

It's a common tale. People, myself included, often feel like they're "doing it wrong" but as you say, it's rarely actually needed.

Having said that, god I love nom's zero copy stuff. Turning a flat file into a complex AST, all the strings only references into the original one. Luckily my files are small enough I can just read them into memory and don't have to figure out the streaming api.

nom is real good. I�m converting a thing from using hand-rolled parsers (because �I�m only going to need to parse a couple things!�)

I�m trying to not use the macros though, have you found good resources on that? Function calls seem to be the preferred new way, but the documentation still mixes the two and most tutorials are for the macro style.

# ? Dec 30, 2020 03:53

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »