Rust: What the fuck is a borrow checker and why is it yelling at me?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

I'm brushing off my Rust and am lost in closure lifetime semantics. The explain code for this error isn't making it click and I think I'm just not understanding some underlying principle of the lifetimes here. It's a simple hyper server, and this is the meat of it:

code:

type SecretMap = Arc<Mutex<HashMap<String, String>>>;

async fn handle_request(req: Request<Body>, secret_map: SecretMap) -> Result<Response<Body>> {
    let (parts, body) = req.into_parts();

/* ... */
    
}

#[tokio::main]
async fn main() {
    let addr = SocketAddr::from(([127, 0, 0, 1], 5000));

    let secret_map: SecretMap = Arc::new(Mutex::new(HashMap::new()));

    let make_svc = make_service_fn(move |_conn| async { 
        let secret_map = secret_map.clone(); /* HERE */
        Ok::<_, Infallible>(service_fn(move |req| handle_request(req, secret_map.clone())))
    });

    let server = Server::bind(&addr).serve(make_svc);

    if let Err(e) = server.await {
        eprintln!("server error: {}", e);
    }
}

At the marked line I get this error:

code:

error[E0507]: cannot move out of `secret_map`, a captured variable in an `FnMut` closure
  --> src/main.rs:51:55
   |
49 |       let secret_map: SecretMap = Arc::new(Mutex::new(HashMap::new()));
   |           ---------- captured outer variable
50 |
51 |       let make_svc = make_service_fn(move |_conn| async {
   |  _______________________________________________________^
52 | |         Ok::<_, Infallible>(service_fn(move |req| handle_request(req, secret_map.clone())))
   | |                                        --------------------------------------------------
   | |                                        |
   | |                                        move occurs because `secret_map` has type `Arc<std::sync::Mutex<HashMap<std::string::String, std::string::String>>>`, which does not implement the `Copy` trait
   | |                                        move occurs due to use in generator
53 | |     });
   | |_____^ move out of `secret_map` occurs here

error: aborting due to previous error; 1 warning emitted

I feel like this is me being dumb about Arc semantics, but I thought that .clone(), creating a non-ref value, is what I wanted to do here.

Playground link if anyone's bored enough to explain this to me!

# ? Jul 16, 2021 16:04

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 23:38

necrotic: Aug 2, 2005; I owe my brother big time for this!

You�re calling clone inside the closure, which must move the original for that call to work. I think you need to clone outside and move that in? I�m pretty new to rust myself so might be wrong.

# ? Jul 16, 2021 16:17

gonadic io: Feb 16, 2011; >>=

That's what I tried but the future stuff requires the closure to be static too so it's a bit awkward. Maybe you need to find the place between being inside the FnMut and being inside the future. I couldn't quite get it working. There's also two FnMuts which isn't helping things.

gonadic io fucked around with this message at 16:47 on Jul 16, 2021

# ? Jul 16, 2021 16:33

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

Yeah, on the advice of a wise person I made a 'static arc<mutex<map>> and cloned that into the innermost call, since that works fine for my small case here. I would like to some day understand how to do this "properly" since it seems like I should be able to move something into the FnMut and have the lifetime work out. Some other day, that is.

# ? Jul 16, 2021 16:38

Vanadium: Jan 8, 2005

gonadic io posted:

That's what I tried but the future stuff requires the closure to be static too so it's a bit awkward. Maybe you need to find the place between being inside the FnMut and being inside the future. I couldn't quite get it working. There's also two FnMuts which isn't helping things.

It's just https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=73c14a8e6fdb3b3795746f544adc39b7 right?

Rust code:

    let secret_map: SecretMap = Arc::new(Mutex::new(HashMap::new()));

    let make_svc = make_service_fn(move |_conn| {
        let secret_map = secret_map.clone();
        async {
            Ok::<_, Infallible>(service_fn(move |req| handle_request(req, secret_map.clone())))
        }
    });

The hyper server pattern is awkward.

# ? Jul 19, 2021 04:05

gonadic io: Feb 16, 2011; >>=

Ah gently caress, move the clone before the async block. That makes so much sense, thanks.

# ? Jul 19, 2021 08:17

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

Ah ha! Thank you, I think I understand why that works.

# ? Jul 19, 2021 14:50

take boat: Jul 8, 2006; boat: TAKEN

astral posted:

Just a friendly heads-up that Rust is one of the languages now officially supported in the recently-updated [code=thing] tags. Enjoy! I tweaked the language of a code block on this page to help show it off.

Full list of supported languages/language aliases available here.

I'm very late on this, but yessss

# ? Aug 10, 2021 00:44

gonadic io: Feb 16, 2011; >>=

I've got a new pet peeve which is APIs using Fn/FnMut when it could easily be FnOnce instead and loving up my move semantics

# ? Sep 24, 2021 12:41

gonadic io: Feb 16, 2011; >>=

https://twitter.com/jeffvanderstoep/status/1446405080828239912

# ? Oct 9, 2021 01:26

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

I'm curious about hot takes on Rust adoption in the Linux kernel. I saw some talk from Linux a few months back where he basically conceding needing something like Rust to get all the Cool New Kids to start doing kernel development, and Google has some big push going to use it in the kernel. I don't really know how far along it is and if it's consumable yet. I'm trying to make a call on if/when we might want to transition into writing and working with Rust code. I particularly wish I could easily have unit test coverage of a lot of the code I'm running in kernel space.

# ? Oct 12, 2021 19:35

gonadic io: Feb 16, 2011; >>=

from what I've seen (admittedly cherry-picked by rust twitter) he seems reasonably positive in that none of his objections have been fundamental issues. the main one was about needing to statically forbid panics and I think people mostly worked out how to do that

quote:

Torvalds answered, "I think C is a great language. To me, it's really a way to control the hardware at a fairly low level." That said, "it is so close to the hardware that you can do anything with it. It is dangerous. It's like juggling chainsaws. I also see that it does have a lot of pitfalls and they're easy to overlook. And in a kernel, that's not always a good thing. It's good when it comes to things like low-level memory management, where you're literally setting up the virtual memory map for a process. But, in many other situations, you don't want it. Rust was the first language I saw which looked like this might actually be a solution to the other part of the problem, not the control of the machine at the lowest possible level so that you can read the machine code and match it up with the C code. But to the problem where you just want to get the job done of writing a driver or a file system."

At that level, it's much safer to create drivers, file systems, and routines with Rust. That said, Torvalds reminded everyone that while "people have been talking about Rust in the kernel for the longest time and it's not done yet." It's coming though. "We'll probably see some modules next year being written in Rust, and maybe being integrated into the mainline kernel."

here's where it's currently at:
https://rust-for-linux.github.io/docs/kernel/
https://github.com/Rust-for-Linux/linux

gonadic io fucked around with this message at 19:44 on Oct 12, 2021

# ? Oct 12, 2021 19:41

Hed: Mar 31, 2004; Fun Shoe

gonadic io posted:

from what I've seen (admittedly cherry-picked by rust twitter)

Do you or anyone else have any good people to follow on Rust? Thanks.

# ? Oct 12, 2021 20:18

gonadic io: Feb 16, 2011; >>=

Hed posted:

Do you or anyone else have any good people to follow on Rust? Thanks.

how into trans shitposting are you

# ? Oct 12, 2021 20:25

Arcsech: Aug 5, 2008

gonadic io posted:

how into trans shitposting are you

It�s possible to find interesting tech people to follow on Twitter that doesn�t include trans shitposting?

# ? Oct 12, 2021 21:59

gonadic io: Feb 16, 2011; >>=

Arcsech posted:

It�s possible to find interesting tech people to follow on Twitter that doesn�t include trans shitposting?

I mean it's not like I'd follow terfs or whatever but people like
Steve: https://twitter.com/steveklabnik
Ryan: https://twitter.com/ryan_levick
Amos: https://twitter.com/fasterthanlime
Mara: https://twitter.com/m_ou_se
Jorge: https://twitter.com/japaric_io

are on the lower end of shitposting vs people like
Michael: https://twitter.com/mgattozzi
Eliza: https://twitter.com/mycoliza
& co (https://twitter.com/__femb0t)
etc

All are pro follows and most of those either are on one of the committees or otherwise have a prolific library. That's just based on scrolling through my follows in no real order.

e:
https://twitter.com/scrabsha/status/1447885874029547521

# ? Oct 12, 2021 22:34

JawnV6: Jul 4, 2004; So hot ...

hows the amazon takeover going

# ? Oct 13, 2021 16:49

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Can somebody give an impression on the error handling in Rust for an outsider? I was following along with what I was reading at https://doc.rust-lang.org/book until I got to error management. It looks like it's trying to square the circle in systems programming with putting error management next to the offending statement while still having something like exceptions. How does your thinking change in Rust when using this kind of error management?

# ? Jan 27, 2022 16:58

gonadic io: Feb 16, 2011; >>=

The current best practices are that exceptions, as in out-of-control-flow errors, are essentially never used. You should use the enum Result (usually these days with an Error type from the lib anyhow most typically but there's lots of options here) for anything that's recoverable and catch/deal with it in-band, and pass it up with `?`. Any function that needs to signal failure to its parent should do it by returning a result (or an option or some other more specialised enum if you like).

Panics, most often created with the macros panic!, unwrap!, or expect! are mostly used for fatal, un-handle-able exceptions and while you CAN catch and recover from them it's used very rarely.

gonadic io fucked around with this message at 19:00 on Jan 27, 2022

# ? Jan 27, 2022 18:47

crazypenguin: Mar 9, 2005; nothing witty here, move along

There are two main crates that help deal with Results: anyhow and thiserror.

anyhow is great when you�re just trying to consume errors (eg applications) and thiserror is great for building libraries that have to return errors.

The end result is quite pleasant. I feel like it�s a good middle ground between checked and unchecked exceptions. You�re forced to deal with errors all the time, but the common case is �just write ?� and that does the right thing.

# ? Jan 27, 2022 19:20

gonadic io: Feb 16, 2011; >>=

There's also a new generation of error libs trying new poo poo and some of anyhow (specifically error chains iirc) is likely to get merged into std error but for a beginner I would definitely stick with anyhow (it's easier to use than even std error).

e: once you get beyond just messing around to learn the language I would generally recommend pairing at least a `.context("failed to do the thing)?` instead of just raw `?` since there's no stack traces for Errors (compared to panics which do have them) but still yeah nice and unobtrusive

gonadic io fucked around with this message at 19:56 on Jan 27, 2022

# ? Jan 27, 2022 19:36

Progressive JPEG: Feb 19, 2003

I'd argue that all errors are checked errors since the compiler complains if you don't handle or forward a returned Result

# ? Jan 27, 2022 20:13

Radia: Jul 14, 2021; And someday, together.. We'll shine.

crazypenguin posted:

There are two main crates that help deal with Results: anyhow and thiserror.

anyhow is great when you�re just trying to consume errors (eg applications) and thiserror is great for building libraries that have to return errors.

The end result is quite pleasant. I feel like it�s a good middle ground between checked and unchecked exceptions. You�re forced to deal with errors all the time, but the common case is �just write ?� and that does the right thing.

the naming conventions in rustlang are so good, tbh.

# ? Jan 28, 2022 03:39

Vanadium: Jan 8, 2005

anyhow has a feature flag for backtraces, doesn't it?

# ? Jan 28, 2022 09:17

Malloc Voidstar: May 7, 2007; Fuck the cowboys. Unf. Fuck em hard.

it's automatically determined https://github.com/dtolnay/anyhow/blob/master/build.rs

# ? Jan 28, 2022 14:15

Progressive JPEG: Feb 19, 2003

Rust tooling is generally pretty good but what's the deal with rustfmt ignoring the configured edition in Cargo.toml? Apparently for political reasons they want you to duplicate that information a separate rustfmt.toml, instead of just defaulting to whatever Cargo.toml has.

This came up because I'm setting up emacs to automatically rustfmt the current file whenever I save it, when previously I was doing 'cargo fmt' for the whole project every couple weeks or so. Switching to direct 'rustfmt' triggered all kinds of bizarre errors like 'what the hell is this "async fn" thing??' that turned out to be caused by rustfmt deciding to ignore the configured edition for the project.

I've gone with the workaround of just having emacs run '--edition 2021' everywhere so that rustfmt shuts up. That isn't great when I'm on projects that aren't on 2021 yet, but it's better than adding a rustfmt.toml to every project in order to solve a problem that shouldn't exist.

# ? Feb 19, 2022 22:36

Progressive JPEG: Feb 19, 2003

Like I can only imagine the kind of inferiority complex the rustfmt team must have for the cargo team in order for them to ignore the existence of Cargo.toml entirely, and to instead go with requiring everyone to duplicate their project configuration in a separate rustfmt.toml.

e: Meanwhile rustc themselves just point you to Cargo.toml when errors like this come up, which then leads to bugs filed (and closed) against rustfmt because they still don't honor Cargo.toml.

Progressive JPEG fucked around with this message at 22:49 on Feb 19, 2022

# ? Feb 19, 2022 22:42

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

I've never touched a rustfmt.toml in my life, and formatting buffers works fine in emacs, including on async code; maybe you've got a weird old version floating around? I might be going through LSP for the formatting...

# ? Feb 20, 2022 18:18

Ranzear: Jul 25, 2013

Should pop over and send a PR to cargo devs to create a symlink of cargo.toml to rustfmt.toml

No solution speaks louder than the passive aggressive one.

# ? Feb 22, 2022 19:45

luchadornado: Oct 7, 2004; A boombox is not a toy!

Ranzear posted:

Should pop over and send a PR to cargo devs to create a symlink of cargo.toml to rustfmt.toml

No solution speaks louder than the passive aggressive one.

Oh hey, another Minnesota goon!

# ? Feb 23, 2022 01:57

Progressive JPEG: Feb 19, 2003

Ranzear posted:

Should pop over and send a PR to cargo devs to create a symlink of cargo.toml to rustfmt.toml

No solution speaks louder than the passive aggressive one.

I indeed ended up passive aggressively putting up a working PR that adds Cargo.toml lookup into rustfmt - effectively doing what any reasonable person would expect rustfmt to already support. I now use the patched version locally and it works fine.

It sounds like part of the problem is that they're wanting to keep rustfmt as dumb as possible and to instead add individual file support into "cargo fmt", which is already Cargo.toml aware but doesn't support formatting individual files, only the entire project in one go. In any case I'm hoping my badgering them about it helps gets something across the line.

But ultimately it's a thankless volunteer project so I'm not gonna give them too much slack for dragging their feet on the solution even if it's pretty annoying. This is like the third time I've hit this same problem over the years with trying to automate file formatting and it's effectively a regression in rustfmt that they've been ignoring since Editions became a thing. e: Specifically, when async came out, suddenly rustfmt started needing an extra argument to not choke on it.

Progressive JPEG fucked around with this message at 03:59 on Feb 23, 2022

# ? Feb 23, 2022 03:36

Ranzear: Jul 25, 2013

luchadornado posted:

Oh hey, another Minnesota goon!

Seattle, actually. It's the nordic settlers in common.

# ? Feb 27, 2022 21:27

Love Stole the Day: Nov 4, 2012; Please give me free quality professional advice so I can be a baby about it and insult you

Bevy, the game engine, came out with v0.7 recently. Now you can do animations and skeletal meshes with the glTF 3D format. :yotj:

# ? Apr 19, 2022 02:07

Ranzear: Jul 25, 2013

https://twitter.com/m_ou_se/status/1527209443309633536?t=F2wrNi9SYgE_WiFLz2YBtA&s=19

# ? May 19, 2022 10:26

Mata: Dec 23, 2003

I'm trying to learn rust, and I figure I might as well direct my noob questions here.
I really like arrays over vectors. I like how there's no push/remove, the typed size, etc. But I'm struggling a bit to understand the implications of them being stack allocated.

Rust code:

struct Chunk {
    tile_array: [Tile; CHUNK_SIZE], // imho nicer than...
    tile_vec: Vec<Tile>,  // this. How safe is it to index into this really?
}

So I've blown up my stack allocating large arrays, and that makes sense. Is there a way to allocate large arrays directly on the heap?
Does iterating over an array pull the whole thing into the stack? if I do something like

Rust code:

let chunk = hashmap_of_chunks[key];
let tile = chunk.tile_array[5];

does that load the whole array onto the stack? What if I don't reference the tile_array member at all in that function? I assume the chunk struct's fields will be laid out contiguously, and one of those fields will be the whole array, itself contiguous. I assume doing something like

Rust code:

let boxed_array = Box::<[f32; LARGE_NUMBER]>::new([0.0; LARGE_NUMBER]);

will still attempt to allocate the array on the stack before moving it to the heap in the Box::new call. I've found this sized vec crate which I guess is what I want, but I'm a little hesitant about pulling in a crate for such a low level construct.

Mata fucked around with this message at 21:57 on May 19, 2022

# ? May 19, 2022 21:52

repiv: Aug 13, 2009

Large arrays are just annoying to work with right now, the compiler will usually elide the temporary stack allocation in release builds but it's hard to keep your debug builds from blowing up. I think the "box" keyword is supposed to fix that eventually.

If you really want to use arrays and only care about storing zeroable POD types like f32 then the bytemuck crate has a helper that can construct a Box<[T; N]> without an intermediate stack allocation in the meantime

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=33d0630c79f834949a8d3d22e8c7c01e

That works for anything implementing bytemucks Zeroable trait, which can be derived on your own types too if all their fields are also Zeroable

repiv fucked around with this message at 22:53 on May 19, 2022

# ? May 19, 2022 22:44

gonadic io: Feb 16, 2011; >>=

I would strongly recommend just use Vec for now. As repiv said, try running with --release and see if that changes anything. If you want to get fancy and really do want to construct Box<[u8; 1_000_000]> on the heap or whatever, you could jump straight into unsafe and uninitialised memory with

Rust code:

#![feature(new_uninit)]
pub fn main() {
    let b: Box<[f32; 1_000_000]> = unsafe { Box::new_zeroed().assume_init() };
}

But note that this only works with types that are safe to zero, i.e. primitives like float/int/enums containing those, etc.

fakeedit: this requires nightly but not an external crate, but also bytemuck::Zeroable won't let you do it with non-zeroable types.

Mata posted:

Does iterating over an array pull the whole thing into the stack? if I do something like
Rust code:
let chunk = hashmap_of_chunks[key];
let tile = chunk.tile_array[5];

Nope, this is fine to do.

e:

Rust code:

let boxed_array = Box::<[f32; LARGE_NUMBER]>::new([0.0; LARGE_NUMBER]);

This is one that DOES get optimised out in release mode, but 1.0 there wouldn't.

gonadic io fucked around with this message at 23:09 on May 19, 2022

# ? May 19, 2022 22:55

repiv: Aug 13, 2009

It feels like you should just be able to do Box::default() for that really, but Default is still only implemented for arrays up to 32 long

Not sure why there isn't a const-generic implementation of Default for any length array yet

# ? May 19, 2022 22:59

gonadic io: Feb 16, 2011; >>=

repiv posted:

It feels like you should just be able to do Box::default() for that really, but Default is still only implemented for arrays up to 32 long

Not sure why there isn't a const-generic implementation of Default for any length array yet

It's ongoing: https://github.com/rust-lang/rust/issues/61415

e: the holdup (since 4 May 2019) is

quote:

The current Default impl for [T; 0] doesn't have a T: Default bound so this would be a breaking change.

lol time for edition 2022 to just gently caress the old 0<N<=32 Default impls imo

gonadic io fucked around with this message at 23:10 on May 19, 2022

# ? May 19, 2022 23:01

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 23:38

pseudorandom name: May 6, 2007

std::vec::Vec::into_boxed_slice()

# ? May 20, 2022 01:28

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Rust: What the fuck is a borrow checker and why is it yelling at me?

«‹›31 »