Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Gyshall
Feb 24, 2009

Had a couple of drinks.
Saw a couple of things.
Solaar is also good. You might need to make sure you've installed Logitech udev rules as well

Adbot
ADBOT LOVES YOU

Klyith
Aug 3, 2007

GBS Pledge Week

busalover posted:

That looks cool. But my MX518 is not getting recognised. Maybe I need to restart? Weird, can't be because it's too old.

Is this an original MX518? Like, you've had it for 10+ years? If so, it's too old. Piper is only for mice with onboard memory where the mouse then does stuff on its own.

If so, and you're still on Xorg, xbindkeys is powerful but has no GUI. If you're on Wayland, I would TBQH suggest getting a new mouse with onboard profiles.


OTOH if this is a post-2019 MX518, it should work with Piper. It looks like various distros have at times screwed up Piper by shipping incompatible versions of Piper and ratbag (the backend). Running ratbagctl list will tell you if ratbag can see your mouse.

Truga
May 4, 2014
Lipstick Apathy

Klyith posted:

Is this an original MX518? Like, you've had it for 10+ years? If so, it's too old. Piper is only for mice with onboard memory where the mouse then does stuff on its own.

If so, and you're still on Xorg, xbindkeys is powerful but has no GUI. If you're on Wayland, I would TBQH suggest getting a new mouse with onboard profiles.

https://github.com/sezanzeb/input-remapper
this works on wayland

ExcessBLarg!
Sep 1, 2001

pseudorandom name posted:

I like how you completely ignored the "I'm talking about the span roughly from where grub hands control over to the decompressed kernel to the point that systemd takes over" in the original post.
That's my point, GRUB doesn't hand control to a decompressed kernel, but to the kernel's own decompressor stub.

Anyways, if I failed to provide immediate relevant information I hopefully provided a few tangentially-related observations that I know are little else documented otherwise.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!

I eventually got this recommendation elsewhere too and seems to be the best thing there is so far. I'm almost to the point of calling it "the authority" because I haven't found anything else.

There is one very important point I got out of even just skimming that which may be of note to anybody else: there is Linux kernel booting, and then there is Linux kernel initialization. The booting part takes care of the Grub handover and the kernel getting itself up into memory, but initialization takes care of bringing up all the subsystems and getting systemd going at the end. So searching for stuff about the Linux kernel boot process only gave me the first part. That's how we get to stuff like this:

pseudorandom name posted:

I like how you completely ignored the "I'm talking about the span roughly from where grub hands control over to the decompressed kernel to the point that systemd takes over" in the original post.

I did also start off by asking about the "Linux kernel boot process" so the human Google gave the most literal answer based on what I learned about the terminology! Also, this is still true:

ExcessBLarg! posted:

You'll probably mostly have to figure it out on your own.

It's still a bit hosed.

Volguus
Mar 3, 2009
What's your goal? What do you want to achieve?

On a Debian live system (that's what I have experience with, many years ago), init is just a shell script (they use ash), which executes all the other shell scripts in order. They have the concept of stages (I think 4 if I remember correctly, I could be wrong), so you can break before any stage you want to debug things. The main purpose of these scripts is to detect where the root filesystem is, mount it then pass over control to that (systemd).

That means, probing for all the disks, usb flash drives, decrypt anything that may need decrypting, etc. You can create your own scripts, in whatever stage you need/want and have them included in the final initramfs. With hooks you can include kernel modules, binaries, whatever else you may need.

One good piece of documentation that I saw in the past (it's light):
https://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html
https://wiki.ubuntu.com/Initramfs

Is this what you were looking for? How to probe for devices and mount a root filesystem?

cruft
Oct 25, 2007

I haven't been keeping up, but if we're still talking about early user space, just pass "init=/bin/sh" on the kernel command line, and read through scripts until you figure out how to make your system work normally.

Here's something I wrote about all this a while ago: https://woozle.org/papers/pid1/

Here's something else: https://woozle.org/papers/runit-as-init/

cruft fucked around with this message at 03:41 on Oct 5, 2023

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Great articles! You can do Berkeley sockets from bash with /dev/tcp and friends, I think. You’re right that Unix hasn’t been deeply Unixy in a really long time, though.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
init and the shell are too late. I have a few things I intend to do, and some of them are work stuff, but I can vaguely list some stuff:

1. Code for a kernel function seems to get rewritten at some point during the boot and I want to figure out where and why. I have not yet successfully nailed it with a memory watch so I was going to just pace through it and narrow where it mutates, but I need to know the earliest I can hit any debugger in the kernel (or just in QEMU at technically any point, I suppose.)
2. We have a USB serial device brand that is causing panics in 6.x kernels while it is being set up and I am trying to figure out why. Our underlying hardware is not golden either. We use it for the serial console so we are less than three seconds in. It could also not be that but something else but I get nothing useful when kdb gets to it.
3. Personal one here: my Ubuntu setup sometimes still reaches the luks password prompt without initializing the keyboard and I am trying to figure out what is actually happening. I have tried to force the drivers to be loaded by initramfs, but that apparently is not enough.

I have to munge on kernel code for work so dropping into kgdb or even kdb and running around is a thing to already have to do.

So I am hoping that is enough to justify understand what the kernel is doing during boot and initialization.

pseudorandom name
May 6, 2007

The kernel function is probably getting rewritten by the alternatives system.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!

pseudorandom name posted:

The kernel function is probably getting rewritten by the alternatives system.

Thanks for that tip. It doesn't google particularly well though, so I was going to just grep around the kernel source tomorrow and hope it comes up. "Linux kernel alternatives system" brings up all kinds of things to use instead of Linux lol.

ihafarm
Aug 12, 2004

Rocko Bonaparte posted:

Thanks for that tip. It doesn't google particularly well though, so I was going to just grep around the kernel source tomorrow and hope it comes up. "Linux kernel alternatives system" brings up all kinds of things to use instead of Linux lol.

kernel altinstructions is what I think they’re referring to

Well Played Mauer
Jun 1, 2003

We'll always have Cabo
I feel dumb asking this but I recently switched to Plasma and the middle click to paste thing is driving me nuts. Is there a plug-in or something I can use to disable it? Stackexchange had some 10-year-old answers like remapping middle click to another button (not ideal since I still want middle click where it is). Gnome just has a setting to disable paste but I otherwise prefer Plasma for the most part.

Klyith
Aug 3, 2007

GBS Pledge Week

Well Played Mauer posted:

I feel dumb asking this but I recently switched to Plasma and the middle click to paste thing is driving me nuts. Is there a plug-in or something I can use to disable it? Stackexchange had some 10-year-old answers like remapping middle click to another button (not ideal since I still want middle click where it is). Gnome just has a setting to disable paste but I otherwise prefer Plasma for the most part.

Settings -> Workspace Behavior -> General Behavior -> Middle Click Paste selected checkbox

(KDE has settings for everything, but the trouble with that is finding the needle in the checkbox haystack.)



I loving love middle-click c/p selected and am now intensely annoyed every time I go to a windows machine and it doesn't work.

Well Played Mauer
Jun 1, 2003

We'll always have Cabo
Thank you so much!

I actually like it too but stuff like Miro and Roll20 use middle click and drag for navigation and such, so I’ll inadvertently paste something constantly.

busalover
Sep 12, 2020

Klyith posted:

Is this an original MX518? Like, you've had it for 10+ years? If so, it's too old. Piper is only for mice with onboard memory where the mouse then does stuff on its own.


yeah I bought it when HL2 released

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.
Is there a command that gives me the size of the actual current data on a zfs volume? So something that approximates the output of du -s.
The commands I found either include snapshots for space usage or don't count data that is duplicated in a snapshot. Du -s gives me what I want, but I am looking for something that takes less then an hour to run.

Zorak of Michigan
Jun 10, 2006


VictualSquid posted:

Is there a command that gives me the size of the actual current data on a zfs volume? So something that approximates the output of du -s.
The commands I found either include snapshots for space usage or don't count data that is duplicated in a snapshot. Du -s gives me what I want, but I am looking for something that takes less then an hour to run.

Something like

code:
zfs list -t filesystem -o space -r
might give you what you want, assuming you mean a zpool or zfs file system, not an actual zfs volume. Zfs volumes are opaque by nature.

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

Zorak of Michigan posted:

Something like

code:
zfs list -t filesystem -o space -r
might give you what you want, assuming you mean a zpool or zfs file system, not an actual zfs volume. Zfs volumes are opaque by nature.

Yes, I meant a zpool. And the sum of the usedds column seems to be what I wanted.

BlankSystemDaemon
Mar 13, 2009



zfs get logicalused <dataset> will, as the zfsprops(7) manual pages says, give you the logical space - meaning that it ignores compression and ditto blocks, while not accounting for the size of the metadata.

Since compression is enabled by default and is in-line and completely transparent, there's no easy way to work out the on-disk size - but typically that's not very relevant.

VictualSquid posted:

Yes, I meant a zpool. And the sum of the usedds column seems to be what I wanted.
The above command will work if you use it on the parent dataset of everything else.

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

BlankSystemDaemon posted:

zfs get logicalused <dataset> will, as the zfsprops(7) manual pages says, give you the logical space - meaning that it ignores compression and ditto blocks, while not accounting for the size of the metadata.

Since compression is enabled by default and is in-line and completely transparent, there's no easy way to work out the on-disk size - but typically that's not very relevant.

I need the numbers to estimate how much space a copy will need, so no need for on disk stuff.

BlankSystemDaemon posted:

The above command will work if you use it on the parent dataset of everything else.

It gives:
code:
zfs list -t filesystem -o space -r tank/nobackup
NAME                            AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
tank/nobackup                   13.8T  8.06T      196K    103K             0B      8.06T
tank/nobackup/backups           13.8T  1.88T      353G    987G             0B       589G
....
So, the total usage is summed up for the root datasets. But the categories aren't. Makes sense for CHILD, but not for the other ones, I suppose they wanted to be consistent.

BlankSystemDaemon
Mar 13, 2009



VictualSquid posted:

I need the numbers to estimate how much space a copy will need, so no need for on disk stuff.

It gives:
code:
zfs list -t filesystem -o space -r tank/nobackup
NAME                            AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
tank/nobackup                   13.8T  8.06T      196K    103K             0B      8.06T
tank/nobackup/backups           13.8T  1.88T      353G    987G             0B       589G
....
So, the total usage is summed up for the root datasets. But the categories aren't. Makes sense for CHILD, but not for the other ones, I suppose they wanted to be consistent.
You're using the space property, not the logicalused property that I suggested.

pseudorandom name
May 6, 2007

ihafarm posted:

kernel altinstructions is what I think they’re referring to

I'm referring to arch/x86/kernel/alternative.c and arch/x86/include/asm/alternative.h (and similar for several other architectures).

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

BlankSystemDaemon posted:

You're using the space property, not the logicalused property that I suggested.

That logicalused command seems to include snapshots in the count, which is what I wanted to avoid.
code:
root@tobias-manjaro /h/tobias [1]# zfs list -t filesystem -o space -r tank/nobackup/backups/borg
NAME                        AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
tank/nobackup/backups/borg  13.8T   481G      253G    229G             0B         0B
root@tobias-manjaro /h/tobias# zfs get logicalused tank/nobackup/backups/borg
NAME                        PROPERTY     VALUE  SOURCE
tank/nobackup/backups/borg  logicalused  478G   -
root@tobias-manjaro /h/tobias# du -hs /data/backups/borg/
229G    /data/backups/borg/

BlankSystemDaemon
Mar 13, 2009



VictualSquid posted:

That logicalused command seems to include snapshots in the count, which is what I wanted to avoid.
code:
root@tobias-manjaro /h/tobias [1]# zfs list -t filesystem -o space -r tank/nobackup/backups/borg
NAME                        AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
tank/nobackup/backups/borg  13.8T   481G      253G    229G             0B         0B
root@tobias-manjaro /h/tobias# zfs get logicalused tank/nobackup/backups/borg
NAME                        PROPERTY     VALUE  SOURCE
tank/nobackup/backups/borg  logicalused  478G   -
root@tobias-manjaro /h/tobias# du -hs /data/backups/borg/
229G    /data/backups/borg/
Well, firstly you need to stop using du - because just like every other userland utility, it has no way of understanding the dnode structure used in ZFS, so whatever number it's reporting isn't correct.

As for whether it includes snapshots, I don't believe it does considering that it's looking at ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME.

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

BlankSystemDaemon posted:

Well, firstly you need to stop using du - because just like every other userland utility, it has no way of understanding the dnode structure used in ZFS, so whatever number it's reporting isn't correct.

As for whether it includes snapshots, I don't believe it does considering that it's looking at ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME.

So if I tar up all the files in the dataset is should be closer to the 478G then the 229G? I just started the command, it will run for a bit.

BlankSystemDaemon
Mar 13, 2009



VictualSquid posted:

So if I tar up all the files in the dataset is should be closer to the 478G then the 229G? I just started the command, it will run for a bit.
The number you get from du will be an approximation of the on-disk size, becasue it doesn't understand ZFS' notion of compression, metadata, and padding (if using raidz).

Similarly, df doesn't understand the notion of datasets being different from filesystems, and doesn't understand that they can have descendants or snapshots, or how quotas or reservations work as they exist in ZFS. Nor, like du, does it understand compression or how metadata works.

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

BlankSystemDaemon posted:

The number you get from du will be an approximation of the on-disk size, becasue it doesn't understand ZFS' notion of compression, metadata, and padding (if using raidz).

Similarly, df doesn't understand the notion of datasets being different from filesystems, and doesn't understand that they can have descendants or snapshots, or how quotas or reservations work as they exist in ZFS. Nor, like du, does it understand compression or how metadata works.

Yes, I know about all of those options having limitations. And df has always been bad for zfs and btrfs.
I specifically want to know which of those commands most closely approximates the free space I need on my other computer to mirror all the current data, without backing up the old snapshots.

e:Tar has finished running on my test subset. It is of similar size as du's estimate. And half of what your command would have suggested. This is for incompressible data though.

VictualSquid fucked around with this message at 13:11 on Oct 6, 2023

Zorak of Michigan
Jun 10, 2006


Wouldn't the USEDDS be closer to what VictualSquid wants? I believe that's a shortname for "usedbydatasets."

BlankSystemDaemon
Mar 13, 2009



VictualSquid posted:

Yes, I know about all of those options having limitations. And df has always been bad for zfs and btrfs.
I specifically want to know which of those commands most closely approximates the free space I need on my other computer to mirror all the current data, without backing up the old snapshots.

e:Tar has finished running on my test subset. It is of similar size as du's estimate. And half of what your command would have suggested. This is for incompressible data though.
But how are you checking the size of the tarball, and is it on a ZFS pool? If so, it'll likely fall afoul of the exact same issues, because ls is no better than du or df.

Computer viking
May 30, 2011
Now with less breakage.

BlankSystemDaemon posted:

But how are you checking the size of the tarball, and is it on a ZFS pool? If so, it'll likely fall afoul of the exact same issues, because ls is no better than du or df.

Imagine tar without compression piped into wc to count bytes in the stream. That's still "wrong" because metadata and wasted space in blocks and so on being different - but which number would that be closest to?

Computer viking fucked around with this message at 18:29 on Oct 6, 2023

BlankSystemDaemon
Mar 13, 2009



Computer viking posted:

Imagine tar without compression piped into wc to count bytes in the stream. That's still "wrong" because metadata and wasted space in blocks and so on being different - but which number would that be closest to?
Sure, that's one way of doing it - especially if you pipe it to /dev/null.
Is that what was done, though?

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

BlankSystemDaemon posted:

Sure, that's one way of doing it - especially if you pipe it to /dev/null.
Is that what was done, though?

What would you pipe to /dev/null? Doesn’t the data have to go into wc to get counted?

BlankSystemDaemon
Mar 13, 2009



Subjunctive posted:

What would you pipe to /dev/null? Doesn’t the data have to go into wc to get counted?
I meant after wc has counted it, but forgot the word "after".

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

BlankSystemDaemon posted:

I meant after wc has counted it, but forgot the word "after".

wc just outputs the count, no? I think op wants to see the count!

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

BlankSystemDaemon posted:

But how are you checking the size of the tarball, and is it on a ZFS pool? If so, it'll likely fall afoul of the exact same issues, because ls is no better than du or df.
ls and du and df and usedds all give around 230G. Used gave around 478G, for the original dataset and I am still quite certain it includes old snapshots.
I expect copying the taball to an ntfs drive with 300G free to work. It is copying right now.

I know that all those commands give only approximations in most situations. I am just confused which scenario you are envisioning where used is the closest to my actual needs.

BlankSystemDaemon
Mar 13, 2009



Subjunctive posted:

wc just outputs the count, no? I think op wants to see the count!
Oh gently caress, yeah - wc doesn't output to stderr, I read the source wrong.

And since it's Linux, there's no SIGINFO available.

BlankSystemDaemon
Mar 13, 2009



VictualSquid posted:

ls and du and df and usedds all give around 230G. Used gave around 478G, for the original dataset and I am still quite certain it includes old snapshots.
I expect copying the taball to an ntfs drive with 300G free to work. It is copying right now.

I know that all those commands give only approximations in most situations. I am just confused which scenario you are envisioning where used is the closest to my actual needs.
So the solution is programmatically add together the values from fixed]zfsprops(7)[/fixed] zfs get -t snapshot | grep used, then detract it from the value of zfs get logicalused <dataset>.

BlankSystemDaemon fucked around with this message at 19:34 on Oct 6, 2023

VictualSquid
Feb 29, 2012

Gently enveloping the target with indiscriminate love.

BlankSystemDaemon posted:

So the solution is programmatically add together the values from fixed]zfsprops(7)[/fixed] zfs get -t snapshot | grep used, then detract it from the value of zfs get logicalused <dataset>.

That sounds needlessly convoluted.

e: And summing "zfs get -t snapshot used -p tank/nobackup/backups/borg" seems to give me a much too low value. A tenth of the "usedsnap" value.

ee: and zfs get -t snapshot is one of those commands that take forever to run, which is the original reason why I asked for something faster then du.

VictualSquid fucked around with this message at 19:59 on Oct 6, 2023

Adbot
ADBOT LOVES YOU

cruft
Oct 25, 2007

BlankSystemDaemon posted:

Oh gently caress, yeah - wc doesn't output to stderr, I read the source wrong.

And since it's Linux, there's no SIGINFO available.

It's okay, we know you meant pv when you wrote wc

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply