Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Mescal
Jul 23, 2005

.

Mescal fucked around with this message at 23:41 on Aug 25, 2023

Adbot
ADBOT LOVES YOU

spiritual bypass
Feb 19, 2008

Grimey Drawer
It's fine dude, you can post itt. We live for this poo poo

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


Someone just zoom this guy. I’m sure someone here is helpful but it ain’t gonna be me.

BattleMaster
Aug 14, 2000

ExcessBLarg! posted:

While this might be suboptimal behavior it also seems to me like it shouldn't matter much. If your request rate is low enough that you're having extra wakeups when there isn't immediately another connection sitting in the accept queue, then your external load is low enough that overall performance shouldn't be impacted much.

Yeah, that's essentially the logic I've seen about why it's only theoretically a problem rather than a problem. An extra event in your queue that amounts to nothing when you're handling dozens of other events should barely even register. However in modern Linux it's a fairly trivial thing to avoid so I figure why not?

The next thing I'm working on is the accept then fork method. CERN's httpd and I'm sure tons of other things from the 90s or maybe even later use that. It avoids the thundering herd problem by only having one process accepting to begin with. And the cost of the fork is heavily offset by the massive simplification in the I/O logic in the worker process.

For example, httpd has parts in the parsing logic that just stop to do blocking reads whenever it realizes it needs more from the client. You can't do that in an event-based system - you have to go and do other things while waiting for more to arrive and that requires a little more record keeping.

I'm guessing maybe accepting and then handing the socket off to a thread is probably the modern equivalent. So I guess it's soon time for me to figure out threads.

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


jaegerx posted:

Someone just zoom this guy. I’m sure someone here is helpful but it ain’t gonna be me.

hell just send him all your junk from the computer closet you know you have and he'll figure out something hopefully.

e: quote is not edit, eh I don't care.

Klyith
Aug 3, 2007

GBS Pledge Week

please post:
• the exact model of computer you have that you're trying to install linux on
• what model of thumbdrive or whatever external storage that you're trying to put a linux installer on and boot from
• which distro of linux you want to install


All the instructions you are getting are vague handwavey poo poo because your posts of symptoms and descriptions are vague handwavey poo poo.

The last time I talked about this, you said you weren't asking for help just posting the struggle. If you want help, let's go back to basics and do this poo poo from square one. These idiotic quests to format a drive a different way in the hopes that it fixes a problem that probably has nothing to do with formatting are pointless.

BrainDance
May 8, 2007

Disco all night long!

Mr. Crow posted:

works fine op



I dunno then, cuz it's folder and folders of
Yet when that folder's in my main library or I make its own library
And that's just what I was told the last time I asked on IRC (which was a very long time ago)

Everything else it scrapes just fine.

edit: except the snick stuff, which ends up as that Swedish show that I have nothing else to change it to but whatever that's fine

BrainDance fucked around with this message at 04:07 on Apr 28, 2023

ExcessBLarg!
Sep 1, 2001

BattleMaster posted:

However in modern Linux it's a fairly trivial thing to avoid so I figure why not?
1. If you use systemd's socket activation your daemon is handed a socket from which it can accept, but the daemon itself doesn't have enough context (or privilege, even) to create additional sockets to bind/listen with. (At least, I think, I'm not too terribly familiar with the details of socket activation.)

2. If you're using a privileged port then your daemon has to run more code to create/bind/listen the additional sockets before dropping privileges. The naive way would be to fork, create socket, and drop privileges. But if you know how many workers you have you can just pre-create the total number of sockets before forking and close the unused ones in each child (or use threads).

BattleMaster posted:

The next thing I'm working on is the accept then fork method. CERN's httpd and I'm sure tons of other things from the 90s or maybe even later use that.
Does this httpd bound the number of worker processes or does it naively fork for each incoming request without regard to the total number? Because you can actually starve a system by having too many workers such that few of them can make progress timely.

BattleMaster posted:

I'm guessing maybe accepting and then handing the socket off to a thread is probably the modern equivalent. So I guess it's soon time for me to figure out threads.
The modernish method, assuming you want to use threads to manage the request context and not epoll+state machine, is to have a thread pool of fixed-ish size that can handle connections out of accept and then return to the pool when done. This avoids the overload potential of just naively forking when you have too many requests coming in, while also being more efficient by avoiding unnecessary clone/exit calls.

Incidentally I've written a pthreads-based thread pool implementation before, but not C11/C17 threads. I'm curious if the latter is actually viable. I've read that C11 threads were pretty under-spec'd.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

ExcessBLarg! posted:

1. If you use systemd's socket activation your daemon is handed a socket from which it can accept, but the daemon itself doesn't have enough context (or privilege, even) to create additional sockets to bind/listen with. (At least, I think, I'm not too terribly familiar with the details of socket activation.)

What context would it need that getsockname/getpeername don’t provide?

Mr. Crow
May 22, 2008

Snap City mayor for life
Weird, are there errors in the logs?

Your doing basic file / folder organization as recommended by https://jellyfin.org/docs/general/server/media/shows ?

I would guess maybe the # is causing you grief in the file names.


quote:

Avoid special characters such as * in M*A*S*H, use MASH instead.

ExcessBLarg!
Sep 1, 2001

Subjunctive posted:

What context would it need that getsockname/getpeername don’t provide?
Well, the socket protocol for one. Systemd could be configured to create a SOCK_STREAM socket with IPPROTO_SCTP instead of IPPROTO_TCP. I looked it up and there's a SO_PROTOCOL option to getsockopt that would help there.

As a general point though I'd argue that "the thing that creates the first socket should create all the sockets" to avoid the general class of problems that comes up with those things are done by two different entities making different assumptions.

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

Mescal posted:

whatever screwed it up is making the 1tb thumbdrive take about 12 hours to format.

I suspect two possibilities. Either you forgot the '/quick' option from the format command, or you have a fake thumbdrive. If you are really talking about a 1 terabyte USB stick and you also mention you are poor the latter sounds plausible.

Where did you by that thumbdrive and how much did you pay for it?

Mega Comrade
Apr 22, 2004

Listen buddy, we all got problems!

Klyith posted:

please post:
• the exact model of computer you have that you're trying to install linux on
• what model of thumbdrive or whatever external storage that you're trying to put a linux installer on and boot from
• which distro of linux you want to install




I just want to know this out of curiosity. Your posts may have been more normal 20 years ago, but Linux has been so streamlined in the last decade im kinda in awe of all the continuous issues you are hitting.

BrainDance
May 8, 2007

Disco all night long!

Saukkis posted:

If you are really talking about a 1 terabyte USB stick and you also mention you are poor the latter sounds plausible.

I actually just found out there even are real 1TB usb sticks. Went to google it and there are even real 2TB ones.

I've just never seen them around except for the many, many fake ones and never had a need for that much from a usb stick. So, that's neat!

Computer viking
May 30, 2011
Now with less breakage.

The sub-2.5" external SSDs are not quite sticks, but still so small they fit well in pockets. I have a 1TB Samsung T7, and it's kind of great. It's for work, so I didn't pay for it, but they're not prohibitively expensive if you have a use for it.

BlankSystemDaemon
Mar 13, 2009



Computer viking posted:

The sub-2.5" external SSDs are not quite sticks, but still so small they fit well in pockets. I have a 1TB Samsung T7, and it's kind of great. It's for work, so I didn't pay for it, but they're not prohibitively expensive if you have a use for it.
The real trick is to buy a USB to M.2 adapter that does both NVMe, SATA, and USB Attached SCSI (instead of USB Bulk-Only Transfer).
That way you can buy pretty much whatever storage size you want, and avoid getting the absolute-bottom-barrel non-volatile flash that's used for normal USB devices.

cruft
Oct 25, 2007

Subjunctive posted:

Duh, of course. I need to think more before posting, thank you.

Don't worry, I missed the reasoning too. OP's explanation was fabulous, I'm glad your question prompted it.

I wish this stuff had existed back when I was writing high performance servers. I could hane squeezed a whole lot more out of a single box.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
I'm not alone in that I still regularly have bad Linux days, despite running LTS Ubuntu, right?

Within the last year, I've had:
  1. Headset jack only wanting to be headphones, had to fix this by doing some really nasty poo poo with hdajackretask to reassign audio jacks after I couldn't figure it out by installing drivers or explicitly saying what model my Realtek codec was in config files.
  2. Ubuntu upgrading the kernel to 5.19 on me without asking me resulting in encountering a nasty GPU hang from what turns out to be a pretty bad AMDGPU bug: https://www.reddit.com/r/linux_gaming/comments/y7zl45/psa_for_amdgpu_users_kernel_519_and_likely_60_is/. Worked around this by upgrading to the 6.1 or 6.2 OEM kernel instead. Shame on Ubuntu for giving Desktop the HWE kernel by default, I had been expecting it to stay on 5.15.
  3. Trying to print resulting in me having to dig around https://github.com/pdewacht/brlaser with a bunch of fellow dorks who are logging if the open-source Brother Printer driver works or not.

The client linux experience still has some rough edges no matter what distro you're using, or maybe this is self inflicted and I really just need to jump to Debian stable and lower my expectations. I don't think any distro would have supported the ALC223 with a TRRS headset jack that does audio both in and out out of the box.

I'm not new to this, and I've even run into poo poo like this on a work-issued Dell supported Ubuntu laptop a while ago too.

cruft
Oct 25, 2007

Twerk from Home posted:

I'm not alone in that I still regularly have bad Linux computer days, despite running LTS Ubuntu, right?

No, you are not alone.

unruly
May 12, 2002

YES!!!

cruft posted:

No, you are not alone.
For real. Sometimes poo poo just does not want to work, or flakes out constantly. Those days make me regret working in this industry.

The days where everything just dovetails together perfectly, first try, green checkmarks and smooth, performant sailing make it worth it. :discourse:

BlankSystemDaemon
Mar 13, 2009



Yeah, definitely not alone.

If computers worked consistently, there wouldn't be an IT industry of the size that there is today.

Klyith
Aug 3, 2007

GBS Pledge Week

Mega Comrade posted:

I just want to know this out of curiosity. Your posts may have been more normal 20 years ago, but Linux has been so streamlined in the last decade im kinda in awe of all the continuous issues you are hitting.

TBQH the issues mescal is currently having aren't even with linux, since they can't even boot an installer. At least as far as I can understand.


cruft posted:

No, you are not alone.

Absolutely. Coming up on my 1 year of linux full-time on the desktop, I have had small problems with more frequency versus an average year of Windows. I have not had any problem as bad as time that my Windows install progressively corrupted the UWP apps, which began with unimportant things like the calculator and weather apps, and ended with the loving taskbar throwing errors. And that poo poo was absolutely impossible to fix.



Spaceballs was right.

cruft
Oct 25, 2007

I hate computers.

Mega Comrade
Apr 22, 2004

Listen buddy, we all got problems!
For me. I run into little things that I then make big things when I fool around with stuff I don't understand. But it still beats when I have had serious issues with Windows and the only possible fix is "uninstall , delete the app folder and then reinstall... didn't work? Well out of options then I guess"

Computer viking
May 30, 2011
Now with less breakage.

BlankSystemDaemon posted:

The real trick is to buy a USB to M.2 adapter that does both NVMe, SATA, and USB Attached SCSI (instead of USB Bulk-Only Transfer).
That way you can buy pretty much whatever storage size you want, and avoid getting the absolute-bottom-barrel non-volatile flash that's used for normal USB devices.

Those are also very nice, especially as a way to recycle M.2 sticks after an upgrade.

Computer viking
May 30, 2011
Now with less breakage.

Klyith posted:

TBQH the issues mescal is currently having aren't even with linux, since they can't even boot an installer. At least as far as I can understand.

Absolutely. Coming up on my 1 year of linux full-time on the desktop, I have had small problems with more frequency versus an average year of Windows. I have not had any problem as bad as time that my Windows install progressively corrupted the UWP apps, which began with unimportant things like the calculator and weather apps, and ended with the loving taskbar throwing errors. And that poo poo was absolutely impossible to fix.



Spaceballs was right.


About a month ago, my partner's windows machine just spontaneously started installing the bleeding edge developer version. There's no way to revert to the stable branch without reinstalling, and it was broken enough that some games didn't work, so ... yay.

Oh, and he also has a mac mini that absolutely does not want to wake up the monitor over USB-C; apparently a known issue that's been around since before the change to ARM chips, caused by what parts of Displayport Apple has chosen to implement. (Video over USB-C is apparently Displayport.) Every OS sucks in their own unique ways.

Computer viking fucked around with this message at 15:39 on Apr 28, 2023

ExcessBLarg!
Sep 1, 2001

Twerk from Home posted:

Shame on Ubuntu for giving Desktop the HWE kernel by default, I had been expecting it to stay on 5.15.
So the problem is if you buy a new desktop now and install 22.04 LTS on it, there's a good chance 3D acceleration or native video modes straight up don't work. I don't think it's unreasonable for the desktop installer to install the HWE stack since it will solve more hardware compatibility issues than it introduces. But I think it would be better if they gave you the choice.

Once you noticed the problem why not just install the non-HWE kernel and go right back to 5.15?

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

ExcessBLarg! posted:

So the problem is if you buy a new desktop now and install 22.04 LTS on it, there's a good chance 3D acceleration or native video modes straight up don't work. I don't think it's unreasonable for the desktop installer to install the HWE stack since it will solve more hardware compatibility issues than it introduces. But I think it would be better if they gave you the choice.

Once you noticed the problem why not just install the non-HWE kernel and go right back to 5.15?

It's because I'm an absolute glutton for punishment and would prefer to roll forward and encounter new and exiting bugs, maybe even report them. The chatter around this bug mentioned that a ton of work had been going into AMDGPU in general and newer kernels probably have some other nice stuff in there too. It had just been a while since a normal OS update completely blew me up, normally when I break my system I know that I did it myself.

Yaoi Gagarin
Feb 20, 2014

PSA for anyone using Ryzen CPUs: if you have kernel >= 6.1 you can add the amd_pstate=passive parameter to let it clock down lower at idle. Without it the lowest my 5600X would go was ~2GHz but now it goes down to 550MHz.

I think it's still idling warmer than on windows just based on the sound of the fans. But I need to install some dude's custom kernel module to see the real temperature: https://github.com/leogx9r/ryzen_smu

In other news, last night I tried pop OS and then tumbleweed, and ended up back on fedora again. I wish dnf wasn't so slow. Also I don't like that if you update from the GUI it does this Windows-like behavior where it restarts and has an updating screen with the logo and a progress bar and no other info. That's loving weird yo. But it has the least jank out of everything I've tried so I might stick with it

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



VostokProgram posted:

PSA for anyone using Ryzen CPUs: if you have kernel >= 6.1 you can add the amd_pstate=passive parameter to let it clock down lower at idle. Without it the lowest my 5600X would go was ~2GHz but now it goes down to 550MHz.

I think it's still idling warmer than on windows just based on the sound of the fans. But I need to install some dude's custom kernel module to see the real temperature: https://github.com/leogx9r/ryzen_smu

In other news, last night I tried pop OS and then tumbleweed, and ended up back on fedora again. I wish dnf wasn't so slow. Also I don't like that if you update from the GUI it does this Windows-like behavior where it restarts and has an updating screen with the logo and a progress bar and no other info. That's loving weird yo. But it has the least jank out of everything I've tried so I might stick with it

Red Hat is working on a new DNF that will be substantially faster. It might make it into the next version.

Fedora does offline upgrades (what they call the restart update behavior you're talking about) in order to ensure the system doesn't become corrupted by changing runtimes mid flight. You can do updates that don't reboot the system with the terminal by just running "dnf update" but its not recommended.

Klyith
Aug 3, 2007

GBS Pledge Week

Twerk from Home posted:

It's because I'm an absolute glutton for punishment and would prefer to roll forward and encounter new and exiting bugs, maybe even report them. The chatter around this bug mentioned that a ton of work had been going into AMDGPU in general and newer kernels probably have some other nice stuff in there too. It had just been a while since a normal OS update completely blew me up, normally when I break my system I know that I did it myself.

I also had the GPU ring timeout crash with 5.19, for me it was bad enough that I had to switch down to 5.15 for a cycle. Never even posted about it because the obvious solution worked fine.

Luckily I didn't have problems with 6.0 and 6.1, even though that bug wasn't fully resolved until 6.2 according to the internets. OTOH I also upgraded the GPU from a 5700 to a 6700 around that time and I think that helped a lot. The 5700 was just a little bit flaky.

VostokProgram posted:

PSA for anyone using Ryzen CPUs: if you have kernel >= 6.1 you can add the amd_pstate=passive parameter to let it clock down lower at idle. Without it the lowest my 5600X would go was ~2GHz but now it goes down to 550MHz.

I think it's still idling warmer than on windows just based on the sound of the fans. But I need to install some dude's custom kernel module to see the real temperature: https://github.com/leogx9r/ryzen_smu

Weird. I don't have that set, and mine definitely idles that low already:


Currently on 6.2 but I'm pretty sure is was idling that low well before now. Possibly some other software can enable that behavior, maybe KDE powerdevil?
(I already had the ryzen_smu module & ryzen monitor.)

VostokProgram posted:

In other news, last night I tried pop OS and then tumbleweed, and ended up back on fedora again. I wish dnf wasn't so slow. Also I don't like that if you update from the GUI it does this Windows-like behavior where it restarts and has an updating screen with the logo and a progress bar and no other info. That's loving weird yo. But it has the least jank out of everything I've tried so I might stick with it

Can you disable the boot splash screen to also disable that? I always disable boot splash, show me that text like I'm l33t.

Computer viking
May 30, 2011
Now with less breakage.

I dont mind the offline updates. I understand why the splash screen showing "installing updates, 37%" is a good default, too. But pressing Esc and getting a text prompt telling me "installing updates, 37%" with no further details does not fill me with happiness.

Mr. Crow
May 22, 2008

Snap City mayor for life
dnf is just as fast as any other package manager if you tune it a bit, no idea why their defaults are so slow.

Add this to /etc/dnf/dnf.conf

max_parallel_downloads=20
fastestmirror=True

And you can use `dnf -C` if you just want to run your command off the current cache.

Mr. Crow fucked around with this message at 17:56 on Apr 28, 2023

cruft
Oct 25, 2007

I just told new employer I wanted the HP x360 and not the MacBook.

I sure hope somebody asks later whether I want Windows or Linux, because I'm going to be completely and utterly up poo poo creek if I'm supposed to do productive work on this server code if I get a Windows box. :ohdear:

I've been losing sleep over this, actually. At this point I'll even be relieved to get RHEL5.

Mega Comrade
Apr 22, 2004

Listen buddy, we all got problems!

Computer viking posted:

I dont mind the offline updates. I understand why the splash screen showing "installing updates, 37%" is a good default, too. But pressing Esc and getting a text prompt telling me "installing updates, 37%" with no further details does not fill me with happiness.

Remove 'rhgb quite' from your boot parameters. That should switch it to spitting out everything it's doing, including updates.

Mega Comrade fucked around with this message at 17:55 on Apr 28, 2023

Computer viking
May 30, 2011
Now with less breakage.

Mega Comrade posted:

Remove 'rhgb quite' from your boot parameters. That should switch it to spitting out everything it's doing, including updates.

Right, makes sense.

It's still a little bit odd as a default. On most machines with a splash screen boot, choosing to dismiss it probably means you're trying to see more details - going out of their way to hide the details there feels like a misunderstanding.

Kibner
Oct 21, 2008

Acguy Supremacy

cruft posted:

I just told new employer I wanted the HP x360 and not the MacBook.

I sure hope somebody asks later whether I want Windows or Linux, because I'm going to be completely and utterly up poo poo creek if I'm supposed to do productive work on this server code if I get a Windows box. :ohdear:

I've been losing sleep over this, actually. At this point I'll even be relieved to get RHEL5.

The MacBook hardware is just so much nicer than every other laptop, though. :(

I'm guessing too much of the AS Macs are super proprietary so there is no easy way to run *nix on them, right?

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

The port for Apple Silicon is proceeding, but I don’t think is ready to be a daily driver yet.

Yaoi Gagarin
Feb 20, 2014

I'll double check the amd_pstate thing tonight. I did a bit more googling and phoronix says it might perform worse than the usual acpi driver: https://www.phoronix.com/news/AMD-P-State-Built-In-Options

Although he doesn't say what exactly the regression is. :shrug:

By the way, is it possible to dump the list of supported kernel parameters?

Adbot
ADBOT LOVES YOU

BattleMaster
Aug 14, 2000

ExcessBLarg! posted:

Does this httpd bound the number of worker processes or does it naively fork for each incoming request without regard to the total number? Because you can actually starve a system by having too many workers such that few of them can make progress timely.

So httpd is a mess of #defines because it supports like every Unix-like of the mid-90s plus Windows, and it has the ability to support platforms with select and/or fork, or neither. So the main loop of the daemon is little confusing. But as far as I can tell, it does not set a maximum number of clients anywhere. I can kind of get why something originally developed for use by researchers and academics rather than the public might not have been built with the expectation that people would be loving with it.

Here's the source and here's the daemon's main loop.

It would be fairly easy with the available tools, even in the mid 90s. to enforce a maximum number of active requests (trivial, I do in my current experiments) and to keep track of current number of active requests from the same IP address (adding the cost of iterating through your list of active clients and counting, or perhaps keeping a separate mapping of IPs to concurrent connections.) The parent would also need to keep track of which PIDs belong to which IP so that the count can be updated when the worker exits.

The duration of a request should also be managed, but probably not by the parent. Some HTTP requests like the downloading of files could take hours, especially in the 90s. So I think that the worker should at least use the socket options to abort reads after a certain idle time (httpd doesn't even do that as far as I can tell, and I think that existed in the 90s.) If those socket options don't exist then I guess the only real choice would be to use polled I/O, but I think you could replace every normal socket read with a function that wraps select and read without having to actually rewrite anything - have select wait on the socket to be readable for 60 seconds or whatever and return -1 if the timeout occurs, or doing the read normally otherwise. And make sure to be aggressive on ending things as soon as possible if the read doesn't work out.

Without doing those things you could trivially DoS it with a bunch of netcat instances (or 90s equivalent) spawned in a loop and sitting idle until it forks too much and blows up.

Also this made me laugh - it was in one of the header files but I can't remember which one and I don't know how to use github's search:

BattleMaster fucked around with this message at 21:34 on Apr 28, 2023

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply