|
Cannon_Fodder posted:1password rules, thank you for the suggestions
|
#
?
Mar 29, 2024 14:23
|
|
|
|
| # ? Apr 28, 2024 20:41 |
|
|
A developer found that a backdoor had been put in the xz/liblzma library which can affect ssh on certain distros. https://seclists.org/oss-sec/2024/q1/268 Redhat has put out a big warning for Fedora 41/rawhide. https://www.redhat.com/en/blog/urgent-security-alert-fedora-41-and-rawhide-users
|
|
#
?
Mar 29, 2024 19:29
|
|
|
happy no deploy Friday everyone
|
|
#
?
Mar 29, 2024 19:30
|
|
|
Fedora and OpenSuse have had to roll back whereas RHEL and SLES didn't have the code, so it was probably caught soon enough to minimise enterprise use.
|
|
#
?
Mar 29, 2024 19:38
|
|
|
Some knob will have his private Fedora box.
|
|
#
?
Mar 29, 2024 20:33
|
|
|
Pablo Bluth posted:A developer found that a backdoor had been put in the xz/liblzma library which can affect ssh on certain distros. liblzma deez nuts
|
|
#
?
Mar 30, 2024 01:57
|
|
|
Pablo Bluth posted:A developer found that a backdoor had been put in the xz/liblzma library which can affect ssh on certain distros. My arch desktop had the bad xz library on it. No sshd running so in theory nothing happened. Though any process that linked liblzma would have gotten the bad rsa function I think. Either way pretty disconcerting knowing your machine had malicious code on it. Also now there's questions about changes to libarchive nearly two years ago from the same contributor, so we may not have found the extent of this yet. Not to mention how many other libraries with only a couple maintainers and little review end up runtime linked one way or another in critical processes like sshd.
|
|
#
?
Mar 30, 2024 06:48
|
|
|
Jesus loving Christ
|
|
#
?
Mar 30, 2024 15:54
|
|
|
We'll be pushing the year of the Linux desktop back one preemptively, just to be safe
|
|
#
?
Mar 30, 2024 17:19
|
|
|
Also most people are reverting to 5.4 and/or stable distros are on 5.4, but that still contains hundreds of commits from a now known bad actor. Reverting to 5.2 or early 5.3 might be wise but lots of stuff like even dpk will have to be modified/rebuilt so it's going to take real time. Or dropping xz entirely in favor of zstd or something. Everybody is digging through the payload and presumably the other commits this person/organization made to xz as we speak, but there could be all kinds of stuff like code execution from extracting special files or the like lurking on millions of stable release machines out there. With no quick way to repair them, you can't uninstall xz nor rollback. Hopefully we'll know soon but yeah even if you dodged the sshd payload you've possibly been running some truly nasty poo poo for more than a year now.
|
|
#
?
Mar 30, 2024 17:21
|
|
|
Rescue Toaster posted:My arch desktop had the bad xz library on it. No sshd running so in theory nothing happened. Though any process that linked liblzma would have gotten the bad rsa function I think. If I'm understanding this correctly, the function replacement and all the other wackiness only happens if it's loaded as part of /usr/sbin/sshd, so you should be safe from this backdoor. Who knows how many others are yet to be discovered, though.
|
|
#
?
Mar 30, 2024 17:33
|
|
|
Shumagorath posted:I wrote them off for a decade due to their Mac focus, but they’re truly excellent. They still publish patch notes on iOS long after other devs gave up and started insulting their customers. Windows/Linux app got rewritten in electron and is now at pretty much feature parity with the the mac version, with the exception of being able to control the screen to autofill into apps not just the web browser. But that depends on apple specific accessibility apis so w/e. Browser extensions are also pretty great and almost obviate needing to install the app at all. I’d give it another shot.
|
|
#
?
Mar 30, 2024 18:09
|
|
|
Rescue Toaster posted:Also most people are reverting to 5.4 and/or stable distros are on 5.4, but that still contains hundreds of commits from a now known bad actor. Reverting to 5.2 or early 5.3 might be wise but lots of stuff like even dpk will have to be modified/rebuilt so it's going to take real time. Or dropping xz entirely in favor of zstd or something. I'd favour reverting everything and reimplementing what's necessary. It, along with the time period of over two years, is also what makes me almost certain we're dealing with a state-sponsored APT and not some individual who may or may not have been compromised. BlankSystemDaemon fucked around with this message at 19:47 on Mar 30, 2024 |
|
#
?
Mar 30, 2024 19:41
|
|
|
Generic Monk posted:I’d give it another shot.
|
|
#
?
Mar 30, 2024 20:23
|
|
|
Sounds like it might be for remote code exec rather than bypassing authentication. https://bsky.app/profile/filippo.abyssdomain.expert/post/3kowjkx2njy2b
|
|
#
?
Mar 30, 2024 21:02
|
|
|
Pablo Bluth posted:Sounds like it might be for remote code exec rather than bypassing authentication. Passing stuff to system() as root, yeah. Even worse than the attacker having to guess at a sudo-enabled account name or something. BlankSystemDaemon posted:With how fiendishly clever the suspect libarchive PR is, I seriously have my doubts everything will be found, as it looks to be designed to take advantage of terminal escape character sequences. I would be shocked if there isn't at least a special crafted compressed file/buffer -> code execution path in basically every LTS & release distro out there running xz 5.4+, aka every production Linux system out there. Even if they don't find it immediately I don't know how you'd trust any up to date Linux distro in existence right now. It's truly staggering how bad this might be.
|
|
#
?
Mar 30, 2024 21:15
|
|
|
Generic Monk posted:Windows/Linux app got rewritten in electron and is now at pretty much feature parity with the the mac version I think technically they rewrote the Mac version using Electron, so now it's the same shitware on all 3 operating systems.
|
|
#
?
Mar 30, 2024 22:17
|
|
|
As someone entirely outside the industry who does not personally run any Linux distro on any hardware, what threats might this have exposed me to via institutions? Could my personal info have been compromised at financial or educational institutions as a result of this backdoor?
|
|
#
?
Mar 30, 2024 22:38
|
|
|
DoctorWhat posted:As someone entirely outside the industry who does not personally run any Linux distro on any hardware, what threats might this have exposed me to via institutions? Could my personal info have been compromised at financial or educational institutions as a result of this backdoor? I don't think anyone knows yet. The various threads I'm reading seem to be people still scrambling to figure things out, meanwhile doing damage control.
|
|
#
?
Mar 30, 2024 22:48
|
|
|
DoctorWhat posted:As someone entirely outside the industry who does not personally run any Linux distro on any hardware, what threats might this have exposed me to via institutions? Could my personal info have been compromised at financial or educational institutions as a result of this backdoor? This is hard to judge, or rather, maybe not worse than any other common industrywide security failure. This most egregious remote ssh code execution payload doesn't appear to have reached any production builds that are widely used. But nobody knows how many deliberate holes are in xz 5.3 and beyond, or how they might have been exploited in the field in the past two years. Really it's especially concerning because it highlights how easily this can be done if a serious adversary is willing to play a long game (and two years is not that long in the scheme of things). Yes this xz situation is extremely bad, but so are the probably dozens+ of other common libraries that have vulnerabilities either deliberately introduced or discovered but not disclosed so far. These xz guys were not even that good, making some silly valgrind issues and introducing excessive performance issues that were probably all avoidable. Rescue Toaster fucked around with this message at 22:59 on Mar 30, 2024 |
|
#
?
Mar 30, 2024 22:56
|
|
|
DoctorWhat posted:As someone entirely outside the industry who does not personally run any Linux distro on any hardware, what threats might this have exposed me to via institutions? Could my personal info have been compromised at financial or educational institutions as a result of this backdoor? The risk is lateral movement inside a target. No financial institution exposes Linux distros directly to the Internet - the only ways in is via web apps sitting in a DMZ behind firewalls and possibly web application proxies, or via Citrix or VPN client (using MFA) to get to the office LAN - the server LAN should still be segregated by a firewall requiring MFA So your primary risk is an attacker who has broken in to the server network via some method can use the ssh flaw to move laterally in the server LAN, going after Linux-based databases or middleware. Microsegmentation would help. But assuming the statement is true that Fedora is affected but RHEL isn't, this bug poses no significant risk because only loving idiots would deploy Fedora in a banking environment instead of RHEL. Your educational institution, not taking that bet. Rust Martialis fucked around with this message at 00:35 on Mar 31, 2024 |
|
#
?
Mar 31, 2024 00:31
|
|
|
Most people are still focused only on the sshd payload at this point. RHEL and everybody else still almost certainly have versions of xz that have tons of other changes from the malicious actor. Debian apparently has shut down and is rebuilding their entire build/package system. But unless they're rolling back to a 4 year old version of xz I'm not sure what they're hoping to accomplish. Fedora 40 is supposed to be coming soon too, so it'll be interesting to see if they take the risk seriously or just bandaid over the sshd exploit specifically.
|
|
#
?
Mar 31, 2024 00:51
|
|
|
Rescue Toaster posted:Most people are still focused only on the sshd payload at this point. RHEL and everybody else still almost certainly have versions of xz that have tons of other changes from the malicious actor. RHEL wasn't impacted as far as they are aware, even Debian its not everyone. Fedora was affected however. https://www.darkreading.com/vulnerabilities-threats/are-you-affected-by-the-backdoor-in-xz-utils quote:Red Hat: Vulnerable packages are present in Fedora 41 and Fedora Rawhide. No versions of Red Hat Enterprise Linux (RHEL) are affected. Red Hat says users should immediately stop using the affected versions until the company has had a chance to change the xz version. This actually got caught fairly early, thankfully, but it was well on its way. CommieGIR fucked around with this message at 03:23 on Mar 31, 2024 |
|
#
?
Mar 31, 2024 03:19
|
|
|
CommieGIR posted:This actually got caught fairly early, thankfully, but it was well on its way. One of the most disturbing aspects of this story is that it wasn't "caught" per se, it was stumbled upon by a thankfully brilliant Postgres developer who happened to be investigating a performance issue which turned out to be the result of the backdoor. This guy is truly a hero; we'd be in a terrible spot had this been discovered later. The fact that the committer(s) used multiple accounts to get this circulated, and are experienced enough to become primary maintainers of an ubiquitous library is also terrifying. It's likely they have commit access to projects other than xz. The backdoor author ("Jia Tan") is even listed as a reviewer on some xz-related patchsets for the linux kernel itself: https://lore.kernel.org/lkml/20240320183846.19475-2-lasse.collin@tukaani.org/
|
|
#
?
Mar 31, 2024 06:29
|
|
|
CommieGIR posted:RHEL wasn't impacted as far as they are aware, even Debian its not everyone. Fedora was affected however. Again, this is focused on the sshd payload specifically. The same account that placed that has made hundreds of commits to xz for years, and there's clear evidence they were a bad actor right away rather than a stolen account. It is insane to trust the 5.4 version of xz that people are "rolling back to". Half of the drat Linux world links against liblzma in xz 5.4 or later, including package managers, systemd, binutils... They need to roll back to at an absolute minimum 5.3ish or even earlier, and probably end up fixing any compatibility issues as well as re-patch actual CVEs since then without all the countless questionable commits. I haven't heard of a single Linux distro in the last two years that can be easily rolled back to a safe state right now without manual patching and recompiling everything to a much older liblzma.
|
|
#
?
Mar 31, 2024 07:38
|
|
|
Rescue Toaster posted:I haven't heard of a single Linux distro in the last two years that can be easily rolled back to a safe state right now without manual patching and recompiling everything to a much older liblzma. RHEL's weird (to me) habit of settling on a particular release of a package and patching it for eternity means RHEL 8 apparently used xz-5.2.4 so it's possible they dodged a bullet. xz-libs-5.2.4-3.el8.x86_64.rpm Anyone check a live system?
|
|
#
?
Mar 31, 2024 10:44
|
|
|
muskrat posted:The fact that the committer(s) used multiple accounts to get this circulated, and are experienced enough to become primary maintainers of a ubiquitous library is also terrifying. It's likely they have commit access to projects other than xz. It sounded like the original maintainer of xz lost interest because of self admitted mental health issues and also because being an open source maintainer earns poo poo for pay (read: $0). Someone offered to step up and why not? There are hundreds of open source projects like this maintained by 1 person who probably get peanuts for their work.
|
|
#
?
Mar 31, 2024 12:55
|
|
|
Rust Martialis posted:RHEL's weird (to me) habit of settling on a particular release of a package and patching it for eternity means RHEL 8 apparently used xz-5.2.4 so it's possible they dodged a bullet. Well that definitely should be safe, assuming they didn't accidentally backport anything nasty while trying to fix actual security issues. Boris Galerkin posted:It sounded like the original maintainer of xz lost interest because of self admitted mental health issues and also because being an open source maintainer earns poo poo for pay (read: $0). Someone offered to step up and why not? Yeah looking back at some of the clues that this person wasn't legit seem obvious, but it was basically only dumb luck and mistakes on their part that this didn't totally explode in our faces. Hopefully OSS at large does some reflection on this and what to do going forward. Rescue Toaster fucked around with this message at 14:31 on Mar 31, 2024 |
|
#
?
Mar 31, 2024 13:58
|
|
|
Please upload a photograph of your social security card or passport to create a GitHub account.
|
|
#
?
Mar 31, 2024 15:38
|
|
|
Not to sound  but I can't imagine that agencies like the NSA hasn't already planted developers into these open source communities. I mean, they hold and attend conferences and poo poo. It is logical to assume they have people on payroll contributing to things like openssh.
|
|
#
?
Mar 31, 2024 16:19
|
|
|
Boris Galerkin posted:Not to sound Turns out the reason the whole open source developed and maintained by one guy thing works is that half the guys are in fact intelligence agencies. Which is even more scary.
|
|
#
?
Mar 31, 2024 16:35
|
|
|
Boris Galerkin posted:Not to sound If you're talking about the upstream projects, OpenSSH and OpenBSD are two projects where I would least expect this, and if the NSA et al are in fact contributing code, they're not contributing backdoors. These are two of the most secure projects on the planet; patches like the ones in question would never have been accepted. A more realistic scenario would be one of these government agencies hiring a core developer from e.g. OpenBSD / OpenSSH full-time, or someone with that level of expertise. They would be an unbelievably strong asset in terms of finding and exploiting vulnerabilities in other software. They wouldn't even need to add backdoors. Scary to think of the outcome were these world-class whitehats to have a change of heart.
|
|
#
?
Mar 31, 2024 19:09
|
|
|
Rescue Toaster posted:Hopefully OSS at large does some reflection on this and what to do going forward. More concretely, it's not "OSS at large" but rather all of the companies doing an extraction on it. It's your employer (the general you, I don't know you specifically--it's definitely mine) who's either using Ubuntu Server off a FTP download or thinks paying for a RHEL contract absolves them of further duty--and while they contribute in some lanes as well this absolutely includes Red Hat and even moreso Canonical. The XKCD comic is right; distros aren't paying it forward to the random projects they package, and then oops!-all-vulns happens and everyone just looks shocked. Being freely downloadable doesn't absolve one of pay-it-back and pay-it-forward moral obligations, but that's a foreign concept today I guess. I have a history of burning social capital to at least move the needle a tiny bit and put money in the hands of people whose stuff we rely on, but now that I'm at a megacorp I can't even do that. It sucks. I talked to Tidelift early on in their existence when they were hiring founding engineers and they seemed to at least scope the problem, but they completely failed to do anything about it as they instead tried to capture a sub-RHEL market of OSS by contributing directly instead of enabling maintainers of critical infrastructure. tracecomplete fucked around with this message at 20:01 on Mar 31, 2024 |
|
#
?
Mar 31, 2024 19:49
|
|
|
Rescue Toaster posted:Again, this is focused on the sshd payload specifically. The same account that placed that has made hundreds of commits to xz for years, and there's clear evidence they were a bad actor right away rather than a stolen account. It is insane to trust the 5.4 version of xz that people are "rolling back to". Half of the drat Linux world links against liblzma in xz 5.4 or later, including package managers, systemd, binutils... They need to roll back to at an absolute minimum 5.3ish or even earlier, and probably end up fixing any compatibility issues as well as re-patch actual CVEs since then without all the countless questionable commits. Yes, and hopefully this highlights the issue of nobody really scrutinizing these commits and patches. However, this wasn't broadly released outside dev and testing releases so unless you are running that in prod, which is a hilariously bad idea, this one got close but didn't make it to production releases.
|
|
#
?
Mar 31, 2024 21:43
|
|
|
CommieGIR posted:Yes, and hopefully this highlights the issue of nobody really scrutinizing these commits and patches. The problem is this was only the first vulnerability found from a threat actor that has been operating for years. At this point it's necessary to assume that recentish distros are running backdoors from him.
|
|
#
?
Apr 1, 2024 04:10
|
|
|
Rust Martialis posted:RHEL's weird (to me) habit of settling on a particular release of a package and patching it for eternity means RHEL 8 apparently used xz-5.2.4 so it's possible they dodged a bullet. My RHEL8 machines have 5.2.4, which is reassuring. Looking forward to whatever else is discovered as a result.
|
|
#
?
Apr 1, 2024 04:18
|
|
|
So I checked into work where I can query uCMDB about the versions of xz-libs on the myriad versions of Linux running on the 2000+ servers we support: CentOS up to 7.9.2009 - versions up to and including 5.2.2-2.el7_9 Oracle Linux up to version 8.9 - up to 5.2.4-4.el8 RHEL 5 and 6 (don't laugh, cry with me) - 4.999.9-0.5 RHEL 7 - 5.2.2-2.el7_9 latest RHEL 8 - 5.2.4-4.el8_6 latest (even on 8.9) RHEL 9 - 9.3 has xz-libs 5.2.5-8.el9_0 sooooooooooo I sort of believe RH
|
|
#
?
Apr 1, 2024 09:27
|
|
|
Rust Martialis posted:So I checked into work where I can query uCMDB about the versions of xz-libs on the myriad versions of Linux running on the 2000+ servers we support: As someone who has about 250 RHEL8/9 servers in his environment, and whose lead Linux admins are both OoO for a week, your post has given me an incredible amount of relief.
|
|
#
?
Apr 1, 2024 09:31
|
|
|
I must ruin any calming effect in pointing out that if an app running on your servers required xz-libs to be upgraded, it's *possible* you're running a more recent version and you should check. My analysis only really applies to baseline OS. For example I needed features in Python 3.9 or something but RHEL is simply not at that release, I had to manually install 3.9, so I've deviated. If you have developers, etc who live to stay bleeding edge...
|
|
#
?
Apr 1, 2024 10:16
|
|
|
|
| # ? Apr 28, 2024 20:41 |
|
|
Saukkis posted:The problem is this was only the first vulnerability found from a threat actor that has been operating for years. At this point it's necessary to assume that recentish distros are running backdoors from him. True, but at least we can find out and review code. This has been a problem before, its the blessing and the curse of Open Source.
|
|
#
?
Apr 1, 2024 14:33
|
|