Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug

Martytoof posted:

I’m still trying to understand how I am somehow at the centre of our incident response despite not being remotely involved in secops or incident response in like three years.

I mean I’m happy to pitch in and I’ve got extensive historical info on this place, but I guess a lot more handoff needs to happen.

Because most places don't understand it at all. So guess who gets voluntold.

I'm absolutely burnt out doing IRs and Incidents for log4j, its going on two weeks now and I'm just done with it.

CommieGIR fucked around with this message at 03:35 on Dec 20, 2021

Adbot
ADBOT LOVES YOU

Mustache Ride
Sep 11, 2001



I had an IR end of last week that absolutely didn't involve log4j one single bit. It was a refreshing change.

Of course they brought it to us as a log4j and it took me hours of scratching my head going "how the hell did you guys get affected by log4j when all you have external is a node app" before I realized.

some kinda jackal
Feb 25, 2003

 
 

CommieGIR posted:

Because most places don't understand it at all. So guess who gets voluntold.

I'm absolutely burnt out doing IRs and Incidents for log4j, its going on two weeks now and I'm just done with it.

I'm the sucker who didn't even get voluntold -- I put my hand up to help identify stuff based on my historical knowledge and somehow ended up running point on 99% of things lol

I'm making a list of notes that shouldn't need to be presented to a major enterprise on how this process needs to be fixed. To be clear I'm expecting none of these suggestions to actually be taken seriously or acted on once this emergency is over.

BaseballPCHiker
Jan 16, 2006

Well I ended up working some this weekend. Log4J is the gift that keeps on giving.

I really hope that we get some more software/people out of this at least. The powers that be are slowly realizing they have some major gaps and flaws in the "security program" and Log4J just reiterated that.

I spent a good chunk of time trying to ID desktops with the vulnerability, contained them, and notifying staff to get hosts updated.

Im ready for new and exciting vulns in 2022.

Baconroll
Feb 6, 2009

BaseballPCHiker posted:

Im ready for new and exciting vulns in 2022.

Theres still plenty of time left in 2021 for another 1 or 2 versions of log4j to be released due to new and exciting CVEs :suicide:

some kinda jackal
Feb 25, 2003

 
 
I'm really hoping that the specific instance of logging configurations that need to be in place for the LAST CVE means that a bunch of our vendors come back and say "nah we have 2.16 but we don't use that weird context log thing" or whatever it was.

That won't stop vulnerability scanners from flashing red all over the place because 2.16 detected but at least puts some context on patch/upgrade priorities. Maybe I'm just being naively optimistic here :D

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug
Its gonna be 6-8 months of this at least, so I'm gearing up to enjoy my week long holiday break so I can come back to this hellscape a little refreshed.

Somaen
Nov 19, 2007

by vyelkin
This seems to have been public since 2016 which is pretty cool

https://youtu.be/Y8a5nB-vy78

Wonder how many incidents were caused by someone paying attention five years ago

Potato Salad
Oct 23, 2014

nobody cares


pretty surprised no major fw vendor looked at outbound ldap in costumer metrics and thought, "you know what, this should be blocked by default"

Absurd Alhazred
Mar 27, 2010

by Athanatos

Potato Salad posted:

pretty surprised no major fw vendor looked at outbound ldap in costumer metrics and thought, "you know what, this should be blocked by default"

I know the thread title was just changed, but...

Rust Martialis
May 8, 2007

At night, Bavovnyatko quietly comes to the occupiers’ bases, depots, airfields, oil refineries and other places full of flammable items and starts playing with fire there
I thought you could engineer the callback to any port, so blocking LDAP can't be port-based

some kinda jackal
Feb 25, 2003

 
 
lmao I am about to lose it with some subcontractors doing "analysis" for days and being stuck waiting for vendor response.

Like fuckin show some initiative, go look at the loving version of the thing that's installed and literally google "<software> log4j" and see for yourself if there's a loving update my dudes.

Then I went and googled it myself and the vendor released a package two days ago and now I'm gonna go have some words.

Submarine Sandpaper
May 27, 2007


I had one vendor when I was a msp that would not release patches without a ticket, so you could not get the update package unless there was a ticket.

RFC2324
Jun 7, 2012

http 418

Submarine Sandpaper posted:

I had one vendor when I was a msp that would not release patches without a ticket, so you could not get the update package unless there was a ticket.

Shame no one ever reposts those to bypass irresponsible vendors

Rust Martialis
May 8, 2007

At night, Bavovnyatko quietly comes to the occupiers’ bases, depots, airfields, oil refineries and other places full of flammable items and starts playing with fire there
New! Improved! Script!

Scans everything under root for jar files, then looks to see if the jar file has JNDILookup.class in it, and if it does, checks the MANIFEST.MF for a version.

pre:
$ cat manifestscan.sh
#!/bin/bash

printf "%-16s|%-8s|%-64s|%-20s\n" "Hostname" "Version" "Jar File" "Class"
for n in $(find / -name *.jar )
do
        CLASS="$(unzip -l $n | grep -i 'jndilookup.class' | awk '{print  $4}' )"
        if [[ -n ${CLASS} ]]; then
                VERSION="$(unzip -q -c $n META-INF/MANIFEST.MF | grep "Log4jReleaseVersion:" | awk '{print $2}' | sed 's/\s//g' )"
        printf "%16s|%8s|%64s|%20s\n" $HOSTNAME $VERSION $n $CLASS
        fi

done

$ ./manifestscan.sh
Hostname        |Version |Jar File                                                        |Class
        hostname|  2.17.0|             /TEST/apache-log4j-2.17.0-bin/log4j-core-2.17.0.jar|org/apache/logging/log4j/core/lookup/JndiLookup.class

Rust Martialis fucked around with this message at 13:02 on Dec 22, 2021

Fart Amplifier
Apr 12, 2003

The state of k-12 education: Insecure browsers on every device! Insert image of Oprah handing out CVEs to everyone in the audience.

https://old.reddit.com/r/k12sysadmin/comments/rlp1q1/alert_check_your_google_console_for_updating/

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Rust Martialis posted:

I thought you could engineer the callback to any port, so blocking LDAP can't be port-based

In log4j's case you are correct. It doesn't even really use LDAP (or need to involve LDAP at all--that's just the most common version that's being passed around).

But in the general case of "why are my customer metrics pinging LDAP like that?", there may be good cause to just blanket block that poo poo anyhow.

Cup Runneth Over
Aug 8, 2009

She said life's
Too short to worry
Life's too long to wait
It's too short
Not to love everybody
Life's too long to hate


Here's a little levity for you as the log4j nightmare continues

https://twitter.com/hilare_belloc/status/1474126244199424002

(also maybe you should check your smart card readers)

RFC2324
Jun 7, 2012

http 418

Cup Runneth Over posted:

Here's a little levity for you as the log4j nightmare continues

https://twitter.com/hilare_belloc/status/1474126244199424002

(also maybe you should check your smart card readers)

Follow up on this and it turned out one of their shithead devs had inserted an extra thing that pulled from the card reader using log4j and never documented it

Double check your shithead devs

spankmeister
Jun 15, 2008






Cup Runneth Over posted:

Here's a little levity for you as the log4j nightmare continues

https://twitter.com/hilare_belloc/status/1474126244199424002

(also maybe you should check your smart card readers)

You could just link to the post, in the thread, on the something awful forums.

Takes No Damage
Nov 20, 2004

The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.


Grimey Drawer

spankmeister posted:

You could just link to the post, in the thread, on the something awful forums.

Is this the forums equivalent of getting an email attachment of a phone picture of a monitor screen in a Word document?

astral
Apr 26, 2004

Takes No Damage posted:

Is this the forums equivalent of getting an email attachment of a phone picture of a monitor screen in a Word document?

I'd say "try tracking that" but it's twitter; they are tracking that.

Absurd Alhazred
Mar 27, 2010

by Athanatos
I found it!

Sarah Problem posted:

Some gently caress head is randomly scanning a rfid at our smart card readers on the outside of our offices with a god drat jndi payload and the loving controller actually tried to call back. God drat you log4j. I only found it when scanning for outbound traffic attempts on our firewall. I found the payload when I grepped it’s logs. gently caress this poo poo

Powered Descent
Jul 13, 2008

We haven't had that spirit here since 1969.

Takes No Damage
Nov 20, 2004

The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.


Grimey Drawer
Posted that in my work's Teams chat, it got 2 👍 and a 🤣

Cup Runneth Over
Aug 8, 2009

She said life's
Too short to worry
Life's too long to wait
It's too short
Not to love everybody
Life's too long to hate


spankmeister posted:

You could just link to the post, in the thread, on the something awful forums.

I hadn't seen it! Information ecosystems, man


Thanks for getting OP the credit!

spankmeister
Jun 15, 2008






Cup Runneth Over posted:

I hadn't seen it! Information ecosystems, man

Thanks for getting OP the credit!

I just thought it was funny to see a yospos secfuckthread screenshot show up here.

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

Rust Martialis posted:

New! Improved! Script!

Scans everything under root for jar files, then looks to see if the jar file has JNDILookup.class in it, and if it does, checks the MANIFEST.MF for a version.


Now go down the next level and make the script scan for any .jars inside the .jars, unpack them and look for JNDI. Oh, and also include .wars in your scan.

I have to say this has been more more than the usual vulnerability. As a sys op group we usually try to figure how much to panic, then we twiddle our thumbs while waiting a patch from Red Hat, Ubuntu or VMware. Screw trying to binary patch or compile from sources. But this time the interesting part has been trying to locate where we have it and then we can just zip-delete the problem or drop in patched jars from the upstream.

First round was lsof-grepping for any log4j jars. Then we extended that to locate and find. Then we find out they can be inside .wars and need to start recursive searching. Biggest annoyance has been containers since we can't as easily fix them.

Rust Martialis
May 8, 2007

At night, Bavovnyatko quietly comes to the occupiers’ bases, depots, airfields, oil refineries and other places full of flammable items and starts playing with fire there

Saukkis posted:

Now go down the next level and make the script scan for any .jars inside the .jars, unpack them and look for JNDI. Oh, and also include .wars in your scan.

I have to say this has been more more than the usual vulnerability. As a sys op group we usually try to figure how much to panic, then we twiddle our thumbs while waiting a patch from Red Hat, Ubuntu or VMware. Screw trying to binary patch or compile from sources. But this time the interesting part has been trying to locate where we have it and then we can just zip-delete the problem or drop in patched jars from the upstream.

First round was lsof-grepping for any log4j jars. Then we extended that to locate and find. Then we find out they can be inside .wars and need to start recursive searching. Biggest annoyance has been containers since we can't as easily fix them.

Could I invoke auditd somehow on Linux to scream any time a fopen( ) or open( ) was done on any file with jndi in the name, I wonder

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

The internal structure of jar files doesn’t surface to the kernel, so I’m not sure that would help.

SSJ_naruto_2003
Oct 12, 2012



You guys might like this story.

I'm not very knowledgeable in network stuff or infosec but am very comfortable with computers.

I worked briefly as a medical receptionist at a very large company that deals with addiction. One of the responsibilities of the medical receptionist here was to photograph each new patient (with a digital camera from 2002) and upload their photo to the system the pharmacy uses, so that pharmacists can visually verify who they're handing meds to.

The policy at the company was that usb devices(that had memory) were not allowed to be plugged in, they would not function if you did, and your pc would shut off and you not be allowed on until IT talked to you.

The only part that I found true is that usb devices wouldn't function. No one from it ever bothered to follow up or lock my account, I guess that system was disabled.


The infosec gently caress up, is that the USB camera was also disabled by default. So clearly they just need to find a better system, or enable that one device for people in the medical receptionist group, or something. I provided screenshots of the error message, the MAC address, my network user id, the ip and what the device was to IT both in an email and over the phone.


Do you want to know how they got it working, and apparently the way its been done for at least a decade?

They just manually gave every medical receptionist admin power to the pc they normally logged into, and disabled all peripheral controls, so they could plug ANY USB device in with no problems. This was not a separate pc or anything, it had full access to medical records company wide (thousands of patients at any time)

The lowest paid, highest turnover , worst scheduled job in the whole company was given pretty much full network privileges. At a company where HIPAA theoretically applies.

I tried to explain both to the director of nursing (my boss), as well as the facility manager how much of a bad idea this was. I don't think either one of them understood.

Rojo_Sombrero
May 8, 2006
I ebayed my EQ account and all I got was an SA account

SSJ_naruto_2003 posted:

I tried to explain both to the director of nursing (my boss), as well as the facility manager how much of a bad idea this was. I don't think either one of them understood.

Sometimes the weakest link is the human element in InfoSec. I'm no "hacker" or any kind of guru in this field. But sometimes it is just common sense with prevention.

stoopidmunkey
May 21, 2005

yep

Rojo_Sombrero posted:

Sometimes the weakest link is the human element in InfoSec. I'm no "hacker" or any kind of guru in this field. But sometimes it is just common sense with prevention.

Humans are always the weakest link in cyber security. If they weren’t, phishing wouldn’t work.

BonHair
Apr 28, 2007

The best cyber attack weapon is probably a cleaning trolley. That'll get you past most physical perimeters, and from there you can probably install a small wireless keylogger in selected computers or docking stations. Congratulations, you now have full access to anything not protected by two factor. Unless you use being on the internal network as a second factor, in which case you also have that.

Rust Martialis
May 8, 2007

At night, Bavovnyatko quietly comes to the occupiers’ bases, depots, airfields, oil refineries and other places full of flammable items and starts playing with fire there

BonHair posted:

The best cyber attack weapon is probably a cleaning trolley. That'll get you past most physical perimeters, and from there you can probably install a small wireless keylogger in selected computers or docking stations. Congratulations, you now have full access to anything not protected by two factor. Unless you use being on the internal network as a second factor, in which case you also have that.

Plus with certain attacks like stealing session cookies you can break some 2FA anyhow.

KillHour
Oct 28, 2007


In case anyone hasn't seen it

https://youtu.be/qM3imMiERdU

Absurd Alhazred
Mar 27, 2010

by Athanatos

KillHour posted:

In case anyone hasn't seen it

https://youtu.be/qM3imMiERdU

Holy poo poo that intro! :tviv:

Cup Runneth Over
Aug 8, 2009

She said life's
Too short to worry
Life's too long to wait
It's too short
Not to love everybody
Life's too long to hate


https://news.ycombinator.com/item?id=29705957

LastPass might be getting hit by a credential stuffing incident with master passwords, reportedly

lol, lmao

Thanks Ants
May 21, 2004

#essereFerrari


KillHour posted:

In case anyone hasn't seen it

https://youtu.be/qM3imMiERdU

Haha the badge scanning last time I attended a Plasa show was more thorough than that

Adbot
ADBOT LOVES YOU

Ynglaur
Oct 9, 2013

The Malta Conference, anyone?

Cup Runneth Over posted:

https://news.ycombinator.com/item?id=29705957

LastPass might be getting hit by a credential stuffing incident with master passwords, reportedly

lol, lmao

I'm halfway through the thread and it's just nerds arguing with someone to update a blog post from 2017 because apparently if to write a piece of code ever you have an obligation to update your blog posts about it in perpetuity.

Also the OP got their master password compromised and didn't have their original MFA token. I mean, maybe a credential stuffing attack is there, but I don't think that's the only potential cause of the problem and it's a single report on a forum.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply