Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Woof Blitzer
Dec 29, 2012

[-]

ChubbyThePhat posted:

Surely this is because you don't wanna be physically in the office to deal with that poo poo and not because it has taken three days to fix right....


More than three days now, might be because it’s managed by AT&T.

Adbot
ADBOT LOVES YOU

Woof Blitzer
Dec 29, 2012

[-]
Btw speaking of AT&T, does anyone use one of their dedicated business wireless broadband plans?

klosterdev
Oct 10, 2006

Na na na na na na na na Batman!
WELCOME TO AT&T

WOULD YOU LIKE TO REPORT A TROUBLE

THANKS FOR REPORTING THE TROUBLE *click*

BaseballPCHiker
Jan 16, 2006

Nuclearmonkee posted:

You'd have to do something insanely bad to be able to gently caress a network up so badly that spanning tree is causing an office to be dead for three days. I can't even imagine what that would be. Worst case I can think of is if you increase network diameter until it just fails and is unable to work, which can be remedied by removing the switch you added and being less dumb, or even just loving with timers a little bit until you can get someone in there who knows a thing to unfuck it over time.


:psyduck: you can influence root election with a single command.

Correct. But when its not your equipment and you cant access it, yet it touches your network what are you supposed to do? Like I said its a very strange setup, that will thankfully be going away as that particular customer goes out of business in the next year or so.

abigserve
Sep 13, 2009

this is a better avatar than what I had before
You should use root guard not bpdufilter in that scenario fyi

Methanar
Sep 26, 2013

by the sex ghost

BaseballPCHiker posted:

Correct. But when its not your equipment and you cant access it, yet it touches your network what are you supposed to do? Like I said its a very strange setup, that will thankfully be going away as that particular customer goes out of business in the next year or so.

Sharing an l2 domain with an untrusted 3rd party is an interesting choice.

tortilla_chip
Jun 13, 2007

k-partite
Every MetroE deployment ever

Thanks Ants
May 21, 2004

#essereFerrari


BaseballPCHiker posted:

Correct. But when its not your equipment and you cant access it, yet it touches your network what are you supposed to do? Like I said its a very strange setup, that will thankfully be going away as that particular customer goes out of business in the next year or so.

As Methanar wrote, can it connect on an L3 interface?

Nuclearmonkee
Jun 10, 2009


3rd layer best layer. Even if you have some goofy thing that requires L2 between sites I would not bridge entire networks through a metro-e connection. Can do layer 2 tunneling where required.

abigserve posted:

You should use root guard not bpdufilter in that scenario fyi

Nuclearmonkee fucked around with this message at 23:33 on Feb 20, 2020

Thanks Ants
May 21, 2004

#essereFerrari


If you think you need to split a broadcast domain between sites then try literally any other way of achieving what you want

Darchangel
Feb 12, 2009

Tell him about the blower!


Defenestrategy posted:

Just be in an office space where your leadership team is mostly older technology inept people who insist on printing everything to a common room printer rather than have their own printer for ~reasons~ and then don't promptly get what ever they printed out/forget about it entirely.

Let me tell you about the long-time administrative assistant we had who printed out emails and kept them in (several!) binders to refer to.
We very much enjoyed throwing those away when she finally retired.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Thanks Ants posted:

If you think you need to split a broadcast domain between sites then try literally any other way of achieving what you want

Three jobs ago my company got bought by a company that had a datacenter, a main office and a satellite office all using a combined 192.168.1.0/24 range.

It was insane. They had a single DHCP server in the data center that would gleefully hand out up addresses to anything that asked, their production servers showed up and were browsable in windows network, and they were so IP constrained (more than 254 devices on a network spanning three different sites? Who knew?) that their IP lease time was fifteen minutes. Hibernate your laptop and when it wakes up you probably lost your IP address and connectivity to the network. Printers were on leased IP addresses as well and not reserved either with predictable results.

When we asked why they would use a single class C space for three different sites including production, their IT guy said “it’s way easier this way”.

Nuclearmonkee
Jun 10, 2009


Thanks Ants posted:

If you think you need to split a broadcast domain between sites then try literally any other way of achieving what you want

I have ancient industrial devices that have their IP address set via rotary switch and do not understand what a gateway is :suicide:

Fortunately, there are only two instances on our network in which they have to talk to something far away where I have configured a shamecube VLAN for them to sit in and bridged them back to the rest of their IO network via L2TP.

klosterdev
Oct 10, 2006

Na na na na na na na na Batman!

Agrikk posted:

when we asked why they would use a single class C space for three different sites including production, their IT guy said “the gently caress are subnets"

Thanks Ants
May 21, 2004

#essereFerrari


Nuclearmonkee posted:

I have ancient industrial devices that have their IP address set via rotary switch and do not understand what a gateway is :suicide:

Fortunately, there are only two instances on our network in which they have to talk to something far away where I have configured a shamecube VLAN for them to sit in and bridged them back to the rest of their IO network via L2TP.

Shove something that can do proxy ARP between it and the rest of the network

BaseballPCHiker
Jan 16, 2006

Methanar posted:

Sharing an l2 domain with an untrusted 3rd party is an interesting choice.

Yeah not my choice. Hopefully soon we're rid of this customer and I'll have one less thing to worry about.

The amount of layer2 on this network is staggering, no one in their right mind would design it like it is now if they were starting fresh but its what I get to deal with day to day.

Antigravitas
Dec 8, 2019

Die Rettung fuer die Landwirte:

Methanar posted:

Sharing an l2 domain with an untrusted 3rd party is an interesting choice.

We have, for contrived reasons that are complex even for university IT, a /26 allocated to one of our computer pools. However, policy apparently was to allocate VLANs per building, so we are actually within an l2 we share with the other networks carved from the /24 allocated to the building.

It's fascinating to watch. One neighbor institute has a server 2003 machine running there that is broadcasting garbage and there's nothing we can do about it because we don't know who that is or who is responsible. Listening in on their traffic is equal parts entertaining in a "ha ha look at those freaks" kind of way, but also horrifying because we are network room mates…

GnarlyCharlie4u
Sep 23, 2007

I have an unhealthy obsession with motorcycles.

Proof
Whew I picked a real good day to be sick.
We got a call from DHS saying that they have poof that our systems have been compromised, and now it's all-hands-on-deck.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


GnarlyCharlie4u posted:

Whew I picked a real good day to be sick.
We got a call from DHS saying that they have poof that our systems have been compromised, and now it's all-hands-on-deck.

Do you work for Slickwraps?

https://www.droid-life.com/2020/02/21/slickwraps-appears-to-have-suffered-a-massive-data-breach/

siggy2021
Mar 8, 2010

klosterdev posted:

WELCOME TO AT&T

WOULD YOU LIKE TO REPORT A TROUBLE

THANKS FOR REPORTING THE TROUBLE *click*

You forgot the intermediary step where you wait on hold for two hours and then someone tells you you called the wrong number for your particular issue and you need to call this other number that is not actually documented anywhere.

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

siggy2021 posted:

You forgot the intermediary step where you wait on hold for two hours and then someone tells you you called the wrong number for your particular issue and you need to call this other number that is not actually documented anywhere.

:argh:

Woof Blitzer
Dec 29, 2012

[-]
Our executive PM thought that a data field in our application wasn’t updating, so we started a big managed incident call with 20+ people, only to find out that according to the developer that data doesn’t always update. Efficiency in action!

GnarlyCharlie4u
Sep 23, 2007

I have an unhealthy obsession with motorcycles.

Proof

oof. Thankfully, no.

And since the Medium post is down, here's a wayback snap:
https://web.archive.org/web/20200221151606/https://medium.com/@lynx0x00/i-hacked-slickwraps-this-is-how-8b0806358fbb

GnarlyCharlie4u
Sep 23, 2007

I have an unhealthy obsession with motorcycles.

Proof
Today is my 5 year anniversary which means I am officially vested in the pension.
...and here I am resetting passwords for users on a Sunday because I am on call and we nuked everyone's accounts on Friday.

Hughmoris
Apr 21, 2007
Let's go to the abyss!

GnarlyCharlie4u posted:

Today is my 5 year anniversary which means I am officially vested in the pension.
...and here I am resetting passwords for users on a Sunday because I am on call and we nuked everyone's accounts on Friday.

Congrats on hitting the 5 year point!

And speaking of call... listening to some of of my new team members talk about how the past 2 people hired before me quit before they were assigned their first week of call. My internal dialogue was thinking it's soon to be 3.

Judge Schnoopy
Nov 2, 2005

dont even TRY it, pal
I just built a firewall auditing tool to check our 200+ firewalls against a database of accepted and rejected rules. It collects real time data from two different firewall vendors using two other micro services I spun up in the last four months. The tool is 100% rest API driven.

I built a front end auditing interface in powershell, using out-gridviews and prompts. This ui guides admins through each firewall audit from a single command. 90% of all firewall rules match the database definitions so admins only need to audit the never-before-seen one off rules per firewall. The focus this provides allows us to flag many rules that don't belong or are way too wide open.

And I did it in 3.5 days because somebody royally messed up the audit scheduling. 5 MRs, hundreds of commits, ~3000 lines of code. I feel like a drat superhero.

Zero VGS
Aug 16, 2002
ASK ME ABOUT HOW HUMAN LIVES THAT MADE VIDEO GAME CONTROLLERS ARE WORTH MORE
Lipstick Apathy

GnarlyCharlie4u posted:

Today is my 5 year anniversary which means I am officially vested in the pension.
...and here I am resetting passwords for users on a Sunday because I am on call and we nuked everyone's accounts on Friday.

You get a pension after 5 years? I think I got a t-shirt.

CrazyLittle
Sep 11, 2001





Clapping Larry

Nuclearmonkee posted:

You'd have to do something insanely bad to be able to gently caress a network up so badly that spanning tree is causing an office to be dead for three days. I can't even imagine what that would be. Worst case I can think of is if you increase network diameter until it just fails and is unable to work, which can be remedied by removing the switch you added and being less dumb, or even just loving with timers a little bit until you can get someone in there who knows a thing to unfuck it over time.


:psyduck: you can influence root election with a single command.

Nuclearmonkee
Jun 10, 2009


Thanks Ants posted:

Shove something that can do proxy ARP between it and the rest of the network

It also uses broadcast UDP packets for data transmission (modern CIP stuff uses multicast but this system is old), so you'd have to do proxy ARPing and directed broadcasts which is doing basically the same thing in a different way.

devmd01
Mar 7, 2006

Elektronik
Supersonik
I’ve had a domain controller down since Sunday morning because of patching.

Does it count against SLA if nobody noticed? 🤔

Methanar
Sep 26, 2013

by the sex ghost

devmd01 posted:

I’ve had a domain controller down since Sunday morning because of patching.

Does it count against SLA if nobody noticed? 🤔

Literally no. Microsoft doesn't payout unless you have SCOM or something proving there was a violation.

devmd01
Mar 7, 2006

Elektronik
Supersonik
I don’t even care. It took me all of an hour to excise the old domain controller properly, shut down the old physical server and disable the switchport, provision a 2019 vm from template and get it re-joined with the same Dc name as the physical.

devmd01 fucked around with this message at 00:34 on Feb 25, 2020

Judge Schnoopy
Nov 2, 2005

dont even TRY it, pal

Judge Schnoopy posted:

And I did it in 3.5 days because somebody royally messed up the audit scheduling. 5 MRs, hundreds of commits, ~3000 lines of code.

I feel like shouting into the void to say that I used this tool to single handedly audit 400 firewalls, 13,000 rules today, each being individually tagged with an audit determination. Only took 3 hours of auditing with the rest of the day dedicated to bug fixes.

The Fool
Oct 16, 2003


Judge Schnoopy posted:

I feel like shouting into the void to say that I used this tool to single handedly audit 400 firewalls, 13,000 rules today, each being individually tagged with an audit determination. Only took 3 hours of auditing with the rest of the day dedicated to bug fixes.

lol

You did good man. It can be frustrating when you're really proud of a piece of work and no-one else seems to get it, but don't let that take away from the fact that you built something good and it is a real accomplishment.

Judge Schnoopy
Nov 2, 2005

dont even TRY it, pal

The Fool posted:

lol

You did good man. It can be frustrating when you're really proud of a piece of work and no-one else seems to get it, but don't let that take away from the fact that you built something good and it is a real accomplishment.

I mean this is just every day at my job to be honest. My supervisor gave me a 'woohoo' when I finished with 49 minutes to spare on the audit deadline, despite it being impossible for anybody else to have accomplished the same thing.

He won't sing my praises to anyone higher up, this project won't be noticed beyond the engineer team, and my efforts will be forgotten.

Woof Blitzer
Dec 29, 2012

[-]

Judge Schnoopy posted:

He won't sing my praises to anyone higher up, this project won't be noticed beyond the engineer team, and my efforts will be forgotten.

IT.txt

PBS
Sep 21, 2015

Judge Schnoopy posted:

I feel like shouting into the void to say that I used this tool to single handedly audit 400 firewalls, 13,000 rules today, each being individually tagged with an audit determination. Only took 3 hours of auditing with the rest of the day dedicated to bug fixes.

Jfc, you guys have that many firewalls? Is it all on-prem?

Shut up Meg
Jan 8, 2019

You're safe here.

PBS posted:

Jfc, you guys have that many firewalls? Is it all on-prem?

I could believe that it's all on a single server.

PBS
Sep 21, 2015

Shut up Meg posted:

I could believe that it's all on a single server.

Fair, I was thinking network appliance as opposed to iptables. My company disables iptables because managing firewalls on servers is hard. (/s)

Adbot
ADBOT LOVES YOU

Shut up Meg
Jan 8, 2019

You're safe here.
I was actually being facetious about 400 firewalls installed on a single machine : like how you encounter desktops with 4 antivirus packages installed, to be extra safe.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply