|
Well I may or may not have hosed up and bricked a laptop, I'm still waiting to see if it comes online or if the user gets back to me, as I ran a system restore without establishing a way to talk to the user outside of the laptop and it's been a while.
|
|
#
?
Apr 15, 2024 23:25
|
|
|
|
| # ? Apr 29, 2024 20:31 |
|
|
tokin opposition posted:Well I may or may not have hosed up and bricked a laptop, I'm still waiting to see if it comes online or if the user gets back to me, as I ran a system restore without establishing a way to talk to the user outside of the laptop and it's been a while. Welcome to working in IT. Collect your "remote bricking 1" merit badge. Ask me about remote bricking an AD controller once. I was able to restore it from the hypervisor, but it was still like 30 minutes of dull raw panic.
|
|
#
?
Apr 15, 2024 23:32
|
|
DeathSandwich posted:Welcome to working in IT. Collect your "remote bricking 1" merit badge. Gonna need to give that badge back, laptop came back after 2+ hours, within five minutes of me calling the user. I had to ask their boss for a number to get in touch. The worst part is that the issue she contacted us about isn't fixed, but at this point I've taken like five hours of her day and I'm amazed they were a civil as they were at the end.
|
|
|
#
?
Apr 16, 2024 00:24
|
|
|
tokin opposition posted:I was assured on Friday that the switches had "no special configurations set," since how else could we have moved around patch cables? At a previous job, I was once told that we could not possibly move another group over to our (networking department's) gear because their contractor had set up a Very Special Configuration and it would never work. Upon inspection, their Very Special Configuration was literally no configuration at all.
|
|
#
?
Apr 16, 2024 00:36
|
|
|
DeathSandwich posted:Welcome to working in IT. Collect your "remote bricking 1" merit badge. I managed to hose our only physical DC doing windows updates. We still were on ADFS 2012 at the time, and it only authenticated to the PDC for any external logins. Most of our workforce is remote. Oops. Seized the roles to get things back in order, cleaned out all the old metadata then drove up to the datacenter and had it flattened and ready to be repromoted by mid afternoon.
|
|
#
?
Apr 16, 2024 01:25
|
|
|
https://www.itprotoday.com/attacks-and-breaches/cisco-duos-multifactor-authentication-service-breached
|
|
#
?
Apr 16, 2024 03:29
|
|
|
tokin opposition posted:Gonna need to give that badge back, laptop came back after 2+ hours, within five minutes of me calling the user. I had to ask their boss for a number to get in touch. You'll run into that a lot in the IT world. Even in the rote helpdesk stuff you'll wind up getting the perfect storm of hardware / software issue and PEBKAC issue that means you're about to have like a password reset call that takes 90 minutes as the person immediately forgets, then can't figure their phone out, then "oh by the way" - s you right as you were about to cut him loose, then calls you back anyway because he forgot a second time.
|
|
#
?
Apr 16, 2024 06:36
|
|
|
One time I enabled Windows firewall on an RRAS server. That I was remoted in through. It was fun.
|
|
#
?
Apr 17, 2024 06:35
|
|
|
Just had my first phone screen since January, not excited about the job but excited about money. It would be the one in-house IT person meaning a mix of hands-on repair and MSP wrangling, and hopefully being able to convince management to let me update their infrastructure from what I assume is very old and backwards but the owner apparently used to be in tech in the early '00s which has me worried that he'll be all "back in my day we did it like ___" when his day was two decades ago. Also it's on-site which I hate, but not as much as I hate not having stable income.
|
|
#
?
Apr 17, 2024 23:24
|
|
|
22 Eargesplitten posted:Just had my first phone screen since January, not excited about the job but excited about money. It would be the one in-house IT person meaning a mix of hands-on repair and MSP wrangling, and hopefully being able to convince management to let me update their infrastructure from what I assume is very old and backwards but the owner apparently used to be in tech in the early '00s which has me worried that he'll be all "back in my day we did it like ___" when his day was two decades ago. I'd definitely check the vibes if the call you in for an in-person interview. Being the sole IT person could be bad if they expect you on call 100%. But if your boss knows the IT field a bit, he may be a bit more understanding and realistic. Of course, you could just take the job if offered and then bail as soon as you find something better.
|
|
#
?
Apr 17, 2024 23:55
|
|
|
As long as it’s 9-5 I don’t mind being the only in-house. Gives an opportunity to make decisions. I just had 5 interviews put on my schedule tomorrow for our open help desk contractor position and I am not looking forward to an entire day of that.
|
|
#
?
Apr 18, 2024 00:14
|
|
|
Silly Newbie posted:One time I enabled Windows firewall on an RRAS server. That I was remoted in through. It was fun. I did that by screwing up an iptables rules update. While remote. On a Saturday. Sometimes in interviews I talk about my experience in terms of mistakes. "Unix? Let's see, I've done an rm -rf * at the root level of a machine, etc., etc." That should show that you have hands on experience in the real world, and you're not just parroting exam material.
|
|
#
?
Apr 18, 2024 00:18
|
|
|
I had the best day at work ever today! Did not do any IT, did not talk to my boss or coworker, and spent the day doing a DEI retreat with other people in the org I actually enjoy talking to or being near
|
|
|
#
?
Apr 18, 2024 01:17
|
|
|
mllaneza posted:I did that by screwing up an iptables rules update. While remote. On a Saturday. A lot of times people are hesitant to answer “what was your worst mistake that impacted production?” so I have to tell them the time I took down a courthouse in the middle of the day to get a real answer. I like it as a question because good answers get down to the real question which is “and what did you learn from that awful experience?” and you can talk about stupid technology that you have to build controls around to protect it from falling over too easily. If they have no answer or a fake one, then I know they’re either full of poo poo or they don’t actually do anything. Everybody has at least one mistake, though maybe the impact may not have been severe if you only worked at places with excellent control and deployment practices.
|
|
#
?
Apr 18, 2024 01:17
|
|
|
I'm sure I've blown stuff up before, but I can't remember anything well enough to have a great "I hosed up" story. Which honestly kind of bums me out, because I don't think I'm infallible, and I don't want others to think I'm infallible. Or even worse, for people to think that I think I'm infallible.
|
|
#
?
Apr 18, 2024 01:36
|
|
|
Those stories are very good to use in interviews, always have one ready and how you successfully rolled back and recovered
|
|
#
?
Apr 18, 2024 01:39
|
|
|
I think I’ve caused… at least four or five production outages? Nothing quite compares to the particularly ill thought out DNS changes to support private link/endpoint resolution I made last Easter Sunday though. Who really needs to resolve management.azure.com anyways?
|
|
#
?
Apr 18, 2024 02:20
|
|
Cat Army |
I haven't had any major gently caress ups that affected more than one person, but that's also because I've had to fight tooth and nail to actually start touching (and actually have someone actively manage) the servers. The good news is that I now can access the APs to diagnose our wifi that still isn't working
|
|
|
#
?
Apr 18, 2024 02:23
|
|
|
tokin opposition posted:I haven't had any major gently caress ups that affected more than one person, but that's also because I've had to fight tooth and nail to actually start touching (and actually have someone actively manage) the servers. Accidentallied did something that got our entire company domain flagged as malicious by google. That was a fun week of apologizing to my teammates as we desperately worked to get that fixed. As someone once told me in this thread, if you're not in position to ever make huge mistakes you're not doing important IT work.
|
|
#
?
Apr 18, 2024 02:34
|
|
|
The Iron Rose posted:ill thought out DNS changes to support private link/endpoint resolution I made last Easter Sunday though. Who really needs to resolve management.azure.com anyways? hybrid DNS and azure private endpoints are a nightmare to begin with, I don't believe anyone can get it right on the first pass
|
|
#
?
Apr 18, 2024 02:39
|
|
|
Vargatron posted:I'd definitely check the vibes if the call you in for an in-person interview. Being the sole IT person could be bad if they expect you on call 100%. But if your boss knows the IT field a bit, he may be a bit more understanding and realistic. That's a good point, I didn't think to ask about on-call expectations. I don't think I can reject this job if I get an offer without losing unemployment, so if I get the offer I think I'll take it and keep applying for stuff that doesn't require going into the office. I'd be reporting to a finance person who also currently manages the MSP, then her boss is the one that apparently used to work in tech. There's definitely a lot of red flags, but the pay seems good and I'm not really in a position to refuse the offer if I get it. Really aside from the on-site requirement the job being good or bad seems to come down to "will management let me do my job or are they going to second-guess everything that they are paying me to know?" 22 Eargesplitten fucked around with this message at 03:38 on Apr 18, 2024 |
|
#
?
Apr 18, 2024 02:42
|
|
|
Defenestrategy posted:Accidentallied did something that got our entire company domain flagged as malicious by google. That was a fun week of apologizing to my teammates as we desperately worked to get that fixed. Lmao I was doing some questionable searching with Google operands once, leading to the entire campus of like 5k students and teachers to have to do a captcha for every Google search from our IP I think that was when I learned about load balancing outbound nat
|
|
#
?
Apr 18, 2024 04:33
|
|
|
I've broken so many things. If you're at smaller shops where poo poo isn't usually done quite right and no one has the time or expertise it can get pretty touch and go at times.
|
|
#
?
Apr 18, 2024 05:05
|
|
|
skipdogg posted:So it's like that Farmers' Insurance commercial. "We know a thing or two, because we've seen a thing or two". You just run into weird poo poo over the years, or crazy edge cases, stuff like that you pick up along your career. Good reply, I am at a point where I am pretty good at my job because I've been doing this for over a decade and for some very, very big customers. My last projects was divesture for 100k users splitting domains/tenants, 1,000+ SaaS Application migration and then just helping a client get everything setup in Azure (PIM,MFA,CA,etc. Landing Zone, etc.) but in a gov tenant. I feel like I sort of got pigeonholed too because I know quite a bit about the insides of the Entra ID Connect sync. engine, the relationship between Office 365, Entra ID, Intune, etc. but now I'm getting asked what do I know about SAML or OIDC? Too tell you truth, I don't that much off the top of my head but it's a web based AuthN protocol and I feel at this point in my career I'll just learn fast - because that's just what I've done for the past decade? Hell, I didn't know anything how Quests tool for the migration project I was on earlier but I was able to communicate with their engineer to coordinate everything. I just read their docs and made my own small lab.  I feel like the only thing that's left more me aside from applying  is to get more certs but is there such as thing as a all encompassing identity certification? is there something that'll give me a seal of approval with Kerberos, SAML, OIDC and SCIM? Or do I start getting things like the CISSP, CISM, CISA, SANS, Security+, etc.? Should I just say gently caress it and make my own SAML App or IdP and learn the whole spec?
Gucci Loafers fucked around with this message at 11:45 on Apr 18, 2024 |
|
#
?
Apr 18, 2024 06:19
|
|
|
I was the domino that destroyed six figures worth of hardware once, a poorly built server room was primed to cause a catastrophe and me changing a password caused it all to tip over. Sprinklers were involved. No one ever blamed me formally but it still haunts me.
|
|
#
?
Apr 18, 2024 11:51
|
|
|
I dropped a production database during an outage during my first ever mainframe job, thankfully my boss was a kind soul and I got to learn how to restore from backups. Thought my career was over right then and there
|
|
#
?
Apr 18, 2024 12:04
|
|
|
CommieGIR posted:I dropped a production database during an outage during my first ever mainframe job, thankfully my boss was a kind soul and I got to learn how to restore from backups. Only crappy tyrants would fire someone for making an honest mistake that can be chalked up to: 1) lack of training 2) lack of documentation 3) lack of process control 4) piece of poo poo computer have bug If you go outside of a process and cause an outage out of negligence when you knew better, that's when the serious discussions are had.
|
|
#
?
Apr 18, 2024 15:07
|
|
|
I killed the entire network for an indoor farm that relied on automation to function during a grow cycle.
|
|
#
?
Apr 18, 2024 19:31
|
|
|
Welp I'm not paid off yet but I'm being marginalized and got told explicitly there's no path to grow my skills here, so I've probably got ~1-3 months here On the upside I'll be WFH full time so I'll have a lot more time to job hunt and get a new cert. Debating between sex+ and net+ E: sec+ but that typo is too good to properly edit
|
|
|
#
?
Apr 18, 2024 19:36
|
|
|
IT nerds are definitely lacking in sex+
|
|
#
?
Apr 18, 2024 19:37
|
|
|
sec+
|
|
#
?
Apr 18, 2024 19:37
|
|
|
same tokin. had my 2 year review yesterday and it was grim tbh
|
|
#
?
Apr 18, 2024 19:40
|
|
|
FISHMANPET posted:I'm sure I've blown stuff up before, but I can't remember anything well enough to have a great "I hosed up" story. Which honestly kind of bums me out, because I don't think I'm infallible, and I don't want others to think I'm infallible. Or even worse, for people to think that I think I'm infallible. Same here. I've not done anything to take out a whole company or anything, but certainly made a boneheaded maneuver or two that cost me a hours or days rebuilding a user machine or something. Mainly because I don't typically touch things that could cause a production outage. I do have access to our network IDFs and server room, and have yet to unplug or kick anything critical, so there's that. tokin opposition posted:Welp I'm not paid off yet but I'm being marginalized and got told explicitly there's no path to grow my skills here, so I've probably got ~1-3 months here Well, from what you've said, you have no growth path explicitly because your boss isn't interested in growth, so... Still sucks. Chalk it up to you weren't enough of a punching bag for them, and take that as a positive.
|
|
#
?
Apr 18, 2024 20:15
|
|
|
Cyks posted:As long as it’s 9-5 I don’t mind being the only in-house. Gives an opportunity to make decisions. It’s not fair to the interviewees but I hit a wall 15 minutes into number 4. This is exhausting and we never should have scheduled so many.
|
|
#
?
Apr 18, 2024 20:29
|
|
|
Can you reschedule the last one? Back-to-back-to-back-to-back-to-back interviews sounds unfeasible
|
|
#
?
Apr 18, 2024 21:44
|
|
|
I have broken the website of our nations biggest telco and isp on multiple occaisions (always due to different root causes though). Most times it was just seconds to a few minutes but one time it took 3 hours to flush a cache or invalidate a cdn that was misconfigured and kept getting stale data from the wrong backend. It didn’t help that it was maintained by a 3rd party who had very little knowledge of the ancient system and could not be reached directly. There’s nothing like a flock of sales / marketing people literally standing behind you asking how long it’ll take to fix at 5 min intervals because a campaign/sale just went life. Especially the time it happened when a new iphone was released. Them doing stupid poo poo I had warned them about multiple times was usually the problem to begin with.
|
|
#
?
Apr 18, 2024 21:50
|
|
|
I took down all client logins for one of the country's leading mortgage vendors for half an hour because i scaled a node pool from six to seven. Luckily I also found out we had no monitoring on it, and fixed that at the same time.
|
|
#
?
Apr 18, 2024 22:12
|
|
|
I accepted a new job on Tuesday, and today a BETTER job emails me asking for a second interview, 2 hours in person. Start date for the position I accepted is a month out, so... Guess I get to see if I can land a better job and bail on the first one if it works out?
|
|
#
?
Apr 18, 2024 23:26
|
|
|
chin up everything sucks posted:I accepted a new job on Tuesday, and today a BETTER job emails me asking for a second interview, 2 hours in person. Start date for the position I accepted is a month out, so... Guess I get to see if I can land a better job and bail on the first one if it works out? Yes. With absolutely no sympathy or fucks given to the first job, because they'd probably fire you in a heartbeat if it were convenient.
|
|
#
?
Apr 18, 2024 23:30
|
|
|
|
| # ? Apr 29, 2024 20:31 |
|
|
If the first job ran into a problem that required they decide not to hire you the week before you were due to start, they'd have no problems telling you that without feeling bad about it.
|
|
#
?
Apr 18, 2024 23:33
|
|
