|
jaegerx posted:A goon is telling all about Tesla IT. https://forums.somethingawful.com/showthread.php?threadid=3862643 Right in my veins. That’s the good stuff.
|
#
?
Aug 26, 2018 03:09
|
|
|
|
| # ? Jun 10, 2024 20:46 |
|
|
Corsair Pool Boy posted:Yeah it's SQL DBs. They have chosen to manually power on all their servers (probably 60 or so VMs and 30 or so physical boxes) because nothing works if SQL doesn't come up first. If they've got pre-6.7 vCenter, then I could almost see this being a thing where they'd want SQL up first before trying to boot vCenter.
|
|
#
?
Aug 26, 2018 03:52
|
|
|
jaegerx posted:A goon is telling all about Tesla IT. https://forums.somethingawful.com/showthread.php?threadid=3862643 
|
|
#
?
Aug 26, 2018 05:55
|
|
|
Lord Dudeguy posted:If they've got pre-6.7 vCenter, then I could almost see this being a thing where they'd want SQL up first before trying to boot vCenter. Hyper V environment.
|
|
#
?
Aug 26, 2018 06:12
|
|
|
Corsair Pool Boy posted:Yeah it's SQL DBs. They have chosen to manually power on all their servers (probably 60 or so VMs and 30 or so physical boxes) because nothing works if SQL doesn't come up first. This should be done via script and it seems goofy to have more than a handful of VMs and do it manually. Mine finds vcenter, DNS, and dbs and boots them, waits 180 seconds and then boots everything that was powered on before shutdown. Still having ~30 physical boxes is cancerous but I assume they could also be scripted using IPMI or whatever.
|
|
#
?
Aug 27, 2018 18:51
|
|
|
Long time no post. Looks like I have a new job lined up and am free from MSP hell. Less travel, better hours, 4 weeks vacation (week between Christmas and New Years is free cause office is simply closed), and a 21k salary increase. Not too shabby if I do say so 
|
|
#
?
Aug 27, 2018 19:43
|
|
|
ChubbyThePhat posted:Long time no post. Looks like I have a new job lined up and am free from MSP hell. Less travel, better hours, 4 weeks vacation (week between Christmas and New Years is free cause office is simply closed), and a 21k salary increase. Not too shabby if I do say so 
|
|
#
?
Aug 27, 2018 19:44
|
|
|
ChubbyThePhat posted:Long time no post. Looks like I have a new job lined up and am free from MSP hell. Less travel, better hours, 4 weeks vacation (week between Christmas and New Years is free cause office is simply closed), and a 21k salary increase. Not too shabby if I do say so Congrats!
|
|
#
?
Aug 27, 2018 19:48
|
|
|
Hell yea!
|
|
#
?
Aug 27, 2018 19:56
|
|
|
I swear I saw this in this thread: what's the enterprise DNS / network monitor / logging software that has a ridiculous dark / cyberpunk / Tom Clancy bullshit GUI? Lots of lines on a map / node / graph stuff. Did I imagine this?
|
|
#
?
Aug 28, 2018 00:58
|
|
|
JehovahsWetness posted:I swear I saw this in this thread: what's the enterprise DNS / network monitor / logging software that has a ridiculous dark / cyberpunk / Tom Clancy bullshit GUI? Lots of lines on a map / node / graph stuff. Did I imagine this? Grafana? 
|
|
#
?
Aug 28, 2018 01:07
|
|
|
No, it's not something well-known. It's very obviously an lovely enterprise product that does DNS logging but the interface looks like someone re-made http://map.norsecorp.com with the intent of impressing executives who like cyberhack interfaces. I swear this exists. e: I also force everyone in my group to use the Grafana light theme because I hate the default grey one.
|
|
#
?
Aug 28, 2018 01:10
|
|
|
There's some security software that has an interface that looks exactly like something out of Ghost In The Shell but I cannot for the life of me remember the name of it right now.
|
|
#
?
Aug 28, 2018 02:23
|
|
|
JehovahsWetness posted:I also force everyone in my group to use the Grafana light theme 
|
|
#
?
Aug 28, 2018 02:32
|
|
|
You're probably looking for Plixer. They have a few tools, like Scrutinizer for netflow. All of them are designed to look as dark and sci-fi as possible.
|
|
#
?
Aug 28, 2018 02:36
|
|
|
None of that stuff is even remotely unexpected, also Tesla uses Buildroot now and not Yocto. Car infotainment code is pure dumpster fire across the board, Tesla included.
|
|
#
?
Aug 28, 2018 02:55
|
|
|
JehovahsWetness posted:I swear I saw this in this thread: what's the enterprise DNS / network monitor / logging software that has a ridiculous dark / cyberpunk / Tom Clancy bullshit GUI? Lots of lines on a map / node / graph stuff. Did I imagine this? You're thinking of Darktrace
|
|
#
?
Aug 28, 2018 03:06
|
|
Cat Army
|
The Iron Rose posted:You're thinking of Darktrace Oh lmao those screenshots are something else Holy poo poo how is that real
|
|
#
?
Aug 28, 2018 03:18
|
|
|
The Iron Rose posted:You're thinking of Darktrace This website is amazing! I'm the Data Theft from Under the Floorboards  
|
|
#
?
Aug 28, 2018 03:49
|
|
|
nullfunction posted:This website is amazing! I'm the Data Theft from Under the Floorboards It always cracked me up when our colo providers asked us if we needed the cages to extend to the ceiling and into the raised floor.
|
|
#
?
Aug 28, 2018 04:06
|
|
|
Things I've experienced about working in IT so far: 1. My boss insists that usernames are case sensitive for the website we handle accounts for for our staff (they're not) 2. We use Outlook for email and calendars but Google Drive for sharing documents 3. Zoom went down today and bc I was the first one to notice, everyone thought I broke something 4. Turns out in order for there to be documentation on how to do things, someone has to write it. Looks like that someone is going to be me. At least once we have the knowledge base done, there will be fewer tickets coming in for common issues. Right? Right?
|
|
#
?
Aug 28, 2018 04:25
|
|
|
AnonymousNarcotics posted:
hahahahahaha
|
|
#
?
Aug 28, 2018 04:48
|
|
|
AnonymousNarcotics posted:
The tickets will then be about changes that needed to be made to the kb.
|
|
#
?
Aug 28, 2018 04:50
|
|
|
AnonymousNarcotics posted:
“It’s so much to read can you just tell me what to do? I’m on a deadline and just need it done”
|
|
#
?
Aug 28, 2018 11:30
|
|
|
Can anyone recommend some resources for learning more about wireless troubleshooting? I don't actually have a wireless issue, I have a helpdesk and helpdesk management issue where the helpdesk refuses to admit that the wireless network is fine and they have a software problem. My approach so far has been "Look, I put six devices on the wifi network and they've been streaming video for an hour without a hiccup," but I really need something more concrete and RSSI/SNR isn't a full enough picture. I think the best approach would be to demonstrate the specific reason why a user station loses connectivity; I'm open to books on the subject, software tools, hardware tools, whatever. I looked at Wireshark, but I will mostly be able to use Windows machines for this and the documentation suggests that I probably won't be able to put the wireless adapter on my laptop into promiscuous mode, and I don't know if packet analysis is even the way to go. The wireless vendor control panel does include a client monitor but its results are in what I would describe as broken English and I find it very hard to interpret; for example, does "(user station) is de-authenticated because of notification of driver" actually point to a client driver issue, or is it just written poorly?
|
|
#
?
Aug 28, 2018 12:08
|
|
|
AnonymousNarcotics posted:Things I've experienced about working in IT so far:
|
|
#
?
Aug 28, 2018 12:40
|
|
|
That's really dependent on what the actual problem is. Does Windows report as disconnected? Are the machines laptops, and are they moving? Do you share space with another company, or a microwave testing factory? Is it an application connected to a database that complains about being disconnected? Have you looked at the packet status on the switch interface ports feeding the APs? Have you checked Windows event viewer for Ethernet adapter messages? Most importantly, have you looked at DNS? Seriously, always check DNS, even if it doesn't seem like it could possibly be DNS. Wireless problems are abundant and rich.
|
|
#
?
Aug 28, 2018 12:45
|
|
|
Judge Schnoopy posted:That's really dependent on what the actual problem is. Does Windows report as disconnected? Are the machines laptops, and are they moving? Do you share space with another company, or a microwave testing factory? Is it an application connected to a database that complains about being disconnected? Yeah, if you don't have access to the actual infrastructure wireless can be anything from clients are hosed to AP are hosed to it's just a little too humid today. Ideally, you want to packet cap from the BSSIDs so you can get L2/L3 and dot11 events. Barring that, you can also see if there's a "debug" level of logging you can redirect. If it's a wireless issue, the device should be logging 802.11 reason codes. (I'm assuming this is enterprise level). If there's a controller in charge of the APs, there may be additional debugging tools available. https://community.arubanetworks.com/t5/Technology-Blog/802-11-Reason-Codes-and-Status-Codes/ba-p/257893
|
|
#
?
Aug 28, 2018 12:53
|
|
|
Definitely enterprise level, and we do not share space with anyone. All devices experiencing issues are laptops -- desktops are wired -- and are mostly the same model, and they do not need to be moving to experience this issue. I have full access to the infrastructure so that's not a problem. The specific issue is that these devices will associate with the AP, everything will work for a while, and then the client will stop sending and receiving data. Visually it doesn't appear to dissociate from the AP, just goes to "limited connectivity." I'm usually not there when it happens so it's hard for me to get additional data from the client machine. I'll take a look at that link and follow up on the other suggestions, thanks. There's no on-site controller, it's an Aerohive solution and they report back to a central server that's not on site. (They don't actually need to be able to communicate with the server to work, although they should always be able to if there's not a network problem preventing it, it's just how you monitor them and apply configurations and updates.)
|
|
#
?
Aug 28, 2018 13:04
|
|
|
Get them to prove a network issue rather than you having to prove it isn’t. Edit: I’ve torn out Aerohive deployments where everything looks fine in the logs but clients just can’t connect. Their support have a go-to excuse of blaming RF interference, which caused a six week ticket about devices not getting DHCP leases despite looking fine in the dashboard before they admitted it was a software bug. They’re on my poo poo list, and judging how the wireless industry is I assume they are either due to get acquired or disappear. Thanks Ants fucked around with this message at 13:07 on Aug 28, 2018 |
|
#
?
Aug 28, 2018 13:04
|
|
|
guppy posted:just goes to "limited connectivity." At least on our w10 machines that happens if the dns or the gateway doesn't reply to pings fast or reliably enough for MS standards. Can you duplicate the issue and see if gw and the dns works while the client is into this error state?
|
|
#
?
Aug 28, 2018 13:09
|
|
|
guppy posted:Definitely enterprise level, and we do not share space with anyone. All devices experiencing issues are laptops -- desktops are wired -- and are mostly the same model, and they do not need to be moving to experience this issue. I have full access to the infrastructure so that's not a problem. The specific issue is that these devices will associate with the AP, everything will work for a while, and then the client will stop sending and receiving data. Visually it doesn't appear to dissociate from the AP, just goes to "limited connectivity." I'm usually not there when it happens so it's hard for me to get additional data from the client machine. I'll take a look at that link and follow up on the other suggestions, thanks. There's no on-site controller, it's an Aerohive solution and they report back to a central server that's not on site. (They don't actually need to be able to communicate with the server to work, although they should always be able to if there's not a network problem preventing it, it's just how you monitor them and apply configurations and updates.) You could setup a scheduled task to run Test-NetConnection (Windows 8/10) or Test-Connection (7+?) every second and write to a log file. This would allow you to determine if the machines are really losing network connectivity, and provide specific timestamps for when the issue is occuring. (You would need to do a little scripting to include a timestamp with the connection tests, check out the powershell thread if you need any help) Depending on what's happening, you may also see something in event viewer.
|
|
#
?
Aug 28, 2018 14:01
|
|
|
Further, DNS issues usually show up in Windows event viewer. You can narrow the problem down in that way and start building a smaller scope. And Aerohive in an Enterprise environment can absolutely be your problem. Even if all devices seem healthy and your test devices are working ok, you could easily run into intermittent density and rf issues. Lower end devices don't work as well when you have dozens of phones filling the air with traffic to other networks. If people are really pissed about it, pitch another (Enterprise) wireless solution to put in a test area.
|
|
#
?
Aug 28, 2018 14:06
|
|
|
H110Hawk posted:It always cracked me up when our colo providers asked us if we needed the cages to extend to the ceiling and into the raised floor. When I had our col move our gear into a half-cabinet, I was expecting to have my OWN half-cabinet. Nope. We were sharing a full cabinet with another customer. We had full access to each other's gear, cabling, spare external drives left in the cabinet, everything. These were the same clowns that did a generator test without a single UPS in-circuit. They flipped the switch on mains power and everything shut down. Everything. The whole data center. I went in to nursemaid starting everything up again and the place was packed with angry sysadmins. I would not have been surprised to see a few rolls of carpet and bags of lyme.
|
|
#
?
Aug 28, 2018 14:28
|
|
|
ChubbyThePhat posted:Long time no post. Looks like I have a new job lined up and am free from MSP hell. Less travel, better hours, 4 weeks vacation (week between Christmas and New Years is free cause office is simply closed), and a 21k salary increase. Not too shabby if I do say so
|
|
#
?
Aug 28, 2018 14:35
|
|
|
mllaneza posted:When I had our col move our gear into a half-cabinet, I was expecting to have my OWN half-cabinet. Nope. We were sharing a full cabinet with another customer. We had full access to each other's gear, cabling, spare external drives left in the cabinet, everything. This sounds amazing. Name and shame.
|
|
#
?
Aug 28, 2018 15:19
|
|
|
Virigoth posted:“It’s so much to read can you just tell me what to do? I’m on a deadline and just need it done” This made my blood pressure shoot up to pre-stroke levels. "Vargatron, your documentation is too detailed, can you make it so that the level 1 tech support can recover the nightly maintenance process when it crashes???"
|
|
#
?
Aug 28, 2018 16:11
|
|
|
To be fair if there's a process that should be performed by your phone monkeys, it should be provided in a simple cookbook style step-by-step instruction. There's no need to dump a hundred pages of architectural and design documentation on someone just to show them how to restart a service.
|
|
#
?
Aug 28, 2018 16:17
|
|
|
Some genius plugged the redundant cables in the same pdu for all our servers in several racks. Got that fixed before tragedy struck
|
|
#
?
Aug 28, 2018 16:20
|
|
|
|
| # ? Jun 10, 2024 20:46 |
|
|
We never had a software side to our previous phone system. People now open up the phone client. Just start double-clicking all over the gently caress. Dialing numbers. Etc. "This is too complicated *close*"  Just use the phone then, bitch.
|
|
#
?
Aug 28, 2018 16:28
|
|
