Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
rock2much
Feb 6, 2004

Grimey Drawer

Antioch posted:

45 tickets came in.

I was on vacation for 2 weeks. As far as I can tell, no one looked at the queue the entire time I was gone.

There's tickets in here that were assigned the day I left that are now a week+ overdue before they even get looked at.

There's 7 people on the team. I know we have projects, but seriously assholes, I shouldn't be the only one looking at these things.

What happened to me last year was:
rock2much requests 2wks vacation, team lead designates 2 people to pick up the slack
rock2much returns 2wks later, all old cases reassigned to him, team lead wants to know why cases are 2wks+ overdue

Adbot
ADBOT LOVES YOU

blackswordca
Apr 25, 2010

Just 'cause you pour syrup on something doesn't make it pancakes!
So a meeting came in.

There was a meeting at the client site asking for more information about the great server recycling fiasco of last week. I spent over an hour basically saying "I was here, I don't know what happened, I know as much as you do."

I did get a call from the general manager though. I got in a little trouble for telling the Sr to go gently caress himself last week. Nothing major actually just got asked to "be a bit more politically correct next time"

They are starting to be nice to me, I'm getting a bit scared.

Alliterate Addict
Jul 10, 2012

dreaming of that face again

it's bright and blue and shimmering

grinning wide and comforting me with it's three warm and wild eyes

blackswordca posted:

They are starting to be nice to me, I'm getting a bit scared.

Probably realizing how hosed they'd be if you left. Not enough to actually give you a raise, but all the more reason to quit now so you can see the look of despair on their faces in real time, as opposed to imagining it when they're calling you two weeks after the fact.

Mustache Ride
Sep 11, 2001



Maybe the client contact has said how much they like you, and how they would hate to see you go.

A Frosty Witch
Apr 21, 2005

I was just looking at it and I suddenly got this urge to get inside. No, not just an urge - more than that. It was my destiny to be here; in the box.
I had probably the single greatest thing so far happen to me a few minutes ago.

I get a panicked, urgent page from one of our reporters.

:stare: "I need you to come down here right now. I'm trying to make it quote and it's just L-ing at me. You make it quit, you hear?"

I go to check out what he's freaking out about. Apparently, he's trying to make quotation marks in his story editing program and it's making lowercase Ls instead. I sit down and try it myself and get the same issue. Curious, but a restart would probably fix it. End of story, right?

Nope, this reporter is convinced that his computer is now literally home to some sort of actual evil spirit and wants nothing to do with it. He begins fretfully praying in Creole, dons his fedora, and quickly leaves the premises.

The entire newsroom just sat in stunned silence for a minute and stared at me. I restarted the computer, shrugged at the manager, left, and now I can't quit laughing.

Malkar
Aug 19, 2010

Taste the cloud

larchesdanrew posted:

I had probably the single greatest thing so far happen to me a few minutes ago.

I get a panicked, urgent page from one of our reporters.

:stare: "I need you to come down here right now. I'm trying to make it quote and it's just L-ing at me. You make it quit, you hear?"

I go to check out what he's freaking out about. Apparently, he's trying to make quotation marks in his story editing program and it's making lowercase Ls instead. I sit down and try it myself and get the same issue. Curious, but a restart would probably fix it. End of story, right?

Nope, this reporter is convinced that his computer is now literally home to some sort of actual evil spirit and wants nothing to do with it. He begins fretfully praying in Creole, dons his fedora, and quickly leaves the premises.

The entire newsroom just sat in stunned silence for a minute and stared at me. I restarted the computer, shrugged at the manager, left, and now I can't quit laughing.

Tomorrow he's going to come in and dump holy water on it.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Malkar posted:

Tomorrow he's going to come in and dump holy water on it.

Nah I'm picturing something like this.

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


Well, this has been a day.

30 spontaneous reboots in the last week. 26 of them in the last 36 hours. WTF.

So now, because HP support are whiny bitches, I'm updating the iLOs on 385 DL 380 G7s. At least we still got 2nd level support to talk to us WHILE we're doing the updates.

I want to blame power, but the reboots are across several racks and between 2 machine rooms about 300 miles apart. What a day.

RFC2324
Jun 7, 2012

http 418

AlexDeGruven posted:

Well, this has been a day.

30 spontaneous reboots in the last week. 26 of them in the last 36 hours. WTF.

So now, because HP support are whiny bitches, I'm updating the iLOs on 385 DL 380 G7s. At least we still got 2nd level support to talk to us WHILE we're doing the updates.

I want to blame power, but the reboots are across several racks and between 2 machine rooms about 300 miles apart. What a day.

I almost envy you. My company won't update poo poo without a six month review process, which seems to get reset every single time a vendor says 'all your issues will go away if you patch/update'

I understand and support controlling new poo poo like that, but gently caress, we could prevent so much downtime.

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


RFC2324 posted:

I almost envy you. My company won't update poo poo without a six month review process, which seems to get reset every single time a vendor says 'all your issues will go away if you patch/update'

I understand and support controlling new poo poo like that, but gently caress, we could prevent so much downtime.

iLOs are really good at loving things up. We're moving toward a policy of pushing out within days of release due to some issues we've had.

1.57 was great. It would throw shittons of false positive temperature alerts.

Mattress Man
Apr 3, 2009

:dukedog:
A ticket came in. Evidently last night's PBX update reset some of the toll group permissions so you could call anyone but internal extensions. All it was supposed to do was update daylight savings time on the phone configs.

RFC2324
Jun 7, 2012

http 418

AlexDeGruven posted:

iLOs are really good at loving things up. We're moving toward a policy of pushing out within days of release due to some issues we've had.

1.57 was great. It would throw shittons of false positive temperature alerts.

We have an issue with a handful of our mission criticals constantly throwing false positives on power supplies. One of them, a core NIS server, has had both power supplies and the system board replaced, yet keeps insisting it has a failed power supply.

loving HPs.

evol262
Nov 30, 2010
#!/usr/bin/perl

RFC2324 posted:

We have an issue with a handful of our mission criticals constantly throwing false positives on power supplies. One of them, a core NIS server, has had both power supplies and the system board replaced, yet keeps insisting it has a failed power supply.

loving HPs.

Why do you even have NIS in 2014?

RFC2324
Jun 7, 2012

http 418

evol262 posted:

Why do you even have NIS in 2014?

I wish I knew. Until tomorrow, I have just been doing monitoring, and not a drat one of our servers outside of Japan and China uses NIS, but they are still considered critical to the point that one of them having issues requires shutting down multiple fabs.

Maybe once I am doing more than monitoring it will become clear(I doubt it will).

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


RFC2324 posted:

We have an issue with a handful of our mission criticals constantly throwing false positives on power supplies. One of them, a core NIS server, has had both power supplies and the system board replaced, yet keeps insisting it has a failed power supply.

loving HPs.

That's pretty hosed up, lol. So far we've been lucky and firmware updates have taken care of most of our problems. False positives go away and don't come back.

Though we are starting to suspect there might be a bug in the core iLO3 code that effectively creates a 45 day timer under certain circumstances. The hosts that started rebooting in the last week and a half or so were all updated to 1.65 just about 45 days ago. A few right at the end of January, and then a fuckton updated in the first week or so of February. The flood of spontaneous reboots coincides with the big flood of updates.

Lum
Aug 13, 2003

At the behest of my boss I am now basically trolling our slow and overly bureaucratic head office IT department.

I'm a full time home worker, and asked boss if I could claim for a new toner cartridge on expenses. Told him that the only place I could find one was ebay as it's quite an old printer. When he saw how old he told me to put a ticket in and see what head office make of it.

The printer: A 1993 vintage HP Laserjet 4Si, from the days when HP were a printer company, not an ink company!

ColdStreamNL
Jul 24, 2013
Last Friday two virtual machines went down on a customer location, upon inspection it turned out snapshots created by Veeam backups filled the datastore and VMware couldn't create the vswap files to boot them. This customer has a problematic backup server which takes 3 weeks for a full backup to complete and thus snapshots exist way too long. Exactly the reason why I had disabled all snapshot based backups and 'strongly suggested' the customer to finally replace the backup server (which is about 9 yrs old). Whoever enabled those backup again, thanks!

So after about half an hour of deleting snapshots, booting the virtual machines and disable all backups everything is working correctly. Unfortunately that's not the end of this, of course we had to take "immediate and decisive action". Which resulted in my manager pushing a change done by a jr. tech where one of the virtual machines has been shrunk 20GB so the backups can be enabled again since "there's 125GB available space for snapshots" :woop:

Excellent example of addressing the symptom and not the cause, we still need to fix the problem only now we're 6 billable hours further down the road. Not to mention the most logical fix (purchase new storage for the backups and relocate Veeam) requires us to expand the virtual machine that the jr. tech took 6 hours to shrink, good luck explaining that one...

QuiteEasilyDone
Jul 2, 2010

Won't you play with me?
A ticket came in...

Assist field tech onsite last minute with a PC deployment.

I get onsite and I see the shithead account manager onsite so I know that this is going to be good. Turns out that I'm right.

12 PCs straight from the manufacturer, equipped with office 2013 on a 2003 exchange environment , never seen this model before, don't have even have a shadow protect image let alone an SCCM deployment package, and Lenovo factory shitware to top off this turd cake.

Oh and it has to be done by 3.

FlapYoJacks
Feb 12, 2009

QuiteEasilyDone posted:

A ticket came in...

Assist field tech onsite last minute with a PC deployment.

I get onsite and I see the shithead account manager onsite so I know that this is going to be good. Turns out that I'm right.

12 PCs straight from the manufacturer, equipped with office 2013 on a 2003 exchange environment , never seen this model before, don't have even have a shadow protect image let alone an SCCM deployment package, and Lenovo factory shitware to top off this turd cake.

Oh and it has to be done by 3.

1) Get drivers
2) Inject into fresh install of Windows 7
3) Create new ghost image
4) Flatten and reinstall.

Entropic
Feb 21, 2007

patriarchy sucks

Lum posted:

At the behest of my boss I am now basically trolling our slow and overly bureaucratic head office IT department.

I'm a full time home worker, and asked boss if I could claim for a new toner cartridge on expenses. Told him that the only place I could find one was ebay as it's quite an old printer. When he saw how old he told me to put a ticket in and see what head office make of it.

The printer: A 1993 vintage HP Laserjet 4Si, from the days when HP were a printer company, not an ink company!

Those '90s vintage HP laserjets are fuckin' tanks and no IT guy worth his salt should bat an eye at seeing one still running.

skooma512
Feb 8, 2012

You couldn't grok my race car, but you dug the roadside blur.
Boss wants to pull me off a night shift because the last time I was tired after doing it.

... 7 days straight of 12 hour night shifts.


:downsbravo:

Lum
Aug 13, 2003

Entropic posted:

Those '90s vintage HP laserjets are fuckin' tanks and no IT guy worth his salt should bat an eye at seeing one still running.

Quite, but no IT person worth their salt works at head office.

Well maybe one, there's a woman who usually gets most of my tickets, and she's good, knows me and my boss well and will probably pass this one on to join in on the trolling.


Beyond that there's the helpdesk guy who when told "My govt. customers have only approved webex for remote support, no other options" bought us a LogMeIn licence and there is Mr. Virtualise All The Things, it's the solution to every problem. His worldview was completely broken when a Defence project required the systems to be airgapped.

TITTIEKISSER69
Mar 19, 2005

SAVE THE BEES
PLANT MORE TREES
CLEAN THE SEAS
KISS TITTIESS




ratbert90 posted:

1) Get drivers
2) Inject into fresh install of Windows 7
3) Create new ghost image
4) Flatten and reinstall.

Outlook 2013 does not support Exchange 2003.

sfwarlock
Aug 11, 2007

Wilford Cutlery posted:

Outlook 2013 does not support Exchange 2003.

OWA for everyone! :suicide:

Trastion
Jul 24, 2003
The one and only.

sfwarlock posted:

AOL for everyone! :suicide:

Fixed that for you!

QuiteEasilyDone
Jul 2, 2010

Won't you play with me?
The account manager in question apparently elected not to have a PM or use any technical resource in his decision making process of sending factory new machines onsite to the client, have absolutely no process for vetting the machines beforehand, and the only reason I'm involved was because the senior field tech that we have put up an SOS

ConfusedUs
Feb 24, 2004

Bees?
You want fucking bees?
Here you go!
ROLL INITIATIVE!!





ColdStreamNL posted:

Last Friday two virtual machines went down on a customer location, upon inspection it turned out snapshots created by Veeam backups filled the datastore and VMware couldn't create the vswap files to boot them. This customer has a problematic backup server which takes 3 weeks for a full backup to complete and thus snapshots exist way too long. Exactly the reason why I had disabled all snapshot based backups and 'strongly suggested' the customer to finally replace the backup server (which is about 9 yrs old). Whoever enabled those backup again, thanks!

So after about half an hour of deleting snapshots, booting the virtual machines and disable all backups everything is working correctly. Unfortunately that's not the end of this, of course we had to take "immediate and decisive action". Which resulted in my manager pushing a change done by a jr. tech where one of the virtual machines has been shrunk 20GB so the backups can be enabled again since "there's 125GB available space for snapshots" :woop:

Excellent example of addressing the symptom and not the cause, we still need to fix the problem only now we're 6 billable hours further down the road. Not to mention the most logical fix (purchase new storage for the backups and relocate Veeam) requires us to expand the virtual machine that the jr. tech took 6 hours to shrink, good luck explaining that one...

Welcome to my world. I'm always amused (and slightly horrified) by the backup and restore issues that can easily be solved by "buy more storage", "no, we can't make your 10TB of data upload any faster than your 1.5Mbps DSL line allows", or "you didn't back that up, so no we cannot restore it."

Rhymenoserous
May 23, 2008

ColdStreamNL posted:

Last Friday two virtual machines went down on a customer location, upon inspection it turned out snapshots created by Veeam backups filled the datastore and VMware couldn't create the vswap files to boot them. This customer has a problematic backup server which takes 3 weeks for a full backup to complete and thus snapshots exist way too long. Exactly the reason why I had disabled all snapshot based backups and 'strongly suggested' the customer to finally replace the backup server (which is about 9 yrs old). Whoever enabled those backup again, thanks!

So after about half an hour of deleting snapshots, booting the virtual machines and disable all backups everything is working correctly. Unfortunately that's not the end of this, of course we had to take "immediate and decisive action". Which resulted in my manager pushing a change done by a jr. tech where one of the virtual machines has been shrunk 20GB so the backups can be enabled again since "there's 125GB available space for snapshots" :woop:

Excellent example of addressing the symptom and not the cause, we still need to fix the problem only now we're 6 billable hours further down the road. Not to mention the most logical fix (purchase new storage for the backups and relocate Veeam) requires us to expand the virtual machine that the jr. tech took 6 hours to shrink, good luck explaining that one...

I had an SQL server go down the other day. Reason? My manager setup the drives to shadowcopy because he didn't trust our snapshots. 200GB db. 500 GB of shadowcopies.

odiv
Jan 12, 2003

If you keep calling with a problem that's a symptom of poo poo not being set up correctly and I tell you this and you still refuse to take the time to help me set things up correctly, then guess what will happen? Yes, I know you're important and busy which is why I'm trying to save us all some time here.

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


odiv posted:

If you keep calling with a problem that's a symptom of poo poo not being set up correctly and I tell you this and you still refuse to take the time to help me set things up correctly, then guess what will happen? Yes, I know you're important and busy which is why I'm trying to save us all some time here.

In fairness, this is precisely why first line tech support is so frustrating to call when you're experienced. The people behind the scenes have figured out that most of the callers are stupid and haven't done the basic steps yet, and so they now refuse to go any further until it's confirmed to have been done.

Stanos
Sep 22, 2009

The best 57 in hockey.
I always felt rude explaining that I've already unplugged my modem, disconnected my wireless and tried a direct physical connection so I just humor the first line people.

A c E
Jun 18, 2007

Is this weird? Is this too weird? Do you need to sit down?

Stanos posted:

I always felt rude explaining that I've already unplugged my modem, disconnected my wireless and tried a direct physical connection so I just humor the first line people.

I gave up on fighting this and just say I'm performing the steps, and will actually perform any if I've missed something suggested.

Humbug Scoolbus
Apr 25, 2008

The scarlet letter was her passport into regions where other women dared not tread. Shame, Despair, Solitude! These had been her teachers, stern and wild ones, and they had made her strong, but taught her much amiss.
Clapping Larry

Entropic posted:

Those '90s vintage HP laserjets are fuckin' tanks and no IT guy worth his salt should bat an eye at seeing one still running.

This. I have a 2100TN that is loving indestructible.

EoRaptor
Sep 13, 2003

by Fluffdaddy

Lum posted:

At the behest of my boss I am now basically trolling our slow and overly bureaucratic head office IT department.

I'm a full time home worker, and asked boss if I could claim for a new toner cartridge on expenses. Told him that the only place I could find one was ebay as it's quite an old printer. When he saw how old he told me to put a ticket in and see what head office make of it.

The printer: A 1993 vintage HP Laserjet 4Si, from the days when HP were a printer company, not an ink company!

Somewhere in my office I have a brand new in-box genuine HP cartridge for that. I also have another one that some idiot opened to try out in their brother printer, but isn't actually used.

nexxai
Jul 17, 2002

quack quack bjork
Fun Shoe
Just a quick shout-out to Ubiquiti for their magical UniFi series of devices. I had 10 APs configured and working around our office in under 30 minutes (including walking time). Goddamn, why can't all hardware be this easy to setup?

Helushune
Oct 5, 2011

nexxai posted:

Just a quick shout-out to Ubiquiti for their magical UniFi series of devices. I had 10 APs configured and working around our office in under 30 minutes (including walking time). Goddamn, why can't all hardware be this easy to setup?

We have a bunch of Aruba Wireless APs that basically do the same thing and it's an absolute godsend. Getting the first master going was trial and error since we had a couple that wanted to be in isolation mode but to add more, we just plug them in. They figure out which one is the master, pull the config off it, talk to the others in the area to figure out which channels are free and what is optimal to use, and just go from there.

babies havin rabies
Feb 24, 2006

Helushune posted:

We have a bunch of Aruba Wireless APs that basically do the same thing and it's an absolute godsend. Getting the first master going was trial and error since we had a couple that wanted to be in isolation mode but to add more, we just plug them in. They figure out which one is the master, pull the config off it, talk to the others in the area to figure out which channels are free and what is optimal to use, and just go from there.

I didn't set them up originally, but our WatchGuard APs seem to do this type of stuff as well.

SyNack Sassimov
May 4, 2006

Let the robot win.
            --Captain James T. Vader


nexxai posted:

Just a quick shout-out to Ubiquiti for their magical UniFi series of devices. I had 10 APs configured and working around our office in under 30 minutes (including walking time). Goddamn, why can't all hardware be this easy to setup?

While I initially shared your optimism and delight, there are a couple of issues we've run into. Their v3 firmware has been in beta for more than a year now and while mostly solid seems to have some problems, and the expensive (well for UBNT) AC APs seem to have a LOT of problems. We've been sticking to the Pros and not seen a lot of issues, other than one client in SF which has huge wireless interference from all around and where the users were insisting on always being on wireless everywhere (i.e. not plugging in at their desks).

Also, the fact that the controller doesn't do SNMP and the APs only do basic SNMP v1 and don't seem to offer the stats on clients / traffic through SNMP is rather annoying for an "enterprise" system.

All that said, it's dead simple to setup, the controller can be installed in two seconds (though if installing as a Windows service, they STILL don't have it working with 64-bit Java so make sure you install 32-bit Java because the setup program by default installs 64-bit Java, and while the controller will start when you manually run it like that, it won't launch the service properly), and the power and range of the things is pretty goddamn impressive.

We've also set up AirVision and use a LOT of their PoE ToughSwitches, and the capabilities of their NVR software and cameras are pretty fantastic for the price, and the switches do what we need without a lot of screwing around. So all in all, I like them a lot, but I wish they'd fix the last little bits and release software faster.

sfwarlock
Aug 11, 2007

nexxai posted:

Just a quick shout-out to Ubiquiti for their magical UniFi series of devices. I had 10 APs configured and working around our office in under 30 minutes (including walking time). Goddamn, why can't all hardware be this easy to setup?

Fair warning: They start acting weird if they're too close. They recommend 100-120 meters inbetween.

Adbot
ADBOT LOVES YOU

Inspector_666
Oct 7, 2003

benny with the good hair

Helushune posted:

We have a bunch of Aruba Wireless APs that basically do the same thing and it's an absolute godsend. Getting the first master going was trial and error since we had a couple that wanted to be in isolation mode but to add more, we just plug them in. They figure out which one is the master, pull the config off it, talk to the others in the area to figure out which channels are free and what is optimal to use, and just go from there.

I had a moment with some Merakis a little while back. I did all the setup without even taking them out of the boxes, and on-site I decided one of them would need to be a wireless repeater I unboxed it and plugged in the power, and while I was thinking "Ah, crap, I'll probably have to at least get it online to begin with" it automatically set itself up as if I had already told it what to do.

:swoon:

  • Locked thread