Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
nitrogen
May 21, 2004

Oh, what's a 217°C difference between friends?

Agrikk posted:


Note that web servers behind hardware load balancers should also not be vMotion-ed, IMHO. If they flip to a different host, the momentary outage can disrupt sessions and cause it to fail a heartbeat that'll drop it from the VIP temporarily and you'll get member servers flapping in the pool.

I have NEVER seen this, and I have set this up countless times. Am I just super lucky, or what?

Adbot
ADBOT LOVES YOU

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


In my AIX shop, we never used LPM (VMotion for the VMWare people, Live Migration for the Xen/KVM people) as an HA measure, even though it was explicitly spelled out as an option by IBM. Too much of our poo poo reacted too poorly to that kind of behavior.

So we set up PowerVM (NPIV is a MUST) so that we could LPM anything to anywhere, but kept our HA clusters split between the rooms.

The unexpected loss of a host created an HA failover which gave us ~5 minutes or less service downtime.

For maintenance, we would move all of the LPARs to an empty host (we always had at least one host spinning idle in either room, either by design or just not used yet) with no downtime and do whatever we needed to on the host.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

nitrogen posted:

I have NEVER seen this, and I have set this up countless times. Am I just super lucky, or what?

You may or may not be lucky, but I assure you that I am unlucky. :smith:

I have never worked for a profitable company and have seen the death of many a startup. I have been present for so many projects that the contractor said "would only take an hour to deploy" that a coworker once as a joke made me a calendar in which a hour time slot took up three days, and you know how some people arrive and mysteriously fix problems "Well it wasn't working just a moment ago"? I have the opposite "Dammit, it was working a second ago" effect on machines.

Splashy Gravy
Dec 21, 2004

I HAVE FURY!
Slippery Tilde
Those of you who got in to that Powershell course, report back with your thoughts. It seems pretty enticing even for $100. - I just noticed there's some free lectures. I'll have to check those out.

blackswordca
Apr 25, 2010

Just 'cause you pour syrup on something doesn't make it pancakes!
My spidey sense is tingling... this is going to be a complete and total gong show on Monday when the line is switched over.

Ive spoken to the Bell support agent about this, the reason they stayed with Bell was so that the external facing IP didn't change. When i talked to the agent at Bell, I was told that the routing for the external IP and the IP itself was managed by the Intranet provider. This didn't make much sense to me but I've never dealt with any kind of multi site internet/intranet setup before so I could very well br wrong. I contacted the intranet provider, they weren't sure if they did. When the agent I was on the phone with spoke to his senior tech and to the addressing team, the still had no idea. They did an IP lookup though and told me the IP was owned by Bell. When I asked if they needed the updated IP information if it changed for any of their routing for us, the senior NOC guy replied to me with "Uh.. I dunno.. maybe?"

I tried asking the Account lead and Sr tech on the account for more information, neither of them know anything about it. The person that did know how all of the routing worked, was fired almost a year ago and of course there is no documentation from him.

On a related note, I have some prospects with a couple different energy companies, one of them came to me for a resume. I am hoping something comes of this before this completely blows up, but I don't think I am that lucky.

blackswordca fucked around with this message at 22:04 on Nov 14, 2013

door.jar
Mar 17, 2010

Agrikk posted:

Note that web servers behind hardware load balancers should also not be vMotion-ed, IMHO. If they flip to a different host, the momentary outage can disrupt sessions and cause it to fail a heartbeat that'll drop it from the VIP temporarily and you'll get member servers flapping in the pool.

I have also seen this happen to an application I support. Was amazing fun to try and show the root cause when I don't have any access to the VM architecture. I'm somewhat sure that I also came across a vMotion in the middle of a java full garbage collection causing hilarious issues but I'm not 100% sure.

stubblyhead
Sep 13, 2007

That is treason, Johnny!

Fun Shoe

BabyFur Denny posted:

In Germany November 11th 11:11 is the official start of Carnival season. :colbert:

I knew the Lutherans had a bunch of issues with the Catholic church, but moving lent from Feb/Mar. to November? Madness!

blackswordca posted:

My spidey sense is tingling... this is going to be a complete and total gong show on Monday when the line is switched over.

Keep this handy, should come in useful: https://www.youtube.com/watch?v=ZnHmskwqCCQ

blackswordca
Apr 25, 2010

Just 'cause you pour syrup on something doesn't make it pancakes!

stubblyhead posted:

I knew the Lutherans had a bunch of issues with the Catholic church, but moving lent from Feb/Mar. to November? Madness!


Keep this handy, should come in useful: https://www.youtube.com/watch?v=ZnHmskwqCCQ

Im playing this right now..

Just got off the phone with the local Bell line tech, apparently nobody at Bell knew that we had existing Bell service.

MJP
Jun 17, 2007

Are you looking at me Senpai?

Grimey Drawer
A new job came in...

So 216.86.148.111 is unblocked, but forums.somethingawful.com isn't. I can get to the login page, but logging in is done through forums., bringing me back to block even if I replace the hostname with IP.

Is there any way to log on via the IP only to SA (which is blocked by OpenDNS as Weapons...)?

Trastion
Jul 24, 2003
The one and only.

MJP posted:

(which is blocked by OpenDNS as Weapons...)?

The favorite logo is a grenade. :)

Sorry cant help you with the blocking part. I run the web filter at my place so i am exempt to it. :)

Paladine_PSoT
Jan 2, 2010

If you have a problem Yo, I'll solve it

stubblyhead posted:

I knew the Lutherans had a bunch of issues with the Catholic church, but moving lent from Feb/Mar. to November? Madness!


Keep this handy, should come in useful: https://www.youtube.com/watch?v=ZnHmskwqCCQ

Lent begins at the end of carnival, who says carnival can't be 3 months long? It's germany, gatta do something when octoberfest hangovers wear off...

Lum
Aug 13, 2003

MJP posted:

A new job came in...

So 216.86.148.111 is unblocked, but forums.somethingawful.com isn't. I can get to the login page, but logging in is done through forums., bringing me back to block even if I replace the hostname with IP.

Create a new Firefox profile at home, log in, bring Firefox profile into work on a USB stick?

Add an entry for forums.somethingawful.com in your local hosts file?

E4C85D38
Feb 7, 2010

Doesn't that thing only
hold six rounds...?

Or just use your SSH client of choice for tunneling traffic (I prefer Tunnelier) to route all your personal traffic away from prying eyes/web filters.

Vin BioEthanol
Jan 18, 2002

by Ralp
I used to tunnel http (rdp and other stuff too) over ssh a lot. Now we can't access anything on the internet except port 80 or 443.

I setup on my router so that 443 will forward to my ssh (only allowing work IP to do that) and it works but I worry that someone's watching and all 'hey that's not https traffic'

Swink
Apr 18, 2006
Left Side <--- Many Whelps
Before you drop $100 on Powershell, watch this 4 hour presentation from Don Jones - https://www.youtube.com/watch?v=-Ya1dQ1Igkc

This gave me the grounding and then I went from there.

stubblyhead
Sep 13, 2007

That is treason, Johnny!

Fun Shoe

Swink posted:

Before you drop $100 on Powershell, watch this 4 hour presentation from Don Jones - https://www.youtube.com/watch?v=-Ya1dQ1Igkc

This gave me the grounding and then I went from there.

Don Jones is a pretty awesome guy who knows his poo poo.

nitrogen
May 21, 2004

Oh, what's a 217°C difference between friends?

Vin BioEthanol posted:

I used to tunnel http (rdp and other stuff too) over ssh a lot. Now we can't access anything on the internet except port 80 or 443.

I setup on my router so that 443 will forward to my ssh (only allowing work IP to do that) and it works but I worry that someone's watching and all 'hey that's not https traffic'

IF they were really serious they would proxy your traffic.

KweezNArt
Jul 30, 2007
I wish we could include images in our outbound ticket updates.

quote:

I am trying to print and it says the printer is offline; how do I get the printer back on line?

EAT THE EGGS RICOLA
May 29, 2008

KweezNArt posted:

I wish we could include images in our outbound ticket updates.



Go show them, record a video of their doing it, then make a gif using gifcam that you can send back to them if they ask again.

Alctel
Jan 16, 2004

I love snails


blackswordca posted:

Im playing this right now..

Just got off the phone with the local Bell line tech, apparently nobody at Bell knew that we had existing Bell service.

This really really does not surprise me.

We have a 5 site MPLS through Bell, and they are spectacularly useless. Not maliciously useless like some other telecoms but just clueless.

EAT THE EGGS RICOLA
May 29, 2008

Alctel posted:

This really really does not surprise me.

We have a 5 site MPLS through Bell, and they are spectacularly useless. Not maliciously useless like some other telecoms but just clueless.

We're on our third day of waiting for Epik to make a relatively urgent change to DNS for us. We used to have the ability to do it ourselves but somehow they took that away from us and we didn't notice until it needed to be done.

Allstream was better somehow, and I hated working with Allstream.

Citizen Z
Jul 13, 2009

~Hanzo Steel~


This was my ticket today.




That thing. It's at a bare minimum 15 years old, is probably completely custom or drat close to it, and is the control unit for a big test stand that causes a production stoppage when it's down.

It also runs NT4. I had the misfortune of being the only guy in the office when it went down, and the only guy on the team with any experience with NT4. Not to mention not smart enough to be scared shitless of it.

GreenNight
Feb 19, 2006
Turning the light on the darkest places, you and I know we got to face this now. We got to face this now.

Rip one of the caps off the motherboard, then take a nap. Tell people you worked on it all night and couldn't get it running. You think it's a dead motherboard.

Citizen Z
Jul 13, 2009

~Hanzo Steel~


GreenNight posted:

Rip one of the caps off the motherboard, then take a nap. Tell people you worked on it all night and couldn't get it running. You think it's a dead motherboard.

I'm genuinely not certain where the motherboard is. There's a central card in the back of the frame that everything plugs into, but I think it's just the bus. There are two rows of connectors that look like old riser/daughterboard connectors. Most of the modules are enclosed in a metal case. I suspect it's the module next to the one with the SCSI port on it, but I'm not positive.

I was able to fix the problem(Corrupt registry hive in NT4). Since there's no optical drive(And no IDE connector that I can find) and it won't boot from USB, I had to get the drives out. Then I had to call around town, found a parts guy with an old P3 system(Dual Xeons, 2 GB of RAM!) with the right kind of SCSI controller. Got it up into a live CD, imported the damaged hive and repaired it, then put it back in.

corgski
Feb 6, 2007

Silly goose, you're here forever.

Next time instead of prolonging the inevitable, just point a butane lighter at a capacitor on the backplane until it bulges and leaks. Also, take an image of that drive before it reboots to a click of death.

EAT THE EGGS RICOLA
May 29, 2008

Tell them that it fixed itself and you have no idea how so that you don't get called next time that happens.

GreenNight
Feb 19, 2006
Turning the light on the darkest places, you and I know we got to face this now. We got to face this now.

Fixing something that old is just asking yourself to get blamed when it eventually does fail. Welcome to a world of getting everyones finger pointed at you.

Citizen Z
Jul 13, 2009

~Hanzo Steel~


thelightguy posted:

Next time instead of prolonging the inevitable, just point a butane lighter at a capacitor on the backplane until it bulges and leaks. Also, take an image of that drive before it reboots to a click of death.

I did grab an image while I had the drive available. When I asked the engineer in charge of the stand why no one had taken one in the 15 years the thing had been in production he did that home vanishing into the bushes thing.

Citizen Z fucked around with this message at 02:06 on Nov 15, 2013

Devor
Nov 30, 2004
Lurking more.


Citizen Z posted:

I did grab an image while I had the drive available. When I asked the engineer in charge of the stand why no one had taken one in the 15 years the thing had been in production he did that home vanishing into the bushes thing.

Lazy

Daylen Drazzi
Mar 10, 2007

Why do I root for Notre Dame? Because I like pain, and disappointment, and anguish. Notre Dame Football has destroyed more dreams than the Irish Potato Famine, and that is the kind of suffering I can get behind.
Jesus, that looks like something I would have encountered back in my desktop support days at Kodak. There was one time when we had a contact-testing station running on a computer that was about 15 years old running one of the earlier versions of DOS. The one time it went down I literally had the plant manager and a couple vice-presidents screaming at me to get it fixed now Now NOW!! Fortunately it turned out to just need a CMOS battery, but I thought the horrified expressions in reaction to my suggestion that they pay to replace this critical piece of equipment with something made in this decade was a bit over-the-top.

QuiteEasilyDone
Jul 2, 2010

Won't you play with me?
"So you're fine with this computer failing on its own terms and bringing your entire assembly line to a halt for however long it takes to to repair or inevitably replace when it catches fire one of these days. Gotcha"

dennyk
Jan 2, 2005

Cheese-Buyer's Remorse

QuiteEasilyDone posted:

"So you're fine with this computer failing on its own terms and bringing your entire assembly line to a halt for however long it takes to to repair or inevitably replace when it catches fire one of these days. Gotcha"

Of course they are; if they decide to spend money and take downtime to replace it now, then the resulting loss of profit is their fault. If the computer dies, then it's the IT guy's fault, and management gets bonuses for successfully minimizing the impact of a difficult unforeseen emergency by yelling at the IT guy to fix the problem that he created. :v:

MisterOblivious
Mar 17, 2010

by sebmojo
So I had this summer job and the line next to me had a "problematic" piece of equipment. We're talking daily shutdowns of an entire production line, sometimes for hours at a time, sort of problematic. 1600 units per minute down to 0. Paying 6 union workers overtime wages to sit around while 2 others wrench the equipment back to life.

The machine was designed to have 1/3rd of it overhauled each year. Parts checked, fixed, replaced, etc. As it's a highly seasonal industry (voluntary and mandatory layoffs in the off season) it's no problem to schedule a week of downtime on that one line.

It should have been fully rebuilt at least 7 times in it's lifetime.

The company that built the equipment couldn't believe it was running at all considering it had never received a single overhaul.



Remember this the next time you wonder why a can of sugar water costs so drat much.

Citizen Z
Jul 13, 2009

~Hanzo Steel~


Daylen Drazzi posted:

Jesus, that looks like something I would have encountered back in my desktop support days at Kodak. There was one time when we had a contact-testing station running on a computer that was about 15 years old running one of the earlier versions of DOS. The one time it went down I literally had the plant manager and a couple vice-presidents screaming at me to get it fixed now Now NOW!! Fortunately it turned out to just need a CMOS battery, but I thought the horrified expressions in reaction to my suggestion that they pay to replace this critical piece of equipment with something made in this decade was a bit over-the-top.

We're actually keeping this thing going for another year or so, then replacing it. It's rated to run 800-900 units a day, but only does about 30-40 and just doesn't justify the floor space it takes up. They're going to replace it with a smaller unit more in line with what it needs to do. The problem is the removal of this thing is going to take a lot of time and money.

I should go take a picture or two, it's kind of cool. Big conveyor platforms and crazy armatures.

EAT THE EGGS RICOLA
May 29, 2008

Admin person just emailed me, called me and left a voicemail message, then came and interrupted me while I was speaking to my boss and his boss because she wanted help printing a bunch of colour pictures of her grandkids.

Lum
Aug 13, 2003

Citizen Z posted:

This was my ticket today.




That thing. It's at a bare minimum 15 years old, is probably completely custom or drat close to it, and is the control unit for a big test stand that causes a production stoppage when it's down.

It also runs NT4. I had the misfortune of being the only guy in the office when it went down, and the only guy on the team with any experience with NT4. Not to mention not smart enough to be scared shitless of it.

Is it bad that I'd actually love to work for a place that still had a few things like that, in addition to a modern infrastructure.

As you can probably tell from the discussion in the other thread, I have a lot of happy memories of that time, also hello job security.

Though I have a sneaking suspicion I'd end up bringing in some PCs from my parts graveyard to try and get that poo poo up and running.

MJP
Jun 17, 2007

Are you looking at me Senpai?

Grimey Drawer
All glory to the hosts file option!

First casual Friday in three years (last two places didn't allow it, gotta love MSPs where you might have to go to a ~finance professional office~ for an onsite and actuarial firms where ~we might have clients~ even in the IT space, where there's no clients ever) and I have an office with the helpdesk guy. Not bad from a door-closable cube.

Plus natural light and around 40 minutes door-to-door.

Now all that matters is the first needful...

user on probation
Nov 1, 2012

removed
Half of management likes to come tell me that a toner cartridge in a printer is low and When are you going to replace it? I keep seeing "toner low" in my printer settings!

The other half insists that I not change toner cartridges until they absolutely will not print a single page more, even if this occurs when some coach is printing 800 pages at 10pm, because That's a waste of toner! These things are expensive!

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
"why didn't you fix X your company doesn't manage???"

Adbot
ADBOT LOVES YOU

guppy
Sep 21, 2004

sting like a byob
I just got a ticket assigned to me for a user who -- this is my description, hers is nigh incomprehensible -- no longer has her recent recipients auto completing in Outlook. She had previously spoken to our helpdesk for a different email-related issue, and among other things they rebuilt her email profile, nuking the file that holds that information. Fortunately the helpdesk rep had the presence of mind to mention in the ticket that he'd done that, so at least I knew what happened. That file is stored locally and no earlier versions are available, so she's out of luck. Which, whatever, you want external contacts saved, add them to your address book. The user is still very upset because "I took the initiative to know that they'd be there." Motherfucker, that is not what "taking the initiative" means. In fact, you did the opposite of taking the initiative. You didn't do poo poo, and that's why you now have a problem.

Same user was also concerned that she wasn't getting email from external senders and thought it might be related, because she texted her son to send her a test email half an hour ago and hadn't gotten anything. Actual problem: no one external to the organization has sent her anything yet today.

  • Locked thread