|
One time I almost got crushed by a SAN rack (I think it was a 3PAR storage rack?) when we were trying to decommission it from the datacenter floor, it was this big fucker, like full height 42u unit, that we had removed all the drives and sent them off with our e-waste company. I think the rack weighed like 2500 LBS? but yeah either way we had 4 people trying to get it down off the raised floor via the ramp at the back of the datahall and yeah you can't stop 2500 lbs on an incline no matter how many people you have in front of it if it wants to move. It put a big dent in the wall lol but nobody died.
|
#
?
Jul 15, 2020 07:07
|
|
|
|
| # ? May 25, 2024 17:32 |
|
|
orange juche posted:One time I almost got crushed by a SAN rack (I think it was a 3PAR storage rack?) when we were trying to decommission it from the datacenter floor, it was this big fucker, like full height 42u unit, that we had removed all the drives and sent them off with our e-waste company. I think the rack weighed like 2500 LBS? but yeah either way we had 4 people trying to get it down off the raised floor via the ramp at the back of the datahall and yeah you can't stop 2500 lbs on an incline no matter how many people you have in front of it if it wants to move. It put a big dent in the wall lol but nobody died. 
|
|
#
?
Jul 15, 2020 07:36
|
|
|
mllaneza posted:I switched from a full to a half cage at a lovely colo once and paid their hands to do the move. Next time I'm in the colo I was very surprised to find that we had half of a full-height cage. Someone else had the other half, so we had full access to each other's stuff. I'm confused. What was your expected outcome here? A shorty rack that's taking up the same amount of floor space just for your stuff plus a price reduction?
|
|
#
?
Jul 15, 2020 14:09
|
|
|
larchesdanrew posted:Tickets haven't come in yet because I'm in training. I'm happy for you. I've enjoyed reading and getting vicariously angry at your stories for years.
|
|
#
?
Jul 15, 2020 14:16
|
|
|
Always tip your server.
|
|
#
?
Jul 15, 2020 14:18
|
|
|
doesn't look too micro to me
|
|
#
?
Jul 15, 2020 14:22
|
|
|
Motronic posted:I'm confused. What was your expected outcome here? A shorty rack that's taking up the same amount of floor space just for your stuff plus a price reduction? I'm thinking a full-height cage with two independently locking halves stacked vertically.
|
|
#
?
Jul 15, 2020 14:23
|
|
|
Motronic posted:I'm confused. What was your expected outcome here? A shorty rack that's taking up the same amount of floor space just for your stuff plus a price reduction?  Much better than another product we sold. Shared cabinet, pay per U. Ended with expected disaster and nearly stolen equipment.
|
|
#
?
Jul 15, 2020 14:34
|
|
|
OmniCorp posted:
I mean, those certainly exist, but I've rarely seen one in the wild. Unless the facility you were at has those (many do not) and you specified one......yeah, I would have expected exactly what you got.
|
|
#
?
Jul 15, 2020 14:54
|
|
|
Motronic posted:I mean, those certainly exist, but I've rarely seen one in the wild. Unless the facility you were at has those (many do not) and you specified one......yeah, I would have expected exactly what you got. Yeah, when I'm renting from explicitly a colo place I expect to be mixed in with everyone else, that's what the "colocation" means to me, though I'd hope everyone would only have access in the presence of the person who's actually renting the rack. Though that said, a lot of places rent by the half rack and I'd probably expect to be in literally a half rack cabinet, but perhaps that's naive.
|
|
#
?
Jul 15, 2020 15:08
|
|
|
larchesdanrew posted:Tickets haven't come in yet because I'm in training. Hopefully there'll always be dumbassery to tell us, but I hope more so that it's someone else's dumbassery. You amongst all of us deserve a decades-long break.
|
|
#
?
Jul 15, 2020 15:18
|
|
|
Jaded Burnout posted:Yeah, when I'm renting from explicitly a colo place I expect to be mixed in with everyone else, that's what the "colocation" means to me, though I'd hope everyone would only have access in the presence of the person who's actually renting the rack. any colo I have been in(5 or 6), if you you don't pay for the square footage for a cage, you don't get a server secured against the other customers touching it.
|
|
#
?
Jul 15, 2020 15:19
|
|
|
Jaded Burnout posted:Though that said, a lot of places rent by the half rack and I'd probably expect to be in literally a half rack cabinet, but perhaps that's naive. In the facilities I've been using for the last......really long time (a mic of everything including Equinix/DRT/Telicity/random local places), yes that expectation would indeed be naive unless it was called out in the contract explicitly. In some very in demand places it's not even an option, and many people are renting by the RU not even half rack (Sovereign Telhouse, I'm looking at you). RFC2324 posted:any colo I have been in(5 or 6), if you you don't pay for the square footage for a cage, you don't get a server secured against the other customers touching it. This is the case in a place with a lot of square footage that they can't fill because they are built a long time ago for something like 75w per square foot, so they've got nothing but floor space they can't fill because they don't have enough power to do so. Think DRT SFR2 (aka 365 Main in San Francisco). You go to some place at 300w/sq. ft. and they are a lot less likely to have copious floor space available and you will be paying for it (Equinix SG3 is an example of that) unless you are buying enough power - like 5kva per rack - in which case it's buried in your power cost. If you want a cage expect to bump that kva per rack higher to account for the floor space you are using that could have been racks.
|
|
#
?
Jul 15, 2020 15:35
|
|
|
RFC2324 posted:any colo I have been in(5 or 6), if you you don't pay for the square footage for a cage, you don't get a server secured against the other customers touching it. Guess I got lucky, I assume because the guy I was renting from didn't have a better way to give people access to the building.
|
|
#
?
Jul 15, 2020 15:35
|
|
|
Motronic posted:In the facilities I've been using for the last......really long time (a mic of everything including Equinix/DRT/Telicity/random local places), yes that expectation would indeed be naive unless it was called out in the contract explicitly. In some very in demand places it's not even an option, and many people are renting by the RU not even half rack (Sovereign Telhouse, I'm looking at you). I was renting 1U so I knew what I was getting there, but at the time I remember assuming that if I forked out for a half rack from the big boys that I'd get isolation, definitely willing to file that under late-teen naivety.
|
|
#
?
Jul 15, 2020 15:37
|
|
|
Jaded Burnout posted:I was renting 1U so I knew what I was getting there, but at the time I remember assuming that if I forked out for a half rack from the big boys that I'd get isolation, definitely willing to file that under late-teen naivety. The big boys often don't even bother selling less than a full cabinet, and will refer you to one of their in-house resellers. That's when you're right back to "is it in the contract?"
|
|
#
?
Jul 15, 2020 15:40
|
|
|
Motronic posted:This is the case in a place with a lot of square footage that they can't fill because they are built a long time ago for something like 75w per square foot, so they've got nothing but floor space they can't fill because they don't have enough power to do so. Think DRT SFR2 (aka 365 Main in San Francisco). You go to some place at 300w/sq. ft. and they are a lot less likely to have copious floor space available and you will be paying for it (Equinix SG3 is an example of that) unless you are buying enough power - like 5kva per rack - in which case it's buried in your power cost. If you want a cage expect to bump that kvay per rack higher to account for the floor space you are using that could have been racks. This makes sense Jaded Burnout posted:Guess I got lucky, I assume because the guy I was renting from didn't have a better way to give people access to the building. This also fits. The places I have dealt with that had poo poo floor security had impressive door security.
|
|
#
?
Jul 15, 2020 15:41
|
|
|
Motronic posted:In the facilities I've been using for the last......really long time (a mic of everything including Equinix/DRT/Telicity/random local places), yes that expectation would indeed be naive unless it was called out in the contract explicitly. In some very in demand places it's not even an option, and many people are renting by the RU not even half rack (Sovereign Telhouse, I'm looking at you).
|
|
#
?
Jul 15, 2020 16:03
|
|
|
The colo I used to work for did offer a few half racks, and there was a row of two of third-racks in there as well. The only shared rack I can recall is the employee-only/lab rack. Sure, someone could have poked a pen in through the grates, but every aisle had video monitoring and three layers of badge/biometrics to get through. The suspect list would have been in the low single digits.
|
|
#
?
Jul 15, 2020 19:00
|
|
|
Jaded Burnout posted:Yeah, when I'm renting from explicitly a colo place I expect to be mixed in with everyone else, that's what the "colocation" means to me, though I'd hope everyone would only have access in the presence of the person who's actually renting the rack. If you rent a half rack and state that you don't want a neighbor in your contract, you will get a half rack that is half empty and full of blanks, but you will be paying the price for a full floor tile. I don't know any place that gives a discount unless you have neighbors in your rack.
|
|
#
?
Jul 15, 2020 19:27
|
|
|
Wow those are some amazing power densities. In CA I have more than one full rack (42) with a total of 15A to it - no ability to increase, if want more then have to pick up another full rack due to power density issues. Some of my friends use their 42U for servers on top, misc storage (snowboards, etc) on bottom. Anyone in the bay area can probably guess which DC this is.
|
|
#
?
Jul 15, 2020 19:42
|
|
|
Biowarfare posted:Wow those are some amazing power densities. In CA I have more than one full rack (42) with a total of 15A to it - no ability to increase, if want more then have to pick up another full rack due to power density issues. Some of my friends use their 42U for servers on top, misc storage (snowboards, etc) on bottom. The ancient one I already mentioned, or a different ancient one?
|
|
#
?
Jul 15, 2020 19:44
|
|
|
I have 2x 32A 240v to a half cabinet in east London, but it's not really surprising that the land of data centres also has lots of power.
|
|
#
?
Jul 15, 2020 19:45
|
|
|
Motronic posted:The ancient one I already mentioned, or a different ancient one? Nah it's in Fremont.
|
|
#
?
Jul 15, 2020 19:52
|
|
|
Biowarfare posted:Wow those are some amazing power densities. In CA I have more than one full rack (42) with a total of 15A to it - no ability to increase, if want more then have to pick up another full rack due to power density issues. Some of my friends use their 42U for servers on top, misc storage (snowboards, etc) on bottom. 200 Paul? edit: a few jobs ago, our french sister company got a honking big 3Par array that was a couple racks big, but needed 4 racks of power due to power density requirements, so the flanking racks were empty but very much needed for power. The colo, in all their colo wisdom, cut power to those racks "because nothing was in them" and that's the story of how we had to guide a major french website through restoring postgres from backup luminalflux fucked around with this message at 19:56 on Jul 15, 2020 |
|
#
?
Jul 15, 2020 19:54
|
|
|
luminalflux posted:200 Paul? All our 3PAR racks were fed straight 240 instead of 120v that had been stepped down. Even then, since the 3PAR arrays took up so much power we set down blank tiles for like 2 positions on either side and drew red Xs on them in electrical tape and labelled the breaker switches for those tiles for the 3PAR so we would remember that the power under those tiles was fed into the 3 3PAR cabinets in that row. This was in a quite modern, large datacenter in Atlanta GA, 42u storage arrays take a lot of power. orange juche fucked around with this message at 20:05 on Jul 15, 2020 |
|
#
?
Jul 15, 2020 20:03
|
|
|
orange juche posted:All our 3PAR racks were fed straight 240 instead of 120v that had been stepped down. Huh? This is not how US split phase power works. 240 is two legs. 120 is one leg and a neutral. All of my power hungry racks are fed three phase from the floor/ceiling where I break them out on PDUs where I can balance 240/208 v things over the X/Y, Y/Z, and Z/X phases.
|
|
#
?
Jul 15, 2020 20:07
|
|
|
Thanks Ants posted:I have 2x 32A 240v to a half cabinet in east London, but it's not really surprising that the land of data centres also has lots of power. Pour one out for Redbus
|
|
#
?
Jul 15, 2020 20:22
|
|
|
Motronic posted:Huh? This is not how US split phase power works. 240 is two legs. 120 is one leg and a neutral. Maybe I'm thinking of a different setup, but I remember us having some stupid ones that took 240, it's been years since i've been inside of a rack, and I don't plan on going back to that because it didn't pay well ($16-18/hr, because honestly remote hands is unskilled work), I'm trying to remember how the PDUs in the racks worked and drawing a blank (E: looked up a picture of the PDUs we used "HP B4G22A" and yeah it was black/white, you spread out your devices across the phases and redundant across your A and B PDUs, we also had some devices that did take power straight from the floor and we didn't open up to cable them, and it depended on the customer cabling things redundantly in a correct manner if they shipped in their own preconfigured racks.). I definitively remember taping red tape over the adjacent floor position breakers to the 3PAR to keep wandering hands from potentially throwing a breaker that shouldn't be touched and pissing off a customer mightily. I also remember when we were doing scheduled load failover testing, one of the equipment racks that was provided to us by a customer had not been properly redundantly cabled by the customer's technicians, so instead of feeding both A and B power to each module in the racks, they fed A side breaker power to both A and B redundant power supplies in the left side of the rack and B side power to the right hand side A and B power supplies. When we threw the A side master breaker on the row, half the lights on the rack winked out and it was a mixture of laughing and despair because we knew that whatever was in the rack was toast. I don't know exactly what kind of hardware was in the rack but when the call came in to open the rack and see what was what, it was pretty funny because the power routing diagram was on the inside of the door plainly, and it was completely different than what the customer's techs had wired up. orange juche fucked around with this message at 20:49 on Jul 15, 2020 |
|
#
?
Jul 15, 2020 20:27
|
|
|
Super Soaker Party! posted:Dammit it sounds like we're not gonna get a final conclusion if it basically depends on your boss' overheard gossip, but still, doesn't sound good for our buddy Mitch. We'll get a final conclusion, but right now all I have is gossip. He's fired, he's been "allowed to resign", he's still employed but missed out on another promotion. The unanticipated cost of the new license etc got him fired, the cost was in the budget for this fiscal year anyway, the cost was too much so we're moving to another platform. Right now the internal directory just lists him as a manager in IT. I remember that he was hired as staff but really wanted to take a leadership role, so he got upped to team lead then promoted to manager. Re: edit: Possibly, but given that it's been ... a lot of years since I was on that campus and I'm not one of those people who ties their identity to what college they attended... (CAL 25 STANFORD 20.) RFC2324 posted:depends on the DC, however, it sounds like he had a ticket to do physical decommissions of some servers, found the stick in a machine and proceeded to decommission both the dongle and the licensing server that may have actually cached the license so it could be verified So this is what I've put together about what happened there: * The dongle was plugged into, let's call it, LAX-UTIL-05. * Right next to that (under, possibly) was LAX-STOR-12. * Both were on the decom list; no one was quite clear about what UTIL05 actually did, but it had passed the scream test. * While actually removing the servers from the racks, he pulled the dongle out of the back of the server, went around front to read the label, read the wrong label, and thought it had been plugged into the storage server. * Chaos ensued. orange juche posted:I think warlock and their boss got cut out of the loop on the saga of Mitch. Mitch is probably done though in infosec role, everywhere, especially if the company is as big as it is implied to be. RFC2324 posted:I don't think Mitch was in infosec, I think Mitch was onsite hardware team/remote hands person Honestly, I think he's going to survive this, simply because 1) he did follow procedure, even if immediately destroying the dongle was a bit stupid, and 2) even if he hadn't been there, once we'd e-wasted/whatever both the licensing server and the dongle, we'd be in the exact same boat. He might get slapped on the wrist about the tone of the email, but I doubt that's a term. On a tangent: larchesdanrew posted:but I think we can consider the saga of larchesdanrew finally over. Congrats!
|
|
#
?
Jul 15, 2020 21:16
|
|
|
It all seems pretty normal and safe for now, but a few weeks from now he'll be sitting in a staff meeting, and someone will reach into their purse and pull out a pork chop...
|
|
#
?
Jul 15, 2020 21:45
|
|
|
Powered Descent posted:Always tip your server.  I laughed. Well done.
|
|
#
?
Jul 15, 2020 21:46
|
|
|
Bit late to the what did I break party but here you go We had 6 data cabinets that, for some reason the previous IT guy ran 2 fibres from the server room then created a very random mesh of Cat5 between these 6 cabs. I don't think we ever fully understood where the cabling went... it just worked. At least, until I knocked one of the 2 fibre tails in the server room with my butt (I assume) and broke it. This meant I knocked a number of those 6 buildings offline and we had to trace all the cables to get everyone back online My boss told his boss a rat ate the cable and I got I I that one. I think since I left, the new IT guy has had fibres put in to all the cabs. The other one I did was to do with our web proxy it was sent to sync with AD once a day and at the time, no one was allowed access to it apart from the Head of IT because omg internet access!! There was a glitch in the sync process where if you changed name of an AD user, it broke the proxy and you had to get the head of IT to go and fix it. Someone at my place got married, I changed her name. She then reported loss of internet - I think as a youngster I panicked because I couldnt reach the big boss and came up with the idea of deleting her AD account and recreating it new with her new name, that was my web proxy work around - which worked well My boss has fun restoring her exchange mailbox though 
|
|
#
?
Jul 15, 2020 23:45
|
|
|
angry armadillo posted:Someone at my place got married, I changed her name. She then reported loss of internet - I think as a youngster I panicked because I couldnt reach the big boss and came up with the idea of deleting her AD account and recreating it new with her new name, that was my web proxy work around - which worked well I'm always amazed at how something as simple and common as a name change is such a difficult IT problem. One of my co-workers put in a name change request after getting married. She managed to get divorced before they managed to change her name in her email address.
|
|
#
?
Jul 16, 2020 14:42
|
|
|
Soylent Pudding posted:I'm always amazed at how something as simple and common as a name change is such a difficult IT problem. One of my co-workers put in a name change request after getting married. She managed to get divorced before they managed to change her name in her email address. Is that really a commentary on how slow/bad IT systems are, or a commentary on how terrible the institution of marriage is? 
|
|
#
?
Jul 16, 2020 14:55
|
|
|
DelphiAegis posted:Is that really a commentary on how slow/bad IT systems are, or a commentary on how terrible the institution of marriage is? Yes.
|
|
#
?
Jul 16, 2020 15:00
|
|
|
DelphiAegis posted:Is that really a commentary on how slow/bad IT systems are, or a commentary on how terrible the institution of marriage is? Plenty of reasons to change your name that has nothing to do with marriage.
|
|
#
?
Jul 16, 2020 15:01
|
|
|
Motronic posted:(aka 365 Main in San Francisco) Yup, them. When I think back to just how loving stupid Coloserve management was back in the late 00s I should be glad that my new half-rack was all in the same "half cage". They were the DC that did a generator test with the UPSes out-of-circuit. So when they cut the mains, well the generator did start... They were not the DC that did a generator test with the mains in-circuit and blew power for SOMA and the Financial District for 5 hours.
|
|
#
?
Jul 16, 2020 15:04
|
|
|
Soylent Pudding posted:I'm always amazed at how something as simple and common as a name change is such a difficult IT problem. One of my co-workers put in a name change request after getting married. She managed to get divorced before they managed to change her name in her email address. We can’t change samaccountname because someone in their infinite wisdom a long time ago decided to use it for username in service now, so if we rename a user’s samaccountname service now sees it as an entirely new account.
|
|
#
?
Jul 16, 2020 15:09
|
|
|
|
| # ? May 25, 2024 17:32 |
|
|
Sirotan posted:Plenty of reasons to change your name that has nothing to do with marriage. Absolutely, I was just shitposting since they mentioned the person in question getting divorced.  To be fair, my wife and I still find places where they have her previous name recorded, even after we think we got them all on the last round.
|
|
#
?
Jul 16, 2020 15:13
|
|