Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Misogynist posted:


"DevOps Engineer" (quotes for all the reasons JHVH-1 has already gone into) is, in almost all cases, a T-shaped generalist who is an integration point between various technologies in use within the organization. It doesn't mean you're responsible for operations in every application and every service. Most organizations that practice DevOps have at least some degree of "you build it, you run it" product teams as opposed to centralized operations like corporate IT.

This is essentially the way we run. We're a SaaS and we try to make sure the devs have the tools and support necessary to build their project (for example, an MVC app for a new client that plugs into our larger application framework) from the ground up while integrating it into an environment as close to production as possible. So, we make sure their environment is setup right, give support on the deployment integration into the larger framework and they are responsible for making sure that their app is packaged up and has all the documentation necessary to deploy properly.

We also help give them perspective on how what they are working on fits into the greater whole. "Oh, your new app uses a 3rd party component that's incompatible with one that's already in use by an app on the same server. We'll spin up a new set of VMs for deployment in this case and segregate it from the conflicting pieces." When poo poo goes wrong during testing in our various code promotion environments, we're the ones digging into the logs and metrics on the servers to give feedback to the developers to help them solve the issue. We just had a round where we had a bunch of aggregation calcs absolutely pummeling and crashing our test mongodb server so my team worked directly with the teams that wrote the code to do the post mortem to figure out where things went south. In that case, it was a slightly underspec'd VM combined with some mongodb driver settings that would occasionally create the perfect storm of locked collections and sharp escalation of memory usage that would eventually tank the server.

Now, where we fall into the devops 'trap' that is talked about here (sometimes not having enough people) is that there's no dedicated IT staff outside of my department so we're also responsible for office desktop support as well. It's annoying and something we've talked about eventually correcting by outsourcing our desktop support, but for right now the workload isn't quite enough to keep a whole dedicated team busy so it lingers.

But, no environment is perfect. One thing that has been great about this sort of role is the sheer number of technologies you get to touch. Every once and awhile I think it would be nicer to dig deeper into something and master it, but then something new and shiney comes by. Digging out the issues remains the fun part and things break in new and interesting ways with the rate we're growing as a company and how quickly we are evolving our software.

Adbot
ADBOT LOVES YOU

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


We just get a straight 3% of whatever your gross salary is in one lump payment per year, fully vested from day one. You don't even have to contribute and you get it.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


The real perk of my employment is I pay zero for healthcare. Zip, zilch, nada. It's awesome coverage too. No deductibles (except $100 for emergency room visits). Just copays.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


0-5 years with company, 10 days
5-10 years with company, 12 days
10+ years with company, 14 days

That and 3 sick days. That's pretty much it. Official policy is no rollover, but people are allowed to do it unless it's abused.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I think Paramount and Sony need to get together and get some rehab for whatever condition causes them to churn out disappointing movies for franchises that should be money printing powerhouses.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Sickening posted:

Its also easier and more rewarding to tell people what is broken more so than fixing it with realistic goals in mind.

I think this is it, right there.

Everyone is hosed and it's much more fun to tell someone they are hosed and walk away than it is to actually try to fix things in a meaningful way.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Not free, but we've been really happy with NewRelic for application level monitoring. They have some basic server monitoring with it as well.

We use that in combination with an internal Nagios server to raise alerts to PagerDuty for off hours escalation.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I have an ancient (and isolated) Linux server running some old reporting software that has an uptime right now of 1627 days. I have to move it to a new rack soon :(. Attempts to p2v it thus far have been problematic. Looks like I'm going to be rolling the dice.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


For us, our client contracts forbid us from using public cloud infrastructure (much to my annoyance as scaling would be so much easier.)

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


evol262 posted:



Scaling is the same with private clouds as long as you have the excess hardware. If they have no room to scale at all, why are they bothering to use private infrastructure?


That's the rub, hardware procurement and installation. Takes time away from things that I would much rather be doing plus makes us more inflexible to peaky demand.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


evol262 posted:

This isn't a rub. Private cloud installations should already have the hardware procured and installed, waiting for instances.

It's more of a failing on the business side when specing out clients/projects, but we've had instances where demand for resources as doubled or quadrupled in less than a month with no prior warning. I can't double my local infrastructure with 15 days lead time.

That's where we are having trouble. Our growth rate is outstripping our ability to get hardware in place.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I had a fun time with 1099 at my first gig out of college due to an employer doing something illegal.

The company ran out of money and couldn't make payroll. We were all full time W2 employees. The CEO cut personal checks out to employees to cover their net pay. At the time, he stated that everything would be reconciled with payroll once one of the contracts cleared and the company got paid. Two weeks later when the same situation was going to happen again, a whole bunch of us said "lay us off so we can at least collect unemployment", which he did.

Tax season rolls around and I'm at my new job. I get my tax documents mailed to me and look what happens. I get a 1099-MISC for the net pay that he paid us out of pocket. It was never reconciled with payroll and squared away and now I was essentially being double taxed. Not to mention it was incredibly illegal to issue a 1099 for work done by an employee when they were full time W2. What made things even more fun is I was indeed supposed to have a 1099-MISC as I did contract work for them part time while I was looking for work after I was laid off.

So, I lined things up to file my taxes with an exception, basically disputing my tax documents and let him know I was intending to do so. A few days later I get a check in the mail more than double the difference I was going to have to pay out of pocket for taxes, SS, and medicare. So, I let the matter drop rather than draw it out since I eventually made out better in the end.

As to why I went back to work after they couldn't make payroll and laid me off, I refused to work enough hours to cut into my unemployment benefits until he agreed to pre-pay me for my hours. The situation only lasted about 20 days anyways until I got my new job lined up.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


That article is by no means limited to IT. It's sadly applicable to pretty much any sort of professional field, especially one that's salary rather than hourly.

A combination of the push to do more with fewer people on top of sheer management incompetence that flat out prevents them from effectively utilizing more resources makes this far too common in every industry.

Unfortunately, the people that really need to read it and be affected by the message are the ones least likely to be moved by it.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I only mention it because I have friends that are getting hit just as hard even though they are not IT. For example, I have a friend that does design work and she's getting handed the same types of impossible deadlines and 24/7/365 response time for issues. It's all overwhelmingly due to a lack of proper planning/project specification and lack of people to share the load.

Hell, I haven't been able to do anything with my friend that prints signs for a few weeks due to forced overtime 6 day work weeks.

It's all squeeze squeeze squeeze. Until we get some sort of overtime legislation with teeth, the American worker is going to keep getting poo poo on from top to bottom.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


There are slower weeks?

As far as I can tell, weeks are just on a feedback loop of acceleration.

I usually eat at my desk and then wander around outside for close to an hour. But I haven't been able to do that in months due to the weather.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I have to say that Dell's lifecycle management controller is pretty magical.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Inspector_666 posted:

Speaking of, does anybody know exactly how the NIC config works with the Lifecycle controller? I set up a server the other day and had issues with it until we set it to just use DHCP. Does it create subinterface on the selected NIC?

This just for my own edification.

Not so much a subinterface, think of it as dual booting into another OS where the Lifecycle controller is the other OS. So, you could set the IP address for the lifecycle controller to be the same as your OS if you want. It isn't active when you aren't in the lifecycle controller.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Inspector_666 posted:

Weird, when I assigned the Lifecycle Controller the same IP as the OS, it couldn't communicate with the Dell FTP server, and as soon as we just set it to DHCP it worked fine.

I guess it doesn't matter since we'll almost never be pulling a lease with it.

Yeah, that is odd. I actually just setup an R630 last week and I statically assigned the IP for the lifecycle controller and it didn't have any issue. I ended up using the same IP for the OS after I set it up.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


It's one of those things that's hard to look at in absolute terms.

Being a small company and the head of my department, It's pretty much understood that if poo poo hits the fan in any super serious way, I'm likely to be called for it because of the whole "expert subject matter thing."

At the same time, I go out of my way to architect things in a way that this can in no way shape or form happen or if it does happen it can be resolved by me providing minimal guidance for someone else to resolve.

So, in a lot of ways I'm master of my own fate in that regard. If I come up with a lovely plan for something, don't disseminate knowledge well enough, or it have poor automation and redundancy, then it serves me right for having my free time interrupted when something goes bump.

Where it falls down a bit is where things intersect with issues I don't have complete control over. So, that's where the challenge is.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I won't deny the engineering is nice, but I really can't see why for $9k.

Hell, we just bought a new database server that was scarcely more than that and it was loaded with 750gb of ram and two 18 core Xeons.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


As a manager, I'm only permitted to give out between 0%-5% and 5% is a real outlier that I have to prove the person to be Jesus incarnate to get approved. Anything more than that basically requires a title change.

It's not impossible. I myself got two >10% raises back to back in the first few years after I started. That was because I was originally brought in for deployment only, then moved into an infrastructure role, to heading up both internal and production infrastructure after someone left. The past few years though I"ve only gotten 4% because that basically all the structure currently allows.

In the grand scheme of things, I'm making about 50% more than I was 6 years ago so there has been growth there. I just also know I probably won't see another significant jump unless I jump ship somewhere else. I also know I'm in the upper bracket salary-wise for what I do in the area, so I know I'll have to move if I want significantly more.

I do live in one of the wealthier zip codes in the region and my single income household is 26% above the median income for my zip code so I don't really feel I'm doing all that badly.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


My favorite is the call saying "I just wanted to confirm that you received the whitepaper I forwarded over. Oh you didn't? I can certainly resend that, what's your email for confirmation?"

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Internet Explorer posted:

I have to ask, why do you want to do this over a lovely WAN link? Is there an fat application on both sides that needs access to the data? You are almost certainly better served by just using RDS / XenApp.

One reason I could see is a transactionally consistent off-site backup of the database. I know I'm planning on adding availability groups to my current SQL cluster for that reason. Local SQL cluster for redundancy, with a standalone remote database as part of the group. We take transaction log backups every 10 minutes, but even a loss of 10 minutes worth of transactions would be a major shitstorm with our clients.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I would say it's crazy. Legal doesn't let most places do anything other than confirm or deny that the person was employed there for the period stated.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I wouldn't do it.

It's a black hole that you'll have trouble escaping from. The issue is, how do you define your free time? You get home from a stressful day of real job and they have some sort of emergency that they want you for. Now you have to juggle between feeling guilty for leaving them hanging or adding to your stress level by not having time to unwind. Are they going to pay you under the table? If not, you'll have the whole bullshit of working out a 1099 when tax season rolls around. If you aren't making enough from there for it to be taxable, then it's not worth your time and effort.

If it's a employer that you have a good relationship with, I can see giving a helping hand for a period of time to maintain relationships. But absolutely have an end date confirmed ahead of time when you will no longer be providing support. Otherwise, they just have incentive not to replace your position if you can keep them limping along.

There's no such thing as 'casual' work if you are providing any sort of support that's critical to keeping a business running.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Someone needs to cut all telephone lines to Comodo's sales dept. Every as soon as a cert goes to 89 days 59min and 59 seconds from expiration, they call like 3 times a day posing as the company you bought the cert from trying to get you to renew with them. Pain in the rear end.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


DrBouvenstein posted:

Because we use such an outdated system for paging, it took me and another tech almost an hour to set up our system to send me a text if I get a page this weekend instead of a page, because I'll be out of the range of our pagers.

It involved setting up the software to send an email to my phone number, so then AT&T sends it as a text. But even after that, there were like three schedules and two action routines in the drat software that had to be edited to include that.

We're set to demo a smartphone paging system next month, and I can't wait. Right now everything is one way, so even if I get a page I know is pointless, I have to get up, VPN in, remote to the server the software runs on, and acknowledge the drat page.

In theory, I should be able to use our intranet page for it and not have to remote to the server, but that page flakes out and only works 50% of the time.

With the smartphone-based system we can acknowledge the alert in the app, in theory, so I can just hit a button while still lying in bed.

Right now, we're using PagerDuty. It's been working out well for us.

PagerDuty+NewRelic is pretty much magic.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Internet Explorer posted:

If you have more than 8 VMs (eh.. that link posted above says 5, maybe pricing has changed since I did the math) per host and can vMotion between hosts Datacenter makes sense from a financial standpoint

Yeah, for the host. That doesn't mean your guests need to run the same edition.

Back in 2008/2008 R2 there was a certain logic to running Datacenter as a guest because Datacenter had hot-add support and Standard did not. So, if your VM ran short on CPU or RAM, you could increase it without having to bounce the box.

Now though, there's no advantage to running Datacenter over Standard in the guest. It doesn't hurt anything either since they are technically identical. The only reason why I would worry about someone insisting on running Datacenter as a guest for 2012 R2 is that it's a sign they don't understand the product they are running.

We have datacenter licenses for our 6 hosts to cover the unlimited virtualization rights, but we only run Standard in guest. There's not much point in running anything else.

bull3964 fucked around with this message at 03:38 on Apr 11, 2015

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Internet Explorer posted:

If you are licensed for Datacenter on the host why would you run Standard in a VM...? So you are telling me you buy Datacenter licenses, install something other than Hyper-V, and then install Standard? That makes no sense.

Why not? We're ESXi. We have datacenter licenses to cover all our hosts. The only difference between Datacenter and Standard is what they report as under system properties. It makes gently caress all difference which edition you run.

At one point in time we DID have some hyper-v hosts alongside the ESXi. It was nice from an organizational standpoint when looking at scans and whatnot when sorted by OS to use the edition to quickly identify which were guests and which were hosts.


Edit:

adorai posted:

Until the day you have to move your existing VM to another host, that is not licensed for datacenter, and you are hosed into doing so.

I always install standard or core unless I have a compelling reason to install something else.

Yeah, that too. It's conceivable that you might want to move a VM to a standard edition licensed host and you wouldn't be able to do so if your edition was too high.

bull3964 fucked around with this message at 03:48 on Apr 11, 2015

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Internet Explorer posted:

Why would you have some hosts licensed with Standard and some licensed with Datacenter? So you have something else to worry about at night?

Because it's conceivable that you may want to split off a high load VM on to dedicated standalone hardware at some point in the future or you may want to clone a limited environment for test purposes that you wouldn't want affecting your production cluster.

I'm actually IN that situation right now. Our environment's backup software is virtualized and the load of the software is starting to affect performance of the host and lower the speed of the backups. I'm not going to put a dedicated host in the cluster with several grand of Datacenter Windows licensing to run one dumb VM when I already have a few standard edition licenses knocking around and hardware to run it on. I can pull out that one VM, stick it on dedicated hardware, and obtain performance isolation without having to rebuild anything.

bull3964 fucked around with this message at 04:17 on Apr 11, 2015

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


We had long ago had to disable Kernel cache due to another obscure bug, so I guess we already implemented the workaround! Yay!

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


We say we're a windows shop since our stack for our main product is IIS/.NET/SQL, but the reality is, we use whatever is best for the job at hand. We have mongodb servers running on CentOS, as does our memcache servers. Our support system also runs on Apache/PHP/MySQL. We even have a group of webservers for 'lite' content (read simple sites that we don't need full devs working on) that are running Apache/PHP.

If a piece of software we want to use runs better and is better supported on linux than windows, then we run it on linux. Vice versa as well.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


skipdogg posted:

I'm never buying another SuperMicro product again. The hassle is not worth the savings at this point.

Reached that point about 3 years ago.

I haven't had a single one that hasn't puked on storage one way or another.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Vulture Culture posted:



SuperMicro is a bad fit for use cases where your response to failure is something more complex than "throw it out and get a new one."

It's also a bad fit for anyone who expects their hardware vendor to do more than 5 minutes of validation of the hardware configuration.

I've seen Supermicro boxes ship out with hard drives that aren't on the HCL for the raid card (or firmware of the raid card) installed. That poo poo just shouldn't happen.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


One flaw I can see with that study is the reason they gave for them not having effect.

quote:

One reason for the apparent lack of effect may be that primary care physicians already identify and intervene when they suspect a patient to be at high risk of developing disease when they see them for other reasons.

That's sort of a chicken and a egg thing isn't it? If you aren't going to the doctor for other reasons, how is something going to get identified and treated? In the past 20 years or so, checkups would be the only compelling reason for me to go to the doctor. There hasn't been a need otherwise.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


I think it's pretty obvious at this point that if you didn't show up, he's going to leave you a batshit crazy email which would be perfectly good enough proof to anyone involved that it was an abusive environment.

You also seem like you would have a pretty good case for short term disability benefits if you can't get unemployment. I think I would start out with sick days, seek professional help during that time, and then get on short term disability for depression issues stemming from work load. While you are sorting that out, get a new job and never look back.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Housing talk makes me appreciate Pittsburgh so much. I bought a decent (though, older) 3 bedroom house for $130k in one of the wealthier suburbs in the area. I also ended up buying it when I was making about 30% less than I currently am which has worked out pretty well in retrospect. I could buy a loving mansion around here for $250k-$280k, but it's much more fun having a ton of disposable income.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Zero VGS posted:



If I can't set the static IP of a network adapter in less than 6 clicks I get pissed off pretty fast. Win 8/10 both keep developing new ways to hide it from me.

You do realize that in 8.1/2012 R2 you can just right click the start button and go directly to Network Connections right?

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Potato Alley posted:

Also I rather disagree with the processing power remark - VMs are noticeably faster on newer hardware.

A 2009 server is going to have Nehalem microarch in it. Sandy bridge is going to be about 20% faster clock for clock to say nothing of the other platform improvements such as memory speed.

We're flat out RECYCLING poo poo of that vintage. It's junk compared to modern.

Adbot
ADBOT LOVES YOU

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


iDRAC is dreamy.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply