Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
MagnumOpus
Dec 7, 2006

I still have not had a chance to actually run Chaos Monkey in a professional environment. Got real close to having a 100% crash-only upper stack, but no one ever wanted to actually pull the trigger.

Adbot
ADBOT LOVES YOU

ghostinmyshell
Sep 17, 2004



I am very particular about biscuits, I'll have you know.
I was asked to setup patch management for our centos servers. Must be free obviously.

I looked at Spacewalk since that was the obvious choice, but then I read that the latest version Satellite was was based off a different framework and that version of spacewalk isn't out yet. One person told me to look at Katello but I'm getting a bad vibe from it especially when they recommend turning off SELinux. Anyone have experiences with this?

Proud Christian Mom
Dec 20, 2006
READING COMPREHENSION IS HARD
i love it when the discussion is only viewed through the narrow lens of fortune 500

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug

Misogynist posted:

Chaos Monkey is great, but Chaos Gorilla is even better. Chaos Gorilla will, once every couple of months, pick an entire AWS region worth of production infrastructure and just pull the plug to see what happens.
I think programs like this is one of the best examples between the "new" and "old" way. Imagine trying to convince your boss to run a program that takes down production instances deliberately. There's NO way I could have ever gotten that through management, in any company I have ever worked at.

tl/dr on satellite server: old version / spacewalk bad, new one goodish.

evol262
Nov 30, 2010
#!/usr/bin/perl

ghostinmyshell posted:

I was asked to setup patch management for our centos servers. Must be free obviously.

I looked at Spacewalk since that was the obvious choice, but then I read that the latest version Satellite was was based off a different framework and that version of spacewalk isn't out yet. One person told me to look at Katello but I'm getting a bad vibe from it especially when they recommend turning off SELinux. Anyone have experiences with this?

Satellite 6 is a combination of Katello (which includes Foreman), Pulp, and Candlepin.

Basically a combination of this gets you Satellite 6. It should work wine with SElinux.

But bear in mind that Satellite 5/6 (and hence Spacewalk/Katello+Pulp+Candlepin) are lifecycle management, not patch management, and it can take a significant amount of effort compared to just pushing changes into a qa repo then promoting them once you've smoke tested it or qa says it's fine. They really work best when you have specific package sets that mean "this is a prod server running this particular version of our software" or "this is a dev server", so you can easily deploy the whole shebang over PXE or whatever.

How many servers do you have?

SubjectVerbObject
Jul 27, 2009

Misogynist posted:

Future shock is here and job roles are getting more specialized as automation takes over the industry, and uptime of even cheap ISPs is more than good enough to facilitate moving business-critical services offsite. Generalist roles will be increasingly shifted off to managed service providers, consultancies, and pre-sales engineering gigs for the major players selling on-premises equipment.

I'm surprised that you've barely heard of AWS. Amazon is by far the world's largest hosting provider. Some metrics put them at double the size of runner-up OVH. Many players with tens or hundreds of thousands of server instances, like Netflix, run most of their infrastructure on AWS.



And future shock is coming to business telecom. All the endpoints are getting much smarter and more flexible, there are a lot of options for small business that don't involve a spending a fortune on hardware, and what's left is big iron type systems that process 500k calls an hour for large companies. A lot of the players are former hardware companies that desperately want to stop competing in commoditized markets and sell software and services, hence the drive to the cloud. Unfortunately like the example of Dropbox for business, I think the cloud is going to provide an alternative to 20 year old software with high licensing fees.

And I am a telecom geek who tries not to think too much about software outside of work, so unfortunately I haven't kept up.

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug

ghostinmyshell posted:

One person told me to look at Katello but I'm getting a bad vibe from it especially when they recommend turning off SELinux.
Good luck finding a commercial software product OR OSS product that doesn't recommend disabling SELinux...

evol262
Nov 30, 2010
#!/usr/bin/perl

Bhodi posted:

Good luck finding a commercial software product OR OSS product that doesn't recommend disabling SELinux...

Every single Red Hat project upstream or downstream should either run with SElinux enabled or have an outstanding bug about it not working in enforcing mode. We care, or somebody at Red Hat cares.

If Katello doesn't work with SElinux enabled, file a bug. Seriously.

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
We've talked about this man... you guys make heroic efforts to make packages work out of the box, by default, but any customization requires commensurate selinux configuration... and didn't you say you polled stats to see what percentage was using enforcing and it was like, low single digits?

I'm pretty sure some of the optional administration / housekeeping stuff doesn't work by default under SELinux, but that's just a guess from past experience.

evol262
Nov 30, 2010
#!/usr/bin/perl

Bhodi posted:

We've talked about this man... you guys make heroic efforts to make packages work out of the box, by default, but any customization requires commensurate selinux configuration... and didn't you say you polled stats to see what percentage was using enforcing and it was like, low single digits?

I'm pretty sure some of the optional administration / housekeeping stuff doesn't work by default under SELinux, but that's just a guess from past experience.

It is low single digits, yeah. And a lot of customization requires SElinux rules. But I mean there should not be Red Hat project which have "disable SElinux" in the documentation, and normal usage of software (clicking around and configuring stuff in the Katello web UI, say) should not result in denials. If it does, it's a bug.

If some other vendor's software needs it disabled or your developers want to run CGI scripts as root or you enabled SElinux MLS mode or changed the context on something or "cp -arZ /somedir /path/to/somewhere/else" or whatever, fine. But it really, really is a bug if installation or normal usage of Katello or other products breaks because of SElinux.

E: if any of the optional bits that come with it don't work with SElinux enforcing, that's also a bug.

ghostinmyshell
Sep 17, 2004



I am very particular about biscuits, I'll have you know.
We have about 20 servers, but for some compliance coming down the road I don't know which one yet, requires some kind of patch management/lifecycle system.

http://www.katello.org/docs/2.1/upgrade/index.html is where it describes running SELinux in Permissive. I need to find out when 2.2 is coming out or just deal with it for now. Depends on the hard timeline I'm given.

Docjowles
Apr 9, 2009

edit: thanks for the mention of Katello, hadn't seen that before. Looks interesting. Although now that I dig into it, everything I'm interested in is marked "TODO" in the docs lol.

Misogynist posted:

You're living in the Satya Nadella era. I'm sure SCCM is going to get some really nice Azure management capabilities over the next several years. (Learn. Azure.)

In general, if you're learning configuration management on the Windows side, which tool you learn is going to be determined by where you want to use it. I don't see SCCM being unseated as a killer piece of software for managing changes across thousands of desktops. For servers, it looks like PowerShell DSC is a more forward-looking tool.

Chef is going all-in on supporting Windows via DSC. So learning Chef is a decent hedge for a skill that's already very valuable on Linux, and gaining a foothold in Windows land.

Zaepho posted:

I sure hope SCCM improves from the Server/Application perspective. It's great for base OS deployment, Patching, some Config monitoring and so forth but it could really use a Distributed application concept would be much appreciated. It helps that the entire suite is licensed together so it is much easier to advocate SCCM on servers if they've already got SCOM on the servers.

Jeffrey Snover (the guy who invented PowerShell) has been making the podcast rounds recently. At one point he straight up said "SCCM and friends are great for managing enterprise desktops. They are crap for managing servers. PowerShell and DSC are where Microsoft is going in that regard". So if you care about that side of things, I'd make sure you're paying attention to DSC as it matures.

Docjowles fucked around with this message at 23:35 on Feb 19, 2015

Zaepho
Oct 31, 2013

Docjowles posted:

Chef is going all-in on supporting Windows via DSC. So learning Chef is a decent hedge for a skill that's already very valuable on Linux, and gaining a foothold in Windows land.


Jeffrey Snover (the guy who invented PowerShell) has been making the podcast rounds recently. At one point he straight up said "SCCM and friends are great for managing enterprise desktops. They are crap for managing servers. PowerShell and DSC are where Microsoft is going in that regard". So if you care about that side of things, I'd make sure you're paying attention to DSC as it matures.

Been keeping an eye on DSC but need to actually make some time to play with it for something useful. Maybe some DSC policies for at least various System Center Pre-Requisites i could probably convince the boss to hook up a little "bench" time for that.

beepsandboops
Jan 28, 2014

Docjowles posted:

Jeffrey Snover (the guy who invented PowerShell) has been making the podcast rounds recently. At one point he straight up said "SCCM and friends are great for managing enterprise desktops. They are crap for managing servers. PowerShell and DSC are where Microsoft is going in that regard". So if you care about that side of things, I'd make sure you're paying attention to DSC as it matures.
What podcasts are you listening to? I'd like to hear more from Snover

psydude
Apr 1, 2008

This is kind of a silly question, but we could probably use one to break up all of the serious discussion. My boss has been out of town all week and won't be back until Monday, so I haven't had the opportunity to hand in my resignation. I'm wondering if I should just go ahead and formally resign to his boss (our PM) tomorrow and call him, or wait until Monday (which would technically be inside the two weeks) so I can do it in person.

Proteus Jones
Feb 28, 2013



psydude posted:

This is kind of a silly question, but we could probably use one to break up all of the serious discussion. My boss has been out of town all week and won't be back until Monday, so I haven't had the opportunity to hand in my resignation. I'm wondering if I should just go ahead and formally resign to his boss (our PM) tomorrow and call him, or wait until Monday (which would technically be inside the two weeks) so I can do it in person.

It depends on what your relationship is.

Personally, I'd wait until Monday and tell him then and the last day you will be available (don't be surprised if they tell you "no, today is the day" and have you leave then. I've seen it). Two weeks is simply courtesy. If you're a M-F it would be that M-F then next M-F and then adios.

mewse
May 2, 2006

psydude posted:

This is kind of a silly question, but we could probably use one to break up all of the serious discussion. My boss has been out of town all week and won't be back until Monday, so I haven't had the opportunity to hand in my resignation. I'm wondering if I should just go ahead and formally resign to his boss (our PM) tomorrow and call him, or wait until Monday (which would technically be inside the two weeks) so I can do it in person.

You could tell him on the phone and then hand your written resignation to the other guy. The letter is the formality.

Wizard of the Deep
Sep 25, 2005

Another productive workday

psydude posted:

This is kind of a silly question, but we could probably use one to break up all of the serious discussion. My boss has been out of town all week and won't be back until Monday, so I haven't had the opportunity to hand in my resignation. I'm wondering if I should just go ahead and formally resign to his boss (our PM) tomorrow and call him, or wait until Monday (which would technically be inside the two weeks) so I can do it in person.

I'd save it until Monday morning too. Don't ruin his weekend/end of his vacation. Not first thing Monday, but before lunch definitely.

psydude
Apr 1, 2008

flosofl posted:

It depends on what your relationship is.

Personally, I'd wait until Monday and tell him then and the last day you will be available (don't be surprised if they tell you "no, today is the day" and have you leave then. I've seen it). Two weeks is simply courtesy. If you're a M-F it would be that M-F then next M-F and then adios.

I like and respect him, so I'll probably do that. I'm not worried about them giving me the boot, because they can still bill for me during the remaining two weeks and I've also been working on several projects that need to be handed over.

Sacred Cow
Aug 13, 2007

Docjowles posted:

Chef is going all-in on supporting Windows via DSC. So learning Chef is a decent hedge for a skill that's already very valuable on Linux, and gaining a foothold in Windows land.


Jeffrey Snover (the guy who invented PowerShell) has been making the podcast rounds recently. At one point he straight up said "SCCM and friends are great for managing enterprise desktops. They are crap for managing servers. PowerShell and DSC are where Microsoft is going in that regard". So if you care about that side of things, I'd make sure you're paying attention to DSC as it matures.

I've been following some blogs that have been picking apart all the DSC stuff showing up in PS v5 and it feels like this is all just going to end up being an extension of SCCM similar to MDT and OSD. I would not be at all surprised if it shows up built into the Compliance module of the next iteration of CM next to Baseline Items.

Docjowles
Apr 9, 2009

beepsandboops posted:

What podcasts are you listening to? I'd like to hear more from Snover

Pretty sure it was the most recent Arrested DevOps. But I've heard him pop up on several that I listen to, like RunAs Radio and DevOps Cafe, too.

Chickenwalker
Apr 21, 2011

by FactsAreUseless
.

Chickenwalker fucked around with this message at 03:00 on Mar 1, 2019

Japanese Dating Sim
Nov 12, 2003

hehe
Lipstick Apathy
Sometimes reading this thread makes me feel so overwhelmed. I'm at the point in my career where I'm just starting to fiddle with scripting/Powershell etc. and wondering how I'll ever get to a level where I can understand most of this. :/

Then again, all of you people have to retire some day and it's not like anyone's jumping into IT behind me with their CCIEs/MCSE's ready to go. At least it should be interesting...

evol262
Nov 30, 2010
#!/usr/bin/perl

Japanese Dating Sim posted:

Sometimes reading this thread makes me feel so overwhelmed. I'm at the point in my career where I'm just starting to fiddle with scripting/Powershell etc. and wondering how I'll ever get to a level where I can understand most of this. :/

Then again, all of you people have to retire some day and it's not like anyone's jumping into IT behind me with their CCIEs/MCSE's ready to go. At least it should be interesting...

We're probably not that much older than you. New technologies aren't that different from old ones, and getting the basics down helps you learn new stuff exponentially faster since you can correlate it to things you know already

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
Nah, it's just specialist knowledge. When I wander into one of the programming threads I am mostly lost despite half my job right now being to squirt out ruby code.

I guess there's some interest in all this cloud stuff and I feel like we kind of poo poo this thread up today, if tomorrow's slow at work I'm going to start a cloud thread. Join me over there for some effortposts / angerposts?

Zaepho
Oct 31, 2013

Bhodi posted:

if tomorrow's slow at work I'm going to start a cloud thread. Join me over there for some effortposts / angerposts?

I can probably promise a little of both. I do the Microsoft version of "private cloud" so there's plenty of opportunity to vent anger with much effort.

Japanese Dating Sim posted:

Sometimes reading this thread makes me feel so overwhelmed. I'm at the point in my career where I'm just starting to fiddle with scripting/Powershell etc. and wondering how I'll ever get to a level where I can understand most of this. :/
As was said. Learn the basics and build on that. and for fucks sake learn to properly troubleshoot! When presented with a problem ask some critical thinking questions that allow you to rule things out as issues and learn the stack involved so you can work your way Up or Down the stack to find the problem. That right there is the essence of 99% of troubleshooting. I would put forth as well that 99% of IT is troubleshooting in one form or another.

evol262
Nov 30, 2010
#!/usr/bin/perl

Bhodi posted:

Nah, it's just specialist knowledge. When I wander into one of the programming threads I am mostly lost despite half my job right now being to squirt out ruby code.

I guess there's some interest in all this cloud stuff and I feel like we kind of poo poo this thread up today, if tomorrow's slow at work I'm going to start a cloud thread. Join me over there for some effortposts / angerposts?

I mean, I mostly write Python and Go these days, though I guess I get to write JS now, too. And I find the programming threads to be really niche stuff. A lot of the python optimization is above and beyond what I know or care about.

But fortunately, admin/engineering is more general, at least in the general threads, until we delve into minutiae in the SAN/Network/Linux/Enterprise Windows threads.

I'll definitely watch for the cloud thread. Hopefully it stays more active than the config management thread.

Dr. Arbitrary
Mar 15, 2006

Bleak Gremlin

evol262 posted:

Hopefully it stays more active than the config management thread.

Where's that one? I'm playing around with Powershell DSC and I'd love to know more (also the other config management tools)

Gucci Loafers
May 20, 2006

Ask yourself, do you really want to talk to pair of really nice gaudy shoes?


Japanese Dating Sim posted:

Sometimes reading this thread makes me feel so overwhelmed. I'm at the point in my career where I'm just starting to fiddle with scripting/Powershell etc. and wondering how I'll ever get to a level where I can understand most of this. :/

Then again, all of you people have to retire some day and it's not like anyone's jumping into IT behind me with their CCIEs/MCSE's ready to go. At least it should be interesting...

Heh,

It's obvious the game is changing but then again Midrange and Mainframe computing is still around.

socialsecurity
Aug 30, 2003

Yeah I read about how cloud is taking over but then I live in a major city and most of my clients struggle to find a 5 megabit upstream so I've got some time to learn.

evol262
Nov 30, 2010
#!/usr/bin/perl

Dr. Arbitrary posted:

Where's that one? I'm playing around with Powershell DSC and I'd love to know more (also the other config management tools)

I think it may actually be an attempt at a puppet megathread. I'll find it...

Griffon
May 14, 2003

ghostinmyshell posted:

I was asked to setup patch management for our centos servers. Must be free obviously.

I looked at Spacewalk since that was the obvious choice, but then I read that the latest version Satellite was was based off a different framework and that version of spacewalk isn't out yet. One person told me to look at Katello but I'm getting a bad vibe from it especially when they recommend turning off SELinux. Anyone have experiences with this?

I feel that it's not mature enough to migrate to it right now after trying it out. Spacewalk is easy to use and still being updated.

Here's the best setup guide I've found for it: http://htfdidt.blogspot.ca/2013/12/spacewalk-20-setup-on-centos-6.html

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
If you go down the spacewalk route, you will regret it for ever and ever.

Griffon
May 14, 2003

Okay? Go on...

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
I've repressed most of it, and maybe some of this is fixed since I haven't been responsible for it for about 4 years, but when I had to deal with the enterprise version it in it's default installation from redhat (satellite server, embedded oracle database) I found it very crashy. The internal message handler was junk, the logging was full of endless unreadable tracebacks, it stores all RPMs in some arcane multi-level layout with some sort of database identifier as the directory name and it renames all the rpms to md5sums or something. To communicate, it uses a custom yum handler instead of the tried and true Packages.gz metadata format.

There's more but I've forgotten, if you dig in the previous iterations of this thread, you can find my posts where I bitch about it. Back then, even trying to figure out how to update the ssl certs on the box was not well documented, though I assume documentation has caught up for things like that.

evol262
Nov 30, 2010
#!/usr/bin/perl

Bhodi posted:

I've repressed most of it, and maybe some of this is fixed since I haven't been responsible for it for about 4 years, but when I had to deal with the enterprise version it in it's default installation from redhat (satellite server, embedded oracle database) I found it very crashy. The internal message handler was junk, the logging was full of endless unreadable tracebacks, it stores all RPMs in some arcane multi-level layout with some sort of database identifier as the directory name and it renames all the rpms to md5sums or something. To communicate, it uses a custom yum handler instead of the tried and true Packages.gz metadata format.

There's more but I've forgotten, if you dig in the previous iterations of this thread, you can find my posts where I bitch about it. Back then, even trying to figure out how to update the ssl certs on the box was not well documented, though I assume documentation has caught up for things like that.

Satellite5/Spacewalk is dead, in the "we'll support this for the next eon but nothing really new is coming" long term support way. You should use katello instead now anyway, but...

Spacewalk supports postgres, at least. You don't need to use Oracle. And I think you've been able to use an external database since 2013 or something even downstream.

Satellite was a public offering of the internal RHN bits because some very large customers asked for it, not because the code was great or in good shape. Nobody likes it.

It actually doesn't use a custom yum handler in any real way. It uses the same backend as rhn-client and as up2date used to use, because, again, it's RHN running on your infrastructure (and Satellite was released with support for RHEL4, which didn't have yum anyway). It presents entitlements, not a yum repository. The "tried and true Packages.gz metadata format" was not in use when Satellite/RHN Classic was developed.

The "arcane multi-level layout..." is to avoid wasting gobs of disk space if the same packages are included in multiple channels.

None of these are great design decisions in 2015, but nobody knew any of that in 2000 when they started doing it.

Spacewalk/Satellite have the ability to present traditional yum repos if you want them for unmanaged clients which can't attach to an activation stream and just want plain repositories.

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
You really don't have to white knight a product I used 4 years ago when someone asked why I thought it sucked, I'm never going to use it again and you don't even sell it anymore. But he asked about spacewalk, and that was spacewalk for me.

evol262
Nov 30, 2010
#!/usr/bin/perl

Bhodi posted:

You really don't have to white knight a product I used 4 years ago when someone asked why I thought it sucked, I'm never going to use it again and you don't even sell it anymore. But he asked about spacewalk, and that was spacewalk for me.

Satellite 5 still gets sold. It's :homebrew:

But giving some background on your complaints and the reasons behind them isn't "white knight"-ing anything. I'm not recommending Satellite/Spacewalk either. But when somebody comes by and says "oh my god, look at all this weird/stupid poo poo something's doing", a little explanation as to why it's doing it goes a long way, even if it's going towards explaining to somebody why they'd regret using it, because knowing that it's a 15 year old codebase designed to run before the package management utility we're deprecating now was more than a twinkle in the eye of some weird people running Linux on PPC Macs is a better reason to avoid it than "it renames RPMs to md5sums or something" (which the average user wouldn't see anyway, since you should be managing it from the web UI, which doesn't do any renaming at all).

Erwin
Feb 17, 2006

evol262 posted:

I think it may actually be an attempt at a puppet megathread. I'll find it...

It's here: http://forums.somethingawful.com/showthread.php?threadid=3654103

Considering the OP of that Puppet thread is a big warning not to use Puppet, I think it makes more sense to make a general config management thread. I'd really like one since I'm trying to find a good fit for my environment between Chef and DSC. I can make the thread if no one else wants to, but the OP will be mostly pulled from marketing info since I'm not very familiar yet with all the offerings.

Adbot
ADBOT LOVES YOU

ghostinmyshell
Sep 17, 2004



I am very particular about biscuits, I'll have you know.

Erwin posted:

It's here: http://forums.somethingawful.com/showthread.php?threadid=3654103

Considering the OP of that Puppet thread is a big warning not to use Puppet, I think it makes more sense to make a general config management thread. I'd really like one since I'm trying to find a good fit for my environment between Chef and DSC. I can make the thread if no one else wants to, but the OP will be mostly pulled from marketing info since I'm not very familiar yet with all the offerings.

I would really like that better and maybe address which of these products named after food groups are somewhat stable and actually useful.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply