Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
The Fool
Oct 16, 2003


Vulture Culture posted:

I'm getting really, seriously twitchy at this particular use of the word "DevOps".

These days it's just a catchphrase used to describe a style of engineering that combines developer and operations type roles.

There are associated best practices and technologies. Like Orchestration, Containerization, CI/CD, etc.

There are also plenty of shops that describe themselves as devops and the reality is that they fired their operations team and make their devs reboot servers.

So really, his use of the word isn't that out of bounds, but he should have been asking about using small scale orchestration.

Adbot
ADBOT LOVES YOU

Warbird
May 23, 2012

America's Favorite Dumbass

It’s a real hoot to interview for a DevOps position only to find out they want something completely different than what you have. It’s getting to the point of “Engineer” now. Everybody’s definition is wildly different.

poemdexter
Feb 18, 2005

Hooray Indie Games!

College Slice

Warbird posted:

It’s a real hoot to interview for a DevOps position only to find out they want something completely different than what you have. It’s getting to the point of “Engineer” now. Everybody’s definition is wildly different.

Current job is a DevOps position and it's my first time not in a pure software developer role. It's always been my opinion that all software devs should understand how their software is built and deployed in production. DevOps shouldn't be a black box.

Warbird
May 23, 2012

America's Favorite Dumbass

DevOps at my current workplace is pretty much just Ops. I’m mostly doing automation and being the middle man for people that need environments for the project. It’s a bit of a pain as most places in the area more closely associate it with AWS/Azure and/or CI. I can read up on it all day, but I’m somewhat concerned about my finding something else when/if this contract ends.

JehovahsWetness
Dec 9, 2005

bang that shit retarded
My title is "Cloud DevOps Engineer", which is just gobbleygook.

I basically do infrastructure-level automation / codification / etc and help other groups move projects to The Butt without doing a lift-and-shift. The crappy part is the teams that need my groups help to move and re-architect are the teams that are so unskilled they shouldn't be involved / doing it at all. And most of that stuff is vendor trash. There's really only like 2 "real" software projects our group is looped into the cycle of.

Scikar
Nov 20, 2005

5? Seriously?

Rocko Bonaparte posted:

The big thing here is somebody would want to add a step towards a regression, but we don't want unrelated commits that don't even have the matching code to fail. I expect the incompatible code to get through because it has already happened two of three times we did this. We were about to start adding more steps more often, and I don't want it to become a game of us having to get together and turn our keys at the same time.

You absolutely don't want to be trying to synchronise the manual parts of the process in order to make the automatic parts work properly. I think TeamCity leads you into a trap here, because it has so many options for build runner steps that it's far too easy to add everything as a new step, but since the TeamCity config doesn't mirror the branches on your source control you end up breaking all of the branches every time you make a config change.

The only way to keep your sanity is to keep the steps in TeamCity as high level as you possibly can, i.e. 1. Build Project (by running a build script in the same repo as the code, so your build config always matches the code it's building no matter what), 2. Run Regression. Here you have 2 options, most likely you keep the regression script in the same repo as well together with the build script (so again it's always correct for any given build regardless of how old the branch is or what has been changed). If you have developers who like to make lots of tiny commits that only ever change one thing and push them continuously then you might have to train them to squash the commits or put up with breaking the build every time they don't update the regression script in lockstep.

The other way is to keep the regression suite in a different repo, in which case your code repo needs to have some config file which defines the version of the build script that it depends on. That might just be a case of using the commit hash from the regression repo, or you can get more advanced and use nuget packages or whatever depending on how complicated your regression suite is.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
There is a great need for people that know what they’re doing in most organizations because most places over time just plain rot on the operations side because it’s typically a cost center and leads to horrible attrition rates. One place I’m familiar with wound up trying to aggressively recruit kids just out of school to replace senior ops engineers that were leaving by the droves and even the contractors were not biting.

smackfu
Jun 7, 2004

I feel like DevOps just means we wait for the OpenShift admins to fix our broken pods instead of the sysadmins to fix our servers. Where’s my magical future dammit?

Warbird
May 23, 2012

America's Favorite Dumbass

VMWare has poo poo the bed at the workplace and we've been unable to spin up Windows VMs for close to a month now. Current lead times now that it is kinda working are longer that it would be to get a physical server in place. Yaaaaaaaay.

freeasinbeer
Mar 26, 2015

by Fluffdaddy

smackfu posted:

I feel like DevOps just means we wait for the OpenShift admins to fix our broken pods instead of the sysadmins to fix our servers. Where’s my magical future dammit?

If your “devops” use openshift, they are the trash.


Also with k8s I’d make you all fix your own poo poo. poo poo I don’t even go on call anymore.

Edit: the sysadmin thread had me in a combative mood. I think that there are legit draw backs to openshift that makes it less optimal then just using regular kubernetes. They subvert more then a few workflows and endpoints behind what I think is worse abstractions.

freeasinbeer fucked around with this message at 02:00 on Feb 22, 2018

Internet Explorer
Jun 1, 2005





Warbird posted:

VMWare has poo poo the bed at the workplace and we've been unable to spin up Windows VMs for close to a month now. Current lead times now that it is kinda working are longer that it would be to get a physical server in place. Yaaaaaaaay.

I'm not even sure I can guess at what level of incompetence this requires.

Also, it's VMware. It looks weird the other way and generally shows you don't have familiarity with what you're talking about.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
For the longest time VMware’s internal IT didn’t use any virtualization (as of 2011). Why? More management headaches than it’s worth for systems that run software provided by vendors and packaged up nicely rather than developed in-house - the precise scenario that most VMware customers use vSphere for.

StabbinHobo
Oct 18, 2002

by Jeffrey of YOSPOS
edit: oh crap i was pages back

in a well actually
Jan 26, 2011

dude, you gotta end it on the rhyme

In a low stakes, low intensity, onprem existing puppet managed environment is the basic docker puppet module a reasonable way to manage a handful of containers?

Warbird
May 23, 2012

America's Favorite Dumbass

Internet Explorer posted:

Also, it's VMware. It looks weird the other way and generally shows you don't have familiarity with what you're talking about.

My understanding is that it was something with load balancers compounding with some other issues. My team isn’t involved with any of that, so who even knows. I just want some test/dev environments and lord knows that isn’t happening any time soon.

Mao Zedong Thot
Oct 16, 2008


freeasinbeer posted:

If your “devops” use openshift, they are the trash.


Also with k8s I’d make you all fix your own poo poo. poo poo I don’t even go on call anymore.

Edit: the sysadmin thread had me in a combative mood. I think that there are legit draw backs to openshift that makes it less optimal then just using regular kubernetes. They subvert more then a few workflows and endpoints behind what I think is worse abstractions.

Yeah, I'm on call for things like 'something is wrong with the ingresses or kube-router'. "My service is broken" is solidly Not My Problem. :capitalism: except DevOps.

Mao Zedong Thot
Oct 16, 2008


PCjr sidecar posted:

In a low stakes, low intensity, onprem existing puppet managed environment is the basic docker puppet module a reasonable way to manage a handful of containers?

It's fine in an appropriately low intensity environment, but Docker is utter trash at every turn, and I would recommend using literally anything to orchestrate scheduling other than some puppet module that runs docker commands (or unit/service files that do the same).

Lily Catts
Oct 17, 2012

Show me the way to you
(Heavy Metal)
Are there vendor-agnostic certifications for DevOps? I might take the AWS Associate ones within the year but I'm wondering if there are options that don't lock you into a specific vendor. Or would AWS be good enough for most companies?

I found this from the Linux Professional Institute and it looks promising... Any thoughts?

Walked
Apr 14, 2003

Are there any CI systems that people have had good success with? I'm looking at alternatives to Jenkins that we can run on-prem (some projects have this as a requirement).

I have an email out to the Travis CI guys for enterprise pricing; but I'm also looking at GoCD; what other options are reasonable for on-prem work?

Methanar
Sep 26, 2013

by the sex ghost

Walked posted:

Are there any CI systems that people have had good success with? I'm looking at alternatives to Jenkins that we can run on-prem (some projects have this as a requirement).

I have an email out to the Travis CI guys for enterprise pricing; but I'm also looking at GoCD; what other options are reasonable for on-prem work?

What's wrong with Jenkins?

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


Concourse I guess. But really. Run jenkins

Mao Zedong Thot
Oct 16, 2008


Concourse is literally the worst software I've ever used. It's not that it's breaks, but it's insanely opinionated and built around incredibly strange and awful abstractions. You can't parameterize builds, you can't rebuild builds, you can't tell where a build is running without curling poo poo in your job. Build config lives 50% in your repo and 50% in concourse. Running a command takes like 4 different files of config.

cheque_some
Dec 6, 2006
The Wizard of Menlo Park
I worked at a place once that used TeamCity, I didn't have any major complaints.

freeasinbeer
Mar 26, 2015

by Fluffdaddy
Basically Jenkins is what we keep coming back to but we hate it. Once I am back from vacation I am going to just use it as a shim for cloudbuilder.


The problem I have with Jenkins is that we already do all our builds in docker, so something like Travis, Concourse or drone make a lot of sense to us where we can define a super simple set of steps in yaml that wrap what the devs are doing already.

Hadlock
Nov 9, 2004

I would really like to see a clone of Jenkins not written in Java, plus Jenkinsfile support in a language other than groovy.

We've used bamboo and Jenkins primarily. And one guy used team city, which was just dead reliable. But 95% of everything I've seen was running Jenkins.

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


Mao Zedong Thot posted:

Concourse is literally the worst software I've ever used. It's not that it's breaks, but it's insanely opinionated and built around incredibly strange and awful abstractions. You can't parameterize builds, you can't rebuild builds, you can't tell where a build is running without curling poo poo in your job. Build config lives 50% in your repo and 50% in concourse. Running a command takes like 4 different files of config.

100% agree but he wanted alternatives.

Volguus
Mar 3, 2009

Hadlock posted:

I would really like to see a clone of Jenkins not written in Java, plus Jenkinsfile support in a language other than groovy.

We've used bamboo and Jenkins primarily. And one guy used team city, which was just dead reliable. But 95% of everything I've seen was running Jenkins.

Do you have a wishlist for such a system? The language is written in I doubt matters very much, though probably support for scripts/instructions written in many languages would be a bonus.
If you can outline a set of needed features, i can guarantee that there are developers out there able and willing to implement said features, even if it would mean to fork an existing system (jenkins....).

kitten emergency
Jan 13, 2008

get meow this wack-ass crystal prison
TeamCity is very reliable and works well for us. Depends on what you’re building at the end of the day though.

PerOlus
Jan 26, 2003

We'r even, señor!
Has anyone here used Buildbot, in a larger setting than with 3 developers? I've heard good things about it, but only at small scale.

Walked
Apr 14, 2003

Methanar posted:

What's wrong with Jenkins?

Just doing my due diligence; I'm our cloud services 'lead' but have started taking on a role of supporting some development teams as well - a couple of which are new contracts/projects without a CI/CD platform in place yet.

So I'm just making sure that we're having this conversation - even if it's super brief - before it's too late to even discuss.

poemdexter
Feb 18, 2005

Hooray Indie Games!

College Slice

Hadlock posted:

I would really like to see a clone of Jenkins not written in Java, plus Jenkinsfile support in a language other than groovy.

We've used bamboo and Jenkins primarily. And one guy used team city, which was just dead reliable. But 95% of everything I've seen was running Jenkins.

I would love for Jenkins to support the full Groovy language and not sandbox poo poo in weird ways.

Erwin
Feb 17, 2006

poemdexter posted:

I would love for Jenkins to support the full Groovy language and not sandbox poo poo in weird ways.

I'm currently developing a shared pipeline library for some Jenkins stuff and I loving hate it. You end up with @NonCPS all over the place because of its weird sandboxing poo poo, they want you to use declarative syntax because it's new and better but it doesn't support anything mildly complicated, and the only way to test changes that can't be unit tested is to commit and push it and run a job on Jenkins because god forbid they support loading the library from anything besides source control, so you end up with a hundred commit messages like "gently caress it let's try putting this here."

Seriously just support rake libraries or something.

poemdexter
Feb 18, 2005

Hooray Indie Games!

College Slice

Erwin posted:

I'm currently developing a shared pipeline library for some Jenkins stuff and I loving hate it. You end up with @NonCPS all over the place because of its weird sandboxing poo poo, they want you to use declarative syntax because it's new and better but it doesn't support anything mildly complicated, and the only way to test changes that can't be unit tested is to commit and push it and run a job on Jenkins because god forbid they support loading the library from anything besides source control, so you end up with a hundred commit messages like "gently caress it let's try putting this here."

Seriously just support rake libraries or something.

Abuse the replay functionality in Jenkins to test library changes so you don't make a million commits to test changes. Lucky for us, we haven't needed @NonCPS but in a few spots.

Also peek at https://github.com/jenkinsci/JenkinsPipelineUnit for unit testing pipeline things. We have over 150 unit tests for our shared pipeline library and this made it possible.

spiritual bypass
Feb 19, 2008

Grimey Drawer
Gitlab is cool

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

Gitlab is the first and only CI system I've used but I like it a lot.

(I actually made things a bit more complicated for myself than I had to because I stubbornly decided I wanted to deploy Gitlab itself, the CI runner, and the glorified cronjob cleaning up old runner caches as a single docker-compose stack, and then I wanted to use the docker build executor on top of that. A plain old bare metal install is probably much easier to get up and running.)

NihilCredo fucked around with this message at 22:53 on Mar 9, 2018

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


Git lab is pretty cool. They run all their builds on DigitalOcean too. Plug for my old company.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
I already said my piece earlier in the thread for why Jenkins is an abomination that costs you in time and is therefore the worst option for smaller companies as a rule. Circle CI and Travis are all worth looking at for merely building software and deploying their artifacts - that’s that. Bitbucket pipelines and Gitlab CI may work for limited orchestration cases but Code Pipeline may be worth a try if you’re an AWS shop. Bitbucket pipelines is really barebones on features when I looked at it a couple weeeks ago but I know they’re trying to beef it up.

If you are large and can afford the engineer time on Groovy DSL bullshit in Jenkins pipelines or can use Jenkins job builder, Jenkins may be viable. If you’re trying to build stuff without the horrors of Jenkins stateful configurations and node management, Concourse and Drone.io are your primary options.

As a comedy option, maybe using Ansible Tower as CI with playbooks as builds could work.

Mr. Crow
May 22, 2008

Snap City mayor for life
Teamcity is real good. Every other CI server I've used I've been unimpressed with (Jenkins and Travis pretty much).

Cerberus911
Dec 26, 2005
Guarding the damned since '05
I'm a fan of concourse from using it at my last 2 workplaces. Once you get over the abstractions and realize it's just generic tasks running in a pipeline it becomes very flexible.

Question for the more experienced devops people here. How do you normally interview for people devops positions? I might have to that soon, and I've only done dev interviews in the past.

Adbot
ADBOT LOVES YOU

Warbird
May 23, 2012

America's Favorite Dumbass

Have the interviewer quantity their definition of DevOps to you at the outset. That’ll get you a decent idea of where they’re at as far as ops v admin v CI v lord knows what else.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply