|
what are people using for build pipelines on greenfield GKE projects? I have a use case where I want to add very thin custom layers on top of pre-existing container images as fast as conceivably possible (ideally closer to hundreds of milliseconds than seconds).
|
#
?
Mar 12, 2018 04:47
|
|
|
|
| # ? May 15, 2024 03:51 |
|
|
StabbinHobo posted:what are people using for build pipelines on greenfield GKE projects? I have a use case where I want to add very thin custom layers on top of pre-existing container images as fast as conceivably possible (ideally closer to hundreds of milliseconds than seconds). Jenkins, but about to use that to wrap cloudbuilder. If you use a hosted github or bitbucket I’d just use that. Looked at drone.io and concourse, but drone gates some basic features behind a paywall and concourse seems overbuilt.
|
|
#
?
Mar 12, 2018 04:58
|
|
|
We've flat out stopped recommending clients host their own CI systems, unless they actually have a security, or a specific engineering need for them to self-host. Between gitlab, Bitbucket pipelines, Circle CI, etc, etc, there's a lot of options to point folks to if they don't necessarily want to dedicate engineer time to janitoring a ci setup. If you have to, Jenkins is fine though. My Windows DevOps friends swears by TeamCity and OctopusDeploy.
|
|
#
?
Mar 15, 2018 21:19
|
|
|
Am I shooting myself in the foot if my first foray into CI/CD is VSTS? It's been super easy to set up and I haven't run into anything I wanted to do that I haven't been able to do yet, but since it doesn't seem to have a lot of community uptake, I can't help but think my time would be spent better with other tools.
|
|
#
?
Mar 15, 2018 22:49
|
|
|
The Fool posted:Am I shooting myself in the foot if my first foray into CI/CD is VSTS? It's been super easy to set up and I haven't run into anything I wanted to do that I haven't been able to do yet, but since it doesn't seem to have a lot of community uptake, I can't help but think my time would be spent better with other tools. Nope. It's an awesome platform and the basic tenents of continuous integration/delivery are basically tool agnostic. [Full disclosure: I work for a Microsoft partner and do a lot of work in VSTS]
|
|
#
?
Mar 15, 2018 23:02
|
|
|
OWLS! posted:My Windows DevOps friends swears by TeamCity and OctopusDeploy. Hell yeah. I've been using this combo quite successfully for about 5 years in 3 companies for .NET web apps. The key to any CI/CD pipeline is to not allow your inner tinkerer to get free and over-complicate things. If you find yourself writing tons of custom scripts, step back and consider that what you're doing is unnecessary, stupid, or both.
|
|
#
?
Mar 15, 2018 23:38
|
|
|
That's only the second time I've ever heard of Octopus Deploy in nearly five years.poemdexter posted:I would love for Jenkins to support the full Groovy language and not sandbox poo poo in weird ways. Yeah next week I am diving in to building out our CD system for a single-tennant version of our multitenant product and not looking forward to this. The build engineer at my last company just figured out how to execute bash from inside jenkins-groovy and our jenkinsfiles were just huge quoted text blocks of "run this series of bash scripts". I think Jenkins has at least a ruby plugin, I would imagine there's Python support too via a plugin. Volguus posted:Do you have a wishlist for such a system? The language is written in I doubt matters very much, though probably support for scripts/instructions written in many languages would be a bonus. Are you offering to build me an enterprise-grade CI system for free...?
|
|
#
?
Mar 16, 2018 03:18
|
|
|
if you need windows stuff (and can do cloud), appveyor is pretty decent and it has cloud or on-prem options.
|
|
#
?
Mar 16, 2018 03:33
|
|
|
I've done linux and windows builds in bamboo and pretty happy with it for the most part, but if you don't use other atlassian products to integrate it all together it would be silly. Also keeping it up to date is kinda annoying. Its fine when its up and working.
|
|
#
?
Mar 16, 2018 16:25
|
|
|
Hadlock posted:Are you offering to build me an enterprise-grade CI system for free...? Not a chance in hell. But if you can outline your needs, needs that maybe other build systems do not cover, surely there must be people out there able and willing to implement them, even if in existing build systems. The idea is: just like with any other field, the experts in that field have their needs and wants that a developer may not encounter or even be aware of. You (or some other dude) complained about jenkins: it's java, and it doesn't do what you want. ok, perfect, what do you want? Make a request to the jenkins bug tracking system. Or, write a blog article outlining the ideal build system for you, the person who has to babysit them every day. Maybe they could be integrated into existing applications (the world needs another build system like we need another email client, bug tracking software, im/irc client,etc.) or maybe they're so outlandish that conflict with every existing application out there and therefore something brand new is required. the point is, complaining about a build system without providing solutions won't help you. or anyone.
|
|
#
?
Mar 16, 2018 19:55
|
|
|
Volguus posted:Not a chance in hell. But if you can outline your needs, needs that maybe other build systems do not cover, surely there must be people out there able and willing to implement them, even if in existing build systems.
|
|
#
?
Mar 16, 2018 20:59
|
|
|
Vulture Culture posted:i suspect you have a problem speaking others' love languages since i have no idea what that even means ... i guess i do.
|
|
#
?
Mar 16, 2018 22:25
|
|
|
OWLS! posted:We've flat out stopped recommending clients host their own CI systems, unless they actually have a security, or a specific engineering need for them to self-host. Do any of those places do CI for private personal repositories for free (either free free, or free for students)? I have a few things for school that I would love to throw into a CI system if anything just to keep my documentation and pdf/latex files consistently up to date but last I checked they were free for public repositories only.
|
|
#
?
Mar 17, 2018 17:32
|
|
|
I think Travis CI is free for public repositories but I don’t think you’re supposed to make your schoolwork public. I know that AWS CodePipeline is free for a single pipeline active per month and that the first 30 days are free. That may work for you.
|
|
#
?
Mar 17, 2018 17:35
|
|
|
Boris Galerkin posted:Do any of those places do CI for private personal repositories for free (either free free, or free for students)? I have a few things for school that I would love to throw into a CI system if anything just to keep my documentation and pdf/latex files consistently up to date but last I checked they were free for public repositories only. VSTS is free for 5 users
|
|
#
?
Mar 17, 2018 17:41
|
|
|
Boris Galerkin posted:Do any of those places do CI for private personal repositories for free (either free free, or free for students)? I have a few things for school that I would love to throw into a CI system if anything just to keep my documentation and pdf/latex files consistently up to date but last I checked they were free for public repositories only. CircleCI does
|
|
#
?
Mar 17, 2018 18:29
|
|
|
Most cloud CI services are free for public repos.
|
|
#
?
Mar 17, 2018 18:44
|
|
|
StabbinHobo posted:what are people using for build pipelines on greenfield GKE projects? I have a use case where I want to add very thin custom layers on top of pre-existing container images as fast as conceivably possible (ideally closer to hundreds of milliseconds than seconds). Is this for layering config inside the image? Are ConfigMaps somehow unsuitable?
|
|
#
?
Mar 19, 2018 06:51
|
|
|
The Fool posted:VSTS is free for 5 users For anyone lazy: https://www.visualstudio.com/team-services/ Thanks you both. I’ll look into those when I get some time. uncurable mlady posted:Most cloud CI services are free for public repos. Right, but I said private repositories.
|
|
#
?
Mar 19, 2018 12:07
|
|
|
Got a question about user permissions. I'm building AMIs and when I deploy them, I take the EBS volume attached to the old one and then plug it into the new one. The problem I ran into was that the new AMI didn't have the users created in the same order (telegraf, elasticsearch, etc) and thus my elasticsearch storage volume on this EBS volume ended up having the owner changed from elasticsearch:elasticsearch to telegraf:telegraf, because the telegraf user happned to be created with the same uid/gid that the elasticsearch user was using previously. What's a good way to handle this? I was thinking maybe I need to create my users with known uid/gid before installing the RPMs, so that the RPM install doesn't create a user with whatever the hell ids it wants. Another option would be to kinda fix it after the fact, and just chown it when I reconnect the EBS volume fletcher fucked around with this message at 01:40 on Apr 10, 2018 |
|
#
?
Apr 10, 2018 01:21
|
|
|
fletcher posted:I'm building AMIs Probably want to take a look at Packer to automate it entirely instead of copying around EBS volumes.
|
|
#
?
Apr 10, 2018 01:54
|
|
|
dox posted:Probably want to take a look at Packer to automate it entirely instead of copying around EBS volumes. I'm already using Packer to build the AMIs. I'm not copying the EBS volume, just unplugging it from the old instance and plugging it into the new instance (via terraform) for the data that needs to be persistent.
|
|
#
?
Apr 10, 2018 01:59
|
|
|
fletcher posted:I'm already using Packer to build the AMIs. I'm not copying the EBS volume, just unplugging it from the old instance and plugging it into the new instance (via terraform) for the data that needs to be persistent. I think doing a chown post install will prevent any other kinds of issues like the one your described from popping up again in the future.
|
|
#
?
Apr 10, 2018 02:11
|
|
|
Would you not be able to create the users and set the uid/grid before installing what ever packages you need to ensure consistency?
|
|
#
?
Apr 10, 2018 06:57
|
|
|
fletcher posted:I was thinking maybe I need to create my users with known uid/gid before installing the RPMs, so that the RPM install doesn't create a user with whatever the hell ids it wants. As a general rule you want to make sure that whatever provisioning software you're running via Packer is idempotent. If the uids and gids are changing then you need to make sure that you always add service users ahead of time so that their identifiers are consistent.
|
|
#
?
Apr 10, 2018 12:32
|
|
|
Yeah creating the service users before the RPM install tries to do it seems like the way to go. Thanks guys!
|
|
#
?
Apr 10, 2018 21:40
|
|
|
Does anyone know of a proper guide for setting up triggers such that Jenkins will kick off jobs in response to 1) Github PRs being merged 2) Certain text being in a comment made on a PR 3) Github PRs being merged but only with certain text in a comment Jenkins' documentation is unbelievably bad
|
|
#
?
Apr 11, 2018 20:55
|
|
|
Methanar posted:Does anyone know of a proper guide for setting up triggers such that Jenkins will kick off jobs in response to * Set up hooks in Github. * In your SCM setup on Jenkins, you can click "Additional Behaviours" and get the ignoring/building based on commit message text working there. All of this assumes you have the correct plugins and such based on your SCM and you have the correct connectivity set between Github and Jenkins.
|
|
#
?
Apr 11, 2018 21:33
|
|
|
Methanar posted:Does anyone know of a proper guide for setting up triggers such that Jenkins will kick off jobs in response to This might help with interacting with jobs via GitHub: https://github.com/jenkinsci/ghprb-plugin
|
|
#
?
Apr 11, 2018 23:39
|
|
|
Trying to get a rough idea of what’s expected stress / responsibilities compared to others that have broader experience than myself. Is it normal for companies to hire “devops” engineers as a hero engineer that are expected to take completely garbage, stateful, poorly documented, unautomated legacy (5 - 15 years old) software and have exactly one engineer out of 8 - 30 engineers take over most of infrastructure ownership, deployments, release management, and deliver a CI/CD pipeline in less than half a year while being on-call? I’ve talked to dozens of companies (large, small, b2c, enterprise - the full gamut) in several non-tech hubs for years and all but 3 companies seem to want / need exactly this (in veiled or not so veiled intent) while paying maybe 20% more for said engineer(s). It’s getting super old being deployment dave when I spend 30% of my time documenting and making deployments push-button easy for others and getting stuck with marching orders like Dockerizing super stateful, brittle software intended to be pushed into a K8S cluster.
|
|
#
?
Apr 15, 2018 22:16
|
|
|
necrobobsledder posted:Is it normal for companies to hire “devops” engineers as a hero engineer that are expected to take completely garbage, stateful, poorly documented, unautomated legacy (5 - 15 years old) software and have exactly one engineer out of 8 - 30 engineers take over most of infrastructure ownership, deployments, release management, and deliver a CI/CD pipeline in less than half a year while being on-call? Tale as old as time. Companies that hire for "Senior DevOps Engineer" don't actually understand that it's a methodology rather than a job title, and so they wind up renaming Operations to DevOps because they think it's about hiring KTLO janitors who know Python, rather than getting developers to own more of their app's process and lifecycle beyond committing something to source control. Now the hot stuff is to hire SREs, which at least has the convenience of being a job title that also implies a particular methodology, so it's a little harder for HR jobsworths to mess up. My longtime experience has been that companies routinely underestimate and undervalue the importance of deployment plumbing because they assume it just magically happens. Simplicity is hard. Keeping devs on the rails is triply hard. Building automation that doesn't fail open and pour molten steel all over everything when something misfires is infinitely hard.
|
|
#
?
Apr 15, 2018 23:27
|
|
|
necrobobsledder posted:Trying to get a rough idea of what’s expected stress / responsibilities compared to others that have broader experience than myself. yes ask for a raise
|
|
#
?
Apr 16, 2018 01:02
|
|
|
chutwig posted:Now the hot stuff is to hire SREs, which at least has the convenience of being a job title that also implies a particular methodology, so it's a little harder for HR jobsworths to mess up.
|
|
#
?
Apr 16, 2018 01:51
|
|
|
necrobobsledder posted:Trying to get a rough idea of what’s expected stress / responsibilities compared to others that have broader experience than myself. yep, I call it "the devops trap". I've turned down three "director of devops" jobs just in the last year because I simply can't bring myself to step into that awful loving life again. Its not just a lovely work dynamic, it really fucks up your brain and your life to spend the majority of your waking hours mentally crouching in the fetal position, being interrupt-bombarded with broken poo poo that somehow the onus is on you now for. you inevitably grow to resent everyone around you, and then your behavior changes such that they inevitably grow to resent you. this is why every "devops guy" is a 1 - 3 year per company job-hopper, and some kind of functional-alcoholic/stoner. the naive think that if they just job hop another time or two they'll find the place that does it right. I'm 0 for 9.
|
|
#
?
Apr 16, 2018 01:58
|
|
|
Vulture Culture posted:I changed my job title on LinkedIn from Lead Site Reliability Engineer to Engineering Manager last month because of how untrue this is. Yeah I was gonna say, what I am seeing in job postings doesn't really bear this out. Companies are more than happy to spam out listings for ~* SRE *~ that are exactly the same thing traditional Ops people have been doing since time immemorial. Just like they would for DevOps Engineers before that. The SRE model is cool and good. If I don't have to pass a pretty legit software dev interview to earn that title, maybe that's not actually the model your company is using. StabbinHobo posted:being interrupt-bombarded with broken poo poo that somehow the onus is on you now for This is too real. I am ready to come hang out with you at your local Docjowles fucked around with this message at 03:57 on Apr 16, 2018 |
|
#
?
Apr 16, 2018 03:52
|
|
|
necrobobsledder posted:Trying to get a rough idea of what’s expected stress / responsibilities compared to others that have broader experience than myself. This is an SRE job description, welcome to my world. In theory Google is full of SREs that work side by side with developers in complete harmony, but outside of the ivory tower it seems to be something like 1 SRE who architects/builds the system(s) described above, to a ratio of 4 "Devops" engineers at a company who do most of the toil/microconfiguration of said system.
|
|
#
?
Apr 16, 2018 04:12
|
|
|
I have spent a lot of time recently trying to kill of the idea of devops as being a role, and pushing hard for infrastructure engineers and developers to come together and actually build things co-cooperatively, rather than throwing things over the fence to each other and then playing the blame game when things do no run smoothly. I am very fortunate that the company I work for has a fairly good culture and we have a good graduate program that means that change can be influenced by the influx of new talent that joins us. I have been in that hero sysadmin/devops position for many years and all it does is create burnouts, which is terrible, because a lot of the talent of being able to problem solve under extreme pressure and having a really sound understand of a platform as a whole, is really valuable to a product/development team.
|
|
#
?
Apr 16, 2018 10:25
|
|
|
The things you guys are saying echo so true to me. I'm technically a "senior cloud engineer" but also our DevOps Practice Lead (it's not a title yet it somehow still is!) Unfortunately rather than being a function of enabling teams to better own their products through the lifecycle at all layers, it's turned into "empowering developers to feel more comfortable chucking all manner of issues to others because we're all DevOps now" Burnout rings very true
|
|
#
?
Apr 16, 2018 11:55
|
|
|
The success or failure of a collaborative environment, DevOps or otherwise, is predicated almost entirely on the ability of the people involved to say "no" or "not right now"
|
|
#
?
Apr 16, 2018 14:20
|
|
|
|
| # ? May 15, 2024 03:51 |
|
|
I’ve been practicing Docker in my free time and finally had an opportunity to put what I’ve learned to the test. Things went retry well, and I even internalized some gotchas that’ll make future work go a lot more smoothly. That said, a common pain point I had was coordinating logging across the five or six services I’m standing up. I’ve managed to get a few sources of log files going, but none of it gives me great confidence that I could use that logging to troubleshoot anything that might go wrong. Are there any good guides on best practices for capturing log output from containers? For the scale of what I’m supporting, it’d be great to get a robust local logging setup. I know at some point, though, I’ll need to look at services I can use to aggregate data. For now, though, I’m more interested in higher level fundamentals to gain more confidence in Docker.
|
|
#
?
Apr 17, 2018 03:10
|
|
