|
Those of you claiming a ~15 minute build time, how many microservices are you talking about?
|
#
?
Sep 27, 2018 23:50
|
|
|
|
| # ? May 16, 2024 13:46 |
|
|
Our monolith takes about 15 minutes to build, other services are in the 3-10 minute range. Full commit to live smoke test including generating private dns record etc is probably close to 20 minutes
|
|
#
?
Sep 27, 2018 23:51
|
|
|
Hadlock posted:Our monolith takes about 15 minutes to build, other services are in the 3-10 minute range. Do you guys do load tests? Any integration with internal services outside your control?
|
|
#
?
Sep 28, 2018 01:19
|
|
|
Ok goons, continue doing only puppet work at a job I largely don't like or take a 50k take home pay cut to take a swing at DevOps consulting at a place I've been wanting to go? And no, they won't come up a drat penny.
|
|
#
?
Sep 28, 2018 02:18
|
|
|
Warbird posted:Ok goons, continue doing only puppet work at a job I largely don't like or take a 50k take home pay cut to take a swing at DevOps consulting at a place I've been wanting to go? And no, they won't come up a drat penny. Stick around and continue looking for a similar DevOps job you do like. If you don’t have to, never take a paycut. Your negotiation position completely depends on your ability to walk away from the offer if you don’t like it. Read more about that in negotiation thread in BFC. Don’t settle for a job at a run of the mill conultancy firm. The hourly rates are usually through the roof so if they don’t make you a decent offer it teaches you a few things about the way they run their company. You can and will not for the life of you get a decent raise after you start working there. They’re penny pinchers. If they really want you, they’ll come up with a better offer. Otherwise they’re just interested in putting a warm body in a seat. For me this would already be a red flag and I wouldn’t want to work for them. It sounds like you have a rather cushy job which you don’t have to leave straight away. Take this interview as experience. Update your resume and LinkedIn profile and start looking at companies you’d like to work at for opportunities.
|
|
#
?
Sep 28, 2018 09:03
|
|
|
I tend to agree with you. I'm raw as hell for anything that isn't Puppet (which I'm so so with at best) and I'm extremely concerned that when/if I leave where I am that I will be pigeonholed into more Puppet stuff. I don't have a drat clue what most of you are saying most of the time and my main draw for these guys is that I'd get a good base overview of most everything. The consulting guys know I'm green and I'm more or less over a barrel. The offer is a bit above market for this area to boot. It would for sure be a better career move down the line, but $50k isn't exactly a trivial amount. Bah. Good problem to have though.
|
|
#
?
Sep 28, 2018 17:28
|
|
|
you should absolutely continue shoveling poo poo in puppet if taking a $50k pay cut is your only other option. full stop when it comes to getting a better job, the technologies you work with are not as important as the personal connections you make. do not take a pay cut just for the opportunity to work with different tech
|
|
#
?
Sep 28, 2018 19:00
|
|
|
Point made. Let me ask you all this then, how do go about gaining enough experience in non-work related tech or applications to be able to use it down the line? Certs? I've only ever built directly off of what I did in the last job(s), so I'm super certain how one branches out in a meaningful way.
|
|
#
?
Sep 28, 2018 20:23
|
|
|
Warbird posted:Point made. Let me ask you all this then, how do go about gaining enough experience in non-work related tech or applications to be able to use it down the line? Certs? I've only ever built directly off of what I did in the last job(s), so I'm super certain how one branches out in a meaningful way. I installed docker on my PC at home and spun up Jenkins to play with setting up builds and stuff for random hello world projects. I'm pretty sure I would go the same route for playing with other tech stacks in the future. Every company is going to be different with how they setup and use tech, so as long as you know enough to get through the door and not be completely useless day 1, you can pick up the rest during business hours.
|
|
#
?
Sep 28, 2018 20:25
|
|
|
Warbird posted:Point made. Let me ask you all this then, how do go about gaining enough experience in non-work related tech or applications to be able to use it down the line? Certs? I've only ever built directly off of what I did in the last job(s), so I'm super certain how one branches out in a meaningful way. A lot of it is just a mindset thing, IMO, and demonstrating a strong grasp of fundamentals. If you want to do "devops" stuff I think you just need to be able to speak about, roughly: - 12 factor - pets vs cattle - 0 touch provisioning - task orchestration (build / deploy) - environment management or infrastructure as code It's unreasonable to expect a new hire to be an expert at a particular tool (unless the job posting is like Sr. Engineer - OpenStack or whatever). You can totally answer a question about deploying a new kubernetes application with the puppet+VMware version, or whatever. As an interviewer I'd be mostly looking for you to demonstrate knowledge about the above items anyway, so if you give me the answer for your own tech stack that's really just an even better opportunity for me to get at those fundamentals instead of talking about replica set pod bullshit which ultimately does not matter. Comedy option though I usually learn a new tech stack during technical exercises at job interviews.
|
|
#
?
Sep 28, 2018 22:29
|
|
|
Warbird posted:Point made. Let me ask you all this then, how do go about gaining enough experience in non-work related tech or applications to be able to use it down the line? Certs? I've only ever built directly off of what I did in the last job(s), so I'm super certain how one branches out in a meaningful way. Generally, I learn new technologies on the job site, either when switching jobs or when trying new tech at the current job. 12 rats tied together posted:A lot of it is just a mindset thing, IMO, and demonstrating a strong grasp of fundamentals. Good answer. OP, if you can't readily switch jobs, I advise you to read up on these devops concepts and figure out ways to apply them to problems at your current job. If you can come up with a plan to improve something and execute it, that's a great story to tell at your future interviews.
|
|
#
?
Sep 28, 2018 22:57
|
|
|
12 rats tied together posted:A lot of it is just a mindset thing, IMO, and demonstrating a strong grasp of fundamentals. Thanks, I appreciate the breakdown! My entrance into Ops/DevOps was by complete chance and I never got a chance to become acquainted with most of what you listed; too many fires to put out. I'll set aside this weekend and read up on those points.
|
|
#
?
Sep 28, 2018 23:53
|
|
|
Warbird posted:Thanks, I appreciate the breakdown! My entrance into Ops/DevOps was by complete chance and I never got a chance to become acquainted with most of what you listed; too many fires to put out. I'll set aside this weekend and read up on those points. Working with Puppet already gets you like 3 out of those 5 elements so you're not off as far as you might think.
|
|
#
?
Sep 29, 2018 00:02
|
|
|
Helianthus Annuus posted:Good answer. OP, if you can't readily switch jobs, I advise you to read up on these devops concepts and figure out ways to apply them to problems at your current job. If you can come up with a plan to improve something and execute it, that's a great story to tell at your future interviews. This is basically how I learned puppet/config management years ago. I had started to hear about this stuff from various sources and it sounded very interesting, but had no hands on experience whatsoever. A project came up at work where I needed to configure a whole bunch of nearly identical Linux boxes and I said gently caress it, I can totally justify learning Puppet on company time to knock this out. The project was successful, people loved seeing the power of automation vs hand configuring a bunch of crap over and over. I got to learn a hot new thing (and then parlay that into a new job at a 50% raise shortly after lmao, though I had no such Machiavellian scheme when I started out). I’m not outright advocating Resume Driven Development here. Don’t just set up kubernetes to run your company’s old rear end Java monolith for shits and giggles. But if you can find some way to get what you want to learn into a project where it actually makes sense, by all means push for that.
|
|
#
?
Sep 29, 2018 01:45
|
|
|
what the hell is 12 factor, do i have yet another bullshit invented term to learn? jesus, i hate computers I interviewed a guy today whose resume looked like it was a word jumble from tech trends magazine. "Implemented machine learning and AI neural networks to solve automation and monitoring" sort of stuff. Don't be like that guy. if you actually know how to install and manage kuberbetes jesus let me hire you please because 95% of candidates have flubbed the "what network layer provider did you decide on" because they used EKS or equiv and it's all done for you On that note, gently caress you amazon give me EKS and route 53 in govcloud already It is pretty funny that almost everyone has moved away from puppet/chef to ansible which makes sense if you aren't worried about state validation/enforcement because you just rebuild golden image containers all day errday but dammit, we've got 3 years invested in this stack and you'll have to pry chef from our senior devs's cold dead hands it's rough hearing candidate after candidate suck air though their teeth and say well I'm pretty rusty on chef, I've been doing ansible for the last three years and i just sit there silently nodding but we won't budge no sir Bhodi fucked around with this message at 03:14 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 02:41
|
|
|
quote is not edit also i should probably also stop drunkposting in serious threads but you're not the boss of me 
Bhodi fucked around with this message at 03:08 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 02:54
|
|
|
I know of a company that doubled down on Chef so hard they built their own monstrous artisinal distributed chef cluster to handle tens of thousands of nodes being provisioned at once, multiple times a day, rather than use pre-built AMIs, or something else immutable.
|
|
#
?
Sep 29, 2018 03:16
|
|
|
Bhodi posted:if you actually know how to install and manage kuberbetes jesus let me hire you please because 95% of candidates have flubbed the "what network layer provider did you decide on" because they used EKS or equiv and it's all done for you How would you feel about "flannel because it was the first one I tried and it worked?" I don't have k8s on my resume, no sir.
|
|
#
?
Sep 29, 2018 03:57
|
|
|
Zorak of Michigan posted:How would you feel about "flannel because it was the first one I tried and it worked?" 99% of people who didn't use eks just mashed "I'm feeling lucky" and then "brew install kops" I tell everyone who's trying to get into or up on tech that you have no idea the kind of talent you're up against Docjowles posted:set up kubernetes to run your company’s old rear end Java monolith Bhodi fucked around with this message at 04:14 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 04:01
|
|
|
lol i'd start ranting about the awsvpc cni for eks being the same drat thing as the one for ecs so you're still pod-count limited to the number of enis if you want security groups to mean anything, flip the table, and storm out
|
|
#
?
Sep 29, 2018 04:15
|
|
|
Bhodi posted:what the hell is 12 factor, do i have yet another bullshit invented term to learn? jesus, i hate computers Are you vaguely aware of how to write and operate modern applications, where modern is like TYOOL 2012? It is that. https://12factor.net/. Plus the usual "make your app stateless!!!" "ok but the state has to live someplace, what do we do with that?" "???????????" Which is not at all to say that you should stop hating computers. gently caress computers. Methanar posted:I know of a company that doubled down on Chef so hard they built their own monstrous artisinal distributed chef cluster to handle tens of thousands of nodes being provisioned at once, multiple times a day, rather than use pre-built AMIs, or something else immutable. I see you've met my coworker. Guy has a simple decision making process. Is $thing written in Ruby and Chef (ideally by him)? It is good. No? It is utter trash and needs to be reimplemented in Ruby and Chef, by him. Corollary: if it was written in Ruby/Chef but not by him, it sucks, and needs to be rewritten by him. I swear 95% of this dude's tickets are "rewrite cookbook for no goddam reason", or "build tool nobody asked for". Opened by himself. edit: Bhodi posted:don't doxx me I mean this is basically how we got k8s into our org at my current job, except it was from the Director level instead of a rando ops engineer so we actually had power to tell people to rewrite their poo poo to not monopolize an entire worker node with one 64GB Java heap container that cannot tolerate downtime ever Docjowles fucked around with this message at 04:18 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 04:15
|
|
|
Docjowles posted:Are you vaguely aware of how to write and operate modern applications, where modern is like TYOOL 2012? It is that. https://12factor.net/. Plus the usual "make your app stateless!!!" "ok but the state has to live someplace, what do we do with that?" "???????????" Actually more than hate it, the fact that someone typed that inane drivel up and is apparently treating it like the new testament is THERE IS NO CAROL level crazy. I lose respect for anyone who takes self-evident do-the-thing-like-you'd-do-it as some enlightened idea. "Run admin/management tasks as one-off processes" come the gently caress on. This advice is on the same level as "Use e-mail or chat to coordinate tasks with your coworkers!" Docjowles posted:I mean this is basically how we got k8s into our org at my current job, except it was from the Director level instead of a rando ops engineer so we actually had power to tell people to rewrite their poo poo to not monopolize an entire worker node with one 64GB Java heap container that cannot tolerate downtime ever Bhodi fucked around with this message at 04:42 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 04:33
|
|
|
Bhodi posted:Thanks, I hate it See, if I showed 12 factor to some of the devs I work with, they'd tell me it was a beautiful dream but this is real world and they can't write apps like that. It's only self-evident if you already get it. Otherwise, you go right ahead sucking up 64gb Java heaps and complaining that the infrastructure team won't give you enough resources.
|
|
#
?
Sep 29, 2018 06:47
|
|
|
Docjowles posted:Which is not at all to say that you should stop hating computers. gently caress computers.
|
|
#
?
Sep 29, 2018 07:01
|
|
|
CNI doesn’t matter til it does and when it does you realize they are all varying levels of poo poo.
|
|
#
?
Sep 29, 2018 07:49
|
|
|
freeasinbeer posted:CNI doesn’t matter til it does and when it does you realize they are all varying levels of poo poo. Actually calico is the best
|
|
#
?
Sep 29, 2018 07:49
|
|
|
Warbird posted:Thanks, I appreciate the breakdown! My entrance into Ops/DevOps was by complete chance and I never got a chance to become acquainted with most of what you listed; too many fires to put out. I'll set aside this weekend and read up on those points. You are me, 3-4 years ago. I had a lot of experience in one specific area and wanted to branch out and learn more skills than just being a 1 trick pony (I wasn’t, but felt like one). I got a gig at a consultancy firm that saw the potential of getting me a gig in my current expertise but also let me touch new stuff. Getting new roles becomes easier and easier as you exposed to more and more technologies. To speed things up be sure to always have a project you pick up at home or if you have downtime at the office. I don’t mean you need to spend 10 hours each week in your spare time, but looking into stuff you don’t know yet and hear a lot about certainly helps in getting a better view of the big picture. As others have mentioned, just start playing with stuff and go from there. Methanar made a huge effort-post in the general IT thread some time (months probably?) about a good way to get started learning devops skills. If you want I can repost it here, it was an excellent post and helped several goons on their way already.
|
|
#
?
Sep 29, 2018 07:57
|
|
|
Methanar posted:Actually calico is the best It breaks and can take etcd down with it. In our case it seems to break if you scale nodes too quickly. We had a production outage because of it.
|
|
#
?
Sep 29, 2018 07:58
|
|
|
Bhodi posted:
I got a job offer 2 months ago half just because when I was asked the difference between ClusterIP and NodePort I ended up giving a 15 minute explanation of iptables, IPVS, snat abuse and why the Kubernetes networking model actually sucks rear end, especially on-prem. All the guy was hoping for was 'Nodeport is externally accessible'. I was apparently the only one they could find at all that knew even the slightest bit of how it actually worked. Because exactly as you said quote:because they used EKS or equiv and it's all done for you This company was spending a bajillion dollars on AWS and had some pretty serious financial incentives to disentangle themselves from the grip of Amazon and move their substantial baseload the gently caress out of the cloud and into their own on-prem DC. Part of that was migrating their stuff from AWS-specific orchestration to Kubernetes which is obviously a bit more portable than the AWS API. The problem for them then became 'what do you mean I won't have kops, or ELB.' Its actually really really hard to find people who have any experience outside of AWS. Methanar fucked around with this message at 09:07 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 08:54
|
|
|
I have strong opinions on running datacenters.
|
|
#
?
Sep 29, 2018 09:02
|
|
|
12 factor is idealistic. But we work mono repo and store our code logically:code:Apps is the app specific code the SRE team writes to support infrastructure for the company's apps. Network is the mutable or semi mutable stuff such as Akamai or whatever. Jobs are Jenkins jobs stored in YAML and parsed by Jenkins Job Builder. Modules are the common reusable components the team builds referenced by other projects or code. Services are all the SRE team specific apps we support, such as Jenkins, Artifactory, or Sonar.
|
|
#
?
Sep 29, 2018 13:34
|
|
|
Methanar posted:I know of a company that doubled down on Chef so hard they built their own monstrous artisinal distributed chef cluster to handle tens of thousands of nodes being provisioned at once, multiple times a day, rather than use pre-built AMIs, or something else immutable. e: Phil Dibowitz did a talk at ChefConf 2013 about their approach called Scaling Systems Configuration at Facebook if anyone wants to dig it out Vulture Culture fucked around with this message at 17:18 on Sep 29, 2018 |
|
#
?
Sep 29, 2018 17:13
|
|
|
I don't know where you guys are that a director of engineering could realistically order a bunch of overworked engineers to containerize anything remotely legacy let alone make it 12F whatsoever. That's a major rewrite basically (and if you're rewriting more than 25% of the software, you should just rewrite the whole thing) - this kind of software inertia is a major reason why Innovator's Dilemma hits so many software companies regardless of how smart and well-thought out the software was. In my experience, the struggle to acquire and even retain customers has been the far more important factor than anything engineering is concerned with and technical debt was wildly or soon to be wildly out of control while teams got slower and slower to deliver unless you were lucky to be on a greenfield effort, which was also luck or nepotism driven. Most developers worth a salt left before anything could be done with their experience / expertise, so even refactors tend to be deathknells. I'd kill for an actually justifiable reason to use immutable infrastructure practices but it's truly nonsense anywhere I've been due to developers being given all the reins to dump stuff everywhere and push out whatever because management was terrified of retaining developers and operations is basically dime-a-dozen. Granted, it's kinda strange how even companies like Chik-Fil-A and Target are using Kubernetes but that just makes the actual tech companies still on ancient practices and stacks look even worse. Supporting crufty, bloated Java apps that can't handle the slightest wiff of anything going wrong at a whole scale of under 100 nodes and maybe 50 requests per second (40 of which are from your own company) is not good for one's career in this game because relevant experience also means modern and scalable experience when it comes to a field around technology. Serverless may be arguably trendy hipster crap but I'll take managing a bunch of mismanaged Lambda functions over distributed snowflake monoliths each hand-crafted with the blood of 3 ops engineers in a ritual deployment at 2 am on a Saturday involving a bottle of Jim Bean and a half-assed, untested Ansible playbook any day.
|
|
#
?
Sep 30, 2018 00:13
|
|
|
necrobobsledder posted:I don't know where you guys are that a director of engineering could realistically order a bunch of overworked engineers to containerize anything remotely legacy let alone make it 12F whatsoever. That's a major rewrite basically (and if you're rewriting more than 25% of the software, you should just rewrite the whole thing) - this kind of software inertia is a major reason why Innovator's Dilemma hits so many software companies regardless of how smart and well-thought out the software was. In my experience, the struggle to acquire and even retain customers has been the far more important factor than anything engineering is concerned with and technical debt was wildly or soon to be wildly out of control while teams got slower and slower to deliver unless you were lucky to be on a greenfield effort, which was also luck or nepotism driven. Most developers worth a salt left before anything could be done with their experience / expertise, so even refactors tend to be deathknells. The decades-old recommendation on reimplementing legacy projects greenfield is based on a combination of voodoo and horseshit, rooted in mythologies and fables around waterfall development cycles, and I am unconvinced any of it is grounded in reality. Legacy systems are legacy because they're too important to outright discontinue but too burdensome to forklift—it was too hard to replace it ten years ago, so why is it suddenly a worthwhile investment with another decade of accreted inertia? The more people, processes, and workflows you have using them, the more coordination is required to replace them wholesale, requiring your efforts to shift into complex, error-prone human migration processes instead of complex, slightly less error-prone refactors. The complexity is irreducible. A lot of people choose the full rewrite because the rewrite kicks the complexity can down the road instead of figuring out how to break down the software into constituent parts that can be refactored and rewritten piecemeal. The only thing that should be greenfielded in a business-critical rewrite of any magnitude is a prototype, with the intention of breaking down and reworking the parts of that prototype that prove themselves to be ready. The backporting of Rust/Servo components into Mozilla Firefox is a first-class example of how to do this in a way that creates a platform for moving forward without attempting to displace twenty years of accumulated work in one fell swoop. Pioneers/settlers/town planners is basically a required organizational structure in any company dealing with legacy software/processes of any kind. Vulture Culture fucked around with this message at 17:49 on Sep 30, 2018 |
|
#
?
Sep 30, 2018 17:40
|
|
|
Yeah, but at it's root it's a problem with a monetary figure large enough that it tends to get decided on business interests way above the technical level and as we all know business interests are quarter-based with maybe thinking out up to a year. So the solution is keep the Frankenstein stack shambling along until it falls over or you can't buy parts for it anymore. At least when things were physical there was a solid "Sorry, we have to rebuild it anyway, we're moving datacenters / hardware is being EOL'd". With containers, it's going to be interesting in a decade since in theory you could run garbage code in perpetuity, there's no hard requirement to replace/rebuild. Or maybe the next thing will be incompatible with containers in the same way containers are incompatible with vcenter / xen. It's probably going to be worse than I even expect because at some point there will come a time where you literally cannot build another container due to your CI/CD software being EOL'd but your current containers run just fine so I fully expect to see some container built a decade prior running somewhere like those windows NT installs still running in water treatment plants, only connected up to the internet Bhodi fucked around with this message at 19:48 on Sep 30, 2018 |
|
#
?
Sep 30, 2018 19:38
|
|
|
Since the whole pipeline is probably an amalgam of third-party hosted services in the first place, I dread finding out what that even looks like in a decade
|
|
#
?
Sep 30, 2018 20:41
|
|
|
I think I’m going to accept that full time consulting gig tomorrow. Pay cut or no I think it would be more beneficial for my career by way of establishing a solid base and having the ability to branch out. My contracting firm also recommended I commit tax fraud so I could get extra cash, so it might be best to not be associated with them. Silver lining: Since I don’t much care for pissing off the IRS the pay cut is only 25k or so. Which would be about where I would be if I converted at my current place.
|
|
#
?
Oct 1, 2018 03:13
|
|
|
Warbird posted:I think I’m going to accept that full time consulting gig tomorrow. Pay cut or no I think it would be more beneficial for my career by way of establishing a solid base and having the ability to branch out. My contracting firm also recommended I commit tax fraud so I could get extra cash, so it might be best to not be associated with them. That’d be a really dumb move. Find a consultancy gig that’ll at least pay you the same as you earn now. With accepting the offer your new employer will also know that you’re a weak negotiator and will stil accept their terms even if it hurts your own bottom line (like it does now). The fact that they won’t come up a dime now means you’re not likely to ever get a meaningful raise because they know you’ll accept their terms anyway. Do yourself a favor and read the negotiation thread in BFC before accepting this offer. There are so many goons who earn thousands of dollars a year more thanks to the advice there. This job offer is not unique, you can get a dozen like it in a matter of days. The IT market is booming, at least see if you can find another interview or two to see what other companies are willing to pay you. Lowball offer + not willing to negotiate = major red flag! Edit: negotiation thread link LochNessMonster fucked around with this message at 05:25 on Oct 1, 2018 |
|
#
?
Oct 1, 2018 05:22
|
|
|
if you're taking a paycut to get into devops you're either a surgeon or doing it wrong
|
|
#
?
Oct 1, 2018 05:24
|
|
|
|
| # ? May 16, 2024 13:46 |
|
|
StabbinHobo posted:if you're taking a paycut to get into devops you're doing it wrong
|
|
#
?
Oct 1, 2018 05:39
|
|