|
Spring Heeled Jack posted:As a vmware shop I’m super interested in whatever their plans are for the integrated k8s product, whenever it decides to actually surface. Rancher owns incredibly hard
|
#
?
Feb 5, 2020 06:59
|
|
|
|
| # ? May 15, 2024 05:38 |
|
|
Rancher has come a long ways, v1 was an interesting experiment, v2 is a total rewrite and dead simple to install, other than playing around with it in our dev env at my last company I haven't had a chance to use it in prod, but it works if you have on prem servers and need kubernetes... it might meet your needs, and you can always buy a support contract when things go sideways. They also built k3s which is... kubernetes compatible api server... thing. It's sort like minikube but as a single golang binary. It works so long as you're not trying to do anything super fancy with it. More of a functional toy. I personally really like prometheus, we were dumping ~4GB of stats/day into it and had ~2TB of data from the last two years, the DB guys loved being able to query data and do year over year comparisons as we were super seasonal. I think eventually we had to upgrade to a 2 core 16gb server My current company has no coherent ops strategy but we have like... three or more different prometheus servers, they just purr along in the background with zero maintenance, whereas our CSO manages to break nagios almost monthly and it needs lots of care and feeding.
|
|
#
?
Feb 5, 2020 07:57
|
|
|
How difficult is the AWS Devops Pro Certificate exam? My company needs people to gain this cert in order for the company to gain super status partnership and they've asked if I'd be willing to take it. I have a very small amount of experience using AWS. On a previous project, I helped set up infrastructure as code using terraform and a number of different AWS services. This is the only real experience I have in the AWS world. I also studied for the basic AWS cert but never actually took the exam because of other commitments. I'm quite good at cramming/exams but I haven't touched AWS in over a year.
|
|
#
?
Feb 5, 2020 15:53
|
|
|
Requires you to know code commit/deploy/pipeline beanstalk opsworks and cloudformation and how to do HA RDS
|
|
#
?
Feb 5, 2020 17:51
|
|
|
fluppet posted:Requires you to know code commit/deploy/pipeline beanstalk opsworks and cloudformation and how to do HA RDS Ugh really? The only things out of that list I've ever used, or am likely to ever use, are cloudformation and RDS. My work is also looking for people to get AWS certs.
|
|
#
?
Feb 5, 2020 19:05
|
|
|
Cloudformation is a fantastic service that everyone who works in AWS should know, even if you don't use it actively, simply because the Cloudformation resource reference doubles as the best API documentation available for the platform.
|
|
#
?
Feb 5, 2020 19:14
|
|
|
Nomnom Cookie posted:Ugh really? The only things out of that list I've ever used, or am likely to ever use, are cloudformation and RDS. My work is also looking for people to get AWS certs. Do they specifically want the DevOps cert? If you aren't touching the code pipeline poo poo for work the Solutions Architect and/or SysOps certs may be more up your alley.
|
|
#
?
Feb 5, 2020 19:19
|
|
|
Junkiebev posted:Rancher owns incredibly hard Are you referring to Rancher itself or RKE? As I get time I'm getting further down the path of having clusters I can bring up quickly with Ansible so I've been looking at RKE occasionally.
|
|
#
?
Feb 5, 2020 19:24
|
|
|
Docjowles posted:Do they specifically want the DevOps cert? If you aren't touching the code pipeline poo poo for work the Solutions Architect and/or SysOps certs may be more up your alley. I'm on "the DevOps team", so... More to the point I do a lot of build and deploy work just not with the AWS services.
|
|
#
?
Feb 5, 2020 19:33
|
|
|
Nomnom Cookie posted:I'm on "the DevOps team", so... If they're looking for a certain APN partner-level status, likely they're after Pro certs, not DevOps Engineer specifically. Probably a good idea to check on that though, and whether they still need people with Associate-level certs as those are much more easily obtained. I just wrote this exam for the first time a bit over a week ago, having previously written(and failed) the SA Pro exam. I can guarantee you aren't going to walk into that and pass if you haven't used AWS in a year. While some general devops principles apply, obviously this is AWS-centric. DevOps Eng Pro goes deep, whereas SA Pro is wide on topics(or so I've heard it described, and would agree). I had questions on Step Functions, Lambda, API Gateway, Trusted Advisor, Macie, CodeBuild/CodePipelineCodeDeploy, Cloudwatch, Kinesis, DynamoDB, along with tons on S3/EC2/ASG/VPC/etc. Definitely get more info on what your org needs, then form a plan!
|
|
#
?
Feb 5, 2020 19:57
|
|
|
12 rats tied together posted:Cloudformation is a fantastic service that everyone who works in AWS should know, even if you don't use it actively, simply because the Cloudformation resource reference doubles as the best API documentation available for the platform. Cloudformation is awful and no one will ever convince me otherwise 
|
|
#
?
Feb 5, 2020 23:05
|
|
|
Everyone with extensive experience knows the measured, professional, evidence-based answer - everything sucks and you should burn it all down, quit, and look for a new job, all in no particular order.
|
|
#
?
Feb 6, 2020 03:28
|
|
|
I think most tools are pretty alright, as long as you don't do something silly like subscribe to fad-ops the mailing list and curl bash a new thing into production every 3 days, or decide that there can only ever be one tool and have to spend all of your time doing mental and operational gymnastics to provision ec2 instances with a chef cookbook or do a production deploy by chaining null_resource and depends_on. There will always be tools to glue together and you'll always have to glue them together with a mixture of automation and human process. With this in mind I judge a tool mostly by its ability to accomplish what I need it to, and for it to play nicely with other tools and arbitrary code. There's a sweet spot in there that obviously hugely varies based on your org, but in mine Terraform has a long way to go before it is "better enough" than a locally optimal piece of tech (ARM, Cloudformation, ROS, etc) to justify using, just as an example. IMO the most difficult thing about doing infrastructure work is finding ways to successfully work in large teams of varying skillsets and specializations while pissing off the fewest amount of people and keep everyone interested and engaged with the tool's development.
|
|
#
?
Feb 7, 2020 20:17
|
|
|
r3s being able to use sql in lieu of etcd is a huge deal in my opinion. Makes it something way easier to grok and for you to build into existing things that do backups.
|
|
#
?
Feb 7, 2020 20:25
|
|
|
I'm hoping somebody here can explain the process and experience for generating some documentation in GitLab (GitLab Pages) to be hosted with it. I have a bunch of Markdown that I can point to from the HEAD of master in my repository. However, I also want to at least: 1. Show a table I have to generate based on the current state of the repository. It serves as something like a catalog. 2. Post the API documentation some place. I believe I need to create a separate pages project for this and just link to it from my Markdown. Do I put stuff in my source project's .gitlab-ci YAML to create and publish that? How do I point it to the pages project? Can I do this without a separate pages project?
|
|
#
?
Feb 10, 2020 18:41
|
|
|
12 rats tied together posted:There will always be tools to glue together and you'll always have to glue them together with a mixture of automation and human process. With this in mind I judge a tool mostly by its ability to accomplish what I need it to, and for it to play nicely with other tools and arbitrary code. There's a sweet spot in there that obviously hugely varies based on your org, but in mine Terraform has a long way to go before it is "better enough" than a locally optimal piece of tech (ARM, Cloudformation, ROS, etc) to justify using, just as an example. We have a gently caress terraform "FTF" box on the whiteboard and everyone adds a tic mark when they discover something dumb, like the fact you can't use index on modules, along the auto-closed or stale as hell git issue with people begging for support.
|
|
#
?
Feb 10, 2020 22:41
|
|
|
So maybe it's just me but is EKS intentionally designed to be as opaquely unusable as possible or what I've done kops and gke and both I was up and running in almost a dangerously quick fashion. Less than 8 minutes in all cases. Even if you pick dangerously wrong settings at least you can get in and deploy a Prometheus helm chart or whatever EKS: After four stabs at using the GUI/wizards and the worker nodes not being able to talk to the master due to lack of NAT, I gave up and used some sort of pre-rolled cloud formation template to setup a new box, security groups and other poo poo. This generally takes at least 15 minutes, and even longer to tear down if you gently caress it up somehow After that step there's, I guess at least two ways to go 1. Run pre-rolled cloud formation template that involves a bunch of manual leg work to compare values in you eks cluster and hunt down a compatible Ami for your specific AZ.... Then paste it all in manually... Note that these nodes will never show up in the eks GUI despite being addressable via kubectl 2. Use the GUI and click through the wizard, this will take approximately 15 minutes for the nodes to attach... The plus side being that these nodes show up in the GUI as part of the cluster If we didn't have some kind of weird mandate from the guy who hates kubetnetes* that we use EKS I'd just stick with gke or even roll the dice and use kops over trying to deal with this Maybe it's just me *Maybe his hatred for kubetnetes is because he first touched it using EKS? That would be a valid reaction
|
|
#
?
Feb 12, 2020 08:49
|
|
|
We've used it, but I wasn't directly involved. I heard it was a pain and I haven't heard anything good from anyone else who's used it.
|
|
#
?
Feb 12, 2020 14:34
|
|
|
AWS interfaces are kind of baffling to me in general. I thought going from "I have some docker images" to "service up and running" would be quick and easy but did not find that to be case at all. Perhaps they've dogfooded so much that everyone there speaks/thinks in whatever that interface represents.
|
|
#
?
Feb 12, 2020 15:09
|
|
|
I think fargate is what they're trying to sell in that niche but I've never used it. If you are trying to go to docker images ->  the golden standard is gke and everyone else has a long way to catch up. EKS is a pretty lovely offering because you still have to manage all the hard poo poo yourself, and they make you pay a premium for it on top of the hassle.It's kind of like directory service which is another sub-par offering that is almost worse than just running the poo poo on your own. Bhodi fucked around with this message at 23:19 on Feb 12, 2020 |
|
#
?
Feb 12, 2020 17:43
|
|
|
Osmosisch posted:AWS interfaces There's your first problem. Don't use the AWS console to build or set up anything. It's just not worth the pain and frustration.
|
|
#
?
Feb 12, 2020 18:56
|
|
|
Blinkz0rz posted:There's your first problem. Don't use the AWS console to build or set up anything. It's just not worth the pain and frustration. Oh no i meant their overloaded web interface. I'd love any pointers because trying to pick this stuff up on my own has been extremely frustrating. Just as an example, mounting ssl key/cert pairs into the file system from a secrets volume is apparently rocket surgery or I'm just missing something obvious (or both).
|
|
#
?
Feb 12, 2020 19:16
|
|
|
I barely even remember setting up our first couple EKS clusters, they're just an ASGs with a cloudformation signal right? We have an ansible playbook that rolls all of our EKS clusters, including deploying org-wide defaults like namespaces and addons that should exist everywhere. It took like 20 minutes to write, but we're very practiced with pumping out ASGs with random signals and poo poo in them, so ymmv. The same playbook also deploys applications to clusters but this is optional and some teams prefer to use their own thing. Since ansible is driving during provisioning the playbook takes a yaml declaration of what types of worker ASGs should belong to each cluster and it handles the magic of putting the correct userdata blobs into each launch config, adding a new set of nodes to a cluster is like a 3 line PR that we barely code review. Everything is run through cloudformation change sets so we have a persistent record of exactly who ran exactly what, and exactly what the changes were going to be. Node maintenance is also absolutely trivial since you can adjust your scaling processes and termination policy to roll nodes in/out on demand. My only complaint about the whole thing was that we started out using jinja2 to render all of the manifests but halfway through a coworker wrote a kustomize ansible plugin so like half of the cluster components are kustomize templated and the other half are jinja2. Other than that it works perfectly and it like instantly increased the amount of engineers who can support a k8s cluster by 500% via "documentation as code". Blinkz0rz posted:There's your first problem. Don't use the AWS console to build or set up anything. It's just not worth the pain and frustration. The ECS interface is good though. The only thing EKS really does is fart out a master endpoint for you and have some logging hooks so I don't even know what they would put in there, to be honest. edit: Osmosisch posted:Oh no i meant their overloaded web interface. You pretty much have it, you can mount secrets directly to pods through a volume. If it seems like it sucks rear end and is way more confusing than just using normal config management, that's because it does, and it is. It will get exponentially worse as you add namespaces and clusters, too. 12 rats tied together fucked around with this message at 19:27 on Feb 12, 2020 |
|
#
?
Feb 12, 2020 19:18
|
|
|
Bhodi posted:I think fargate is what they're trying to sell in that niche but I've never used it. If you are trying to go to docker images -> Yeah we're actually ON gke right now and it works great but one developer does not like it so we're switching to EKS to support his needs Actually I think he just likes aws code pipelines and spinning up ec2 instances because that's just what he did for the 10 years before he worked here. What's the emoji for throwing away production ready GCP/GKE to appease your coworker and rewriting all your container tooling for EKS
|
|
#
?
Feb 12, 2020 21:08
|
|
|
Hadlock posted:Yeah we're actually ON gke right now and it works great but one developer does not like it so we're switching to EKS to support his needs 
|
|
#
?
Feb 12, 2020 21:28
|
|
|
|
|
#
?
Feb 12, 2020 22:28
|
|
|
Imagine using the AWS web dashboard to set up infrastructure in TYOOL2020
|
|
#
?
Feb 12, 2020 22:53
|
|
|
Gyshall posted:Imagine using the AWS web dashboard to set up infrastructure in TYOOL2020 My company built and maintains their own internal version of the aws web console with flask. This is how all the devs set up LTs/ASGs/iam/SGs/dns/environment variables. lol
|
|
#
?
Feb 12, 2020 23:55
|
|
|
Osmosisch posted:Oh no i meant their overloaded web interface. Yeah, their web interface is called "The Console". Methanar posted:My company built and maintains their own internal version of the aws web console with flask. lmao this is awful
|
|
#
?
Feb 13, 2020 01:26
|
|
|
Blinkz0rz posted:lmao this is awful As awful as having all cloud infrastructure maintained by a separate team time-shifted by about 10 hours, driven by service now tickets?
|
|
#
?
Feb 13, 2020 01:35
|
|
|
New Yorp New Yorp posted:As awful as having all cloud infrastructure maintained by a separate team time-shifted by about 10 hours, driven by service now tickets? Nope that's way worse!
|
|
#
?
Feb 13, 2020 01:43
|
|
|
Blinkz0rz posted:Yeah, their web interface is called "The Console". Its one of those things that kind of made sense in 2011. Terraform didn't come out until 2014. Packer 2013. CF might not have existed either. And I can imagine that the real aws console was even shittier back then. Having a UI purpose built to actually template and clone out aws resources, create DNS records etc seemed like the way to go 10 years ago. The environment variables thing is basically Consul, but again predates Consul by quite a bit.
|
|
#
?
Feb 13, 2020 05:36
|
|
|
Blinkz0rz posted:Yeah, their web interface is called "The Console". o h yeah of course it is 
|
|
#
?
Feb 13, 2020 12:48
|
|
|
Hadlock posted:What's the emoji for throwing away production ready GCP/GKE to appease your coworker and rewriting all your container tooling for EKS It kind of sounds like this guy is doing you a favor by giving you a good reason to refactor container tooling that is pinned to a cloud vendor instead of the k8s api. Sucks that he's using codedeploy though that thing is garbage. Methanar posted:Its one of those things that kind of made sense in 2011. Terraform didn't come out until 2014. Packer 2013. CF might not have existed either. And I can imagine that the real aws console was even shittier back then. Cloudformation released in early 2011, so it was just as bad of an idea then as it is today, unfortunately.
|
|
#
?
Feb 13, 2020 15:29
|
|
|
Every time you use the AWS console, Jeff Bezos eats a kitten
|
|
#
?
Feb 13, 2020 16:11
|
|
|
Remember: AWS hates you.
|
|
#
?
Feb 13, 2020 16:47
|
|
|
Bhodi posted:Remember: AWS hates you. I had a fun one earlier this week. In the encryption policy of S3 buckets AWS lets you reference the KMS key via key ID or key ARN. Of course what they don't tell you is that if you're trying to use that bucket from another account (e.g. sending VPC flow logs to a bucket in another account) you have to use key ARN otherwise poo poo just won't work! It makes sense I guess as the key ID wouldn't be resolvable to a principal in another account but then why even let you use anything other than key ARN!? Lost just over an hour to troubleshooting that bullshit. Edit: also yesterday I discovered that AWS::EC2::LaunchTemplate TagSpecification just straight up doesn't seem to work at all. If you've got any tags specified the resource will create successfully however when you try to launch an instance from the template it just spits out some bullshit error. Pile Of Garbage fucked around with this message at 19:29 on Feb 13, 2020 |
|
#
?
Feb 13, 2020 19:26
|
|
|
Pile Of Garbage posted:It makes sense I guess as the key ID wouldn't be resolvable to a principal in another account but then why even let you use anything other than key ARN!? I’m willing to bet there’s an enormous amount of poo poo they accidentally allowed, discovered was a bad idea, but oops, customers are using it, it can never be fixed!
|
|
#
?
Feb 14, 2020 16:07
|
|
|
12 rats tied together posted:Sucks that he's using codedeploy though that thing is garbage. As a former member of the CodeDeploy team, I wholeheartedly agree.
|
|
#
?
Feb 20, 2020 06:50
|
|
|
|
| # ? May 15, 2024 05:38 |
|
|
EkardNT posted:As a former member of the CodeDeploy team, I wholeheartedly agree. I'm sorry, it's not cool to be so dismissive about products in a relatively public place. There is definitely a type of customer that can get a huge amount of value from CodeDeploy and the other related services -- I would lump like Beanstalk and OpsWorks and poo poo like that in here too. I've worked at a bunch of places where we don't even have a git repository, and in those places being able to push an AWS migration as kind of an all-inclusive solution where you can click buttons and get a bunch of really important poo poo for "free", and a bunch of pretty decent support too, is a big deal. CodeDeploy is way better than hiring contractors to create a github account for you, for example. I think most people posting about this kind of stuff on the internet though are going to have their own code repository, their own opinions, and they're probably going to have their own set of deploy hooks and poo poo like that, so the intersection rate between "people posting about my work online" and "people who benefit from the thing I work on" would probably be pretty low. I would not have enjoyed working on CodeDelpoy. I'd totally work for the CloudFormation team though, that seems pretty dope. I interviewed for the ELB team too like 3-4 years ago and everyone I spoke to was super cool but I was extremely underqualified and we all knew it pretty much immediately. I'd probably work for that team though, seems like a cool product.
|
|
#
?
Feb 20, 2020 21:11
|
|