|
pretty much all of vault's announcements this year have been "we added a feature to our piece of poo poo cloud platform, please give us money"
|
#
?
Oct 13, 2023 09:56
|
|
|
|
| # ? May 16, 2024 19:29 |
|
|
vanity slug posted:pretty much all of vault's announcements this year have been "we added a feature to our piece of poo poo cloud platform, please give us money"
|
|
#
?
Oct 13, 2023 10:14
|
|
|
I wonder how many new Vault customers they're still getting. A lot of people I've been talking to have figured out that after a few years, they aren't using any of its differentiating features at all. Even for hybrid companies, most of the secrets functionality is handled better by something like AWS Secrets Manager.
|
|
#
?
Oct 13, 2023 16:29
|
|
|
What's the best way to get Prometheus metrics out of ECS services into Grafana Cloud without using Cloudwatch? We have a couple of services with auto-scaling rules spread across a couple of clusters. Each task is a .net core web service. I want to expose some custom metrics from these services so we can track a bunch of stats for support. It's easy to scrape the metrics endpoints when there's only one instance of each task running, but if it scales up then I don't know how I'd scrape each distinct container from Grafana Cloud's Prometheus. Is my best bet to add Grafana Agent to my containers and use remote_write to push the metrics to Prometheus instead of trying to scrape? As you can probably tell, we are a small dev team who are not at all mature in terms of cloud engineering capabilities.
|
|
#
?
Oct 16, 2023 15:06
|
|
|
We use New Relic instead of Prometheus, but yeah the pattern is to run an OpenTel collector agent as a sidecar to your .NET app's container. That's not the same as installing the agent in your Dockerfile, which I suspect is what you were thinking. AWS has a sample task definition. If you're using Fargate your options are pretty limited and this one is probably the least poo poo.
|
|
#
?
Oct 17, 2023 07:44
|
|
|
Extremely Penetrated posted:We use New Relic instead of Prometheus, but yeah the pattern is to run an OpenTel collector agent as a sidecar to your .NET app's container. That's not the same as installing the agent in your Dockerfile, which I suspect is what you were thinking. AWS has a sample task definition. If you're using Fargate your options are pretty limited and this one is probably the least poo poo. Thanks, that looks helpful. Yeah we are using fargate, and I was originally looking at installing the agent in my dockerfile… lots more research to do!
|
|
#
?
Oct 17, 2023 22:37
|
|
|
beuges posted:What's the best way to get Prometheus metrics out of ECS services into Grafana Cloud without using Cloudwatch? Do you not use the Prometheus Operator and have a per-cluster prom? If you do, it’s just a matter of adding a service monitor to your deployment manifests. If you don’t, it’s harder.
|
|
#
?
Oct 18, 2023 05:09
|
|
|
Derp - thought I saw eks there
|
|
#
?
Oct 18, 2023 05:10
|
|
|
What is the pitch the founder of a company wants to hear when doing the final sign off on a new hire Seems like DevOps is old hat, everyone wants "devX" now as that has greater more measurable impact on developer experience and velocity
|
|
#
?
Oct 18, 2023 06:01
|
|
|
Hadlock posted:What is the pitch the founder of a company wants to hear when doing the final sign off on a new hire DevX quality is a direct and linear function of how good one’s automation/tooling/platform is/are, and DevOps is responsible for delivering that at scale, in an easily-digestible format. Call it whatever you want, but that’s the gig (in my view)
|
|
#
?
Oct 18, 2023 07:05
|
|
|
its only devx it if comes from the soma district otherwise its sparkling release engineering
|
|
#
?
Oct 18, 2023 13:34
|
|
|
Hadlock posted:What is the pitch the founder of a company wants to hear when doing the final sign off on a new hire Generally, DevOps/releng/DX/whatever is something they explicitly don't care about under a dozen employees or so. In basest terms possible, someone making devs 10% more productive YoY needs to do that 10 times over to earn their salary. Claim much higher than that and you'll butt heads with the piss and vinegar seed round devs who, in their heads, sure aren't working at an early startup because they're low performers With my last company, I was a) versatile, and they could give me plenty of stuff to do if there wasn't enough high value ops/RelEng work b) able to fit into the company-culture needle they were threading c) a person you'd much rather have and not need just yet than need and not have later Vulture Culture fucked around with this message at 19:27 on Oct 25, 2023 |
|
#
?
Oct 25, 2023 19:22
|
|
|
Vulture Culture posted:Depends on the size of the company, their trajectory towards a scalable product/market fit, broader hiring goals, etc. At small shops you just get crafty with defining SLO/SLIs and just focus on getting reportable DORA metrics or something similar. A lot of times this is just something stupid simple like making things actually done in a GHA workflow or something stupid simple like that. It doesn't scale out well, but hopefully you would be altering objectives/indicators as time progresses to be sensible for representing the "DX".
|
|
#
?
Oct 25, 2023 23:33
|
|
|
Would you go for OpenTofu or stick to Hashicorp Terraform if you had the choice to start a greenfield project today?
|
|
#
?
Oct 26, 2023 13:36
|
|
|
"It depends" If this is for a larger enterprise that is going to care about support contracts, and other enterprise nonsense, I'd still go with terraform If this is for a startup or a personal project and I'm ok with all the baggage around the open source community, probably tofu
|
|
#
?
Oct 26, 2023 14:48
|
|
|
Actually I'd probably stick with the pre-license change version of terraform until things have a chance to shake out a little bit more, you aren't missing anything in 1.6.
|
|
#
?
Oct 26, 2023 14:50
|
|
|
For AWS nowadays, I'd probably just use CDK and sidestep the hybrid cloud nonsense. There's no compelling need to get an entire business caught in this crossfire right now
|
|
#
?
Oct 26, 2023 15:05
|
|
|
Terraform is good for static long lived infrastructure and not a lot else, which is almost always the least interesting component of your project. If you can avoid using TF at all, do that. If you have to use it just use terraform. The licensing changes are both reasonable and incredibly minor, opentofu has very little reach and will be significantly less supported going forward, and using it amounts to pointless signalling for a stupid cause while increasing your environment’s risk and complexity.
|
|
#
?
Oct 26, 2023 15:09
|
|
Cat Army
|
Oh, yeah, I forgot. If you want to pick something without big-brand backing right now, look at System Initiative. It's got Adam Jacob from Chef behind it and, while it doesn't fulfill a useful need for me, it seems like it could be really cool for Terraform-type use cases
|
|
#
?
Oct 26, 2023 15:23
|
|
|
Appreciate the thoughts. It’s for a small department in a large corporate. It’s only for provisioning the base infra layer. Anything application related is done through git and ci/cd pipelines. I’m not sure why I didn’t think of cdk. I’m comfortable with both tf and cdk. One of the two other people I work with knows cdk but none of them know tf. Not sure if Pulumi is worth looking into for an env this small.
|
|
#
?
Oct 26, 2023 20:32
|
|
|
imho pulumi is strictly an upgrade over the cdk, for the same reasons terraform might be an upgrade over cloudformation, except hcl isn't there to get in the way this time
|
|
#
?
Oct 26, 2023 20:42
|
|
|
Vulture Culture posted:Oh, yeah, I forgot. If you want to pick something without big-brand backing right now, look at System Initiative. It's got Adam Jacob from Chef behind it and, while it doesn't fulfill a useful need for me, it seems like it could be really cool for Terraform-type use cases this thing owns actually. i might try and pitch OSS version for my team at current job. thanks for sharing it!
|
|
#
?
Oct 27, 2023 18:35
|
|
|
Feeling a little frustrated in my job search as pretty much any job I want is going to involve at least some Kubernetes and Terraform, but my org is so backwards that we don't actually use either of those, so I'm left having to convince them that while I may have only a little exposure now, that I could quickly become an expert if exposed to it.
|
|
#
?
Oct 27, 2023 22:37
|
|
|
Vulture Culture posted:Oh, yeah, I forgot. If you want to pick something without big-brand backing right now, look at System Initiative. It's got Adam Jacob from Chef behind it and, while it doesn't fulfill a useful need for me, it seems like it could be really cool for Terraform-type use cases I know they have better things to do right now like develop their product but man that is a weird looking website. It’s in the uncanny valley between modern design and like early 2000s styling FISHMANPET posted:Feeling a little frustrated in my job search as pretty much any job I want is going to involve at least some Kubernetes and Terraform, but my org is so backwards that we don't actually use either of those, so I'm left having to convince them that while I may have only a little exposure now, that I could quickly become an expert if exposed to it. I’ll cop to addressing this issue with some good old fashioned resume driven development early in my career. Puppet and config management were the hot topic and I had zero exposure. So I found a reason to bring it into our company and start hacking on it. Not a bullshit reason, it was a legitimate project solving a real problem that puppet was a good match for. My boss didn’t particularly care about the technology choice so I ran with it.
|
|
#
?
Oct 27, 2023 23:14
|
|
|
Not sure if this is the best thread for this question, but I'm sure there are a lot of container/docker experts in here so I thought I'd ask. I'm trying to run a Docker container using the bitnami/wordpress image. If I set up the database on another container using a compose file, the WordPress container happily connects to the database just fine. But if I try to put a remote database host in there it can't connect to it, it's like the container has no internet access. I don't know enough about the networking side of containers to understand what's going on and unfortunately the image has been stripped of any helpful tooling so debugging it is tricky. What I have done is built a separate container with code inside it that DOES connect to the database just fine, just to validate that it's not some general Docker or database configuration issue, it's something specific to this image. My suspicion is that maybe this is deliberate as a security feature, and I'm supposed to follow some specific pattern for allowing this remote connection, but I can't find any documentation on it or even any evidence that other people are having this problem.
|
|
#
?
Oct 27, 2023 23:55
|
|
|
Do the logs show an attempt to setup that connection? Are you running this locally, in a k8s cluster, in fargate...?
|
|
#
?
Oct 28, 2023 00:51
|
|
|
Trapick posted:Do the logs show an attempt to setup that connection? Are you running this locally, in a k8s cluster, in fargate...? I got it sorted finally. Turns out it wasn't as general as I thought, it did actually have database access it just couldn't connect to THIS database. The answer was setting the env variable WORDPRESS_ENABLE_DATABASE_SSL to "yes", it's "no" by default.
|
|
#
?
Oct 28, 2023 03:27
|
|
|
I have weird conflicting feelings about how many issues are solved by such a simple solution as switching “no” to “yes” or whatever. But I also just lost my job so I’m especially tuned into the capricious nature of the universe at the moment. Which is really just to say if any of y’all need an AWS/Terraform person, I’m your guy. I’ll probably bump the job seeking thread on Monday but for the time being, oof.
|
|
#
?
Oct 29, 2023 01:34
|
|
|
Docjowles posted:I know they have better things to do right now like develop their product but man that is a weird looking website. It’s in the uncanny valley between modern design and like early 2000s styling
|
|
#
?
Oct 30, 2023 17:19
|
|
|
Docjowles posted:I’ll cop to addressing this issue with some good old fashioned resume driven development early in my career. Puppet and config management were the hot topic and I had zero exposure. So I found a reason to bring it into our company and start hacking on it. Not a bullshit reason, it was a legitimate project solving a real problem that puppet was a good match for. My boss didn’t particularly care about the technology choice so I ran with it. I was able to do some TF in a project I was working on, but that project got shutdown and also I got fired, so now I'm left stuck without any opportunity for resume driven development, just coasting on what I've got. Maybe I'm a little salty because I had an interview on Wednesday and then Friday found out I didn't get the job and I felt like it was my lack of Kubernetes and Terraform experience that did me in. I'm also in a weird place with Azure where I'm pretty strong at the particular pieces I've touched but less so on pieces I haven't touched. So I have to convince interviewers that I can pivot the experience I do have into quickly learning whatever specific technologies they need. Necronomicon posted:I have weird conflicting feelings about how many issues are solved by such a simple solution as switching “no” to “yes” or whatever. But I also just lost my job so I’m especially tuned into the capricious nature of the universe at the moment. Which is really just to say if any of y’all need an AWS/Terraform person, I’m your guy. I’ll probably bump the job seeking thread on Monday but for the time being, oof. 
|
|
#
?
Oct 30, 2023 17:45
|
|
|
FISHMANPET posted:I was able to do some TF in a project I was working on, but that project got shutdown and also I got fired, so now I'm left stuck without any opportunity for resume driven development, just coasting on what I've got. Maybe I'm a little salty because I had an interview on Wednesday and then Friday found out I didn't get the job and I felt like it was my lack of Kubernetes and Terraform experience that did me in. I'm also in a weird place with Azure where I'm pretty strong at the particular pieces I've touched but less so on pieces I haven't touched. So I have to convince interviewers that I can pivot the experience I do have into quickly learning whatever specific technologies they need. Cloud is too large to be strong everywhere. As long as you have a few of the most common services covered you’re probably good for most of the jobs you’re aiming for. If you want to get better at k8s, get the Kodekloud courses on udemy, they come with lab environments and are completely worth it for that alone. Besides that they also happen to be the best entry level k8s courses I’ve encountered. 12-15 bucks and doing ckad amd cka will get you enough knowledge to talk ablut during interviews. As for TF, I think Andrew Brown just did an entry level course which he’s going to post on freecodecamp. No personal experience but I heard he puts out great content.
|
|
#
?
Oct 30, 2023 19:34
|
|
|
Necronomicon posted:I have weird conflicting feelings about how many issues are solved by such a simple solution as switching “no” to “yes” or whatever. But I also just lost my job so I’m especially tuned into the capricious nature of the universe at the moment. Which is really just to say if any of y’all need an AWS/Terraform person, I’m your guy. I’ll probably bump the job seeking thread on Monday but for the time being, oof. It can be very frustrating! Especially on images with no tooling to actually interrogate the state of the container. But on the other hand it was such a relief to get the problem solved with such a simple solution even if it did take an annoying amount of digging to get there. As for the rest of your post, sorry to hear that, friend. I'm assuming you're in the US, but if you're in Melb I can maybe help? putin is a cunt fucked around with this message at 10:30 on Oct 31, 2023 |
|
#
?
Oct 31, 2023 10:27
|
|
|
putin is a oval office posted:As for the rest of your post, sorry to hear that, friend. I'm assuming you're in the US, but if you're in Melb I can maybe help? Yeah, I'm in Massachusetts, about two hours west of Boston. I appreciate it though.
|
|
#
?
Oct 31, 2023 19:22
|
|
|
I'm trying to get less dumb so I set up a kubernetes cluster on some spare hardware at home so I can get a better understanding of how it works internally. After much flailing around I got all the basics working, ingress works from any machine on my lan and I can access my hello world destination. So setup done. But one thing that I absolutely could not figure out was haproxy as the ingress controller. I could talk to haproxy, get its status page and get a 404 error trying to hit 80 or 443. But no matter what I tried I could not get it to proxy to my hello world echo server. I did find a note at the bottom of some random article that haproxy has issues if you don't install with helm which I did not use.. I downloaded the yaml manifest and applied it myself. Does that sound plausible? Or am I just bad at writing yaml? nginx worked first try with zero issues.
|
|
#
?
Nov 8, 2023 03:06
|
|
|
helm is just a means to template, apply, and rollback yaml files at the end of the day. There’s nothing magical about using it, but it tends to be the tool de jour for third party apps since someone other than you gets to maintain the manifest files.
|
|
#
?
Nov 8, 2023 03:20
|
|
|
That's kind of why I avoided helm in this case, I didn't want to be shielded from the nuts and bolts since this is all educational. Next time I'm puttering I'll try haproxy with helm and see if I get different results, in the meantime I figured I'd ask around.
|
|
#
?
Nov 8, 2023 03:52
|
|
|
Kubernetes is moving away from ingress controllers Unless you have a very specific reason, stick with the nginx ingress controller
|
|
#
?
Nov 8, 2023 08:53
|
|
|
You talking about the availability of the gateway api? I'll definitely play around with that next but I assume it's gonna be years before ingress is is gone.
|
|
#
?
Nov 8, 2023 14:51
|
|
|
xzzy posted:You talking about the availability of the gateway api? I'll definitely play around with that next but I assume it's gonna be years before ingress is is gone. Look at Gateway seriously if you're using service mesh. Ingress is fine and will continue to be fine for exposing cluster services on the network.
|
|
#
?
Nov 8, 2023 15:52
|
|
|
|
| # ? May 16, 2024 19:29 |
|
|
I need to manage kubernetes minor version updates (I.e. 1.27-1.28) across 3 clouds and 60 k8s clusters. somewhat to my surprise, it seems like nobody has built a tool to do this. Both GCP and Azure have auto updates, but AWS EKS does not. Doing this all in terraform sucks regardless. I basically want to: - take as input a target minor version - check for deprecated APIs in use using an existing tool like kubent - identify the latest available patch version for the target minor version (handles both patch upgrades and minor version upgrades) - update any EKS addons (AWS clusters only) - update the control plane - update one node pool at a time if an update isn’t ongoing. My initial thought was to just do this all in gitlab CI, but since node pool updates can take hours or days the orchestration got ugly fairly quickly. Accordingly, I’m leaning towards writing my own service to handle this. I’ll probably just use a lambda function or three and use dynamodb to store the couple bits of state I need on the cheap. Before I do though, am I reinventing the wheel here?
|
|
#
?
Nov 9, 2023 01:28
|
|