|
hey guys stop humbling me
|
#
?
Mar 9, 2023 07:52
|
|
|
|
| # ? Jun 4, 2024 19:05 |
|
|
Methanar posted:hey guys stop humbling me I didn't get a chance to read this thread today, it was a long rear end day.
|
|
#
?
Mar 9, 2023 07:54
|
|
|
Hadlock posted:The market just fell off a cliff after the first half of January Well, hilariously I've been offered an opportunity to completely change careers away from being a computer toucher, so I'm probably just gonna do that.
|
|
#
?
Mar 9, 2023 13:30
|
|
|
please, do it for all of us
|
|
#
?
Mar 9, 2023 14:14
|
|
|
minato posted:I'm intrigued about the Vault solution, but I'm bewildered about how that actually works. I grok'd all the nouns in that explanation, but I have no idea how they all connect together. This setup works for us because pipelines are required to store any needed secrets in Vault anyway, so it's not a big leap to using it for STS creds. 1: Configure AWS Secrets Engine w/ an IAM User (000000000:vault-user) that has sts:AssumeRole * (plus whatever tightening your want here). 2: Rotate the IAM User creds via the Secrets Engine so only Vault knows them. 3: Configure your target IAM Role(s) (111111111:target-role) with the trust policy sts:AssumeRole for "000000000:vault-user". 4: Create an AWS Secret Engine role "my-target-role" of credential_type assume_role for target roles. 5: Update / attach a policy to each pipeline's auth role to include "read aws/sts/my-target-role" that needs STS credentials for that role. 6: Pipelines auth to Vault, read "aws/sts/my-target-role" get creds w/ a default TTL, etc. Steps 4-6 can all be done in the IaC of your choice so a single pipeline has a declarative list of target roles it can generate credentials for. We do the whole thing in Terraform so anytime a dev spins up a new repo/pipeline that needs access to a set of IAM Roles they update the TF to include the repo and role list, go through PR, etc. and then they can get creds without any of the target roles needed to be updated. Since it's brokered by Vault you can also allow access to the "aws/sts/..." paths as needed from other workloads without them needing to directly interact with the target IAM Roles, which is handy for workloads that have no AWS identity (like GCP crap, local devs, etc) but have a way to auth to Vault. (We also have some standard templating and extensions in CI so the Vault request includes info about the CI pipeline so you can more easily trace back the assumed role in CloudTrail to the original request without directly mucking around in the Vault audit logs.)
|
|
#
?
Mar 9, 2023 15:32
|
|
|
jaegerx posted:Oh. I work with ebpf. Are you a bee?
|
|
#
?
Mar 9, 2023 16:22
|
|
|
Another dumb Kubernetes question. Say you have persistent storage using a particular CSI driver that doesn't like having multiple mounts of the same destination filesystem, each with different mount options, attached to the same pod. Is there a way to hammer in a different behavior, maybe by doing something like shimming in an overlayfs with noacl enabled? I'm not particularly creative here, and my brain is mostly trying to bruteforce EC2 solutions into the EKS world. Enterprise software is wonderful.
|
|
#
?
Mar 9, 2023 19:37
|
|
|
Vulture Culture posted:Another dumb Kubernetes question. Say you have persistent storage using a particular CSI driver that doesn't like having multiple mounts of the same destination filesystem, each with different mount options, attached to the same pod. Is there a way to hammer in a different behavior, maybe by doing something like shimming in an overlayfs with noacl enabled? I'm not particularly creative here, and my brain is mostly trying to bruteforce EC2 solutions into the EKS world. Can you go one layer up and block pod creation if it violates what you want?
|
|
#
?
Mar 9, 2023 20:44
|
|
|
freeasinbeer posted:Can you go one layer up and block pod creation if it violates what you want?
|
|
#
?
Mar 9, 2023 21:55
|
|
|
This is not the right place to put this, but kind of falls under "cloud engineering" https://protomaps.com/blog/serverless-maps-now-open-source example: https://protomaps.github.io/basemaps/examples/maplibre-basemap.html Provides medium-zoom level maps without google maps api which is kind of neat. And it's serverless. I've seen too many old hobbyist websites broken by the fact that their API key expired or they cancelled that feature because some web scanning robot got stuck in a loop and ate up all the free monthly api requests or something.
|
|
#
?
Mar 10, 2023 15:26
|
|
|
what does serverless mean here, because it doesnt sound like the regular use
|
|
#
?
Mar 10, 2023 15:50
|
|
|
The Fool posted:what does serverless mean here, because it doesnt sound like the regular use
|
|
#
?
Mar 10, 2023 15:55
|
|
|
I didn't click any links.
|
|
#
?
Mar 10, 2023 16:00
|
|
|
You're referencing static assets hosted on, probably, cloudflare, PNG or SVG image tiles and location information of stuff (like city names and major street names) at a size of ~2.9GB and then javascript pulls down the relevant ones glues the SVG together seamlessly and overlays the location information on top in the right spots. Even if you used a private API you'd still have to run a DB and server of some sort to serve up this data (which might save money vs just using google maps API since gmaps api can run $100-thousands of dollars a month depending on usage) I worked at a medical services company and you had to pick your prefered pharmacy to pickup your drugs, usually 3-4 options across a city. You don't need super zoom for that, this would have been an easy way to cut $12k/year off our cloud hosting budget
|
|
#
?
Mar 10, 2023 16:04
|
|
|
The Fool posted:I didn't click any links. N/P combo
|
|
#
?
Mar 10, 2023 16:04
|
|
|
Hadlock posted:You're referencing static assets hosted on, probably, cloudflare, PNG or SVG image tiles and location information of stuff (like city names and major street names) at a size of ~2.9GB and then javascript pulls down the relevant ones glues the SVG together seamlessly and overlays the location information on top in the right spots. Even if you used a private API you'd still have to run a DB and server of some sort to serve up this data (which might save money vs just using google maps API since gmaps api can run $100-thousands of dollars a month depending on usage) That's clever, thank you. Hadlock posted:N/P combo 
|
|
#
?
Mar 10, 2023 16:17
|
|
|
The Fool posted:I didn't click any links. Decent thread title
|
|
#
?
Mar 10, 2023 17:37
|
|
|
The Fool posted:I didn't click any links. congrats you have completed the annual mandatory security compliance training module. We hope the 2 hours of unskippable slides on the difference between phishing and spear phishing have been informative (it's not just my company that makes people do this right?)
|
|
#
?
Mar 10, 2023 17:43
|
|
|
Docjowles posted:congrats you have completed the annual mandatory security compliance training module. We hope the 2 hours of unskippable slides on the difference between phishing and spear phishing have been informative I've definitely had the same training. But hey, the CEO is really happy with the reports that we've all completed the required training, and wants to get everyone some mobile gift cards. Can you go grab $250 worth of Apple and Google Play cards, and send me the numbers? I'll send them out to the team. Thanks!
|
|
#
?
Mar 10, 2023 17:47
|
|
|
Blame the lawyers with their pedantic BS causing so much lost productivity across the world with all this drat training that hasn't demonstrably shown much preventative benefits. Even worse is when people that understand tech start to try to appease lawyers that have basically zero idea because the technical people got strong-armed. It was really freakin' funny when as a security company I had to take that course. I heard before though that there was a great spear phishing exercise that almost all the engineers failed when the systems lead sent out an e-mail with the phishing link in it with the subject "Free pizza on Thursday" and "sign up <here>" That's the kind of ingenuity and creativity you need in your training materials to keep people on their toes and engaged if you ask me.
|
|
#
?
Mar 10, 2023 17:55
|
|
|
JehovahsWetness posted:This setup works for us because pipelines are required to store any needed secrets in Vault anyway, so it's not a big leap to using it for STS creds.
|
|
#
?
Mar 10, 2023 20:58
|
|
|
Docjowles posted:congrats you have completed the annual mandatory security compliance training module. We hope the 2 hours of unskippable slides on the difference between phishing and spear phishing have been informative Hey that is my afternoon today
|
|
#
?
Mar 10, 2023 21:19
|
|
|
Ours lets you skip slides but because it's forty different 30 second videos I've found it's less distracting to just leave it playing (muted) on a second monitor and occasionally hitting next than to actually bother skipping through it.
|
|
#
?
Mar 10, 2023 21:25
|
|
|
just completed “Anti-Vishing” module not opening emails and not answering phone calls stays winning, baby
|
|
#
?
Mar 10, 2023 21:59
|
|
|
I am absolutely done with terraform and I'm gonna try to move all my poo poo to pulami. gently caress you hashicorp.
|
|
#
?
Mar 11, 2023 04:48
|
|
|
jaegerx posted:I am absolutely done with terraform and I'm gonna try to move all my poo poo to pulami. gently caress you hashicorp.  new thread title
|
|
#
?
Mar 11, 2023 05:14
|
|
|
jaegerx posted:I am absolutely done with terraform and I'm gonna try to move all my poo poo to pulami. gently caress you hashicorp. I am intrigued by Pulumi as a Third Way. Because terraform does kind of suck, but I also hate the CDK since at the end of the day it's still loving CloudFormation and also the documentation is impenetrable*  * for the love of god Amazon do something to make your deprecated CDKv1 not the top hit for every search. Insert ASCII dicks that only Google's crawler can parse, I don't care, just figure it out Docjowles fucked around with this message at 05:20 on Mar 11, 2023 |
|
#
?
Mar 11, 2023 05:17
|
|
|
The thing I hate about pulumi is that it does not provide an opinionated method of defining your resource graph. You literally can have teams implement the same poo poo over and over in their own lil languages. Sure, you COULD get around this...but be realistic. Like I get having the deployment code be as close to the product code as possible with ownership, but when you start to get to larger segmentation of business units and organizations that you're trying to "synergize" (heh), it becomes a living hell just having the conversation cross teams on how you normalize the poo poo in a sane manner. Edit: Going to edit to say, that while I do not see pulumi as a replacement, I do see it as a very good addition to the stack while having rigid resource definitions provided by TF and using something like pulumi to fill the gap of end to end testing on actual resources with more depth than you get with terratest.
|
|
#
?
Mar 11, 2023 06:24
|
|
|
Life is too short to be using yaml and HCL as programming languages
|
|
#
?
Mar 11, 2023 06:34
|
|
|
Methanar posted:Life is too short to be using yaml and HCL as programming languages You're doing it wrong. If you can seriously think that it's a sound idea to have your base infrastructure resource definitions defined by what ever language is hot at the moment, I got another think coming for ya. We've literally been here before...it didn't work great in the past.
|
|
#
?
Mar 11, 2023 06:41
|
|
|
its fair to put HCL on blast because it has serious, deep-rooted problems. terraform too. putting yaml on blast otoh is labeling yourself as not having read the specification or the documentation for your parser. this markup format supports everything you could possibly need, it is XML 2, and if you don't like it its because you're holding it wrong
|
|
#
?
Mar 11, 2023 06:53
|
|
|
drunk mutt posted:You're doing it wrong. Interesting topic but I refuse to accept "this thing is worse, which is better at a large enough org size" as valid reasoning, is that where you're going with this? Have you seen/used the cross-language code gen features for sharing pulumi code? I have not used them "at scale" but it's not clear to me where they would break down but that e.g. CloudFormation YAML would chug along fine.
|
|
#
?
Mar 11, 2023 06:59
|
|
|
12 rats tied together posted:Interesting topic but I refuse to accept "this thing is worse, which is better at a large enough org size" as valid reasoning, is that where you're going with this? Have you seen/used the cross-language code gen features for sharing pulumi code? I have not used them "at scale" but it's not clear to me where they would break down but that e.g. CloudFormation YAML would chug along fine. While I have not seen them "at scale", I have seen them attempt to mature beyond scope of the individual team and it is not a fun chore. This is why I added the "sure you COULD make it work", but seriously, why do that when you can lean on a technology that does one aspect of the tooling well and leverage the newer to expand on it? Like, this is literally what we've been doing with TF since I've been doing TF... Hell, we didn't have shared state files..so we used a third-party product...then they recognized that same product gave the aspect of workspaces...now we have workspaces. The point I'm working towards, is that pulumi is a new tool into the realm and is starting to get hype. While I can see value into the tooling, there isn't a very sound reasoning of replacing a rigid definition of resources in a singular known configuration language be it HCL or YAML, which even makes me consider "oh yeah, replace everything in what ever language people want" as an ideal thought.
|
|
#
?
Mar 11, 2023 07:41
|
|
|
My experience has largely evolved to the opposite I guess. The biggest problem I see for greenfield infrastructure-code is adoption rate and then later, lack of code sharing or a surplus of the wrong kind of code sharing. Eventually this metastasizes to "distributed holding it wrong" where one team can make a clerical error in a module caller relationship that causes DNS reflection attack sized work amplification in all of the other teams that use their code needing to update module versions and adjust their plumbing due to a language limitation thats primarily caused by HCL being invented by a company whose largest infrastructure at the time was the vagrant docs. I think by this point most people ITT are on board with the idea that HCL is "simply bad" and we don't need to go into the technical minutiae. IMO, IME, Pulumi is "the answer" for this generation of the tooling because it has solutions to both of these problems. Adoption rate: meet people where they're at, in their preferred IDE, in the language they already know. It's not hard to get a developer excited about cutting out the SRE team because they can write Java code that manages their infrastructure. Code sharing: we know how to share Java code, it's a solved problem. What about cross-language? Well they use some code gen horseshit that is honestly pretty cumbersome, but it's pretty easily translatable to a devex concern. Feature team doesn't have the GCP expertise to figure out why their C# module is bad? Publish it as a python package and let ops look at it. Problem solved. I pretty much only work as an IC, and I only ever plan to work as an IC, which I think explains a lot of my perceptions here. I'm willing to acknowledge that things look different from a director role, but I doooon't think that "worse is better" applies here.
|
|
#
?
Mar 11, 2023 07:58
|
|
|
Having been responsible for hiring both SREs and devs I think it depends a bit upon the kinds of people you can potentially hire + train and the scope of the IAC elements in question over time when it comes to cross-cutting concerns like languages used in the codebase. I have some conservative opinions about introducing more and more languages and tooling for an already high cognitive load situation and would prefer One Language to Rule Them All to some extent. With some broad strokes of generalizations and lack of nuance, I would recommend Pulumi over Terraform in a heartbeat if your hiring pipeline looks more like software engineers that happen to do some ops-stuff than ops folks that can code because I've seen what happens repeatedly when you hand day-to-day developers Terraform or CloudFormation as work output. If you're one of the lucky ones that can command a huge hiring pipeline with lots and lots of really talented folks (read: you're one of the Big Boys) then it doesn't matter in a sense because you probably already have written your own alternative of sorts internally. For lots of folks in the middle they use Terraform because the community tooling support absolutely destroys Pulumi still and probably always will. Developers writing IAC is a bit of an organizational smell to me depending upon a lot of factors I can't really enumerate now. There's already a ton of responsibilities developers have and letting them focus on features and the software that kinda keeps the company able to make money is a decent separation of concerns. The shared responsibility v. separation of concerns line for "make my software run fine in both dev and prod, dammit" optimality is somewhere around developers writing Helm charts and advising on the Terraform necessary when using cloud provider services like proprietary DBs (insert comedy "write Helm charts deployed via Terraform" option that is literally what my company does). Maybe I can be more opinionated, reject both, and suggest K8S crossplane as a go-to but then we get into the K8S can of sandworms and loss of the community ecosystem compared to Terraform which could make it a worst-of-all-options situation rather than best-of-all middle.
|
|
#
?
Mar 11, 2023 10:27
|
|
|
At our company we're really enjoying crossplane, and it's got a lot better since we started a year ago. Of course, we had the advantage of being a tiny company (5 devs + 5 AI) doing greenfield IaC and no one had any experience so it was easy to choose something new once we were convinced it would be easier and probably gain adoption. That also means the chance we made the wrong choice approaches 100%, but it's been fairly smooth. Also a niche product with few enough users I can tell you all their names, so it's a completely different scale than what you guys are all doing. But I recommend at least consider it for some part of your infrastructure and try a POC. IaC with GitOps and reconciliation loops is pretty nice. We have a composite resource that creates a dev environment with a bucket, cloudsql with user/db created, both populated with sample data, and a service account with the key available as a Secret in their team's namespace. They just need to merge an 8 line yaml, they love it. I guess this is the benefit of any IaC but it completely changed my life. The last headache left for me is the lack of iteration or optional resources so if your new composite resource needs 2 buckets instead of 1 you have to make a whole new composition instead of adding an nBuckets field. Silly example but you get the idea.
|
|
#
?
Mar 11, 2023 12:06
|
|
|
What if your hiring pipeline is empty because all your jobs are being camped in by people who would have trouble with Duplo blocks, which IaC tool makes sense there.
|
|
#
?
Mar 11, 2023 13:50
|
|
|
FISHMANPET posted:What if your hiring pipeline is empty because all your jobs are being camped in by people who would have trouble with Duplo blocks, which IaC tool makes sense there. Depending upon the actual demands to scale as a business eventually the skill + attitude bar to even have a chance to keep up with modern practices won't be enough and you'll have to sink or swim as an org. If your org's fundamental business function is to babysit 200 - 1k servers and it's basically static with not a great deal of variation or demands to increase rates of change it doesn't matter what tools you're using.
|
|
#
?
Mar 11, 2023 14:33
|
|
|
Fun fact, working for a 150 year old public research institution means that no matter how badly IT fails the "business" we can never truly fail and force any kind of reckoning or change. Anyways I'm curious about the problems with Terraform and HCL. I suspect we'll always be fractured and independent enough that problems at scale will never ever really come up.
|
|
#
?
Mar 11, 2023 15:18
|
|
|
|
| # ? Jun 4, 2024 19:05 |
|
|
I’ve been on a greenfield TF/TFC project for two years that has probably a thousand+ workspaces for more individual teams than I can recall, it works fine. TF is TF, the warts are what they are but it’s way better than most of the alternatives. The project isn’t a top down deal and folks are free to chose Pulumi or native cloud IAC or roll their own and so far everyone has chosen TF unless you run into a situation where you use some kind of scripting in a pipeline to work around some weird bullshit with the provider or TF itself.
|
|
|
#
?
Mar 11, 2023 15:34
|
|
