|
we looked at migrating from tfe to tfc and the change in pricing would have made it significantly more expensive
|
# ? Mar 14, 2023 02:16 |
|
|
# ? May 22, 2024 11:07 |
Yea I don’t pay the bills, TFC ain’t cheap. Client has an organizational directive to use SaaS first, PaaS second, and IaaS/on premises as an absolute last resort. So unless someone’s SaaS implementation is horrible, it leads to stuff like using TFC and kinda shrugging at the cost Edit: they’re interesting cause drat near everything was already containerized which IME is a bit rare for a company in their vertical and as old as it is. Makes shifting workloads to AKS insanely easy
|
|
# ? Mar 14, 2023 02:19 |
|
12 rats tied together posted:CloudFormation is really good at this but is unintuitive because the way to do it is to use none of the features for it. StackSets are garbage, cross-stack references are also garbage. Control Tower shouldn't exist.
|
# ? Mar 14, 2023 02:20 |
|
The Fool posted:You say the plumbing is error-prone like it's not automated and abstracted away. JavaScript code:
Additionally, what happens when the networking module changes? My root state here is full of load-bearing string literals: "private_subnet_ids". "bucket_arn". If these ever become wrong my app suddenly starts failing to plan and I have no idea why, I need to trace the change up to whoever published my module, maybe find an updating guide or release notes, etc. This is a lot of work, risk, and ongoing maintenance for the really simple ask of "using the subnet that I'm supposed to". CloudFormation sucks at this too, to be fair. The only benefit it has is that you dont have to write HCL, but Terraform also has that, so it's not really a W. A better way to solve this problem is using a programming language, because then you can write factory methods. You can use dependency injection. You can use inheritance and composition mechanism unavailable in HCL. A lot of people do this already and just serialize to terraform json, or cloudformation json/yaml, but it's still kind of a lot of work. This is where Pulumi comes in, which has a standard library full of useful stuff like enums for every instance type so its impossible to run an apply and get "c5.8xlarge is not a valid instance type" errors. necrobobsledder posted:A large part of the modularity problems in CF come directly from how exported variables work and how there's basically no such thing as namespaces enforced by the system. It's too flexible in a sense.
|
# ? Mar 14, 2023 02:54 |
I’m on my phone so can’t effortpost, but I think the difference in opinions here might be that if you’re fully leaning into TFC/TFE some of that isn’t really an issue. And for_each loops, dynamic blocks, and other improvements made to TF actually work really well in the latest TF versions.
|
|
# ? Mar 14, 2023 03:03 |
|
I do think a lot of this stuff is "better now", in terraform. Like I'm pretty sure you can output an entire module by just declaring an output with a name and a value of "module.name", which gives anyone who calls you a big fat json blob containing all the outputs of your child module. You used to have to thread these through individually by name, and it sucked rear end, especially if any of them were conditionally created, because terraform still sucks poo poo at null and does random poo poo any time you feed it one. You also used to not be able to output anything except for string values, so you had to join your subnet ids with a pipe and then split them in your caller, but again I'm also pretty sure you can output complex types. Really though, what I want to be able to do, is define e.g. a bunch of different types of ways of routing, and then I want a module that takes a RoutingType parameter, and I can just pass it 12RatsCoolVpnRoutingBehavior, and the module will pull it in and call "build()" on it, and everything just works. And then I can be like hey all these types of module can take any sort of RoutingBehavior from this enumerated list of different types of routing behavior, which I've imported from a package that was built by my devops team, and they've either baked in the subnet ids or theyve written some code to figure out the right ones, and I never have to care or think even a little bit about it except for typing "import companyname.infra.routing" and picking one in an intellisense drop down menu.
|
# ? Mar 14, 2023 03:05 |
|
12 rats tied together posted:plumbing I mean, for that specific scenario we just read the outputs from the hub network workspace
|
# ? Mar 14, 2023 03:29 |
|
What's the latest poo poo people are using for CI these days? Last thing I worked on was jenkins and a bunch of plugins
|
# ? Mar 14, 2023 03:58 |
|
Github Actions
|
# ? Mar 14, 2023 04:08 |
|
Hadlock posted:Github Actions
|
# ? Mar 14, 2023 05:48 |
|
My little SaaS shop still has the majority of its components on EC2, with newer stuff developed locally in docker and then GitHub Actions'd into ECS. Almost every component is a dependency; you need the full stack in every dev environment. But a dev environment is created by getting EC2 going, then editing + manually running terraform in every component's repo to get ECS up, with a PR per repo. This ECS pattern is obviously poo poo and won't scale beyond a handful of components. Is this finally a good time for us to use EKS? Have a helm chart from each repo, then just pull down which ones you want when a new environment is needed and deploy it to a new namespace in some premade cluster? How do I sell my Devs on it, it took them like a year to stop bitching about managing docker locally.
|
# ? Mar 14, 2023 07:03 |
|
12 rats tied together posted:I do think a lot of this stuff is "better now", in terraform. Like I'm pretty sure you can output an entire module by just declaring an output with a name and a value of "module.name", which gives anyone who calls you a big fat json blob containing all the outputs of your child module. You used to have to thread these through individually by name, and it sucked rear end, especially if any of them were conditionally created, because terraform still sucks poo poo at null and does random poo poo any time you feed it one. You also used to not be able to output anything except for string values, so you had to join your subnet ids with a pipe and then split them in your caller, but again I'm also pretty sure you can output complex types. e: this is fine from root modules from the DAG perspective (though mixed sensitivity still remains an issue) Vulture Culture fucked around with this message at 16:10 on Mar 14, 2023 |
# ? Mar 14, 2023 15:55 |
|
Feral Integral posted:What's the latest poo poo people are using for CI these days? Last thing I worked on was jenkins and a bunch of plugins I'm "still" using Azure DevOps. We don't have access to GitHub Actions, but even if we did, a lot of the stuff I'm doing in Azure DevOps doesn't seem to be possible in GitHub Actions. Microsoft isn't really saying much about what their preferred future is, considering they have two competing products (Azure DevOps Pipelines and GitHub Actions). In true Microsoft fashion, they're not going to truly "deprecate" anything when people are using it and, importantly, paying for it, but it does seem like a lot of their tutorials are more and more geared towards deploying the thing in GitHub Actions instead of Azure DevOps Pipelines. But they don't really seem to have feature parity either. Azure DevOps does feel much more "enterprisey" to me, which is beneficial for some of the stuff I'm doing with it.
|
# ? Mar 14, 2023 17:52 |
|
Hadlock posted:Github Actions This or Azure Pipelines. GH Actions lacks out of the box approval workflows which makes setting up gated continuous delivery a pain in the rear end.
|
# ? Mar 14, 2023 18:30 |
|
New Yorp New Yorp posted:This or Azure Pipelines. GH Actions lacks out of the box approval workflows which makes setting up gated continuous delivery a pain in the rear end. Eh, we've just created a workflow in a project template repo that runs to setup the "environment" and the approval block. It is kind of lovely, but in the end isn't all that much of a pain.
|
# ? Mar 14, 2023 18:52 |
|
The Fool posted:I mean, for that specific scenario we just read the outputs from the hub network workspace i am a moron posted:I’m on my phone so can’t effortpost, but I think the difference in opinions here might be that if you’re fully leaning into TFC/TFE some of that isn’t really an issue. Vulture Culture posted:I strongly recommend, in general, not exporting complete module outputs as a single variable. [...] For the hidden C of terraform-docs: I don't use it, when I had to write terraform still I used the terraform language server instead. B, I never put passwords in terraform because it's not worth the labor of having to audit access to state files, and I recommend the same to anyone who asks. I don't understand what you mean in A.
|
# ? Mar 14, 2023 19:06 |
|
Extremely Penetrated posted:Is this finally a good time for us to use EKS? I want to clarify first to any of my coworkers reading this thread that I mean this in the most charitable possible way: when it comes to ECS vs EKS the default choice is ECS and there are 2 reasons to ever pick EKS: 1, you have a strong dependency on code someone else wrote, or 2, you just feel like it. If you just feel like it, by all means, switch to helm and eks. I would probably start by investigating why helm looks easier and seeing if you can't instead make things easier for yourself with no new tools. For example: Extremely Penetrated posted:getting EC2 going, then editing + manually running terraform in every component's repo to get ECS up, with a PR per repo.
|
# ? Mar 14, 2023 19:18 |
|
12 rats tied together posted:I don't understand what you mean in A. If you have a module that deploys multiple resources and output the entire module as a single block, you aren't able to access any of those values unless all of those resources are deployed. Where if you have individual outputs they become available as soon as the dependent resource makes them available.
|
# ? Mar 14, 2023 19:27 |
|
12 rats tied together posted:Even in TFE, or with a hub module to my understanding, you declare your dependency on a module and you gain access to its outputs. You still have to, in code, take those outputs and pass them to other things like local resources or more modules. Terraform lets you pass data around freely (ish) but the only way to pass behavior around is to invoke a module. With a workplace model, changes can be run independently and are immediately available through the workspace outputs. The consuming team runs a data call to read them. With a module model, the consuming team needs to update their config to ensure using the new module version, and if the module deploys resources, needs to potentially account for that as well. (Yes, I know we were talking about a data only module) Both models have their use case, and we actually use both. The most common use for workspace outputs is networking related, but we have a data only module that provides a bunch of environment info for the current run.
|
# ? Mar 14, 2023 19:42 |
|
I think I understand but when presented with the axes of "cleaner code" but "dag walk takes longer in terraform apply due to dependency blocking" I will slam that gauge to max on the cleaner code side every time.
|
# ? Mar 14, 2023 19:52 |
|
When you're doing modules for the consumption of others making GBS threads out an entire modules worth of outputs at once usually isnt the cleaner option. Especially with TFC/TFE good variables and good outputs are self documenting.
|
# ? Mar 14, 2023 19:59 |
Docjowles posted:Loops and conditionals were bolted on after the language was largely “done” in the weirdest, shittiest way imaginable and despite writing terraform professionally for like 5 years I still gently caress them up or run into bizarre issues all the time. Ugh I was dealing with this the other day. I have like 40 of these repeated in my terraform code and I could not for the life of me figure out how to just create a list variable to do it: code:
|
|
# ? Mar 14, 2023 21:40 |
|
fletcher posted:Ugh I was dealing with this the other day. I have like 40 of these repeated in my terraform code and I could not for the life of me figure out how to just create a list variable to do it: For each and locals? There is also dynamic blocks but that doesn’t fit this use case based on a quick over view
|
# ? Mar 14, 2023 22:54 |
|
fletcher posted:Ugh I was dealing with this the other day. I have like 40 of these repeated in my terraform code and I could not for the life of me figure out how to just create a list variable to do it: Sure, we've implemented this. Here's the lifecycle rules, we wanted ours to keep images tagged for an environment and purge the rest: code:
code:
|
# ? Mar 15, 2023 00:39 |
|
For what they are trying to do I think this is what they want: code:
|
# ? Mar 15, 2023 17:56 |
|
fletcher posted:Ugh I was dealing with this the other day. I have like 40 of these repeated in my terraform code and I could not for the life of me figure out how to just create a list variable to do it: Creating ECR repos in terraform has been more headache than it's worth. we just opportunistically create the repo from CI using the aws-ecr circleci orb.
|
# ? Mar 15, 2023 21:44 |
|
luminalflux posted:Creating ECR repos in terraform has been more headache than it's worth. we just opportunistically create the repo from CI using the aws-ecr circleci orb. You can also trigger a lambda on a failed docker vis cloudtrail to event bridge push to ECR to create it for you Not mine, but a sample of someone else’s that’s very close to what my old company did: https://github.com/tradeparadigm/terraform-aws-ecr-repo-lambda freeasinbeer fucked around with this message at 21:57 on Mar 15, 2023 |
# ? Mar 15, 2023 21:53 |
|
Not everything that can go in terraform should go in terraform On a related note I had to migrate 136 resources from one state file to two other state files, kill me The source was a rat’s nest of unversioned modules on top of modules on top of modules. Sometimes you should repeat yourself, ahhhh.
|
# ? Mar 15, 2023 22:22 |
|
was something like terraformer not an option?
|
# ? Mar 15, 2023 22:23 |
|
The Iron Rose posted:
This seems like ideal work to hand off to a contractor I'm not sure how to sell this angle but the next time I get assigned this
|
# ? Mar 15, 2023 22:28 |
|
I mean, it's tedious but as long as you're not having to also make configuration changes its not too bad
|
# ? Mar 15, 2023 22:30 |
|
The Iron Rose posted:Not everything that can go in terraform should go in terraform i had 6k lines of moved{} yesterday lol
|
# ? Mar 15, 2023 22:35 |
|
The Fool posted:was something like terraformer not an option? Probably if I knew it existed before today! But I am changing up the structure a tad w/ some very minor config edits, and it strains credulity to imagine terraformer knows when to use a for loop over a complex map versus modularize and so on. Some simple shell scripting worked fine for the most part to handle it though. There really is nothing more soul sucking than terraform janitoring though. Totally get why devs hate it. If it can go in application code, it probably should.
|
# ? Mar 15, 2023 22:50 |
|
luminalflux posted:circleci orb. Heh, people are still expecting to use CCI going forward and aren't moving off of it?
|
# ? Mar 15, 2023 23:54 |
|
drunk mutt posted:Heh, people are still expecting to use CCI going forward and aren't moving off of it? i'm describing what we currently do but yeah I had a very prescient meeting yesterday morning about moving off CircleCI and what that would look like. Then later in the day their poo poo went sideways again.
|
# ? Mar 16, 2023 00:04 |
freeasinbeer posted:For what they are trying to do I think this is what they want: Thanks! That looks promising, it's similar to what I had attempted previously but a little different. I wish I had saved what I tried before to compare it. Ran into a strange error on terraform plan though, perhaps it doesn't like switching from my old way of doing things to this new way. I'll give it a shot on a fresh terraform state file to see if I can sort it out, maybe just need to blow away my existing state and then import my repos and policies into this style of doing it. code:
|
|
# ? Mar 16, 2023 16:22 |
|
fletcher posted:Thanks! That looks promising, it's similar to what I had attempted previously but a little different. I wish I had saved what I tried before to compare it. 🤔 I have zero idea why it’s spitting that out but I did actually run it to confirm it worked prior to posting.
|
# ? Mar 16, 2023 17:29 |
freeasinbeer posted:🤔 I have zero idea why it’s spitting that out but I did actually run it to confirm it worked prior to posting. Thank you for even going as far as testing it out! Much appreciated
|
|
# ? Mar 16, 2023 17:41 |
|
there is a pretty good discord link over in this post https://forums.somethingawful.com/showthread.php?threadid=3753052&pagenumber=393#post530539723 and we have a reasonably active cloud yelling channel in it, where questions of this nature and other types of terraform yelling are always on topic
|
# ? Mar 17, 2023 21:25 |
|
|
# ? May 22, 2024 11:07 |
|
Feral Integral posted:What's the latest poo poo people are using for CI these days? Last thing I worked on was jenkins and a bunch of plugins teamcity for onprem, gh for cLoUD. no complaints about either
|
# ? Mar 18, 2023 04:30 |