|
Be careful, Terraform can't make subscriptions. We do the same thing. We already had a pretty robust provisioning automation system built up around Google Forms, Google Sheets, and Azure Automation, that we were able to easily expand for cloud provisioning.
|
#
?
Apr 21, 2020 03:57
|
|
|
|
| # ? May 15, 2024 22:50 |
|
|
cheque_some posted:Use Azure Blueprints or ARM templates. What's wrong with doing things in a "vendor specific" way when it's an activity that literally only applies to that vendor? Blueprints can be run from the az cli (https://docs.microsoft.com/en-us/cli/azure/ext/blueprint/blueprint?view=azure-cli-latest) or from powershell (https://docs.microsoft.com/en-us/powershell/module/az.blueprint/?view=azps-3.7.0). So can ARM templates.
|
|
#
?
Apr 21, 2020 05:07
|
|
|
New Yorp New Yorp posted:What's wrong with doing things in a "vendor specific" way when it's an activity that literally only applies to that vendor? Pretty much this, when/if your org goes multi-cloud, you're going to be using vendor specific tech. At best, you might have a few stacks or components where you define some cloud agnostic porcelain, but you're still going to be writing vendor specific plumbing to actually make that porcelain function. Terraform bills itself as cloud agnostic but it's all marketing bullshit, a cursory examination of Terraform's aws_subnet vs azurerm_subnet will prepare you for the levels of tomfoolery you will be involved in if you ever try to write a DSL or resource model that is truly cloud agnostic instead of just having different plugins for each cloud vendor sdk. There are ton of ways to apply arm crap. We use ansible's azure_rm_deployment, we use the template lookup plugin to render the deployments with jinja2 which cuts out all of the boilerplate. We use the same pattern you would normally use for switching on OS types to switch on cloud vendor instead, which dovetails nicely with cloudformation, alicloud ROS, and even terraform, to answer your question about where you would run your terraform from. It's really hard to earnestly recommend using jenkins if you don't already have a jenkins nightmare to maintain.
|
|
#
?
Apr 21, 2020 05:27
|
|
|
I agree trying to use terraform in a vendor neutral way is nuts at best and impossible at worst. But what they may have meant is that with a vendor specific tool, if you end up switching clouds (or jobs) your knowledge of the old tool is pretty much useless. Whereas once you know how to use terraform on Azure, you know how to use it on Amazon or GCP or a bunch of other things. You don’t have to relearn from zero even if you can’t directly reuse your code. And that has value. For where/how to run terraform, is anyone loving with Atlantis? My teammates are all hot to try it out. Interested if anyone has hands on experience.
|
|
#
?
Apr 21, 2020 17:36
|
|
|
The power in terraform being cloud agnostic is that I can reference multiple things with a single project, like deploying something in AWS, adding SSH deploy keys to a github repo, and then adding config to Jenkins. Edit: Atlantis is fine, big gotcha is no in progress logs, and commit to master after it’s applied makes people angry.
|
|
#
?
Apr 21, 2020 19:08
|
|
|
I am a man in desperate need of a better job. I got recommended devops focus because stuff like automation and standards scratch my lizard brain. The question is, what would the best thing(s) to learn so I can get my foot in the door? I’ve got a small bit of docker/docker-compose experience from homelab BS. I can function in AWS/Azure. I’m just trying to find a direction.
|
|
#
?
Apr 22, 2020 02:31
|
|
|
Thanks, all.Docjowles posted:I agree trying to use terraform in a vendor neutral way is nuts at best and impossible at worst. But what they may have meant is that with a vendor specific tool, if you end up switching clouds (or jobs) your knowledge of the old tool is pretty much useless. Whereas once you know how to use terraform on Azure, you know how to use it on Amazon or GCP or a bunch of other things. You don’t have to relearn from zero even if you can’t directly reuse your code. And that has value. Yes, I meant more from a personal development and skill development standpoint I'd rather improve my Terraform skills than my ARM template skills. FISHMANPET posted:Be careful, Terraform can't make subscriptions. Yeah, already discovered Terraform can't create subscriptions because they didn't want to deal with Azure's weird limitation on their being a lifetime limit on the number of times the create subscription API endpoint can be called (dealing with Azure is an exercise in discovering weird limitations). After I discovered we're using Managament Groups in Azure I think Terraform may not be the way to go, because we'd have to maintain state of those management groups, when I don't think we can guarantee all subscription changes (that would be add/removed to/from those Management Groups) will be done through Terraform. So I'm probably going to be back to using az CLI or REST API.
|
|
#
?
Apr 22, 2020 02:39
|
|
|
Just checking: is it a bad practice for mydomain.com/.env.production.local to happily serve up a file with, eg, admin database credentials? edit: asking for a friend
|
|
#
?
Apr 22, 2020 20:37
|
|
|
Newf posted:Just checking: is it a bad practice for mydomain.com/.env.production.local to happily serve up a file with, eg, admin database credentials? Not at all. You can tell your friend that the internet is more than happy to backup said file for ... archival purposes. Also, at no point in time should he or she ever change said credentials. After all we do not want the backups to become useless.
|
|
#
?
Apr 22, 2020 21:18
|
|
|
Newf posted:Just checking: is it a bad practice for mydomain.com/.env.production.local to happily serve up a file with, eg, admin database credentials? It's fine, think of all the extra admins you'll get, free of charge.
|
|
#
?
Apr 23, 2020 07:41
|
|
|
Irritated Goat posted:I am a man in desperate need of a better job. I got recommended devops focus because stuff like automation and standards scratch my lizard brain. YAML writing skills. seriously nothing is more difficult than un-loving templated YAML i swear to god
|
|
#
?
Apr 23, 2020 08:11
|
|
|
CMYK BLYAT! posted:YAML writing skills. I unironically like YAML but I wouldn't even consider the idea of templating it any farther than extremely simple individual values: code:
|
|
#
?
Apr 23, 2020 09:37
|
|
|
I generally agree but building a full object is a little overkill for most ops tasks. Here's what that would look like in ansible:code:code:Your example looks like something out of helm which is irredeemable garbage even as a templating language, which is fair, because it's just stdlib templating in a general purpose programming language. It would be totally unreasonable for golang templating to try and compete with something like Jinja2 which is purpose built for rendering complex documents like web views and has like 15 years of bugfixes and feature releases. freeasinbeer posted:The power in terraform being cloud agnostic is that I can reference multiple things with a single project, like deploying something in AWS, adding SSH deploy keys to a github repo, and then adding config to Jenkins. This is what I meant by having plugins per vendor sdk. Plenty of non-terraform systems have plugins, even sparkleformation which is maintained by like 1 dude has plugins for all of the clouds and openstack stuff, with an added bonus of actually having an agnostic DSL that automatically translates itself into provider-specific implementation. You still have to learn aws, github, and jenkins, or re: the OP, you still have to learn ARM or whatever other azure resources you provision. You can wrap it in Terraform if you want, but it's a workflow that tends to become worse and worse the more you rely on it.
|
|
#
?
Apr 23, 2020 18:12
|
|
|
Code reuse for configuration code - still the same garbage heap as 2002  . It's bad enough trying to reuse code in normal application code, let's crank the stupid up to 11 when we try to computer janitor faster, guys.A wise principal engineer a number of jobs ago said "do NOT try to write configuration code like you're writing normal code to cover edge cases." It's how we wind up with these huge repos of community configuration code that gets abandoned and crumbles over time. It really does seem like we can't figure out how to inherit behavior and data together as much as to compose interfaces. NihilCredo posted:I unironically like YAML but I wouldn't even consider the idea of templating it any farther than extremely simple individual values: I strongly prefer Saltstack's Jinja based (or whatever other templating you'd prefer, it's all pluggable) loops over the 6+ different ways to do loops we've had in Ansible since initial release. Salt codebases I've worked with have been simply easier to maintain over a few years than an Ansible playbook across the same number of engineers, hands down. I remember about 10 years ago a Python templating library author gave up trying to do what Ansible has attempted to do and constantly failed at doing since. YAML variables and block substitutions can go straight to hell though. Good luck getting whatever parser you have to support everything across every other language's YAML implementation.
|
|
#
?
Apr 23, 2020 18:18
|
|
|
I agree the various "with_", "loop_control", etc, features in ansible have all been really bad. J2's "{% for %}" though is totally fine and coincidentially it also hasn't changed since 2011 or whatever. An ansible feature that a lot of people ignore because it sounds scary is custom filter plugins. Add a file to your repo under ./plugins/filter/ and define a very simple pluginbase subclass and you can pipe inputs from ansible directly into a full python interpreter and then get results back, instead of trying to juggle a series of "info modules -> set fact -> set fact transpose to list -> loop_control_double_nested_iterator_with_index" or whatever other nonsense exists today. I've reviewed coworker PRs where we turn an entire tasks file in a role into a single item with a single filter call. Same deal if you're trying to get values from some external source and you don't want to chain together all of the various info modules: just write a lookup plugin, which you drop inside ./plugins/lookup. I have a lookup plugin that turns the XML response from AWS' vpn connection service into a simple dictionary that I plug into a single downstream template call, that configures a full mesh sts vpn network on a bunch of vyos routers and a bunch of vpn gateways. It's like 15 lines of python, and I get to use an actual xml library, instead of 15 pages of module calls and trying to use "| from_xml" or whatever the gently caress. The "stdlib" ansible-playbook interface is extremely bloated because they're trying to cater to all kinds of people with wildly variant levels of python experience. If you know what subclassing is, though, just pop the hood and go hog wild. It is extremely friendly to you doing this. e: I am extremely interested in salt though (esp. salt reactor, which I think I discovered exists from you linking it in this very thread), unfortunately it seems like not a lot of people are actually using it so my chances of job hopping to get some experience with it are pretty slim. 12 rats tied together fucked around with this message at 19:04 on Apr 23, 2020 |
|
#
?
Apr 23, 2020 19:01
|
|
|
I’ve been at more then one place that had salt in place and then ended up running from it at some point. Edit: to clarify I had no hand in deciding to go into or out of salt, and was only briefly exposed in the course of troubleshooting. It seemed ok, but had a lot of places where folks were too clever by half. Both places ended back at ansible and then eventually k8s. freeasinbeer fucked around with this message at 20:49 on Apr 23, 2020 |
|
#
?
Apr 23, 2020 20:45
|
|
|
Eh, I've been at now 3 places using Salt and 3 places using Ansible. My current place uses pretty much everything though honestly because we have to support what everyone uses basically in our business strategy. On a day-to-day basis I like using Salt for its execution model (zeromq makes command output latency hands down the fastest of anything I've ever used), and in the completely garbage SSH jumphost bonanzas I've had to work with the only choice if just ssh-ing to a machine gives you pause. If you really want to build event-driven automation a likely superior tool than Salt's thorium reactor is Stackstorm and LinkedIn uses it for responding to common problems in their backends and helped them drop pages down a couple orders of magnitudes, and I believe LinkedIn also is a Salt shop, too.
|
|
#
?
Apr 23, 2020 22:03
|
|
|
I'm doing a POC for a Lambda pipeline and I'm about to pull my hair out. This documentation is good, but assumes that you just let AWS create service roles willy nilly automatically. I can't do that due to security restrictions. How in the hell do you make a service role that can show up in the Role ARN dropdown in CodePipeline? I barely know what I'm doing already and this added level of detail is throwing me a loop.
|
|
#
?
Apr 23, 2020 22:04
|
|
|
12 rats tied together posted:I agree the various "with_", "loop_control", etc, features in ansible have all been really bad. J2's "{% for %}" though is totally fine and coincidentially it also hasn't changed since 2011 or whatever. I'm starting in on ansible hard for the first time and yeah maybe i should just make a custom filter i guess? It shouldn't be this hard to parse and be able to make the index names 7.1_repo1 7.1_repo2 and 7.2_repo3: pre:repos:
rhel7:
"7.1":
- name: repo1
baseurl: http://whatever1
- name: repo2
baseurl: http://whatever2
"7.2":
- name: repo3
baseurl: http://whatever3
|
|
#
?
Apr 24, 2020 01:00
|
|
|
I still like chef because it uses a real language as it's DSL and if I at any moment need to break into ruby to parse some json, I can. God help if you if you ever try to craft non-trivial IAM trust documents or something in terraform and need to use that garbage DSL to generate json. The bad thing of course about Chef is that people start feeling empowered with writing their own libraries to do really, really dumb poo poo like building their own scheme for registering and de-registering to a hilariously complicated in-house DNS infrastructure. Then actually using SRV records as a form of service discovery. Document templating is just always going to be bad. Any way what I'm trying to say is pulumi or get the gently caress out.
|
|
#
?
Apr 24, 2020 02:31
|
|
|
My company wrote an entire terraform provider for spinnaker so we could have devs write helm values files as their app config. On commit a jenkins hook would kick off running some disaster amalgamation of helm template to get kubernetes object definitions and then kick off 2000 lines of jsonnet render to get spinnaker pipeline definitions in json format which would be fed into the spinnaker provider to ultimately end up with spinnaker API commands being fed to spinnaker and spinnaker in GovCloud (which is entirely different in lol ways to work around the isolation requirements) to actually get a pipeline for every app for every environment and somehow this was okay. The replacement which just got funding to go ahead with implementation this week is every bit as convoluted but will have the features of: Uses kafka Written in golang Includes a web UI step for devs to type into rather than a text file in git to set their app config Might include writing a custom k8s admission controller Methanar fucked around with this message at 02:47 on Apr 24, 2020 |
|
#
?
Apr 24, 2020 02:43
|
|
|
12 rats tied together posted:helm which is irredeemable garbage even as a templating language heyoooo guess what I get to write. Nothing like a bunch of nested control flow templating text that needs to match the indentation it will have in the rendered template, rather than lining up with the control flow directives. You can sometimes make the indentation sane, but not often.
|
|
#
?
Apr 24, 2020 03:29
|
|
|
Warbird posted:I'm doing a POC for a Lambda pipeline and I'm about to pull my hair out. This documentation is good, but assumes that you just let AWS create service roles willy nilly automatically. I can't do that due to security restrictions. How in the hell do you make a service role that can show up in the Role ARN dropdown in CodePipeline? I barely know what I'm doing already and this added level of detail is throwing me a loop. You should generally be able to trust AWS to create service roles, since the roles aren't especially useful on their own until you start giving people/other services the ability to interact with them. In general the easiest way forward for weird stuff like this is to just create the service role in a test account and copy paste it to production with whatever extra stuff you need included. Probably the best way would be for you to have blacklist managed policies for your various compliance needs and then let AWS create the service role and attach your blacklists to it. I spent a few minutes in my personal test account with this. The role you create just needs to have a trust relationship that allows the codepipline service to assume it, unfortunately, there's no easy way in the web UI to do this. I create all of my roles through cloudformation because the role creation ui really sucks rear end these days. Hit the Trust Relationship tab on your role and click Edit Trust Relationship. You want something like this in it: code:The trust relationship thing especially took a really long time to "stick" for me because there are so many moving parts that it was really easy to have the words for things just go in and out without really triggering any kind of understanding. To try and briefly describe, roles have a permission set (what the role is allowed to do, in AWS) and a trust relationship, also sometimes called an AssumeRolePolicyDocument, which defines who is allowed to "be" the role, and how they are supposed to become it. In this case it makes perfect sense that the CodePipeline UI is only listing roles that allow the CodePipeline service to use it. My employer's prod account has like 400 roles or something and the service would be totally unusable unless it did at least this level of filtering. Bhodi posted:I'm stuck in this hell right now, trying to find the best way to build a multi-platform file using yum_repository from a yaml structure but apparently you can't have an inner and outer loop in ansible that traverses a tree with different size leaves without include_task? Because the jinja2 product filter only let you reduce not reference? Like, WTF? Yeah this kind of sounds like a filter situation. I don't have a good ansible install except on my work laptop which I refuse to open after 5pm, but I'll try to come up with a decent example tomorrow. The plugins are unfortunately not super well documented. CMYK BLYAT! posted:heyoooo guess what I get to write. TBH this kinda sucks rear end in ansible/j2 too but one of my coworkers came up with an, IMO, really clever technique for addressing it: code:They use it heavily on the neteng team where they're trying to manage basically a global devicefootprint from the same set of j2 templates, and it's working pretty well actually. The templates are very, very big, but are recently a lot smaller since we're moving to a model where we try to define behavior as much as possible instead of enumerating configuration. Simple example: we always know that the top-of-rack switch on rack 3 is going to have the 36th ip address in whatever subnet is appropriate for this datacenter, so, we just have like management_ip: "{{ subnet_allocations[datacenter_id][rack_id] | ipsubnet(tor_switch) | ipaddr(36) }}". I forgot the exact syntax for chaining ipsubnet/ipaddr to get the desired outcome here but it results in a surprisingly small code footprint given the device count.
|
|
#
?
Apr 24, 2020 07:42
|
|
|
12 rats tied together posted:You should generally be able to trust AWS to create service roles, since the roles aren't especially useful on their own until you start giving people/other services the ability to interact with them. I appreciated the input, thanks. It appears that cloudsec has that whole thing locked down and I can’t even messes with the trust permissions even with the highest level of federated permissions. Plus I think the permission boundary that has to be attached won’t allow for it either. Turns out what I’m doing is WAY more greenfield that first thought. It seems like most folks are kicking CodePipeline duties over to Jenkins/Bamboo and I’m likely going to do the same. The kicker was that doing this all in AWS was supposed to be the “easy” way. Such is DevOps.
|
|
#
?
Apr 24, 2020 14:38
|
|
|
Methanar posted:Any way what I'm trying to say is pulumi or get the gently caress out.
|
|
#
?
Apr 24, 2020 15:25
|
|
|
12 rats tied together posted:TBH this kinda sucks rear end in ansible/j2 too but one of my coworkers came up with an, IMO, really clever technique for addressing it: This is a very good suggestion. We do this as well and it really helps improve readability.
|
|
#
?
Apr 24, 2020 20:01
|
|
|
What are you all using to test your ansible code? I just spent 3 weeks getting molecule working on ec2. It is a loving huge pain in the balls because there’s so much lovely undocumented behavior. And don’t get me started on ansible-lint which has to be the worst loving piece of garbage ever written.
|
|
#
?
Apr 27, 2020 12:31
|
|
|
Take a look at testinfra
|
|
#
?
Apr 27, 2020 12:50
|
|
|
Gyshall posted:Take a look at testinfra We're using testinfra already for some of our code, but we're using Molecule in our CI process because people kept pushing breaking changes to ansible roles that weren't caught until they were deployed. Molecule is actually pretty good at what it does, but man does it suck to get started because the documentation is incredibly bad. Like, maybe worse than Jenkins bad.
|
|
#
?
Apr 27, 2020 13:21
|
|
|
Bhodi posted:I'm stuck in this hell right now, trying to find the best way to build a multi-platform file using yum_repository from a yaml structure but apparently you can't have an inner and outer loop in ansible that traverses a tree with different size leaves without include_task? Because the jinja2 product filter only let you reduce not reference? Like, WTF? I spent some time with this and yeah, this definitely sucks/is a good time to use a filter plugin. I would suggest that, in general, a yaml file (interacted w/ via single templated parameter value in ansible-playbook) is not the place for this data. If you asked me to do it from scratch: code:You can also totally do a plugin, quick example that worked for me. Add a section to your ansible.cfg to extend the lookup plugin path to somewhere inside your repo, and then drop this file into it. I was calling it like this: code:deedee megadoodoo posted:What are you all using to test your ansible code? I just spent 3 weeks getting molecule working on ec2. It is a loving huge pain in the balls because there's so much lovely undocumented behavior. And don't get me started on ansible-lint which has to be the worst loving piece of garbage ever written. Both of those tools are really bad and it's a travesty that molecule was adopted into the official project, it unfortunately casts all of the other addon projects in kind of a bad light IMO and makes it look like they're really reaching for things to add. Instead of molecule we just use normal roles that do the same thing, without any of the molecule project structure. It's trivial to spin up infrastructure with some kind of hash suffix name and then run your ansible against it, that's exactly what ansible does all of the time. Molecule doesn't add anything except some lovely assumptions that absolutely destroy principal of least astonishment. Ansible in general really benefits from a good test/staging environment. If you can afford to have one, I superrecommend it.
|
|
#
?
Apr 27, 2020 18:22
|
|
|
The pain with Molecule is the absolute lack of anything even resembling usable documentation. If you're just using it to spin up docker containers and don't need to do any customization it is probably going to work out of the box, and that seems to be the only use case that ever gets tested. But if you need to customize anything then the documentation won't get you anywhere. There is an "official" molecule ec2 module but it doesn't work out of the box with molecule 3. It's also designed as a singleton so you'll run into weird race conditions with the infrastructure it builds if you're running concurrent molecule tests (like if you have multiple people working on branches at the same time). I had to completely redesign the create and destroy playbooks to get ec2 supported. It's really weird that the people who built out a test framework don't actually know how testing works in the real world. This is from the official documentation: “It is the developers responsiblity to properly map the modules’s fact data into the instance_conf_dict fact in the create playbook. This allows Molecule to properly configure Ansible inventory.” That is the only place in the entire documentation that "instance_conf_dict" is mentioned in the documentation, yet it plays a pivotal role in how molecule connects to the instances it spins up. They don't tell you what the variable contains or how to configure it, just that it exists. I was able to dig through the source and the broken molecule-ec2 module to figure out how to populate the variable, but that's how everything related to molecule works. It's just undocumented behavior after undocumented behavior that you need to comb through source and write a bunch of debug statements to figure out. It's amazing that they've gotten to version 3 of this software and it's now part of the official project, but it just completely fragile and not user friendly at all. end rant
|
|
#
?
Apr 27, 2020 19:57
|
|
|
12 rats tied together posted:I spent some time with this and yeah, this definitely sucks/is a good time to use a filter plugin. I would suggest that, in general, a yaml file (interacted w/ via single templated parameter value in ansible-playbook) is not the place for this data. If you asked me to do it from scratch: Followup question for ansible people; what's the best practice way of using a local boto .aws/confiig profile to run aws commands on remote hosts? I'd think there would be a best practice way of doing this, but docs seem to infer copying your entire .aws/ over temporarily or export a mess of environment variables (which you'd need to parse from local profile first). Has anyone developed a task block or plugin to streamline this or do I get to reinvent the wheel?
|
|
#
?
Apr 28, 2020 15:28
|
|
|
In general I would say that you should not do that -- your aws credentials identify you, so they should remain on your laptop, which is something you are responsible for maintaining as part of having administrative access to an AWS account. AWS commands should be run from your own laptop, and if you're using ansible for this, the only thing it does is import boto3(or botocore) which transparently uses your own aws config. The easiest way to do this is to use the local connection plugin. All of my current employer's AWS playbooks start pretty much like this, which is a pattern I copied from a previous employer: code:If the node needs to be able to run commands, it's best to dodge this entirely and just give the node an IAM role via instance profiles. Ansible will SSH to the node and inherit the permissions attached to the node in the same way that it inherits the permissions you configure on your laptop. If your IAM user must proxy commands through an ec2 instance for some reason, I'd probably create an IAM role, sts:AssumeRole it, and ship the return values to the remote node, export them as env vars, and then run your command. The response from AssumeRole is time limited and, while it identifies you personally, it can't be used to recover or otherwise extend the period where it is valid to run adminstrative commands like your normal credentials would. You can also revoke active role assumptions through a policy addon, but I've never tried it. I'd probably just assume role with an expiration of expected playbook length + wiggle room, run commands, and then use a post_task to revoke active role sessions or something. This is definitely kind of a bad idea though, IMO, or at least strictly worse than the other two options.
|
|
#
?
Apr 28, 2020 17:47
|
|
|
In this particular case, I needed to proxy my own permissions to copy some stuff to a S3 bucket the instance normally doesn't have access to. I could pass them through environment variables, but to use the profile I'd need to first parse my local boto profile config for them through a localhost connection and register a variable and I was just kinda was hoping there was something built-in to do it all for me and make them ephemeral. most of the aws related tasks support profiles now, it's just that those have to be local to the instance they're being run on. The need was definitely an edge case, since most everything can be done through local commands as you noted. In the end I got lazy and gave the machine role additional creds for a few hours.
|
|
#
?
Apr 29, 2020 04:52
|
|
|
So my current company just announced to management that we lost our last round of VC funding and pretty much any team non-service oriented will probably be laid off in the near future. For context, we're heavily Professional Services based and I'm part of the new dev org just under a year old, we've been working on building a platform of the current SaaS offering, yada yada. Anyway, where did you goons find your current job? I'm kind of going to try something different than just the usual recruiter spam nonsense this time around.
|
|
#
?
Apr 29, 2020 12:39
|
|
|
Bhodi posted:In this particular case, I needed to proxy my own permissions to copy some stuff to a S3 bucket the instance normally doesn't have access to. I could pass them through environment variables, but to use the profile I'd need to first parse my local boto profile config for them through a localhost connection and register a variable and I was just kinda was hoping there was something built-in to do it all for me and make them ephemeral. most of the aws related tasks support profiles now, it's just that those have to be local to the instance they're being run on. For stuff like that I tend to just fetch the file to my laptop, upload it to S3 using my local profile and then remove it. Three plays that work for anything outside of insanely hefty files and means I don't need to hassle with IAM. Obviously not great practice if you've got concerns with moving data out of environments. The talk about using Anisble to manage Terraform has actually really made me strongly consider moving our existing Terraform stuff to it. At the moment, we run it via Buildkite, and it's essentially just a chain of scripts that init, sync some stuff with secrets manager, output a plan for verification and then apply the plan. But thinking about it, it could probably be all replaced by a single Ansible playbook that makes everything way more readable and easier to maintain. Am I going down the right path here or is it just a 'grass is greener' thing?
|
|
#
?
Apr 29, 2020 13:53
|
|
|
Gyshall posted:Anyway, where did you goons find your current job? I'm kind of going to try something different than just the usual recruiter spam nonsense this time around. whats for dinner posted:The talk about using Anisble to manage Terraform has actually really made me strongly consider moving our existing Terraform stuff to it. At the moment, we run it via Buildkite, and it's essentially just a chain of scripts that init, sync some stuff with secrets manager, output a plan for verification and then apply the plan. But thinking about it, it could probably be all replaced by a single Ansible playbook that makes everything way more readable and easier to maintain. Am I going down the right path here or is it just a 'grass is greener' thing?
|
|
#
?
Apr 29, 2020 15:30
|
|
|
I have a small footprint of ansible running Terraform in that way. I like it well enough. We're j2 templating the terraform files so there are some extra steps involved, but the module works as well as can be expected, for example there is a force_init flag so you don't have to constantly init plan apply all over the place, you can generally interact with it a single module call -> state file at a time. It also plugs nicely into roles w/ dependencies. If I had a bigger footprint and more free time I would probably fork the module and hook it into tf landscape, I remember not being particularly impressed with check and diff mode support from it.
|
|
#
?
Apr 29, 2020 17:07
|
|
|
To anyone running k8s, how are you handling database migrations with deployments? We’re a .net shop and our pipelines (azure devops) are triggered to build a container for the migration and one for the service and run them in that order (the migration container runs as a job). We’ve been running this way for a while without issue but it seems like a better option may be to run it as a initcontainer with the service deployment. We have permissions separated out so the job runs as a user with scheme update rights and the service can only r/w data. On another note, how is everyone handling deployments? We’re reviewing our setup to make sure we’re doing things efficiently and while we don’t have any real issues, we know there’s probably something better we could be doing. Currently we write out manifests in our repo and have the pipeline download/do a variable replacement for the container tag and some other things. For secrets I have an appsettings file that gets transformed and turned into a cluster secret/mounted as a volume on the pod. We’re currently reviewing azure managed identities to lessen the amount of secrets we actually need to pipe into these files. Helm still seems like a hassle as it looks like we would need 15 separate charts to cover all of our services.
|
|
#
?
Apr 30, 2020 01:40
|
|
|
|
| # ? May 15, 2024 22:50 |
|
|
Spring Heeled Jack posted:To anyone running k8s, how are you handling database migrations with deployments? We’re a .net shop and our pipelines (azure devops) are triggered to build a container for the migration and one for the service and run them in that order (the migration container runs as a job). do never helm. If what you have works, keep using it.
|
|
#
?
Apr 30, 2020 01:48
|
|