|
Hello DevOps thread, I am extremely tired of building and deploying on my local machine, particularly when I'm on rural internet and uploading sourcemaps takes 40 minutes. I'm looking for a CI/CD solution that can do the following: - Test, build, and deploy (to S3 and Cloudfront) React applications - Test, build, and deploy NodeJS apps to EKS - Test and deploy Rails apps to EKS - Integrate with BitBucket - Can perform branch-specific operations (i.e. only do stuff when we push to develop or master, deploy to different endpoints based on which branch was pushed) - Not become my full time job to set up and maintain We are a really small org (2-3 developers) but we do release a lot. My initial thought was to get running with Bitbucket Pipelines, GitLab CI, or CircleCI, but the lead dev is cagey about putting our Amazon keys on a third party provider. So now I'm looking at self-hosted solutions. Can anyone recommend something that I could run on an EC2 instance or in its own container cluster (is that a thing?) Alternately, should I push back on the not using cloud providers thing? GitLab CI looks like it has a pretty generous free tier. I also set up a test and build pipeline with Bitbucket and it was super easy, but they're by far the most expensive as far as build minutes go. Is it secure to store your keys with these types of providers? I have to imagine a lot of people do it, but I guess I can also see why our lead would be nervous about it. Thanks!
|
#
?
Jul 10, 2019 14:23
|
|
|
|
| # ? May 16, 2024 00:13 |
|
|
CircleCI. A 2 to 3 person org is way too small to take on running the thing. Plus: a hosted service has more people worried about the security of your keys, than you running some platform on VM.
|
|
#
?
Jul 10, 2019 15:38
|
|
|
Or CodeBuild/CodePipeline. Or run GitLab in your EKS cluster.
|
|
#
?
Jul 10, 2019 17:30
|
|
|
Azure Devops is solid too
|
|
#
?
Jul 10, 2019 17:32
|
|
|
Most 3rd party services that want access to your amazon account should really be asking for role arns and the ability to sts:AssumeRole that role. You can add an extra piece of "security" here by requiring that the role assumer pass an external id, they go into detail on this here (especially note the "confused deputy problem" diagram): https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html Broadly, giving external entities access to your AWS account is a totally safe, expected, and supported thing to do. You still have to be smart about it, of course, you shouldn't give CircleCI "delete my only vpc" permissions or anything like that, but properly integrating your account with 3rd party services is just an operational reality of using AWS in 2019. Most services will have a documentation page that goes over the API calls they need access to (in general, I wouldn't approve a service that does not have this). You should cross reference the api calls they need with the context and condition keys documentation for the related service to see if you can further limit their permissions in some way. For a really bad, do-not-actually-do-this example, https://docs.aws.amazon.com/IAM/latest/UserGuide/list_amazonec2.html you can see in this chart that one of the available conditions for TerminateInstances is "ec2:Region", so you could put all of your CircleCI managed infrastructure in us-east-2, for example, and then only allow the CircleCI service to terminate instances in us-east-2. This way you can be sure that no CircleCI code bug will accidentally delete something critical of yours in us-east-1, but the service role itself will hum along happily as if nothing were different. More realistically, for things that you cannot ever tolerate going down, you should be setting appropriate termination protection and update policies, and ideally propagating tags to those resources through something like CloudFormation. Most api calls will let you condition on tag key/tag value in some way, so you can be like "you can terminate any ec2 instance except those marked MoneyFirehose".
|
|
#
?
Jul 10, 2019 19:39
|
|
|
Nomnom Cookie posted:Or CodeBuild/CodePipeline. Or run GitLab in your EKS cluster. I wasn't crazy with how it integrated with BitBucket when I tried it. I was checking it out because I wanted to see if we could start using it in place of Bamboo but it seemed like more work. Bamboo is kinda nice if you already manage your users and have Jira as well as it can automatically link back to issues and commits in the corresponding builds. I know people hate atlassian stuff a lot (and I was even thinking I might try CircleCI), but its one of those things where if you use more than one service its becomes more useful.
|
|
#
?
Jul 10, 2019 20:16
|
|
|
I think we would probably just be running the AWS CLI on the CI/CD server so we would be creating a role with the same permissions we would give to a regular developer if we just keep doing deploys from our machines. Which is also imo another point in favour of going down this road because it means we can limit deploys by limiting push permissions to certain branches and we don't need to worry about anyone screwing up any manual deploy steps. The AWS offerings look interesting and would be an easier sell since we're already using so many other AWS products but they don't have a BitBucket integration so that's kind of a roadblock.
|
|
#
?
Jul 10, 2019 21:00
|
|
|
prom candy posted:I think we would probably just be running the AWS CLI on the CI/CD server so we would be creating a role with the same permissions we would give to a regular developer if we just keep doing deploys from our machines. Which is also imo another point in favour of going down this road because it means we can limit deploys by limiting push permissions to certain branches and we don't need to worry about anyone screwing up any manual deploy steps. It’s possible they could add it in the future if enough people request. There’s a way to poll your bitbucket repo via lambda but that’s kinda round about. Right now I create roles and then pass them as variables in bamboo which you can override per branch. I’ve also started running cloud formation templates through this https://github.com/stelligent/cfn_nag And for new contracted projects where I don’t write the stuff myself I’m going to request things either pass that first or preferably be wrapped in something like serverless framework, AWS cdk, SAM or amplify because it’s a lot cleaner.
|
|
#
?
Jul 10, 2019 23:18
|
|
|
I'm looking for a CI server to put on my tiny ARM server. I've spent the last week dabbling in lots of different things. I'd really love for Concourse or CDS to work, but neither functions properly on my ARM server. It seems like my only two maintained options are Jenkins and Buildbot. Neither feels ideal to me, but Buildbot is at least easy to run and bearable to configure. Any suggestions for ARM-compatible CI servers?
|
|
#
?
Jul 12, 2019 02:49
|
|
|
prom candy posted:The AWS offerings look interesting and would be an easier sell since we're already using so many other AWS products but they don't have a BitBucket integration so that's kind of a roadblock. I know CodeBuild has "BitBucket" as a source but yeah it's super light weight. I might check with our enterprise support person to see if they're going to grow that integration. We use TeamCity which has great integration with BitBucket and "ok-ish" AWS integration; it has plugins for ECS, CodeDeploy etc, but it's a bit clunky. We tend to ship artifacts to Octopus deploy as we can Poweshell/C#/Whatever the deploy using AWS APIs instead of trying to build our own custom plugins for TeamCity. Cancelbot fucked around with this message at 23:50 on Jul 13, 2019 |
|
#
?
Jul 13, 2019 23:44
|
|
|
Cancelbot posted:We use BitBucket a lot and have enterprise support with AWS, next time I see our account manager (This Wednesday) I'll put in a request as they seem to listen to the enterprise people more Cool, thank you! I ended up spinning up an EC2 instance and installing our apps on there because I'm on 3/1mbps Internet this weekend and I don't want to wait 45 minutes to deploy. Hopefully we can get something a bit more suited to the purpose set up but at least I can deploy now.
|
|
#
?
Jul 13, 2019 23:52
|
|
we've just never had to use CodeBuild/Deploy as we're predominantly TeamCity + Octopus.
|
Cancelbot posted:I know CodeBuild has "BitBucket" as a source but yeah it's super light weight. I might check with our enterprise support person to see if they're going to grow that integration. We use TeamCity which has great integration with BitBucket and "ok-ish" AWS integration; it has plugins for ECS, CodeDeploy etc, but it's a bit clunky. We tend to ship artifacts to Octopus deploy as we can Poweshell/C#/Whatever the deploy using AWS APIs instead of trying to build our own custom plugins for TeamCity. Hello fellow TeamCity user! Have you tried the Kotlin DSL? We're getting tired of testing build changes in production and considering Kotlin as an alternative to "poke in the UI and hope nothing breaks this time".
|
|
#
?
Jul 14, 2019 02:42
|
|
|
Nomnom Cookie posted:Hello fellow TeamCity user! Have you tried the Kotlin DSL? We're getting tired of testing build changes in production and considering Kotlin as an alternative to "poke in the UI and hope nothing breaks this time". We have but the process is kinda clunky if you have an existing project - it'll go off and happily commit the changes but unless you set the right options people can still make changes in the UI. The problems we have here are mostly cultural - we're 95% Windows and this means that people get scared of command lines, configuration languages etc.
|
|
#
?
Jul 16, 2019 08:27
|
|
|
How is bamboo compared to team city? I have a feeling we will be switching at work and my only experience is with team city.
|
|
#
?
Jul 16, 2019 08:55
|
|
|
Taitale posted:How is bamboo compared to team city? I have a feeling we will be switching at work and my only experience is with team city. Bamboo is tolerable, like every other Atlassian product. I prefer TeamCity.
|
|
#
?
Jul 16, 2019 15:41
|
|
|
We use Gitlab runner. It's.. OK I guess.
|
|
#
?
Jul 16, 2019 16:38
|
|
|
Does anyone know or do anything around canary releases of data? Servers/instances/containers all are well established, but pushing out a data change progressively seems like it'd be stupidly difficult to do; my initial thoughts are that every potential data set would have to be versioned and whilst that's fairly straightforward in a NoSQL database like Mongo or Dynamo due to the ability to do versioned data or point in time recovery. In SQL databases this looks like a huge rearchitecting effort. I really think an arbitary rollback mechanism might work out easier.
|
|
#
?
Jul 17, 2019 08:16
|
|
|
Yeah, it's really hard to do. It helps if your apps have no state beyond what's in the database and can be quickly pointed at an arbitrary database. Canarying really needs to be architected in from the beginning. It's only easy to add on after the fact for trivial apps.
|
|
#
?
Jul 17, 2019 17:18
|
|
|
Cancelbot posted:Does anyone know or do anything around canary releases of data? Servers/instances/containers all are well established, but pushing out a data change progressively seems like it'd be stupidly difficult to do; my initial thoughts are that every potential data set would have to be versioned and whilst that's fairly straightforward in a NoSQL database like Mongo or Dynamo due to the ability to do versioned data or point in time recovery. In SQL databases this looks like a huge rearchitecting effort. Having your datasets as immutable lumps helps a lot. Attach metadata to them and you can track where they came from, how they got there, what validation they've passed, etc. It becomes trivial to diff the result of a pipeline change. Downstream you spin up a new DB cluster, bulk load it, and swap with the old one. If your current model is "here is some data, let's have a party", then...I dunno. You could cry a lot, maybe.
|
|
#
?
Jul 17, 2019 18:53
|
|
|
does anyone have experience breaking up a monolith into something more resembling service oriented architecture? at my company, we don't exactly have a monolith. what we have is several legacy services which get tested and deployed together on a regular basis. we have unit tests and integration tests, and we currently require all integration tests to run and pass before deploying the whole stack to production. today, this deploy happens once per week. we also have newer services, which get deployed at-will. these generally don't have downstream dependencies, so fuckups generally don't cause pain outside of the team that owns the new service. the tests for these are pretty lightweight. but for our legacy services, testing and deploying is very centralized. our team owns it for now, but the company wants to decentralize it and let individual teams deploy their stuff independently. that sounds great to me, but i'm wondering what to do with the integration tests, which will somehow need to be owned in common. these integration tests take a long time to run, of course. some of them can run in CI, and others need to run in staging. we only have one staging environment. running all of these tests for every deploy is going to really slow us down, so the prevailing idea is to let individual teams decide which tests to run and which tests to skip on a service-by-service basis. to me, it seems impossible to guarantee the same level of coverage we currently get by just running all the tests and then deploying everything after we get a passing build. so i'm looking for help resolving this apparent tension between thorough integration testing with strong guarantees of coverage and fast decentralized deployments. my instinct is that you can't have your cake and eat it too, but i'm open to hearing other's experiences dealing with this
|
|
#
?
Jul 18, 2019 06:31
|
|
|
How many tests do you have? Is it possible to manually comb through them and come up with some mapping like Test 1 =calls=> services a, d, e Test 2 =calls=> services b,d,c Test 3 =calls=> service a Then when service x deploys only run the tests that call it. If it's more a case of every test tests part of every service then you are just kinda boned. Either run all tests with every deployment, or do the work to formalize the contracts between these services and replace a large chunk of the tests with smaller ones that test the contracts. Keep a small set of full integration tests for smoke testing. This is often a large amount of work for legacy services and often not really worth it. There are probably hundreds or thousands of tiny decisions made by devs over the years based on the assumption that this was all one product and they didn't need to consider backwards compat scenarios.
|
|
#
?
Jul 18, 2019 19:23
|
|
|
it does seem like a lot of work. but i want to at least make all our tests available to teams, and let them run whichever ones they decide are necessary. it would be good if that decision were as fool proof as possible in theory it should be possible to determine programmatically which tests touch which services. if a test posts or produces to component A, then checks for a result from component B, we can easily find out about those. but there are cases where an intermediary service is involved, and the only way to know about that is to have specific knowledge of the systems under test. the dev teams should have that knowledge, but it’s difficult to guarantee that i’m considering using jacoco or similar tools to determine which services are active when a certain test runs. maybe debug logging can work instead, but not everything an application does is necessary going to generate a log entry (although it should be imo). it honestly seems very labor intensive to do this in a rigorous way the staging tests will likely need to be refactored to run in CI, or just trashed. they aren’t necessarily parallelizable in staging thx for your reply. it’s kind of a spicy situation to say the least, so i’m trying to make sure i’ve thought this thru as best i can. i appreciate your perspective
|
|
#
?
Jul 19, 2019 06:22
|
|
|
Nomnom Cookie posted:If your current model is "here is some data, let's have a party", then...I dunno. You could cry a lot, maybe. Get out of my head. But I've never thought about blue/green but with database servers 
|
|
#
?
Jul 19, 2019 08:06
|
|
|
I've posted in here before and thanks for all the help, I've gotten pretty far in getting everything I deal with into ci/cd (gitlab flavor) Android apps build+test in a gradle docker Python backend build+test in a python docker javascript frontend, well, I wish they had tests but lol at getting frontend developers to do anything right. even arm embedded builds+tests in a custom docker with the appropriate environment and runs the tests in arm under qemu, although something's wrong with the binary it spits out that I need to debug. Probably library version mismatch. What I'm missing is deploy. I see how to trigger a deploy job but I have no idea what to put in one - do I have a shell runner on the target machine that runs a script, some web-service that gitlab calls out to or what? The only one I've done is a terrible hacked-up thing with a transient image + ssh key as a gitlab secret that rsyncs to the target server. It's awful and basically a RCE because anyone who gets creds or injects a commit can run whatever they want on the staging server under the staging account. What does a good deploy of a docker image look like? What triggers the rollover on the docker-server? What does a good deploy of static files for the webserver look like? Especially looking for atomicity/all-or-nothing instead of update-in-place. App releases are currently manual but I'll get to that eventually.
|
|
#
?
Jul 24, 2019 18:13
|
|
|
I don't do a ton of webserver work anymore but a common approach I see to atomic static content is to keep the last n releases as local directories, have your application read content from a "latest" symlink, and then update the symlink as you deploy. Pretty sure capistrano will do this for you, for example, if you have a rails app. You could do something similar with docker applications by building volumes-per-release and treating your container's volume spec the same as you would normally treat a symlink I guess? That's probably where I would start, anyway. Docker deploys, it depends on what you're scheduling containers with and the rest of their operational context. I've worked with containerized applications where we just ssh -> change compose.yaml -> docker-compose down -> docker-compose up on production deploys. If your containers just serve web traffic you can lean heavily on load balancer health checks, if your containers are task workers or cluster members you might need to restart them in some type of sequence or batched sequence, etc. For example, if you have containerized kafka brokers then you might want to restart the current ActiveController last, if you have a zookeeper cluster you want to do a rolling restart in a way that means you still have a quorum throughout the deployment, etc. Ansible is really good at handling batch updates or other types of complex deployment logic, it's what I'm currently pushing for at my employer and it handles servers as well as container orchestration, but it's a pretty heavyweight solution unless you're already using it elsewhere or have other ansible experience. You would run AWX (like jenkins, but with ansible), embed api credentials in your gitlab runner jobs, and then use ansible to describe your deployment logic and configure your repo to "run job `production deploy for $app` on merge to master". A more standard recommendation would be that your scheduler probably has some primitives for declaratively managing certain application requirements. If you use k8s, probably a good place to start reading is about pod disruption budgets and deployment strategies. If you're using docker swarm, I have less experience here but I believe service update behavior is a similarly good place to start. These are both really similar to AWS AutoScaling UpdatePolicies if you've done any work with them, and is in my experience a very common thing to be working on/thinking about in 2019. Generally in this scenario your runner, or deploy script run by your runner, is just going to authenticate to your scheduler and run some basic cli commands. It's hard to go into more detail, IMO, without a specific description of the application and environment.
|
|
#
?
Jul 25, 2019 16:51
|
|
|
Queue based applications have an inverted deployment problem compared to synchronously run applications that just get deregistered from the load balancer / reverse proxy. At my last place we had applications that took forever (20+ minutes) to stop reading from queues (because people stopped caring about trying to do anything more than massive locks across large swaths of the long lived transactions) and it took hours to roll over our production system. If the underlying messages in the queue changed format / schema you had to handle both versions of messages correctly as well. Some people advocate removing access to the queue to applications but this approach is not correct / safe always because of system state involving error handling. Distributed system upgrades are unique almost to each one unless they’re stateless in which case you can get away with a YOLO deploy. So what remains is persistent backends like Kafka, RabbitMQ, RDBMSes, Cassandra, etc. The good news is that K8S operators are taking care of those. The bad news is that most people’s applications developed before 5+ years ago and not properly ported to containers / 12F-ish basically need their own custom K8S operator because they are stateful as hell by default. AWS autoscaling update policies in CloudFormation tend to require a fair bit of work with firing back to callback endpoints in CloudFormation resources along with some decent CloudWatch monitoring and alerts setup to be able to work effectively but they’re workable if your applications have external network access and you’re using CF already.
|
|
#
?
Jul 25, 2019 17:41
|
|
|
I haven't done a lot of work with operators yet (although "ansible as a k8s operator" is a hot topic right now in ansible) but I've always found the procedural playbook approach easier to deal with than trying to wrap deployment logic in some kind of declarative state manager. Almost certainly this is because I've spent the past ~6 years working heavily with ansible, combined with a general distaste for any type of polling or watching, but I think investing in robust deployment logic is superior to the operator's definition -> watch -> reconcile pattern. It also seems like no matter what you do in k8s you end up writing a bunch of yaml, so if I'm going to do that anyway I'd really rather just write the playbook. I haven't yet encountered a compelling argument for, for example, baking some kind of partition-aware rolling update logic into a kafka broker operator instead of just having the kafka broker playbook be partition-aware and perform rolling updates, especially considering that writing the operator means maintaining code, and given the choice I'd pick maintaining yaml over maintaining golang.
|
|
#
?
Jul 25, 2019 18:27
|
|
|
I need some direction with moving some existing web apps from VMs into AWS. Not sure if there's a better thread for it than this. We have a code-ignitor PHP app on a VM. I have so far been able to create and deploy a docker image containing the app into Elastic Beanstalk. There are 2 issues that I still need to work out though. Be aware that I have almost no experience with either docker or EB, but neither does anyone else on our team either. Issue 1: code ignitor stores stuff like database credentials in files like application/config/database.php. The docker image I've created has a database.php inside it which connects to the db in the test environment, and it works. But I'd obviously want to be able to deploy the same docker image into the prod environment once we're ready to migrate over. What's the best way to inject settings into a container? Do I set environment variables and then run scripts via Dockerfile that copy the environment variables out into the proper places, or is there a better way to do this? Issue 2: the app in EB works great for me, but others are having difficulty whenever submitting a form, so they can't even log in. AWS logs have a "upstream sent too big header while reading response header from upstream" error, which apparently means that the client is trying send session data that's too big for the EB Nginx to handle. We apparently came across the same issue with our VM, so the Nginx config there was updated to increase the proxy buffer size. Problem is, the Nginx instance that's forwarding requests to my web app sits outside my docker container, so how would I update the config for something outside my environment?
|
|
#
?
Jul 25, 2019 21:02
|
|
|
beuges posted:Issue 1: code ignitor stores stuff like database credentials in files like application/config/database.php. The docker image I've created has a database.php inside it which connects to the db in the test environment, and it works. But I'd obviously want to be able to deploy the same docker image into the prod environment once we're ready to migrate over. What's the best way to inject settings into a container? Do I set environment variables and then run scripts via Dockerfile that copy the environment variables out into the proper places, or is there a better way to do this? - pass in the credentials with environment variables, and change your database.php to be like "$dbuser = getenv('DATABASE_USER')", etc. This is easy to do but has the disadvantage that your credentials are easier to find for some hacker on the box. - Put the creds into a file outside the container, and bindmount that file into the container (docker run... -v /path/to/file/on/host:/path/to/file/in/container ...) and again, update your database.php to read credentials from that file (or even simpler, just bindmount the database.php file itself). Slightly safer than the envvars method, as long as you protect the credentials file on the host. - Use a vault or other type of credentials server, and then at container startup time your app will reach out to the server and ask for the credentials over the network. This is safest because the credentials only exist in the app's memory, but it's also way more hassle to setup.
|
|
#
?
Jul 25, 2019 23:18
|
|
|
Since you're already in AWS I would recommend SSM parameters or Secrets Manager secrets. If neither service directly ties into beanstalk, you can always pass IAM credentials to your beanstalk servers and then write a simple startup script that pulls and decrypts your secrets before launching your application. Regarding your nginx settings, I'm not really sure -- I don't spend a lot of time in beanstalk, or working with php, and when I do I usually just use an ELB.
|
|
#
?
Jul 26, 2019 15:13
|
|
|
RDS has an API to get temporary passwords through IAM roles, so use that and get rid of the long-lived password entirely: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html Requires a little change to the application, but it's the most secure path. e: ...which I suppose you'd be trying to avoid, since I'm sure you don't want to muck about with changing a 3rd party application. But worth at least googling to see if there's a config option or plugin or something that might do it before going another route. crazypenguin fucked around with this message at 06:23 on Jul 27, 2019 |
|
#
?
Jul 27, 2019 06:17
|
|
|
Thanks for all the suggestions, much appreciated!
|
|
#
?
Jul 29, 2019 11:32
|
|
|
Please don't use Elastic Beanstalk. Especially if you're running in container mode. Just use ECS.
|
|
#
?
Jul 30, 2019 16:13
|
|
|
Ajaxify posted:Please don't use Elastic Beanstalk. Especially if you're running in container mode. Just use ECS. I wish ECS was mature enough when some of our teams decided to use beanstalk around 18 months ago. Now we have a ton of unpicking to do. Friends don't let friends continue using Windows workloads.
|
|
#
?
Jul 31, 2019 08:54
|
|
|
I also should have piled on at the time. But for things that aren’t like a hack day or college class project please do not use BeanStalk.
|
|
#
?
Aug 1, 2019 04:35
|
|
|
Azure: "Hey you can deploy web apps using a docker compose file now!" Also Azure: "...But, uhh, it's actually just a hack over an existing CLI parameter that used to only contain a Linux version, so please make that file 4000 characters or less. ASCII characters." Never thought I'd find myself minifying a glorified configuration file in 2019, yet here we are  YAML code:
|
|
#
?
Aug 1, 2019 08:54
|
|
|
what is even the use case for this
|
|
#
?
Aug 1, 2019 17:21
|
|
|
Well, I managed to get vagrant to boot up VMs for testing our code individually, but I'm trying to figure out how to combine all this so I launch all test VMs simultaneously. Vagrant has a multi-vm mode where I just specify all the different configurations in on Vagrantfile. I tried this and it appeared to try to run them serially. I put my test in the provisioning logic, so I am not too surprised something like that might happen. However, I wanted to double check if that's the case. Generally, does vagrant provision boxes sequentially? My other situation is figuring out the best way to run each of the different VMs in a different mode. Specifically, I need to test both for Python 2 and Python 3, and I'm trying to do them on separate VMs. The stuff I am running can switch modes basically with an environment variable. Does anybody know a good way to duplicate each VM with a different toggle like that? I'm trying to avoid having to write a bunch of automation around this. I'm already bummed about the multi-vm sequential thing and hope I'm just wrong. Being able to kick the whole thing off from one Vagrantfile would be great.
|
|
#
?
Aug 2, 2019 04:10
|
|
|
Rocko Bonaparte posted:Well, I managed to get vagrant to boot up VMs for testing our code individually, but I'm trying to figure out how to combine all this so I launch all test VMs simultaneously. Vagrant has a multi-vm mode where I just specify all the different configurations in on Vagrantfile. I tried this and it appeared to try to run them serially. I put my test in the provisioning logic, so I am not too surprised something like that might happen. However, I wanted to double check if that's the case. Generally, does vagrant provision boxes sequentially? Serial vs parallel depends on the provider. VirtualBox (the default) is serial only. Other ones, particularly ones that are just making API calls to a thing, can usually run in parallel. Fundamentally a Vagrantfile is just Ruby, so once you unravel how it sets up the VMs, you can write whatever loops and such that you want to set up lots of VMs.
|
|
#
?
Aug 2, 2019 15:36
|
|
|
|
| # ? May 16, 2024 00:13 |
|
|
chutwig posted:Serial vs parallel depends on the provider. VirtualBox (the default) is serial only. Other ones, particularly ones that are just making API calls to a thing, can usually run in parallel. Ahh that's good to know. I'm awful at Ruby but I did manage to squeeze out a little code to try to do something involving loading the files separately into one Vagrantfile. However, it's pointless if they run sequentially. I'm betting that specifying all this in separate threads doesn't mean I can make them run in parallel due to what are probably serious race conditions in the Virtualbox provider. I'm assuming that if I'm launching these from separate processes that I'll have to manage assigning the forwarded WinRM and SSH ports too, but I guess I'll try it without any management like an animal first.
|
|
#
?
Aug 2, 2019 17:25
|
|