Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
The Fool
Oct 16, 2003


Vulture Culture posted:

I'm getting really, seriously twitchy at this particular use of the word "DevOps".

These days it's just a catchphrase used to describe a style of engineering that combines developer and operations type roles.

There are associated best practices and technologies. Like Orchestration, Containerization, CI/CD, etc.

There are also plenty of shops that describe themselves as devops and the reality is that they fired their operations team and make their devs reboot servers.

So really, his use of the word isn't that out of bounds, but he should have been asking about using small scale orchestration.

Adbot
ADBOT LOVES YOU

The Fool
Oct 16, 2003


Am I shooting myself in the foot if my first foray into CI/CD is VSTS? It's been super easy to set up and I haven't run into anything I wanted to do that I haven't been able to do yet, but since it doesn't seem to have a lot of community uptake, I can't help but think my time would be spent better with other tools.

The Fool
Oct 16, 2003


Boris Galerkin posted:

Do any of those places do CI for private personal repositories for free (either free free, or free for students)? I have a few things for school that I would love to throw into a CI system if anything just to keep my documentation and pdf/latex files consistently up to date but last I checked they were free for public repositories only.

VSTS is free for 5 users

The Fool
Oct 16, 2003


Just run another container with rsyslog

The Fool
Oct 16, 2003


VSTS is cool and good if on-prem is not a requirement.

The Fool
Oct 16, 2003


It's containers all the way down

The Fool
Oct 16, 2003


22 Eargesplitten posted:

I’m going to stop making GBS threads up the Working in IT thread with Docker stuff so I can poo poo it up with other topics. I’ve been doing some beginner tutorials at work, but I want to set something up at home. My desktop can sometimes be inconvenient to work from by virtue of being loving gigantic and stuck in one place. I want to set up a MEAN stack CRUD application, and it seems like being able to VPN into it from my laptop would be good. Here’s what I’m thinking I’ll need:

Container running VPN software of my choice.

Container running Mongodb.

Container running Express.

Container running Angular 2.

All of this on a Linux VM.

What am I missing?

I wouldn't do a VPN in a container. For your use case, I wouldn't bother with a VPN at all. Just set up SSH on your linux vm.

I would also look into Docker Compose, which will allow you to define all of your containers, their network and storage in one configuration file.

The Fool
Oct 16, 2003


Should be ngninx but MENN isnt a cool acronym

E: unless it's raining

The Fool
Oct 16, 2003


22 Eargesplitten posted:

Well that sounds like a pain in the rear end.

Are there generally recommended starter Docker projects? Since I should probably wait until I get Docker and maybe k8s before doing something else new on it. I’d like to mess with mongodb if possible too, learn the basic differences between that and SQL.

You don't actually need to containerize your entire application. You could very easily just get a mongodb container, start running that on it's own, get it working with your app, and then expand as you feel comfortable.

The Fool
Oct 16, 2003


Hadlock posted:

If Prometheus/Grafana is the open source monitoring solution

What is the log management equivalent these days

Bonus points if there's already a helm chart for it

Graylog?

I don't know about a helm chart, but there are official docs for docker compose.


Docker compose
Helm chart

The Fool fucked around with this message at 18:49 on Jul 27, 2018

The Fool
Oct 16, 2003


NPM installs with node, and express is distributed as an npm package.

Any other questions related to this part of your project may be better suited for the JavaScript thread

The Fool
Oct 16, 2003


Opulent Ceremony posted:

I appreciate your thoughts but this is my goal. Docker for Windows also appears to have issues running Linux and Windows containers side-by-side so everything else has to be a Windows container too.

This will theoretically not be an issue in the near future. Docker had a PoC demo in a session at ignite running Windows and Linux containers side by side.

The Fool
Oct 16, 2003


All of them?

I would put the python code and the php code in separate repos, but you should otherwise not have any issues with your deployment tool of choice.

The Fool
Oct 16, 2003


Lately MS seems to be doing a decent job of identifying tools the community prefers to use and then putting support behind those tools.

The Fool
Oct 16, 2003


What IaaC type tool should I be using to manage Hyper-V guests?

The Fool
Oct 16, 2003


In addition, terraform maintaining it's own state allows you to have terraformed resources along side other resources and not worry too much about terraform loving poo poo up.

The Fool
Oct 16, 2003


Erwin posted:

Terraform taking an hour to create a resource is probably the Azure provider's fault. If you're sure you're not doing something wrong in your configuration, then go look at the provider's repo for issues related to whatever you're seeing. The Azure provider is what defines how Terraform interacts with the Azure API to kick off the resource creation and to know when it's finished. If it was working correctly, it would take the same amount of time as your ARM template. There's nothing about Terraform that would make it take longer for Azure to do things. Solve that issue and your other points are moot.

Terraform sucks in a lot of ways, but not in any of the ways you think it does. It's the best tool for what it does, and it's one of those things that you grow to hate because it's indispensable.

The azure provider is heavily contributed to by Microsoft, there is really no reason for it to have this kind of issue.

The Fool
Oct 16, 2003


Does anyone have any opinions about Digital Oceans managed K8s offering?

The Fool
Oct 16, 2003


Messing around in my lab and trying to figure out a reliable way to deploy a node app to Windows.

No containers, just a Window Server VM with nothing installed.

I'm using Azure Devops Pipelines, and deploying the code from repo isn't a problem, I'm just not sure how to reliably ensure Node is present and if it is already running, how to reload the app.

The Fool
Oct 16, 2003


Methanar posted:

Do you really need it to be windows server?

I'm attempting to combine a hobby (the node app) with work (Windows Server Admin). If I was doing this 100% for myself I would just build a container on a linux vm and rebuild when I needed to deploy an update.

The Fool
Oct 16, 2003


Azure Devops is solid too

The Fool
Oct 16, 2003


crazysim posted:

I like the pricing difference for self hosted runners running on private repositories: $0/mo. $15/mo on Azure DevOps.

The GitHub Actions page seems to imply that will change after beta

The Fool
Oct 16, 2003


Part of the issue with terraform is that it's not easy to recover if your state file gets deleted/corrupted. Which is not something I've had happen to myself, but have heard of it happening enough times that I'm wary.

The Fool
Oct 16, 2003


Gyshall posted:

That sounds like an absolute hellscape

Also known as “the real world” for a ton of enterprises

The Fool
Oct 16, 2003


I need to add some devopsy/cloud/automation/config management voices to my Twitter feed anyone have some good follow recommendations?

The Fool
Oct 16, 2003


12 rats tied together posted:

He's not specifically in this space but I recommend @sbellware as a pro twitter follow for anyone working SaaS.

Blinkz0rz posted:

Not a comprehensive list but I follow Seth Vargo (@sethvargo), Charity Majors (@mipsytipsy), Corey Quinn (@QuinnyPig), Mitchell Hashimoto (@mitchellh), and @SimpsonsOps and find them to be pretty good

Matt Zerella posted:

Also IanColdwater

PCjr sidecar posted:

Kelsey Hightower, Liz Fong Jones, Erowid Recruiter

whats for dinner posted:

Alex Hidalgo and Amy Tobey are really good too

freeasinbeer posted:

Also the dude behind envoy, who’s name escapes me.

FamDav posted:

matt klein

Thanks for all the suggestions guys
Couple of them I'm already following, like Quinn and Coldwater. Adding the rest

The Fool
Oct 16, 2003


Are there any good existing scaffolding tools for terraform?

The Fool
Oct 16, 2003


The Fool posted:

Are there any good existing scaffolding tools for terraform?

Or something generic like Yeoman, only done in Python or Go since I don’t want to recommend installing node if I don’t have to.

The Fool
Oct 16, 2003


Ok, my question was pretty vague and I think some of you were searching for an xy problem.

The team that I’m on is building an infrastructure pipeline workflow so that our app teams can just say “I want resource1, resource2, and it needs to be load balanced” and our tools take that information and builds out all the required infrastructure to make it work, the nsg’s, the storage accounts, makes sure asp’s are in the right ase’s and a bunch of other stuff. It also enables easy promotion from dev to load testing to prod.

Right now the app teams interact with this by writing terraform using modules that we built, which when checked in trigger azure devops pipelines and tfe.

This is having a heavier support burden for teams that are less familiar with tf and we are having to troubleshoot and help them deploy their environments.

My idea was to explore the possibility of having the app teams use a scaffolding/code generation tool to ask them a couple questions then generates a base folder structure and tf files that would deploy what they need based on some common design patterns.

Mostly inspired by web dev tools like create-react-app and django.

The Fool
Oct 16, 2003


12 rats tied together posted:

I don't think it is a good idea to trigger Terraform runs from automation (as part of build pipelines or whatever).

We do, but in order to actually apply you have to log in to TFE and click a button

The Fool
Oct 16, 2003


https://cloud.google.com/run/docs/mapping-custom-domains

The Fool
Oct 16, 2003


the talent deficit posted:

terraform consumes so much time and attention at every place i've been that used it that i'm convinced it's a scam to ensure full employment of programmers who don't want to program

We have a team (that I am on) of 8 people who’s primary responsibility is janitoring terraform

However, we support multiple teams of developers and create and maintain modules because they’re not allowed to deploy resources directly

The Fool
Oct 16, 2003



Terraform for infrastructure deployment then ansible or some other config management is a pretty common pattern and probably what you should be doing

The Fool
Oct 16, 2003


The Fool posted:

We have a team (that I am on) of 8 people who’s primary responsibility is janitoring terraform

However, we support multiple teams of developers and create and maintain modules because they’re not allowed to deploy resources directly

12 rats tied together posted:

If you're on an infrastructure engineering team providing that PaaS abstraction to other feature teams, it's a really bad tool and you shouldn't use it, you'll be able to come up with something way better yourselves.

the talent deficit posted:

this is basically where i land. if you can do it in an afternoon terraform is fine (but also most things are going to be fine and it comes down mostly to taste and experience). if you are writing terraform to enable other teams to write more terraform you end up with awful messes

:hmmyes:

The Fool
Oct 16, 2003


I’m sure this is fine

The Fool
Oct 16, 2003


Yeah, my picture was from a single plan.

Specifically, that plan was using 3 modules, but those modules had dependencies on other modules resulting in needing to dig through 21 different modules to find the one where someone set the provider version requirement to ~> 2.30.0 while other modules in the plan had required higher than 2.30.x

Basically, if you have a choice don’t do what my employer is doing

If you don’t have a choice, have a team to maintain and support the house of cards you are building

The Fool
Oct 16, 2003


so far my only professional golang has been writing terratest tests

I think that’s ok

The Fool
Oct 16, 2003


Looking for a way to run terraform init against a tfe workspace without installing terraform cli, any suggestions?

Might be an x-y problem so I’ll post more info if you need it

The Fool
Oct 16, 2003


Yeah fair enough

The target audience for this is our app teams. We build and maintain a bunch of modules for them to use so that they can manage their applications infrastructure by just setting some variables for the modules they need then pushing their repo up to azure devops, then the build pipeline handles the rest

The actual terraform deployment is done as an api-driven run through terraform enterprise, the applications terraform state is stored here in a workspace as well.

Right now we are working on migrating from 12.29 to 13.5, all of our modules are updated, with the idea that the app team should be able to make sure they are using the right module versions, then set their workspace to 13.5 and deploy.

This works with the exception of an issue with provider namespaces. The issue is resolved by either running terraform init or terraform replace-provider against the remote backend

We want to eliminate the need for the app teams to have to install terraform and configure the remote backend just for this one task

Adbot
ADBOT LOVES YOU

The Fool
Oct 16, 2003


The Iron Rose posted:

How on earth do y'all use terraform version management? I have a few dozen different repos, all of which have "required_version = "0.12.18" or something similarly archaic in there, usually whatever version was latest at the time the repo was first built. There is vast institutional opposition to simply using the latest version whenever you make a new PR, mostly because people are (foolishly) scared of state file surgery.

We do have terraform cloud.


also while I'm at it, hot take...


i kinda hate terraform modules. I mean I get it, there's a few very simple ones that I've used before, but I often find it more work to use and grok an existing module rather than create it all greenfield myself.

As a policy we peg modules to the minimum compatible version, ie: >=12.26, we do the same thing for provider versions

If the app breaks because of some change in a newer version terraform or the provider, they can peg it to whatever is the newest that works in their workspace until we fix the module they’re using

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply