|
EssOEss posted:Is there anything better? I find that most of "system setup" tooling and automation is poo poo - I can certainly say so about Packer and Vagrant, which I have had the displeasure of using. Yet I know of nothing better. It's all incredibly poo poo. Puppet and chef are super poo poo, but work mostlyish. Ansible is... limited. Salt is okay, but I've never used it in production, where it probably falls apart. Nix/ops/os is incredibly good, but also utterly insane. At least these days you don't need to use them as much: they can be relegated to install your orchestration layer.
|
#
?
Dec 12, 2017 07:59
|
|
|
|
| # ? May 15, 2024 13:14 |
|
|
Apparently no one uses nixops, even in the rarefied circle of nix-people.
|
|
#
?
Dec 12, 2017 08:16
|
|
|
I’m a strong NixOps and NixOS proponent and would like to use it in prod professionally.  Similarly, the guys at Fugue AKA Luminal are insane and have written a declarative cloud management system that actually keeps enforcing changes... as a Haskell DSL.Saltstack is used in prod at a few places I’ve interviewed with and everyone seems to have no regrets - this alone gets me interested. It seems like it may have too small of a community and not ambitious enough of a backing company to compete with Chef and PuppetLabs though. Ansible is a pain with the many ways to fail to access a variable (there’s a cheat sheet a guy wrote showing dozens of ways to access specific variable kinds) while minor version upgrades completely break or silently change behavior making integration tests absolutely essential for it to not shoot your family after you go to sleep instead of adjusting a cron job schedule (seriously, all the cron job CM tooling I’ve ever seen has failed to work after more than 2 non-trivial changes - wtf is so hard about crontab management?). I’m kind of curious if anyone’s tried to seriously deploy CloudFoundry and actually understand bosh and stem cells without becoming a High Priest at Pivotal. From what I’ve heard, one company in China used it to go from nothing to a fully fleshed out infrastructure of 10k+ nodes serving millions of users in a couple months from scratch, and that already beats anything near that scale I’ve done with any CM tool I’ve used.
|
|
#
?
Dec 12, 2017 09:35
|
|
|
This is the only tool every devops person should have: https://github.com/nvbn/thefuck
|
|
#
?
Dec 12, 2017 16:20
|
|
|
How good most of these tools are highly depends on the environment you work in and the application or type of servers/containers you are managing. If you learn Puppet or Chef you will be covered understanding how to make stacks based around pulling instead of pushing like Ansible. I would suggest Chef personally. It ended up totally not fitting in with my project I last worked on, but it worked pretty well in my previous job when I had to automate windows. Puppet I found was way more of a pain to deal with, and without the commercial version more work to set up. Its pretty popular now to use Chef and have everything run through tests to validate it before it gets deployed, so it could be a worthwhile skill. Ansible really is more of a swiss army knife tool, and how good it is depends more on how the stack is laid out and how you manage your data because it leaves so much of is up to the person authoring the playbook. Its really like learning a programming language, once you get decently proficient at one and have built something usable it isn't a huge leap to switch assuming you are smart and capable.
|
|
#
?
Dec 12, 2017 16:44
|
|
|
Sedro posted:I've used puppet for a few things and it's always been a disaster. I'll find a package which does exactly what I want, great! Hours later I'm debugging compatibility issues in an package 3 dependencies deep. Oh, there's a PR to fix it. Opened a year ago. Mr. Crow posted:Also y'all act like there isn't a huge market and industry of companies and services running private clouds and guess what you need to use to set that up.
|
|
#
?
Dec 12, 2017 16:49
|
|
|
Vulture Culture posted:By "package", are you referring to third-party modules on the Puppet Forge? I've found both Puppet and Chef to be incredibly competent and flexible server automation platforms, but I've run really, really far away from any reusable third-party modules other than resources that wrap APIs. At my last job we used Puppet a lot for machines where we couldn't (or hadn't yet) shoved it into container. If we used a forge module the first thing we would do is fork it and make a private-repo'ed version of it because inevitably we'd have to make a bunch of changes and the few times we tried to PR / push upstream it went nowhere. We even started making private repos for a couple of puppetlabs's authored modules because of dumb long-standing bugs. (One of which we submitted a PR for, got merged, then reverted because it failed a test for a puppet version the module's version_requirement didn't include. And it's _still_ a bug according to their JIRA.)
|
|
#
?
Dec 12, 2017 17:19
|
|
|
As someone who learned Puppet over the last 2 years, the biggest pain was the fact that most of the popular links you'll hit when you search for how to do something are way out of date, even stuff that has been published recently. You run through the learning VM and feel like you've got a decent handle on how to get things done, and then you start setting things up for real and you find out that what Puppet calls "environments" are not what everybody else calls environments. And all the experienced people you can find suggest using a control repo to solve your site module problems that you didn't know you had. Now you're trying to figure out why r10k bombs out with a super vague "protocol error" every time you try to download a module that you don't even understand but is required for yet another module that installs and manages the thing you actually want... I'm finally at the point where I'm managing everything I care about through a control repo, I can monitor it all through PuppetBoard and any changes get automatically tested, deployed and announced to Slack. But it was a lot of pain getting here and I'm not totally convinced it was worth it. If I was starting fresh I would probably go masterless.
|
|
#
?
Dec 13, 2017 01:15
|
|
|
Scikar posted:As someone who learned Puppet over the last 2 years, the biggest pain was the fact that most of the popular
|
|
#
?
Dec 13, 2017 02:34
|
|
|
FamDav posted:aws gives you practically all the tools necessary to have a restrictive IT policy if you want. for the above example, you can restrict the ability to create a publicly readable bucket organization-wide in about 10 lines of json. Ok so you are being argumentative over a one-off anecdote of apparently dubious quality and then arriving at the same conclusion.  I'm genuinely surprised so many of y'all dislike ansible and call it lacking, when was the last time you used it? Everyone else I've talked on and offline has loved it, myself included. I feel it has chef and puppet beat in almost all cases; usability, readability, getting new people or servers up and running etc. What specifically is it lacking?
|
|
#
?
Dec 13, 2017 20:36
|
|
|
Vulture Culture posted:Consultants? Heh
|
|
#
?
Dec 13, 2017 20:38
|
|
|
Mr. Crow posted:Ok so you are being argumentative over a one-off anecdote of apparently dubious quality and then arriving at the same conclusion. I'm not arguing against you because its now p obvious that it isn't your opinion, I'm arguing against the idea that there's flexibility in one direction (ex. the ability make an s3 bucket world readable with the click of a button) and not the other (restrict almost any interesting option across any service, organization-wide).
|
|
#
?
Dec 13, 2017 23:14
|
|
|
Mr. Crow posted:Ok so you are being argumentative over a one-off anecdote of apparently dubious quality and then arriving at the same conclusion. we use ansible and it's pretty great. it's even good at dealing with windows servers now.
|
|
#
?
Dec 13, 2017 23:27
|
|
|
ansible is fine as a way to fill in some variables in templates and install some apt packages or Ruby gems or whatever. it's a total nightmare once you move beyond that
|
|
#
?
Dec 14, 2017 01:11
|
|
|
Mr. Crow posted:Ok so you are being argumentative over a one-off anecdote of apparently dubious quality and then arriving at the same conclusion. A way to use it without ssh access to the node?
|
|
#
?
Dec 14, 2017 13:49
|
|
|
the talent deficit posted:ansible is fine as a way to fill in some variables in templates and install some apt packages or Ruby gems or whatever. it's a total nightmare once you move beyond that Vulture Culture fucked around with this message at 16:21 on Dec 14, 2017 |
|
#
?
Dec 14, 2017 16:18
|
|
|
I find ansible great for one offs and initial configuration but completely hopeless when it comes to consistency and conformity remediation (did someone edit a file where they shouldn't? revert it), it's not great at working through baton hosts, and the static nature of the hosts file is at odds with the fast moving and mercurial nature of cloud vms. Including files depending on variables is it's token nod to modularization, but puppet has an entire dependency tree with resolution. They're mostly different tools and i don't feel like they overlap all that much, which is why every company using ansible is ansible plus something else.
|
|
#
?
Dec 14, 2017 16:48
|
|
|
Bhodi posted:I find ansible great for one offs and initial configuration but completely hopeless when it comes to consistency and conformity remediation (did someone edit a file where they shouldn't? revert it), it's not great at working through baton hosts, and the static nature of the hosts file is at odds with the fast moving and mercurial nature of cloud vms. Including files depending on variables is it's token nod to modularization, but puppet has an entire dependency tree with resolution. They're mostly different tools and i don't feel like they overlap all that much, which is why every company using ansible is ansible plus something else.
|
|
#
?
Dec 14, 2017 17:09
|
|
|
Bhodi posted:I find ansible great for one offs and initial configuration but completely hopeless when it comes to consistency and conformity remediation (did someone edit a file where they shouldn't? revert it), it's not great at working through baton hosts, and the static nature of the hosts file is at odds with the fast moving and mercurial nature of cloud vms. Including files depending on variables is it's token nod to modularization, but puppet has an entire dependency tree with resolution. They're mostly different tools and i don't feel like they overlap all that much, which is why every company using ansible is ansible plus something else. why do you have servers where this is a problem? cattle, not pets.
|
|
#
?
Dec 15, 2017 03:07
|
|
|
Vulture Culture posted:It works fine with bastion hosts if you feed it a proper SSH config, and dynamic inventory scripts have been supported, with cloud examples shipped in contrib, for years now  uncurable mlady posted:why do you have servers where this is a problem? cattle, not pets.
|
|
#
?
Dec 15, 2017 04:43
|
|
|
Bhodi posted:I remember trying dynamic inventory but there was some requirement that made me unable to use it, I can't recall what anymore though. probably due the booze I used to cope 
|
|
#
?
Dec 15, 2017 04:54
|
|
|
Vulture Culture posted:The requirement is an executable bit. Hope this helps!
|
|
#
?
Dec 15, 2017 04:59
|
|
|
We'll see how long this lasts for, but for the moment, after some good proof of concept work on containerizing two of our core products, my boss is giving me carte blanche to move his company's systems out of the dark ages of managed VMs and in to the light of kubernetes on AWS. Trying to compile a "2018-era devops starter kit", looks something like this, what would you add/delete/modify? Hosting: AWS Orchestration: Kubernetes Reverse proxy: Traefik Monitoring/Alerting: Prometheus/Grafana Log management: Graylog Build system: Jenkins Secret management: Vault Source control: Github Release management: Coreroller
|
|
#
?
Jan 7, 2018 04:58
|
|
|
Hadlock posted:We'll see how long this lasts for, but for the moment, after some good proof of concept work on containerizing two of our core products, my boss is giving me carte blanche to move his company's systems out of the dark ages of managed VMs and in to the light of kubernetes on AWS. if you're going to use aws i'd probably use ALB instead of traefik and cloudwatch logs -> es instead of graylog. you might want to replace jenkins with codebuild too
|
|
#
?
Jan 7, 2018 05:02
|
|
|
Yeah I’d do nginx ingress controller instead of treafik. It’s a cool project but not super well integrated into kubernetes. To really run it you need to expose a key value store for it to hold state that nginx-ingress + kube LEGO does for you. Edit also use kops, unless you really want to roll by hand. And if you aren’t already in AWS I’d take a real hard look at GKE instead of managing it yourself. freeasinbeer fucked around with this message at 06:47 on Jan 7, 2018 |
|
#
?
Jan 7, 2018 06:44
|
|
|
Punkbob posted:Yeah I’d do nginx ingress controller instead of treafik. EKS is right around the corner, and there is kops in the intervening time period
|
|
#
?
Jan 7, 2018 17:14
|
|
|
FamDav posted:EKS is right around the corner, and there is kops in the intervening time period Until I see EKS I still think GKE/GCP is a better product if you aren’t already in AWS. It’s magnitudes simpler then kops and pretty much does it all for you. Upgrades are push button gcp is more secure then AWS by default and the concept of projects in GCP is a superior then trying to manually separate things in AWS.
|
|
#
?
Jan 7, 2018 17:26
|
|
|
I don’t disagree with much of that butPunkbob posted:gcp is more secure then AWS by default What examples led you to that belief?
|
|
#
?
Jan 7, 2018 21:12
|
|
|
At my new job we have both a gke cluster and a cluster created with kops on normal google cloud instances and the latter is firmly in the "horrible poo poo we can't wait to get rid of." It's not like gke is flawless magic, it's just an opportunity for you to defer learning every detail of how to operate a cluster the hard way. You can always switch your workload to a new cluster later. E: Kubernetes is a fancy new thing, and will use up a lot of your fanciness budget, so you'll benefit from making every other part of your platform boring. Doc Hawkins fucked around with this message at 21:45 on Jan 7, 2018 |
|
#
?
Jan 7, 2018 21:42
|
|
|
the talent deficit posted:if you're going to use aws i'd probably use ALB instead of traefik and cloudwatch logs -> es instead of graylog. you might want to replace jenkins with codebuild too We want to build this in AWS but also run a similar if not exact setup in our environment hosting our managed VMs, so trying to avoid complete AWS lock-in and at least keep the appearance of vendor neutral deployment. Being able to build a very small k8s inside our existing infrastructure and prove that it works there makes it a lot easier to get buy in from those that hold the purse strings to give us some additional spend to migrate to the cloud. Punkbob posted:Yeah I'd do nginx ingress controller instead of treafik. Re: GKS we did that at my last company and for the most part it worked out great minus some small cock-ups. The plan is to do K8s on AWS via kops as proof of concept/mvp for some greenfield projects; and then when AKS becomes avalable switch over to that. I'm a big fan of kops so far. What do you recommend instead of Traefik and/or what is best practice here, can you link me to something good? The problem with k8s is it's almost too modular, and there's 1000 articles out there about how to use someone's pet project of the week as X module in k8s instead of just "this is best practice in 2017/8" Hadlock fucked around with this message at 23:32 on Jan 7, 2018 |
|
#
?
Jan 7, 2018 23:30
|
|
|
I always feel like I have a handle on what I do until I read this thread. So, we’re talked with automating the install and config of a software stack. Problem is, it appears the msi switches don’t exist for some of the settings that are being tweaked. I’ve got a workaround via making a Chocolatey package and autoIT, but this doesn’t seem to be the “right” way to do things. Any suggestions?
|
|
#
?
Jan 8, 2018 06:11
|
|
|
Stop using Windows 
|
|
#
?
Jan 8, 2018 06:43
|
|
|
FamDav posted:I don’t disagree with much of that but Basically GCP encrypts everything(like storage and network) by default you don’t have to add your own stuff on top. I used to have to do HIPAA compliance in the cloud and GCP basically starts as default secure while AWS requires a lot of work. If we didn’t have $100k in credit with them I would of pushed harder to dump them. Hadlock posted:
I’d go with ingress-nginx(but not the one by nginx, inc) it’s a first class project of kubernetes and has good support. I’d really go with this both on prem and in the cloud as it’s the most full featured ingress controller at the moment. The reason why I dislike k8s and traefik is that it isn’t well integrated into k8s yet and to run it in HA you’d need to standup your own key value store(consul or etcd). It’s config syntax also doesn’t use the native ingress resource last time I checked, which is a bummer, and it had no context for kubernetes specfic resources like secrets(used to store tls keys) or configmaps.
|
|
#
?
Jan 8, 2018 07:47
|
|
|
Warbird posted:I always feel like I have a handle on what I do until I read this thread. What on earth settings are you "tweaking"?
|
|
#
?
Jan 8, 2018 14:37
|
|
|
I think maybe you want to create an MST "transform" file to go along with your MSI.
|
|
#
?
Jan 8, 2018 14:50
|
|
|
Speaking of.. has anyone ever used MST files to create arbitrary Product IDs (I think that's the right property, I'm not great at MSI stuff) in order to install multiple instances of an application with an MSI? And it really needs to be an arbitrary number of instances. If it was just like 5 or 10, I could hard code those into the MSI. It seems like it might be possible to generate MST files on the fly which would allow you to do that without having a predefined set of ProductId transforms. I'm not 100% sure though. Might be more appropriate for the .NET/Windows development thread, but I want to use it to deploy internal test environments for on-premise software.
|
|
#
?
Jan 8, 2018 15:22
|
|
|
Punkbob posted:Basically GCP encrypts everything(like storage and network) by default you don’t have to add your own stuff on top. I used to have to do HIPAA compliance in the cloud and GCP basically starts as default secure while AWS requires a lot of work. If we didn’t have $100k in credit with them I would of pushed harder to dump them. for storage on aws, you don't have to add any of your own stuff on top. you click the "encrypt this at rest" button and then either use aws encryption keys or your own, which you can do with just about any service that stores customer data. as for networking, gcp doesn't encrypt all networking either (similiar to aws vpc, traffic is authenticated but not encrypted within a ~region), so if you're not supply some form of encryption between hosts in your kubernetes cluster that traffic is flowing unencrypted. i've seen a couple of people make this mistake because gcp does do encryption in transit for some aspects of their infrastructure.
|
|
#
?
Jan 8, 2018 15:40
|
|
|
Warbird posted:it appears the msi switches don’t exist for some of the settings that are being tweaked. As in, some GUI options in the installer? If they just change runtime settings and do not affect the installer, I would simply do a post-MSI step that sets the appropriate regkeys or whatnot.
|
|
#
?
Jan 8, 2018 17:34
|
|
|
EssOEss posted:As in, some GUI options in the installer? If they just change runtime settings and do not affect the installer, I would simply do a post-MSI step that sets the appropriate regkeys or whatnot. RegShot is great for this sort of thing. And now that I've embarrassed and aged myself, I will go back to lurking and Googling every other word in this thread.
|
|
#
?
Jan 8, 2018 17:37
|
|
|
|
| # ? May 15, 2024 13:14 |
|
|
FamDav posted:for storage on aws, you don't have to add any of your own stuff on top. you click the "encrypt this at rest" button and then either use aws encryption keys or your own, which you can do with just about any service that stores customer data. Mea culpa. I guess google just wasn’t worried about in transit and HIPAA peering. That’s a bummer that I misunderstood. Still GKE is pretty dope, and you don’t have to wait for EKS. Also the rbac and google stuff is much easier then accomplishing the same with rear end/IAM. But EKS hopefully should make kubernetes more turn key for more people.
|
|
#
?
Jan 9, 2018 06:54
|
|