|
FISHMANPET posted:I want a front end to it, that's all. Rundeck is insane overkill for that, but nobody else seems to offer it, which itself makes me worried that I'm just missing something huge. the issue is that there are actually a million different ways to do this. From rolling a web app with an api shim, to config files in a repo, to forms being submitted to a webhook.
|
#
?
Jan 30, 2024 17:24
|
|
|
|
| # ? May 29, 2024 10:17 |
|
|
I made this for a slide deck so now I have to subject you to it also
|
|
#
?
Jan 30, 2024 19:43
|
|
|
The Fool posted:the issue is that there are actually a million different ways to do this. From rolling a web app with an api shim, to config files in a repo, to forms being submitted to a webhook.
|
|
#
?
Jan 30, 2024 19:46
|
|
|
Vulture Culture posted:I made this for a slide deck so now I have to subject you to it also foone's death generator has this scene
|
|
#
?
Jan 30, 2024 19:53
|
|
|
|
|
#
?
Jan 30, 2024 19:55
|
|
|
The Fool posted:the issue is that there are actually a million different ways to do this. From rolling a web app with an api shim, to config files in a repo, to forms being submitted to a webhook. I get nervous coming upon stuff like this, especially when I can't find anybody talking about how they're _actually_ doing it. I'm not that smart, and it's already 2024. I'm not the first person that wants to manage a dynamic set of environments. I could roll out a solution completely home grown in a million different ways, and probably screw up such that the whole thing comes falling down. Where are the blog posts, or conference talk recordings, or whatever, of people talking about how they're actually solving this problem. I'm concerned that the fact that I can't find this information means that it's largely been deemed not to work, and people are instead doing [i[something[/i] and I'd like to learn from their experience but it just feels like this massive black hole. In my last job I had a Google Form that passed data via webhook to trigger an Azure Automation job that would build a new VM. It always felt hacky, and people wanted "smarter" forms that would be more dynamic based on stuff like who was filling it out and what they were selecting and what the state of something was (for example, the storage team wanted a form for quota increases that would be able to determine which storage shares the user was authorized to request modifications for and present them to the user, instead of just a bunch of freeform text fields). And back then I could never find anything that came anywhere close to suiting my needs, short of just becoming a full stack developer and building my own front end and potentially accompanying back end application to do it. And I kind of accepted it working in higher ed where the problems were truly a bit unique from the "normal" world. But now I'm working at a young tech company and coming across problems that I have to believe have been faced by other tech companies before, yet I have absolutely no idea what they did to solve them. I'm totally in the dark here, which again tells me that I'm really missing something about the problem space, but I don't even know what I'm missing to begin filling that gap in my knowledge.
|
|
#
?
Jan 30, 2024 20:01
|
|
|
what you're missing is that the specific problem you're trying to solve is boring and no-one cares about it It's also something that can look wildly different for different organizations. Are you running k8s? Are you on Azure or AWS? Are you using Terraform? TFC? Spacelift? Atlantis? What about CICD? Do you let your teams have their own infrastructure choices? Or are they chosen for them and they just need a place to put a container? Depending on the answers to those questions your automation can look pretty different. Some of those things (TFC and Spacelift for sure) have built in automation to help enable ephemeral environments, but it's a small enough and boring enough problem that no-one is going to make a dedicated product for it.
|
|
#
?
Jan 30, 2024 20:21
|
|
|
there's also the question of whether or not this isn't better handled by your application tech stack. why does the infrastructure specifically have to be ephemeral, and what does that solve that e.g. having a django route that loads application code from a particular git hash wouldn't? just duplicating infrastructure is usually not that helpful.
|
|
#
?
Jan 30, 2024 20:30
|
|
|
12 rats tied together posted:why does the infrastructure specifically have to be ephemeral, and what does that solve that e.g. having a django route that loads application code from a particular git hash wouldn't? How do you validate all your infrastructure is actually managed by code if you don't redeploy from scratch at least annually
|
|
#
?
Jan 30, 2024 20:59
|
|
|
Hadlock posted:How do you validate all your infrastructure is actually managed by code if you don't redeploy from scratch at least annually That's a separate conversation from making environments available to devs. It's entirely possible for the dev's concept of an environment to have no infrastructure concerns whatsoever, and if that's the case your automation shouldn't either.
|
|
#
?
Jan 30, 2024 21:02
|
|
|
interesting question. i don't because i don't care if it's all managed by code or not, the value that iac provides is rooted in my own productivity and my team's ability to react to business demands. the infracode is useful when it makes those things go up and it's counterproductive when it makes them go down. it's not a goal by itself. in my current role it would take >1 year to redeploy our infrastructure from scratch for no benefit to the company or our team so we would never even consider it, but i'm on a dbre team now, so the usual advice doesn't apply
|
|
#
?
Jan 30, 2024 21:06
|
|
|
My instinct is that "differentiation in an environment that the devs care about" is different from the differentiation that I care about. For example, I don't think my devs care if their code is running in a vm or a container or in kubernetes, though I as the DevOps person obviously care. And so that's why my brain is primarily concerned with how to set guardrails for the information the devs are giving me, and then let everything else go from there. Also thinking about it, I may not be perfectly clear in my terminology. We just have the one product, and so when I say "dev environment" I'm referring to an independent instance of that product. We use them for various purposes though as I've just started I don't fully understand all the intricacies. But it might just be time to step away from this for a while, before I get too frustrated at it, and until I can learn more about what problem we're trying to solve here exactly.
|
|
#
?
Jan 30, 2024 21:32
|
|
|
Applications have dependencies and it's managing the entire stack of dependencies that makes this a hard problem, not making a single leaf route invoke the right thing In simpler times, most of those dependencies were configurations and libraries and code modules, now they're mostly living organisms, tardigrades floating in clouds
|
|
#
?
Jan 30, 2024 22:09
|
|
|
The Fool posted:what you're missing is that the specific problem you're trying to solve is boring and no-one cares about it Weirdly this made me think of Bill Gates' "replace the toilet with something cheaper" moonshot prize. Cash money to fix a problem nobody cares about and there's zero prestige in building a better https://m.economictimes.com/magazines/panache/bill-gates-wants-to-reinvent-the-toilet-save-233-billion-while-at-it/articleshow/66520139.cms I think the idea is that if all 10 billion people had adequate access to sanitation you'd eliminate an entire class of pathogens like cholera and polio (which is back, there was community transmission in 2021 in a NYC suburb,I think) Anyways yeah we already have cron jobs in Django. RoR, spring Boot/Grails and any other framework you want to look at has this already We have Django cron jobs for all sorts of stupid reports needed by people I've never met. And I just got done rolling out Apache superset as a crutch until the business owners give engineering a clearer idea of what it is they need
|
|
#
?
Jan 30, 2024 23:19
|
|
|
Hadlock posted:polio (which is back, there was community transmission in 2021 in a NYC suburb,I think) Wildly offtopic but it's a stretch to say polio is "back". A specific community that tends not to vaccinate had a case, it's not like there is a wider epidemic that people need to worry about
|
|
#
?
Jan 30, 2024 23:52
|
|
|
World Health Organization had ~0 reported cases worldwide in 2015 and it's been a hockey stick graph up and to the right growing each year to ~1000 cases worldwide in 2021 , one of which was the first reported case of community spread in the USA in ~70 years. Call it what you want. We declared "victory" long ago and then the cases started popping up again Community spread is important, it means not only is there a positive case, but they've been there long enough to infect someone, and in polios case most people don't seek treatment until their fingers start tingling at which point they've been contagious for months or years, as the NYC case was. TL;DR community spread is a leading indicator of a potential outbreak And yeah we're wildly off topic Maybe Bill Gates will pour $200 billion into run deck next year Hadlock fucked around with this message at 00:09 on Jan 31, 2024 |
|
#
?
Jan 31, 2024 00:02
|
|
|
There's also a major education issue in the poorer and less medically serviced parts of the world, where you have this horrendous combination of live virus vaccines and people who don't get their full course of immunizations. So you end up with places like rural India where there are minor outbreaks due to vaccine-derived polio strains—something not rare in immunocompromised people but where it won't spread through communities—and it creates this perfect shitstorm. What doesn't help our chances in the US is that we see dead diseases popping up in the same communities over and over, and they happen to be ones where large crowds of almost universally unvaccinated people get together for large ceremonies and observances. Though, I guess that's going to be common in the political religions too in a generation. Vulture Culture fucked around with this message at 14:16 on Jan 31, 2024 |
|
#
?
Jan 31, 2024 14:13
|
|
|
Income tax history and the world state of polio in TYOOL 2024, welcome to the CI/CD thread. Also, Rundeck was started by the devs - stated by their own docs at least - as an OSS successor to the product I worked on, and it’s kind of odd because I thought it was a tiny community and I’d have known everyone using personally but I never met them. Having extensively used Jenkins and being familiar with the architecture of Rundeck and the original product, I’d say that Rundeck is fine if you want a task executor with minimal need for updates compared to Jenkins because security in that product is something rather lacking. Being able to show and distribute common tasks that may have cross cutting business concerns (the real power of any orchestration engine IMO and I think the formal definition requires it) with ACLs is clutch. Having support staff, sales, engineering, etc. all able to consume stuff from the same portal with similar ease of use is a sort of holy grail given the rather vast needs of everyone. Really, it comes to how well a team has simply given a poo poo about everyone rather than aiming a product at just one specific kind of user (the general failure of BPEL engines being this IMO) and you make a Swiss Army knife product of sorts. Well, the Swiss Army Knife foot shotgun of our world is, in fact, Jenkins. But good god is exposing Jenkins to the public Internet something that would keep me awake at night.
|
|
#
?
Jan 31, 2024 19:23
|
|
|
Rundeck’s primary difference is organizational; all scripts are tied and owned by a “project” which also has a single pregenerated list of nodes not unlike an ansible inventory hostfile. Scripts cannot be “shared” amongst projects (though can be administratively moved/cloned/copied) and the nodelist must exist or be pregenerated before any script is run. You can run on a filtered subset of this nodelist, set as a job param, but a project and nodelist are 1:1. Nodes themselves are not validated and are just strings passed to ssh and so can be part of multiple project’s nodelists and are typically generated via periodic inventory scraping scripts or static, again not unlike ansible.
Bhodi fucked around with this message at 14:08 on Feb 1, 2024 |
|
#
?
Feb 1, 2024 14:05
|
|
|
Bunch of other convo, but maybe this still applies. The helm integration with ArgoCD kind of sucks and I would honestly suggest looking at kustomize, which IMO takes better use of merge strategies to render down the resource manifests.
|
|
#
?
Feb 3, 2024 16:22
|
|
|
We started to migrate all our third-party apps installed by ArgoCD from helm to kustomize (via HelmChartInflationGenerator or whatever it's called) and it's a bit of a pita at times but I think it will be nice to have a consistent mechanism for environments, ability to add arbitrary resources or overlay anything even if not exposed by the chart. I'm not sure there is a "good" option, but so far it seems better than the nonsense we were doing before with subcharts.
|
|
#
?
Feb 3, 2024 20:04
|
|
|
SurgicalOntologist posted:We started to migrate all our third-party apps installed by ArgoCD from helm to kustomize (via HelmChartInflationGenerator or whatever it's called) and it's a bit of a pita at times but I think it will be nice to have a consistent mechanism for environments, ability to add arbitrary resources or overlay anything even if not exposed by the chart. Can you describe how your system worked and why you're switching away from Argo/helm, because I'm going to be doing Argo/helm on Monday probably
|
|
#
?
Feb 4, 2024 01:14
|
|
|
We switched away from Argo+Kustomize towards Argo+Helm. Helm is awful, don’t get me wrong, but we manage 40 “Addons” (and counting) across 38 clusters (and counting) across four different cloud providers (and counting) and the amount of Kustomize overlays required was unwieldy at best. Plus, a lot of OSS “Addons” use a Helm chart as their default/official install method so it’s fairly easy to consume. With Argo+Helm things are much cleaner in the repo and much easier to grok. We also have an in-house tool that runs as a pipeline to render a diff between main and the branch just to ensure the changes we’re intending to make are what ArgoCD will make. All in all it works well enough. We’ve explored other tools and patterns but they haven’t ever been better. Helm itself is quite bad. Templating YAML is horrible. But it is what it is.
|
|
#
?
Feb 4, 2024 07:21
|
|
|
ThanksThe Fool posted:what you're missing is that the specific problem you're trying to solve is boring and no-one cares about it Weave Works closing their doors https://www.crn.com/news/cloud/2024/aws-backed-kubernetes-company-weaveworks-closes-ceo-blames-failed-m-a I never really understood their product or value add, what I knew them for was they had written a front end for Flux 2 After working for a devops adjacent startup, who ultimately failed and pivoted to auditing/compliance, it sure seems like devops isn't an investible vehicle. The golden era of k8s/devops tooling to build consensus and become the market leader seems to have passed. Looks like cloud bees (Jenkins) is claiming $100mm in revenue, of which almost half (or more?) Comes from ~20 companies? https://www.techtarget.com/searchitoperations/news/252523469/Another-new-CloudBees-CEO-faces-fierce-CI-CD-tools-rivalry Weave Works is listed as a "fierce competitor" they aspire to be, article dated August 2022. Gitlab is mentioned too. With the Advent of Google cloud run, GitHub actions, and whatever gitlab has.... Runners? Seems like the market shifted from underneath Jenkins. Jenkinsfile allows for true git ops but the product needed a total rewrite for at least 7 years now. Even if they had a "Jenkins cloud" product, why would you deal with the added complexity of moving outside of the services offered but GitHub/gitlab Hashicorp seems like the other potentially profitable option but as soon as they tried to pivot to a profitable business model, the community split off ASAP. Terraform underpins Oracle cloud but they're pinned to such an old version they'll be able to pivot to opentofu should they choose to Anyways TL;DR not predicting much additional investment in this space unless you want to compete directly with like, GitHub and GitHub actions
|
|
#
?
Feb 6, 2024 16:34
|
|
|
I thought weave works made flux itself? Not just their (overpriced) front-end. Wonder if they're gonna open source that now
|
|
#
?
Feb 6, 2024 17:22
|
|
|
FluxCD has always been open source? I think the bigger question is who's going to be maintaining it going forward.
|
|
#
?
Feb 6, 2024 17:27
|
|
|
Hadlock posted:Hashicorp seems like the other potentially profitable option but as soon as they tried to pivot to a profitable business model, the community split off ASAP. I think there's room for profitable companies in this space, but Hashicorp (and others) keep making pants on head stupid decisions for investor story time. No-one wants to make a "reliable product or service that makes a little bit of money consistently"
|
|
#
?
Feb 6, 2024 17:29
|
|
|
The Fool posted:FluxCD has always been open source? I think the bigger question is who's going to be maintaining it going forward. No I mean open-source their gui. They have a free one but it's pretty gimped compared to the enterprise version
|
|
#
?
Feb 6, 2024 17:34
|
|
|
Resdfru posted:I thought weave works made flux itself? Not just their (overpriced) front-end. Wonder if they're gonna open source that now I looked at the release history and I don't think this is correct. I think Stefan took over ownership of flux 1 and happened to work at weave works when flux2 was being developed. He left the company and is still the maintainer of flux2
|
|
#
?
Feb 6, 2024 17:53
|
|
|
Ah, I misunderstood something in the ceos linkedin post about this I think
|
|
#
?
Feb 6, 2024 18:58
|
|
|
How big are your gitops monorepos? Some smart heads in the org I work with came up with the design that after a year has ❯ git rev-list --objects --all | wc -l 68656106 So I’m curious how it compares.
|
|
#
?
Feb 7, 2024 03:56
|
|
|
kaaj posted:How big are your gitops monorepos? Some smart heads in the org I work with came up with the design that after a year has 100% running this on every repo I have read access to tomorrow, out of now-massive curiosity
|
|
#
?
Feb 7, 2024 05:04
|
|
|
we have a repo sprawl problem, so I don't think any of our repos would out up big numbers but now I'm curious how many we actually have
|
|
#
?
Feb 7, 2024 05:13
|
|
|
The Fool posted:we have a repo sprawl problem, so I don't think any of our repos would out up big numbers but now I'm curious how many we actually have “How many git repos do we have?” is one of those seemingly innocuous questions that, when asked about a large enough org, at a high enough level, ends up burning hundreds of man-hours to produce the most disjointed excel spreadsheet you have ever laid eyes on.
|
|
#
?
Feb 7, 2024 05:24
|
|
|
The Fool posted:I think there's room for profitable companies in this space, but Hashicorp (and others) keep making pants on head stupid decisions for investor story time. No-one wants to make a "reliable product or service that makes a little bit of money consistently"
|
|
#
?
Feb 7, 2024 14:06
|
|
|
I'm not super happy with ArgoCD but I'm too far along the implementation path to back out and switch to flux because I need to get this delivered ArgoCD is pretty good for what it does. But then to update the image tag of the container you need to.... Install a third party plugin that's v0.12 and loudly points out that it could change at any time? Looks like there's a PR ready to merge* but the guy who maintains the plugin has abandoned it and wants someone else to take over the plugin, but doesn't offer any way to contact them  also a bunch of proceduralists are adding red tapeThird there's no first class support for AWS ECR, gently caress me, guys come on. Ok fine I'll install a weird third party helm chart to get the ecr login secret, I guess. Now I have to create a local fork of this third party chart to support my CD system I'm all for "do one thing, and do it well" but it doesn't seem like these functions need to be independent of the main helm chart, you've already broken ArgoCD into five+ services Of interest, it looks like the guys who started ArgoCD gave up on it, literally forked argocd-image-updater and built a new CD system on top of it, Kargo (although they've since fully rewritten the image updater code). Kargo is too new for my tastes but I'm not loving this "band of merry helm charts" approach to building a functional CD system; flux would have been a very choice at this point I think. *Edit there's a PR to merge his plugin into the main line ArgoCD to remove this glaring amateur hour oversight Hadlock fucked around with this message at 20:02 on Feb 8, 2024 |
|
#
?
Feb 8, 2024 19:42
|
|
|
Hadlock posted:I'm not super happy with ArgoCD but I'm too far along the implementation path to back out and switch to flux because I need to get this delivered
|
|
#
?
Feb 8, 2024 20:02
|
|
|
so weaveworks made this thing called flamingo that basically melds Argo and flux together. Not sure if it would let you get around any of those issues but if you wanna make things even weirder it's there
|
|
#
?
Feb 8, 2024 20:03
|
|
|
e: nm, this thread already talked about the Weaveworks shutdown
|
|
#
?
Feb 8, 2024 20:04
|
|
|
|
| # ? May 29, 2024 10:17 |
|
|
I just use gitlab pipelines/GitHub actions 
|
|
#
?
Feb 8, 2024 20:06
|
|
Cat Army