|
It will probably be fine OP
|
#
?
Feb 12, 2021 16:11
|
|
|
|
| # ? Jun 10, 2024 01:16 |
|
|
carry on then posted:It will probably be fine OP An optimist, In yospos ? It's more likely than you think
|
|
#
?
Feb 12, 2021 16:16
|
|
|
Can someone give me a tl;dr about why k8s is bad, or maybe what kind of use case its good for, and how developers ruin it?
|
|
#
?
Feb 13, 2021 03:19
|
|
|
ate poo poo on live tv posted:Can someone give me a tl;dr about why k8s is bad, or maybe what kind of use case its good for, and how developers ruin it? k8s was not built for your cloud provider, and your cloud provider was not built for k8s (save maybe gcp, but even then). you will inevitably run up against weird edge cases and poorly built "drivers" for things like networking and storage that have subtle limitations and edge cases that are incongrous with what k8s expects.
|
|
#
?
Feb 13, 2021 03:29
|
|
|
gke or fargate are pretty ok.
|
|
#
?
Feb 13, 2021 03:39
|
|
|
honestly the biggest mysterious k8s fuckups we've had had to deal with are around certificate revocations and updates for mtls and so forth rather than networking
|
|
#
?
Feb 13, 2021 03:40
|
|
|
FamDav posted:k8s was not built for your cloud provider, and your cloud provider was not built for k8s (save maybe gcp, but even then). you will inevitably run up against weird edge cases and poorly built "drivers" for things like networking and storage that have subtle limitations and edge cases that are incongrous with what k8s expects. iawtp, and expanding on it: k8s in practice is about halfway between a spec and a working implementation of the spec that meets your company’s needs. the cool thing about k8s is that it’s flexible enough you can probably find the right tinkertoy bits to get something that does exactly what you want. but that means you’re going to end up with a combination of tinkertoy bits that is unique, or only shared by a few other users. every k8s user is on k8s, but how many are using traefik for an ingress controller, with flannel, on kops/AWS. are you still on kube-dns or did you migrate to coredns. fluentd, or fluent bit, or filebeat, or something else? it’s inevitable while bringing up a k8s cluster to production ready status that you’re going to run into novel problems, and you’re going to run into more novel problems if and when you start scaling
|
|
#
?
Feb 13, 2021 03:44
|
|
|
google made it
|
|
#
?
Feb 13, 2021 03:55
|
|
|
https://twitter.com/SeinfeldOps/status/1350976533171757056
|
|
#
?
Feb 13, 2021 04:17
|
|
|
it is written in go
|
|
#
?
Feb 13, 2021 04:42
|
|
|
it’s basically the devops equivalent of a gamer pc with water cooling and rgb lighting, op
|
|
#
?
Feb 13, 2021 04:55
|
|
|
i set up rancher (pre k8s version) ages ago for running a few internal tools that have wild plang dependencies and it's been rock solid. is their k8s version any good?
|
|
#
?
Feb 13, 2021 05:01
|
|
|
Gentle Autist posted:it’s basically the devops equivalent of a gamer pc with water cooling and rgb lighting, op water cooling is functional and rgb lighting appeals to a certain demographic. k8s is neither functional nor appealing to anyone
|
|
#
?
Feb 13, 2021 05:58
|
|
|
k8s is pretty decent if you design your app to run on it.
|
|
#
?
Feb 13, 2021 06:09
|
|
|
kitten emergency posted:k8s is pretty decent if you design your app to run on it. and so, everyone proceeded to take their existing applications, which were not designed with kubernetes in mind, tacked on strange hacks and middleware, and made them run in kubernetes despite the applications' many protestations
|
|
#
?
Feb 13, 2021 06:21
|
|
|
kitten emergency posted:k8s is pretty decent if you design your app to run on it. one nice thing about php apps is they're stateless by nature so nicely suited to scale-out and containerization
|
|
#
?
Feb 13, 2021 07:04
|
|
|
my stepdads beer posted:i set up rancher (pre k8s version) ages ago for running a few internal tools that have wild plang dependencies and it's been rock solid. is their k8s version any good? it’s not managed k8s so it sucks. use eks
|
|
#
?
Feb 13, 2021 07:13
|
|
|
i’ll also try and make an effort post from the perspective of providing managed k8s
|
|
#
?
Feb 13, 2021 07:46
|
|
|
the raison d'etre for kubernetes is ostensibly getting economies of scale for your ops and release teams. if you're trying to solve those two specific problems, then it's not bad. if you don't have either of those problems I don't know why you would use it whicvh is also why you should be suspicious of devs who are pushing it, the kubernetes is not *for* them.
|
|
#
?
Feb 13, 2021 10:39
|
|
|
my homie dhall posted:the raison d'etre for kubernetes is ostensibly getting economies of scale for your ops and release teams. if you're trying to solve those two specific problems, then it's not bad. if you don't have either of those problems I don't know why you would use it is k8s better for this than other container orchestrators, though? I’m thinking of nomad, specifically, because that’s what I’ve had exposure to. it seemed simpler to set up and operate
|
|
#
?
Feb 13, 2021 16:31
|
|
|
nomad is indeed simple and nice and id prefer to use it if i had the choice
|
|
#
?
Feb 13, 2021 17:33
|
|
|
Nomnom Cookie posted:is k8s better for this than other container orchestrators, though? I’m thinking of nomad, specifically, because that’s what I’ve had exposure to. it seemed simpler to set up and operate no clue, I only have experience with k8s, but it’s the one that “won” so it’s going to be here for a while also, at least in terms of its logical model, it’s not very difficult to understand. you have workload units (pods) and then a bunch of abstractions to create and manage them in different ways. there’s incidental complexity mostly in implementing networking and storage and everything that’s involved in initially standing a cluster up, but if you can make those things Someone Else’s Problem then it’s not too bad imo
|
|
#
?
Feb 14, 2021 02:19
|
|
|
CMYK BLYAT! posted:and so, everyone proceeded to take their existing applications, which were not designed with kubernetes in mind, tacked on strange hacks and middleware, and made them run in kubernetes despite the applications' many protestations well those people are dumb
|
|
#
?
Feb 14, 2021 02:38
|
|
|
CMYK BLYAT! posted:and so, everyone proceeded to take their existing applications, which were not designed with kubernetes in mind, tacked on strange hacks and middleware, and made them run in kubernetes despite the applications' many protestations the more pieces of software with names like "Splort" and "⚡gesundheit" in your stack, the better off you are
|
|
#
?
Feb 14, 2021 03:13
|
|
|
Some SREs want to start using terraform to deploy to kubernetes. Their intention is to replace kustomize so I don’t fault them, but it still doesn’t seem like a good idea. Anyone gone down that path before?
|
|
#
?
Feb 18, 2021 04:53
|
|
|
Cerberus911 posted:Some SREs want to start using terraform to deploy to kubernetes. Their intention is to replace kustomize so I don’t fault them, but it still doesn’t seem like a good idea. Anyone gone down that path before? we’re using the kustomize provider with terraform. the kubernetes provider is kinda lovely, and helm is lovely so I can’t see how adding terraform to helm would help anything. we did fork the kustomize provider to allow creating an overlay in terraform code rather than needing one on the filesystem—I think that’s the missing piece to make terraform with k8s not suck. if that sounds interesting, DM me and I’ll link you to the repo. we do occasionally use the k8s provider, but only for cases that kustomize can’t handle. creating resource names dynamically, for instance I say the kubernetes provider is kinda lovely for mainly two reasons. first, fields that you don’t fill in on a resource don’t get the k8s default, they get zeroed. that means naively translating manifests from helm or kustomize to the k8s provider won’t work—stuff may apply, but it won’t do what you expect, or what it does for everyone else. second, you have to translate every manifest you find from yaml to .tf, which adds significant drag
|
|
#
?
Feb 18, 2021 05:11
|
|
|
Cerberus911 posted:Some SREs want to start using terraform to deploy to kubernetes. Their intention is to replace kustomize so I don’t fault them, but it still doesn’t seem like a good idea. Anyone gone down that path before? sounds like its going to be their problem
|
|
#
?
Feb 18, 2021 09:30
|
|
|
ive started telling folks that koobernetis is a good solution if you require a massive distributed ad serving infrastructure platform that mostly works. it basically guarantees you will have some requests fail, which can be good or bad for you and your developers. its good in the sense that your developers must design error handling and retry into whatever the hell is calling into koobernetis: the js in the browser or java applet or whatever. the devs should have been doing this all along and itll kinda force them to at least think about it before they declare its too hard and just make the end user hitting the page reload button be their planned error recovery solution. its bad because see prev statement. your devs arent that good anyway don’t worry about it the end user will be the retry logic. oh, you wanted to run something that doesnt speak http/https ??? oh, its stateful? oh its stateful aaaand it has transactions that span out to other service providers that are really really painful and expensive to unwind if our transaction fails?? yeah maybe dont run that in koob. oh sure you caaaan run that in koob i guess, if your devs are really really good. ( they are not that good ) is it just me, or does it seem like all the silver bullet hotness for infrastructure over the last ~15 years has been built around solving one specific problem for one of the cloud providers and they all do it by punting all of the other harder problems higher up the stack to make it the app devs problem?
|
|
#
?
Feb 18, 2021 14:07
|
|
|
yeah. I use terraform to deploy my kubernetes clusters but even there it kinda sucks. as for deploying things into K8s, terraforms ideas about state don’t mesh well with k8s. tell them that Argocd is the sorta refactor tool of choice there.
|
|
#
?
Feb 18, 2021 14:09
|
|
|
freeasinbeer posted:yeah. I use terraform to deploy my kubernetes clusters but even there it kinda sucks. oh yeah i forgot to mention that we don't use terraform at all in our CI/CD process. that's all handled by separate tooling. terraform and the kustomize provider are for managing the cluster and infrastructure services: kube-proxy, coredns, traefik, datadog agent, fluent bit, etc
|
|
#
?
Feb 18, 2021 18:56
|
|
|
Nomnom Cookie posted:oh yeah i forgot to mention that we don't use terraform at all in our CI/CD process. that's all handled by separate tooling. terraform and the kustomize provider are for managing the cluster and infrastructure services: kube-proxy, coredns, traefik, datadog agent, fluent bit, etc argocd and app of apps is the de facto choice imo, if using helm. if just kustomize then flux works I guess.
|
|
#
?
Feb 18, 2021 22:25
|
|
|
like I can’t stress enough how bad terraform and K8s api play together.
|
|
#
?
Feb 18, 2021 22:26
|
|
|
grafana does a deece job of hiding conveniences behind the paywall i feel like
|
|
#
?
Feb 19, 2021 16:54
|
|
|
gently caress kubernetes
|
|
#
?
Feb 20, 2021 07:56
|
|
|
does this make nomad the canonically good orchestrator or are they all bad
|
|
#
?
Feb 21, 2021 00:16
|
|
|
more like no users kinda hard to believe that what’s exciting to everyone right now is basically just a process scheduler, but here we are
|
|
#
?
Feb 21, 2021 08:06
|
|
|
as a young lad i was always a bit perplexed with old greybeards who refused to learn all the cooler modern stuff and went on about ye olde trash. but as i have opted to never learn the intricacies of web tech or container orchestration i now see that i am well on the path myself.
|
|
#
?
Feb 21, 2021 13:46
|
|
|
the whole k8s thing feels like everybody collectively gave up on building server applications that can be configured and reloaded and just agreed to install factories that could create dumb murderable micro computers by the thousand. i don't care for it in most cases
|
|
#
?
Feb 21, 2021 19:33
|
|
|
I’ve seen several homegrown dynamic config schemes in my career and they all sucked in different ways. same for reloading. nginx does an ok job of it but almost nothing else does
|
|
#
?
Feb 21, 2021 21:34
|
|
|
|
| # ? Jun 10, 2024 01:16 |
|
|
Jonny 290 posted:the whole k8s thing feels like everybody collectively gave up on building server applications that can be configured and reloaded and just agreed to install factories that could create dumb murderable micro computers by the thousand. i don't care for it in most cases Farmer Crack-rear end posted:i don't deal with code so my cynical assumption is that a lot of "scaling" is basically "if we can fully automate spinning up and tearing down server instances, it won't matter how often our shoddy code crashes!"
|
|
#
?
Feb 21, 2021 21:49
|
|
