|
this is genuinely no different than it has been for the past two decades at least i've been told a couple of times the design philosophy of websphere was "make bad apps run good"
|
#
?
Feb 21, 2021 21:55
|
|
|
|
| # ? May 20, 2024 03:07 |
|
|
I guess I can see how k8s would be unwelcome if your job is setting up and decommissioning servers scaling is mostly about not caring about how bloated and slow your code is. it is for when you need to throw more hardware at simple problems so often that this process is worth automating
|
|
#
?
Feb 21, 2021 21:57
|
|
|
back in my drupal days customers would scale out their web or DB tiers astronomically and be clenching hard when poo poo kept slowing down and not coping with traffic spikes very often, you could spend less than half an hour checking logs and figure out that some checkbox or config at app level could be changed (like, turning on caching) and poo poo would come right and would use like 1/8th of their scaled up capacity in some senses having the apps deployed on the butt was good because poo poo kept working, but it was all band aids on band aids. you are an infra person its easier to just throw more capacity at it than yell at developers in a domain you don't know much about
|
|
#
?
Feb 21, 2021 23:39
|
|
|
poo poo, devops means its the devs slapping on the band aids instead. one of our teams fairly recently had to actually look one of their services when they made the pod too big to schedule. that service's entire responsibility was querying mysql and putting the results in elasticsearch. our standard worker nodes have 64 gigs of ram k8s can be a good force multiplier but it can't replace knowing wtf you're doing
|
|
#
?
Feb 22, 2021 01:55
|
|
|
There's a reason why real good SREs can command eyewatering salaries at a FAANMG. There's very little cross-domain talent out there.
|
|
#
?
Feb 22, 2021 03:44
|
|
|
nomad requires you to reinvent a bunch of included kubernetes use cases, if you want beyond the most basic use case. on the flip side kubernetes is becoming super bloated. everything sucks. in the end I’d say K8s > nomad, because you don’t need to write inner source, or deal with some companies half rear end open source replacement for key features.
|
|
#
?
Feb 22, 2021 05:25
|
|
|
nomad has me writing a bunch of go templating which sucks rear end
|
|
#
?
Feb 22, 2021 07:25
|
|
|
freeasinbeer posted:deal with some companies half rear end open source replacement for key features. But that's 100% the kubernetes experience ?
|
|
#
?
Feb 22, 2021 10:01
|
|
|
k8s exists due to google cargo culting and associated marketing, FOMO, and need for new HR screening check boxes
|
|
#
?
Feb 22, 2021 13:52
|
|
|
if your application makes money by attaching itself to the internet firehose, kubernetes is worth running, especially if you make money off of serving internet ads or if you have any other kind of 2-phase revenue generating process that runs into fun scaling problems like "this floor of the data center is full". in those situations being able to quickly rebalance various internal processes based on metrics like "ads we bought but haven't redistributed updated budget information for" and poo poo like that is arguably worth the complexity. using kubernetes to host web applications or random tools crap though is absolutely just google cargo culting, resume driven infrastructure, etc.
|
|
#
?
Feb 22, 2021 17:29
|
|
|
Kazinsal posted:k8s exists due to google cargo culting and associated marketing, FOMO, and need for new HR screening check boxes all these big data and cloud native scaling stacks are just recreating tech google made 20 years ago
|
|
#
?
Feb 22, 2021 17:46
|
|
|
its extra funny that approx 100% of the big data workflows that exist outside of google are some wrapper around "just put it all in s3". if you told me google just used s3 also i would absolutely believe it
|
|
#
?
Feb 22, 2021 17:50
|
|
|
well the alternative is using gcs *shudder*
|
|
#
?
Feb 22, 2021 18:03
|
|
|
12 rats tied together posted:its extra funny that approx 100% of the big data workflows that exist outside of google are some wrapper around "just put it all in s3". if you told me google just used s3 also i would absolutely believe it s3 is good though
|
|
#
?
Feb 23, 2021 00:28
|
|
|
k8s is good in theory because it turns a bunch of hard operations and provisioning problems into just some declarative files. it's bad in practice because every abstraction layer is software and software is poo poo
|
|
#
?
Feb 23, 2021 01:31
|
|
|
jre posted:s3 is good though s3 is "good" for a specific subset of data storage problems. So of course developers try to use it like a global cache.
|
|
#
?
Feb 23, 2021 05:03
|
|
|
jre posted:s3 is good though ate poo poo on live tv posted:s3 is "good" for a specific subset of data storage problems. So of course developers try to use it like a global cache.
|
|
#
?
Feb 23, 2021 06:40
|
|
|
12 rats tied together posted:yes. stored procedures are the worst.
|
|
#
?
Feb 23, 2021 09:23
|
|
|
s3 has a sql interface
|
|
#
?
Feb 23, 2021 16:21
|
|
|
Nomnom Cookie posted:s3 has a sql interface 
|
|
#
?
Feb 23, 2021 16:22
|
|
|
it's good, actually. it operates on a single object at a time and it requires that the object is in a particular format, parquet is 100% one of the formats but i don't remember any of the other ones. for a lot of use cases the interface is a massive perf and cost boost because you don't need to maintain pools of compute (EMR, lmao) that are equipped to download, unzip, and decode whatever bullshit you're writing
|
|
#
?
Feb 23, 2021 18:41
|
|
|
it’s called athena op there’s also s3 select which is a babby version
|
|
#
?
Feb 23, 2021 19:46
|
|
|
i think redshift has some "just run against these s3 buckets" stuff these days too which is nice since those clusters can be pretty expensive as they size up
|
|
#
?
Feb 23, 2021 20:16
|
|
|
it shouldn’t be that horrifying, s3 isn’t ebs
|
|
#
?
Feb 23, 2021 20:31
|
|
|
love to wake up to a wall of red alarms that might have been prevented if the hardware we ordered from juniper last year wasn't delayed due to the global chip shortage
|
|
#
?
Feb 23, 2021 22:35
|
|
|
hobbesmaster posted:it shouldn’t be that horrifying, s3 isn’t ebs i just get irrationally scared at the thought of s3 being thrown at literally everything
|
|
#
?
Feb 23, 2021 23:33
|
|
|
Asymmetric POSTer posted:i just get irrationally scared at the thought of s3 being thrown at literally everything the reason it exists is to drive a subset of data transformation directly into the s3 servers, if done well dramatically reducing how much data you have to load into your EMR cluster. It's not meant to be a s3 as a database
|
|
#
?
Feb 24, 2021 01:49
|
|
|
12 rats tied together posted:i think redshift has some "just run against these s3 buckets" stuff these days too which is nice since those clusters can be pretty expensive as they size up it’s called redshift spectrum op, so you can run queries seamlessly across data sets with hotter storage in redshift and colder stuff in s3
|
|
#
?
Feb 24, 2021 06:24
|
|
|
thank you, that sounds like a good feature. my experience is that on-staff "data warehouse" people are some of the worst engineers to work with so i'm 100% always the guy saying to put stuff in redshift instead of running vertica on ec2 in addition to our 800k monthly EMR spend for some reason i had a data warehousing team request like 90 something ec2 vertica nodes on i3 class instances, which they wanted us to configure some awful software raid poo poo on, we delivered the servers and the first thing they did was shut over half of them off "to save money", which wiped the instance store volumes and associated configs
|
|
#
?
Feb 24, 2021 17:32
|
|
|
my homie dhall posted:the raison d'etre for kubernetes is ostensibly getting economies of scale for your ops and release teams. if you're trying to solve those two specific problems, then it's not bad. if you don't have either of those problems I don't know why you would use it
|
|
#
?
Feb 24, 2021 18:55
|
|
|
agree except imho "good" ops and release teams were already building those abstractions (they are not very complicated). having done SRE tech interviews for a while though i will readily admit that "good" ops and release teams might as well not exist and the field overall is home to some of the most deranged engineering i've ever seen like, if your choices are "a server that runs chef-solo and updates aws autoscaling group desired counters on every convergence" and "kubernetes", definitely run kubernetes. 12 rats tied together fucked around with this message at 19:26 on Feb 24, 2021 |
|
#
?
Feb 24, 2021 19:24
|
|
|
12 rats tied together posted:i had a data warehousing team request like 90 something ec2 vertica nodes on i3 class instances, which they wanted us to configure some awful software raid poo poo on, we delivered the servers and the first thing they did was shut over half of them off "to save money", which wiped the instance store volumes and associated configs lol
|
|
#
?
Feb 24, 2021 19:33
|
|
|
12 rats tied together posted:i had a data warehousing team request like 90 something ec2 vertica nodes on i3 class instances, which they wanted us to configure some awful software raid poo poo on, we delivered the servers and the first thing they did was shut over half of them off "to save money", which wiped the instance store volumes and associated configs
|
|
#
?
Feb 24, 2021 19:34
|
|
|
my homie dhall posted:the raison d'etre for kubernetes is ostensibly getting economies of scale for your ops and release teams. if you're trying to solve those two specific problems, then it's not bad. if you don't have either of those problems I don't know why you would use it wrong, the raison d'etre is inspiring spicy takes on the name pronunciation
|
|
#
?
Feb 24, 2021 19:42
|
|
|
it's called redshift because you're supposed to run away from it as fast as possible
|
|
#
?
Feb 24, 2021 20:50
|
|
|
12 rats tied together posted:agree except imho "good" ops and release teams were already building those abstractions (they are not very complicated). lomarf
|
|
#
?
Feb 24, 2021 20:55
|
|
|
Progressive JPEG posted:it's called redshift because you're supposed to run away from it as fast as possible it’s called redshift cos it’s supposed to move people off of oracle for OLAP
|
|
#
?
Feb 24, 2021 21:03
|
|
|
its called redshift because you put the servers far away in amazon's cloud and then because they are farther away you think about them less often
|
|
#
?
Feb 24, 2021 21:05
|
|
|
you cant spell "the natural order is chaos" without "NAT"
|
|
#
?
Feb 25, 2021 02:09
|
|
|
|
| # ? May 20, 2024 03:07 |
|
|
Nomnom Cookie posted:we’re using the kustomize provider with terraform. the kubernetes provider is kinda lovely, and helm is lovely so I can’t see how adding terraform to helm would help anything. we did fork the kustomize provider to allow creating an overlay in terraform code rather than needing one on the filesystem—I think that’s the missing piece to make terraform with k8s not suck. if that sounds interesting, DM me and I’ll link you to the repo. we do occasionally use the k8s provider, but only for cases that kustomize can’t handle. creating resource names dynamically, for instance It was a switch from kustomize + kapp (an actually good deployment/diff tool) to directly using the kubernetes provider. Looks like the idea is dead now after they figured out we can't replace all the kustomize we use (for setting up namespaces/rbac/clusterwide resources on clusters) with terraform in one week like they originally planned.
|
|
#
?
Feb 25, 2021 03:13
|
|