|
This is the sort of thing we use ZooKeeper for, though probably a bit much of a hassle to set up for just this one thing.
|
#
¿
Nov 7, 2017 17:49
|
|
|
|
| # ¿ May 20, 2024 23:27 |
|
|
We started our migration from baking AMIs to docker/mesos/marathon deployment a few years ago, kicking and screaming. It has been hugely beneficial: among other things, building and pushing a docker image is miles upon miles faster than bringing up an EC2 instance and imaging it to bake an AMI, especially because many steps like installing infrequently updated base packages are performed less redundantly and easier to break out into individual and clearly defined steps
|
|
#
¿
Nov 11, 2017 04:04
|
|
|
I'd love for someone to help me understand this: https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance/ Specifically: quote:This S3 request rate performance increase removes any previous guidance to randomize object prefixes to achieve faster performance. That means you can now use logical or sequential naming patterns in S3 object naming without any performance implications. I've got a legacy app that, unfortunately, uses a sequential naming pattern that we've known is not ideal but for various reasons is very difficult to fix. This wording reaaallly seems to imply that it's not an issue anymore. However, since that announcement was made we've started seeing failures writing to that bucket: we'll start getting nearly 100% internal server errors and/or slow down 503s for some period of time. There are errors on reads too, but at a much lower rate. There have been several instances of this since this announcement, and it had never been a problem before. We're not seeing peak traffic, so the fact that things have gotten worse right at the same time that they were supposedly made better is highly suspect.
|
|
#
¿
Aug 5, 2018 18:13
|
|
|
Less Fat Luke posted:If you're getting errors you should probably open a support ticket with AWS; my org works with thousands of buckets at crazy levels of usage and I've never seen any of them exhibit that behavior. Oh of course, I've already done that and will continue to do so. But they haven't been of much help yet
|
|
#
¿
Aug 5, 2018 19:34
|
|
|
And the response I've gotten so far is not consistent with that quoted statement. What support has said is that performance is improved over all, but that randomizing keys is still a good idea (I totally believe that to be the case, but it's not consistent with my read of that statement). And maybe it's a coincidence that we started having these issues after the announcement, but our access patterns haven't changed and we are not hitting previous peak request rates where we had no problem before the supposed improved performance, so that's why I am skeptical
|
|
#
¿
Aug 5, 2018 19:37
|
|