|
Sistergodiva posted:I am having a horrible time trying to get websockets to work on elastic beanstalk. How flexible are you with the implementation details? Have you looked at the new API Gateway support for WebSocket?
|
#
¿
Mar 16, 2019 23:32
|
|
|
|
| # ¿ May 10, 2024 15:40 |
|
|
Soaring Kestrel posted:Hello! I am trying to build a DynamoDB table and am getting a little bit confused. I've been reading a lot of the documentation but I'm still not 100% clear, since this is my first time actually using DynamoDB. It is a pretty simple use case so mostly I am asking for a sanity check if what I am putting together makes sense. Some questions: 1. "grab all the comments with a "post_uid" field matching a condition" - is "post_uid" the unique id for the associated blog post? What is the condition that needs to be matched? Equality? 1. Where are the blog posts themselves stored? How big are they? 2. How many comments can be made on one blog post? How big are the comments? 3. I assume "ts" = timestamp, how are you using this? It sounds like you're wanting do some fancy queries, which is outside DDB's capabilities and sweet spot. It seems to me your use case is "grab everything associated with a given blog post", which DDB is very good at. You can definitely do this with one table and without a GSI. Adding either of those would add a lot of complexity and downsides and I don't see any benefits to justify it.
|
|
#
¿
Jun 6, 2019 17:32
|
|
|
Soaring Kestrel posted:Ugh yeah rereading there was some confusion there. With no limit on the number, you'll have to store comments as separate records in your table (the reason I asked is if your limit/size is low enough it could be feasible to store them all in one record for the post). Based on what you're written so far, I would use post_uid as partition key, and then a combination of parent comment id (0 = top level comment), timestamp, and comment id (in case two people post comments at the same time) for the sort key: postId(partition key)-------------------sortKey(sort key) -------------------------------------------------------------------- 075fd5b5-5029-4ee0-b456-eac9b989e2c0----0:2019-06-06T18:12:31+00:00:1 075fd5b5-5029-4ee0-b456-eac9b989e2c0----0:2019-06-06T18:12:42+00:00:2 075fd5b5-5029-4ee0-b456-eac9b989e2c0----0:2019-06-07T18:11:54+00:00:3 075fd5b5-5029-4ee0-b456-eac9b989e2c0----1:2019-06-08T18:15:41+00:00:4 075fd5b5-5029-4ee0-b456-eac9b989e2c0----3:2019-06-09T18:16:18+00:00:5 ... You should still store the constituent parts of the sort key as separate fields (good practice). You can then use a query with a string-based key condition on the sort key to find only top-level comments, and sort it descending (ScanIndexForward=false). Then to tie comments to their parents you can either do separate queries for each top-level comment (begins_with on the sort key), or just do a single second query to find replies and their parent ids with an ascending sort order, and match them up in-memory. It's a trade-off between performance and and Lambda compute cost. You'll also need to deal with paginating comments and cleaning up comments once the post itself is deleted (if that happens). Ideally the post itself would be stored in the same DDB table (with a spillover to S3 if necessary).
|
|
#
¿
Jun 6, 2019 19:37
|
|
|
PierreTheMime posted:Continuing on my adventures in S3, what's the preferred method for unzipping large files when they land in a space? I already have the code ready to extract it via Lambda/Java which works, but a coworker has it in his mind he'd prefer invoking an unzip from an Docker-spawned EC2 which seems a little much to me. What do you mean by “when they land in a space”? “Docker-spawned EC2” doesn’t make sense to me and sounds more complicated than using Lambda.
|
|
#
¿
Jun 22, 2019 17:26
|
|
|
PierreTheMime posted:I mean when a file is uploaded to the S3 bucket. I phrased it that way because I plan to control whether the unzip happens based on a folder metadata value similar to what I have for the movement function. Your coworker is an idiot. Lambda’s memory limit can go up to 3GB, but even if the 512MB were true, you can unzip any size file if you do it in a streaming way.
|
|
#
¿
Jun 22, 2019 18:01
|
|
|
Arzakon posted:Co-worker is an idiot but there is only 512mb of scratch space in tmp on each function invocation. True, but from what PierreTheMime has said that’s still not a problem. What’s supposed to happen after the unzip?
|
|
#
¿
Jun 22, 2019 21:47
|
|
|
For future reference, you should turn on logging for both APIG and Lambda and check the appropriate CloudWatch logs. The first invocation after saving the function will always be a cold start, so the symptoms you were seeing definitely pointed towards something being not quite stateless.
|
|
#
¿
Aug 7, 2019 07:24
|
|
|
Yep that’s spot on. In fact this is what we do internally at AWS. Using IAM users is strongly discouraged and there is a lot of work ongoing to get rid of existing ones.
|
|
#
¿
Aug 19, 2019 06:28
|
|
|
Are you looking for something like Step Functions? If not, can you give a more specific example of what you’re trying to do?
|
|
#
¿
Aug 20, 2019 02:16
|
|
|
Cancelbot posted:I loving got it! Congrats!  What org will you be in?
|
|
#
¿
Sep 10, 2019 17:04
|
|
I'll be a Senior TAM in 2 months (notice period boo).
|
You want to Google for availability SLAs (not failure rates). Here are AWS' own SLAs for availability: https://aws.amazon.com/legal/service-level-agreements/. Your availability is currently 99.9% (100 - 0.1), which is not great, not terrible. At AWS, that's where we start paying our customers credits for most services. Your advertised SLA is apparently 98.5% (100 - 1.5), which is pretty terrible. I wouldn't want to use such a service personally. Seems like you could make a stronger promise there.
|
|
#
¿
Sep 24, 2019 20:08
|
|
|
Agrikk posted:Be careful how you book sessions. If you are sloppy you can walk ten miles in a single day, like a customer did. New service: AWS Exercise.
|
|
#
¿
Nov 21, 2019 03:17
|
|
|
a hot gujju bhabhi posted:Vent time. Please also vent on the feedback forms, there should be feedback links on every doc page. I totally agree our docs are poo poo. As for your .NET question, you could look into .NET Core, if your project builds on it. It runs on Linux too.
|
|
#
¿
Dec 28, 2019 06:13
|
|
|
|
|
#
¿
Dec 31, 2019 09:35
|
|
|
a hot gujju bhabhi posted:Sorry for a pretty newbie question, but I'm looking at this: https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html Not sure if I understand the question right; the three steps in the table are the only ones that happen for the original instances. There is no installation happening on them.
|
|
#
¿
Jan 2, 2020 01:53
|
|
|
dividertabs posted:DynamoDB on-demand pricing. They don't tell you this in their documentation* but it still throttles. It appears to just be a wrapper hiding Dynamo's autoscaling feature plus a different billing scheme. Mildly triggered AWS engineer here (not from the DDB team). Can you provide more detail on the circumstance under which you got throttled from your on demand table? It may throttle for a short time before it scales up, but once scaled it should subsequently be able to handle the same load in the future. You can shortcut the initial throttling by setting an appropriate provisioned capacity on the table before switching it to on demand.
|
|
#
¿
Apr 15, 2020 07:17
|
|
|
Nomnom Cookie posted:Athena is managed presto and totally unsuitable for anything. Fixed.
|
|
#
¿
Apr 15, 2020 07:18
|
|
|
dividertabs posted:Say we want to scale from 0 to 1,000,000 reads/sec in 1 second. S3 can do it (with good P95 latency, and poor p99 latency). It was not surprising to me that Dynamo couldn't do it. It was surprising to me that Dynamo couldn't do it, and the documentation for Dynamo on-demand, which emphasizes that it 'instantly accommodates customers’ workloads as they ramp up or down', didn't mention it. Bolding my own. No judgment on whether my ideal behavior is reasonable. It should only throttle the first time you scale up that table though. Never if you use the shortcut I mentioned. See also: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html#HowItWorks.InitialThroughput I agree our docs are generally poor.
|
|
#
¿
Apr 16, 2020 07:08
|
|
|
From the link you posted:quote:This topic explains authenticating requests using Signature Version 2. Amazon S3 now supports the latest Signature Version 4. This latest signature version is supported in all regions and any new regions after January 30, 2014 will support only Signature Version 4. For more information, go to Authenticating Requests (AWS Signature Version 4) in the Amazon Simple Storage Service API Reference. Make sure you're implementing SigV4, not SigV2, which is ancient. Read this: https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html
|
|
#
¿
Apr 21, 2020 21:10
|
|
|
This library contains the official AWS C implementation for SigV4 signing: https://github.com/awslabs/aws-c-auth. It’s Apache 2 licensed.
|
|
#
¿
Apr 22, 2020 07:18
|
|
|
PierreTheMime posted:What’s the best deployment mechanism for a Step Functions state machine and underlying resources? Lambda, Batch are primary services, but presumably we’d also want associated IAM roles, DynamoDB table, SSM parameter store, secrets, etc. What are you using now? CloudFormation will let you manage all that infrastructure.
|
|
#
¿
Jun 2, 2020 06:27
|
|
|
Is this interview at AWS, or a company using AWS? If it’s at AWS, avoid using a relational databases. I would recommend that in either case actually, but we really don’t like relational databases at AWS and this is definitely in DynamoDB’s sweet spot.
|
|
#
¿
Jul 27, 2020 03:03
|
|
|
Hughmoris posted:Side note: seems like prime opportunity for a good tutorial here. Amazon Training walks you through all this via the console UI but it would be great to then have the same steps via Terraform. If I can complete this then I might try a Terraform write-up. Because Terraform is a competing product. CDK is what we want customers to use. Should be able to find tutorials for it.
|
|
#
¿
Sep 3, 2021 06:45
|
|
|
Internally at AWS we often use Step Functions state machines to implement asynchronous API operations.
|
|
#
¿
May 14, 2022 23:32
|
|
|
Tulenian posted:Huh I figured the folks doing it would bias towards simple workflow service instead. SWF is also used a lot in more old school teams. I’m all about that serverless life though. I’m probably leaving soon too…
|
|
#
¿
May 15, 2022 07:16
|
|
|
Money.
|
|
#
¿
May 16, 2022 07:34
|
|
|
Scrapez posted:Does AWS not pay well or just typical have to move to a different company to get a big bump in $$? I was going to effort post but it’s already been explained pretty well in the last few posts. I joined in 2016 so my cliff was 2020. My compensation for 2022 has dropped 40% compared to 2020. I’m way below the range for my role and the correction for that is way too slow. So I don’t really have a choice. I like my job but can’t afford to stay.
|
|
#
¿
May 17, 2022 04:21
|
|
|
After 6 years at AWS I can assure you that even there, most engineers only vaguely know HTTP, and anything below that might as well be magic.
|
|
#
¿
Jul 12, 2022 02:54
|
|
|
In a stunning display of Bias for Action and Insists on the Highest Standards, I quit my job at AWS last week. But yes, having a solid story for each leadership principle will help enormously in getting the job because every candidate is directly scored on how they performed on each one during interviews.
|
|
#
¿
Jul 30, 2022 05:44
|
|
|
Agrikk posted:Congratulations! And you left as the stock lost a third its value so double well done! Double thanks!
|
|
#
¿
Jul 30, 2022 23:45
|
|
|
And that reason is bucket sniping.
|
|
#
¿
Aug 23, 2022 16:33
|
|
|
It’s a bit more nuanced than that, but pretty much, yeah. There are also more steps between EC2 and Lambda (ECS on Fargate, ECS on EC2).
|
|
#
¿
Oct 3, 2022 20:26
|
|
|
I’m very happy I left AWS in July and now work for a private company. So nice to not have to follow the stock market so closely. Found out recently my last manager also left, and he was largely keeping things afloat in my org.
|
|
#
¿
Nov 3, 2022 23:19
|
|
|
jiffypop45 posted:Bots required manual approval via a team that had like 5 people and there was no process in place for that or many other things. There was no way to join private channels without being added by hand. FC/contract employees were unable to use slack. Emojis gave people with epilepsy seizures at least twice due to bad internal actors. Amazon tried to make retention two weeks likely to cover something up only to brown out slack globally when they rolled it back after hiding whatever they sought out to hide. :party_parrot:
|
|
#
¿
Dec 17, 2022 01:42
|
|
|
CDK for life.
|
|
#
¿
Mar 2, 2023 04:27
|
|
|
Do you need SF here? Sounds like a perfect use case for SQS + DLQ with a Lambda trigger off the queue. Then you get the redrive functionality for free.
|
|
#
¿
Jun 11, 2023 05:54
|
|
|
I would go S3 -> SNS -> SQS -> Lambda. Adding SNS is optional but provides decoupling and allows you to do add some other processing later if you want. SQS (with DLQ) gives you great error recovery and let’s you absorb bursts nicely.
|
|
#
¿
Jun 13, 2023 19:09
|
|
|
|
| # ¿ May 10, 2024 15:40 |
|
|
2 and 3 are the standard two you would use at a minimum and very widely used. Depending on how you are receiving from the queue, you could additionally look at the number of empty receives. If that goes up you are probably overscaled. Ideally your receiving side is autoscaled though (i.e., use Lambda). Generally though, for alerting, the ones you already have are great. Beyond that I just look at the metric page in the console for the queue as needed.
|
|
#
¿
Feb 6, 2024 19:37
|
|