Amazon Web Services - Cloud Giant Hits Hard - The Something Awful Forums

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Amazon Web Services - Cloud Giant Hits Hard

Ajaxify: May 6, 2009

Try loosening your security groups (even if they look ok) and see if that helps.

# ¿ Jan 23, 2019 08:38

Adbot: ADBOT LOVES YOU

# ¿ May 1, 2024 18:28

Ajaxify: May 6, 2009

Docjowles posted:

I just had to pull this out for appreciation. There is always a bucket called $companyname and it is always like the first thing some rando dev ever did in AWS years before the rest of the org thought about using the cloud. It will be a giant dumping ground of poo poo with no lifecycle policy and probably leaking PII.

There will also be some horrible reason you can't just easily fix it.

Verily.

Our company was serving assets from a self-named S3 bucket in the CEO/Founder's personal AWS account up until a couple years ago. The CEO hasn't written any code in probably a decade.

Given we have 2.5m concurrent users on average, It was probably quite the bill.

# ¿ Aug 23, 2022 18:11

Ajaxify: May 6, 2009

calusari posted:

I am trying to update a CloudFormation template to whitelist some countries in a CloudFront distribution:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cloudfront-distribution-georestriction.html

How can I set a default list of strings like ['US', 'CA'] for my Ref?
code:
Params:
  GeoRestrictionLocations:
    Type: List<String>
    Default: [what goes here]

CloudFrontDistribution:
  Type: AWS::CloudFront::Distribution
  Properties:
    DistributionConfig:
      Restrictions:
        GeoRestriction:
          Locations: !Ref GeoRestrictionLocations
          RestrictionType: whitelist

code:

List<Number>

Is an invalid param type.

I think you want this instead:

code:

CommaDelimitedList

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html

# ¿ Sep 2, 2022 22:27

Ajaxify: May 6, 2009

Arzakon posted:

It will not trail, it will exceed in certain markets (NYC/SFO). RSU vesting schedule is 4 years (5/15/40/40%) but first two years will have a signing bonus.

Happiness Commando posted:

As the person above said, that's base pay only. The only way to figure out actual total compensation is to look at levels.fyi and do some fuzzy math. Also depending on the market your total compensation may exceed or trail your comp target, which may get made up for with additional grants, which will be on a (2 year?) vesting schedule

With regards to RSUs, you're both right. The first grant is 4-years with a signing bonus in the first two years, but subsequent refresh grants are 2-years, starting in the next calendar year. You can get an in year refresh of your RSUs if you are promoted.

# ¿ Sep 4, 2022 18:56

Ajaxify: May 6, 2009

S3 buckets have resource-based policies attached to the bucket. It needs to also allow the same actions that you are adding in your Lambda's role.

You can also look into the Policy Simulator, to see if that helps you identify the missing piece of your policy: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html

Otherwise, everything else people have said is also correct. You need appropriate permissions on both accounts for KMS as well if you're using that. You may also be running into S3 ACL issues which are a total nightmare to deal with cross-account. This might provide some more information as well: https://aws.amazon.com/premiumsupport/knowledge-center/cross-account-access-s3/

# ¿ Sep 21, 2022 00:12

Ajaxify: May 6, 2009

ledge posted:

Two steps is not Rube Goldberg. Rube Goldberg is:
file arrives in S3,
triggers lambda to process file line by line into SQS queue, with a dummy entry to indicate the end of the file
SQS sends to Lambda which loads entries into one of two DynamoDBs (one active on empty) based on environment variables in the lambda,
When the dummy eof entry is received call another lambda
this Lambda updates the environment variables on loading lambda and reading lambda and itself about what DynamoDB to target and then deletes and recreates what was previously the active dynamoDB table.

Which I have to do as loading into DynamoDB is slow as poo poo and the file is big enough to take over 15 minutes so I can't do it all in a single lambda.

Hope you're using a FIFO queue

# ¿ Jun 25, 2023 16:43

Adbot: ADBOT LOVES YOU

# ¿ May 1, 2024 18:28

Ajaxify: May 6, 2009

Plank Walker posted:

Been migrating a bunch of AWS resource creation to CDK/Cloudformation vs manual, noticed we are getting hit with a big bill for S3 access so trying to add a VPC gateway. Working off a stack overflow answer here: https://stackoverflow.com/a/72040360/2483451, is it sufficient to just add the gateway endpoint to the VPC configuration or do I have to add some reference to the VPC to the S3 construct as well? The comment on stack overflow says VPC configuration is all that's necessary, but not much more detail.

That's correct, for Gateway endpoints (as opposed to Interface endpoints), you don't need any extra configuration to get them to work. Your machines will still resolve the S3/DDB IPs from public DNS and send packets to the public IPs of those services, but the Internet Gateway in your VPC will now intercept those packets and shunt them directly to the AWS service instead of allowing them to traverse the public internet.

For Interface endpoints, you'll need to setup DNS to resolve those service hostnames to VPC local endpoints. This can be done using AWS Private Zones in Route53. Security groups and network ACLs may also need to be updated to allow traffic from the services to the new internal endpoints.

I agree though, these endpoints really should come out of the box with your VPC. When I coded up a VPC CDK construct for my team, I added all endpoints to the VPC by default as there's no situation I can think of where you wouldn't want these enabled. It was pretty cool implementation too; I wrote an Aspect that searched the CDK application for Security Groups, and magically allowed access from any SG in the VPC to the AWS Service VPC Endpoints.

# ¿ Apr 5, 2024 16:58

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Amazon Web Services - Cloud Giant Hits Hard