Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Put the CSVs in S3, use a glue crawler to read the CSVs and output it to RDS, I think.

Alternatively use Athena to query the tabular data in S3 directly.

Adbot
ADBOT LOVES YOU

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

You don't need a WAF to limit ingress IP - you can do that with regular security groups. It's a fine thing to add if you want, though

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

I just got asked about Aurora multi-region multi-master, which doesn't exist. Now I get to have a whole bunch of meetings to determine business requirements and figure out what architecture will actually suffice

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Scrapez posted:

Is there any way to do multi-region with Aurora Serverless? We have a database that has very low utilization with occasional small spikes so it's perfect for serverless but I need to have it be multi-regional.



They claim serverless v2 is compatible with Aurora global, which is multi-region. Mysql only, and it's in preview.


https://aws.amazon.com/rds/aurora/serverless/ posted:


Aurora Serverless v2 (Preview) supports all manner of database workloads, from development and test environments, websites, and applications that have infrequent, intermittent, or unpredictable workloads to the most demanding, business critical applications that require high scale and high availability. It supports the full breadth of Aurora features, including Global Database, Multi-AZ deployments, and read replicas. Aurora Serverless v2 (Preview) is currently available in preview for Aurora with MySQL compatibility only.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Amazon's stock compensation is weighted as follows:

5% granted on the 1y anniversary
15% granted on the 2y anniversary
20% every six months after that.

So for someone hired 2 years ago just entering their 3rd year of tenure, their stock-based compensation (which could be 50% or more of their total comp) just took a 25% dive.

Add to that the fact that 1) compensation changes happen once a year, 2) are frequently heavily weighted 2 years out, 3) Amazon is one of the lowest compensating of the FAANGs (gotta love that frugality leadership principle) and 4) new hires get paid current market rate but tenured peoples' comp lags quite a bit... Well, job hopping starts to look pretty dang attractive

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

You might be able to get a proactive dive and save (without another offer in hand) depending on your manager and org, but it might be just as easy to take another job and boomerang if you want. #pay-equity and #pay-equity-discussion are good resources if you're not aware of them.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Quebec Bagnet posted:

You may know this already, but I'll point it out anyway. The CLI is automatically generated from the API specification, and the CFN schema closely wraps the underlying create/modify APIs (to the point where CFN documentation sometimes describes parameters in ways which only make sense if you consider the underlying API call). So figuring out how to do something in CLI or API will be very easy to translate to CFN.

The console also usually reflects the API fairly well, and it almost always issues the same API calls so you could sniff traffic in your browser's development tools to figure out what it's doing.

This is usually true, but it's also the case that some things simply aren't implemented in CFn yet that you can do either through the CLI or the console

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Yes. A bastion host gets you into the vpc, and then either manage directly from the bastion host or double hop onto another ec2 server for management

And/Or direct connect / vpn / workspaces

Edit: not just gui management. You couldn't read from or write to it with SQL statements unless you had network connectivity. That's either public access (bad), vpc access, or dx/vpn

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

I tried doing an earthquake tracker too, but I wanted to see if I could do the ETL with Glue.

Turns out I couldn't, so I gave up.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

If you're a megacorp with a sweet private licensing agreement with Microsoft

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Corey Quinn's company just posted an opening for a cloud economist. I am definitely not qualified and I'm also applying.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Oh drat, good for you. They're probably a great place to work.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Yeah, no, I read your post. Good for you for not rejecting yourself on their behalf. Make them do it.

I have a *very* hard time believing Duckbill would hire me, let alone get me past the resume screening stage. But if they're going to say no, I'm not going to do it for them.

Happiness Commando fucked around with this message at 23:53 on Jul 19, 2022

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Happiness Commando posted:

Corey Quinn's company just posted an opening for a cloud economist. I am definitely not qualified and I'm also applying.

One of the things that I entered into the application form was that Duckbill prioritizes caring about people and having no oncall requirements. And then LOL got woken up at midnight when I got paged for something that was complete BS

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Duckbill rejected me very directly and kindly. Then I asked for feedback and got some, again, direct and kind and well targeted. If I'm still in the workforce in 10 years, I might try again.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

They are pr bullshit. It's just that the call is coming from inside the house

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Does anyone here have extensive operational experience with Lambda and want to pick up a short consulting gig? $Oldjob is starting to look at it and they want to compare notes with someone with practical experience. As far as I can tell, the scope is advisory only, not building a solution. Language is C# / Powershell, FWIW

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

EC2, S3, RDS are usually the big ones for most enterprises.

Cost savings measures:

* Are they using reserved instances or compute savings plans?
* How much inter-AZ or inter-region traffic is there, and can it be rearchitected around?
* VPC endpoints instead of making API calls over the public internet?
* GP3 vs. GP2 EBS volumes?
* too many old snapshots sticking around
* Are they using any S3 tiers other than the standard one?
* modern instances classes vs. old ones
* rightsizing with Compute Optimizer
* rightsizing EBS volume size
* are they using datacenter-like usage patterns (like querying an S3 API every second 24/7 for if a new file has showed up vs using a message bus)
* transit gateways are expensive


Finops tools:
CUDOS and other CID dashboards
Cost Anomaly detector
Cost Categories/Budgets when paired with intelligent tagging strategies

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

As the person above said, that's base pay only. The only way to figure out actual total compensation is to look at levels.fyi and do some fuzzy math. Also depending on the market your total compensation may exceed or trail your comp target, which may get made up for with additional grants, which will be on a (2 year?) vesting schedule

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

These numbers are totally made up. OP gets hired when AMZN is at $100/share. If OPs total comp target is 200k, comprised of 150k base and 50k RSUs and then the share price takes a dive to $80/share, OP is not guaranteed an in-year vesting grant of RSUs to make up the shortfall. OP might be required to take it on the chin, because "we're all builders and when we build well, the market does well, and then your actual comp will exceed the TCT"

If the market tanks right before a vesting date, OPs compensation may very well trail their TCT. If it stays down, they wont get a top off of RSUs to get them to hit their TCT until the next annual review (edit: or promo), and those RSUs will likely vest on a 2 year schedule.

Happiness Commando fucked around with this message at 19:03 on Sep 4, 2022

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

I bailed from my AWS team after one year. Pursued an internal transfer (while also applying externally) and got it. So loving glad to be out of that nightmare.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Yeah. You use a two or three tier architecture. Here's a random search result

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Today I learned that if you try to cfn-lint a YAML template that has !!Ref in it (instead of the correct !Ref) everything blows up in spectacular fashion. Since I'm running python on Windows and not particularly good at it, this took me an hour to resolve and included multiple reboots, reinstalls, path changes, etc.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

That should be enough. Sometimes arns or resource names can leak sensitive info if you're a corporation. Frequently account IDs are blurred out, even though they're not actually sensitive. Corey Quinn has a tweet somewhere of some aws exec definitively stating that account IDs shouldn't be considered sensitive. But it's usually done anyway.

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Dumb question but are you sure you're running on EC2 vs Fargate?

Adbot
ADBOT LOVES YOU

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

Hed posted:

I have a pretty small Postgres RDSinstance on a db.t4g.medium that is capping out of IOPS and as a result there's a whole lot of WAL wait. We have a 20GB disk on gp2 so expected IOPS is 60.

I shouldn't have let it get to this point, but is there a graceful way to increase IOPS with no downtime? My options appear to be:

  • convert the disk to gp3 (no way this can be done online right?)
  • make the disk bigger to scale the "IOPS = Volume size [GB] * 3", but I don't need it larger

Is there a good way to spin up another instance into the cluster with enough IOPS and gracefully transition to it? I know I could pgdump/restore but would rather not have downtime if possible.

gp2 to gp3 should be totally safe and without downtime. Also you should have a backup, because that's a good practice. Documentation for aws cli here

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply