Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
crazypenguin
Mar 9, 2005
nothing witty here, move along

Hughlander posted:

However, my understanding is to prevent side channel attacks the Kernel, the VM, and an LXC in a k8s pod (I guess that's just the kernel again.) . All will work against you to invalidate those optimizations. Is that correct?

I don’t think so, no. Why do you think effort to reduce cache misses wouldn’t be helpful?

Everything about side channel attacks involves context switches across permissions boundaries.

And at any rate this is something a simple benchmark would answer.

Adbot
ADBOT LOVES YOU

crazypenguin
Mar 9, 2005
nothing witty here, move along

PierreTheMime posted:

default to a “standard” client connection

Yeah, this (just use DefaultCredentialsProvider) in conjunction with assigning roles to things is the way to go.

Another thing you might check is whether the role is setup right so that batch (or ecs or whatever) can grant it properly. I don't know if this is different between ec2/lambda/ecs/whatever. But it's at least conceivable that ec2/lambda can grant the role to an instance, but ecs is unable to grant it to a container.

crazypenguin
Mar 9, 2005
nothing witty here, move along

Thermopyle posted:

I don't know AWS well and I've come in to admin a project that is currently hosted across multiple EC2 instances and I have questions.

1. One thing that I think could be better is that the project has a redis server acting as a task queue and python workers running on one instance. If i understand correctly, if I'm using AWS, I should probably move those python workers over to Lambda, no? Then I can just eliminate redis and replace the code that sends tasks to the workers via redis with code that starts lambda tasks (or whatever the lambda terminology is)?

2. Some of these instances call HTTP endpoints on other instances via public dns addresses...I should just use the VPC local address, right?

1. You're probably looking for: https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html

2. I don't understand the question, are you looking for this? https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/hosted-zones-private.html

e:fb; at least I've got links!

crazypenguin
Mar 9, 2005
nothing witty here, move along
IMO, you should never actually be logged into bastions.

Create local SSH configs that use ProxyJump to bounce through the bastion. Then it’s totally transparent you’re using a bastion at all. scp away.

crazypenguin
Mar 9, 2005
nothing witty here, move along
If you’re using Put Metric, you might want to look into Embedded Metric Format with logs instead. IIRC, it can be cheaper from lambdas because lambdas can’t do batching or something like that.

That might do away with that cost problem.

crazypenguin
Mar 9, 2005
nothing witty here, move along
Maybe look at AWS CDK. It's basically a tool to generate CloudFormation.

It seems to me like you're going for ASGs because "that's the thing that spins up a lot of servers at once" but you can just write a loop in CDK and make whatever you want.

crazypenguin
Mar 9, 2005
nothing witty here, move along

Fcdts26 posted:

I did some more work on it last night and it seems that the issue comes from switching the load balancer listeners from instance based target groups to ip based ones. Even deleting the listeners and recreating them it has issues.

NLB has different behavior in how it forwards packets between IP and instance target groups. With instances, it preserves the client IP in packets and so target security groups have to be set up to allow traffic accordingly.

With IP targets the packets from the NLB always have the NLBs ip in the return address. Awkwardly, this means you can’t really use security groups to control access to an NLB targeting IPs, lol.

crazypenguin
Mar 9, 2005
nothing witty here, move along
It's probably pointless to try to restrict a CD role much, unless I'm mistaken. Don't forget to threat model.

Your attacker gets access to your CD role. What are their goals? What's the worst-case? Are they thwarted because the role diligently doesn't allow creating an AppStream fleet? Or do they have the keys to the whole account anyway, because they can just iam:CreateRole whatever they please and use that?

I'm not sure there's any way to make a CD role less valuable (wait, no: SCPs), so you have to make it really well protected instead.

crazypenguin
Mar 9, 2005
nothing witty here, move along

Pile Of Garbage posted:

Amazon Linux 2 question: can anyone explain to me or point me in the direction of a page that clearly outlines the differences between the standard 4.14 kernel and the "next generation" 5.4 kernel (Installed via sudo amazon-linux-extras kernel-ng)?

Newer kernel versions have some new features (io_uring comes immediately to mind because I'm wild excited about it's potential... someday), and often have slight performance improvements. (Sometimes large ones in niche areas.) That's all.

Note: the -ng kernel has obviously already gone from 4.19 to 5.4 once. I suspect we'll see it jump to 5.10 this summer.

So on the one hand, maybe slightly better performance (or niche features if you know of them), and on the other hand, you might see major version jumps in the future. If you got a good system validating patches before deploying to production servers, then maybe it's worthwhile to go -ng. If not, you probably want to be more conservative with the kernel upgrades.

crazypenguin
Mar 9, 2005
nothing witty here, move along
Yeah. It’s in amazon-linux-extras

I forget what version exactly but definitely a newer 5.x

crazypenguin
Mar 9, 2005
nothing witty here, move along
If you’re having the instance assign itself an ENI on startup you could instead just have it assign itself an elastic IP, no? Never tried this, so dunno about gotchas, but seems like it’d work. The closest I did was have instances update their DNS on startup

(I assume this is one of those “ASGs of size exactly 1” situations right?)

crazypenguin
Mar 9, 2005
nothing witty here, move along
Maybe an S3 Access Point? I haven’t tried it so there could be quirks, but looks feasible.

Edit: Maybe don’t even need the access point. I had an S3 bucket with just a resource policy allowing based on “aws:sourceVpc”. Might work if the vpcs have S3 gateway endpoints, and you just need to allow by vpc and not some general IP thing.

crazypenguin fucked around with this message at 23:59 on Mar 8, 2022

crazypenguin
Mar 9, 2005
nothing witty here, move along
Does Access Analyzer tell you about it?

crazypenguin
Mar 9, 2005
nothing witty here, move along

Falcon2001 posted:

Is this setup going to be able to deliver anything near acceptable performance, or is the startup/ephemeral nature of lambdas going to be a problem?

It will likely work fine. I’ve built plenty of internal apps on lambda.

You havent defined what acceptable performance means, but a python cold start should be short. Well under a second. (idk about Flask, but the answer is just: try it)

Once a lambda instance is warmed, it sticks around for awhile and serves quickly

crazypenguin
Mar 9, 2005
nothing witty here, move along
VPC peering doesn't have a cost anymore I think https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/

The only use case I've had for multiple VPCs in the same account is associating a Route 53 Private Zone for `.` as a tool for allowing internal VPC DNS to work, but suppress public DNS resolution in an "air gapped" VPC. Neat trick, but the association is VPC-wide, so to have some things WITH public DNS in there, you have to peer two VPCs together.

crazypenguin
Mar 9, 2005
nothing witty here, move along
Is this a bit? If not, have you heard about graviton?

crazypenguin
Mar 9, 2005
nothing witty here, move along
Fast possibly useless thought, but the error message "Adding cross-account target is not permitted." does not strike me as a IAM policy problem, but a limitation of what the service will accept.

Adbot
ADBOT LOVES YOU

crazypenguin
Mar 9, 2005
nothing witty here, move along
It's been too long since I touched SQS, and those look good, but you missed at least one important one. *After* an incident explodes the queue, you want to be able to easily say "how long until it's back to normal?"

It looks like you have size of queue and incoming (I think?) rate, but not the outgoing rate. That's going to be important to compute "size / (out - in)" to answer "queue back to normal in X hours".

(And as a very generalized bit of advice for anyone, don't just think about alarmable metrics for dashboards. Informing operators during incidents, or providing data to make choices for runbooks, are also important considerations. And for any SQS queue an important runbook entry to have is "the queue is too big, it won't drain in an acceptable timeframe (e.g. days/weeks!! months?!), and scaling up the fleet consuming it just ran into other scaling bottlenecks, so now what do we do?!" SQS is a deliberately unbounded queue, so this is something that definitely needs answers thought through before you're in the middle of a stressful incident.)

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply