Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Methanar
Sep 26, 2013

by the sex ghost
Idiot question:

What is the difference between https://bucket.s3-us-east-1.amazonaws.com and https://bucket.s3.amazonaws.com

https://github.com/jie123108/lua-resty-s3 This library for accessing s3 over lua has helpfully hardcoded in the string s3-us-east-1 into all requests. Aside from hardcoding a region into a library is stupid, what is the difference between s3 and s3-region in the subdomain? Why does my bucket only respond to .s3.?

➜ Scripts ping bucket.s3-us-east-1.amazonaws.com
ping: cannot resolve bunstributor.s3-us-east-1.amazonaws.com: Unknown host
➜ Scripts ping bucket.s3.amazonaws.com
PING s3-1-w.amazonaws.com (52.216.225.176): 56 data bytes
64 bytes from 52.216.225.176: icmp_seq=0 ttl=44 time=68.532 ms

Adbot
ADBOT LOVES YOU

Methanar
Sep 26, 2013

by the sex ghost

Volguus posted:

Ok, thanks for the explanations so far. So if I would be to put a job posting tomorrow for a person to help with the build and deployment infrastructure ... what title should I ask for? At the end of the day, I agree with you that the person needs to be a lot more involved in the development and planning aspect, but we're a 4 people startup , 2 of them being scientists that shouldn't be trusted with a computer, much less with C++, me, the developer, architect, tester, build and release manager and AWS expert (god help us) and a CEO who is ... a CEO.

At the end of the day AWS is so drat big that one needs to do that crap full time to even dream of getting anywhere much less to take advantage of it in the most efficient manner. And I would be very happy to not have to touch AWS. The only thing I care about it is to be up and how much does it cost.

What are your scientists doing that doesn't involve computers.

Methanar
Sep 26, 2013

by the sex ghost
Maybe you could use the ami launch index for that
curl -s http://169.254.169.254/latest/meta-data/ami-launch-index

Comedy answer: use metal instances and use virtualbox to run windows on each metal instance.

Methanar fucked around with this message at 01:05 on Feb 19, 2020

Methanar
Sep 26, 2013

by the sex ghost

Vanadium posted:

Eh, they're trying to cut down on handing out routable CIDR ranges. I'm pretty sure asking for a /16 would raise eyebrows, for an actual "gonna run a bunch of computey things" account I think they just gave me a /22 or /21 or something.

So far this account is mostly metrics and IAM roles and that sort of thing, and these couple lambdas that are periodically doing things to CloudWatch alarms, based on data from this service I'm wanting to call via privatelink. Right now it calls the legacy version of the thing via API Gateway but the shiny new version isn't going to have an API Gateway.

Having a tiny VPC just for these lambdas and nothing else seems least likely to cause conflicts down the line, and in the worst case I can spin up this stuff somewhere else and delete the VPC again. If they get a lot of scope creep the lambdas should probably move to a better platform than this pile of terraform, too, but I shouldn't invest a lot of time in that right now.

Having multiple VPCs is basically always a bad idea because of cross-vpc billing. Just make one mega VPC. and make it big enough Forever.

Methanar
Sep 26, 2013

by the sex ghost

crazypenguin posted:

VPC peering doesn't have a cost anymore I think https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/

The only use case I've had for multiple VPCs in the same account is associating a Route 53 Private Zone for `.` as a tool for allowing internal VPC DNS to work, but suppress public DNS resolution in an "air gapped" VPC. Neat trick, but the association is VPC-wide, so to have some things WITH public DNS in there, you have to peer two VPCs together.

huh. I somehow missed this change. Because we used to pay out of the nose from some questionable multiple VPC architecture.

Methanar
Sep 26, 2013

by the sex ghost

Docjowles posted:

Yeah AWS SSO is quite cool and good these days. Really my only gripe (and this applies to regular old IAM too) is that you still can’t be logged into more than one console at once. So if I want to look at things in more than one account simultaneously I have to open incognito windows or multiple browsers or some poo poo. Which is very awkward. I don’t know how they would reasonably make that work, though.

I die a little bit every time I'm logged into the prod aws account and then realize I need to do the whole login MFA dance again for the dev account in incognito.

Methanar
Sep 26, 2013

by the sex ghost
Graviton is good tho.

duck monster posted:

Told him, lets just stick with intel for now but setup an experiment to see how it works. The main issue is likely more that our CI might take some tweaking to build docker containers that wont explode if we ask it to build ARM

Watch out for the difference in integer processing vs float processing performance for graviton vs Intel, which may or may not be a thing that's relevant to your apps.

Do you even need to do anything special at all for building your java app for arm?
I guess if you have any c libraries, but I actually can't think of any off hand that would make sense for a java app.



If my company wasn't balls deep in investing in our own bare metal datacenter presence, I'd be spearheading my own initiative to introduce graviton as a cost savings initiative.

Methanar fucked around with this message at 07:24 on Dec 21, 2022

Methanar
Sep 26, 2013

by the sex ghost

duck monster posted:

Kafka and Elastic search are a high probability. We are currently using OCS's OSI which is a loving horrible timeseries db (decision made a long time ago before I started) that fails one the basic premise of timeseries databases of "usecond latency" for writes. From same bad prehistoric time we have a custom built ETL setup that is basically hot trash and we're speccing out a replacement based around a custom go engine [its too weird a workload for the off the shelf solutions] and kafka with queues in and out w something running all our weird maths pouring all the finished product into elastic

I have no idea what you're doing, but I'm fairly confident that's not how I'd build my tech stack.

usecond write latency is a big ask, but cassandra would be my completely uninformed recommendation as a replacement for your weirdo time series DB I've never heard of. Clickhouse might be worth looking into as well.


Even if you don't care, you should still watch how Cassandra works as an inspiration of how to build a good write-path.
https://www.youtube.com/watch?v=B_HTdrTgGNs

Methanar fucked around with this message at 07:49 on Dec 21, 2022

Adbot
ADBOT LOVES YOU

Methanar
Sep 26, 2013

by the sex ghost

jiffypop45 posted:

Multicloud is a joke and I'm sorry your upper management got pilled by someone selling something.

If you sell software licenses of basically anything to Google, they love to negotiate that as part of the deal they will pay you in GCP credits. I'm pretty sure Oracle does this as well. I can't imagine microsoft not also being on that train.

It's a great idea from their standpoint.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply