|
Comb Your Beard posted:First time poster ITT, this is both an aws question and noSQL question, let me know if not appropriate. You could have the sk be a string of code:You could have each item have an attribute witch is a list of all things that have occurred during that second. Both of these could also work with 30 second buckets, etc. A lot will depend on your query patterns.
|
#
?
Jan 28, 2023 06:06
|
|
|
|
| # ? May 15, 2024 21:19 |
|
|
can someone recommend a decent tutorial project for terraform and/or ansible that has a bit more complexity than "here's an ec2 that prints hello world"
|
|
#
?
Jan 28, 2023 06:16
|
|
|
kalel posted:can someone recommend a decent tutorial project for terraform and/or ansible that has a bit more complexity than "here's an ec2 that prints hello world" setting up a pihole in the cloud and make it HA with shared configs and blocklists using ASGs, spot instances, and EFS is usually my go to for the people I mentor. Add an ALB or NLB, monitoring with Cloudwatch, alerting and logging, and so on. Make it run in a container and use certbot and HTTPS for your internal domain. Restrict access to only your public IP of course so AWS doesn’t yell at you for running an open resolver, or set up a openVPN along with it with profiles for iPhone/android, computers, and so on. Configure DNS over HTTPS. Deploy your terraform and ansible with CI/CD using GitHub actions. Lose the load balancer and you can do this all in the free tier.
|
|
#
?
Jan 28, 2023 07:29
|
|
Cat Army 
|
kalel posted:can someone recommend a decent tutorial project for terraform and/or ansible that has a bit more complexity than "here's an ec2 that prints hello world" Terraform up and running by oreilly has a lot of more involved examples later in the book. It's what I used to learn for my company. Unrelated rant but does anyone at their company have issues getting people to use the loving terraform linter? I put in a PR the other day and this guy from another org absolutely ripped apart the formatting and it was all the linter. This is exactly what linters are for. To prevent this sort of lovely pedantic bickering. I was so pissed. jiffypop45 fucked around with this message at 19:30 on Jan 28, 2023 |
|
#
?
Jan 28, 2023 19:21
|
|
|
Yes, we run tflint as part of pre-commit-hooks for the repo. We also run pre-commit in CI on all our repos to ensure that someone doesn’t miss running it. Branch protection in GitHub means that you need a PR to merge and checks must be green to merge. Edit: we also use Atlantis to plan/apply so you don’t have to handhold people into their AWS setup
|
|
#
?
Jan 28, 2023 19:44
|
|
|
luminalflux posted:Yes, we run tflint as part of pre-commit-hooks for the repo. We also run pre-commit in CI on all our repos to ensure that someone doesn’t miss running it. Branch protection in GitHub means that you need a PR to merge and checks must be green to merge. I should look into that. You will comply. Or else. 
|
|
#
?
Jan 28, 2023 19:49
|
|
|
The Iron Rose posted:setting up a pihole in the cloud and make it HA with shared configs and blocklists using ASGs, spot instances, and EFS is usually my go to for the people I mentor. Add an ALB or NLB, monitoring with Cloudwatch, alerting and logging, and so on. Make it run in a container and use certbot and HTTPS for your internal domain. Restrict access to only your public IP of course so AWS doesn’t yell at you for running an open resolver, or set up a openVPN along with it with profiles for iPhone/android, computers, and so on. Configure DNS over HTTPS. Deploy your terraform and ansible with CI/CD using GitHub actions. This is a great idea! Going to work on that in an upcoming weekend. I still run pihole on an old raspberry pi b that I worry is not long for this earth.
|
|
#
?
Jan 28, 2023 20:06
|
|
|
luminalflux posted:Yes, we run tflint as part of pre-commit-hooks for the repo. We also run pre-commit in CI on all our repos to ensure that someone doesn’t miss running it. Branch protection in GitHub means that you need a PR to merge and checks must be green to merge. Yeah we actually do all of this stuff too. https://pre-commit.com makes sure various linters run before poo poo even gets committed, let alone to the code review stage. Pedantic assholes arguing about the number of spaces or whatever is such an obnoxious waste of time, just make a tool enforce it. And I enjoy Atlantis too. Docjowles fucked around with this message at 21:03 on Jan 28, 2023 |
|
#
?
Jan 28, 2023 21:00
|
|
|
Docjowles posted:Yeah we actually do all of this stuff too. https://pre-commit.com makes sure various linters run before poo poo even gets committed, let alone to the code review stage. Pedantic assholes arguing about the number of spaces or whatever is such an obnoxious waste of time, just make a tool enforce it. Yep, that’s the framework we use. You need to run it in CI as well, since people sometimes “forget” to install it in repos they don’t normally work with. “Linters are failing. Please ensure you’ve installed pre-commit hooks on this repo with pre-commit install” is something I use as a “Request changes” review way too often.
|
|
#
?
Jan 28, 2023 21:19
|
|
|
The Iron Rose posted:setting up a pihole in the cloud and make it HA with shared configs and blocklists using ASGs, spot instances, and EFS is usually my go to for the people I mentor. Add an ALB or NLB, monitoring with Cloudwatch, alerting and logging, and so on. Make it run in a container and use certbot and HTTPS for your internal domain. Restrict access to only your public IP of course so AWS doesn’t yell at you for running an open resolver, or set up a openVPN along with it with profiles for iPhone/android, computers, and so on. Configure DNS over HTTPS. Deploy your terraform and ansible with CI/CD using GitHub actions. sounds cool. is there a tutorial walkthrough you would recommend because I'm not smart/experienced enough to figure that all out on my own
|
|
#
?
Jan 28, 2023 22:26
|
|
|
Docjowles posted:Yeah we actually do all of this stuff too. https://pre-commit.com makes sure various linters run before poo poo even gets committed, let alone to the code review stage. Pedantic assholes arguing about the number of spaces or whatever is such an obnoxious waste of time, just make a tool enforce it. Nthing this; pick a linter and force compliance in CI for basically any language; if folks want to argue about the right formats to use, agree as a team and setup your CI to use it (like PEP8 for Python says 80 char lines, but line lengths of 120 is perfectly reasonable. The rest of it should really be consistent with a given style guide used by whatever linter you're using)
|
|
#
?
Jan 29, 2023 05:54
|
|
|
Question about AWS api service. We're trying to integrate a fairly heavy pair of dockerized apis (One a fastapi with the ability to offload some fairly gnarly spatial transformations [multi-gig datasets] to a backend worker process) and a geoserver instance (giant ancient crusty java erver that wants a few gigs of ram and probably terrabytes of storage xto do its thing, oh and its start up time is a good 7-10 mins), with our lambda jumble. Obviously neither are good candidates for lambda-izing, which is fine, we've come to accept that short of rewriting the whole thing (including the very standard 3rd party geoserver, something beyond our scope) we'll have a bit of a hybrid setup. So far so good. One thing we want to do however is put the fastapi behind the AWS api front-end so it forms an effective singular api with the lambda stuff. The problem seems to be that attempting to load in the OpenAPI.json generated by FastAPI (which is a great little framework that heavily utilizes the static type annotations of the recent python 3.x's to build schemas) it wants to overwrite the whole setup. Thats obviously going to be a problem. So I guess the question is, is there another way to do this? Can I just tell the api front end service to just redirect everything under a particular path to the service and yolo it, or perhaps have a secondary api front end feed into a primary one? Whats the best practice here?
|
|
#
?
Jan 30, 2023 03:07
|
|
|
BaseballPCHiker posted:Speaking of! I got tasked with tracking down Marketplace subscriptions, whose subscribing, why are they doing that, etc. And for the life of me I dont see any marketplace API calls in our CloudTrail. Looking at the reference I think I need to be searching for aws-marketplace:Subscribe but I cant find anything marketplace related. I verified our org CloudTrail doesnt have any exclusions. Confirmed! Buying poo poo from marketplace is not captured by CloudTrail. You can kind of find it digging around in cost explorer.
|
|
#
?
Feb 1, 2023 21:53
|
|
|
I just want to make sure I'm not missing something obvious in GCP GPU pricing: it looks like the best value is always latest gen GPUs, which tells me that the old ones are only kept around for customers who don't want to bother with the effort of moving onto newer instances: https://cloud.google.com/compute/gpus-pricing. Is that a sane read of the situation? I know that's broadly how it works with CPU based instance type, but per-core CPU performance has been advancing so slowly it's less noticeable there, while pricing a Pascal GPU higher than a Turing one is just weird as hell.
|
|
#
?
Feb 6, 2023 23:31
|
|
|
Elastic Beanstalk instance security group question: Trying to spin up EB in two different regions but want it to use a custom security group. I know I can add a securitygroup.config file to the .ebextensions directory of the application to tell it which security group to use like this: option_settings: - namespace: aws:autoscaling:launchconfiguration option_name: SecurityGroups value: sgEBSecurityGroup The issue I have is that I'm deploying into VPCs so I need to specify the security group ID rather than the security group name. The security group ID is obviously different in each region so it would require me to have two different versions of the application with a different securitygroup.config within each. Is there a way to reference the region and select the corresponding security group ID that exists within that region? Scrapez fucked around with this message at 21:16 on Feb 17, 2023 |
|
#
?
Feb 17, 2023 20:05
|
|
|
Anyone have that link for the alternate AWS status/health checks? It was called something like Honest or True AWS health report? Ringing any bells for anyone. Cant for the life of me remember what it was called, but it was always way more accurate than the official AWS health status page.
|
|
#
?
Feb 21, 2023 15:34
|
|
|
Thinking of lying.cloud? That was a quinnypig thing but he took it down a while ago.
|
|
#
?
Feb 21, 2023 16:19
|
|
|
That sounds right. Bummer, that worked pretty well. Any good alternatives?
|
|
#
?
Feb 21, 2023 18:10
|
|
|
Wonder if he was just scraping Twitter. I've seen other companies do that for status pages with accurate results.
|
|
#
?
Feb 22, 2023 00:45
|
|
|
A question about IaC and version control: Say I have a single-pay webapp that just says 'Hello' when you navigate to it. I want to host it on EC2 + nginx. I want to deploy it with Terraform. I want to use Github for version control and later on I might add pictures to the app. Do I put my Terraform files inside of my Webapp github repo? Or is it good practice to host terraform files separate from the webapp it is building infrastructure for?
|
|
#
?
Mar 1, 2023 00:43
|
|
|
put them in separate folders from the root ie /infra /webapp
|
|
#
?
Mar 1, 2023 00:49
|
|
|
The Fool posted:put them in separate folders from the root Ok that makes sense, thanks!
|
|
#
?
Mar 1, 2023 01:14
|
|
|
putting them in separate repos isnt a terrible idea either, but imo for smaller projects it just makes more sense to keep everything together
|
|
#
?
Mar 1, 2023 01:17
|
|
|
keeping them adjacent in the same repo is usually better in my experience, although i usually name the folder terraform/ instead of infra/
|
|
#
?
Mar 1, 2023 01:21
|
|
|
I see it in larger projects that split microservices up into separate repos but they still share infrastructure so the iac config is in its own repo
|
|
#
?
Mar 1, 2023 02:01
|
|
|
Going against the grain to say I prefer my IaC code to be in its own repo. Especially if there are more people than you touching it/the app. It's related to the app but it's also got nothing to do with the app, if you know what I mean. But that's just like, my opinion, man.
|
|
#
?
Mar 1, 2023 16:46
|
|
|
I find that folks also get tied up in git semantics around PRs and workflows a lot and that can mean some really awkward workflows if they share a repo.
|
|
#
?
Mar 1, 2023 16:49
|
|
|
One iac repo per microservice. Anything more will drive you insane and doesn't scale. We currently have this guest engineering team at my job that stuck like 3 microservice worth of iac in a single repo and when I submitted a 1500 line cr for it absolutely screamed at me. So we shelved my cr and are just going to wait until they go away and then merge it. (Rewriting peoples tf code to be idiomatic doesn't make friends but they could have read the book the same as me but elected not to so they don't have anything to stand on, as noted previously I'm sticking that tf linter on our hooks as soon as I get back from leave)
|
|
#
?
Mar 2, 2023 02:27
|
|
|
Pulumi seems pretty cool and seems to have some neat usability benefits over tf like interactive debugging, I think I'll give it a shot w/ a new project sometime soon. https://www.pulumi.com
|
|
#
?
Mar 2, 2023 02:53
|
|
|
FormatAmerica posted:Pulumi seems pretty cool and seems to have some neat usability benefits over tf like interactive debugging, I think I'll give it a shot w/ a new project sometime soon. I’ve heard folks saying it doesn’t live up to hype lately.
|
|
#
?
Mar 2, 2023 02:55
|
|
|
Terraform cloud does a lot of the same stuff but market share is still a bit limited.
|
|
#
?
Mar 2, 2023 04:00
|
|
|
CDK for life.
|
|
#
?
Mar 2, 2023 04:27
|
|
|
freeasinbeer posted:I find that folks also get tied up in git semantics around PRs and workflows a lot and that can mean some really awkward workflows if they share a repo.
|
|
#
?
Mar 2, 2023 19:01
|
|
|
Does anyone who works at Amazon ITT work on the ADC team?
|
|
#
?
Mar 8, 2023 23:30
|
|
|
Woof Blitzer posted:Does anyone who works at Amazon ITT work on the ADC team? I did for awhile what's up?
|
|
#
?
Mar 9, 2023 00:18
|
|
|
jiffypop45 posted:I did for awhile what's up? Let's take this... to PMs!
|
|
#
?
Mar 9, 2023 01:18
|
|
|
What's the best way to get AWS credits for a startup? I've tried this before when starting my consulting company, but don't know what magic incantation will result in getting anything at all, let alone the maximum possible. Appreciate any sage advice from Amazon folk or other knowledgeable types!
|
|
#
?
Mar 22, 2023 18:26
|
|
|
Disclaimer: my experience in this area is limited and I might be big-time wrong. Edit: see below, much better info. Pile Of Garbage fucked around with this message at 19:28 on Mar 22, 2023 |
|
#
?
Mar 22, 2023 18:52
|
|
|
You won't get anywhere through support/TAM unless you have an Enterprise Support contract and by then you probably have a named account/tech resource assigned to you in addition to the TAM. The best a support ticket is going to do is get you connected to your account manager. There are half a billion different programs that help startups with funding, the best possible paths are: 1 - Reach out to your account manager and ask about programs for startups. AWS Activate is the big one - but there are many more buckets an account manager can pull from depending on what your spend is and what it could be based on your architecture and potential customer growth. 2 - Look at the Amazon Partner Network if your startup has any co-sell/better together story (are your customers also likely to be AWS customers and therefore co-marketable?). Even if not, joining is free and will get you identified by a "Partner Development Rep" which is kind of another account manager contact but for moving through partner programs - some of which are focused on Startups. There could be more credits here, joint marketing dollars, etc. Feel free to hit me up via PM if you don't have a support contact to ask who your account manager is. If you have any spend at all you are assigned to someone's territory and I can help connect the dots for you.
|
|
#
?
Mar 22, 2023 19:20
|
|
|
|
| # ? May 15, 2024 21:19 |
|
|
Thanks, that's really helpful! Right now we're pre-launch, and serverless dev environments don't exactly rack up the spend so we won't have an account manager any time soon. Very interesting re: APN as we're bootstrapping via consulting, so this might be applicable. I was kind of under the impression that these were "try out/launch your startup" credits, I didn't realize this was meant for organizations who already had significant operations.
|
|
#
?
Mar 22, 2023 23:39
|
|