|
If you just want to learn IAC in general and not terraform specifically, I’ve found AWS CDK to be a lot easier. Choose your language, get some IDE integration going and you can slam some poo poo out pretty fast as you learn the conventions.
|
#
?
Sep 2, 2021 12:41
|
|
|
|
| # ? May 15, 2024 20:24 |
|
|
Granite Octopus posted:If you just want to learn IAC in general and not terraform specifically, I’ve found AWS CDK to be a lot easier. Choose your language, get some IDE integration going and you can slam some poo poo out pretty fast as you learn the conventions. Thanks, I added that to my list of things to research. I'm in that stage where I don't know what I don't know. My immediate goal is to learn some skills and build a few projects that can convince a hiring manager that I'm at least worth a phone call for a junior/associate position. I landed on learning Terraform because that seems to be a hot commodity from browsing SA and LinkedIn etc...
|
|
#
?
Sep 2, 2021 13:48
|
|
|
terraform is the most popular tool in this space, yeah. i prefer pulumi to the aws cdk but they are both fine e: if you decide you want to get a job terraform touching i would look up the following addon tools asap to make your eventual job suck less: - terraform-landscape - terragrunt 12 rats tied together fucked around with this message at 01:20 on Sep 3, 2021 |
|
#
?
Sep 3, 2021 01:12
|
|
|
Hughmoris posted:Side note: seems like prime opportunity for a good tutorial here. Amazon Training walks you through all this via the console UI but it would be great to then have the same steps via Terraform. If I can complete this then I might try a Terraform write-up. Because Terraform is a competing product. CDK is what we want customers to use. Should be able to find tutorials for it.
|
|
#
?
Sep 3, 2021 06:45
|
|
|
12 rats tied together posted:terraform is the most popular tool in this space, yeah. i prefer pulumi to the aws cdk but they are both fine Landscape is pointless if you're using TF >= 0.12 (which you should should be unless you inherited an older codebase in which case you should be upgrading asap)
|
|
#
?
Sep 3, 2021 13:45
|
|
|
The full gamut of stuff people have been using for IAC is beyond just Terraform, CDK, and Pulumi. The major players from the configuration management era have all had attempts:
Terragrunt is something people reach for when they'd like some more formality around the hodge-podge of modules strung together across a ton of places and Makefiles aren't cutting it for you. There's some repository layout patterns I'd recommend before it, but compared to the 500+ line Makefiles we had everywhere and the code duplication issues, Terragrunt fit our needs swell. Much of why it exists is to fill in gaps prior to Terraform 0.12 existing, but I like to use Terragrunt as a driver for Terraform commands and to have Makefiles look more uniform in our monorepo.
|
|
#
?
Sep 3, 2021 16:41
|
|
|
Rookie question... My Goal: to automate the launch of an EC2 instance and host a simple static site using nginx I've walked through the console and launched an EC2 instance using Amazon Linux 2 AMI. I then manually installed nginx and configured it so I can reach it from a public computer. Success. Next, I've used Terraform to launch an EC2 instance but I'll still need to manually install/configure nginx. What direction do I want to look if I want to automate the deployment of the EC2, and then automatically install and start nginx? Am I on the right track with researching EC2 User Data, or is best practice something else?
|
|
#
?
Sep 13, 2021 05:03
|
|
|
I don't think there exists a "best practice" in this space, yet. EC2 lets you run cloud-init through userdata which is one of the most portable and effective tools in this space, I highly recommend getting started with it. In larger environments it has some ergonomics issues that make it difficult to scale past a certain size, geographical, or complexity threshold, but it's very very good and even in those larger environments is often still used. The "terraform only" answer for that is provisioners, they're pretty bad though (more info in the link). Another solution in this space is to build an AMI with your nginx stuff already configured, and then just launch that AMI. There are a bunch of ways to build AMIs but packer is a great place to start.
|
|
#
?
Sep 13, 2021 05:40
|
|
|
Generally speaking, it is common to half-bake an AMI with the basic packages and applications preinstalled. Then one can use User Data to pull config files/scripts/whatnot from a location and run them. Preinstalling necessary applications speeds up launch times when all you have to do is load a config file and restart.
|
|
#
?
Sep 13, 2021 06:25
|
|
|
12 rats tied together posted:I don't think there exists a "best practice" in this space, yet. EC2 lets you run cloud-init through userdata which is one of the most portable and effective tools in this space, I highly recommend getting started with it. In larger environments it has some ergonomics issues that make it difficult to scale past a certain size, geographical, or complexity threshold, but it's very very good and even in those larger environments is often still used. Agrikk posted:Generally speaking, it is common to half-bake an AMI with the basic packages and applications preinstalled. Then one can use User Data to pull config files/scripts/whatnot from a location and run them. Exactly what I was looking for. Thanks! I pivoted from nginx to a LAMP stack since AWS has some tutorials on it. I've been able to create an EC2 instance, get LAMP up and running, then created a custom AMI from a snapshot and spun that up. I was immediately able to access static content on the new instance over http. I know this is all the basics but its some pretty cool stuff.
|
|
#
?
Sep 13, 2021 15:37
|
|
|
When you get down to it, cloud poo poo is pretty amazing. I mean think about it: once you’ve set it up properly you can build a site capable of huge horizontal elasticity and launch it in ninety seconds. Compare this to the olden days when horizontal scaling required physically racking and cabling a box, and then lovingly crafting your new web server, “Hobbes”, with fresh bios, OS install, drivers, and patches. You would monitor its health across a dozen performance metrics and care for it for several years, getting paged when a dashboard turns read somewhere, despite its effective load hovering right around zero.
|
|
#
?
Sep 13, 2021 15:57
|
|
|
Agrikk posted:When you get down to it, cloud poo poo is pretty amazing. Its incredible. The scale and complexity of things you can design, all remotely without touching any hardware, is mind boggling. The pace of change and complexity is equally amazing in my mind. I'm struggling right now to learn as much as I can about AWS core services for my job in InfoSec and I have a hard time keeping up.
|
|
#
?
Sep 13, 2021 16:20
|
|
|
Are there jobs with AWS that just revolve around a single service? Like I eat and breath GuardDuty at my current position. I'm the go to person on our team with questions about findings, logs being read by GuardDuty, etc. I think I'd really enjoy doing work around that service most of the day. Helping orgs spin it up, or troubleshoot for example. Or would I be better off just working for some MSP doing that?
|
|
#
?
Sep 14, 2021 17:37
|
|
|
Absolutely. AWS has technical program managers and project managers and subject matter experts dedicated to a single service. If a customer needs a deep dive on any specific service the first thing I do is reach out to the TPM or PM of that service for engagement.
|
|
#
?
Sep 14, 2021 18:42
|
|
|
Agrikk (and anyone else): I've been using an old post of yours to roughly guide my AWS learning. You had recently mentioned that you would approach it now days with containers and container registry. At a high level, would you be willing to point out where/why you would use containers on this path? I understand the gist of containers and have done simple Hello World projects with Docker but I'm not sure how to interweave it in to this pipeline and deliver a better solution. quote:Because at the free tier to build a web site on a t3micro in a VPC with public/private subnets and an ELB acting as a NAT gateway and firewall. The web site can be something simple, like displaying the host name and local time.
|
|
#
?
Sep 14, 2021 18:45
|
|
|
Anywhere you have EC2 running, really. Build an Apache container (or containers- for HA) fir your web site. Build a container to host the app that parses the Apache web logs and inserts them into your database. Stuff like that. Raise your game by converting anywhere there is compute with Lambda or ECS.
|
|
#
?
Sep 14, 2021 23:22
|
|
|
Agrikk posted:Anywhere you have EC2 running, really. Ahhhh. Ok, I was looking at it from the completely wrong angle. I was thinking that I would build out my own custom EC2, deploy that, then kick off a container inside that...  Just watched an Amazon ECS video and now better understand what you're hinting at. Thanks!
|
|
#
?
Sep 15, 2021 00:57
|
|
|
BaseballPCHiker posted:Are there jobs with AWS that just revolve around a single service? I know Agrikk answered your question but it got me thinking: In trying to land my first gig, am I better off going a mile deep on one or two particular services? Or should my knowledge be a foot deep and a mile wide? What would be useful as a newcomer to the team? I passed my AWS Cloud Practioner cert yesterday, and have been flipping through this 450 pg Solution Architect book. Maybe I can become the ECS guy, or the CloudWatch guru?
|
|
#
?
Sep 16, 2021 05:21
|
|
|
Depends on the job, and the org, but if you had to pick only one service to fully understand it should be IAM (because it's the only hard part about all the other services).
|
|
#
?
Sep 16, 2021 05:26
|
|
|
I hate IAM. I mean, I really hate working with IAM. This means that 12 rats is correct in that you should learn IAM so there's one more person to whom I can redirect IAM questions. Also, I'd make sure you can speak to the foundational stuff (EC2, S3, VPC, xLB, RDS/DynamoDB) before heading off into specialty land. Getting too deep into one topic without the foundational stuff is like being really, really good at sparkplugs. Or the timing belt. :caranalogy:
|
|
#
?
Sep 16, 2021 05:35
|
|
|
I mainly deal with Connect (the contact centre service) but that then involves a lot of lambdas and dynamoDB, and s3, oh and cloudwatch and and I just had to figure out how to get a .net core MVC app to work with ElasticBeanstalk and Cognito. Which then also involves buggering around with Route 53, oh and cloudtrail and SNS. And whatever else. Plus API gateway, then I get asked to stand up EC2 instances and now apparently use Transcribe and Comprehend. And lucky me I get to be the IAM guy as well. So good luck being an expert in one thing. And a couple of weeks back I did a course so I can get the Associate SysOp cert (having already got Associate Developer), and by then end of three days I was just about ready to shoot myself.
|
|
#
?
Sep 16, 2021 07:12
|
|
|
12 rats tied together posted:Depends on the job, and the org, but if you had to pick only one service to fully understand it should be IAM (because it's the only hard part about all the other services). Yeah the other large part of my job is sorting through IAM policies. Just refer to AWS's handy & simple flow chart!  I would agree though, you should have a foundational level of knowledge in things like S3, IAM, EC2, VPCs, etc. If there is a service your company works with a lot in particular try to learn at least the fundamentals, in my case that would something like Glue or Lake Formation which I hate, but eh thems the breaks. Also as I get more and more involved I've accepted that I will just have to learn coding somewhat and have started learning python. Im not good at it, I dont particularly love it, but I am at the point where I can look at code now and understand what is trying to be accomplished. I'm pretty much a "copy and paste" coder right now which seems to be good enough.
|
|
#
?
Sep 16, 2021 13:56
|
|
|
BaseballPCHiker posted:I'm pretty much a "copy and paste" coder right now which seems to be good enough. Think that applies to basically everyone and even people with a dev/eng background will turn to Google/Stack overflow etc  I mean no point reinventing the wheel
|
|
#
?
Sep 16, 2021 14:13
|
|
|
I'm the IAM person on my team and even I want to shoot myself or at the least have a drink before I read through any moderately complex set of IAM policy documents. What bothers me most are some of the really odd or misleading error messages that can happen and send you into an alternate dimension when it's just being denied access to a key. For example, with a KMS key policy that was denying access to AWS services I was getting a random, generic failure message that conflates with 100+ other possible root failures while in most IAM failure modes you get pretty straightforward permission errors. Similar happens with S3 object and bucket policies.
|
|
#
?
Sep 16, 2021 14:50
|
|
|
necrobobsledder posted:Similar happens with S3 object and bucket policies. Ugh, these are the worst and most painful for me to help with. I want to pull my hair out every single time.
|
|
#
?
Sep 16, 2021 15:06
|
|
|
i actually love it, sorry, and my suggestion was only half joking. you can google "iam actions conditions" + service name and the first result will be one of the best references in the docs for understanding a new service in my experience. AWS IAM is also best in class of the various cloud providers, second only to probably alibaba cloud which is the exact same service but it's called RAM instead. GCP's IAM is horrible by comparison, Azure is also extremely bad unless you have an existing AD domain plus a team of active directory janitors to deal with it.
|
|
#
?
Sep 16, 2021 16:26
|
|
|
This one time a customer asked me to look over his cloudformation template for standing up a new VPC in their infrastructure. He sent me a fifteen page pdf and said “what do you think?” “It look great!” I said having not looked at it at all. Deciphering someone else’s code? Yeah nah.
|
|
#
?
Sep 16, 2021 18:23
|
|
|
necrobobsledder posted:really odd or misleading error messages that can happen and send you into an alternate dimension [...] Similar happens with S3 object and bucket policies. Hear hear! My first bit of advice to my team on diagnosing weird AWS errors that aren't obvious are, do you have permissions to access what you want? Yes. How do you know? Did you test it? Yes. Escalate to support. 1 in 5 issues gets escalated.
|
|
#
?
Sep 16, 2021 21:20
|
|
|
CarForumPoster posted:Hear hear! My first bit of advice to my team on diagnosing weird AWS errors that aren't obvious are, do you have permissions to access what you want? Yes. How do you know? Did you test it? Yes. Escalate to support. Even obvious errors like this one from s3 "Insufficient permissions You need s3:CopyObject permission to perform this action" but s3:CopyObject does not exist as a permission. Don't know if this dialog has been fixed yet.
|
|
#
?
Sep 17, 2021 03:19
|
|
|
ledge posted:Even obvious errors like this one from s3 "Insufficient permissions You need s3:CopyObject permission to perform this action" but s3:CopyObject does not exist as a permission. Don't know if this dialog has been fixed yet. i suspect this is an issue with the console more than anything else. iam policies are fundamentally just json text, so there is nothing stopping you from add "s3:CopyObject" to your policy.
|
|
#
?
Sep 17, 2021 03:59
|
|
|
yea the problem with that one is that CopyObject isn't a real permission. you need to have PutObject and GetObject on the destination/source respectively the api docs kind of mention it obliquely: https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html
|
|
#
?
Sep 17, 2021 04:18
|
|
|
i guess, as always with s3, the reality is way more confusing than it should be
|
|
#
?
Sep 17, 2021 04:52
|
|
|
an extra fun part of CopyObject is that you can end up with objects in your account that people with AdministratorAccess, also in your account, don't have access to it happens if you allow a user in a different account to PutObject, and they do a CopyObject, which as noted in the API docs does not preserve the Object ACL but replaces it with "private". "private" is a Canned ACL which basically grants the source account owner full control over the object, which is normally fine, except because the user belongs to a different account, the source account owner is that different account, not the bucket owner account you can still HeadObject, pretty sure you can DeleteObject, but you can definitely be a full rear end admin and get an access denied trying to GetObject a thing in your own account. it rules
|
|
#
?
Sep 17, 2021 05:21
|
|
|
Working my way through this SAA book to try and build a solid foundation of knowledge. EC2s, Databases, Security all kind of make natural sense from working with computers. Networking is a whole different world. Setting up home WIFI is the extent of my current networking knowledge. So all of this VPC, VPNs, NATs, Routing etc... is a lot of new crap to straight up memorize. I know this is a broad question, and excluding the network engineers, how would you all rate your AWS Networking knowledge? Are you using it daily in your engineering/DevOps work, or is it something you typically don't dig in to much poo poo is broke?
|
|
#
?
Sep 18, 2021 21:39
|
|
|
Hughmoris posted:I know this is a broad question, and excluding the network engineers, how would you all rate your AWS Networking knowledge? Are you using it daily in your engineering/DevOps work, or is it something you typically don't dig in to much poo poo is broke? I did most of my networking work when I was doing migrations into AWS. Now I rarely touch any of it because once you get things setup they just don’t change that often. Every once in a while we need to add some routes or something but those changes are few and far between.
|
|
#
?
Sep 18, 2021 21:46
|
|
|
One of the issues I had was that coming from a background that had more traditional networking knowledge of Cisco ASAs for example a lot of things were weird. For example, firewall rules oftentimes have preferences in order that impact performance, but not in AWS! I had an argument with a manager before that while normally quite intelligent didn't understand the subtleties between traditional constructs in networks vs how it's done in AWS and demanded that I order NACL rules a certain way due to how routing rules usually resolve historically. I told him that even if it mattered AWS APIs basically re-arranged them anyway. Another fun part is when security groups reference each other, and lots of people intuitively think that resolution is fully transitive and resolves IP groups as a union when it's really just pointers to another group. But really, EC2 VPC networking isn't too awkward to understand as long as you test some of your own assumptions from time to time before committing to anything as architecture. Things get weird when we talk about VPC peering sometimes (like resolving DNS entries between regions) and on occasion some constructs that used to be impossible become no big deal later on with new features. For the longest time VPCs with overlapping IPs were a big no-no for many architecture but with Transit Gateway it's not the same death sentence anymore. Heck, I remember when we used to be unable to launch EC2 instances with a new KMS key defined, and I was having to deal with a bunch more encrypted AMIs for our workflow, then when AWS announced the feature my project was done with literally a single line of code. It definitely can be a problem going from traditional networking constructs into AWS, Azure, GCP, etc. and I've seen a theme with a ton of graybeards making some Bad Decisions when it comes to cloud environments despite supposedly being the SMEs on networking in theory. For example, traditionally we're used to trying to define subnets as small as possible for application spaces but in AWS we generally want to use larger subnets for the sake of growth and because we don't have VLANs use VPCs instead.
|
|
#
?
Sep 19, 2021 18:15
|
|
|
necrobobsledder posted:Another fun part is when security groups reference each other, and lots of people intuitively think that resolution is fully transitive and resolves IP groups as a union when it's really just pointers to another group. Yeah this is an interesting thing in AWS, whenever you start dealing with groups they start being references-to-references instead of explicit objects themselves like you would find in active directory etc. Of particular note an IAM group is not a valid value for use in a policy's principal field: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html This means, in a resource based policy, you can't be like "members of group ThingDoers are allowed to Do Thing on this resource", you have to be like "group ThingDoers are allowed to assume role ThingDoer, role ThingDoer is allowed to Do Thing". A principal based policy attached to a group is interesting in that the principal is implicitly "self", but that implicit-ness AFAICT is used as a pass-through, each user resolves the policy individually, where "self" becomes the IAM user "me". Basically, querying groups and resolving group membership is expensive computationally, and AWS' model dodges that problem by just not supporting it. e: My biggest recommendation for OP is to actually read the documentation. They worked very hard on it, and it's very good. 12 rats tied together fucked around with this message at 18:37 on Sep 19, 2021 |
|
#
?
Sep 19, 2021 18:25
|
|
|
12 rats tied together posted:Network stuff... The AWS documentation really is fantastic. As I'm starting from scratch and poking about, I don't think I've yet run into a situation where I thought the documentation should be better. A lot of that couple be the fact that I don't know jack but I've appreciated it so far.
|
|
#
?
Sep 20, 2021 04:56
|
|
|
Is there any viable reason to be using MS SQL on an EC2 instance rather than in an RDS instance? My suspicion is it's due to a lift and shift from on-prem to AWS and wanting to just copy everything over from the old on-prem DB setup. I know they were fussed about the price of MSSQL on RDS compared to on Azure but I'm not sure if a license for an on-prem version would transfer over to EC2 and save them the subscription cost. Then again, if they want HA, they would probably need two licenses for the two different EC2 instances and have to deal with cross-region replication of data between the two which sounds like a pain.
|
|
#
?
Sep 21, 2021 02:18
|
|
|
|
| # ? May 15, 2024 20:24 |
|
|
22 Eargesplitten posted:Is there any viable reason to be using MS SQL on an EC2 instance rather than in an RDS instance? My suspicion is it's due to a lift and shift from on-prem to AWS and wanting to just copy everything over from the old on-prem DB setup. I know they were fussed about the price of MSSQL on RDS compared to on Azure but I'm not sure if a license for an on-prem version would transfer over to EC2 and save them the subscription cost. RDS is very expensive. Usually a database needs the features RDS has to offer, so it can be worth it. I've had a few unusual use cases where a SQL database was needed but without any replication, failover, etc that we used EC2 for instead of RDS. Those were not typical use cases of a database, which usually needs the features RDS has to offer. You pay a premium for it though!
|
|
#
?
Sep 21, 2021 07:01
|
|
