Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
pissinthewind
Nov 11, 2021

Are there jobs in this? Someone is like "You should get AWS certified and you can get a job!" and I went and looked at jobs in my area and there are none. Is the training source in the OP still good? If not what do people recommend?

Adbot
ADBOT LOVES YOU

Arzakon
Nov 24, 2002

"I hereby retire from Mafia"
Please turbo me if you catch me in a game.
What are you doing now, where are you at in your career, and how far out in the backwoods are you? I only have a few thousand options that are all remote for you.

pissinthewind
Nov 11, 2021

Arzakon posted:

What are you doing now, where are you at in your career, and how far out in the backwoods are you? I only have a few thousand options that are all remote for you.

I'm unemployed, my career died because I got laid off(the local unemployment office thinks I should go to school to be a welder) and I'm a short distance out of town. Really, I'm just curious if this AWS thing is worth learning at this point or if its already played out as it seems like everything is by the time someone recommends I put time and effort into something.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
I don’t have any AWS certifications and have gotten plenty of high paying jobs managing and writing against these services and helping companies be productive with them. An AWS certification like most tech certifications are not comparable to certifications in most other industries and is commonly ridiculed by people in the industry except for those that are obligated to have them due to regulations or professional obligation in itself such as for partnership requirements and simple conditions of employment contracts. If one does not have much of a background in computers already to begin with I do not think it’s going to be sufficient on its own to be viable for employment prospects. It’s excellent if you are going into a consulting role and want an edge over the competition in terms of labor and other consulting companies. With that said, to address the question more directly, no, AWS certifications aren’t really going away but there are bigger dynamics at work that need to be addressed before looking at these certifications

pissinthewind
Nov 11, 2021

necrobobsledder posted:

bigger dynamics at work that need to be addressed before looking at these certifications

Liiiiiike? Will a certification convince some suit who doesn't know better to hire a middle aged person looking to switch into a new career? I'm sure as poo poo not going back to school. In college I did all sorts of coding and other tech poo poo, but that was decades ago. I don't want to make zillions of dollars, I just want to eat and pay bills and have some stability while touching myselfcomputers

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord
It’s been a few years since I’ve checked in on this so I figured I’d ask people active in the AWS world: what’s the current standard for scheduling executions of a Docker image and passing in a dynamic command? I’ve got a bunch of different processes that handle large-ish data so I can’t use Lamdba due to time constraints. Previously I used a combo of Step Functions, Batch, and ECR which I assume still works, but is there a more efficient/better way to do it?

Just-In-Timeberlake
Aug 18, 2003
I’ve been a computer toucher for 2 decades, being a welder sounds pretty sweet right now.

Learn how to build bike frames, dentists will throw money at you.

Docjowles
Apr 9, 2009

Yeah welding seems pretty cool tbh although maybe not as a full time job. I used to be heavy into homebrewing beer and I was super jealous of the guys who whipped up these sick “brew sculptures” to hold all their equipment. You could buy them premade from a vendor for thousands of dollars bur it seemed like if you knew what you were doing it was like a day or two of work and some scrap metal.

CarForumPoster
Jun 26, 2013

⚡POWER⚡
Welding for money is a lot less "make cool beer sculpture" and a lot more "repair/install this pipe, girder, etc" by number of welders employed. IMO welding sewer pipe, conduit, pressure vessels, etc. is boring and hot. Same opinion about sitting at a sheet metal brake. Welding and machining stuff for fun is awesome, but I quit CNC machining to touch computers and wouldnt go back. I totally plan to have my own machine shop with a nice MIG and TIG setup if I ever sell my business or get rich enough to have that just sitting around. That said, two of my acquaintances are welders and really like it.

pissinthewind - If you want, check out the resume/interviewing thread in BFC. Lots of hiring managers helping people with career questions as well as resumes. To answer your question directly: no AWS is not in any way "played out". It seems like understanding and setting up systems to power internet services must not be of interest to you, or you'd probably know that already. If you're not interested in that subject, it's likely not a good career choice.

luminalflux
May 27, 2005



PierreTheMime posted:

It’s been a few years since I’ve checked in on this so I figured I’d ask people active in the AWS world: what’s the current standard for scheduling executions of a Docker image and passing in a dynamic command? I’ve got a bunch of different processes that handle large-ish data so I can’t use Lamdba due to time constraints. Previously I used a combo of Step Functions, Batch, and ECR which I assume still works, but is there a more efficient/better way to do it?

You have a few options. If this is just a cron-like thing, you can use EventBridge to trigger an ECS task on a schedule (which is how we're moving our crons from a single host to something more blue/green deployable). You can override the command, entrypoint and environment to the task - we have one single "cron" task definition and pass in from the eventbridge triggers each various cron's arguments.

If you have more complicated stuff with dependencies that's a bit more data sciencey - "This job pulls today's data from Salesforce into S3, this other job pulls today's data from our transactional database, and a third job reconciles them and pushes the result into Redshift. Both need to have run before the reconciliation, and ideally we would like the option to retrigger a date's run if something's hosed up" you will want a DAG scheduler system like Airflow, especially if you're running Spark jobs. I would however not recommend using EMR or Batch but go to Databricks, and probably look at running your own Airflow instead of AWS Managed Workflows with Apache Airflow (MWAA). We've had huge issues with all of them and are looking to migrate off MWAA to our own Airflow.

pissinthewind
Nov 11, 2021

Welding is something I want to do for fun, I'm just not sure its going to be a good choice for dollarydoos compared to computer touching. Seems like welding jobs cap out around the mid/low 40s around here and don't really offer remote work. I'm not really interested in catching omegacron, drowning to death on my own fluids isn't necessarily my death of choice.

pissinthewind fucked around with this message at 19:07 on Dec 4, 2021

Thanks Ants
May 21, 2004

#essereFerrari


I had this video randomly appear on YouTube and just nope nope nope nope, whatever these people get paid is worth it

https://youtu.be/AK47SC6kr_A

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord

luminalflux posted:

You have a few options. If this is just a cron-like thing, you can use EventBridge to trigger an ECS task on a schedule (which is how we're moving our crons from a single host to something more blue/green deployable). You can override the command, entrypoint and environment to the task - we have one single "cron" task definition and pass in from the eventbridge triggers each various cron's arguments.

If you have more complicated stuff with dependencies that's a bit more data sciencey - "This job pulls today's data from Salesforce into S3, this other job pulls today's data from our transactional database, and a third job reconciles them and pushes the result into Redshift. Both need to have run before the reconciliation, and ideally we would like the option to retrigger a date's run if something's hosed up" you will want a DAG scheduler system like Airflow, especially if you're running Spark jobs. I would however not recommend using EMR or Batch but go to Databricks, and probably look at running your own Airflow instead of AWS Managed Workflows with Apache Airflow (MWAA). We've had huge issues with all of them and are looking to migrate off MWAA to our own Airflow.

I appreciate the info. I suppose I should clarify my ask in that the desire is for it to be completely ephemeral, as this would be something we could deploy on many disparate client AWS accounts instead of being centrally controlled. Airflow can certainly do the job, but if we're hosting something continuously and need to setup cross-account control we have a traditional workload automation platform we could use.

luminalflux
May 27, 2005



PierreTheMime posted:

I appreciate the info. I suppose I should clarify my ask in that the desire is for it to be completely ephemeral, as this would be something we could deploy on many disparate client AWS accounts instead of being centrally controlled. Airflow can certainly do the job, but if we're hosting something continuously and need to setup cross-account control we have a traditional workload automation platform we could use.

Yeah, spinning up an Airflow per account is ... doable but not fun. If your workflow can fit EventBridge + ECS Fargate task constraints that's really easy to deploy.

pissinthewind
Nov 11, 2021

Thanks Ants posted:

I had this video randomly appear on YouTube and just nope nope nope nope, whatever these people get paid is worth it

https://youtu.be/AK47SC6kr_A

I had a couple friends go into underwater welding, my understanding is that job is so dangerous and most of them die or retire by their 30s. I'm too old for that poo poo. I liked my non-tech computer toucher work from home 60k a year job, but apparently that career is in decline. I'll just get the basic AWS certification and see if I can shmoooz my way into something. Even without much knowledge beyond generic computer stuff schooling from decades ago the practice tests make it seem super loving easy to get.

cage-free egghead
Mar 8, 2004
What are your guys' opinions on the AWS Resume Challenge for someone not quite done with school, has a bunch of years of IT support, the CCP, Cloud+, other certs?

CarForumPoster
Jun 26, 2013

⚡POWER⚡

cage-free egghead posted:

What are your guys' opinions on the AWS Resume Challenge for someone not quite done with school, has a bunch of years of IT support, the CCP, Cloud+, other certs?

I'd not head of this. Assuming that it is this: https://cloudresumechallenge.dev/docs/the-challenge/aws/

My thoughts are:
Why?
-and-
For what job?

If the why is "so that I know that I can do this" yea this project seems like a decent entry-level project. If a new-grad engineer 1 had this as a project on github with a readme explaining what it was and does, it'd count as a useful project to do with me.

If the why is "This website is how I will share my resume. This and only this." Thats a poo poo answer, do not do that.

cage-free egghead
Mar 8, 2004

CarForumPoster posted:

I'd not head of this. Assuming that it is this: https://cloudresumechallenge.dev/docs/the-challenge/aws/

My thoughts are:
Why?
-and-
For what job?

If the why is "so that I know that I can do this" yea this project seems like a decent entry-level project. If a new-grad engineer 1 had this as a project on github with a readme explaining what it was and does, it'd count as a useful project to do with me.

If the why is "This website is how I will share my resume. This and only this." Thats a poo poo answer, do not do that.

I'm trying to get out of end user support and into any sort of cloud stuff. Getting close to finishing the Cloud Computer bachelors at WGU and came across this little challenge as a way to get my feet wet with some stuff, plus have a talking point for interviews and such.

CarForumPoster
Jun 26, 2013

⚡POWER⚡

cage-free egghead posted:

I'm trying to get out of end user support and into any sort of cloud stuff. Getting close to finishing the Cloud Computer bachelors at WGU and came across this little challenge as a way to get my feet wet with some stuff, plus have a talking point for interviews and such.

Yes seems pretty good for that.

BaseballPCHiker
Jan 16, 2006

cage-free egghead posted:

I'm trying to get out of end user support and into any sort of cloud stuff. Getting close to finishing the Cloud Computer bachelors at WGU and came across this little challenge as a way to get my feet wet with some stuff, plus have a talking point for interviews and such.

I was a former network engineer, who was trying to break into security and then specifically cloud security.

I made a lovely wordpress blog, but had cloudtrail, guardduty, shield, etc (all as cheaply done as possible) enabled so that I could show my work to an extent. It helped me land my current job, so I think its definitely useful.

air-
Sep 24, 2007

Who will win the greatest battle of them all?

cage-free egghead posted:

I'm trying to get out of end user support and into any sort of cloud stuff. Getting close to finishing the Cloud Computer bachelors at WGU and came across this little challenge as a way to get my feet wet with some stuff, plus have a talking point for interviews and such.

Would spend more time on this and Digital Ocean gives you credits

https://www.digitalocean.com/community/pages/kubernetes-challenge

Arzakon
Nov 24, 2002

"I hereby retire from Mafia"
Please turbo me if you catch me in a game.
For those of you looking to switch career paths into tech my group hires using Tech U for entry level talent, usually new grads or people with professional careers who are looking to switch into tech and have some fundamentals (can talk one of software development, compute, storage, databases at a 200-300 level). If you've built literally anything you are better than about half of candidates, if you get your SA Associate certification that is another leg up. If anyone is interested in more details shoot me a PM, I don't see our reqs listed but we are looking for Seattle and Austin, maybe New York at the moment.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
Didn’t need the east coast anyways.

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord
What is the best way to transfer a large amount of large-ish data files from one S3 bucket to another, with the complication being that each bucket has a separate access credential and the source bucket owner cannot grant permissions directly to a user/role at the destination? If one account had access to both ends I could do a number of things, but I don't know if there's anything better than just having the source connection stream the data across to the destination using code as an intermediary. Right now I have a multithreaded process that's trying it's best, but it's something like 10,000 files averaging 200MB a piece so that's going to take forever.

vanity slug
Jul 20, 2010

PierreTheMime posted:

What is the best way to transfer a large amount of large-ish data files from one S3 bucket to another, with the complication being that each bucket has a separate access credential and the source bucket owner cannot grant permissions directly to a user/role at the destination? If one account had access to both ends I could do a number of things, but I don't know if there's anything better than just having the source connection stream the data across to the destination using code as an intermediary. Right now I have a multithreaded process that's trying it's best, but it's something like 10,000 files averaging 200MB a piece so that's going to take forever.

AWS DataSync? Or just s5cmd.

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord

Jeoh posted:

AWS DataSync? Or just s5cmd.

Datasync requires you to apply a policy to the source bucket, right? Can s5cmd use a source and destination profile or is it just a single credential?

BaseballPCHiker
Jan 16, 2006

Is there like a big zip file of sanitized cloudtrail logs floating around online you can use as a teaching resource?

Trying to get some of my team up to speed on reading the logs as they would relate to GuardDuty alerts and I am not finding much online aside from one off log examples.

BaseballPCHiker
Jan 16, 2006

Has anyone heard any more specifics off of this announcement yet?
https://aws.amazon.com/blogs/security/using-aws-security-services-to-protect-against-detect-and-respond-to-the-log4j-vulnerability/

Are they creating new GuardDuty alerts or just updating IOCs on the backend for the same type of alerts?

BaseballPCHiker
Jan 16, 2006

BaseballPCHiker posted:

Are they creating new GuardDuty alerts or just updating IOCs on the backend for the same type of alerts?

Answering my own question. GuardDuty team is updating IOCs for existing alerts.

Also this is just the crisis I needed to finally get GuardDuty enabled on all of our accounts! Yeah you read that right, GuardDuty was previously thought to be to expensive for this poo poo org.

cage-free egghead
Mar 8, 2004
I guess I'll crosspost this question here since it's more appropriate!

cage-free egghead posted:

Anyone know anything about associate positions at AWS? I applied to a bunch of positions through Amazon Student's job portal and a recruiter reached out to me. Haven't heard or read much about anyone who's worked at AWS in one of these entry level jobs so I know nothing about them. I'd assume at the very least it'd be worth it for experience in AWS which is exactly what I'm after right now.

Edit: I'm also not seeing any sort of indication that the position can be done remotely, which is a must for me.

https://www.amazon.jobs/en/jobs/1632446/cloud-support-associate-2022

cage-free egghead fucked around with this message at 21:50 on Dec 22, 2021

Plank Walker
Aug 11, 2005
Is there a way to default to a different credentials profile for a specific subdirectory for the CLI besides using the --profile flag on each command or setting an environment variable at the beginning of the terminal session? Like, can I have a local .aws file that overrides the one in my home directory?

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost

Plank Walker posted:

Is there a way to default to a different credentials profile for a specific subdirectory for the CLI besides using the --profile flag on each command or setting an environment variable at the beginning of the terminal session? Like, can I have a local .aws file that overrides the one in my home directory?

Try the AWS_CONFIG_FILE environment variable. I've used it to transfer STS tokens across Jenkins jobs on separate nodes before. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
I dynamically add functions that are just the names of my aws profiles for easy switching between them on the command line; stick this in your .zshrc or whatever file and just type the name of the profile to switch contexts to it:
pre:
for profile in $(awk '/\[profile/ { sub("\]","",$2); print $2 }' ~/.aws/config)
do
  $profile() {
    export AWS_PROFILE=$0
  }
done
If you're using zsh, you can use the builtin function chpwd() which auto-executes every time you switch directories to auto-switch between environments depending on your directory as well, dumping something like this in your .zhsrc:

pre:
AWS_DIR_DEV=("git/devrepo")
AWS_DIR_PROD=("git/prodrepo")
function chpwd() {
  if   [[ " ${PWD} " =~ " ${HOME}/${AWS_DIR_DEV[@]}"  ]]; then export AWS_PROFILE=dev-profile;
  elif [[ " ${PWD} " =~ " ${HOME}/${AWS_DIR_PROD[@]}" ]]; then export AWS_PROFILE=prod-profile;
  else export AWS_PROFILE=''
  fi
}
And if you're running oh-my-zsh/powershell10k you can configure it to fill in your bar with whatever profile is currently active, and like, make prod in red or something.

Bhodi fucked around with this message at 22:34 on Dec 23, 2021

12 rats tied together
Sep 7, 2006

for this scenario I use a single profile and the credentials in that profile have only one permission, the ability to sts:AssumeRole a management principal for each account/context, and then i have a shell command for switching between those roles

Hughmoris
Apr 21, 2007
Let's go to the abyss!
Anyone have knowledge/experience with the AWS Certified Data Analytics – Specialty (DAS-C01) cert?

I've been doing data analyst work for 5+ years now in healthcare. I recently earned my AWS CCP cert and am trying to figure out what next. Plus, my new job is DOD related and I'm curious if this would open any doors.

BaseballPCHiker
Jan 16, 2006

Hughmoris posted:

Anyone have knowledge/experience with the AWS Certified Data Analytics – Specialty (DAS-C01) cert?

I've been doing data analyst work for 5+ years now in healthcare. I recently earned my AWS CCP cert and am trying to figure out what next. Plus, my new job is DOD related and I'm curious if this would open any doors.

My coworker just got it. Seemed to be heavily focused on various DB services and things like Athena, DataLake, etc.

I cant say that I've ever seen it listed on any job postings though.

Hughmoris
Apr 21, 2007
Let's go to the abyss!

BaseballPCHiker posted:

My coworker just got it. Seemed to be heavily focused on various DB services and things like Athena, DataLake, etc.

I cant say that I've ever seen it listed on any job postings though.

Yeah, neither have I. :smith:

Scrapez
Feb 27, 2004

Anyone know if you can create a VPC with cloudformation that contains two CIDRs?

All the documentation I see talks about creating the VPC with a single CIDR and then adding a secondary after the fact. The documentation is all focused on manually creating things, though instead of cloudformation.

12 rats tied together
Sep 7, 2006

The aws::ec2::vpc resource only accept a primary cidr block: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html#cfn-aws-ec2-vpc-cidrblock

You would have to create an aws::ec2::vpccidrblock: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpccidrblock.html which lets you bind a cidr block and a vpc together.

Adbot
ADBOT LOVES YOU

Scrapez
Feb 27, 2004

12 rats tied together posted:

The aws::ec2::vpc resource only accept a primary cidr block: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html#cfn-aws-ec2-vpc-cidrblock

You would have to create an aws::ec2::vpccidrblock: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpccidrblock.html which lets you bind a cidr block and a vpc together.

Thank you! Exactly what I needed.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply