Amazon Web Services - Cloud Giant Hits Hard - The Something Awful Forums

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Amazon Web Services - Cloud Giant Hits Hard

«‹›61 »

Seventh Arrow: Jan 26, 2005

Seventh Arrow posted:

We were talking about setting up accounts for the teachers, admins, TA's and students. For the students, I told him that AWS allows federated logins, so that people can log in to the AWS environment via Google, FB, or Amazon.com, and that piqued his interest. I wasn't sure about security though, but upon reading about it further, it seems like AWS takes the federated accounts and slaps a role on them. I'll have to find out what role to look for so I can grant it the necessary access.

Ok so the setup for this seems to be a bit more involved than I thought it would be. Is anyone using Cognito enough that they might be able to comment? To reiterate, the school that I volunteer for has data science students who need to use S3, spin up EC2 instances, use DynamoDB, etc. Previously, all the students were using one login but the instructor agrees that this is not only insecure, but makes it difficult to track stuff down like 'who didn't shut down their EMR instance.' So I told him that AWS allowed people to sign in via Facebook, Google, and/or Amazon.com and it dumps them in the system and slaps a role on them. So I set that all up as best as I could:

I set up a domain name:

But apparently I had to create an app client as well:

And set it up:

But then when I try to go to the URL ( https://weclouddata.auth.us-east-1....srpgoa5t009k27s ), I get an "invalid request" error. I submitted this on the AWS forums, but they take a long time to respond. Anyone have any ideas in the meantime?

# ? Feb 22, 2018 23:06

Adbot: ADBOT LOVES YOU

# ? May 17, 2024 01:44

putin is a cunt: Apr 5, 2007; BOY DO I SURE ENJOY TRASH. THERE'S NOTHING MORE I LOVE THAN TO SIT DOWN IN FRONT OF THE BIG SCREEN AND EAT A BIIIIG STEAMY BOWL OF SHIT. WARNER BROS CAN COME OVER TO MY HOUSE AND ASSFUCK MY MOM WHILE I WATCH AND I WOULD CERTIFY IT FRESH, NO QUESTION

SnatchRabbit posted:

Thanks, those are very useful links. Re: Lambda, I wasn't sure how feasible it was, it was more like an idea for us to dip our toe into serverless. Yeah, I'll bring that up that we are supposed to notify AWS. the problem with flow logs is that this is a build/test environment so I don't think there's much traffic flowing through as of yet.

If it's just a one-time attempt to access a certain port to see whether it's open, a Lambda behind API Gateway is way overcomplicated.

If you want to monitor the port that's a different story, but Lambda is still not really the appropriate tool. I don't have a lot of experience with pen. testing otherwise I'd offer you some suggestions on tools to use.

# ? Feb 23, 2018 01:03

Startyde: Apr 19, 2007; come post with us, forever and ever and ever

If you really want it in lambda and not sitting on your monitoring you could just use the cron schedule rule to fire it off

# ? Feb 23, 2018 09:35

SnatchRabbit: Feb 23, 2006; by sebmojo

Thanks for the replies. Another question: can anyone recommend a good S3 viewer for Mac OS?

# ? Mar 2, 2018 20:01

fluppet: Feb 10, 2009

SnatchRabbit posted:

Thanks for the replies. Another question: can anyone recommend a good S3 viewer for Mac OS?

Cyber duck is one of the least bad options if you need a gui
If not stick to s3cmd

# ? Mar 3, 2018 10:40

Less Fat Luke: May 23, 2003; Exciting Lemon

SnatchRabbit posted:

Thanks for the replies. Another question: can anyone recommend a good S3 viewer for Mac OS?

Transmit is great (especially if you can expense it).

# ? Mar 3, 2018 15:03

JHVH-1: Jun 28, 2002

Less Fat Luke posted:

Transmit is great (especially if you can expense it).

Does it let you set encryption options these days? I think I was using it maybe 3 years ago before I started my last job, but I found I couldn't set the default encryption options for s3.
Like now you can at least have bucket policies and force it, but at the time it was annoying. Cyberduck was the only client I found that had this as an option so I stuck with it. Plus it can generate temporary signed URLs easily.
If I am just syncing a directory even I use cyberduck to grab the s3:// url and then jump on command like and use 'aws s3 sync'

# ? Mar 3, 2018 20:21

jiffypop45: Dec 30, 2011

Cyberduck is what I have used as well. You can try storage gateway then mount it but I've never been able to get that to work right.

# ? Mar 4, 2018 05:34

SnatchRabbit: Feb 23, 2006; by sebmojo

Thanks again, yet another question: Is there a tool or command I can use to get a list of all the AMIs that are currently being used with EC2 in my environment? Essentially, we want to be able to prove for regulatory reasons we are only using base AMIs with all services disabled by default.

# ? Mar 14, 2018 16:10

deedee megadoodoo: Sep 28, 2000; Two roads diverged in a wood, and I, I took the one to Flavortown, and that has made all the difference.

SnatchRabbit posted:

Thanks again, yet another question: Is there a tool or command I can use to get a list of all the AMIs that are currently being used with EC2 in my environment? Essentially, we want to be able to prove for regulatory reasons we are only using base AMIs with all services disabled by default.

I just wrote some code that does this as part of another larger script. This may be overkill but it does what you need.

code:

#!/usr/bin/python
import boto3

ec2 = boto3.client('ec2')
instances = []
paginator = ec2.get_paginator('describe_instances')
for page in paginator.paginate():
    for reservation in page['Reservations']:
        instances += reservation['Instances']

autoscaling = boto3.client('autoscaling')
launch_configurations = []
paginator = autoscaling.get_paginator('describe_launch_configurations')
for page in paginator.paginate():
    launch_configurations += page['LaunchConfigurations']

used_image_ids = list(set([ i['ImageId'] for i in instances + launch_configurations ]))
print used_image_ids

deedee megadoodoo fucked around with this message at 16:45 on Mar 14, 2018

# ? Mar 14, 2018 16:28

vanity slug: Jul 20, 2010

Aww yeah, Frankfurt now supports inter-VPC peering.

# ? Mar 14, 2018 16:38

fluppet: Feb 10, 2009

Where has decent study material/practice papers for the professional level exams?

# ? Mar 14, 2018 20:02

vanity slug: Jul 20, 2010

fluppet posted:

Where has decent study material/practice papers for the professional level exams?

My colleagues are using the Pluralsight stuff which they say is good. They passed their exams so I guess that's true?

# ? Mar 14, 2018 20:26

Rapner: May 7, 2013

fluppet posted:

Where has decent study material/practice papers for the professional level exams?

I did both using only A Cloud Guru, and whitepapers.

# ? Mar 14, 2018 22:43

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

Anyone implemented something like cloud-custodian with any luck? We have something like 60 accounts and are struggling for a good auditing and remediation solution that won't cost an arm and a leg *cough* evident.io *cough*

# ? Mar 14, 2018 23:40

jiffypop45: Dec 30, 2011

AWS released their docs on GitHub today. I'm not 100% certain what the difference between what's online now and what's in there but they're also accepting commits and merge requests on top of it which is cool from the sake of improving documentation.

# ? Mar 15, 2018 00:47

SnatchRabbit: Feb 23, 2006; by sebmojo

Has anyone used Lambda to parse SNS event? I've been trying to parse an AWSConfig rule's SNS events and using the data to do various things. I'm able to parse the data up to a point, but I can't seem to get the message JSON data that I want. I'm using python 2.7, and json, boto3 imports. I think my issue is the JSON is being read as a single key. Anyone know how to do this?

code:

from __future__ import print_function
import boto3
import json

def lambda_handler(event, context):
   message = event['Records'][0]['Sns']['Message']
   message = json.loads(message)
   print(message)

output:

code:

{u'configRuleNames': [u'restricted-sshv2'], u'awsRegion': u'us-west-2', u'messageType': u'ConfigRulesEvaluationStarted', u'recordVersion': u'1.0', u'notificationCreationTime': u'2018-03-29T22:08:34.631Z', u'awsAccountId': u'########'}

event data: https://dumptext.com/jyTfBcNl

SnatchRabbit fucked around with this message at 23:30 on Mar 29, 2018

# ? Mar 29, 2018 23:13

Seventh Arrow: Jan 26, 2005

I have a script that requires the admin account (or any account, really) to assume a role, but it's not having it. I don't think it's necessary to trudge through the script (especially if you're not familiar with python), but just in case, it can be found here.

I get the following: User: arn:aws:iam::197306934454:user/admin is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::197306934454:role/Cognito_WCD_studentsAuth_Role

The admin account has a trust set up with the role in question:

Someone on the AWS forums suggested removing any conditions from the trust relationship and I did so, but no dice. Any ideas?

# ? Apr 6, 2018 02:51

Virigoth: Apr 28, 2009; Corona rules everything around me
C.R.E.A.M. get the virus
In the ICU y'all......

Does the user have iam permissions to assume roles?

I�m actually not sure if you can use PolicySim to test that now that I think about it.

# ? Apr 6, 2018 03:55

Seventh Arrow: Jan 26, 2005

Is there really a specific permission in to assume roles? I didn't see anything like that in IAM, and it seems like the Admin account has most of the bases covered:

# ? Apr 6, 2018 07:16

Vanadium: Jan 8, 2005

I don't think you can use an IAM user ARN in a role's trust relationship. You need to add the account itself (arn:aws:iam::YOURACCOUNTIDHERE:root) as a trusted entity, and then your AdministratorAccess policy can allow users in that account to assume the role.

Edit: maybe not, I guess I confused myself by trying to assume the role too quickly after changing the trust relationships, when the change takes a bunch of seconds to take effect.

Vanadium fucked around with this message at 12:15 on Apr 6, 2018

# ? Apr 6, 2018 11:39

Cancelbot: Nov 22, 2006; Canceling spam since 1928

This is what we do to allow a common root "descend into child" permission;

First the role

Then trust relationship

code:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::0000000000000:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}

edit: forgot a piece, if you want to be explicit about assuming roles from the root account we use this policy for our automatic auditing:

code:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": [
                "arn:aws:iam::00000000000:role/RoleName",
                "arn:aws:iam::00000000001:role/RoleName",
                "arn:aws:iam::00000000002:role/RoleName"
            ]
        }
    ]
}

Cancelbot fucked around with this message at 13:55 on Apr 6, 2018

# ? Apr 6, 2018 13:30

Walked: Apr 14, 2003

Cancelbot posted:

This is what we do to allow a common root "descend into child" permission;

First the role

Then trust relationship
code:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::0000000000000:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}

This is pretty much what we do. I wrote a bootstrap scipt with Python/Boto3 that will bring our new accounts into alignment with this and allow assuming roles. Works pretty great.

# ? Apr 6, 2018 13:39

Agrikk: Oct 17, 2003; Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

jiffypop45 posted:

AWS released their docs on GitHub today. I'm not 100% certain what the difference between what's online now and what's in there but they're also accepting commits and merge requests on top of it which is cool from the sake of improving documentation.

FWIW:

I have customers who now monitor the AWS repositories so they can be notified when docs are added or modified. This allows them to stay on top of feature releases and best practices.

# ? Apr 8, 2018 21:51

jiffypop45: Dec 30, 2011

Customers? Are you a TAM at AWS? Or a reseller? That's a cool use case. I didn't think about that option.

# ? Apr 9, 2018 00:13

Agrikk: Oct 17, 2003; Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

jiffypop45 posted:

Customers? Are you a TAM at AWS? Or a reseller? That's a cool use case. I didn't think about that option.

I�m a TAM at AWS, yes.

It�s funny because notifications has been the biggest excitement generator for most enterprise customers I know.

Updating someone else�s docs? Not so much.

# ? Apr 9, 2018 04:36

jiffypop45: Dec 30, 2011

Agrikk posted:

I’m a TAM at AWS, yes.

It’s funny because notifications has been the biggest excitement generator for most enterprise customers I know.

Updating someone else’s docs? Not so much.

I'm a SysEng on AWS. It's neat to hear that perspective .

# ? Apr 9, 2018 05:38

Acidian: Nov 24, 2006

This thread made me set up an AWS free tier account and run an ec2 instance for my web server as well as a mariadb RDS.

I just got a mail saying I have spent 640 hours of my 750 hours monthly alotment for my EC2. I don't understand why. 750 hours is enough to leave it running all month, and I am only running one EC2 instance.

What am I not understanding?

# ? Apr 16, 2018 09:26

Rapner: May 7, 2013

Acidian posted:

This thread made me set up an AWS free tier account and run an ec2 instance for my web server as well as a mariadb RDS.

I just got a mail saying I have spent 640 hours of my 750 hours monthly alotment for my EC2. I don't understand why. 750 hours is enough to leave it running all month, and I am only running one EC2 instance.

What am I not understanding?

When did you set it up?

Also are they saying you're projected to spend it, or you have spent it?

# ? Apr 16, 2018 09:37

lil bip: Mar 13, 2004; That ain't workin', that's the way you do it

Agrikk posted:

I�m a TAM at AWS, yes.

It�s funny because notifications has been the biggest excitement generator for most enterprise customers I know.

Updating someone else�s docs? Not so much.

I gotta say, my AWS TAM(s)/Solutions Architects are amazing and super helpful when I need any questions answered or just to bounce ideas off when trying to design something new.

# ? Apr 16, 2018 10:02

Acidian: Nov 24, 2006

Rapner posted:

When did you set it up?

Also are they saying you're projected to spend it, or you have spent it?

I can't remember when I set it up he account. I first set up a an Amazon linux distro, but then when I logged in a few weeks later to set up the server it was gone. So I set up an ubuntu distro in stead on March 27th, using the t2.micro.

Under "instances" in my admin panel, there is only one EC2 running (well I stopped it now, because I didn't want to go over the limit).

I have spent 650 hours and I am projected to go 80% over my monthly limit.

Edit: Even though my server is turned off, my hours are still ticking. I think my account is bugged, so I will contact support.

Acidian fucked around with this message at 10:18 on Apr 16, 2018

# ? Apr 16, 2018 10:11

Rapner: May 7, 2013

Acidian posted:

I can't remember when I set it up he account. I first set up a an Amazon linux distro, but then when I logged in a few weeks later to set up the server it was gone. So I set up an ubuntu distro in stead on March 27th, using the t2.micro.

Under "instances" in my admin panel, there is only one EC2 running (well I stopped it now, because I didn't want to go over the limit).

I have spent 650 hours and I am projected to go 80% over my monthly limit.

Edit: Even though my server is turned off, my hours are still ticking. I think my account is bugged, so I will contact support.

Sounds like you have one up in another region.

# ? Apr 16, 2018 10:32

Acidian: Nov 24, 2006

Rapner posted:

Sounds like you have one up in another region.

That's exactly what it was. I had a feeling it had to do with me being retarded.

Thank you!

# ? Apr 16, 2018 14:40

AWWNAW: Dec 30, 2008

Are there any better options for creating IAM users with access keys via cloud formation than outputting the access key as part of the template?

# ? Apr 16, 2018 15:15

jiffypop45: Dec 30, 2011

AWWNAW posted:

Are there any better options for creating IAM users with access keys via cloud formation than outputting the access key as part of the template?

Did you look into the new Secrets Manager?

# ? Apr 16, 2018 15:24

Arzakon: Nov 24, 2002; "I hereby retire from Mafia"
Please turbo me if you catch me in a game.

No, federate or everyone will be sad.

# ? Apr 16, 2018 16:35

Thanks Ants: May 21, 2004; #essereFerrari

The region thing in AWS being a global setting is fairly annoying - I�d like to see all instances returned and then a column for region that can be filtered.

I assume there�s a good technical reason for this and presumably it ensures that each region is separated from another so you don�t have issues with your local region meaning you also lose management of other regions but I�ve not read anything that explains why it�s the way it is.

# ? Apr 16, 2018 16:38

the talent deficit: Dec 20, 2003; self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture

AWWNAW posted:

Are there any better options for creating IAM users with access keys via cloud formation than outputting the access key as part of the template?

don't create access keys in cfn. grant users the ability to create and manage their own keys if they need them or use federation to allow them to assume roles instead

# ? Apr 16, 2018 16:43

jiffypop45: Dec 30, 2011

Arzakon posted:

No, federate or everyone will be sad.

Edit: I think I misunderstood what you were asking. Nevermind.

# ? Apr 16, 2018 16:43

Adbot: ADBOT LOVES YOU

# ? May 17, 2024 01:44

JHVH-1: Jun 28, 2002

Thanks Ants posted:

The region thing in AWS being a global setting is fairly annoying - I�d like to see all instances returned and then a column for region that can be filtered.

I assume there�s a good technical reason for this and presumably it ensures that each region is separated from another so you don�t have issues with your local region meaning you also lose management of other regions but I�ve not read anything that explains why it�s the way it is.

You can't move stuff between regions though, and then you would have to pick region every time you create anything. If it just uses a default you would have to remember to switch it or re-create things all the time.

# ? Apr 16, 2018 17:05

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Amazon Web Services - Cloud Giant Hits Hard

«‹›61 »