Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Scrapez
Feb 27, 2004

JHVH-1 posted:

Probably have to play around with the alarms and getting the right metrics so you have both scale up and scale down criteria based on something that covers the whole cluster.
My last company we had a developer that was populating a metric in their code and never thought about creating a scale down one, so the thing would get busy or a bug would scale it out like crazy and then never reduce it.

Definitely. I need to get my head around how I'm going to do that. I'll basically be starting with 3 instances, each having a call capacity of 500 calls. So the thought on scale up was just to spin up another instance if any of the 3 reach 450 concurrent calls. The scale down side becomes a bit more difficult. I'm thinking I can write the concurrent calls value from all active instances and if the average of that number falls below a certain value I terminate one of the instances. That will also require keeping track of how many of this type of instance are active.

The good thing about this platform, at least right now, is that it will be very predicable. There won't be huge peak calling events or anything like that. It likely will just continue to slowly scale up over time.

One of my fears is that I do this wrong and instances start launching willy nilly all crazy.

Adbot
ADBOT LOVES YOU

JHVH-1
Jun 28, 2002
You can do some tweaking based on time at least. I've done that before reducing the capacity over the weekends when I knew it wouldn't be that important.

At least you can make the metrics and create alarms and see what they do before making them the scaling criteria. Like if you know you have 0 calls during long periods on a regular basis you could set an alarm to scale down then or something.

Scrapez
Feb 27, 2004

JHVH-1 posted:

You can do some tweaking based on time at least. I've done that before reducing the capacity over the weekends when I knew it wouldn't be that important.

At least you can make the metrics and create alarms and see what they do before making them the scaling criteria. Like if you know you have 0 calls during long periods on a regular basis you could set an alarm to scale down then or something.

Yeah that makes sense. Overnight or on weekends potentially. At this point, I don't know exactly who the customer base is going to be so it's possible that it will be a lot of international and there won't be a truly "slow time."

Being able to send anything you want back to Cloudwatch via the CLI is really awesome and I can use that for a whole host of other things from alarming problems to reports on various things.

JHVH-1
Jun 28, 2002
Ya know I just remembered, I think you can also send your logs to cloudwatch and then create metric filters based on that.
The newer cloudwatch agent also collects some other stats for you too. I haven't yet played with it, just the older cloudwatch logs one https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html

abelwingnut
Dec 23, 2002


i'm new to aws and ec2, so forgive me if this is trivial. but is it possible to create a ubuntu instance somewhere in aws that's essentially a remote desktop? i'd love to be able to log on to this gui-ed instance from any browser or computer, and just do some dev work on there, which would entail remoting in to other servers and what not.

fluppet
Feb 10, 2009
https://aws.amazon.com/workspaces/ I think it now handles Linux desktops

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

PierreTheMime posted:

I'm positive that this has been covered at some point, but what's considered the best S3 GUI for Windows? I'm perfectly fine working in CLI/API, but I have a few users asking and I honestly don't know. Someone mentioned wanting to have the S3 bucket as a mounted drive, which I'm sure is done. I figure I'd trust the working knowledge here more than just a quick Googling.

My customers use S3 Browser by the NetSDK folks. Most use the free version, but that is essentially for up and down stuff. For actual bucket management you’ll need the pro version which starts at thirty bucks a seat.

YMMV.

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord

Agrikk posted:

My customers use S3 Browser by the NetSDK folks. Most use the free version, but that is essentially for up and down stuff. For actual bucket management you’ll need the pro version which starts at thirty bucks a seat.

YMMV.

Thanks. Ultimately the purchasing is out of my hands but I’m interested to know what’s being used. With as simple as the interface is I’m strongly considering just whipping up a Swing drag and drop explorer for fun and see how it compares.

JHVH-1
Jun 28, 2002
I use cyberduck on the mac, but no idea if its any good on windows. Also versioned buckets confuse the hell out of it if you delete something so stuff still shows up because of the way s3 handles "directories"

Cloudberry Explorer is supposed to be pretty good. Sometimes I wish there was an easy web based front end that I could hand out creds for and not have to manage keys or have people go on the dashboard. We sometimes have the problem that people were sharing keys. The last time I got a request related to that I ended up adding a role we mapped to an AD group and let them log in via an application in office.com. That way they just click an icon and their role only allows the s3 bucket they have permissions to access to in the aws dashboard.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

PierreTheMime posted:

Thanks. Ultimately the purchasing is out of my hands but I’m interested to know what’s being used. With as simple as the interface is I’m strongly considering just whipping up a Swing drag and drop explorer for fun and see how it compares.

Here are some more alternative options for graphical S3 management I have researched:

- AWS ElasticWolf Client Console (Fully Supported by AWS) Free software.

- Cyberduck - (3rd party) Free software.

Cyberduck is a Libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift browser for Mac and Windows.
It just requires your AWS credentials and you can use the simple interface to download/upload/sync any of your buckets/folders/files.


- CloudBerry Explorer - (3rd party) Has a free version. Full feature costs $40.

CloudBerry Explorer for Amazon S3 provides a user interface to Amazon S3 accounts allowing to access, move and manage files across your local storage and S3 buckets. Amazon S3 file manager by CloudBerry is available in two versions – Freeware and PRO.
Freeware version. Free S3 browser comes with full support for such AWS features and services as Server Side Encryption, Lifecycle rules, Amazon CloudFront, Bucket Policies and more.

fluppet
Feb 10, 2009
Why is this the first time I've ever heard of elasticwolf?

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord

fluppet posted:

Why is this the first time I've ever heard of elasticwolf?

Probably the same reason I hadn’t when I started asking. When you Google “aws s3 gui” or “aws s3 browser” it’s not in the first two pages.

JHVH-1
Jun 28, 2002
Though elasticwolf sounded interesting but the last release was in 2017. It probably does basic stuff fine. The problem I had a few years ago when evaluating s3 clients is that so little of them at the time supported modern s3 features like server side encryption. Though now you can set buckets to force that by default, so it wouldn’t matter.

Umbreon
May 21, 2011
If I wanted to get into cloud networking and infrastructure stuff, would it be a good idea to pick up any AWS certs?

CarForumPoster
Jun 26, 2013

⚡POWER⚡
Hey, I am trying to run my first EBS instance with a flask/dash app that runs fine locally. I am on windows and everything is bitching when I try to use the AWS CLI, so I am setting it up through the web browser. This is a file I downloaded a zip from GitHub for and then uploaded to EBS through a browser upload.

code:
[Mon Feb 04 19:32:21.434707 2019] [mpm_prefork:notice] [pid 4594] AH00163: Apache/2.4.37 (Amazon) mod_wsgi/3.5 Python/3.6.7 configured -- resuming normal operations
[Mon Feb 04 19:32:21.434727 2019] [core:notice] [pid 4594] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Mon Feb 04 19:32:24.209122 2019] [:error] [pid 4599] [remote 127.0.0.1:0] mod_wsgi (pid=4599): Target WSGI script '/opt/python/current/app/app-folder-name/application.py' cannot be loaded as Python module.
[Mon Feb 04 19:32:24.209181 2019] [:error] [pid 4599] [remote 127.0.0.1:0] mod_wsgi (pid=4599): Exception occurred processing WSGI script '/opt/python/current/app/app-folder-name/application.py'.
[Mon Feb 04 19:32:24.209391 2019] [:error] [pid 4599] [remote 127.0.0.1:0] Traceback (most recent call last):
[Mon Feb 04 19:32:24.209413 2019] [:error] [pid 4599] [remote 127.0.0.1:0]   File "/opt/python/current/app/app-folder-name/application.py", line 1, in <module>
----->   [Mon Feb 04 19:32:24.209417 2019] [:error] [pid 4599] [remote 127.0.0.1:0]     import dash
But there's a requirements.txt file that I thought would install automatically, and it says to install dash:

code:
-----> dash==0.35.2
dash-core-components==0.42.1
dash-html-components==0.13.5
dash-renderer==0.16.2
decorator==4.3.0
Am I able to force it to install requirements.txt somehow?

EDIT: Posting about it made AWS EB CLI work. Gonna try to ssh in now.

CarForumPoster fucked around with this message at 21:19 on Feb 4, 2019

Docjowles
Apr 9, 2009

This isn't really anything related to AWS. The requirements.txt file doesn't just magically do anything on its own. You need to do something like "pip install -r requirements.txt" first to actually download and install the dependencies. Then your app should work.

CarForumPoster
Jun 26, 2013

⚡POWER⚡

Docjowles posted:

This isn't really anything related to AWS. The requirements.txt file doesn't just magically do anything on its own. You need to do something like "pip install -r requirements.txt" first to actually download and install the dependencies. Then your app should work.

I thought EBS did that automagically when I uploaded my app. Apparently not. I'm SSH'd in as ec2-user, but ls/dir reveals no files. I know my requirements.txt is in

???Somewhere???/opt/python/current/app/app-folder-name/

....any idea where that is


EDIT: Posting about it made me not a dumbass and I installed it.

Docjowles
Apr 9, 2009

Sorry I misunderstood. Didn’t realize you meant Elastic Beanstalk instead of Elastic Block Storage by EBS. Beanstalk should be installing your requirements when the app is deployed, yes.

JHVH-1
Jun 28, 2002

Docjowles posted:

This isn't really anything related to AWS. The requirements.txt file doesn't just magically do anything on its own. You need to do something like "pip install -r requirements.txt" first to actually download and install the dependencies. Then your app should work.

I think it does install the packages if you format it correctly and put it in your application bundle:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html

Also says if it fails validation it won't work and you have to check the logs for that as it shows up as a warning.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!
Does storing a parameter as a “secure string” in system manager: parameter store cost me anything? I seems like it’s free cause I don’t see any pricing, but “secret manager” says it costs $1 per secret or whatever so I don’t understand the difference here.

Arzakon
Nov 24, 2002

"I hereby retire from Mafia"
Please turbo me if you catch me in a game.

Boris Galerkin posted:

Does storing a parameter as a “secure string” in system manager: parameter store cost me anything? I seems like it’s free cause I don’t see any pricing, but “secret manager” says it costs $1 per secret or whatever so I don’t understand the difference here.

Systems Manager is free including the Parameter Store feature. Secrets Manager has some built-in integrations with services like RDS but if you don't need those specific features just use Parameter Store.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Umbreon posted:

If I wanted to get into cloud networking and infrastructure stuff, would it be a good idea to pick up any AWS certs?

Yes.

Get your AWS Solutions Architect - Pro cert.

It’s all about the AWS core services and putting them together to do things.

Boris Galerkin
Dec 17, 2011

I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

Arzakon posted:

Systems Manager is free including the Parameter Store feature. Secrets Manager has some built-in integrations with services like RDS but if you don't need those specific features just use Parameter Store.

Thanks. And just make sure, it’s ok to store things like say slack api tokens in parameter store right?

Umbreon
May 21, 2011

Agrikk posted:

Yes.

Get your AWS Solutions Architect - Pro cert.

It’s all about the AWS core services and putting them together to do things.

Awesome, is there a cert path to that, or do I just go straight for it?

Arzakon
Nov 24, 2002

"I hereby retire from Mafia"
Please turbo me if you catch me in a game.

Boris Galerkin posted:

Thanks. And just make sure, it’s ok to store things like say slack api tokens in parameter store right?

Sure, take a look at granular permissions and general IAM strategy if a bunch of other people also use your AWS account who you don't want having access to that key but otherwise go hog wild.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Umbreon posted:

Awesome, is there a cert path to that, or do I just go straight for it?

You need to get the SA associate very first. Then sit for the SA Pro.

Scrapez
Feb 27, 2004

I'm sure there's a very reasonable explanation but why can't you set DHCP Options Sets at the subnet level?

I have different types of machines in a single VPC and was hoping to be able to give them hostnames that would identify which type they are.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Scrapez posted:

I'm sure there's a very reasonable explanation but why can't you set DHCP Options Sets at the subnet level?

This is a perfectly reasonable request and one that I have heard countless times before.

At a very high level, it’s a performance issue. Allowing dhcp option sets per vpc is one thing. Allowing option sets for subnets, that can exist at a ratio of several hundred to one, is something else.

But yeah, option sets for subnets would be awesome.

Scrapez
Feb 27, 2004

Agrikk posted:

This is a perfectly reasonable request and one that I have heard countless times before.

At a very high level, it’s a performance issue. Allowing dhcp option sets per vpc is one thing. Allowing option sets for subnets, that can exist at a ratio of several hundred to one, is something else.

But yeah, option sets for subnets would be awesome.

Yeah, I guess I understand that. Perhaps they could make it so that only subnets of a certain size would be allowed to have dhcp options sets.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Scrapez posted:

Yeah, I guess I understand that. Perhaps they could make it so that only subnets of a certain size would be allowed to have dhcp options sets.

Speaking personally, I believe there should just be a fixed limit, say 5 or 10, of option sets per vpc. One could implement a default option set for the vpc and then allocate subnet option sets for special cases that could override the default.

But I don’t know the exact details from the networking guys, so I don’t know the real roadblocks to implementation.

Scrapez
Feb 27, 2004

Agrikk posted:

Speaking personally, I believe there should just be a fixed limit, say 5 or 10, of option sets per vpc. One could implement a default option set for the vpc and then allocate subnet option sets for special cases that could override the default.

But I don’t know the exact details from the networking guys, so I don’t know the real roadblocks to implementation.

Yeah that would be great. Second on my list of wants behind ELBs that can do UDP.

Walked
Apr 14, 2003

What do I need to do to get a TAM assigned to my organization? My organization's monthly spend is growing pretty rapidly - and that's ok - but as the IT director I'd like to establish a POC for escalation / coordinating in the event we need it (haven't yet, thankfully)

Can't seem to find a good line of communication and have submitted a request to support and haven't gotten any useful info yet - just want to be sure I'm not missing a correct route for asking.

RVWinkle
Aug 24, 2004

In relating the circumstances which have led to my confinement within this refuge for the demented, I am aware that my present position will create a natural doubt of the authenticity of my narrative.
Nap Ghost

Walked posted:

What do I need to do to get a TAM assigned to my organization? My organization's monthly spend is growing pretty rapidly - and that's ok - but as the IT director I'd like to establish a POC for escalation / coordinating in the event we need it (haven't yet, thankfully)

Can't seem to find a good line of communication and have submitted a request to support and haven't gotten any useful info yet - just want to be sure I'm not missing a correct route for asking.

Are you using the Enterprise Support Plan?

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Walked posted:

What do I need to do to get a TAM assigned to my organization? My organization's monthly spend is growing pretty rapidly - and that's ok - but as the IT director I'd like to establish a POC for escalation / coordinating in the event we need it (haven't yet, thankfully)

Can't seem to find a good line of communication and have submitted a request to support and haven't gotten any useful info yet - just want to be sure I'm not missing a correct route for asking.

The only way to get a TAM assigned to your account is to sign an enterprise support agreement. Where is your company based and what is your average approximate monthly spend?

Walked
Apr 14, 2003

Agrikk posted:

The only way to get a TAM assigned to your account is to sign an enterprise support agreement. Where is your company based and what is your average approximate monthly spend?

Thanks guys.

We've gone from about ~$5k/mo to ~$40k/mo over the last year. So we're inching towards the enterprise support contract being super worthwhile. I'll have to look at some numbers and see if I can make a compelling case.

I really just want an escalation point for when support isn't quite getting it (and by "getting it" - I mean moving unreasonably slowly on something / passing the buck). Doesnt happen too often; but every so often I'll catch a case we have open that seems to move unreasonably slowly. (Almost all of them seem to revolve around GovCloud items, admittedly.)

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
Note that enterprise support starts at $15,000 per month and goes up from there.

I’m not sure your monthly spend makes having a TAM and the other perks worth it yet.

WRT your cases getting handled poorly, there is absolutely nothing wrong with copy/pasting the following text into your case:

“Dear [Blank]- I am feel frustrated with how this case has been handled thus far. Please engage with me more closely so we can resolve this case quickly and to our satisfaction. When can we schedule a call to talk about this?”

Docjowles
Apr 9, 2009

Agrikk posted:

Note that enterprise support starts at $15,000 per month and goes up from there.

I’m not sure your monthly spend makes having a TAM and the other perks worth it yet.

WRT your cases getting handled poorly, there is absolutely nothing wrong with copy/pasting the following text into your case:

“Dear [Blank]- I am feel frustrated with how this case has been handled thus far. Please engage with me more closely so we can resolve this case quickly and to our satisfaction. When can we schedule a call to talk about this?”

You or some other Amazon goon gave me the advice that you always want to do your support interactions over the phone, and that has held extremely true for me. You'll need to block out some time to talk to someone, but it's worth it. My phone cases are resolved in like an hour. The asynchronous "web" option or whatever they call it where you post a message will take days to weeks unless it's something dead simple like "increase my EC2 instance limit in this region by 100".

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
Yep.

The "web" (email) option is for poo poo you just want to toss over the wall and forget about until you get a response.

The chat or phone options are the only way to ensure results. Sure, there won't be any multitasking (or even leaving your desk) by you while the case is being actively handled but it's being actively handled so it's a win.

freeasinbeer
Mar 26, 2015

by Fluffdaddy
Stupid question do I need to signup my sub accounts for enterprise? It’s the first time I’ve set one up in awhile and as far as I know our dedicated spend contract should just have that roll down right?


I guess I could bug our TAM, but :effort:

Adbot
ADBOT LOVES YOU

RVWinkle
Aug 24, 2004

In relating the circumstances which have led to my confinement within this refuge for the demented, I am aware that my present position will create a natural doubt of the authenticity of my narrative.
Nap Ghost
I was checking a reporting tool a work today and unlimited S3 bucket size is no joke. We have one bucket that is more than a petabyte!

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply