Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Red Mike
Jul 11, 2011

Lutha Mahtin posted:

The Lambda service might work for me, but again I don't really know. When I was just trying to learn about EC2, I had the vague outline of an idea for a setup where basically a server is spawned that has my code on it already, or it automatically loads it somehow, and if the server gets destroyed or shut down or whatever, a trigger is in place to spawn it again when possible. Data storage would be taken care of by moving my output (XML from APIs and maybe some text files) to a storage thing that doesn't care if my server gets nuked. The code on the server would also maybe read some from the data store to determine a few things, such as when the last queries were made, or which queries I am interested in at the moment; this would allow me to not be hard-coding state into the code or the server image. Is this sounding like something that makes some sort of sense, in terms of ~~the cloud~~?

There are a number of ways to set it up so that you can instantly spawn a server that has/gets your code, and keeps at least one instance up. Here's two options:
  • Set up an EC2 instance, remote in to it, configure it the way you want (while making sure none of your code is using the exact public/private IP you have on that specific instance). Then you can go to the EC2 dashboard, and make an image of that instance (an AMI) so that you can then go to the AMI list and launch a new instance exactly like that one. Ideally, you'd want to create a launch configuration that uses that AMI now for the next step.
  • Make a script (sh script for Linux, powershell for Windows) that automatically downloads and configures the software you want on the machine. Upload it to S3 to a public folder/folder with permissions set up to allow EC2 to access it. Go to Launch Configurations and make a new one based off of an Amazon AMI preferably (only for Linux) or just whatever OS you want installed. In the 'details' section, there's a 'userdata' input, write a small script there (sh for Linux, powershell for Windows; google for info on how to format this) that downloads your script off of S3 and runs it. This will make it so whenever the launch configuration is used for an instance, it'll run this userdata script automatically.

Now once you've done this, go to Autoscaling and set up an autoscaling group with a min of 1 and a max of 1 with your created launch configurations. If the instance goes down for whatever reason, autoscaling will kick in and create a new one. If you ever need to re-deploy that instance, set max to 2 temporarily and desired to 2, then once the second instance is live, set it back down to 1. That means that if you have a load balancer attached to this autoscaling group, you have literally no downtime for the re-deploy.

Regarding data storage:

  • Set up an EC2 instance that is used for storage independently of your computing instances. Tie in their drives via nfs-server or similar. (Not recommended)
  • Use S3. Always save data to S3, always read data from S3.
  • Use RDS (as a database) and save to/read from there. (Recommended only if your data is a good fit for a database obviously.)

Regarding potential trip-ups with EC2:

  • A "VPC" is basically like a private LAN within EC2 that many instances can share. Nowadays with the new instance types (m4, etc) you require a VPC whether or not you want one. Not sure if this is true for t2 as well, but I'd expect so. Practically, the VPC means all your instances in the same VPC get the same sort of private IP (10.0.100.x) and can access each other over that network. This is useful for allowing, for example, a cluster of instances to talk to each other, but having a bunch of them not accessible from the internet.
  • When you first start an instance in a VPC, you'll get an 'auto-assign public IP' option, which effectively means 'this instance you're creating in this VPC, do you want it accessible from the internet as well'.
  • Security groups are annoying and you will have to check them any time you have connectivity issues. A security group is like a firewall on the machine, blocking input/output ports. By default, instances start out allowing SSH/RDP from anywhere, and not accepting any other input, while allowing any output. If you're hosting a webserver, you'll want to allow 80, 443 from anywhere (or just from your IP for testing).
  • "Availability zones" are basically different datacentres. If you have an EC2 instance A in eu-west-a, and an EC2 instance B in eu-west-b, and an RDS instance they both talk to in eu-west-c, that means all communications between those are across availability zones. Some things start costing (more) money if you go out of your availability zone with your traffic, always check rates.
  • "Elastic IPs" are basically a static IP that you can re-assign from one machine to another. You would allocate one, and use it in your front-end/whatever. Then when you want to switch from instance A (which is out of date or has gone down) to your instance B (which is up to date or up) you take the elastic IP from A (or shut down A) and assign it to B (which needs to not already have an elastic IP). I don't recommend you rely on them for anything and instead add a domain name to AWS, which gives you way more control at the cost of a few minutes/hours latency on DNS updates, but that's me thinking of larger scale.
  • Load balancers do exactly what it says on the tin. If you have a load balancer that has 3 instances on it, then any request has a chance to go to any of those 3 instances. If you have it set to sticky connections, then your first request has that chance, then the same session will keep going to the same instance. If you don't need sticky connections, then you can fake a load balancer with Route 53 (the domain name service) by making a DNS record in R53 with 'weighted' policy with 2 sessions.

Summary: If you're trying to learn ~~the cloud~~ then look into Lambda/ElasticBeanstalk/etc, but be aware your knowledge will literally only be useful for AWS ever. If you're just trying to learn server development at small scale, maybe don't use AWS. If you're trying to learn mid/large-scale development, then use AWS, but also pretend that you have a large number of servers everywhere and you can't afford downtime. Otherwise everything you learn will have to be changed or re-learned when you actually have such a system in production.

e: And if you just want to make your app in peace, go for a VPS provider, preferably a very very cheap one. You have providers that offer year-long machines with 1 modest CPU, 256 MB RAM and enough storage, all for £5/year or similar. You can also try DigitalOcean, which constantly have 'promotions' where you can sign up with a promocode and get more than one month's credit when you start up. Working with a VPS provider will also tell you all you need about small scale development, and the knowledge transfers to some degree (depending on provider) over to EC2 once you learn the lingo.

Red Mike fucked around with this message at 12:07 on Oct 8, 2016

Adbot
ADBOT LOVES YOU

Red Mike
Jul 11, 2011

xenilk posted:

- Is running a micro instance sounds like a viable option for a set of 5 sites that get probably <10,000 views daily total and probably less than a gb of database storage.

t2.micro should be more than enough, assuming you don't do incredibly expensive queries. If you run the CPU up to 100% constantly because you're running a query with 3 dependent subqueries for every one of those daily views, then no it's not going to work.

xenilk posted:

- I noticed that the network is marked as "LOW" for t2.micro instances, does that mean that it will be slow as hell?

That "network" thing just relates to your bandwidth limits (this is how many bytes you can push into/out of the instance in a single instant). It's also misleading across different classes (t2, m4, m3, etc) since t2.micro actually has slightly higher bandwidth limits compared to the cheapest m4.

xenilk posted:

- For EC2, instances have CPU Credits for bursts, is it the same for RDS? Does it mean my instance can go down for any reason? That would blow.

CPU credits don't mean instances going down for either EC2 or RDS. You get baseline performance unless you have cpu credits to spend in order to burst up to higher performance. When you're not using up all your resources, you recover cpu credits slowly. This means that if you only end up needing to burst for 50% of the day, you should be able to maintain it permanently.

Ideally, you should be ignoring CPU credits entirely and making sure that your baseline performance is enough to handle whatever you're throwing at the machine. CPU credits are dumb and will cause you to lock up your machine at 100% CPU in the middle of the night because oops suddenly your machine became quite a bit slower at peak times.

Unrelating to your questions, if you're not also moving to EC2, double check your traffic costs. Data transfer costs money unless your database is talking to an EC2 instance in the same availability zone using private IPs.

Red Mike
Jul 11, 2011

zerofunk posted:

Red Mike brings up a good point about additional costs due to data transfer as well. If you kept it all within AWS, you wouldn't have that issue. Amazon just announced a new VPS product called Lightsail that is supposed to be more akin to Digital Ocean's offering (I haven't read too much about it myself) than EC2. It may be worth looking into that if you did want to move everything over, but keep a similar setup aside from database hosting.

Lightsail is looking competitively priced, although setting up a bridge between AWS and it (so you can access your RDS instance) will mean you'll have to learn about AWS specific notions (which is a good thing if you're trying to learn; annoying if not).

It does however seem to be US-only for now. Otherwise you'll need EC2 instances which end up costing more money.

Red Mike
Jul 11, 2011
Assuming I'm understanding correctly, the setup I've generally seen (especially with Windows servers that take ages to start up if it's a custom AMI) is that the AMI never changes (and preferably is an Amazon one, so that instances are brought up from the waiting pool of pre-instanced servers) but Launch Configurations are used instead, with a userdata script being passed in that downloads and sets up everything as needed. Launch Configurations are instantly created and available, and there's no overhead on bringing instances online from them.

Only downside: you're limited to 300 or so configurations at any one time, and I don't believe you can increase the limit.

Red Mike
Jul 11, 2011
I can cover the SSL part, although be aware that a load balancer for this might be a bit overkill. It sounds more like you've got an issue/not enough resources on your instance. If scaling up is an option, consider that first.


Importing a third party certificate into the certificate manager will not let you renew it and it won't happen automatically. It's solely a way to get third party certs to be used on load balancers in the same way Amazon certificates are used.

You can use the certificate manager to get Amazon-issued certificates (that you can't download to use on your own) which will let you use them in stuff like the load balancer, their CDN, etc. Just go into the certificate manager and go through the process of getting a new certificate. You'll need to prove you 'own' the domain via a couple methods (DNS validation, email validation, etc, they're all explained there), or if you want to make it simpler in future just transfer it to Route 53. These certificates will renew on their own behind-the-scenes. You set it up once and just leave it (like Let's Encrypt when set up properly).

When you get that certificate, you can bind it to a load balancer, or a cloudfront zone, etc. The load balancer then serves HTTPS under an SSL certificate issued by Amazon, but you can set it up to talk to your instances over HTTP. That means the traffic between the load balancer and your instances is not encrypted, but is otherwise the same to an end-user.

Caution though: depending on how your app works, you may have to wrangle it into functioning properly. This is because it'll be serving requests off of port 80 over HTTP, however as far as the user's concerned it's running on HTTPS. I know plenty of web software that breaks down horribly under that setup (forums software, CMSes, etc) ending up with infinite redirects, or refusing requests because they're not secure.

Red Mike
Jul 11, 2011
That's a good point, although I would argue that you should weigh it against the ease of load balancing.

If it ends up being one of those bits of software that is difficult to load balance (or load balancer SSL terminate), and the cost is an extra $10/mo or something, you might consider it worth it to not spend weeks fixing things. Time is as much a resource as money.

Red Mike
Jul 11, 2011

Lumpy posted:

So if I do this and get a cert / SSL on the load balancer, if my front-end makes a call to https://myservice.blah.com the request from *and* response to the client will be encrypted, but the traffic from the load balancer to the ec2 instance will not? (this is fine, I just don't want to have any CORS issues because the front-end is https and doing AJAX to http will cause issues)

Correct. From the end-user (so from the browser) it's as if everything is HTTPS.

It's just on the app server itself that it knows it's not HTTPS, so it'll have to be configured to serve on port 80, non-secure (unless you want to have to get another certificate to encrypt app server <-> load balancer traffic which defeats the point).

Adbot
ADBOT LOVES YOU

Red Mike
Jul 11, 2011
You can leave whatever options you already have alone, unless it breaks later (in which case you need to figure out why it's breaking and won't be easy to explain in a forum post).

The only thing you need to make sure of is that you're listening over port 80, with no SSL certificate. You then tie the load balancer to the instance's port 80.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply