|
I've done SQL installed on a VM before where it was hosting a really badly written app that the developer weren't interested in making any changes to in order for it to be compatible with database products. If it's something being developed in-house then you need to make it in a way that is compatible with RDS etc.
|
# ? Jun 14, 2022 19:34 |
|
|
# ? Jun 5, 2024 03:25 |
|
Just-In-Timeberlake posted:Go to CloudFront > Functions For some reason this didn't work, but I think I found something. While using the website origin you can't use OAI but you can add a custom header. I added a referer header and an S3 bucket policy to only allow access with that header, I'll put in a massive randomly generated key to effectively lock down access to only the distribution. Does anyone see a flaw in this? Normally the referer header specifies the website it's coming from and can be easily spoofed but in this case it seems like it should have a similar effect to requiring an SSH key or something. Right?
|
# ? Jun 14, 2022 19:36 |
|
There's a bunch of off the shelf software companies that won't work with / support anything other than a database on a VM.
|
# ? Jun 14, 2022 19:37 |
|
It's also cheaper to run bare EC2 instances than run RDS. But this argument falls apart pretty quickly if you put any value at all on your own time spent managing those instances, because operating databases loving sucks. And while it's true that certain things work differently or are disallowed in RDS, the vast majority of apps will be just fine. As you branch out from standard mysql/postgres/mssql in RDS to Aurora, or Aurora serverless, etc. then the number of caveats grow. Docjowles fucked around with this message at 21:31 on Jun 14, 2022 |
# ? Jun 14, 2022 21:27 |
|
If you're a megacorp with a sweet private licensing agreement with Microsoft
|
# ? Jun 14, 2022 21:47 |
|
Happiness Commando posted:If you're a megacorp with a sweet private licensing agreement with Microsoft We're ending SA on a bunch of stuff and teams are scrambling to migrate from MSSQL run on-prem or on azure vm's to cosmos or azure sql It's fun to watch, super glad I'm not in the trenches for any of those teams
|
# ? Jun 14, 2022 21:51 |
|
Ending SA shouldn't automatically result in having to scramble to do anything, it just means they have to hold on the release they're running
|
# ? Jun 14, 2022 22:51 |
|
system administration? software automation? ...something awful?
|
# ? Jun 14, 2022 23:39 |
|
yes. (software assurance)
|
# ? Jun 14, 2022 23:44 |
|
kalel posted:Out of curiosity, would there ever be a reason to use an EC2 with a database image instead of RDS? I have some microservices inside fargate tasks which connect to a MySQL RDS (or at least, I'm trying to and am currently in the process of debugging). My understanding is that the typical industry standard way to manage a database is through RDS for scalability and convenience, but is there ever a motivation not to do that? There are plenty of reasons, most of them edge cases. As mentioned above, the most common reason is regarding RDS MSSQL and it's cost per hour of operation (but like mentioned upthread, the cost is purely for it's operation and does not take into account maintenance operations, cost of management, etc). Backup/restore in the classic sense is wonky if you don't use native AWS capabilities, and using special procedures and functions can be tricky. For RDS you still need an EC2 instance if you are doing any kind of ETL (if you aren't using native AWS ETL services and SSIS isn't available as a SaaS offering). But mostly it's that some folks can't give up control and delight/need to have access "under the hood". These same folks still name their servers after planets/cartoon characters/SciFi references and have obsessively nursed them along for years. I have two MSSQL database clusters running on EC2 and they run great and are very performant (especially with FSx ONTAP!). But I also have a couple of Aurora MySQL and Aurora Postgres instances that I haven't touched in ages and couldn't imagine running those workloads on an EC2 instance.
|
# ? Jun 15, 2022 01:42 |
|
In our case (also MSSQL) we needed to fine tune storage performance without breaking the bank. Tempdb on ephemeral storage, databases on their own disks because their performance requirements were predictable, stuff like that. And at the time AOAG wasn't properly supported on RDS either.
|
# ? Jun 15, 2022 02:19 |
|
There's plenty of other reasons usually having to do with legacy or wanting to embed the database under full control of an application's database (you're creating your own janky version of RDS, for example). When we talk about microservices it really just means "take care of your own hard dependencies like databases and keep your surface area judiciously small." Things start to get awkward when you provision a new database as part of your application v an application is started and presumed to have a database already provisioned for it because these are very different workflows with vastly different sets of requirements and permissions for said application. IMO, if one is cost sensitive much at all, has relatively small scale (a scale where people can get by doing classic sysadmin and still be business viable), growth is not super duper fast, and doesn't have to run stuff in AWS due to customer requirements sticking with a colo and being careful about which cloud services to use is plenty good enough. At previous companies that sucked donkey balls at scale and didn't really need to the systems we always had the least trouble with in terms of time spent in meetings were our bare metal hosted machines. Of course they couldn't scale anything up or down, but we blew so much time on cloud services like AWS and Azure that we kept having political problems over in our dysfunctions that it sunk the company and wasted more money for no good reason, which only hastens the inevitable demise of the company while enriching the hyperscalers. Absolutely go with Big Cloud if you know you need them though and have the budget to afford several people like the pros in the thread at $250k+ total comp in the US. Most businesses doing cloud anything are basically doing it for FOMO and / or can't retain people without having cloud in their job descriptions. I say this as a former hiring manager some years ago.
|
# ? Jun 15, 2022 12:34 |
|
“I have three domain controllers sitting in three regions on AWS. Two database servers and a file server in my corp VPC. I have four web servers and two app servers (both in fixed pools )and an RDS MSSQL instance in My Prod VPC. How come it’s sooo expensive? The cloud was supposed to SaVe Me mOnEy!”
|
# ? Jun 16, 2022 17:37 |
|
Agrikk posted:“I have three domain controllers sitting in three regions on AWS. Two database servers and a file server in my corp VPC. I have four web servers and two app servers (both in fixed pools )and an RDS MSSQL instance in My Prod VPC. How come it’s sooo expensive? The cloud was supposed to SaVe Me mOnEy!” Oh god this is my company to a T. I cant wait to hit a year and bounce so I dont have to pay back my sign on bonus. Why we as a 99% windows shop decided to go with AWS instead of Azure, and then just do a lift of shift of everything is beyond me.
|
# ? Jun 20, 2022 14:04 |
|
BaseballPCHiker posted:Oh god this is my company to a T. I cant wait to hit a year and bounce so I dont have to pay back my sign on bonus. Management: it’s ok we just need to get the move done asap so we can terminate our data center lease and avoid paying for the next hardware refresh. We’ll optimize later Also management: what the gently caress do you mean you want another 9 months to rewrite the app to use s3 and containers vs oversized EC2 instances with a petabyte of high iops EFS volumes. We just spent a year moving to the cloud it’s fine
|
# ? Jun 20, 2022 14:45 |
|
Mods, please enforce trigger warnings kthx
|
# ? Jun 21, 2022 20:27 |
|
Docjowles posted:Management: it’s ok we just need to get the move done asap so we can My yearly metrics just said A cloud...not WHICH cloud.
|
# ? Jun 22, 2022 14:50 |
|
welcome to the oracle cloud
|
# ? Jun 22, 2022 15:38 |
|
Reasonably new to cloud native design stuff - wondering about a proposed setup. We have a tool that's going to be used sporadically (during certain types of events only) - we wanted to design it as a webapp as a reasonably straightforward DDB backend, with the frontend basically delivered as a static file that calls to various lambdas for dynamic content, using Flask on Lambda to handle calls to the DDB backend for reads/writes/etc. Is this setup going to be able to deliver anything near acceptable performance, or is the startup/ephemeral nature of lambdas going to be a problem? My fallback plan is going to Fargate or something for a container, or otherwise just setting up some sort of containerized server. Falcon2001 fucked around with this message at 19:17 on Jun 22, 2022 |
# ? Jun 22, 2022 18:27 |
|
it's hard to say without benchmarking, especially if your flask app does a lot of lazy loading. my intuition is that most of your fears can be allayed by lambda's provisioned concurrency feature, which basically prewarms a bunch of executors for you. since you'd be running in flask on lambda, it shouldn't be too complicated to switch towards flask in a container later, if you find that it's not working
|
# ? Jun 22, 2022 20:41 |
|
Falcon2001 posted:We have a tool that's going to be used sporadically Sometimes the obvious solution can also be the cheapest in the right circumstances. If you have times when the tool is active and times when it isn’t used at all, you can save a lot of money and complexity by building a container or even an EC2 instance that points to your back end. Then write a script that turns everything on when you need it, and another script for turning everything off when you are done. Of course if your utilization is not zero during the downtimes then this won’t work, obviously.
|
# ? Jun 22, 2022 21:29 |
|
Falcon2001 posted:Is this setup going to be able to deliver anything near acceptable performance, or is the startup/ephemeral nature of lambdas going to be a problem? It will likely work fine. I’ve built plenty of internal apps on lambda. You havent defined what acceptable performance means, but a python cold start should be short. Well under a second. (idk about Flask, but the answer is just: try it) Once a lambda instance is warmed, it sticks around for awhile and serves quickly
|
# ? Jun 22, 2022 22:32 |
|
Agrikk posted:Sometimes the obvious solution can also be the cheapest in the right circumstances. If you have times when the tool is active and times when it isn’t used at all, you can save a lot of money and complexity by building a container or even an EC2 instance that points to your back end. In this case, the problem would be that it's a service that needs to work very quickly when we need it to - basically for realtime response stuff, otherwise I'd agree that's a pretty good approach. For the others, it sounds like the idea is at least sane enough to get up to the 'testing' phase. 'Acceptable performance' mostly meant 'Is the latency going to be high enough that a user would find the delay irritating' and it doesn't sound like there's a significant problem here.
|
# ? Jun 22, 2022 23:34 |
|
crazypenguin posted:It will likely work fine. I’ve built plenty of internal apps on lambda. Just schedule a keep warm function for every 4 minutes or so
|
# ? Jun 23, 2022 01:30 |
|
Falcon2001 posted:In this case, the problem would be that it's a service that needs to work very quickly when we need it to - basically for realtime response stuff, otherwise I'd agree that's a pretty good approach. I just tested a Django site served by Lambda that’s got good backend features (DRF, some analytics) and a cheap RDS instance behind it but almost no content on the page I loaded. GTmetrix gave it a 100% A on us-east-1 from Vancouver. So Lambda will not be what makes it slow.
|
# ? Jun 23, 2022 01:35 |
|
its not engineering if you're not using numbers "sporadic" doesn't mean anything. neither does "acceptable".
|
# ? Jun 23, 2022 03:25 |
|
accessing my load balancer's DNS yields a 503 service unavailable error. I have an ECS cluster, my fargate containers are "running," I have health checks defined... so why is my target group not allowing me to register any targets. why are there no available instances in the resource list. wtf is going on fuuuck
|
# ? Jun 23, 2022 04:14 |
|
aws sucks, dev ops sucks. Lol
|
# ? Jun 23, 2022 04:15 |
|
MightyBigMinus posted:its not engineering if you're not using numbers This is totally fair, but I'm also describing things that I don't have clear measurements on. This is a tool used during response to certain types of incidents; based on past experience I expect to use it 6-12 times a year for a few days at a time, but you can't predict incidents, so I'm trying to make sure it's at least somewhat scalable. By 'acceptable', I'm trying to ask 'is this going to deliver a level of responsiveness that humans won't find to be weirdly slow or stilted'; I don't have a lot of frontend experience so web user experience stuff is something I don't have a ton of experience on yet, so I'm not sure what the right term would be. CarForumPoster's response covers it though; basically I just wanted to check if my entire design was fundamentally flawed from the get-go, and it doesn't really sound like it.
|
# ? Jun 23, 2022 04:27 |
|
kalel posted:aws sucks Drake nah kalel posted:dev ops sucks. Lol Drake yeah
|
# ? Jun 23, 2022 05:53 |
|
I'm trying to get our ECS containers tagged so we can see what particular task definitions/services are costing us the most. The problem is that when the services were set up while they had AWS-managed tags enabled they weren't set to make the tasks inherit tags from the service or task definition. Is there a way to do this after the fact? E: Found it, I can do it through the CLI. Except now I have to wait for our security guy to give me access to create an access key for myself. 22 Eargesplitten fucked around with this message at 20:21 on Jun 29, 2022 |
# ? Jun 29, 2022 19:38 |
|
Cloudshell sib, if your dash creds do it, you can do it. CLI’s installed, you also get a small per-region persistent store.
|
# ? Jun 30, 2022 07:41 |
|
Has anyone used Azure for IoT applications, like sensors uploading recordings? How is it, especially compared to AWS? My team has some applications that mainly use AWS for managing our sensors. The company that owns my division (bought us a year after we started on our application) favors Microsoft and Azure for everything, for no particular reason. I'm worried that bringing everything under one umbrella will be a priority at one point, and I don't have any feel for how hard that will be, but my guess is a big pain.
|
# ? Jul 11, 2022 01:08 |
|
You need to elaborate more. Sensors like you’re reading camera data?
|
# ? Jul 11, 2022 01:28 |
|
Uh... sensors mean a lot of different things across different industries. Should probably clarify that in terms of kinds of data collected, bandwidth per sensor, etc.
|
# ? Jul 11, 2022 03:26 |
|
Yeah, sorry, it's not video but it takes moderately high rate measurements, maybe 240-1000 kb/s, but the device is used for engineering tests, not continuous monitoring, so we upload recordings of a few seconds to a few minutes up to S3, process it into some characteristics we put into the database, and let the user download it if they want. I feel like the big questions might be how good is Azures IoT support, and how much would everything need to restructure if we're DynamoDB based now and have to move to Cosmos or whatever Azure has. Also, sorry if Azure is too off topic. I thought this was a general Web Service thread. It doesn't look like there's an Azure thread, so maybe that says something about Azure vs AWS?
|
# ? Jul 11, 2022 03:41 |
|
StumblyWumbly posted:Yeah, sorry, it's not video but it takes moderately high rate measurements, maybe 240-1000 kb/s, but the device is used for engineering tests, not continuous monitoring, so we upload recordings of a few seconds to a few minutes up to S3, process it into some characteristics we put into the database, and let the user download it if they want. There's a bunch of Azure guys around, we also post in the other IT threads. I'm not doing anything with Azures IoT stuff, but I have soooo much cosmos db in my environment.
|
# ? Jul 11, 2022 15:25 |
|
I've been assigned to make some kind of short presentation and KB article for my new job about an AWS topic that would be useful for our developers. What topic do you all wish that developers understood better in AWS? We use a mix of EC2 VMs and Fargate EC2 containers, I'm thinking maybe something about how auto-scaling works so that if developers make something that causes a huge resource-intensive spike they can understand why everything moves slow for a little bit until autoscaling catches up.
|
# ? Jul 11, 2022 20:42 |
|
22 Eargesplitten posted:I've been assigned to make some kind of short presentation and KB article for my new job about an AWS topic that would be useful for our developers. What topic do you all wish that developers understood better in AWS? We use a mix of EC2 VMs and Fargate EC2 containers, I'm thinking maybe something about how auto-scaling works so that if developers make something that causes a huge resource-intensive spike they can understand why everything moves slow for a little bit until autoscaling catches up. the devs I support are all hilariously bad at secret management
|
# ? Jul 11, 2022 20:44 |
|
|
# ? Jun 5, 2024 03:25 |
|
For my money it’s some very basic info on how networking and/or dns work. I’ve had some absolutely conversations with senior devs where you suddenly realize that what they’re trying to explain only makes sense if their world view about how two computers communicate is totally and fundamentally broken.
|
# ? Jul 12, 2022 01:11 |