|
Random question, when the bill shows discounts and mentions savings plans discounts, is that the same figure that you'd see in cost explorer under the utilization report for savings plans as on-demand spend equivalent? The two numbers look almost exactly the same for me but there's a discrepancy of a few cents.
|
#
?
Dec 19, 2022 15:31
|
|
|
|
| # ? May 17, 2024 19:04 |
|
|
Blinkz0rz posted:The intent is kind of in the name, isn't it? Sounds a lot like PEBCAK imo. Sure, if there was a separate thing you could do that was just access, then yeah, but there isn't. That's more of a Slack problem than an implementation one, fwiw.
|
|
#
?
Dec 19, 2022 17:00
|
|
|
Our new baby project manager just tried to submit to the boss an architecture plan based almost entirely on ARM vms. Thankfully I intercepted and explained to the kid we are NOT going to move our infrastructure from a kubernetes cluster to a bunch of virtualboxes running on raspberry PIs (or however the gently caress those amazon arm ec2s work), and to seriously rethink the plan before the boss does a boss and fires him. EKS is good. EKS is nice. EKS works, lets not fix what isn't broken, and most importantly lets not abandon the grand plan of "all VMs must die", if it aint in a lamba, docker/k8 container, or in strict "we just wanna test it on a temporary vm" , it stays off my loving cluster. While I love my bosses commitment to a diverse workplace with lots of "give em a chance" (read "cheap") fresh out of uni hires, I feel like I'm the only one keeping half these kids from fireballing their own careers. (He just recently got himself a MacM1 and is pretty much "oh my god, ARMs are so fast!", and I kinda had to explain that actually Apple ARMs are fast, but nobody else has access to those yet, so the best you can hope for is RaspberryPi4/mobile-phone performance. duck monster fucked around with this message at 06:30 on Dec 21, 2022 |
|
#
?
Dec 21, 2022 06:25
|
|
|
Is this a bit? If not, have you heard about graviton?
|
|
#
?
Dec 21, 2022 06:35
|
|
|
duck monster posted:Our new baby project manager just tried to submit to the boss an architecture plan based almost entirely on ARM vms. Thankfully I intercepted and explained to the kid we are NOT going to move our infrastructure from a kubernetes cluster to a bunch of virtualboxes running on raspberry PIs (or however the gently caress those amazon arm ec2s work), and to seriously rethink the plan before the boss does a boss and fires him. EKS is good. EKS is nice. EKS works, lets not fix what isn't broken, and most importantly lets not abandon the grand plan of "all VMs must die", if it aint in a lamba, docker/k8 container, or in strict "we just wanna test it on a temporary vm" , it stays off my loving cluster. Can't you run EKS on ARM instances? If he's got such a boner for 'em figure that'd be an easy place to start and might offer some savings, depending on what that looks like.
|
|
#
?
Dec 21, 2022 06:36
|
|
|
You absolutely can, this feels like a bit
|
|
#
?
Dec 21, 2022 06:40
|
|
|
Yeah the Graviton chips are absolutely not “raspberry pi in the cloud” lol. They are great price and performance options if you don’t mind doing the work to get your workload on there. And you can use them as nodes in an EKS cluster. I feel like you may not be up to speed with what Amazon is doing with ARM. Agree that abandoning containers to return to fleets of bare EC2 nodes is probably a terrible idea though.
|
|
#
?
Dec 21, 2022 06:42
|
|
|
Oh ok. Consider me schooled then. I'll do some reading. But I'm still gonna veto moving EKS over till we've proven its are up to the task. edit: Appologised to baby project manager. Told him, lets just stick with intel for now but setup an experiment to see how it works. The main issue is likely more that our CI might take some tweaking to build docker containers that wont explode if we ask it to build ARM Does anyone know how the Gravitons handle heavy Java processing loads. We have a crusty old JVM based geospatial app (doesnt use GPU style compute, but we might be running some ML loads soon for handling Sentinal imagery) and that might be the easiest saving, cos that thing burns white hot sometimes and I'm not convinced it really does anything that couldnt benefit from "just do it cheaper" duck monster fucked around with this message at 06:58 on Dec 21, 2022 |
|
#
?
Dec 21, 2022 06:50
|
|
|
Graviton is good tho.duck monster posted:Told him, lets just stick with intel for now but setup an experiment to see how it works. The main issue is likely more that our CI might take some tweaking to build docker containers that wont explode if we ask it to build ARM Watch out for the difference in integer processing vs float processing performance for graviton vs Intel, which may or may not be a thing that's relevant to your apps. Do you even need to do anything special at all for building your java app for arm? I guess if you have any c libraries, but I actually can't think of any off hand that would make sense for a java app. If my company wasn't balls deep in investing in our own bare metal datacenter presence, I'd be spearheading my own initiative to introduce graviton as a cost savings initiative. Methanar fucked around with this message at 07:24 on Dec 21, 2022 |
|
#
?
Dec 21, 2022 07:16
|
|
|
Methanar posted:Graviton is good tho. Kafka and Elastic search are a high probability. We are currently using OCS's OSI which is a loving horrible timeseries db (decision made a long time ago before I started) that fails one the basic premise of timeseries databases of "usecond latency" for writes. From same bad prehistoric time we have a custom built ETL setup that is basically hot trash and we're speccing out a replacement based around a custom go engine [its too weird a workload for the off the shelf solutions] and kafka with queues in and out w something running all our weird maths pouring all the finished product into elastic
|
|
#
?
Dec 21, 2022 07:24
|
|
|
duck monster posted:Kafka and Elastic search are a high probability. We are currently using OCS's OSI which is a loving horrible timeseries db (decision made a long time ago before I started) that fails one the basic premise of timeseries databases of "usecond latency" for writes. From same bad prehistoric time we have a custom built ETL setup that is basically hot trash and we're speccing out a replacement based around a custom go engine [its too weird a workload for the off the shelf solutions] and kafka with queues in and out w something running all our weird maths pouring all the finished product into elastic I have no idea what you're doing, but I'm fairly confident that's not how I'd build my tech stack. usecond write latency is a big ask, but cassandra would be my completely uninformed recommendation as a replacement for your weirdo time series DB I've never heard of. Clickhouse might be worth looking into as well. Even if you don't care, you should still watch how Cassandra works as an inspiration of how to build a good write-path. https://www.youtube.com/watch?v=B_HTdrTgGNs Methanar fucked around with this message at 07:49 on Dec 21, 2022 |
|
#
?
Dec 21, 2022 07:46
|
|
|
 We switched to grav as soon as possible, been cheaper and with grav2 it’s faster too. For our loads anyway, it’s good op. Drop-in if you’re running go or python, honest.
|
|
#
?
Dec 21, 2022 11:45
|
|
|
Methanar posted:I have no idea what you're doing, but I'm fairly confident that's not how I'd build my tech stack. C* also uses the Dynamo System so reading that white paper is useful as well if you haven't already. We just moved off of C* to DDB. C* is a giant pain in the rear end to self manage and requires a dedicated DBA and if you think it doesn't you're going to be in for sad times and your devops engineer is going to hate you. AWS managed C* is probably fine I would highly discourage you from rolling your own on EC2 though.
|
|
#
?
Dec 21, 2022 15:54
|
|
|
Also worth noting that you can run a lot of AWS managed services on graviton like RDS, elasticsearch, elasticache. And there’s no reason not to use them really since they are cheaper and Amazon handles all the compatibility work.
|
|
#
?
Dec 21, 2022 16:28
|
|
|
duck monster posted:
Graviton is outstanding for pure Java workloads. About the only place where it really struggles is native code where someone has hand-written a vectorized path with x86 intrinsics and the fallback alternative is dog slow. If you want to dive deep on this: https://chipsandcheese.com/2022/05/29/graviton-3-first-impressions/ AWS's ARM chips are absolutely targeting Intel and AMD's best. It's not just an "it's worse, but a lot cheaper", they really want to deliver similar performance for less cost. Apples ARM chips are loving enormous and blow away anything that Intel and AMD have, but they're never in a million years going to sell it in a server so it's moot. Apples chips would also be too expensive, they're big. Twerk from Home fucked around with this message at 16:53 on Dec 21, 2022 |
|
#
?
Dec 21, 2022 16:48
|
|
|
on the plus side you've given that kid an important career lesson in why people tune out the grumpy old dude
|
|
#
?
Dec 23, 2022 14:31
|
|
|
Tune out the grumpy people on most issues unless it's something that keeps lasting for years about the same thing. Signed, - graybeard that used punch cards trapped in a grumpy geriatric millennial body
|
|
#
?
Dec 23, 2022 16:01
|
|
|
What is the best course for the 2022 Solutions Architect Professional Exam? I previously used ACloud-Guru but it appears their course is not updated yet and is still for the 2020 version of the certification. Plan to get practice exam from Whiz Labs but would like a course as well.
|
|
#
?
Jan 11, 2023 20:46
|
|
|
Scrapez posted:What is the best course for the 2022 Solutions Architect Professional Exam? I previously used ACloud-Guru but it appears their course is not updated yet and is still for the 2020 version of the certification. I haven't sat for the SA Pro yet but I found Adrian Cantrill's courses to be top notch for the SA Associate. I'd be confident that his SA Pro is just as good. He also puts out some free stuff on his YouTube channel if you want to get a feel for his teaching style. I really wish he'd put one together for Certified Database and/or Certified Data Analytics because there seems to be a huge gap of courses/training for those.
|
|
#
?
Jan 11, 2023 22:01
|
|
|
so one of the dashboards i was tasked with creating in quicksight was supposed to consist of lots of scatter plots with regressions and what not. turns out, quicksight can't do that. HOW THE HELL DO YOU PROVIDE SCATTER PLOT VISUALS AND NOT ACCOMPANY THEM WITH REGRESSION OPTIONS?!?!?! i am utterly loving baffled by this. the biggest reason i can think of for using a scatter plot is to plot a regression on it. hell, i even talked to our rep who connected me with one of the lead engineers for qs. you just...can't do this. at all. apparently i have to either a) calculate the (x,y) for whatever regression i want in a table and then plot that on top of the scatter plot via reference lines, i think he said, (utterly ridiculous) or b) do all of this through sagemaker, which i know nothing about. loving gently caress. unreal.
|
|
#
?
Jan 11, 2023 22:12
|
|
|
abelwingnut posted:quicksight sadness... I heard that QuickSight was the neglected child of AWS services and your discovery certainly doesn't dispute that... Rookie question: I believe it's best practice to keep your database in a private subnet and not exposed to the public internet. If I have a mobile app out in the wild, what is the standard architecture to read/write the RDS database that is residing in a private subnet?
|
|
#
?
Jan 11, 2023 22:30
|
|
|
Hughmoris posted:I haven't sat for the SA Pro yet but I found Adrian Cantrill's courses to be top notch for the SA Associate. I'd be confident that his SA Pro is just as good. He also puts out some free stuff on his YouTube channel if you want to get a feel for his teaching style. Great. I'll check that out. Thank you.
|
|
#
?
Jan 11, 2023 23:26
|
|
|
Hughmoris posted:I really wish he'd put one together for Certified Database and/or Certified Data Analytics because there seems to be a huge gap of courses/training for those. The practice exams on ACloudGuru are great for the Database Specialty exam. Without any prep I took four practice exams and dug deep into questions I got wrong. After the fourth I took the live exam and passed. YMMV
|
|
#
?
Jan 12, 2023 01:02
|
|
|
Agrikk posted:The practice exams on ACloudGuru are great for the Database Specialty exam. Without any prep I took four practice exams and dug deep into questions I got wrong. After the fourth I took the live exam and passed. Yeah but you've got so much experience with this stuff it might as well be called Agrikk's Web Services. I'll check those out, thanks.
|
|
#
?
Jan 12, 2023 01:10
|
|
|
Hughmoris posted:I heard that QuickSight was the neglected child of AWS services and your discovery certainly doesn't dispute that... Wouldn’t you use an api to access the db rather than access the db directly from the app? Regardless of whether the db was on a private subnet or available directly over the internet
|
|
#
?
Jan 12, 2023 05:29
|
|
|
Yeah. You use a two or three tier architecture. Here's a random search result
|
|
#
?
Jan 12, 2023 05:44
|
|
|
Hughmoris posted:I heard that QuickSight was the neglected child of AWS services and your discovery certainly doesn't dispute that... I've only used QuickSight to deploy the CUDOS cost dashboards but it sucks enough to make me wonder why it exists. Are there real people who evaluate it next to any other BI product and go "gently caress yeah we gotta have this"? It's fairly cheap I guess if you have minimal requirements. beuges posted:Wouldn’t you use an api to access the db rather than access the db directly from the app? Regardless of whether the db was on a private subnet or available directly over the internet Yes, please in the name of all that is good, do not write an app that opens direct, synchronous connections from a phone to a database. DB's are not designed for high latency, slow, unreliable links or huge numbers of simultaneous connections. It's also an unnecessary security risk putting it directly on the internet. You want a a backend application that your mobile app talks to over HTTPS, which in turn queries the DB and returns the results. Speaking very broadly you want to write a lightweight frontend that sends requests and renders/reacts to the results, and put as much business logic (I'll include "dealing with behind the scenes infrastructure bullshit like databases" here) as possible on the remote backend. This also gives you more freedom to change stuff about your app without waiting and praying for users to update it on their devices.
|
|
#
?
Jan 12, 2023 07:41
|
|
|
yea, even before my discovery about quicksight’s inability to analyze visuals i was having tons of issues. - the whole ‘calculated fields’ element is terrible and requires you to use their own excel-like syntax for formulas, which would be fine if it weren’t so limited. whereas excel has, what, thousands of functions, and even google sheets has hundreds if not thousands, qs has maybe...100? very constraining having to chain three or four functions to do the job of a basic one in other similar tools. - oh, and to the above, why not allow me to use sql to create a calculated field?!?!!?!?!?!?! this would be SO much easier. - contributing to the overall claustrophobia, there are only like 30 visual styles. that’s more than enough for me, but i have had to compromise some ideas to fit with what’s available. - as i hinted at in my original post, zero analytical capabilities. it can plot fine, but not give you anything to help interpret those plots. utterly insane. - bucketing isn’t possible for a lot of visuals. you have to create a calculated field to say a row falls in a certain bucket, then plot that calculated field vs whatever. very tedious. - it just looks rickety and barebones and clumsy. it’s half a product, at best. e: like, our company's huge project now is going from an infrastructure of snowflake/rds/superset/airflow to an all-inclusive aws package. we've got a contractor working on all of that. and qs is supposed to replace superset as the dashboarding tool. but by god, as bad as superset is, it can at least allow me to analyze data. christ. does anyone know of any free bi tools i might be able to employ given this move? we're only going to get more analytical and i'll be damned if i'm going to calculate INDIVIDUAL loving COORDINATES FOR EACH INDIVIDUAL REGRESSION I WANT IN SQL in order to plot a loving regression on top of a scatter plot. that doesn't just tiptoe over my line, that loving trampolines over it. abelwingnut fucked around with this message at 16:03 on Jan 12, 2023 |
|
#
?
Jan 12, 2023 15:18
|
|
|
My company is going ape poo poo for QuickSight right now. They're investing substantial time in cost/billing dashboards for all of our teams in an effort to cut down costs. You know as opposed to rethinking our lift and shift of data centers to the cloud while doing literally nothing else then complaining about the costs when their oversized SQL server instances cost a ton. Jesus this company... Speaking of! I got tasked with tracking down Marketplace subscriptions, whose subscribing, why are they doing that, etc. And for the life of me I dont see any marketplace API calls in our CloudTrail. Looking at the reference I think I need to be searching for aws-marketplace:Subscribe but I cant find anything marketplace related. I verified our org CloudTrail doesnt have any exclusions. Are these API calls some weird one off that doesnt get written to CloudTrail or something?
|
|
#
?
Jan 12, 2023 15:30
|
|
|
Scrapez posted:What is the best course for the 2022 Solutions Architect Professional Exam? I previously used ACloud-Guru but it appears their course is not updated yet and is still for the 2020 version of the certification. I recently assed mine on Stephane Maareks course on udemy. Also bought the test exams from Jon Bonso which are really good. Cantrills content looks awesome as well. It’s more than just what you need to pass for the exam though. If you’re in a hurry go for Maareks course, if you have time Cantrill will probably teach you more.
|
|
#
?
Jan 12, 2023 15:51
|
|
|
ACG is still worth it for their sandboxes, even if you end paying for other courses. I find it super helpful when Im trying to learn a new service.
|
|
#
?
Jan 12, 2023 15:58
|
|
|
Docjowles posted:Yes, please in the name of all that is good, do not write an app that opens direct, synchronous connections from a phone to a database. DB's are not designed for high latency, slow, unreliable links or huge numbers of simultaneous connections. It's also an unnecessary security risk putting it directly on the internet. You want a a backend application that your mobile app talks to over HTTPS, which in turn queries the DB and returns the results. Speaking very broadly you want to write a lightweight frontend that sends requests and renders/reacts to the results, and put as much business logic (I'll include "dealing with behind the scenes infrastructure bullshit like databases" here) as possible on the remote backend. This also gives you more freedom to change stuff about your app without waiting and praying for users to update it on their devices. Yes, my name is Bobby Tables. Why do you ask?
|
|
#
?
Jan 12, 2023 17:03
|
|
|
Docjowles posted:I've only used QuickSight to deploy the CUDOS cost dashboards but it sucks enough to make me wonder why it exists. Are there real people who evaluate it next to any other BI product and go "gently caress yeah we gotta have this"? It's fairly cheap I guess if you have minimal requirements. I think this is a thing that people going through simple tutorials go 'why is this here it's just some weird added extra layer between my app and the data, like the American health insurance industry! I'll just skip it!' because in a simple tutorial your frontend is just like straight up passing stuff to the DB and isn't doing much.
|
|
#
?
Jan 12, 2023 18:37
|
|
|
Hey chaps. Apologies if this sort of thing is posted a lot, I didn't see it covered in the OP. I recently applied to a fantastic looking Junior Cloud Engineer role here in the UK, which requires no experience so I'm hoping I've got a chance, coming from the Architecture industry looking for a career change. I am going through AWS's 'Cloud Practioner Essentials', I am wondering if anyone knows any more hands-on learning resources for a beginner so I can try and put things into real world practice. ACloudGuru looks great but I don't have the money for that currently. I believe the role would involve AWS and Microsoft Azure. It's a government-adjacent position and from what I've looked up about it, would likely involve migrating over a large amount of legacy apps to these cloud services. Thanks in advance for any advice!
|
|
#
?
Jan 15, 2023 16:32
|
|
|
get free trial accounts and run through https://learntocloud.guide
|
|
#
?
Jan 15, 2023 18:54
|
|
|
The Fool posted:get free trial accounts and run through https://learntocloud.guide First thing you do should be https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/monitor_estimated_charges_with_cloudwatch.html
|
|
#
?
Jan 15, 2023 21:28
|
|
|
Fantastic thank you both. I've also noted that previously posted Learn Cantril also has video walkthroughs for hands on learning. Thanks again!
|
|
#
?
Jan 16, 2023 01:05
|
|
|
Today I learned that if you try to cfn-lint a YAML template that has !!Ref in it (instead of the correct !Ref) everything blows up in spectacular fashion. Since I'm running python on Windows and not particularly good at it, this took me an hour to resolve and included multiple reboots, reinstalls, path changes, etc.
|
|
#
?
Jan 18, 2023 18:44
|
|
|
Learning what failure conditions look like what is part of the joy of AWS because they definitely don't make it obvious. I worked there and ran into issues all the time with init scripts not working and me thinking the issue was with the cfn file itself.
|
|
#
?
Jan 22, 2023 05:12
|
|
|
|
| # ? May 17, 2024 19:04 |
|
|
First time poster ITT, this is both an aws question and noSQL question, let me know if not appropriate. Our dynamo design, our PK is an id unique to our data, it's a carriers database, the carrier_id. The SK is the epoch second but we overload it and sometimes store other strings in it for different types of data. But always the int carrier_id for the PK. We have sometimes run into collisions where an event happened the same second for the same carrier. Some may be dupes in our source data (MySQL) but some could be legit same second different data. How do developers handle something like this? I have some ideas but I want to ask the experts. Our source data doesn't hold milliseconds. Some data my boss wants to use a particular datetime field that frequently breaks down to a even times like :00 or :30 minute for the timestamp so this will be even more of an issue going forward.
|
|
#
?
Jan 27, 2023 14:28
|
|
