|
bob dobbs is dead posted:80% is real optimistic for when orms Just Work, if you're gonna give the standard orm spiel, too ORMs really shine in toy projects, but I've definitely seen a lot of projects fail because of their ORM - they either ended up with performance issues they couldn't figure out or spent longer fixing the performance than it would've taken to just do everything manually. Micro-orm might be a little more work up front but it's much easier to get it to perform and finding database people is a lot easier than finding ORM whisperers.
|
#
?
Sep 25, 2020 03:38
|
|
|
|
| # ? May 27, 2024 02:23 |
|
|
What does failure because of poor ORM performance end up actually looking like? Am I just in a company that tolerates awful performance and excessive spending? We have real data access problems, with inefficient queries, frequently doing application-level joins and more, but people are happy with 4+ second latency on requests and just doubling the size of database instances whenever there's a load issue.
|
|
#
?
Sep 25, 2020 03:44
|
|
|
You just described it. In the worst case it can kill the ability to scale horizontally. Even when the ability to scale isn't killed (like when there's a reasonable sharding strategy) you might end up with smaller shards than you want, worse perf/us, higher cost, etc.
|
|
#
?
Sep 25, 2020 03:46
|
|
|
Why yes I worked at a monolithic rails shop, why do you ask?
|
|
#
?
Sep 25, 2020 03:47
|
|
|
Munkeymon posted:Looking for feedback on a technical interview question for a senior full stack developer. I'm not smart when it comes to semi-complex SQL queries. I can get this to work for a given OrderId. What's the query for getting the latest status name of all orders?
|
|
#
?
Sep 25, 2020 04:15
|
|
|
Achmed Jones posted:Why yes I worked at a monolithic rails shop, why do you ask? This We did report differencing of tens of thousands of items, across hundreds of units, reports would take 8-10 hours to run, to the point that we made the monolith run in report runner worker mode, on a separate machine and that client had an 8 core on prem VM that existed just to run reports otherwise the differencing engine would run too slowly, in addition to the primary server, and we ran a replica db just for the reports. The app is not any more complex than it sounds, it's just that we hired 60 interns to write it in 8 months, and it ran like poo poo, and then we piled a bunch of tightly coupled features on top of it and it was less work to make the customer buy more cpu than it was to fix the product Hadlock fucked around with this message at 05:50 on Sep 25, 2020 |
|
#
?
Sep 25, 2020 05:44
|
|
|
Atma_Weapon posted:I'm not smart when it comes to semi-complex SQL queries. I can get this to work for a given OrderId. What's the query for getting the latest status name of all orders? code:
|
|
#
?
Sep 25, 2020 06:03
|
|
|
I have an interview with a FAANG company and another with an almost-FAANG company, both for remote roles. I'm practicing on leetcode but I'm so incredibly rusty at this algorithms stuff. And I have no idea what to answer systems design questions. What can I do in the next two weeks or so to try and maybe not totally bomb both interviews?
|
|
#
?
Sep 25, 2020 13:09
|
|
|
Doghouse posted:I have an interview with a FAANG company and another with an almost-FAANG company, both for remote roles. I'm practicing on leetcode but I'm so incredibly rusty at this algorithms stuff. And I have no idea what to answer systems design questions. Grinding leetcode is probably the best practice for algo code interviews. Do easy/medium and maybe the high-acceptance rate hards, don't spend too long on it if you're stumped, read the submitted solutions and make sure you understand exactly why it works. Rinse, repeat. (Not joking: while you're doing a problem talk out loud about what / why you're writing. That's what we want to hear from a candidate in our coding exercises, even if they don't get a solution.) For system design, I really like this book: https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321. Talks about common design decisions, backgrounds, scaling problems and solutions in distributed systems. A quick read through should get enough background to see problem spots in system designs and enough bullshittery to be able to discuss it.
|
|
#
?
Sep 25, 2020 13:44
|
|
|
minato posted:The SQL fiddle example seems overly complex: Agreed (with you and the other people pointing it out - I was just slapping something together and that was lazy+bad of me. quote:- The full solution can be done a few ways, but some of them are RDBMS-specific. In Postgres you might use a window function, but older versions of MySQL don't have that. So you can't test that here. Good point! I hadn't considered they might want to change the DB engine - making the schema simple enough to work on PG, Oracle or  MySQL would be even better than just cleaning up the setup script.quote:A FSE position requires broad but not necessarily deep knowledge. What signal do you want to get from this question? I would bet that the above question as originally stated will take ~15-20 mins for an average candidate before you have to move on, and it only tests a specific aspect of SQL. Maybe if you made a simpler question, you'd get the signal "yeah, they know SQL enough to know what a JOIN is" within 5 minutes so you can move on to asking them about the many other things they'll need to know as an FSE. I'm looking for basic knowing how a join works and hoping they have a grasp on concepts window functions, explicit ordering, sub-queries and/or foreign keys. Guinness posted:A diagram would be 100x better to describe the data. Yeah, I wish SSMS could spit out an ASCII diagram... among many other things  Guess I could put a screenshot of the diagram on Imgur and put a link in a comment at the top.Hadlock posted:Seems like at every job I work at, whenever we hire somebody super senior, the very first thing they do is look at top 10 queries by total time, then rewrite them by hand, as they're doing something really dumb in the ORM Side note but that's one of the first things I did when I started here, heh
|
|
#
?
Sep 25, 2020 14:56
|
|
Twerk from Home posted:What does failure because of poor ORM performance end up actually looking like? https://stackoverflow.com/questions/97197/what-is-the-n1-selects-problem-in-orm-object-relational-mapping I see this in UI as well, every row in a table kicks off another web call to retrieve another row. And of course it all works fine in testing
|
|
|
#
?
Sep 26, 2020 00:01
|
|
|
Coffee Jones posted:I see this in UI as well, every row in a table kicks off another web call to retrieve another row. "My test data has three records that I setup on my first day, three years ago, what, you want me to write code AND manage my own test data? That's QAs problem, don't ever email me again" cc: Engineering Manager, cc: VP of Engineering Signed- 40% of all developers I've ever met
|
|
#
?
Sep 26, 2020 01:03
|
|
|
Hadlock posted:"My test data has three records that I setup on my first day, three years ago, what, you want me to write code AND manage my own test data? That's QAs problem, don't ever email me again" cc: Engineering Manager, cc: VP of Engineering I've probably mentioned before being consulted on the project that was expected to work over the public internet in 2010 but hard fail if there was more than 100ms of latency between any of 8 clients. Why yes it worked perfectly on the in office networks!
|
|
#
?
Sep 26, 2020 01:22
|
|
|
Hadlock posted:"My test data has three records that I setup on my first day, three years ago, what, you want me to write code AND manage my own test data? That's QAs problem, don't ever email me again" cc: Engineering Manager, cc: VP of Engineering I remember a team of people working on a HIS (hospital information system) project. It was going to use OData and NHybernate and all the latest technology. They didn't test it with more than 40 studies until the week before release. Real hospital systems might have six or seven million studies. You can guess how well the worklist performed. It was a Japanese company so everyone who worked on that release just... wasn't assigned any work afterwards. They all quit one by one as soon as they could find other employment.
|
|
#
?
Sep 26, 2020 04:17
|
|
|
Yeah our chief front end guy used exactly three records to mock all his stuff I think he had a blood vessel in his eyeball pop when the customer success team came back to him and said it was crashing the JavaScript VM trying to render 80,000 items Took three redesigns and six months to build something that would scale to 50,000 items. The back end guys set a hard limit of 49,000 items.
|
|
#
?
Sep 26, 2020 05:02
|
|
|
That degree of failure makes me  . Literally all you need to get efficient multi-million-row handling for most realistic use cases is just knowing when to use which kind of join or subquery, when and how to efficiently paginate your lookups, how to set up good foreign keys and indexes, and how not to footgun yourself by trying to outsmart the query optimizer.The pagination stuff goes double on the frontend. If you're trying to load 50,000 records in the first place something has already gone wrong in the design phase. The human eye can't even make out that much data, so why aren't you either paginating tables or using some kind of interpolated averages for graphs?
|
|
#
?
Sep 26, 2020 10:25
|
|
|
my biggest problem with ef isn't n + 1 queries. it's when ef can't evaluate an expression in the db so silently pulls every single row into the app, then evaluates the expression there
|
|
#
?
Sep 26, 2020 12:38
|
|
|
One thing that I've taken to asking in interviews for jobs that have an ORM is: suppose you have some nasty slow query, whether it's because of bad database architecture or just a bad ORM generated SQL query. What's the normal approach you take? Lots of places surprisingly don't know, because it's not a problem they've run in to (this is probably bad). Some places say "well our database is well architected" but unless they have some DBA designing it or are insanely careful odds are it's not (nobody is insanely careful). But you can learn a lot about a place this way: do they really try to stick with the ORM? Do they lean on caching? Do they write raw SQL? Do they develop a plan to fix it at the source? Etc. Good way to learn about how a company approaches things. (when I interview for data engineer positions, which my career has flirted with a lot, I adjust accordingly...at my last job the data engineering team was prohibited by engineering leadership from writing raw SQL and they had such a garbage ETL process that I screamed when I saw it and was glad I didn't get put on the data engineering team)
|
|
#
?
Sep 26, 2020 13:03
|
|
|
Roadie posted:The pagination stuff goes double on the frontend. If you're trying to load 50,000 records in the first place something has already gone wrong in the design phase. The human eye can't even make out that much data, so why aren't you either paginating tables or using some kind of interpolated averages for graphs? I've had to explain to multiple people that you don't pull in 50,000 records and then discard all but the first 100 on the front end only. "But the user experience suffers if we have to make another request for the next 100 records!"
|
|
#
?
Sep 26, 2020 13:12
|
|
|
vonnegutt posted:I've had to explain to multiple people that you don't pull in 50,000 records and then discard all but the first 100 on the front end only. "But the user experience suffers if we have to make another request for the next 100 records!"
|
|
#
?
Sep 26, 2020 13:28
|
|
|
Ghost of Reagan Past posted:One thing that I've taken to asking in interviews for jobs that have an ORM is: suppose you have some nasty slow query, whether it's because of bad database architecture or just a bad ORM generated SQL query. What's the normal approach you take? Lots of places surprisingly don't know, because it's not a problem they've run in to (this is probably bad). Some places say "well our database is well architected" but unless they have some DBA designing it or are insanely careful odds are it's not (nobody is insanely careful). But you can learn a lot about a place this way: do they really try to stick with the ORM? Do they lean on caching? Do they write raw SQL? Do they develop a plan to fix it at the source? Etc. Good way to learn about how a company approaches things. One of my favorite hobbies is finding and removing caches that only exist to paper over bad database queries
|
|
#
?
Sep 26, 2020 17:32
|
|
|
redleader posted:my biggest problem with ef isn't n + 1 queries. it's when ef can't evaluate an expression in the db so silently pulls every single row into the app, then evaluates the expression there w---what? why would they implement that? can you at least disable that completely through a config option??
|
|
#
?
Sep 26, 2020 23:18
|
|
|
its why even shaggar says ef sucks
|
|
#
?
Sep 26, 2020 23:21
|
|
|
piratepilates posted:w---what? why would they implement that? can you at least disable that completely through a config option?? i thiiiink in more recent versions, ef will throw an exception if it can't evaluate the query server-side. it's one heck of a gotcha
|
|
#
?
Sep 27, 2020 00:30
|
|
|
Look, my dudes, at least it's not a mongo database.
|
|
#
?
Sep 28, 2020 04:57
|
|
|
kayakyakr posted:Look, my dudes, at least it's not a mongo database. look man, we've all been there. we're just trying to get something done within this sprint and this loving db team is like, unresponsive and poo poo. i didn't get into programming to do poo poo like "data modeling" or "database normalization" - i just want to code, god dammit. If I use mongodb, I don't have to deal with all those old people and i can get everything i need to do done within the sprint. the old people will figure out how to get it to perform once it hits production.
|
|
#
?
Sep 28, 2020 15:23
|
|
|
redleader posted:i thiiiink in more recent versions, ef will throw an exception if it can't evaluate the query server-side. it's one heck of a gotcha Yeah, I'm pretty sure it just errors out and don't remember using a version that doesn't, but didn't use it until 5, I think it was?.
|
|
#
?
Sep 29, 2020 18:27
|
|
|
Ghost of Reagan Past posted:Couple jobs ago we had a view that had frontend-only sorting, and multiple pages. It took me like a year to convince everyone that this was worth fixing. Did you get any credit when it was eventually deemed worth fixing? My current employer has enormous performance problems, but it's nobodies priority to fix and the business keeps beating the drum that "a good experience is worth waiting for" so our time-to-interactive across everything we build remains 20+ seconds and basic operations take 5+ seconds.
|
|
#
?
Sep 30, 2020 16:56
|
|
|
Has anyone told them that sitting and waiting for everything is a bad experience?
|
|
#
?
Sep 30, 2020 17:11
|
|
|
Twerk from Home posted:Did you get any credit when it was eventually deemed worth fixing? My current employer has enormous performance problems, but it's nobodies priority to fix and the business keeps beating the drum that "a good experience is worth waiting for" so our time-to-interactive across everything we build remains 20+ seconds and basic operations take 5+ seconds. At a certain point it's up to the business to decide what they can sell. Perhaps their customers don't care. The problem is that 20 seconds is an eternity for a loading time in all but special cases (e.g. a game). For a website or a business app that puts , say, a form or charts or some other poo poo in a window, it means something went really wrong in design phase - one of the point of moving over to web services is that the web service is always running, so you shouldn't have to worry about startup time. The implications are rarely good when it comes to, say, scalability of the application - e.g. while it might be OK if each individual opens the app in 20 seconds, the business is probably not OK if no one can open the app ever because of load and you will get blamed for that. I would say following conventional UI guidelines for response is probably reasonable when determing whether stuff such as load screens etc. are appropriate.
|
|
#
?
Sep 30, 2020 21:01
|
|
|
Twerk from Home posted:Did you get any credit when it was eventually deemed worth fixing? My current employer has enormous performance problems, but it's nobodies priority to fix and the business keeps beating the drum that "a good experience is worth waiting for" so our time-to-interactive across everything we build remains 20+ seconds and basic operations take 5+ seconds. So yes I did. Performance should be a top priority because most performance bottlenecks today are going to be worse tomorrow without proactive work, and you'll have cleaner code to boot. We deprioritized it because we sold a lovely B2B product that worked well enough and our internal clients didn't bang the performance drum. What counts as a performance bottleneck, though, is quite a bit more complex than just "it's slow!" Sometimes you don't care -- if a page takes 5 seconds to load and your audience doesn't have any performance SLAs and they can't just drop you just because your product sucks (multiyear contracts  ), 5 seconds can be fine (more than that you should at least be doing some profiling and figuring out what the issues are, at least -- we knew exactly where the bottlenecks were but we didn't think they were worth prioritizing all the time). If you're doing a direct-to-consumer product or your customer can switch to a competitor without breaking a contract or a huge amount of effort, your performance needs are likely quite different.
Ghost of Reagan Past fucked around with this message at 22:41 on Sep 30, 2020 |
|
#
?
Sep 30, 2020 22:31
|
|
|
Ghost of Reagan Past posted:and you'll have cleaner code to boot that's one heck of a claim
|
|
#
?
Oct 1, 2020 02:03
|
|
|
Ghost of Reagan Past posted:if a page takes 5 seconds to load and your audience doesn't have any performance SLAs and they can't just drop you just because your product sucks (multiyear contracts 5 seconds is my "this is fine" to "poo poo's on fire" breakdown. And oh man when you're using 4 different JS front-ends across your app, poo poo gets real close to being on fire on every page navigation...
|
|
#
?
Oct 1, 2020 19:03
|
|
|
kayakyakr posted:5 seconds is my "this is fine" to "poo poo's on fire" breakdown. And oh man when you're using 4 different JS front-ends across your app, poo poo gets real close to being on fire on every page navigation... I've seen a single project (not, not product, or application, I'm talking about literally one project in solution containing dozens of projects), that contained - Pages rendered using react - Pages rendered using angular - Pages rendered using asp.net mvc. - Pages rendered using a bunch of vanilla js and createElement. Imagine if the page asking for your username/password was written in Angular, and you click "Forgot My password", and it brings you to a react page. You go to a different page, and it's render in ASP.NET MVC. And all the pages were developed at the same time, so it wasn't as if there was an angular fad then a react fad.
|
|
#
?
Oct 1, 2020 19:47
|
|
|
Bruegels Fuckbooks posted:I've seen a single project (not, not product, or application, I'm talking about literally one project in solution containing dozens of projects), that contained Look, I'm not saying that you're describing our project (because gross, Microsoft languages), but why are you describing our project? (not fully. we do have a deep-cut legacy backbone page rendering a react component rendering a vue component, though.  )
|
|
#
?
Oct 1, 2020 20:52
|
|
|
Bruegels Fuckbooks posted:I've seen a single project (not, not product, or application, I'm talking about literally one project in solution containing dozens of projects), that contained Are they rendering each other via iframes? My personal favorite I've seen at work is an AngularDart application housing an iframe with a React application that hosts another iframe holding its own React root.
|
|
#
?
Oct 1, 2020 22:03
|
|
|
Why does this happen? I understand like the competition of frameworks whatever but like why would a single company allow it to happen instead of enforce standards for their tools.
|
|
#
?
Oct 2, 2020 01:08
|
|
|
barkbell posted:Why does this happen? I understand like the competition of frameworks whatever but like why would a single company allow it to happen instead of enforce standards for their tools. Honestly I've seen at least one developer so disgusted at how badly managed their engineering department was, that they decided to just go with the flow and invent problems and then solve them using esoteric solutions that were impossible to support Our technical project manager is bad, but not that bad
|
|
#
?
Oct 2, 2020 02:59
|
|
|
Gentlemen, a style question. I literally haven't written SQL for 20+ years, and my solution from the days of yore would be: code:Is that style of SQL considered unfashionable these days?
|
|
#
?
Oct 3, 2020 05:36
|
|
|
|
| # ? May 27, 2024 02:23 |
|
|
Presuming we're all "gentlemen" is certainly unfashionable these days  The SQL looks pretty clear to me, though I'm no expert.
|
|
#
?
Oct 3, 2020 08:36
|
|
