|
i would love to see that analysis done on the dumb poo poo queries i wrote 30 years ago against the order tables that ran for 45 minutes or more. it could not have been the real oltp tables, it had to be a datamart extract or they would have had an automated “kill any query that scans more than 20 rows” or something.
|
#
?
Mar 31, 2024 13:45
|
|
|
|
| # ? May 3, 2024 00:03 |
|
|
a long time ago in old job we upgraded from sybase 11 to version 12 l think which is when they re-wrote the query planner we were p aware on how to structure data and have queries that were sub-second response times 99% of the time but after the upgrade there were one or two that went totally pear shaped thankfully there was a "don't do that" syntax that reverted those queries to the v11 query planner which fit the bill until I left for a better job
|
|
#
?
Mar 31, 2024 14:44
|
|
|
Deep Dish Fuckfest posted:you're basically taking a side-gig as a query planner except you don't have the various stats an actual query planner uses to make decisions about which indices/join methods/etc to use and also it turns out you don't get asked the queries you thought you would get asked because someone changed their mind fair enough but the entire point of nosql systems is that your queries should be pretty drat simple. if you find you regularly need sophisticated queries then you probably screwed up your model or as you said someone changed their mind also there’s nosql systems nowadays with query planners & optimizers but they aren’t going to be nearly as sophisticated as sql ones because they don’t need to be
|
|
#
?
Mar 31, 2024 15:25
|
|
|
why not? the extra sophistication is bound to help somewhere
|
|
#
?
Mar 31, 2024 15:59
|
|
|
your queries are not so much simple as they are the focal point of the database. they can be complicated, but answering them as effectively as possible is the point, or else you'd just use a rdbms. sophistication is often bad for nosql systems because if you're using them you probably care about p99, sophistication makes answering questions about p99 harder.
|
|
#
?
Mar 31, 2024 16:26
|
|
|
PIZZA.BAT posted:someone changed their mind luckily this is a rare situation, to the point of not being worth thinking about
|
|
#
?
Apr 1, 2024 05:46
|
|
|
redleader posted:luckily this is a rare situation, to the point of not being worth thinking about 
|
|
#
?
Apr 1, 2024 14:25
|
|
|
redleader posted:query planner moment 
|
|
#
?
Apr 1, 2024 19:27
|
|
|
my hot take about query planners is that they should exist but be completely manual. You look at a high-level IR for your query and add in scheduling and index information. The array programming people have been doing this for a while (https://halide-lang.org/, http://tensor-compiler.org/) and it works surprisingly well. (Unfortunately they're array programming people so they don't know they're doing query planning.)
|
|
#
?
Apr 1, 2024 20:28
|
|
|
who the gently caress is going to manually plan a query?
|
|
#
?
Apr 1, 2024 20:33
|
|
|
yeah one of the big selling points on sql is that non-programmers can pick it up pretty easily without really having to 'get' how computers work and forcing them to build their own query plans sounds like a great way to to get people to stop using it if that's the point then go for it
|
|
#
?
Apr 1, 2024 20:35
|
|
|
i love giving random people the ability to go and query for their own incorrect data
|
|
#
?
Apr 1, 2024 22:21
|
|
|
PIZZA.BAT posted:yeah one of the big selling points on sql is that non-programmers can pick it up pretty easily without really having to 'get' how computers work and forcing them to build their own query plans sounds like a great way to to get people to stop using it you only “force” them to do it if their queries need to be optimized for some reason; otherwise a more naive, easy-to-predict optimization set suffices
|
|
#
?
Apr 2, 2024 00:48
|
|
|
redleader posted:i love giving random people the ability to go and query for their own incorrect data this is the marketing pitch for power bi
|
|
#
?
Apr 2, 2024 01:09
|
|
|
Archduke Frantz Fanon posted:this is the marketing pitch for power bi the best sexuality
|
|
#
?
Apr 2, 2024 01:19
|
|
|
Subjunctive posted:you only “force” them to do it if their queries need to be optimized for some reason; otherwise a more naive, easy-to-predict optimization set suffices yeah this is what halide does. By default it's just the obvious loop nests, then if you need to make it do something smarter you can do that. honestly though it really isn't an accessible programming model. The only people actually using halide are compiler experts. You are always gonna need query planners to do the needful for 99.9% of users. I just want this for myself because I think it would be cool
|
|
#
?
Apr 2, 2024 02:09
|
|
|
i think its funny how sqlite has 5 datatypes. i mean thats all you really need (null, string, int, float, raw data) but still
|
|
#
?
Apr 2, 2024 06:23
|
|
|
that's 4 more types than you need (bytes)
|
|
#
?
Apr 2, 2024 06:30
|
|
|
you only really need strings. the rest are derivable from that
|
|
#
?
Apr 2, 2024 06:55
|
|
|
you only really need chars. the rest are indexable from that
|
|
#
?
Apr 2, 2024 07:09
|
|
|
PIZZA.BAT posted:yeah one of the big selling points on sql is that non-programmers can pick it up pretty easily without really having to 'get' how computers work and forcing them to build their own query plans sounds like a great way to to get people to stop using it one of the intended selling points, it is moot though since non-programmers cannot do sql, and in fact most programmers can't either.
|
|
#
?
Apr 2, 2024 10:50
|
|
|
Cybernetic Vermin posted:one of the intended selling points, it is moot though since non-programmers cannot do sql, and in fact most programmers can't either. idk. my wife knows a bunch of excel crap and she picked up enough sql to make queries for the data she needed at byte dance and it only took her a couple weeks to get the hang of it. i guess you could call excelheads “programmers” tho
|
|
#
?
Apr 2, 2024 10:52
|
|
|
fart simpson posted:idk. my wife knows a bunch of excel crap and she picked up enough sql to make queries for the data she needed at byte dance and it only took her a couple weeks to get the hang of it. i guess you could call excelheads “programmers” tho pretty sure your wife could also have done the query in a sensible order if that had been needed. in general i think the query planner idea overestimates computation and underestimates people. not that people can do general discrete optimization problems better than computers (though computers are way worse at it than people in the 60s expected), but rather the human will almost invariably have a decent idea of what the data is (e.g. there's so many employees but only a couple of departments, etc.), and i think indices are actually really graspable to people (very real-world-relevant after all). just make sure the system is built such that you can gently caress up a couple of times without having it fall down. which, granted, a lot of database systems aren't. my first fintech job you could reliably crash production if you ever entered a query to a time-series database that didn't filter on dates first.
|
|
#
?
Apr 2, 2024 11:14
|
|
|
wouldn't be a problem for me; i definitely get filtered on first dates
|
|
#
?
Apr 2, 2024 11:54
|
|
|
Deep Dish Fuckfest posted:wouldn't be a problem for me; i definitely get filtered on first dates lol
|
|
#
?
Apr 2, 2024 13:55
|
|
|
Deep Dish Fuckfest posted:wouldn't be a problem for me; i definitely get filtered on first dates
|
|
#
?
Apr 2, 2024 13:59
|
|
|
i think i’ve told this story before but the worst i’ve seen was when i was consulting a major health insurer where someone handed the keys to prod to a bunch of interns. they came to my office one day asking if i could come over to a meeting room they had to teach them how to write better queries because the one they had wasn’t working well. i go over and one of them has their screen up on the wall and immediately i see that their problem is that they’re doing the equivalent of a select all with no filters and tons of joins and yadda yadda yadda. it’s sitting there spinning away and my eyes meander to the endpoint url and i poo poo a brick. the interns thought they were all gonna get fired lol that was a fun one to take back to their director
|
|
#
?
Apr 2, 2024 15:00
|
|
|
Archduke Frantz Fanon posted:this is the marketing pitch for power bi I have unironically heard this said by a tech manager and it's like, no not really. and I say that as someone that now uses power bi. I mean if your data sources are built out as specific sets for you then maybe, but that means someone has gone through and done it for you based on spec. or you just did a bunch of excel exports from the front end reporting tooling and manually copied them somewhere which is what actually happened in reality. Feisty-Cadaver posted:a long time ago in old job we upgraded from sybase 11 to version 12 l think which is when they re-wrote the query planner yeah this exact thing happened at my old job whereas I, by dint of having a much much smaller DB with performance tolerance of "see if anyone notices in the frontend" just yolo'd mssql compatibility mode from 2008 to 2016 and it was fine
|
|
#
?
Apr 3, 2024 13:02
|
|
|
PIZZA.BAT posted:fair enough but the entire point of nosql systems is that your queries should be pretty drat simple. if you find you regularly need sophisticated queries then you probably screwed up your model or as you said someone changed their mind the one time I dealt with a nosql database (Dynamo) the app developers also put the data in an Elastic DB because Dynamo wouldn't let them do the queries they needed to do. that had no ramifications in the future when the two data sources diverged and queries didn't return the right data. you might be wondering "if the queries needed to be that complex, why were they in nosql?" and I would answer "because the designer/architect/CTO was a total idiot when it came to application design and cared more about buzzwords and w≋e≋b s≋c≋a≋l≋e than making a working product." which really underlies my issue with nosql: its queries are supposed to remain simple and the model shouldn't change, but all non-failed projects I've seen always get more complex as time goes on, and the initial assumptions made at design time will no longer apply, so how can nosql work in the long run except in extremely limited circumstances?
|
|
#
?
Apr 3, 2024 14:37
|
|
|
it depends? if it's the data model that's getting more complicated that's usually fine because you can just keep heaping more attributes into your documents with a pretty negligible hit on performance. if the access pattern is what's massively changing then stakeholders need to know that there's going to be some pain in the transition. it's not impossible
|
|
#
?
Apr 3, 2024 15:26
|
|
|
this is why in a conversation recently when the other guy said "well we're representing actual documents here so they should be stored in a document database" and I said "yeah except that the model of a given document type evolves over time and changes can have retrospective effect" he went quiet
|
|
#
?
Apr 3, 2024 16:56
|
|
|
cosmosdb y/n
|
|
#
?
Apr 4, 2024 00:59
|
|
|
we don’t support cosby in the pos anymore
|
|
#
?
Apr 4, 2024 08:36
|
|
|
recently started working on a custom relational db at current job. It has: - built in webass runtime - multiple built in orms that you have to use - single version concurrency control (global lock on everything) - partial SQL support i am having a grand old time
|
|
#
?
Apr 7, 2024 23:45
|
|
|
i had a class on databases and i forgot everything the second i handed in my exam so now every database i make is a single table which is funny because i actually did really well on the exam
|
|
#
?
Apr 7, 2024 23:55
|
|
|
hbag posted:i had a class on databases and i forgot everything the second i handed in my exam so now every database i make is a single table ah, a nosql user is born
|
|
#
?
Apr 8, 2024 00:13
|
|
|
data based on what?
|
|
#
?
Apr 8, 2024 02:27
|
|
|
I don't even know what a document is
|
|
#
?
Apr 8, 2024 21:40
|
|
|
a miserable little pile of tokens
|
|
#
?
Apr 8, 2024 22:31
|
|
|
|
| # ? May 3, 2024 00:03 |
|
|
Ocean of Milk posted:I don't even know what a document is its a portmanteu of 'doctrine argument'
|
|
#
?
Apr 8, 2024 23:38
|
|
