|
Deep Dish Fuckfest posted:reminds me of that time i got curious how "real" file formats actually look like and read the specs for stuff like elf and dwarf and pdf and who knows what else oh wow, an
|
#
?
Mar 28, 2024 02:17
|
|
|
|
| # ? May 2, 2024 20:14 |
|
|
htonl
|
|
#
?
Mar 28, 2024 02:24
|
|
|
Subjunctive posted:the SQLite disk format is readable by humans if that human is motivated enough i already said to use strings!!!
|
|
#
?
Mar 28, 2024 07:01
|
|
|
rotor posted:fun fact: the first programming language i learned as an adult was postscript were you trying to jailbreak your 202?
|
|
#
?
Mar 28, 2024 07:04
|
|
|
PIZZA.BAT posted:i'm hearing vector databases are the new hotness that's getting all the vc attention makes sense b/c vectords are cool
|
|
#
?
Mar 28, 2024 08:17
|
|
|
Bloody posted:my favorite database is a text file I store next to the executable this ftw
|
|
#
?
Mar 28, 2024 08:19
|
|
|
PIZZA.BAT posted:i was just on a call with one of my customers where our machine learning guy was walking them through various use cases to help them brainstorm things that could be applicable for them. the discussion meanders around for a bit and we hit the inevitable, 'how many records does your dataset contain' question. they answered fifty. we asked, 'fifty... million? billion? what?' that reminds me of being involved in a multi week negotiation w/ a sales guy who was requesting we add a bunch of crap to a soundbar so he could land a big sale in singapore. and after all that negotiation my boss finally asked him how big the sale was and he said "75". as in we had just done all that work to sell 75 additional units
|
|
#
?
Mar 28, 2024 08:21
|
|
|
Well if the units are airbusses...
|
|
#
?
Mar 28, 2024 11:27
|
|
|
Ocean of Milk posted:Well if the units are airbusses... they were soundbars op and our yearly sales were like 50,000 units
|
|
#
?
Mar 28, 2024 11:36
|
|
|
at work we use cosmosDB, and I wish we just used mssql instead. it’s sort of handy because a lot of what we do is basically just posting json from a mobile app to an api and then saving it to the db but we (or rather the consultancy that built this thing originally) did a lot of dumb things that make my life suck cosmosDB has a limit of 2mb per document, which sounds like loads of space, except that we do poo poo like store an entire Customer object in each Report record, even though the Customer object contains n Office objects and each report can only ever relate to a single Office, so we have massive pointless data duplication. this would not have happened if we just had a relational db (probably) fun other db bullshit (this one more self inflicted though): we also have the Partition Key set as /customerName, which is not fixed. we also build a customerId at runtime that includes this non mutable name so sometimes when the customer name changes we unintentionally just soft delete the old one and insert a new one and pretend everything is fine until someone wants to query for something belonging to the client pre-name-change
|
|
#
?
Mar 29, 2024 13:23
|
|
|
sounds like what you actually needed was a file system for storing those json files
|
|
#
?
Mar 29, 2024 13:58
|
|
|
Surprise T Rex posted:this would not have happened if we just had a relational db (probably) lol no dummies are gonna dumb no matter what technology you give them
|
|
#
?
Mar 29, 2024 14:33
|
|
|
Shaggar posted:sounds like what you actually needed was a file system for storing those json files that doesn’t sound very cloud-first
|
|
#
?
Mar 29, 2024 14:38
|
|
|
Surprise T Rex posted:that doesn’t sound very cloud-first What if the file system was… a cloud file system 
|
|
#
?
Mar 29, 2024 14:40
|
|
|
Azure files, bithc
|
|
#
?
Mar 29, 2024 14:42
|
|
|
wait gently caress i just realized, why isn't the thread title "DENORMALIZE YOURSELF AND FACE TO WEB SCALE."?
|
|
#
?
Mar 29, 2024 14:45
|
|
|
Deep Dish Fuckfest posted:wait gently caress i just realized, why isn't the thread title "DENORMALIZE YOURSELF AND FACE TO WEB SCALE."? 
|
|
#
?
Mar 29, 2024 14:49
|
|
|
ADINSX posted:What if the file system was… a cloud file system still boring. now blob storage maybe? files: a concept more than 10 years old, reminiscent of paper (boomer tech), files live on hard drives blob: mysterious, trendy, files live in the sky (?)
|
|
#
?
Mar 29, 2024 15:57
|
|
|
what if we implement our own custom file storage system on top of blob storage??
|
|
#
?
Mar 29, 2024 16:04
|
|
|
marking ticket as closed
|
|
#
?
Mar 29, 2024 18:26
|
|
|
Shaggar posted:what if we implement our own custom file storage system on top of blob storage?? current trends seem to be implementing everything on top of blob storage
|
|
#
?
Mar 30, 2024 01:07
|
|
|
Surprise T Rex posted:at work we use cosmosDB, and I wish we just used mssql instead. nosql databases can definitely be relational, you just need to be pretty sure of your access patterns ahead of time incuding the customer object in reports is dumb cos you can clearly just have the customer id there and look it up, there’s no need to duplicate rick houlihan did a great talk on this which made me understand nosql dbs much better [video type=""]https://m.youtube.com/watch?v=HaEPXoXVf2k[/video] but just loving using sql is almost never a bad choice, people who fart sniff about db engines without a good reason can lick my balls
|
|
#
?
Mar 30, 2024 07:38
|
|
|
All I want from SQL is swapping the clauses around. Let me write FROM bottoms WHERE poo poo=hard SELECT smell. Who decided on the lovely order of the clauses where the last thing you decide is first.
|
|
#
?
Mar 30, 2024 10:24
|
|
|
git apologist posted:nosql databases can definitely be relational, you just need to be pretty sure of your access patterns ahead of time yeah, I suspect what happened is “hmm we need a database, what’s the azure database? oh yeah cosmos that’ll do” with no further thought as to data model or how best to store things because it’s a document db so who cares about schema or planning or anything? just throw it all in there. definitely some strange architectural decisions made on this app.
|
|
#
?
Mar 30, 2024 11:45
|
|
|
git apologist posted:nosql databases can definitely be relational, you just need to be pretty sure of your access patterns ahead of time for years i’ve been tossing around the idea of making one of those massive ‘so you need a’ flowchart diagrams for databases. i’m lazy though
|
|
#
?
Mar 30, 2024 14:39
|
|
|
"money pit" -> oracle "data pit" -> mongo or some other nosql poo poo idk "sanity pit" -> mysql come to think of it, i haven't followed big data stuff (is that even still a thing?) for a long time so it's entirely possible that "data pit" is now an actual thing
|
|
#
?
Mar 30, 2024 16:23
|
|
|
they found that 'lake' was more marketable but its realistically just a moist pit.
|
|
#
?
Mar 30, 2024 17:00
|
|
|
i’m about 3/4 through that video and it’s pretty good. it’s kind of validating because i pretty much give that talk to any new customers we get who aren’t familiar with how the whole nosql thing works. the biggest trap is when they start trying to model their data first before they try to understand the queries they’ll be answering, because that’s how you do it in the relational world. it takes a lot of effort to get people to shake their old habits because the data modeling process is completely upside down to those who’ve lived in that world for a while
|
|
#
?
Mar 30, 2024 17:14
|
|
|
Deep Dish Fuckfest posted:"money pit" -> oracle big data isn’t used anymore people just say scalable. and yes there are lots of applications where the amount of data grows beyond what a single server can handle. once you’re into cluster world you’re in scalable / big data world the particular tool you’ll choose depends on your use case but there’s a reason so few shops choose a relational model once they start clustering
|
|
#
?
Mar 30, 2024 17:18
|
|
|
basically: if your data can exist on a single server then just use whatever you know because at that level the biggest concern is just your ability to get poo poo done
|
|
#
?
Mar 30, 2024 17:22
|
|
|
nielsm posted:All I want from SQL is swapping the clauses around. Let me write FROM bottoms WHERE poo poo=hard SELECT smell. Who decided on the lovely order of the clauses where the last thing you decide is first.
|
|
#
?
Mar 30, 2024 17:24
|
|
|
PIZZA.BAT posted:big data isn’t used anymore people just say scalable. and yes there are lots of applications where the amount of data grows beyond what a single server can handle. once you’re into cluster world you’re in scalable / big data world oh, no, i know the actual tech is still there; i meant strictly from a buzzword perspective. and it seems i was right to ask. imagine using last year's let alone last decade's buzzwords. might as well be talking about vacuum tubes and kerosene lamps
|
|
#
?
Mar 30, 2024 18:00
|
|
|
PIZZA.BAT posted:i’m about 3/4 through that video and it’s pretty good. it’s kind of validating because i pretty much give that talk to any new customers we get who aren’t familiar with how the whole nosql thing works. the biggest trap is when they start trying to model their data first before they try to understand the queries they’ll be answering, because that’s how you do it in the relational world. it takes a lot of effort to get people to shake their old habits because the data modeling process is completely upside down to those who’ve lived in that world for a while you're basically taking a side-gig as a query planner except you don't have the various stats an actual query planner uses to make decisions about which indices/join methods/etc to use and also it turns out you don't get asked the queries you thought you would get asked because someone changed their mind
|
|
#
?
Mar 30, 2024 18:05
|
|
|
at work i tell people to just read the dynamodb documentation for modeling data if they have any questions, even though the nosql database i janitor is not dynamodb. its very good documentationDeep Dish Fuckfest posted:you're basically taking a side-gig as a query planner except you don't have the various stats an actual query planner uses to make decisions about which indices/join methods/etc to use and also it turns out you don't get asked the queries you thought you would get asked because someone changed their mind i'm the query planner except if the user changes the query their app stops performing within its requirements and i tell them "i recommend not doing that and using the queries that we discussed, which is what the database is tuned for" and then their choices are to do what i suggest or to not have a functioning application anymore.
|
|
#
?
Mar 30, 2024 18:32
|
|
|
Cybernetic Vermin posted:they found that 'lake' was more marketable but its realistically just a moist pit. someone coined the phrase "data puddle" for something we're doing (for bad/dumb reasons) and i hate it so much
|
|
#
?
Mar 30, 2024 23:54
|
|
|
i would pay microsoft a lot of money (that is, the amount of money we pay now for azure mssql) to add a query hint that goes "ok just use adaptive joins everywhere; please don't try to get fancy; you're not smart enough to do this; just trust me"
|
|
#
?
Mar 30, 2024 23:57
|
|
|
redleader posted:someone coined the phrase "data puddle" for something we're doing (for bad/dumb reasons) and i hate it so much a data puddle is what pizza.bats client built for their 50 documents
|
|
#
?
Mar 31, 2024 00:04
|
|
|
Asleep Style posted:a data puddle is what pizza.bats client built for their 50 documents
|
|
#
?
Mar 31, 2024 00:14
|
|
|
Asleep Style posted:a data puddle is what pizza.bats client built for their 50 documents
|
|
#
?
Mar 31, 2024 03:52
|
|
|
|
| # ? May 2, 2024 20:14 |
|
|
query planner moment query planner moment  query planner moment  not a query planner moment 
|
|
#
?
Mar 31, 2024 04:03
|
|