|
you got that pretty backwards, the point is that if i have a database significantly smaller than the memory on the machine it is a monumental failure that fetching the complete result set of select * from t where x=y takes orders of magnitude longer than just putting t into some java arrays and making a hashmap from x to indices into this array
|
#
?
Mar 14, 2013 16:22
|
|
|
|
| # ? Jun 2, 2024 18:15 |
|
|
Cybernetic Vermin posted:you got that pretty backwards, the point is that if i have a database significantly smaller than the memory on the machine it is a monumental failure that fetching the complete result set of select * from t where x=y takes orders of magnitude longer than just putting t into some java arrays and making a hashmap from x to indices into this array A few hundredths of a ms to get poo poo like atomicity and the ability for ops to be able to look at the data sounds good to me.
|
|
#
?
Mar 14, 2013 16:32
|
|
|
yeah, no, the "oh well db2/oracle/someshit is an industry standard and we can get dbas for those" is not really a good case for why the old sql dragons aren't total poo poo. the slowdown has nothing to do with atomicity, it is all lovely old data structures with lovely old protocols marshalling single records around in the worst way possible at every turn, with a lovely old query optimizer ready to screw you over in production at any moment because it has no concept of the lovely old lock contention bullshit that the sql dragons pull at the drop of a hat since they process queries so freakishly slow that they are sure to get a backlog at every possible moment
|
|
#
?
Mar 14, 2013 16:39
|
|
|
Run incredibly fast, scalable queries with the power of MongoDB today.
|
|
#
?
Mar 14, 2013 16:42
|
|
|
yeah that doesn't work either, but the nosql movement has at least one thing right, throw sufficiently many monkeys at this problem and someone will sooner or later do better than oracle with relative ease
|
|
#
?
Mar 14, 2013 16:43
|
|
|
Yeah NoSQL is loving dumb *reads a shitload of data from memcached all the time*
|
|
#
?
Mar 14, 2013 16:45
|
|
|
microsoft and postgres did already? just pick either of them, they should suit your needs if your data is small, just use an in memory sqlite database and periodically flush to the drive if you want or some poo poo i dunno
|
|
#
?
Mar 14, 2013 16:49
|
|
|
postgres is too lovely to make the list, never worked with mssql. the small stupid in-process in-memory sql databases do surprisingly ok work though, we seriously ran a pretty critical production job off of hsqldb after db2 threw up all over itself for the nth time, still leaves you to handle the entire persistence and integrity bit though, so there are more complete solutions that are faster still out there
|
|
#
?
Mar 14, 2013 17:02
|
|
|
Zombywuf posted:Anyone got any pro-tips on dealing with a death march?
|
|
#
?
Mar 14, 2013 17:08
|
|
|
Cybernetic Vermin posted:still leaves you to handle the entire persistence and integrity bit though, its a great database once you implement a database on top of it
|
|
#
?
Mar 14, 2013 17:19
|
|
|
Cybernetic Vermin posted:yeah, no, the "oh well db2/oracle/someshit is an industry standard and we can get dbas for those" is not really a good case for why the old sql dragons aren't total poo poo. the slowdown has nothing to do with atomicity, it is all lovely old data structures with lovely old protocols marshalling single records around in the worst way possible at every turn, with a lovely old query optimizer ready to screw you over in production at any moment because it has no concept of the lovely old lock contention bullshit that the sql dragons pull at the drop of a hat since they process queries so freakishly slow that they are sure to get a backlog at every possible moment Yeah, lern 2 database.
|
|
#
?
Mar 14, 2013 17:39
|
|
|
Cybernetic Vermin posted:to be fair p. much all sql databases are incredibly lovely fast databases are ez even with not a lot of resources.
|
|
#
?
Mar 14, 2013 17:47
|
|
|
our database at work is huge and the queries are really fast i think the trick is not designing it like a retard
|
|
#
?
Mar 14, 2013 17:48
|
|
|
nvarchar primary keys are pretty mandatory for good performance
|
|
#
?
Mar 14, 2013 17:50
|
|
|
Cybernetic Vermin posted:yeah, no, the "oh well db2/oracle/someshit is an industry standard and we can get dbas for those" is not really a good case for why the old sql dragons aren't total poo poo. the slowdown has nothing to do with atomicity, it is all lovely old data structures with lovely old protocols marshalling single records around in the worst way possible at every turn, with a lovely old query optimizer ready to screw you over in production at any moment because it has no concept of the lovely old lock contention bullshit that the sql dragons pull at the drop of a hat since they process queries so freakishly slow that they are sure to get a backlog at every possible moment if ur queries are slow its either bad design (for u this is probably it), a bad server (this is also u if ur using MySQL or some other faildb), or if you don't have enough resources (memory/disk). if you let your orm design the database its gonna be total poo poo now and forever cause ur design is poo poo and your code is poo poo and hibernate is poo poo and all of that will combine to make your data design triple-combo poo poo. if ur using a MySQL then just kill urself now. use sql server ftw. lastly, if you can prove its not the first 2 (you probably cant so don't try) then throw some faster disk / more ram at it.
|
|
#
?
Mar 14, 2013 17:52
|
|
|
normalization is like super easy, i dont understand why developers are so bad at it or choose to not do it
|
|
#
?
Mar 14, 2013 17:54
|
|
|
because most of them are bad and then theres web "developers" which don't really count as people.
|
|
#
?
Mar 14, 2013 17:56
|
|
|
persistence model is for lazy shitheads who hate both performance and data
|
|
#
?
Mar 14, 2013 17:57
|
|
|
Can you even ... like *STORE* data man?
|
|
#
?
Mar 14, 2013 18:12
|
|
|
i barely GNU her! posted:3. You can do the equivalent, better, using maps and reduces. this is the exact opposite reaction that the confused "mapreduce sux!!" paper by database experts should cause
|
|
#
?
Mar 14, 2013 18:21
|
|
|
Shaggar posted:lastly, if you can prove its not the first 2 (you probably cant so don't try) then throw some faster disk / more ram at it. did multi-terabyte financial databases for years, including the 200 gigabyte intraday database which we did indeed solve by having 256 gigabytes of ram sql databases are poo poo had an ex-ibm executive on the board that pushed for replacing poo poo with db2, an effort that every time was hugely expensive and always turned out amazingly poo poo (after ibm had gotten a couple of million in consulting fees of course) Cybernetic Vermin fucked around with this message at 18:33 on Mar 14, 2013 |
|
#
?
Mar 14, 2013 18:30
|
|
|
financial dbs are probably the worst designed ever. especially old ones.
|
|
#
?
Mar 14, 2013 18:36
|
|
|
Cybernetic Vermin posted:did multi-terabyte financial databases for years, including the 200 gigabyte intraday database which we did indeed solve by having 256 gigabytes of ram tps?
|
|
#
?
Mar 14, 2013 18:41
|
|
|
idgi what's wrong with postgresql, it seems to be good at doing the sorts of things you should be doing in a database oracle is "good" if a company that makes money selling database licenses tells you that you should do more stuff in the database and you actually listen to them
|
|
#
?
Mar 14, 2013 18:52
|
|
|
Mr Dog posted:idgi what's wrong with postgresql, it seems to be good at doing the sorts of things you should be doing in a database The query optimiser deserves a special place in hell for being excellent right up to the point where it just gives up and gives you a query exponential in the number of joins.
|
|
#
?
Mar 14, 2013 19:09
|
|
|
Ok, a bit of a serious-post: If your sql server works for you, fine, great, have fun. They certainly get the integrity and safe persistence thing done. However, if all you work with are the sql servers you easily get a bit weird expectations. Way too many developers tout the safety of the big sql databases, but then write applications that fetches a bunch of data out of them, finish the transaction, and keep working on that data, or serve it as cached data for later requests. This is often necessary for performance due to database overheads. Then you have squandered a lot of the integrity immediately. There are a lot of weird lighter-weight databases do things a bit differently than the sql dragons, but, really, they cut down on the many overheads, and if you are willing to throw a bit more memory on things, you get into a situation where you are able to actually work directly in the database, always maintaining the integrity. MongoDB is a stupid example, but there are much better ones out there, and DB2/Oracle/MySQL/whatever are rightly getting displaced in many tasks. Ignoring baseline performance the old sql assumptions are also unpredictable, a query optimizer sounds good in theory, but getting a query reoptimized in production is a horror, because even if the new query looks faster based on the internal statistics it may hold some important lock a fraction longer than the previous execution plan, and suddenly the lock contention takes of exponentially for the sky. The complex locking behavior is also a bit of an anachronism, one of the core reasons why it is easy to outperform the sql servers is that it is way better to race to finish for a vast majority of queries, do a coarse lock and get it finished quick, complex locking slowing down the queries means more queries are in flight at once, making locking issues more and more likely.
|
|
#
?
Mar 14, 2013 19:11
|
|
|
Zombywuf posted:Anyone got any pro-tips on dealing with a death march? gently caress off to america for a month and continue phoning it in, as every feature or line of code you write causes yet another snowflake deployment outside of your control.
|
|
#
?
Mar 14, 2013 19:11
|
|
|
i would like a sql that you could just bolt more servers onto to make it faster
|
|
#
?
Mar 14, 2013 19:14
|
|
|
Tiny Bug Child posted:i would like a sql that you could just bolt more servers onto to make it faster same
|
|
#
?
Mar 14, 2013 19:17
|
|
|
2012 has some new replication stuff i guess that does that kinda.
|
|
#
?
Mar 14, 2013 19:19
|
|
|
I'm sure you can't write to those slave servers without loving everything up can you?
|
|
#
?
Mar 14, 2013 19:21
|
|
|
it has some new active/active/active clustering thing iirc. there are probably special things you have to do to the database + tables to get it to work tho. like auto-increment keys probably wouldn't work. idk.
|
|
#
?
Mar 14, 2013 19:28
|
|
|
Shaggar posted:it has some new active/active/active clustering thing iirc. there are probably special things you have to do to the database + tables to get it to work tho. like auto-increment keys probably wouldn't work. idk. yeah that's why I think most surrogate keys are now GUIDS instead of auto increment to alleviate those kinds of problems
|
|
#
?
Mar 14, 2013 19:30
|
|
|
right. guids are ftw.
|
|
#
?
Mar 14, 2013 19:33
|
|
|
Tiny Bug Child posted:i would like a sql that you could just bolt more servers onto to make it faster I like the SQL where I just write some code and it goes faster.
|
|
#
?
Mar 14, 2013 20:49
|
|
|
Shaggar posted:right. guids are ftw. Hope you're not planning on large inserts.
|
|
#
?
Mar 14, 2013 20:50
|
|
|
Been using H2 as my babby's first database, seems ok so far but I only have a couple of tables yet. Anything wrong with it for small database purposes?
|
|
#
?
Mar 14, 2013 21:01
|
|
|
nope.
|
|
#
?
Mar 14, 2013 21:01
|
|
|
polpotpi posted:normalization is like super easy, i dont understand why developers are so bad at it or choose to not do it If I had understood closures and data normalization coming out of (ee) undergrad it would have changed... everything 
|
|
#
?
Mar 14, 2013 22:40
|
|
|
|
| # ? Jun 2, 2024 18:15 |
|
|
code review pfft. who cares schema review. bad schemas end lives
|
|
#
?
Mar 14, 2013 22:43
|
|
