Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
trex eaterofcadrs
Jun 17, 2005
My lack of understanding is only exceeded by my lack of concern.
i really really like how datomic makes temporal data a first class citizen of any transaction in such a way that you can essentially go back in time and discover the truth "in the past"

Adbot
ADBOT LOVES YOU

MononcQc
May 29, 2007

Shaggar posted:

but how does the database determine which data is conflicting, which is bad, and where to send it? if the application failed to write the complete data for an operation, how does the database know that? i guess thats what i dont understand

A single node can have its own ACID semantics and have its own writes being consistent.

Riak and Dynamo will read from more than one node (a configurable N, from 1 node to 100% of the nodes). It will then compare the values received from each of these nodes and see if they all are the same (they can usually work with hashes to make this faster).

If they're all the same, no conflict. If they differ, you can use relative timestamps (using vector clocks or lamport clocks) to figure out a logically correct (as in mathematically correct) winner. If the winner can't be found, then you have a conflict.

Once the conflict is detected, you can do: 1) random pick of a winner 2) last-write wins, if you decide you can trust your absolute timestamps (you shouldn't) 3) pick a majority winner across the reads 4) something else, or 5) push it to the user/application.

The strategy you pick is dependent on the kind of application you write and what you can allow yourself. Banks can thus pick highly reliable factors with safe conflict-resolution semantics, while some guy writing distributed session handling for his website can pick something far less reliable.

That's the model for Riak/Dynamo. Other databases built on other principles can do things differently if they want to.

E: The consistency factors required can also be selected per-request, so that your money transactions have a higher consistency factor and different conflict resolutions from storing events of whoever logged in the system.

Late Edit: The DB knows when writes fail based on some transaction manager, the same way one is used for two-phase commits. The mechanism can vary (it can be a 2 phase commits with only N/M nodes), or use something like Paxos to make it work, but the mechanism has the same challenges as whatever SQL database you're used to may have.

MononcQc fucked around with this message at 18:10 on Oct 23, 2012

JawnV6
Jul 4, 2004

So hot ...
good lord databases are boring

Tiny Bug Child
Sep 11, 2004

Avoid Symmetry, Allow Complexity, Introduce Terror
databases are extremely interesting

this nosql poo poo they're talking about is not

MononcQc
May 29, 2007

Distributed systems are super interesting and fun :3:

Jonny 290
May 5, 2005



[ASK] me about OS/2 Warp
page 219 stop jerking around already


code:
perl -pe "}{ print 'tl;dr'" < this_thread.txt

double sulk
Jul 2, 2010

219 motherfuckers

rotor
Jun 11, 2001

classic case of pineapple derangement syndrome
man if we cant sperge out for two pages about transactions then i dont even know why we have a yospos

trex eaterofcadrs
Jun 17, 2005
My lack of understanding is only exceeded by my lack of concern.

rotor posted:

man if we cant sperge out for two pages about transactions then i dont even know why we have a yospos

goddamn right

MononcQc
May 29, 2007

Your Transaction System is a Piece Of poo poo

Tiny Bug Child
Sep 11, 2004

Avoid Symmetry, Allow Complexity, Introduce Terror
219 pages, every single one a testament to php supremacy

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde
dependency injection and inversion of control are both terrible names for hookin' up components of a system, they should have called it runtime component linking or something

furthermore, what's with ovaltine? they should have called it roundtine

salted hash browns
Mar 26, 2007
ykrop
glorious page 219

Sweeper
Nov 29, 2007
The Joe Buck of Posting
Dinosaur Gum

Gazpacho posted:

dependency injection and inversion of control are both terrible names for hookin' up components of a system, they should have called it runtime component linking or something

furthermore, what's with ovaltine? they should have called it roundtine

Dependency injection is cool, butq I feel like it makes it easier for bad coders to write impossible to follow code

Janitor Prime
Jan 22, 2004

PC LOAD LETTER

What da fuck does that mean

Fun Shoe

Sweeper posted:

Dependency injection is cool, butq I feel like it makes it easier for bad coders to write impossible to follow code

How so?

Also 219 bitches

shrughes
Oct 11, 2008

(call/cc call/cc)
I come to this thread to forget about work, not talk about work :(

gangnam reference
Dec 26, 2010

shut up idiot shut up idiot shut up idiot shut up idiot

shrughes posted:

I come to this thread to forget about work, not talk about work :(

it would be nice if u left thx

Jonnty
Aug 2, 2007

The enemy has become a flaming star!

shrughes posted:

I come to this thread to forget about work, not talk about work :(

you come to this thread, about programming languages, to forget about work, where you use programming languages. good work

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Jonnty posted:

you come to this thread, about programming languages, to forget about work, where you use programming languages. good work

best work, especially if you use modern powerful languages instead of outdated and broken trash like java & php

dragon enthusiast
Jan 1, 2010

Cocoa Crispies posted:

best work, especially if you use modern powerful languages instead of outdated and broken trash like java & php
modern powerful languages like fortran 77

Tiny Bug Child
Sep 11, 2004

Avoid Symmetry, Allow Complexity, Introduce Terror

Cocoa Crispies posted:

especially if you use modern powerful languages like php

Rufus Ping
Dec 27, 2006





I'm a Friend of Rodney Nano

rotor posted:

literaly a door-to-door mongo salesman

mongosteen

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde

Sweeper posted:

Dependency injection is cool, butq I feel like it makes it easier for bad coders to write impossible to follow code

Otoh it facilitates dividing a system among engineers of different skill levels so the better ones don't get throttled by the weaker ones

Shaggar
Apr 26, 2006

MononcQc posted:

A single node can have its own ACID semantics and have its own writes being consistent.

Riak and Dynamo will read from more than one node (a configurable N, from 1 node to 100% of the nodes). It will then compare the values received from each of these nodes and see if they all are the same (they can usually work with hashes to make this faster).

If they're all the same, no conflict. If they differ, you can use relative timestamps (using vector clocks or lamport clocks) to figure out a logically correct (as in mathematically correct) winner. If the winner can't be found, then you have a conflict.

Once the conflict is detected, you can do: 1) random pick of a winner 2) last-write wins, if you decide you can trust your absolute timestamps (you shouldn't) 3) pick a majority winner across the reads 4) something else, or 5) push it to the user/application.

The strategy you pick is dependent on the kind of application you write and what you can allow yourself. Banks can thus pick highly reliable factors with safe conflict-resolution semantics, while some guy writing distributed session handling for his website can pick something far less reliable.

That's the model for Riak/Dynamo. Other databases built on other principles can do things differently if they want to.

E: The consistency factors required can also be selected per-request, so that your money transactions have a higher consistency factor and different conflict resolutions from storing events of whoever logged in the system.

Late Edit: The DB knows when writes fail based on some transaction manager, the same way one is used for two-phase commits. The mechanism can vary (it can be a 2 phase commits with only N/M nodes), or use something like Paxos to make it work, but the mechanism has the same challenges as whatever SQL database you're used to may have.

so thats how notes works. you can set a per database conflict resolution of either 1) refuse to le the user save a conflicing document or 2) let the user save a conflicint document and mark it as conflicing. then the user will usually delete the conflicting document and wonder where their changes went.

for transactions, is it still up to the application code to handle wrapping their code in transactions? is there a stored proc equivilent where you basically pass parameters to a statement (that then handles all the work) instead of orming?

0xB16B00B5
Aug 24, 2006

by Y Kant Ozma Post
PHP is your teenage sweetheart, the girl you first awkwardly fumbled around with that one summer. Just don't try and start a more serious relationship - this girl has serious issues.

Perl is PHP's older sister. She might be a bit old for you, but she was pretty popular back in the 90s. She's looking pretty ugly and scarred now, so you don't hear from her too much.

Ruby is the cool kid of the scripting family. When you first saw her, she took your breath away with her beauty. She was fun, too. At the time she seemed a bit slow and ditzy - though she's matured a lot in the last few years.

Python is Ruby's sensible (and slightly more boring) sister.

Java is a successful career woman. What she lacks in raw intelligence she makes up for in dress - always turned out in immaculateCamelCase, sure to impress the enterprise customers. You might feel like she's the sensible type you should settle down with. Just prepare for years of "NO THAT DOESNT GO THERE GOD YOU ALWAYS USE THE WRONG TYPE AND YOU MISSED A SEMICOLON" nagging.

C++ is Java's cousin. Similar to Java in many ways, the main difference being she grew up in a more innocent time and doesn't believe in condoms/automatic memory management, so be cautious.

C is C++'s mom. Mention her name to some old grey beard hackers and they're sure to reminisce with a twinkle in their eye.

Objective C is another member of the C family. She joined that weird church a while back, and won't date anyone outside of it.

Haskell, Clojure, Scheme and their friends are those hipster, artsy, intellectual girls you probably spent a blissful college summer with a few years ago. The first girls who really challenged you. Of course, it could never have become something more serious (you tell yourself). Though you'll always be left asking "what if?"

You might be put off C# due to her family's reputation. But they've gone legit, the last few years, they tell you. Once you're one of us, you're one of us, you hear? You need a database? Her brother MSSQL will hook you up. Need a place to stay? Heck, her daddy will even buy you your own mansion on Azure avenue. What's that, you're having second thoughts about all these overly friendly relatives? No, you can never leave. You're part of the family, now, ya hear?

Javascript - hey, wasn't that the girl you first kissed, way before even PHP came on the scene? I wonder what she's doing now. I hear her career's really taken off in the last few years. Would be cool to catch up some time, if only for old time's sake... (You catch sight of her, dressed head to toe in designer jQuery)... wow, somebody grew into a beautiful swan...

0xB16B00B5
Aug 24, 2006

by Y Kant Ozma Post
PHP is literally the broken woman seen in TBC's videos

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Shaggar posted:

for transactions, is it still up to the application code to handle wrapping their code in transactions? is there a stored proc equivilent where you basically pass parameters to a statement (that then handles all the work) instead of orming?

not really, most nosql systems don't have stored procedures because the big features are in data storage, not data processing

what usually happens is wrapping the storage layer with an API service that provides the business logic and abstraction from app developers

it's like stored procedures but you can mix and match storage layers in the same api: store users and their relation to organizations in postgresql, their relation to other users in neo4j, their profile pics in riak cs, wall posts in riak kv, etc.

Shaggar
Apr 26, 2006

Hard NOP Life posted:

How so?

Also 219 bitches

usually it makes things not so obvious. ex: in mybatis you create java interfaces and then map their methods to statements in a mapping xml file. normally you have to tie the interface to the mapping file by hand. however, since the standard convention is to name your xml files the same as the interface it seems like this could be automated.

mybatis-spring provides a super easy class that will look in a specified package for interfaces and then try to find matching mapper xml files. when found, it will tie the two together and then generate a spring bean with an id that matches the mapper name. you can then inject this bean into any dependant beans. this is super helpful and saves time.

however, if someone doesnt understand spring or doesnt understand mybatis they may get confused and not understand how the mapping is occuring. new devs will see the java interfaces (a thing they understand) but be confused by the total lack of implementation classes.

now once they understand spring + mybatis it becomes very obvious. this case is pretty normal though. you can do some massively hosed up poo poo in spring that will confuse the poo poo out of even the hardened dev. usually that stuff has to do w/ aop though.

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

0xB16B00B5 posted:

PHP is your teenage sweetheart, the girl you first awkwardly fumbled around with that one summer. Just don't try and start a more serious relationship - this girl has serious issues.

Perl is PHP's older sister. She might be a bit old for you, but she was pretty popular back in the 90s. She's looking pretty ugly and scarred now, so you don't hear from her too much.

Ruby is the cool kid of the scripting family. When you first saw her, she took your breath away with her beauty. She was fun, too. At the time she seemed a bit slow and ditzy - though she's matured a lot in the last few years.

Python is Ruby's sensible (and slightly more boring) sister.

Java is a successful career woman. What she lacks in raw intelligence she makes up for in dress - always turned out in immaculateCamelCase, sure to impress the enterprise customers. You might feel like she's the sensible type you should settle down with. Just prepare for years of "NO THAT DOESNT GO THERE GOD YOU ALWAYS USE THE WRONG TYPE AND YOU MISSED A SEMICOLON" nagging.

C++ is Java's cousin. Similar to Java in many ways, the main difference being she grew up in a more innocent time and doesn't believe in condoms/automatic memory management, so be cautious.

C is C++'s mom. Mention her name to some old grey beard hackers and they're sure to reminisce with a twinkle in their eye.

Objective C is another member of the C family. She joined that weird church a while back, and won't date anyone outside of it.

Haskell, Clojure, Scheme and their friends are those hipster, artsy, intellectual girls you probably spent a blissful college summer with a few years ago. The first girls who really challenged you. Of course, it could never have become something more serious (you tell yourself). Though you'll always be left asking "what if?"

You might be put off C# due to her family's reputation. But they've gone legit, the last few years, they tell you. Once you're one of us, you're one of us, you hear? You need a database? Her brother MSSQL will hook you up. Need a place to stay? Heck, her daddy will even buy you your own mansion on Azure avenue. What's that, you're having second thoughts about all these overly friendly relatives? No, you can never leave. You're part of the family, now, ya hear?

Javascript - hey, wasn't that the girl you first kissed, way before even PHP came on the scene? I wonder what she's doing now. I hear her career's really taken off in the last few years. Would be cool to catch up some time, if only for old time's sake... (You catch sight of her, dressed head to toe in designer jQuery)... wow, somebody grew into a beautiful swan...

tef
May 30, 2004

-> some l-system crap ->

Tiny Bug Child posted:

they can use their name they just can't use weird characters

https://www.youtube.com/watch?v=zyu2jAD6sdo

Catalyst-proof
May 11, 2011

better waste some time with you

0xB16B00B5 posted:

PHP is your teenage sweetheart, the girl you first awkwardly fumbled around with that one summer. Just don't try and start a more serious relationship - this girl has serious issues.

Perl is PHP's older sister. She might be a bit old for you, but she was pretty popular back in the 90s. She's looking pretty ugly and scarred now, so you don't hear from her too much.

Ruby is the cool kid of the scripting family. When you first saw her, she took your breath away with her beauty. She was fun, too. At the time she seemed a bit slow and ditzy - though she's matured a lot in the last few years.

Python is Ruby's sensible (and slightly more boring) sister.

Java is a successful career woman. What she lacks in raw intelligence she makes up for in dress - always turned out in immaculateCamelCase, sure to impress the enterprise customers. You might feel like she's the sensible type you should settle down with. Just prepare for years of "NO THAT DOESNT GO THERE GOD YOU ALWAYS USE THE WRONG TYPE AND YOU MISSED A SEMICOLON" nagging.

C++ is Java's cousin. Similar to Java in many ways, the main difference being she grew up in a more innocent time and doesn't believe in condoms/automatic memory management, so be cautious.

C is C++'s mom. Mention her name to some old grey beard hackers and they're sure to reminisce with a twinkle in their eye.

Objective C is another member of the C family. She joined that weird church a while back, and won't date anyone outside of it.

Haskell, Clojure, Scheme and their friends are those hipster, artsy, intellectual girls you probably spent a blissful college summer with a few years ago. The first girls who really challenged you. Of course, it could never have become something more serious (you tell yourself). Though you'll always be left asking "what if?"

You might be put off C# due to her family's reputation. But they've gone legit, the last few years, they tell you. Once you're one of us, you're one of us, you hear? You need a database? Her brother MSSQL will hook you up. Need a place to stay? Heck, her daddy will even buy you your own mansion on Azure avenue. What's that, you're having second thoughts about all these overly friendly relatives? No, you can never leave. You're part of the family, now, ya hear?

Javascript - hey, wasn't that the girl you first kissed, way before even PHP came on the scene? I wonder what she's doing now. I hear her career's really taken off in the last few years. Would be cool to catch up some time, if only for old time's sake... (You catch sight of her, dressed head to toe in designer jQuery)... wow, somebody grew into a beautiful swan...

jesus christ ban

JawnV6
Jul 4, 2004

So hot ...

0xB16B00B5 posted:

PHP is your teenage sweetheart, the girl you first awkwardly fumbled around with that one summer. Just don't try and start a more serious relationship - this girl has serious issues.

Perl is PHP's older sister. She might be a bit old for you, but she was pretty popular back in the 90s. She's looking pretty ugly and scarred now, so you don't hear from her too much.

Ruby is the cool kid of the scripting family. When you first saw her, she took your breath away with her beauty. She was fun, too. At the time she seemed a bit slow and ditzy - though she's matured a lot in the last few years.

Python is Ruby's sensible (and slightly more boring) sister.

Java is a successful career woman. What she lacks in raw intelligence she makes up for in dress - always turned out in immaculateCamelCase, sure to impress the enterprise customers. You might feel like she's the sensible type you should settle down with. Just prepare for years of "NO THAT DOESNT GO THERE GOD YOU ALWAYS USE THE WRONG TYPE AND YOU MISSED A SEMICOLON" nagging.

C++ is Java's cousin. Similar to Java in many ways, the main difference being she grew up in a more innocent time and doesn't believe in condoms/automatic memory management, so be cautious.

C is C++'s mom. Mention her name to some old grey beard hackers and they're sure to reminisce with a twinkle in their eye.

Objective C is another member of the C family. She joined that weird church a while back, and won't date anyone outside of it.

Haskell, Clojure, Scheme and their friends are those hipster, artsy, intellectual girls you probably spent a blissful college summer with a few years ago. The first girls who really challenged you. Of course, it could never have become something more serious (you tell yourself). Though you'll always be left asking "what if?"

You might be put off C# due to her family's reputation. But they've gone legit, the last few years, they tell you. Once you're one of us, you're one of us, you hear? You need a database? Her brother MSSQL will hook you up. Need a place to stay? Heck, her daddy will even buy you your own mansion on Azure avenue. What's that, you're having second thoughts about all these overly friendly relatives? No, you can never leave. You're part of the family, now, ya hear?

Javascript - hey, wasn't that the girl you first kissed, way before even PHP came on the scene? I wonder what she's doing now. I hear her career's really taken off in the last few years. Would be cool to catch up some time, if only for old time's sake... (You catch sight of her, dressed head to toe in designer jQuery)... wow, somebody grew into a beautiful swan...

BTW you are a MALE and are SEXUALLY INTERESTED in FEMALE

Tiny Bug Child
Sep 11, 2004

Avoid Symmetry, Allow Complexity, Introduce Terror

yeah i guess it would be nice if i automatically stripped out the accents and umlauts and other floating letter doodads, but c'mon, you can't seriously expect an english site that has never been promoted in chinese in any way to work when you put chinese characters into it

tef
May 30, 2004

-> some l-system crap ->

gucci void main posted:

i think that we should gas this now poo poo thread before it gets to page 219 and shames yospos

you're a bad person

Shaggar
Apr 26, 2006

Cocoa Crispies posted:

not really, most nosql systems don't have stored procedures because the big features are in data storage, not data processing

what usually happens is wrapping the storage layer with an API service that provides the business logic and abstraction from app developers

it's like stored procedures but you can mix and match storage layers in the same api: store users and their relation to organizations in postgresql, their relation to other users in neo4j, their profile pics in riak cs, wall posts in riak kv, etc.

those are webservices in soa.

Blotto Skorzany
Nov 7, 2008

He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

JawnV6 posted:

BTW you are a MALE and are SEXUALLY INTERESTED in FEMALE

Blotto Skorzany
Nov 7, 2008

He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip
if you don't <3 cum get the h*ck out of twitter

MononcQc
May 29, 2007

Shaggar posted:

so thats how notes works. you can set a per database conflict resolution of either 1) refuse to le the user save a conflicing document or 2) let the user save a conflicint document and mark it as conflicing. then the user will usually delete the conflicting document and wonder where their changes went.

for transactions, is it still up to the application code to handle wrapping their code in transactions? is there a stored proc equivilent where you basically pass parameters to a statement (that then handles all the work) instead of orming?

I've never used Notes, but it's one possible strategy for conflict resolution. You tend to adapt them to your precise use case. As said above, Amazon would do it automatically, without user input, but it did require for the calling code to make a resolving decision.

Regarding stored procedures, Coacoa Crispies' answer is right, to the extent of my understanding. The API services will help, and sometimes the DB engine will give you a somewhat programmable database (see: "write a map-reduce query in Erlang, also go gently caress yourself" comic, CouchDB has stored Javascript procedures for queries (called views), Datomic has stored procedures, etc.)

Then yes, the software developer may usually tie all kinds of different storage within their own model layer in their application, hiding away as many implementation or storage details as possible.

There is no true uniform solution, because NoSQL basically just means "not relational" to some extent. You have object databases, key/value stores, document databases, some geospatial stores, or whatever, with different APIs available. They're hard to shove into one category.

Some of them (like VoltDB) tend to have full consistency and relational approaches the way most SQL databases do it. While they're often classified in 'NoSQL', the VoltDB guys like to be called 'NewSQL' for a more modern approach to the same design goals as SQL databases. They'll have a drastically different approach from, say, MongoDB, but still be packaged in the same group of databases.


Shaggar posted:

those are webservices in soa.

Yeah, they can be seen that way in many cases. This isn't a bad approach, as long as you don't get inside the ws-* world of endless specifications and architecture astronomy, I guess.

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Shaggar posted:

those are webservices in soa.

yeah exactly

split up the data storage and the data processing responsibilities into independent services so you can scale horizontally easier

∴ stored procedures are a relic and shouldn't be used

Adbot
ADBOT LOVES YOU

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde
haha wow way to be hysterical sweetheart :smuggo:

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply