|
What's a good master-master key value store? The list of DBs is too long and so far I see it's between Voldemort, Riak, ArangoDB, Tarantool, and MongoDB  . Having Java as a dependency is kinda poo poo though, there are a few built on Erlang which can be more self contained with the Beam runtime.
MrMoo fucked around with this message at 01:49 on Mar 30, 2017 |
#
?
Mar 30, 2017 01:26
|
|
|
|
| # ? May 20, 2024 20:37 |
|
|
lmao business logic in abstract classes gently caress you
|
|
#
?
Mar 30, 2017 02:39
|
|
|
Sapozhnik posted:hmm I wish you would disappear up your own rear end in a top hat ... and take youre posting with you!!!
|
|
#
?
Mar 30, 2017 03:06
|
|
|
MrMoo posted:What's a good master-master key value store? The list of DBs is too long and so far I see it's between Voldemort, Riak, ArangoDB, Tarantool, and MongoDB "It depends" Cassandra is one of the few things I don't have anything bad to say about despite being real strange. Evaluated riak once upon a time and it seemed good, but we ended up going with a clusterfuck of redis sentinel shards instead.
|
|
#
?
Mar 30, 2017 03:24
|
|
|
use redis or memcached, they are fine. treat them as a distributed cache, never a primary database. the source of truth should be a normal relational db
|
|
#
?
Mar 30, 2017 04:34
|
|
|
MrMoo posted:What's a good master-master key value store? The list of DBs is too long and so far I see it's between Voldemort, Riak, ArangoDB, Tarantool, and MongoDB uh so what are your constraints? does it have to be embedded are you ok with a hosted solution how many keys, how much read and write tps do you need anything more than get from a key and put to a key
|
|
#
?
Mar 30, 2017 04:57
|
|
|
how do you feel about consistency? how prominently do asteroids obliterating a minority of your datacenters figure into your risk assessment?
|
|
#
?
Mar 30, 2017 05:37
|
|
|
fritz posted:put in the god drat parentheses you animal No.
|
|
#
?
Mar 30, 2017 07:07
|
|
|
MrMoo posted:What's a good master-master key value store? The list of DBs is too long and so far I see it's between Voldemort, Riak, ArangoDB, Tarantool, and MongoDB what others said about "what exactly are your needs" but honestly you'll probably end up with either riak or cassandra absolutely do not use mongodb
|
|
#
?
Mar 30, 2017 07:13
|
|
|
basho is winding down, so i wouldn't use riak
|
|
#
?
Mar 30, 2017 07:36
|
|
|
the talent deficit posted:basho is winding down, so i wouldn't use riak yeah rip (I left in Jan. 2016)
|
|
#
?
Mar 30, 2017 10:45
|
|
|
HoboMan posted:q motherfucking q Same. Methinks it's time to dust off the ol' guillotine.
|
|
#
?
Mar 30, 2017 11:34
|
|
|
oh my god i have explained this one issue to a coworker five times now. we have old pentaho 3.0.0 report files. the new 7.0.0 server can read them and generate reports, yay. unfortunately we can't update our report file format without making changes to our java code. the java code parses the xaction file generated from the report file, extracts the sql query inside the xaction, does some regexp manipulation to the sql to feed it to hibernate, runs the query to determine how many individual reports need to be generated. (i don't understand this bit 100%, but the result is a list of report numbers) then it requests the pentaho server to generate an individual report for each report number in the list. but for some reason the coworker just won't understand that if we convert our old report files to the "new" (8 year old) prpt format, then there are no xaction files anymore and no more sql to extract and run, so our code needs to be changed. like i understand that the process i explained above is insane, but the key points are "the old file is an xml file containing sql" and "the new file is a zip containing xml, containing sql"
|
|
#
?
Mar 30, 2017 13:28
|
|
|
It's really remarkable how quickly basho destroyed itself. I still laugh at "data gravity"
|
|
#
?
Mar 30, 2017 14:01
|
|
|
Wheany posted:pentaho but enough about your mother
|
|
#
?
Mar 30, 2017 14:14
|
|
|
FamDav posted:uh so what are your constraints? It's all of two systems locked down like Fort Nox on the NYSE floor. Required TPS is like all of 200/s read and can be slower on write 🤠 So not embedded, not hosted, limit to GET and PUT verbs with the value being a JSON object. Key count is < 1,000. Eventual consistency is fine. Primary constraint is stability and auto-recovery. Contents is basically configuration of each screen on the floor, e.g. code:
MrMoo fucked around with this message at 14:43 on Mar 30, 2017 |
|
#
?
Mar 30, 2017 14:37
|
|
|
does mongodb still not support taking backups of data without writing a script that will shut down multiple nodes simultaneously, so you get "as close as possible" to being shut down at the same time, so you can have a "maybe consistent" backup?
|
|
#
?
Mar 30, 2017 14:39
|
|
|
CRIP EATIN BREAD posted:does mongodb still not support taking backups of data without writing a script that will shut down multiple nodes simultaneously, so you get "as close as possible" to being shut down at the same time, so you can have a "maybe consistent" backup? just use marklogic
|
|
#
?
Mar 30, 2017 15:04
|
|
|
jfc why did i do this SPA using 100% jquery ajax updating it is a loving paiiiinnnn i pity the person that eventually has to deal with the cobbled together mvc/webApi/REST pile that this is going to become. oh wait that's me.
|
|
#
?
Mar 30, 2017 15:44
|
|
|
MrMoo posted:It's all of two systems locked down like Fort Nox on the NYSE floor. Required TPS is like all of 200/s read and can be slower on write 🤠 not knowing all the options at hand and what the rest of your infrastructure is like/what tools you got, i wouldn't bother with master-master because you're probably ok with decreased availability on writes. if you have access to an object store, use an object store. i know groups that store hundreds of thousands of configuration blobs in s3.
|
|
#
?
Mar 30, 2017 16:19
|
|
|
FamDav posted:not knowing all the options at hand and what the rest of your infrastructure is like/what tools you got, i wouldn't bother with master-master because you're probably ok with decreased availability on writes. Master-master is critical for load sharing of market data flowing through it. If one system fails the floor still needs to be able to make changes, production software changes can only happen out-of-hours, with bigger changes on the weekend-only. That would allow for an out-of-hours resynchronization process but it would be nice for everything to be automatic and not involve the operations team at all, because they're already making GBS threads themselves over the timeline on this project already.
|
|
#
?
Mar 30, 2017 16:39
|
|
|
is that pulling configuration data only or configuration and market data?
|
|
#
?
Mar 30, 2017 16:47
|
|
|
Use cassandra, it's maybe a little overkill, but it will do what you need really well. You have to spend a bit of time setting up compaction and maintenance jobs, but it's chill and stable after that. You can use http://pithos.io/ with it if you'd prefer an s3 interface and don't want to gently caress with cql. Both are p good and I use them in production.
|
|
#
?
Mar 30, 2017 16:54
|
|
|
Shaggar posted:is that pulling configuration data only or configuration and market data? The DB is display configuration only, the rest of the components (the Router below) on each machine are processing market data and the end consumer accesses everything through a single transport. I basically want to minimize the machines talking to each other. If you had the direct consumer of the display configuration data (Admin Handler below) talking to both instances then you could have a master-slave deployment and after hours resync. A challenging option would be a notification mechanism for consumer connected to the other system when an update has been pushed. 
|
|
#
?
Mar 30, 2017 17:59
|
|
|
The device team are using a MySQL cluster for provision of their configuration data, whatever that includes, I am ideally looking for something a little lighter.
|
|
#
?
Mar 30, 2017 18:04
|
|
|
MrMoo posted:The DB is display configuration only, the rest of the components (the Router below) on each machine are processing market data and the end consumer accesses everything through a single transport. when you say machines talking to each other you mean the database machines right? not the consumers talking to each other? How often is the display configuration updated and what are the latency requirements? e: tbh I don't really have any great ideas for you or anything, I'm just interested in the problem.
|
|
#
?
Mar 30, 2017 18:29
|
|
|
Shaggar posted:when you say machines talking to each other you mean the database machines right? not the consumers talking to each other? How often is the display configuration updated and what are the latency requirements? Correct, the database hosts. Naturally the consumers don't talk to each other either. It was on the option list though: use WebRTC to stream data among peers, but the additional complexity of handling the peer that is receiving data from upstream dying was not attractive. Displays are arbitrarily updated by the clerks on the floor during the day, it depends on the market and who is visiting the floor. Latency requirement is basically human patience.
|
|
#
?
Mar 30, 2017 20:11
|
|
|
just use sql server, imo. alternatively Casandra would probably be fine but its still NoSQL do updates to the configuration come from the clients themselves or are they sent from a central admin?
|
|
#
?
Mar 30, 2017 20:52
|
|
|
HoboMan posted:q motherfucking q 
|
|
#
?
Mar 30, 2017 21:35
|
|
|
a witch posted:It's really remarkable how quickly basho destroyed itself. "quickly" investors replaced upper management with bozo fail upward types, what, three, four years ago?
|
|
#
?
Mar 30, 2017 22:24
|
|
|
I'm counting the destruction as roughly the time Tom Santero and cmeik left, but that's an outsider perspective
|
|
#
?
Mar 30, 2017 22:40
|
|
|
for two straight days I was trying to figure out why a daemon wasn't working when everything indicated it should. then it randomly started working, seemingly unrelated to anything I'd changed, so I lost two entire days and didn't even learn anything useful. kill me
|
|
#
?
Mar 31, 2017 01:14
|
|
|
and best of all you will never ever be completely certain that things won't break without any reason in the future. i love that kind of garbage
|
|
#
?
Mar 31, 2017 02:46
|
|
|
a witch posted:I'm counting the destruction as roughly the time Tom Santero and cmeik left, but that's an outsider perspective santero and cmeik were years apart santero was one of the first, his name didn't autocomplete in hipchat one morning i think cmeik left some time in 2015, spent some time at machine zone (they make some f2p shitfest), and now does academia full time, which rules 'cause he's working on cool poo poo
|
|
#
?
Mar 31, 2017 04:05
|
|
|
someone posted itt a website where you write programs to buy and sell items on a virtual exchange and now I can't find it, can anyone remember what it is? tia
|
|
#
?
Mar 31, 2017 10:43
|
|
|
been looking at datomic (ACID nosql database that uses datalog to query + event sourcing) and it's pretty cool but just found this poo poo in their license agreement:http://www.datomic.com/datomic-pro-edition-eula.html posted:The Licensee hereby agrees, without the prior written consent of Cognitect, which may be withheld or conditioned at Cognitect's sole discretion, it will not:
|
|
#
?
Mar 31, 2017 22:46
|
|
|
programming would be so great if there were no such thing as business logic
|
|
#
?
Mar 31, 2017 22:59
|
|
|
St Evan Echoes posted:someone posted itt a website where you write programs to buy and sell items on a virtual exchange and now I can't find it, can anyone remember what it is? tia found it in browser history, it was called stockfighter and it's dead now 
|
|
#
?
Mar 31, 2017 23:00
|
|
|
drop ACID not logic bombs https://www.youtube.com/watch?v=so0BXGEVuoE
|
|
#
?
Mar 31, 2017 23:01
|
|
|
|
| # ? May 20, 2024 20:37 |
|
|
MALE SHOEGAZE posted:programming would be so great if there were no such thing as business logic the perfect program is one that doesn't exist at all. honestly feels like i solve half my problems by pushing back at requirements
|
|
#
?
Mar 31, 2017 23:04
|
|
