Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

ToxicFrog: Apr 26, 2008

it is posted:

Windows. I'm trying to make an AI that can press keyboard keys in relation to graphical input. I don't really care about the language, since it mostly involves shuffling arrays around, which isn't hard in any language.

I guess I'm just asking how to take like 60 screenshots a second, and it's weird that you can't just snoop on whatever is telling my monitor what colors to show me.

You can, that's how screen/game streaming software like OBS or Steam Home Streaming work.

# ? Jul 11, 2016 01:35

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 06:46

mystes: May 31, 2006

ToxicFrog posted:

You can, that's how screen/game streaming software like OBS or Steam Home Streaming work.

Well actually you can't do it quite like the extremely efficient way Steam does it it on Nvidia cards (NVFBC) unless you pay $$$ to Nvidia.

# ? Jul 11, 2016 01:53

pseudorandom name: May 6, 2007

https://blogs.msdn.microsoft.com/dsui_team/2013/03/25/ways-to-capture-the-screen/

# ? Jul 11, 2016 07:25

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

Another security question (do/should we have a general security questions thread?):

We'd like to implement a "Remember me on this computer" feature for our Windows desktop app, which uses a custom user database for authentication. The plan is that on a successful login the server will generate a signed token that contains at least the user's ID and a security stamp. The security stamp will be changed every time the user's password is changed, and probably also in some other cases. The server passes the token back to the client, and the client stores it somewhere in the user's home directory. The next time the user opens the app the client retrieves the persisted token and calls a LoginByToken server method instead of the regular one. The server decrypts the token and verifies that the security stamp matches what's in the database. If it does, the user is logged in.

Please tell me about all the fatal flaws in this plan.

# ? Jul 11, 2016 11:13

mystes: May 31, 2006

LOOK I AM A TURTLE posted:

The plan is that on a successful login the server will generate a signed token that contains at least the user's ID and a security stamp... The server decrypts the token and verifies that the security stamp matches what's in the database. If it does, the user is logged in.

Please tell me about all the fatal flaws in this plan.

Nothing but it's unnecessarily complicated. Rather than using encryption, which is easier to screw up, just generate a random code (the "security stamp") and use that. If you're storing the code on your server there's no reason for a cryptographic signature (and not storing the codes and just relying on signature would probably also be a bad idea, since there would be no easy way for you to revoke them).

# ? Jul 11, 2016 11:20

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

mystes posted:

Nothing but it's unnecessarily complicated. Rather than using encryption, which is easier to screw up, just generate a random code (the "security stamp") and use that. If you're storing the code on your server there's no reason for a cryptographic signature (and not storing the codes and just relying on signature would probably also be a bad idea, since there would be no easy way for you to revoke them).

Ok, fair enough. Right now the security stamps are GUIDs generated by SQL Server NEWID function in a trigger. Would that be considered sufficiently random?

# ? Jul 11, 2016 11:54

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

pokeyman posted:

Erlang uses =< and >=

I think they're inherited from prolog.

(I use neither of these languages so someone's free to tell me how wrong I am.)

They're inherited from mathematics. :smugbert:

But yes, prolog does that too.

# ? Jul 11, 2016 12:08

nielsm: Jun 1, 2009

LOOK I AM A TURTLE posted:

Ok, fair enough. Right now the security stamps are GUIDs generated by SQL Server NEWID function in a trigger. Would that be considered sufficiently random?

GUIDs are not actually/not guaranteed random. You should still use a cryptgraphic randomness generator.

# ? Jul 11, 2016 12:44

ToxicFrog: Apr 26, 2008

mystes posted:

Well actually you can't do it quite like the extremely efficient way Steam does it it on Nvidia cards (NVFBC) unless you pay $$$ to Nvidia.

SHS works fine on AMD and Intel cards as well, as does other streaming software, so clearly this is a solved problem :shrug:

# ? Jul 11, 2016 13:04

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

nielsm posted:

GUIDs are not actually/not guaranteed random. You should still use a cryptgraphic randomness generator.

I know GUIDs aren't particularly random (especially not if you use sequentially generated ones, though I wasn't going to). For the record my original conception of the token algorithm was based on the approach taken in ASP.NET Identity: https://github.com/aspnet/Identity/blob/dev/src/Microsoft.AspNetCore.Identity/DataProtectionTokenProvider.cs#L67 The difference is that that API is written to work either with or without a security stamp, and if there's no security stamp it's obviously necessary for the token to be signed, otherwise anyone could trivially generate a valid token.

# ? Jul 11, 2016 14:25

mystes: May 31, 2006

http://stackoverflow.com/questions/13817452/how-can-i-generate-a-cryptographically-secure-number-in-sql-server

Use CRYPT_GEN_RANDOM

# ? Jul 11, 2016 15:09

CarrKnight: May 24, 2013

I have a simple java program that takes a "input.yaml" file, crunches it for a bit and spits out a "results.yaml" file.
I have a bunch of these files I want processed.
My idea would be to deploy this program over a bunch of rented EC2 machines, let them do the work and send me back the various "results.yaml" files I need.

What I'd really like is a proper book/tutorial on a platform that lets me do this painlessly. I'd like to spin up machines with both java and my program already loaded and perhaps automate the sending of inputs files as well as their retrieval.
Where should I start my journey?

# ? Jul 11, 2016 15:24

it is: Aug 19, 2011; by Smythe

pseudorandom name posted:

https://blogs.msdn.microsoft.com/dsui_team/2013/03/25/ways-to-capture-the-screen/

This is pretty helpful, thanks!

Why isn't this a basic operation? Why are the pixels on the screen something with a million different paths to get to? Why isn't grabbing the pixels something you just, like, can do without thinking about it? Why is "show me these pixels" different enough from "show me the value of x" that it requires a whole bunch of setup and teardown?

# ? Jul 11, 2016 16:50

ultrafilter: Aug 23, 2007; It's okay if you have any questions.

Is there a good quick introduction to Excel's dialect of Visual Basic? I'm trying to reimplement an algorithm that's in that language, and it's tough going cause it's kinda weird.

# ? Jul 11, 2016 16:55

csammis: Aug 26, 2003; Mental Institution

it is posted:

This is pretty helpful, thanks!

Why isn't this a basic operation? Why are the pixels on the screen something with a million different paths to get to? Why isn't grabbing the pixels something you just, like, can do without thinking about it? Why is "show me these pixels" different enough from "show me the value of x" that it requires a whole bunch of setup and teardown?

You're thinking about "the screen" like it's a single raster space. From the hardware and graphics driver's perspective that's a fine viewpoint but from Windows's perspective and an application's perspective that's too simplistic. Each windowed application has its own screen space which it is responsible to draw and that is all it knows about. The window manager tells each application when it is time to draw, when it's being occluded by another window, and so on. The window manager puts everything together at a low level and updates the screen buffer so that the monitor can draw it (or whatever output is being used, like Remote Desktop) This is a purposeful design so that one application doesn't need to worry about all other applications in the system. If one application wants to reach out to another application and figure out what's going on there then that takes effort.

# ? Jul 11, 2016 17:00

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

it is posted:

Why isn't this a basic operation? Why are the pixels on the screen something with a million different paths to get to? Why isn't grabbing the pixels something you just, like, can do without thinking about it? Why is "show me these pixels" different enough from "show me the value of x" that it requires a whole bunch of setup and teardown?

Keep in mind that the screen data (that is, the color of each pixel) doesn't really exist anywhere except at the very end of the graphics rendering pipeline. The OS sends structured data to the graphics card (stuff like "this rectangle here has these drawing commands and transformations applied to it" repeated once for each window or widget you can see), and the graphics card has a very highly-optimized pipeline...for applying those commands and transformations and blitting the results to the screen. Graphics cards were not really made to send data back to the host computer. Getting them to do that thus requires some trickery, hence the hoops you have to jump through.

(This is incidentally one reason why using graphics cards to perform highly-parallelized vector operations in scientific research took a little while to get up and running: the lack of good mechanisms for cards to output data anywhere but to a display)

# ? Jul 11, 2016 17:10

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

There are also security concerns. You don't really want just anything to be able to read any part of the display, but that's the legacy we're stuck with so far.

# ? Jul 11, 2016 17:14

Bob Morales: Aug 18, 2006; ~~Just wear the fucking mask, Bob~~

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Munkeymon posted:

There are also security concerns. You don't really want just anything to be able to read any part of the display, but that's the legacy we're stuck with so far.

Don't forget DRM!

# ? Jul 11, 2016 18:30

overeager overeater: Oct 16, 2011; "The cosmonauts were transfixed with wonderment as the sun set - over the Earth - there lucklessly, untethered Comrade Todd on fire."

I'm looking for an online tool I forgot to bookmark. It let you input a bitwise transformation as a set of bit offsets - "4, 3..." would map the fourth bit to the first, the third to the second and so on - and it would show you a visualization of the transformation, as well as the most efficient way to do it in x86 assembly.

I've been searching for various permutations of "bitwise optimization tool" etc. but all that seems to turn up is SO questions and JS calculators

# ? Jul 11, 2016 19:22

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

csammis posted:

You're thinking about "the screen" like it's a single raster space. From the hardware and graphics driver's perspective that's a fine viewpoint but from Windows's perspective and an application's perspective that's too simplistic. Each windowed application has its own screen space which it is responsible to draw and that is all it knows about. The window manager tells each application when it is time to draw, when it's being occluded by another window, and so on. The window manager puts everything together at a low level and updates the screen buffer so that the monitor can draw it (or whatever output is being used, like Remote Desktop) This is a purposeful design so that one application doesn't need to worry about all other applications in the system. If one application wants to reach out to another application and figure out what's going on there then that takes effort.

This isn't the case in anything past Windows Vista, which added a desktop compositor -- all window contents are now drawn in offscreen buffers, which are then used as textures. One of the ways in which you can capture the screen is to simply get the backing offscreen buffer for every window, but that becomes more complex if you resize the window... you need an event to let you know when the buffer is invalidated, and then you can have race conditions, etc.

TooMuchAbstraction posted:

Keep in mind that the screen data (that is, the color of each pixel) doesn't really exist anywhere except at the very end of the graphics rendering pipeline. The OS sends structured data to the graphics card (stuff like "this rectangle here has these drawing commands and transformations applied to it" repeated once for each window or widget you can see), and the graphics card has a very highly-optimized pipeline...for applying those commands and transformations and blitting the results to the screen. Graphics cards were not really made to send data back to the host computer. Getting them to do that thus requires some trickery, hence the hoops you have to jump through.

(This is incidentally one reason why using graphics cards to perform highly-parallelized vector operations in scientific research took a little while to get up and running: the lack of good mechanisms for cards to output data anywhere but to a display)

Well, that's not true either. Your GPU has memory built-in and the full composited buffer goes there, so it's not like the pixels are just made up in thin air -- they're there, in your GPU memory. Your limiting factor is actually the memory bandwidth here. Getting pixel data might not sound like a lot, but 1920x1060x4, 60 frames per second is actually around half a gigabyte of data per second, which is ridiculous for a PCIe bus.

People have been doing GPGPUs for years by doing texture readback. One of the major issues is actually getting data into the card, since OpenGL isn't designed for terabytes of data to be read into it -- drivers are required to make copies of any buffers you provide to them.

# ? Jul 11, 2016 19:47

mystes: May 31, 2006

ultrafilter posted:

Is there a good quick introduction to Excel's dialect of Visual Basic? I'm trying to reimplement an algorithm that's in that language, and it's tough going cause it's kinda weird.

There used to be books, probably. Googling specific things (with the names of equivalents in other languages, for example) is usually the best bet.There are still some websites abut it and the language itself is the same as vb6. When you're not dealing with the excel object model its pretty simple but extremely limited. It only becomes a nightmare when you have to interact with excel objects in complicated ways.

I've had the misfortune of trying to use it lately so if you have any specific questions I might be able to help.

For data-intensive stuff that doesnt need to directly interact with the object model you might want to look into making an xll plugin with excel DNA if possible; it can be a lot easier than doing everything in VBA.

mystes fucked around with this message at 20:35 on Jul 11, 2016

# ? Jul 11, 2016 20:30

Rigged Death Trap: Feb 13, 2012; BEEP BEEP BEEP BEEP

I've been reading through The Art of Computer Programming, and though it has it's moments of higher maths that I just can't get some exercises have been fun to do.
After I've done one of them I go over to the answer, see if I did better or worse than what the book suggests (usually worse), But I don't get what the answer for the current one im on is trying to convey at all
The book's question and proposed solution are found here.

What I've understood is that t may be the number of 'missing' elements, but what I really don't understand is the seemingly arbitrary instruction to sort them based on their components, which to me seems like spinning your wheels in place. And the decision(among others) to replace one part of the paired names with a number ( file G contains all pairs (i,xi) for n−t<i≤n)
Thanks for any help.

# ? Jul 12, 2016 12:34

it is: Aug 19, 2011; by Smythe

Thanks y'all! It's good to know where some of my bad simplifying assumptions were.

# ? Jul 12, 2016 16:36

HappyHippo: Nov 19, 2003; Do you have an Air Miles Card?

Rigged Death Trap posted:

I've been reading through The Art of Computer Programming, and though it has it's moments of higher maths that I just can't get some exercises have been fun to do.
After I've done one of them I go over to the answer, see if I did better or worse than what the book suggests (usually worse), But I don't get what the answer for the current one im on is trying to convey at all
The book's question and proposed solution are found here.

What I've understood is that t may be the number of 'missing' elements, but what I really don't understand is the seemingly arbitrary instruction to sort them based on their components, which to me seems like spinning your wheels in place. And the decision(among others) to replace one part of the paired names with a number ( file G contains all pairs (i,xi) for n−t<i≤n)
Thanks for any help.

Sorting them helps you find the (x_i, x_(i+t)) pairs. For example if the sequence was [D, C, B, F, A, E] and you had the initial pairs [(B, F), (D, C), (F, A), (C, B), (A, E)], then sorting by first and second gives the lists:

code:

[(A, E), (B, F), (C, B), (D, C), (F, A)]
[(F, A), (C, B), (D, C), (A, E), (B, F)]

Now look at these pairs in sequence, first we have (A, E) and (F, A). This tells us that we must have the sequence F,A,E, so E is two steps after F, so you write (F, E) in the new file. The next two are (B, F) and (C, B), (F is two steps after C) so you write (C, F) in the new file. This continues, however note that (D, C), (A, E) don't share any names because they are the start and end of the sequence. (D, C) must be the start of the sequence, because if D was proceeded by anything it would have been at this point in the second list. And therefore (A, E) must be the end of the list. This lets you write (6, E) and (5, A) to the file of positions (file G, once you have that file filled out you're done). I didn't quite follow how he handles the position file but hopefully this helps. Also the answer appears to use n and N interchangeably so keep that in mind.

# ? Jul 12, 2016 18:45

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

You've gotta use the flowchart!

# ? Jul 12, 2016 21:21

Rigged Death Trap: Feb 13, 2012; BEEP BEEP BEEP BEEP

Those are exactly what I needed. Thank you both.

[e]:Argh I hit a snag.

code:

while (j < F.size())
	{
		if (F.at(j).second == H.at(l).first)
		{
			F_.push_back({ F.at(j).first, H.at(l).second });
			j++;
			l++;
		}
		else if (F.at(j).second == G.at(k).second)
		{
			G_.push_back({ G.at(k).first - t, F.at(j).first });
			j++;
			k++;
		}
		else if (F.at(j).second > G.at(k).second)	{k++;}
		else if (F.at(j).second > H.at(l).first)	{l++;}

	}

j,k and l 'simulate' a magnetic head reading sequentially.

and that block that does this part:

quote:

1. If x′=zx′=z, output (x,z′)(x,z′) to F′F′ and advance files FF and HH.
2. If x′=y′x′=y′, output (y−t,x)(y−t,x) to G′G′ and advance files FF and GG.
3. If x′>y′x′>y′, advance file GG.
4. If x′>zx′>z, advance file HH.

And it's fine, but when it comes this other part:

quote:

When file F is exhausted, sort G′ by second components and merge G with it; then replace t by 2t, F by F′, G by G′. Thus t takes the values 2, 4, 8,��->N; and for fixed t we do O(logN) passes over the data to sort it. Hence the total number of passes is O((logN)^2). Eventually t ≥ N, so F is empty, then we simply sort G on its first components.

when I implement that this way:

code:

	std::stable_sort(G_.begin(), G_.end(), [](const NumName &a, const NumName &b){return a.second < b.second; });
	std::merge(G.begin(), G.end(), G_.begin(), G_.end(), std::back_inserter(temp), 
		[](const NumName &a, const NumName &b){return a.second < b.second; });

	F = F_;
	G = temp;
	F_.clear();
	G_.clear();

	t *= 2;
	j = 0, k=0 ,l=0;

I've tried wrapping my head around it, brute forcing it by zeroing the j/k/l, but always as soon as I hit the third iteration duplicates start showing up in G', and a nagging feeling tells me it's due to H.
'when file F is exhausted' implies iteration or some kind of repetition, but goddamnit this 50 year old answer is vague as hell to me.

tldr: Stuck in the 12-13 loop

Rigged Death Trap fucked around with this message at 18:37 on Jul 13, 2016

# ? Jul 12, 2016 22:10

wargames: Mar 16, 2008; official yospos cat censor

So I am trying to change a mybb forums plugin, but the result has some error that I can't find, can I get a another pair of eyes to what I am missing.

This works great and is awesome, and is the base for my changes.
http://pastebin.com/wWvacVww

This is my changes and it doesn't allow for new post when I use it.
http://pastebin.com/WVmvc1BN

wargames fucked around with this message at 07:54 on Jul 15, 2016

# ? Jul 15, 2016 07:52

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Is there a name for the sorting algorithm that just takes a function to extract a value from each element in a list rather than one that takes an actual comparison function? I'm pretty much talking about the difference between the way sorted works in Python 2 and 3. I see why the key function approach is more efficient, but it can be a lot more annoying to figure out a good key function than to slap together a comparison function that does exactly what I want when working in a language without cmp_to_key

# ? Jul 15, 2016 14:43

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

Munkeymon posted:

Is there a name for the sorting algorithm that just takes a function to extract a value from each element in a list rather than one that takes an actual comparison function? I'm pretty much talking about the difference between the way sorted works in Python 2 and 3. I see why the key function approach is more efficient, but it can be a lot more annoying to figure out a good key function than to slap together a comparison function that does exactly what I want when working in a language without cmp_to_key

Not that I'm aware of, because it's not a change in the sorting algorithm per se, just in how that algorithm accesses the data it needs for performing comparisons. Presumably they use the same logic for which elements to compare and how to shuffle them around, under the hood; the key-based approach is just able to make use of caches (and minimize function calls) because it knows that its comparison function is consistent.

# ? Jul 15, 2016 14:58

ExcessBLarg!: Sep 1, 2001

Munkeymon posted:

Is there a name for the sorting algorithm that just takes a function to extract a value from each element in a list rather than one that takes an actual comparison function?

In Ruby this is the distinction between Enumerable's sort (using a comparator) and sort_by (using a mapper) methods.

# ? Jul 15, 2016 17:33

ExcessBLarg!: Sep 1, 2001

Edit: Double post.

# ? Jul 15, 2016 17:33

pr0k: Jan 16, 2001; "Well if it's gonna be
that kind of party..."

My project is getting kicked out of our lovely splunk instance. We need log handling/parsing/searching.

Right now we are using syslog-ng to ship logs to splunk. I don't really know poo poo about that other than splunk reads that poo poo raw somehow.

We're looking at either

1. AWS ElasticSearch instance. LogStash to do the parsing. Kibana for the interface (what exactly? I don't like sense at all. I just want a simple interface for my tech support guys to look up client numbers or transaction numbers.)

or

2. Amazon has a thing called "AWS CloudWatch Logs" that looks like it would give us a decent basic interface for our app logs.

The logs are coming from a basic log4j spooler in a groovy/grails app running on S3.

Any thoughts or pointers you guys had would be appreciated. This is a medium-term solution, in that I need something right goddamn now, but I can't get the devops people to spec what the solution *should* be yet, other than "something on the ETK stack."

# ? Jul 15, 2016 20:16

LP0 ON FIRE: Jan 25, 2006; _{beep boop}

Databases..

I've been called upon to do some research on what database to use for upcoming projects. Primarily we are looking for speed, read and write. Probably NoSQL, non-relational database. What are your opinions on the best to use? My boss likes Cassandra and CouchDB, but I'm intrigued by Kyoto's language bindings.

# ? Jul 15, 2016 21:58

nielsm: Jun 1, 2009

LP0 ON FIRE posted:

Databases..

I've been called upon to do some research on what database to use for upcoming projects. Primarily we are looking for speed, read and write. Probably NoSQL, non-relational database. What are your opinions on the best to use? My boss likes Cassandra and CouchDB, but I'm intrigued by Kyoto's language bindings.

PostgreSQL is fast too.

Consider your data model first, and what kinds of consistency you require, both in case of crashes, and in case you need to do replication/clustering. Pretty much every document database supposedly has issues with consistency and real risks of data loss in replication.

Don't dismiss technologies before benchmarking them for your specific workload.

# ? Jul 15, 2016 22:40

sarehu: Apr 20, 2007; (call/cc call/cc)

Munkeymon posted:

Is there a name for the sorting algorithm that just takes a function to extract a value from each element in a list rather than one that takes an actual comparison function?

That's just an API. With that API you could do a "Schwatzian transform" as Perlers might call it, there's a Wikipedia page. cmp_to_key's return value presumably throws the objects into a wrapper type that implements the desired comparison, you can always implement such a thing yourself.

# ? Jul 15, 2016 23:17

sarehu: Apr 20, 2007; (call/cc call/cc)

LP0 ON FIRE posted:

Databases..

I've been called upon to do some research on what database to use for upcoming projects. Primarily we are looking for speed, read and write. Probably NoSQL, non-relational database. What are your opinions on the best to use? My boss likes Cassandra and CouchDB, but I'm intrigued by Kyoto's language bindings.

By speed do you mean throughput or latency? For what operations? How large is your dataset? How many concurrent queries? How much I/O? Under what consistency guarantees? Will it run on a single machine or be replicated? Beware that most NoSQL databases are garbage under some conditions or others. And you do need to give specific answers to these questions, with numbers -- they aren't rhetorical questions.

Edit: You say "projects" so I'm going to say PostgreSQL as your generic answer too. For NoSQL I'd recommend RethinkDB for usability and reliability but for performance IIRC you can only expect ~ten thousand writes per second on a given machine, so it's not that good at slurping up large datasets and there's big file size overhead.

sarehu fucked around with this message at 23:34 on Jul 15, 2016

# ? Jul 15, 2016 23:25

MrMoo: Sep 14, 2000

memSQL is pretty hard to top on basic performance, however good luck getting anything but basic queries working.

# ? Jul 16, 2016 00:13

LP0 ON FIRE: Jan 25, 2006; _{beep boop}

nielsm posted:

PostgreSQL is fast too.

Consider your data model first, and what kinds of consistency you require, both in case of crashes, and in case you need to do replication/clustering. Pretty much every document database supposedly has issues with consistency and real risks of data loss in replication.

Don't dismiss technologies before benchmarking them for your specific workload.

sarehu posted:

By speed do you mean throughput or latency? For what operations? How large is your dataset? How many concurrent queries? How much I/O? Under what consistency guarantees? Will it run on a single machine or be replicated? Beware that most NoSQL databases are garbage under some conditions or others. And you do need to give specific answers to these questions, with numbers -- they aren't rhetorical questions.

Edit: You say "projects" so I'm going to say PostgreSQL as your generic answer too. For NoSQL I'd recommend RethinkDB for usability and reliability but for performance IIRC you can only expect ~ten thousand writes per second on a given machine, so it's not that good at slurping up large datasets and there's big file size overhead.

Yeah these are definitely super important things to know, and I also requested to know exactly what kinds of things we are doing before we make a choice. So far it looks like lots of queries writing tiny bits of data, and at times querying a ton of that data to create graphs. So the data as in size isn't a lot, but very busy and lots of it being written.

MrMoo posted:

memSQL is pretty hard to top on basic performance, however good luck getting anything but basic queries working.

Good to know. Makes me scared if I want complicated queries at times.

# ? Jul 16, 2016 03:33

Gravity Pike: Feb 8, 2009; I find this discussion incredibly bland and disinteresting.

By any chance, are all of the graphs going to be of sequential data? If so, a time-series database (like InfluxDB) may be a good choice. They have fast insert times, and are very fast to return or aggregate sequential data, but perform poorly with more general searches.

# ? Jul 16, 2016 04:35

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 06:46

Jewel: May 2, 2009

What kind of db could a an mmo like WoW or eve be using? I know there's a lot of instancing and caching involved but it still seems like there's a high data throughput. Unsure if complicated queries are necessary, though.

# ? Jul 16, 2016 12:11

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »