C/C++ Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›641 »

OneEightHundred: Feb 28, 2008; Soon, we will be unstoppable!

I guess the funny thing about aliasing is that the most technically correct solution to forcing the compiler to alias a block of memory without doing an actual copy is to memmove the data to the same address and pray it just deletes the memmove.

# ? Jan 26, 2018 04:04

Adbot: ADBOT LOVES YOU

# ? Jun 7, 2024 10:45

Jeffrey of YOSPOS: Dec 22, 2005; GET LOSE, YOU CAN'T COMPARE WITH MY POWERS

Jabor posted:

The fact that the "reinterpret the opaque sequence of bytes as a different type" function is called "memcpy" is an unfortunate historical oddity, but it still does what you're actually trying to accomplish in most cases.

Essentially, you can write correct code that also happens to be optimally fast if your compiler works the way you expect it to, or you can write fast code that is only correct if your compiler works the way you expect it to. So what makes the second one preferable to you, again?

I find it odd to treat fno-strict-aliasing as some mysterious and untrustworthy compiler feature not to be relied upon while memcpy elision is to be taken for granted. If my options are fast, zero-copy code or nothing, silent fallback to the slow version is no consolation prize. As far as I'm concerned, my deliverable is what the compiler outputs when it runs on my code and if that include copies of buffers where it shouldn't, it's wrong. Losing aliasing optimizations in other places is unfortunate here and I'd gladly accept a version of C that lets one specify this stuff explicitly, but that's not the universe we're in as far as I know. (attribute ((may_alias)) sounds interesting, maybe I should use that.) I'm far more trusting of the compiler not to make assumptions about aliasing when I explicitly tell it "don't make assumptions about aliasing" than I am in its ability to figure out that's what I'm doing and do the right thing for me wrt optimizing away a memcpy call.

# ? Jan 26, 2018 05:28

eth0.n: Jun 1, 2012

memcpy isn't even needed for most netcode. If you need both struct access and byte-array access, instead of:

C++ code:

struct packet {
  int a;
  int b;
};

char buf[sizeof(struct packet)];
recv(sock, buf, sizeof(buf), 0);
struct packet *p = buf;
printf("%i %i %i", p->a, p->b, buf[0]);

just do:

C++ code:

struct packet p;
recv(sock, &p, sizeof(p), 0);
char *buf = (char*)&p;
printf("%i %i %i", p.a, p.b, buf[0]);

Memcpy is primarily needed for type-punning.

Also, seems a lot easier to verify the compiler is optimizing the relatively few places where an memcpy-to-avoid-aliasing occurs, than to measure how much performance is lost throughout the program by killing a class of optimizations.

OneEightHundred posted:

I guess the funny thing about aliasing is that the most technically correct solution to forcing the compiler to alias a block of memory without doing an actual copy is to memmove the data to the same address and pray it just deletes the memmove.

I don't think that works. It can't change the type of the object in memory, so it doesn't do anything to avoid strict aliasing violations.

C++ can do something similar with placement new, which does work, but that's harder to ensure it's a nop.

# ? Jan 26, 2018 05:52

MrMoo: Sep 14, 2000

It is 2018, compilers will optimize out any redundant memcpy used to bypass aliasing.

# ? Jan 26, 2018 06:03

Jeffrey of YOSPOS: Dec 22, 2005; GET LOSE, YOU CAN'T COMPARE WITH MY POWERS

eth0.n posted:

memcpy isn't even needed for most netcode. If you need both struct access and byte-array access, instead of:

<snipped first block>
C++ code:
struct packet p;
recv(sock, &p, sizeof(p), 0);
char *buf = (char*)&p;
printf("%i %i %i", p.a, p.b, buf[0]);
Memcpy is primarily needed for type-punning.

Also, seems a lot easier to verify the compiler is optimizing the relatively few places where an memcpy-to-avoid-aliasing occurs, than to measure how much performance is lost throughout the program by killing a class of optimizations.

Your second code block breaks my abstraction more than the first one does. My fast path reading bytes doesn't know what the underlying protocol is - it hands off a pointer to another module which handles the specific protocol, perhaps with one struct or another and perhaps some other way. The reasons for wanting to mix byte and struct access are odd perhaps but dictated by the protocols in question.

Fair point on it being easier to verify the impact - definitely not something I can take for granted.

# ? Jan 26, 2018 07:25

VikingofRock: Aug 24, 2008

Jeffrey of YOSPOS posted:

Your second code block breaks my abstraction more than the first one does. My fast path reading bytes doesn't know what the underlying protocol is - it hands off a pointer to another module which handles the specific protocol, perhaps with one struct or another and perhaps some other way. The reasons for wanting to mix byte and struct access are odd perhaps but dictated by the protocols in question.

Fair point on it being easier to verify the impact - definitely not something I can take for granted.

Isn't this just asking for UB from alignment issues (assuming your structs are more-strictly aligned than an array of chars)? Or do I still not understand alignment? I thought I had it understood this time...

# ? Jan 26, 2018 09:01

Jeffrey of YOSPOS: Dec 22, 2005; GET LOSE, YOU CAN'T COMPARE WITH MY POWERS

VikingofRock posted:

Isn't this just asking for UB from alignment issues (assuming your structs are more-strictly aligned than an array of chars)? Or do I still not understand alignment? I thought I had it understood this time...

I use attribute packed on the structs - this forces the compiler to generate code for its platform to do the unaligned accesses if necessary. (It also means any padding has to be done by you.) On x86-64 it just does the unaligned accesses like normal, on (some variants of?) ARM it will generate multiple loads/stores as necessary.

# ? Jan 26, 2018 09:21

Colonel J: Jan 3, 2008

In Visual Studio 2015, when I create a new project, the system creates Debug and Release configurations. From what I understand, I can customize these quite a bit through the project properties.

Any recommendations for creating build targets that are good midpoints between those extremes? As in, it could be good to have "Release with debugging", or "Debug with math optimizations" or other exotic combinations. There are a lot of options in the project properties, and I'm sure many of them could be set to midpoints for a more progressive building experience, but there are so many that it's a bit overwhelming.

On that note, is there a way to make a "super-fast" build that's even more agressive in its optimization than Release?

I hope I'm being clear. Maybe some people here have created such builds? Thanks!

# ? Jan 27, 2018 06:22

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Colonel J posted:

In Visual Studio 2015, when I create a new project, the system creates Debug and Release configurations. From what I understand, I can customize these quite a bit through the project properties.

Any recommendations for creating build targets that are good midpoints between those extremes? As in, it could be good to have "Release with debugging", or "Debug with math optimizations" or other exotic combinations. There are a lot of options in the project properties, and I'm sure many of them could be set to midpoints for a more progressive building experience, but there are so many that it's a bit overwhelming.

On that note, is there a way to make a "super-fast" build that's even more agressive in its optimization than Release?

I hope I'm being clear. Maybe some people here have created such builds? Thanks!

The default configs should be fine unless you have some specific problem with them.

# ? Jan 27, 2018 06:37

OneEightHundred: Feb 28, 2008; Soon, we will be unstoppable!

eth0.n posted:

I don't think that works. It can't change the type of the object in memory, so it doesn't do anything to avoid strict aliasing violations.

memmove is specified as behaving as if the memory was reinterpreted as char to a temp and then copied back to the destination, and both the read-as-char and write-as-char are allowed to alias, so it has that effect. Technically speaking, after the memmove, it shouldn't think that the memory contains a value of the original type any more, it should think it contains a blob of chars.

# ? Jan 27, 2018 08:41

hackbunny: Jul 22, 2007; I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

Colonel J posted:

I hope I'm being clear. Maybe some people here have created such builds? Thanks!

I have, although not in Visual Studio. I have worked on projects that use lots of external libraries, compiled from source. Some of these libraries - think codecs and other compression libraries - are math-heavy and aren't designed to be compiled without optimizations, ever, so I personalized their debug configurations to enable optimizations, because they would run unacceptably slow without, and because it's extremely unlikely you'll actually step through their code in a debugger. In a scenario like this, though, you don't want to slow down the build, so you'll keep support for incremental builds enabled and support for global optimizations disabled: the configuration will still fundamentally be Debug, just with some (not all) optimizations enabled

What I want you to understand by giving you this example is not just what a hybrid configuration looks like, but also that you create such a configuration out of a specific need

Colonel J posted:

On that note, is there a way to make a "super-fast" build that's even more agressive in its optimization than Release?

I don't think it's possible. The most aggressive optimization available is whole program optimization/link-time code generation, and it involves merging your entire program into a single "source" file and recompiling it all again a second time. That takes more time by definition

# ? Jan 27, 2018 14:21

Rottbott: Jul 27, 2006; DMC

I assume they meant fast as in runtime rather than fast to compile. And yes, IIRC VS's release config doesn't enable all optimisations by default so you probably want to go and enable /Ox, /GL and so on (do some profiling).

For debug builds, I just enable /Ob1 (partial inlining) which gives a huge speed boost without hurting debugability much. If anything, it helps not having to step into and back out of every little accessor.

# ? Jan 27, 2018 17:59

School of How: Jul 6, 2013; quite frankly I don't believe this talk about the market

Can someone explain LevelDB to me? It has 12,000 stars on Github, so it must be awesome, but I can't figure out why. Specifically what I don't get are the limitations:

quote:

This is not a SQL database. It does not have a relational data model, it does not support SQL queries, and it has no support for indexes.
Only a single process (possibly multi-threaded) can access a particular database at a time.
There is no client-server support builtin to the library. An application that needs such support will have to wrap their own server around the library.

No index support? Why would you even use a database if it can't make indexes? I come from a Python background where we have Postgres, which has everything you'd ever want.

My rule of thumb is that if I need an index, I'll use a database. If I don't need an index, then I'll just write the data to a file. A database that doesn't have index support seems very pointless to me.

# ? Jan 29, 2018 15:07

Volguus: Mar 3, 2009

School of How posted:

Can someone explain LevelDB to me? It has 12,000 stars on Github, so it must be awesome, but I can't figure out why. Specifically what I don't get are the limitations:

No index support? Why would you even use a database if it can't make indexes? I come from a Python background where we have Postgres, which has everything you'd ever want.

My rule of thumb is that if I need an index, I'll use a database. If I don't need an index, then I'll just write the data to a file. A database that doesn't have index support seems very pointless to me.

Except instead of writing that data to a file using your own protocol, you do it using LevelDBs protocol. What you get is a fast key-value storage. That is, when you lookup-up key Foo it is faster to retrieve the value Bar than your own hand-rolled algorithm when you have millions of keys in that file. If you only have 3 keys, writing them to a file of your your choosing is definitely fine.
Edit: And, why do you specifically need an index for?

# ? Jan 29, 2018 15:13

School of How: Jul 6, 2013; quite frankly I don't believe this talk about the market

Volguus posted:

Except instead of writing that data to a file using your own protocol, you do it using LevelDBs protocol. What you get is a fast key-value storage. That is, when you lookup-up key Foo it is faster to retrieve the value Bar than your own hand-rolled algorithm when you have millions of keys in that file. If you only have 3 keys, writing them to a file of your your choosing is definitely fine.
Edit: And, why do you specifically need an index for?

Maybe I'm mistaken, but if there is no index, then in order to retrieve a key, you have to do a full table scan. An index prevents the need to do a full table scan, as all you need to do is scan the index, which is orders of magnitude smaller and so it scans much faster.

quote:

What you get is a fast key-value storage.

What makes it fast if there is no index?

School of How fucked around with this message at 16:02 on Jan 29, 2018

# ? Jan 29, 2018 16:00

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

You can structure storage such that you don�t need a separate index to make it fast. Consider an in-memory hashtable. I think the LevelDB whitepaper details how it accomplishes this.

# ? Jan 29, 2018 16:20

roomforthetuna: Mar 22, 2005; I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

School of How posted:

What makes it fast if there is no index?

I think maybe the confusion is that it's not "no index", it's "no separate index". The table itself is sorted by key, it just doesn't natively do cross-joins against an index for fast selecting based on a different column. (Though I assume you could implement that on top if you wanted, by making another table keyed on the other value and value'd the key into the real table.)

# ? Jan 29, 2018 16:28

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Indexes are a potential solution to a problem, not a feature that is inherently useful that you would specifically use a DB for.

# ? Jan 29, 2018 18:14

fankey: Aug 31, 2001

In Qt, I'm trying to hook up a QLocalSocket to a local Win32 named pipe server running in PIPE_TYPE_MESSAGE mode. The pipe connects fine but readyRead() never gets called and it doesn't look like the messages I'm sending are making it either. The MSDN docs are a little vague - it doesn't really say how messages are delimited if at all and if there is any handshaking required to make this work. Is there any more indepth documentation on how PIPE_TYPE_MESSAGE? QLocalSocket doesn't have it as an option - is that going to be difficult and should I just write raw WIN32 code?

# ? Jan 29, 2018 19:52

School of How: Jul 6, 2013; quite frankly I don't believe this talk about the market

Plorkyeran posted:

Indexes are a potential solution to a problem, not a feature that is inherently useful that you would specifically use a DB for.

I disagree. In my experience querying a database by something other than the primary key is a very common operation.

# ? Jan 30, 2018 00:15

Jeffrey of YOSPOS: Dec 22, 2005; GET LOSE, YOU CAN'T COMPARE WITH MY POWERS

School of How posted:

I disagree. In my experience querying a database by something other than the primary key is a very common operation.

A DB at its core has nothing to do with indexing though. You would still have them in a universe where indices were never invented, because "look up this stuff by id number" is still really useful. That makes indices a cool, widely-used feature and not a fundamental part of a DB, to me.

# ? Jan 30, 2018 00:30

Nippashish: Nov 2, 2005; Let me see you dance!

Don't let the "db" in the name fool you. LevelDB is not a relational database, and if you want a database you don't want leveldb. If you want a big blob of THINGS that you can look up by key though, it just might be right for you.

# ? Jan 30, 2018 00:35

pseudorandom name: May 6, 2007

You can argue all you want but you're not going to change the fact these things have been called databases since at least 1979 when dbm was first released.

# ? Jan 30, 2018 00:45

The Phlegmatist: Nov 24, 2003

fankey posted:

The pipe connects fine but readyRead() never gets called and it doesn't look like the messages I'm sending are making it either.

It's been a while since I wrote Qt code for Windows but are you sending more data than can fit in the pipe's buffer? QLocalSocket won't handle the error and won't see anything to read in the pipe.

# ? Jan 30, 2018 00:54

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

Nippashish posted:

Don't let the "db" in the name fool you. LevelDB is not a relational database, and if you want a database you don't want leveldb. If you want a big blob of THINGS that you can look up by key though, it just might be right for you.

Not all databases are relational. LevelDB is a database just like Cassandra and memcached. Databases existed before Codd. Don�t be fooled by Big SQL.

# ? Jan 30, 2018 01:27

Volguus: Mar 3, 2009

I think that School of How just got confused about the "database" name. According to google:

quote:

da�ta�base
ˈdadəˌbās,ˈdādəˌbās/
noun
a structured set of data held in a computer, especially one that is accessible in various ways.

A text file with records stored line by line fits that definition. It is a structured set of data (one record per line), held in a computer (a file on the disk), accessible in various ways (via my program). Anything else is just cherry on top: relations, tables, users, permissions, server that listens on a network and can server many users at the same time, indexes, primary keys, all kinds of other features of modern databases.

As you can see, LevelDB is very much a database, as it fits the definition of a database. It is one that only provides 2 things: store and retrieve key-value pairs and it provides access via a library. And is promising to do that very fast.

# ? Jan 30, 2018 03:54

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

School of How posted:

I disagree. In my experience querying a database by something other than the primary key is a very common operation.

Yes, exactly. The functionality you actually want is fast lookups on things other than primary keys. Indexes are a mechanism for achieving that, not an end in themselves.

# ? Jan 30, 2018 04:07

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Embeddable key-value stores are great and have a long tradition. BerkeleyDB, LMDB, etc etc. If all you have are large lists of things that you want to access by just one key each, and you're only concerned about a single process, Postgres is a bloated nightmare by comparison.

Think of it like just storing your data in a regular file, except with tons of great features like transactionality and fast random inserts.

# ? Jan 30, 2018 08:17

Nippashish: Nov 2, 2005; Let me see you dance!

Subjunctive posted:

Not all databases are relational. LevelDB is a database just like Cassandra and memcached. Databases existed before Codd. Don�t be fooled by Big SQL.

Yes, sorry this is what I was trying to say (but not very clearly it seems). Don't expect that having DB in the name means it will do all the SQL things.

# ? Jan 30, 2018 10:17

Xarn: Jun 26, 2015

How fast is levelDB compared to sqlite anyway? Because sqlite owns for embedding.

# ? Jan 30, 2018 10:55

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

Xarn posted:

How fast is levelDB compared to sqlite anyway? Because sqlite owns for embedding.

Depends on what you�re doing, and whether you need compression.

http://www.lmdb.tech/bench/microbench/benchmark.html (but it�s old)

Some kv stores are optimized for different media, like RocksDB for SSDs/flash.

# ? Jan 30, 2018 11:48

School of How: Jul 6, 2013; quite frankly I don't believe this talk about the market

Plorkyeran posted:

Yes, exactly. The functionality you actually want is fast lookups on things other than primary keys. Indexes are a mechanism for achieving that, not an end in themselves.

What other ways are there to have fast lookups to non-primary key columns not using an index?

# ? Jan 30, 2018 18:04

eth0.n: Jun 1, 2012

School of How posted:

What other ways are there to have fast lookups to non-primary key columns not using an index?

Could maintain a second table which maps that secondary key to the primary key of the main table.

But that's besides the point of the post you're replying to. They're just saying that if you don't need fast lookups on non-primary keys, then perhaps indexes aren't needed for you, and using a lower-overhead DB that doesn't support them might be a good idea.

# ? Jan 30, 2018 18:25

nielsm: Jun 1, 2009

An index is just another table, sorted on the index key, pointing to the primary key. You can construct a relational database using a key-value store.

# ? Jan 30, 2018 18:27

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

Hi guys,

Apologies if this is a dumb question. I'm a bit out of my depth here.

I'm trying to compile SDL2_gfx for x64. I'm using MSVC. I'm having an issue because this library uses inline assembly, which causes errors. I can get it to build on x86 with no problems.

It looks like my options are as follows:

Rewrite SDL2_gfx to not use asm. I do not have the expertise to do this.
Convert my project to x86 and give up on x64.

Do I have that about right? Is there some quick and dirty way to make inline x86 assembly work on x64 with MSVC, or a way to build this library as x86 but link my x64 application against it?

My application is written in Rust, not C/C++, if that matters. I'm trying to use dynamic linking. I do have the rest of SDL2 working.

# ? Feb 5, 2018 18:10

nielsm: Jun 1, 2009

Looking through the source, it looks like not defining USE_MMX when compiling will skip all the inline assembly, and just fall back to pure C implementations. Maybe you will need to edit the project files to avoid defining that, but I don't think you need to patch the code itself.

# ? Feb 5, 2018 18:35

xgalaxy: Jan 27, 2004; i write code

nielsm posted:

Looking through the source, it looks like not defining USE_MMX when compiling will skip all the inline assembly, and just fall back to pure C implementations. Maybe you will need to edit the project files to avoid defining that, but I don't think you need to patch the code itself.

code:

./configure --disable-mmx

Right there in the docs.

But if you are using Rust.. why aren't you using one of the crates that already has SDL2, etc built for you with bindings setup?
Or better yet something like: https://crates.io/crates/gfx

xgalaxy fucked around with this message at 21:32 on Feb 5, 2018

# ? Feb 5, 2018 21:26

Bonfire Lit: Jul 9, 2008; If you're one of the sinners who caused this please unfriend me now.

xgalaxy posted:

code:
./configure --disable-mmx
Right there in the docs.

Fergus Mac Roich posted:

I'm using MSVC.

# ? Feb 5, 2018 21:31

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

xgalaxy posted:

code:
./configure --disable-mmx
Right there in the docs.

But if you are using Rust.. why aren't you using one of the crates that already has SDL2, etc built for you with bindings setup?
Or better yet something like: https://crates.io/crates/gfx

I'm using this: https://github.com/Rust-SDL2/rust-sdl2

It still requires you to have the .lib and .dll files. As I said the base SDL2 does work. I was also able to get the other extensions working.

I'm going to explore my options again when I get home today. I don't suppose I could disable MMX if I just switched to the mingw toolchain?

# ? Feb 5, 2018 22:10

Adbot: ADBOT LOVES YOU

# ? Jun 7, 2024 10:45

Doc Block: Apr 15, 2003; Fun Shoe

Bonfire Lit posted:

Just remove USE_MMX from the list of preprocessor defines in the project settings or the configuration header file?

And maybe file a bug report for SDL_gfx?

# ? Feb 5, 2018 22:24

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›641 »