C/C++ Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›641 »

GeneralZod: May 28, 2003; Kneel before Zod!; Grimey Drawer

MGN001 posted:

If I'm writing a function that has an array as a parameter, is there any difference between
code:
type function_name(type * name)
and
code:
type function_name(type name[])

Not to the compiler, at least. In fact, behind the scenes, type function_name(type name[]) is converted into type function_name(type * name) (8.3.5.3).

# ? May 13, 2014 07:36

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 13:07

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

MGN001 posted:

If I'm writing a function that has an array as a parameter, is there any difference between
code:
type function_name(type * name)
and
code:
type function_name(type name[])

No but you should be passing in the length as a size_t and using something like

code:

 type function(size_t len, type name[static len])

If you're using C++, use std::vector or std::array.

# ? May 13, 2014 07:41

OzyMandrill: Aug 12, 2013; Look upon my words
and despair

Hammerite posted:

So I have a C++ book that I've had for a while, and an early chapter covers integer datatypes including signedness vs. unsignedness. It covers what happens when unsigned integers overflow, but then it goes on to say that the equivalent thing happens for unsigned integers. (i.e. it gives the impression that INT_MAX + 1 will be INT_MIN.) And I know from various other sources that this needn't be the case and that assuming that it is can lead to obscure bugs due to e.g. compiler optimisations that take the opportunity to assume that signed integer overflow won't happen.

So I was thinking how would you write a function that you could use when you wanted to add signed integers and reliably get the "wrap around" behaviour that unsigned integer overflow is guaranteed to give. That is, how would you write a function that made the statements about signed integer addition in the book true. Let me check whether this is a solution. Some of it is based on skimming through Stack Overflow pages and other resources.

Not entirely sure I understand what you are on about as there are some errors in the text...
However, for all C/C++ platforms I have personally used, they all do the same thing - a signed integer behaves bit-wise like an unsigned integer for addition & subtraction.

Let's use 4bit for simplicity:

code:

  0111 =  7 = INT4_MAX
+ 0001
= 1000 = -8 = INT4_MIN

This is a signed overflow, but is actually just 7 + 1 = 8 in unsigned maths.

The step from -1 to 0 in signed ints is a 'true' overflow as far as the bits are concerned:

code:

  1111
+ 0001
=10000 = 0000 + the carry bit set in status register

In signed ints, this represents (-1 + 1 = 0) which you would not really call an overflow, but in unsigned ints it is (15 + 1 = 0 + carry bit) which is an overflow!

The point I am trying to make is that the actual bitwise operation is the same for both cases - assembler/machine code does not recognise any distinction between signed and unsigned here - they use ADD/SUB for both. Note that this isn't always the case - there are two forms of shift - arithmetic (for signed ints) and logical (unsigned), but for addition & subtraction - they behave identically.

Now I am not entirely sure what you are trying to achieve with your functions, but:
a + b + 2*INT_MIN does nothing because:
0x1000 * 2 = 0x10000 = 0000 in 4 bit

a + b + 2*INT_MAX does:
INT_MAX = 0111, so 2*INT_MAX = 1110

so:

code:

7 + 1 + (1110) = (1000) + (1110)

   1000
+  1110
= 10110 = b0110 = 6

So your +(2 * INT_MAX) is actually +(-2) and (7+1-2)=6, which doesn't seem right for any overflow scheme really...

If your aim was to get '0' again then what you actually want is: ((a+b)&0xfff) which will strip off the sign bit.

# ? May 13, 2014 11:03

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

OzyMandrill posted:

Not entirely sure I understand what you are on about as there are some errors in the text...

Yes, the second "unsigned integers" should be "signed integers" instead. Edited in original.

quote:

However, for all C/C++ platforms I have personally used, they all do the same thing - a signed integer behaves bit-wise like an unsigned integer for addition & subtraction.

This is my understanding of what will generally happen in practice. Still, it is apparently undefined behaviour and I have received the impression that this may have implications for how code behaves in practice if it (for example) tests for overflow. See this Stack Overflow question.

quote:

Now I am not entirely sure what you are trying to achieve with your functions, but:
a + b + 2*INT_MIN does nothing because:
0x1000 * 2 = 0x10000 = 0000 in 4 bit

The point of the functions is to obtain the behaviour that you are describing without relying on undefined behaviour. For example, the point of writing INT_MAX - a < b (in the first if-condition) rather than somehow testing the result of a + b is that the first does not rely on undefined behaviour in the case where a + b is in fact greater than INT_MAX and thus unrepresentable.

That a + b + 2*INT_MIN does nothing is by design.

quote:

a + b + 2*INT_MAX does:
INT_MAX = 0111, so 2*INT_MAX = 1110

This, on the other hand, is an error in my code. The second if-clause should instead be something like this:

code:

    if (a < 0 && b < 0 && INT_MIN - a > b) {
        return (INT_MAX + (a + 1)) + (INT_MAX + (b + 1));
            // This avoids overflow because
            //     a + b < INT_MIN
            // and therefore
            //     a + b + 2*INT_MAX + 2 < INT_MIN + 2*INT_MAX + 2
            //                           = INT_MIN - 2*INT_MIN
            //                           = -INT_MIN
            //                           = INT_MAX + 1
            // and so (a + b + 2*INT_MAX + 2) is representable.
            // Again, the brackets are important (at least for clarity).
    }

# ? May 13, 2014 16:35

Bonfire Lit: Jul 9, 2008; If you're one of the sinners who caused this please unfriend me now.

OzyMandrill posted:

However, for all C/C++ platforms I have personally used, they all do the same thing - a signed integer behaves bit-wise like an unsigned integer for addition & subtraction.

This is dangerous - gcc's optimizer for example will assume that signed overflow does not occur (if you don't use -fwrapv) and use that assumption when compiling loops or conditionals. This may lead to a range check being removed because it can "never fail".

# ? May 13, 2014 17:13

OzyMandrill: Aug 12, 2013; Look upon my words
and despair

Bonfire Lit posted:

This is dangerous - gcc's optimizer for example will assume that signed overflow does not occur (if you don't use -fwrapv) and use that assumption when compiling loops or conditionals. This may lead to a range check being removed because it can "never fail".

Cool - never knew that! I tend to cast to unsigned if I'm messing with bits, just out of habit.
If I ever find myself relying on signed overflows working right, it's probably a sign that I'm writing terrible code.

OzyMandrill fucked around with this message at 21:27 on May 13, 2014

# ? May 13, 2014 21:25

Heavy neutrino: Sep 16, 2007; You made a fine post for yourself. ...For a casualry, I suppose.

Hey guys, I'm a non-computer person who recently decided to become one, and being hopelessly hard-headed I chose C++ as a starting point via Beginning C++ Through Game Programming, which I've just finished. In a flash of inverse Dunning-Kruger Effect, my big takeaway from the book is that although I learned a lot about barebones C++, programming conventions and logic, and object-oriented programming, I still haven't the first clue how to program anything that actually looks like an application that you'd commonly use -- all the exercises and examples were done through console applications. As such, I have a couple questions:

1) The book that I mentioned was critically light on actual exercise problems. Can someone who's more knowledgeable point me towards a place or a book where I can find exercises that range from extremely basic input/output manipulation all the way to the most advanced topics I learned -- inheritance and polymorphism?

2) If I want to actually learn to use C++ to program actual windows, mobile, iOS and Linux applications that don't look like throwbacks to the glorious 1980's, where would I start? Remember that I never studied computer science, so please be as basic as you can be. What I'm having a hard time expressing here is that I want to figure out how the basic stuff I've learned about C++ fits into the larger project of making computers do stuff.

Don't be afraid to throw math-heavy stuff at me -- it's one of the things that I actually did study, much to my dismay and that of my job prospects.

3) Do you people have recommendations for different languages that I'd be wise to learn concurrently with C++?

# ? May 14, 2014 09:14

Hiowf: Jun 28, 2013; We don't do .DOC in my cave.

Heavy neutrino posted:

2) If I want to actually learn to use C++ to program actual windows, mobile, iOS and Linux applications that don't look like throwbacks to the glorious 1980's, where would I start? Remember that I never studied computer science, so please be as basic as you can be. What I'm having a hard time expressing here is that I want to figure out how the basic stuff I've learned about C++ fits into the larger project of making computers do stuff.

Start by looking at a GUI toolkit. Qt and wxWidgets support many platforms (though not necessarily Android/iOS) and are fairly well documented. Qt comes with a decent IDE (QtCreator) which might be quite useful in your case.

Android requires Java for GUIs, and iOS requires Objective C, so they'll need another leap before you can target them. They run C++ code fine for the "backend" stuff, of course.

If your interest is in games, look at OpenGL, and you may be interested in SDL/GLUT/GLFW more than Qt/wxWidgets.

quote:

3) Do you people have recommendations for different languages that I'd be wise to learn concurrently with C++?

I would learn at least one scripting language. Personally I would pick Python as it mixes good with C++, is very widely used, and has great libraries, many of them built-in. JavaScript is another one that is very in-demand, though obviously more at the webby side.

quote:

Can someone who's more knowledgeable point me towards a place or a book where I can find exercises that range from extremely basic input/output manipulation all the way to the most advanced topics I learned -- inheritance and polymorphism?

You might alternatively ask yourself what you want to actually do and just head in that direction, instead of staying on exercises! What interests you? It may guide your selection of framework and second language as well.

Hiowf fucked around with this message at 09:40 on May 14, 2014

# ? May 14, 2014 09:38

nielsm: Jun 1, 2009

Skuto posted:

Start by looking at a GUI toolkit. Qt and wxWidgets support many platforms (though not necessarily Android/iOS) and are fairly well documented. Qt comes with a decent IDE (QtCreator) which might be quite useful in your case.

Android requires Java for GUIs, and iOS requires Objective C, so they'll need another leap before you can target them. They run C++ code fine for the "backend" stuff, of course.

If your interest is in games, look at OpenGL, and you may be interested in SDL/GLUT/GLFW more than Qt/wxWidgets.

A better start might be something like SDL or Unreal Engine 4.
SDL is a free framework for building games and game-like applications, while Unreal Engine 4 is a subscription-based complete game engine. Both give you some tools useful for getting graphics on the screen, sound in the speakers and raw input from keyboard, mouse and controllers.

SDL is more mature, but is not a complete game engine in itself, while UE4 is still in active development. UE4 is $20 per month, and a percentage of gross income if you release a commercial game, but you can pay for a single month, get a snapshot of it to play with, and never pay again.

# ? May 14, 2014 10:14

UncleBlazer: Jan 27, 2011

Is "The C Programming Language" book still relevant? I got a few pages in and wasn't sure if I'd been spoiled by more modern books for other languages or if it's just outdated.

# ? May 14, 2014 13:33

Joda: Apr 24, 2010; When I'm off, I just like to really let go and have fun, y'know?; Fun Shoe

Heavy neutrino posted:

Hey guys, I'm a non-computer person who recently decided to become one, and being hopelessly hard-headed I chose C++ as a starting point via Beginning C++ Through Game Programming, which I've just finished.

I started on this book way back, but forgot everything once I started uni where everything had to be Java (I'm now kicking myself in the head for not hanging on to it.) One thing I do remember it not covering, though, is make files, headers and conventions for how these are used (unless he changed it in a newer edition.) If you want to start doing projects around 500 lines or more, I'd definitely recommend reading up on that specifically, because debugging/expanding a 500 lines main file with all your classes is making things unnecessarily hard on yourself.

E: Is he still recommending DevCpp? If so, you should get yourself acquainted with a modern IDE (I like NetBeans, but its code assistance does suffer from it being originally made for Java) and learn how to set up a toolchain in it. Most IDEs will set up a default toolchain too, and will generate make files and such for you. Some are also able to analyse your code real-time, auto-complete your function calls/declarations, point out if you're referencing a non-existent member or member function, possible memory leaks, etc.

Joda fucked around with this message at 13:56 on May 14, 2014

# ? May 14, 2014 13:44

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

UncleBlazer posted:

Is "The C Programming Language" book still relevant? I got a few pages in and wasn't sure if I'd been spoiled by more modern books for other languages or if it's just outdated.

It's frankly a much better and easier read than most modern programming books. The style it uses is also not very good for the use cases of C in 2014. It will teach you the syntax and semantics of the language well though, and if you're learning C to do school assignments it will be suffice.

# ? May 14, 2014 13:44

Hiowf: Jun 28, 2013; We don't do .DOC in my cave.

Joda posted:

One thing I do remember it not covering, though, is make files, headers and conventions for how these are used (unless he changed it in a newer edition.) If you want to start doing projects around 500 lines or more, I'd definitely recommend reading up on that specifically, because debugging/expanding a 500 lines main file with all your classes is making things unnecessarily hard on yourself.

Makefile are rather platform/environment specific. I'm not saying you shouldn't know about them, as they're common, but if you work from an IDE you can avoid learning many of the complexities while still being able to split your code into separate modules. That's one reason I recommended looking at QtCreator.

(When I say environment specific, I mean that in many cases people aren't coding all that much Makefiles directly anymore, but using stuff like cmake, scons, gyp, etc)

# ? May 14, 2014 13:53

Ericadia: Oct 31, 2007; Not A Unicorn

Skuto posted:

Android requires Java for GUIs, and iOS requires Objective C, so they'll need another leap before you can target them. They run C++ code fine for the "backend" stuff, of course.

You can actually make an entire iOS app with C++/C using SDL for GUI

That being said, if he wants to learn about objective-C and owns a Mac: ignore books for a little bit and just browse the Apple documentation. There are actually intro topics in the docs that are better than a lot of books out there.

# ? May 14, 2014 14:25

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

contrapants posted:

code:

typedef signed_integer_addition_with_wraparound<int> signed_int_addition_with_wraparound;

I just tried this on a simplified example and couldn't get it to work. It tells me: "error: x<int> does not name a type". (If I comment out the "typedef" line and replace "xint" with "x<int>" in main(), it does work.)

code:

#include <iostream>

template <typename T>
T x (T arg) {
    return 2*arg;
}

typedef x<int> xint;

int main () {
    std::cout << xint(23) << "\n";
    return 0;
}

# ? May 14, 2014 21:43

nielsm: Jun 1, 2009

Hammerite posted:

I just tried this on a simplified example and couldn't get it to work. It tells me: "error: x<int> does not name a type". (If I comment out the "typedef" line and replace "xint" with "x<int>" in main(), it does work.)

x<int> is a function, not a type, so yes you can't typedef it to something.

But I think you should be able to "using" it, at least with a recent compiler:

C++ code:

using xint = x<int>;

# ? May 14, 2014 21:50

Subjunctive: Sep 12, 2006; ✨sparkle and shine✨

nielsm posted:

x<int> is a function, not a type, so yes you can't typedef it to something.

But I think you should be able to "using" it, at least with a recent compiler:
C++ code:
using xint = x<int>;

Or just

code:

inline int xint(int i) { return x<int>(i); }

which should work everywhere?

# ? May 14, 2014 22:16

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

nielsm posted:

x<int> is a function, not a type, so yes you can't typedef it to something.

But I think you should be able to "using" it, at least with a recent compiler:
C++ code:
using xint = x<int>;

This compiler doesn't like that, but never mind. I only wanted to test it out to see whether it would work, because contrapants's post said it would work and it seemed strange that "typedef" (which has a name that seems to suggest it relates to types) would be used to name a specialisation of a templated function.

# ? May 14, 2014 22:17

nebby: Dec 21, 2000; resident mog

Accelerated C++ is a good book for picking up C++ with exercises. Still console-app focused tho, since that is the best way to learn the language IMHO, not really for "real apps."

# ? May 14, 2014 22:52

b0lt: Apr 29, 2005

Subjunctive posted:

Or just
code:
inline int xint(int i) { return x<int>(i); }
which should work everywhere?

The inline keyword does not do what you think it does. inline is a linkage specification that tells the linker not to worry if the symbol shows up multiple times. If you want to enforce inlining, you'd have to use a compiler directive (e.g. __attribute__((always_inline)) on gcc).

# ? May 15, 2014 00:12

Marta Velasquez: Mar 9, 2013; Good thing I was feeling suicidal this morning...; Fallen Rib

Hammerite posted:

I just tried this on a simplified example and couldn't get it to work. It tells me: "error: x<int> does not name a type". (If I comment out the "typedef" line and replace "xint" with "x<int>" in main(), it does work.)

Sorry about that. I have no idea how I missed that you were asking about a function, not a type. :downs:

# ? May 15, 2014 00:19

The Laplace Demon: Jul 23, 2009; "Oh dear! Oh dear! Heisenberg is a douche!"

b0lt posted:

The inline keyword does not do what you think it does. inline is a linkage specification that tells the linker not to worry if the symbol shows up multiple times. If you want to enforce inlining, you'd have to use a compiler directive (e.g. __attribute__((always_inline)) on gcc).

Or in C++11,

C++ code:

[[gnu::always_inline]]
inline int xint(int i) { return x<int>(i); }

which is understood by at least clang++ and g++. Although, if the compiler doesn't implement using correctly...

# ? May 15, 2014 01:00

Heavy neutrino: Sep 16, 2007; You made a fine post for yourself. ...For a casualry, I suppose.

Joda posted:

I started on this book way back, but forgot everything once I started uni where everything had to be Java (I'm now kicking myself in the head for not hanging on to it.) One thing I do remember it not covering, though, is make files, headers and conventions for how these are used (unless he changed it in a newer edition.) If you want to start doing projects around 500 lines or more, I'd definitely recommend reading up on that specifically, because debugging/expanding a 500 lines main file with all your classes is making things unnecessarily hard on yourself.

E: Is he still recommending DevCpp? If so, you should get yourself acquainted with a modern IDE (I like NetBeans, but its code assistance does suffer from it being originally made for Java) and learn how to set up a toolchain in it. Most IDEs will set up a default toolchain too, and will generate make files and such for you. Some are also able to analyse your code real-time, auto-complete your function calls/declarations, point out if you're referencing a non-existent member or member function, possible memory leaks, etc.

You're right; he specifically mentioned during the final chapter's somewhat heavy project that splitting code into several files was outside the scope of the book. I haven't a clue how to do it. As for the IDE, in the third edition he recommends MSVC++ express, but I've been using Code::Blocks instead. I haven't had any trouble with it, but I'm way too green to detect any problems it might have.

I'd certainly like to learn how to make headers. I'll look around for some tutorials on that.

Skuto posted:

You might alternatively ask yourself what you want to actually do and just head in that direction, instead of staying on exercises! What interests you? It may guide your selection of framework and second language as well.

Well, I don't think I expressed very well how starved the book was for exercises -- every full chapter had a stupendous total of three exercise questions, two of which were usually piss easy ("find the obvious error 'hidden' in these five lines of code") and one which asked you to use newly learned concepts to optimize the projects in earlier chapters (pretty good exercise). What I'm looking for is exercises like "here's a problem for you chump; figure out a way to solve it using x/y/z." For instance, I just don't feel like I have a good enough grip of inherited classes and virtual functions to use them in anything concrete yet.

As for figuring out what I want to do, it's like asking an elementary school boy what career he'd like to work in. Sure, he'll probably say he wants to be an astronaut (make games), but you gotta take it with a grain of salt. I picked out Dawson's book because I like video games and figured that book had the highest chance of keeping me hooked until the end, not particularly because I want to work on game design. I've already got decent work, and my goal is just to learn programming for the sake of it.

The plan I've decided to go for is to just ape my alma mater's Software Engineering curriculum and use lab/homework problems that the profs left on their course websites. Maybe not the most ethical thing, but I don't think anyone will mind.

# ? May 15, 2014 05:25

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

Heavy neutrino posted:

You're right; he specifically mentioned during the final chapter's somewhat heavy project that splitting code into several files was outside the scope of the book. I haven't a clue how to do it. As for the IDE, in the third edition he recommends MSVC++ express, but I've been using Code::Blocks instead. I haven't had any trouble with it, but I'm way too green to detect any problems it might have.

I'd certainly like to learn how to make headers. I'll look around for some tutorials on that.

Headers are a basic C++ feature, I don't get why that would be a problem. It's the same fundamental problem as prototyping within a file - the compiler needs to know all the symbols (function names, variables, etc) in a file before it can start linking the source itself*. It's pretty straightforward - the header tells the compiler what symbols are going to be defined (function names and parameters followed by a semicolon, names of structs or even the whole struct prototype, extern [type] [variablename] for global variables, etc) and then the source file actually creates an instance of those (functions with braces and code, [type] [variablename], etc).

The only real trick with them is a symbol can only be declared once for a given scope, so you don't want to have the same file included in two places (eg file A includes B which includes C, and A also includes C), and you especially don't want to get into include loops, where file A includes file B which includes file A. A certain degree of this is really common and the usual way to handle this is the include guard - if the file is myHeader.h, the entire header file is enclosed in an #ifndef preprocessor macro immediately followed by a #define.

code:

#ifndef MYHEADER_H
#define MYHEADER_H

//stuff goes here
	 
#endif

That checks whether the file has already been included and only defines symbols if it hasn't. Most compilers provide a helpful keyword to do this automatically. Put "#pragma once" as the first line of the file and it does the same thing (technically this isn't portable, but it's very common). You also do want to keep some kind of logical ordering - I usually have a file that handles typedefs and defines magic numbers that are going to get used all over the place, maybe one for configuration, and then one header per source file with the prototypes and maybe some fine-grain config stuff. That lets you form a logical hierarchy of includes.

Technically there's no reason that all the functions in a header need to be implemented in one source file, although it's kind of bad practice to split them without a good reason. It can be useful for situations where you potentially need the same thing implemented different ways. For example I need to monitor resource usage (memory and clock time) in Unix and Windows, so I have a header, a file with some common functions, a file that handles the Windows idioms for getting the necessary information, and a file that handles the Unix idioms. I have a makefile set up that builds the project with the Unix file and a VS project file builds with the Windows file. You could also do this within a single file using preprocessor macros like this, but separate files is cleaner.

code:

#ifdef _MSC_VER
#include <windows.h>
#else
#include <sys/resource.h>
#endif
...
#ifdef _MSC_VER
//windows version of platform-specific functions
#else
//unix version of platform-specific functions
#endif

MSVC++ seems like a fairly nice environment to work in but it's definitely got its own unique subset of the C++ standards. It's got about half of the C++11 standard there and it's annoying when stuff is missing (eg the portable printf formatter for size_t, %zu). It also tries to shove precompiled headers down your throat, along with some other stuff that looks a bit weird compared to other C++ (int _tmain(int argv, _TCHAR * argv)) even if there is a somewhat sensible reason. I also find it kind of annoying that a fancy IDE like that won't let you see the value of #defines during debugging. I realize it's not in the symbol table, but c'mon, you can't give me an intellisense for that?

In terms of problems, what kind of stuff have you done? The usual pedagogical approach is to go through the various algorithm styles. I've always thought dynamic programming was a cool algorithm - 0/1 knapsack is the classical example problem here. Branch and bound is a bit easier than that and can be used on the same kind of problems but often has a worse runtime. And there's always lots of fun stuff to do at a real low level, like trying to detect linked lists that loop on themselves (classic interview question, simplest answer is tortoise and hare algorithm).

*As an aside, the need to know symbol names/types is also why the early C revisions require you to declare all variables right up at the front of a function.

Paul MaudDib fucked around with this message at 23:25 on May 15, 2014

# ? May 15, 2014 12:05

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

Am I just blind, or is there no file copy analogue to standard C calls like unlink(), rename(), chmod(), etc.?

(edit: not that it's exactly hard to do with fopen, fwrite, fclose, but dammit it's the principle of the thing)

Ciaphas fucked around with this message at 19:53 on May 15, 2014

# ? May 15, 2014 19:50

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Linux has sendfile and Windows has CopyFile. Traditional unices didn't have helper functions for filesystem stuff that can be done entirely in userland, so C doesn't either.

# ? May 15, 2014 20:16

pseudorandom name: May 6, 2007

TransmitFile() is the Windows equivalent of sendfile(), CopyFile() has no Unix/Linux equivalent.

# ? May 15, 2014 20:49

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

Joda posted:

If you want to start doing projects around 500 lines or more, I'd definitely recommend reading up on that specifically, because debugging/expanding a 500 lines main file with all your classes is making things unnecessarily hard on yourself.

Haha, 500 lines.

One of the things I hate about nvcc is that it doesn't allow CUDA symbols to be used across files or placed inside classes (maybe statically?). Extern is considered an illegal keyword for device symbols. For example, the following is completely illegal in CUDA:

code:

device_constants.h:
#define ROWS 10
extern __device__ __constant__ float myPdf[ROWS];

code:

main.cu:
#include "device_constants.h"
__device__ __constant__ float myPdf[ROWS];
__device__ int function_using_pdf()
{...}

code:

setup.cu:
#include "device_constants.h"
push_to_memory()
{
	float host_PDF[ROWS];
	//do setup

	cudaMemcpyToSymbol(myPdf, host_PDF, sizeof(float) * ROWS, 0, cudaMemcpyHostToDevice);
}

Even that's an improvement over the earlier specs, which didn't allow you to use symbols directly. Instead you had to use a cstring of the symbol name and it would be targeted at runtime. So if you changed the name of a symbol ctrl-h was the most efficient way to do it to avoid runtime errors. Pretty sure that's a coding horror.

code:

	cudaMemcpyToSymbol("myPdf", host_PDF, sizeof(float) * ROWS, 0, cudaMemcpyHostToDevice);

The core spec requires that a single file contain everything needed for the device code, no linking across files. At best you could break out host code (main program loops, etc) and device code, going farther than that was tough. And that exacerbates the lovely compile times since you need to rebuild almost everything all the time. And even if you did it will compile the files one at a time and die if any one of them has an error (template library errors caused by an error in one of your files count as a separate file) without even trying the later files, so you get 5 minutes of compiling broken up by changing two lines at a time. Basically they did everything possible to encourage you to write one big omnibus file and even trivial stuff rapidly swells in the number of lines.

Maybe there's another flag to fix that last thing too, and make it at least try to compile everything? I'm working in Visual Studio mostly. I don't see anything obvious in the project settings at least.

e: Actually I guess you can use extern on device symbols in more recent CUDA toolkits (5.0+), it just doesn't work on all devices (2.0+), isn't enabled by default even when targeting devices that it does work on, and isn't obvious ("generate relocatable device code").

The reason given for these kind of limitations is that nvcc isn't a real compiler, it's just a front end for other tools. I can kind of see where they're probably going with that concept (separate segment-blocks of device code per source file, with the later devices having the ability to peek into other segments). That still seems like kind of a copout, you could do clever parsing to break those limitations. Get the set of all source files sharing at least one extern declaration (for simplicity, say the extern declarations are in a separate header and you just select all source files that include it). Parse the headers together in one file, then start to pick apart the source files. Parse all the includes, then cat all the definitions of all symbols marked as extern together and delete the extern declaration. Then cat the rest of all the source files after that. Net effect, during the build you have everything in one file with the device symbol definitions at the top (stripped of extern keyword) followed by all the rest of the code, just as if you were doing the mothership approach yourself, but you can manage the code as separate source files. There's obviously edge cases like non-extern (file scope) symbols that have the same name and refer to different things in different files.

Overall this seems like the kind of thing you'd be able to do with intermediate output from parsers fairly reasonably. I don't get that one, the devices themselves are pretty amazing but some of the compiler stuff just seems like amateur hour. Another beef, if you are statically allocating shared memory in kernels then the array dimensions have to be literal-constant, you can't do things like statically allocate a shared array dimensioned by the number of threads (often set at launch-time). I get why this can't be done on the CPU (stack vs heap) but I don't think that distinction applies to shared memory since there's no shared-memory stack, shared memory is only allocated when you launch a kernel and you can't call kernels from inside other kernels (dynamic parallelism) until compute capability 3.5. It's not super difficult to handle this yourself, but that denies the compiler the chance to tweak memory access patterns and stuff. It doesn't seem like it would be any harder to have the compiler automatically translate a static allocation to a dynamic allocation and maybe emit a warning if there's a legitimate performance concern.

eg this is illegal:

code:

int blocks, threads;
size_t dynamic_smem_size = 0;
__global__ void kernel()
{
	__shared__ int sharedMem[blockDim.x];	//blockdim = threads
	__shared__ int moreSharedMem[blockDim.x];

	int * mySmemPtr = &sharedMem[threadIdx.x]; 	//threadIdx.x is between (0 and threads-1)
	int * myMoreSmemPtr = &moreSharedMem[threadIdx.x];
}

but I don't see why you can't just translate this to the corresponding dynamic pattern:

code:

int blocks, threads;
size_t dynamic_smem_size = (sizeof(int) + sizeof(int)) * threads;
__global__ void kernel()
{
	extern __shared__ int sharedMemPtr[];	//ptr to 2 * sizeof(int) * blockDim.of shared memory

	int * sharedMem = (int *) &sharedMemPtr[0];
	int * moreSharedMem = (int *) &sharedMem[blockDim.x];	//blockdim = threads
	
	int * mySmemPtr = &sharedMem[threadIdx.x];	//threadIdx.x is between (0 and threads-1)
	int * moreMoreSmemPtr = &moreSharedMem[threadIdx.x];
}

Since the thread number of threads per block can be configured at runtime it's not even like there are theoretical advantages like being able to unroll loops and hardcode offsets and stuff. The heap for __device__ function calls (not kernels) is local memory (cached global memory), not shared memory. Any memory management that you'd have to do to manage the dynamic launches would also have to be done in order to do static launches since all that is scheduled on-chip.

I don't get it.

Paul MaudDib fucked around with this message at 01:08 on May 16, 2014

# ? May 15, 2014 22:13

Edison was a dick: Apr 3, 2010; direct current only

Ciaphas posted:

Am I just blind, or is there no file copy analogue to standard C calls like unlink(), rename(), chmod(), etc.?

(edit: not that it's exactly hard to do with fopen, fwrite, fclose, but dammit it's the principle of the thing)

pseudorandom name posted:

TransmitFile() is the Windows equivalent of sendfile(), CopyFile() has no Unix/Linux equivalent.

There's splice for copying file contents without having to read the data into userland, though you have to create the destination, handle copying permissions and call splice in a loop as necessary.

If you're using btrfs as your file system, there's an ioctl to do a file clone. It's like splice, but with fewer arguments and it copies all the data at once, quickly, since it doesn't do any file data copies, it just shuffles pointers to Copy on Write blocks.

This is the best example I could find for how to use that version of the clone ioctl.

If you're ok shelling out though, GNU cp has a --reflink option, which will use the btrfs clone ioctl under the hood.

# ? May 15, 2014 22:51

pseudorandom name: May 6, 2007

Handling permissions (and extended attributes) and doing IO in a loop is the whole point of CopyFile(), which has no Unix/Linux equivalent.

# ? May 15, 2014 23:11

MrMoo: Sep 14, 2000

Of course there is: g_file_copy, although for the core OS link and symlink are the better solutions. Both exist on Windows now but poorly supported as usual by the GUI (CreateHardLink, CreateSymbolicLink).

MrMoo fucked around with this message at 01:39 on May 16, 2014

# ? May 16, 2014 01:33

b0lt: Apr 29, 2005

Ciaphas posted:

no file copy analogue

MrMoo posted:

Of course there is: g_file_copy, although for the core OS link and symlink are the better solutions.

# ? May 16, 2014 02:01

ExcessBLarg!: Sep 1, 2001

Ciaphas posted:

Am I just blind, or is there no file copy analogue to standard C calls like unlink(), rename(), chmod(), etc.?

There's no "copy file" function in GNU or (to my knowledge) BSD libc although it appears that OS X has one. In the end, this shouldn't be surprising as file copying isn't a fundamental VFS operation (which unlink, rename, chmod, all are), and furthermore, it can get quite complicated with regard to metadata (see all of cp(1)'s options), sparseness, atomicity of overwrites, etc. That is to say, if there was a copy file function in libc, it would inevitably do the wrong thing for many folks.

Neither symlinks nor hard links are a general replacement for copying a file, although one or both may be sufficient depending on the specific circumstances. Also keep in mind that both symlinks and hard links can't be used everywhere (neither are supported by all FSes, hard links can't cross file-systems and in some cases directories).

Surely there are various framework libraries that implement some kind of file copy helper function, g_file_copy sounds like one. That's arguably exactly where such functionality belongs.

All that said, I'm desperately trying to remember I had to implement "copy file" functionality into a program I wrote, that didn't otherwise modify the contents of the file being written. I actually don't think it's ever come up.

# ? May 16, 2014 02:10

The Gay Bean: Apr 19, 2004

I have a custom data class that is formed for and used by a specific algorithm. The data backing it is just a collection of 1D and 2D STL vectors and a cv::Mat (which can be stored using a 2D STL vector). I coded up serialization using boost::serialization because it's easy to read and fairly standard, but it turned out to be slow as poo poo in debug mode - taking several minutes to write / read a 1 GB archive. This is reduced to like 10 seconds in release mode, but for the time being I need to run the code in debug mode a lot, and I can't put up with those kinds of wait times.

I'm sure I could write something that works solely on ofstream objects that would be 100 times faster, but it would be messy and harder to read. But is there some magical compiler flag I'm missing (code is cross-platform MSVC 8 / GCC) or a library that does almost the same thing but isn't slow as poo poo, so I don't have to reinvent the wheel next time I want to serialize a data structure?

edit: I already turned iterator debugging off on MSVC8

The Gay Bean fucked around with this message at 11:00 on May 19, 2014

# ? May 19, 2014 10:52

nielsm: Jun 1, 2009

You can't test on a smaller dataset?

# ? May 19, 2014 11:53

Zopotantor: Feb 24, 2013; ...und ist er drin dann lassen wir ihn niemals wieder raus...

The Gay Bean posted:

I have a custom data class that is formed for and used by a specific algorithm. The data backing it is just a collection of 1D and 2D STL vectors and a cv::Mat (which can be stored using a 2D STL vector). I coded up serialization using boost::serialization because it's easy to read and fairly standard, but it turned out to be slow as poo poo in debug mode - taking several minutes to write / read a 1 GB archive. This is reduced to like 10 seconds in release mode, but for the time being I need to run the code in debug mode a lot, and I can't put up with those kinds of wait times.

Can you split the implementation to put the serialization code in a separate source file (that can be compiled in release mode)? (Of course assuming that you don't want to debug that code, specifically, and that debug mode doesn't cause changes in data layout or other incompatibilities that would preclude mixing debug- and release-compiled code.)

# ? May 19, 2014 11:59

giogadi: Oct 27, 2009

The Gay Bean posted:

I have a custom data class that is formed for and used by a specific algorithm. The data backing it is just a collection of 1D and 2D STL vectors and a cv::Mat (which can be stored using a 2D STL vector). I coded up serialization using boost::serialization because it's easy to read and fairly standard, but it turned out to be slow as poo poo in debug mode - taking several minutes to write / read a 1 GB archive. This is reduced to like 10 seconds in release mode, but for the time being I need to run the code in debug mode a lot, and I can't put up with those kinds of wait times.

I'm sure I could write something that works solely on ofstream objects that would be 100 times faster, but it would be messy and harder to read. But is there some magical compiler flag I'm missing (code is cross-platform MSVC 8 / GCC) or a library that does almost the same thing but isn't slow as poo poo, so I don't have to reinvent the wheel next time I want to serialize a data structure?

edit: I already turned iterator debugging off on MSVC8

Boost::serialization is weird. On the one hand you do get nice DRY ways of saving and loading stuff and it even supports STL; on the other hand, its binary mode is platform dependent, it's slow as poo poo, and they still can't seem to make floating point numbers roundtrip.

If your code is already entrenched in boost::serialization and you just want more speed, look into setting the flag that stops boost from doing extra bookkeeping to serialize pointers. Unfortunately I'm on my phone now and can't find the flag in the docs, but basically as long as you aren't serializing pointers you can get more speed this way.

Another choice is to just use a different library. I haven't tried it out yet, but I've heard that protocol buffers are very fast. This would however require you to do some rewriting.

# ? May 19, 2014 12:32

The Gay Bean: Apr 19, 2004

I ended up just writing my own text file format for the data, and it didn't take very long as well as being much faster. But next time the problem comes up I think I'll look into protobuf.

Thanks.

# ? May 20, 2014 03:43

roomforthetuna: Mar 22, 2005; I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

The Gay Bean posted:

I ended up just writing my own text file format for the data, and it didn't take very long as well as being much faster. But next time the problem comes up I think I'll look into protobuf.

Thanks.

Protobufs can be a bit of a pain to set up - they compile structures from a proto language [aha double meaning] into C++, which means you need to do some work in your makefiles or whatever to do that preprocessing as part of your build, so it's not quite as simple as just installing a library (though installing boost can be a bastard too so relatively it's not so bad).

Once they're set up though, they're pretty great at being convenient (if a little verbose), flexible and fast.

# ? May 20, 2014 04:45

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 13:07

Boz0r: Sep 7, 2006; The Rocketship in action.

How can I use more than 2 GB memory in Windows?

EDIT: Using Visual Studio

Boz0r fucked around with this message at 10:42 on May 27, 2014

# ? May 27, 2014 10:36

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›641 »