Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Beef
Jul 26, 2004

Jabor posted:

I assume you've got some thunk functions that take in parameters, and then instead of doing the work themselves they invoke a routine in the coprocessor.

It's more the inverse situation, there is a bunch of existing codes and they want to magically turn certain function calls into 'run on co-processor' calls.

The compiler does similar transformations for OpenMP support, which turns compound expression into thunks + call to said thunk. In absence of compiler support, you basically have to do the transformation by hand, which my colleague seems to be wanting to avoid.

rjmccall posted:

... they’re being lazy about the port.
:hmmyes:

Adbot
ADBOT LOVES YOU

matti
Mar 31, 2019

Question: I'm under the impression that C++ does not allow "static const" qualified variables to be modified, but does C have that feature?

Talking purely in terms of abstract machines, as defined by language standards.

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
If by "does not allow" you mean "it's undefined behaviour if you try it" then yes, that is also the case in C.

If you mean "will actively prevent you from shooting yourself in the foot" then no, many C++ and C compilers will happily compile many programs that attempt to modify static const values with only the smallest amount of subterfuge.

matti
Mar 31, 2019

Hmmm, reading about it I think I have placed too much faith in random peoples bad interpretations of the language standard.

chglcu
May 17, 2007

I'm so bored with the USA.
There comes a time when you just have to trust your fellow programmers to not be actively malicious. Any bad effects from casting away constness falls pretty squarely into “well, don’t do that, dumbass” territory for me.

matti
Mar 31, 2019

Absolutely. I'm only thinking about this for documentation purposes. Got to be exact.

roomforthetuna
Mar 22, 2005

I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

Jabor posted:

If you mean "will actively prevent you from shooting yourself in the foot" then no, many C++ and C compilers will happily compile many programs that attempt to modify static const values with only the smallest amount of subterfuge.
I think they *sometimes* compile static const things into a memory space that's read-only, right? Like it'll still compile, but then it'll segfault or similar when it reaches the code that tries to write to it?

After looking it up to make sure I'm not making this up, yeah, when the operating system and processor combination supports it, "protected" or "data section" memory, where the operating system and/or processor will deny writes and execution.

Absurd Alhazred
Mar 27, 2010

by Athanatos

roomforthetuna posted:

I think they *sometimes* compile static const things into a memory space that's read-only, right? Like it'll still compile, but then it'll segfault or similar when it reaches the code that tries to write to it?

After looking it up to make sure I'm not making this up, yeah, when the operating system and processor combination supports it, "protected" or "data section" memory, where the operating system and/or processor will deny writes and execution.

Does that mean that an object with just one mutable member would have to be in a separate data section?

Jeffrey of YOSPOS
Dec 22, 2005

GET LOSE, YOU CAN'T COMPARE WITH MY POWERS

roomforthetuna posted:

I think they *sometimes* compile static const things into a memory space that's read-only, right? Like it'll still compile, but then it'll segfault or similar when it reaches the code that tries to write to it?

After looking it up to make sure I'm not making this up, yeah, when the operating system and processor combination supports it, "protected" or "data section" memory, where the operating system and/or processor will deny writes and execution.
Yeah the "rodata" section of an elf executable is for read-only data. Your compile-time strings will end up in there, generally. No idea if static const stuff does.

roomforthetuna
Mar 22, 2005

I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

Absurd Alhazred posted:

Does that mean that an object with just one mutable member would have to be in a separate data section?
An object is always going to be in the regular mutable memory, because you have to allocate and write it in the first place. I think only compile-time const would go in the read-only area, so I guess it's things that could be "constexpr", but I expect compilers would intuit things that are equivalent to constexpr (e.g. static const value types) and put them there.

Edit: but I'm not an authority on this and welcome correction if I'm wrong!

Absurd Alhazred
Mar 27, 2010

by Athanatos

roomforthetuna posted:

An object is always going to be in the regular mutable memory, because you have to allocate and write it in the first place. I think only compile-time const would go in the read-only area, so I guess it's things that could be "constexpr", but I expect compilers would intuit things that are equivalent to constexpr (e.g. static const value types) and put them there.

Edit: but I'm not an authority on this and welcome correction if I'm wrong!

I was thinking something like a static constant that was an object. Possibly could be constexpred, but if it has just one mutable member for later you'd have to put it somewhere else.

roomforthetuna
Mar 22, 2005

I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

Absurd Alhazred posted:

I was thinking something like a static constant that was an object. Possibly could be constexpred, but if it has just one mutable member for later you'd have to put it somewhere else.
Oh wait, do you mean mutable like the keyword mutable?

If it has a normal mutable member (i.e. a member that can be modified) then it can't be const or constexpr. (I mean you can make a const instance of such a class but then the member is not mutable because the instance is const.) And if it has a const member that's a pointer to something mutable, it still can't be constexpr.

If it has the keyword mutable on a member then it's only really pretend const and could not be constexpr.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
There are four situations in which compilers will put const objects of static storage duration in writable memory:

  • If the object contains a mutable member, it must go in writable memory.
  • If the object has a destructor, it must go in writable memory.
  • If the object requires a dynamic initializer, then it will need to be in writable memory during that phase of loading. In principle, the loader could make that memory read-only after global constructors run, but I don't know of any system that does that; among other things, that would require those objects to go onto a separate segment, which is a pretty big memory penalty. But if the compiler can optimize a dynamic initializer into a constant initializer, it might be able to make the object read-only.
  • If the object's initializer contains pointers to other objects or functions (including v-tables), it needs to go in writable memory during load unless all the symbols resolve within the current linked image and the image isn't PIC. Traditionally, that means that the object just goes in writable memory forever. However, a lot of systems have been adding mitigations to put such objects in a special segment which will get remapped read-only after relocations are applied; in ELF, this is called RELRO, and I believe it's still generally opt-in. This covers a lot of very important cases like v-tables and the non-lazy part of the GOT.

Foxfire_
Nov 8, 2010

  • The compiler feels like it
It's never mandatory to use read-only memory (that might not even exist on the target platform) for anything.
Those 4 situations are where read-only can't work, but its still legal to just put absolutely everything in writeable memory even if they can never be legally written.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
Okay, but I was answering an actual question

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
What is current industry standard for C/C++ dependency management? Is it as big of a mess as it seems?

I've seen projects that check in source for all of their dependencies, or projects that use Git submodules to include complete repos of their dependencies, I've seen projects that say "this needs armadillo" with no mention of Armadillo version, I've seen a mix of all three of the above in a single project.

Just today I was installing https://github.com/xqwen/fastenloc, which doesn't have a word about its dependencies anywhere in the documentation, so I built it by running Make, seeing what headers it couldn't find, and installing packages based on header names. Is this normal? Is there a tool that can look at a project and tell me what dependencies it needs so that I wouldn't need to do repeated trial and error?

Also, they included a statically linked target for that application, but when building it I am warned:
code:
/usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function `gomp_target_init':
(.text+0x8b): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
This sounds to me like it will now only work against a single specific version of glibc, meaning that these static binaries are no longer portable between Linux distros with a different version of glibc?

Twerk from Home fucked around with this message at 02:09 on Sep 16, 2021

Foxfire_
Nov 8, 2010

rjmccall posted:

Okay, but I was answering an actual question
Not putting everything that could potentially be in read-only memory into it isn't uncommon behavior.

VS2019 for example takes:
code:
#include <cstdio>

const char formatString[] = "X=%d\n";

class Foo
{
public:
    Foo() : x(5) {}
    void PrintIt() const { printf(formatString, this->x); }

private:
    int x;
};

const static Foo foo1;
const Foo foo2;
static Foo foo3;
Foo foo4;

int main()
{
    foo1.PrintIt();
    foo2.PrintIt();
    foo3.PrintIt();
    foo4.PrintIt();
    
    return 0;
}
and puts all four Foo into normal writeable .data, not .rdata (formatString gets put in .rdata)

qsvui
Aug 23, 2003
some crazy thing

Twerk from Home posted:

What is current industry standard for C/C++ dependency management? Is it as big of a mess as it seems?

From what I've seen, yes. Some options I've come across are vcpkg and Conan. CMake might also be able to do something similar using modules such as FetchContent.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

Foxfire_ posted:

VS2019 for example takes:
and puts all four Foo into normal writeable .data, not .rdata (formatString gets put in .rdata)

Yes, because this uses dynamic initialization.

Foxfire_
Nov 8, 2010

I don't understand what you mean by dynamic initialization. I thought you meant object's whose construction/initialization involved running code at runtime.
In that example, Foo::x is getting values from the loader (.data is load-time initialized, they're not in .bss). They're already set at the very first instruction in the process, before the C runtime starts or any C++ constructors run.

Xarn
Jun 26, 2015

Twerk from Home posted:

What is current industry standard for C/C++ dependency management? Is it as big of a mess as it seems?

I've seen projects that check in source for all of their dependencies, or projects that use Git submodules to include complete repos of their dependencies, I've seen projects that say "this needs armadillo" with no mention of Armadillo version, I've seen a mix of all three of the above in a single project.

Just today I was installing https://github.com/xqwen/fastenloc, which doesn't have a word about its dependencies anywhere in the documentation, so I built it by running Make, seeing what headers it couldn't find, and installing packages based on header names. Is this normal? Is there a tool that can look at a project and tell me what dependencies it needs so that I wouldn't need to do repeated trial and error?

Also, they included a statically linked target for that application, but when building it I am warned:
code:
/usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function `gomp_target_init':
(.text+0x8b): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
This sounds to me like it will now only work against a single specific version of glibc, meaning that these static binaries are no longer portable between Linux distros with a different version of glibc?

It's a mess and will probably get worse before it gets better due to modules being weird and supporting them in build systems being hard.

Use vcpkg or conan, I prefer vcpkg slightly

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
What I am saying is that there are some basic conditions which, for major compilers on platforms that support read-only segments (which, given security necessities, is basically everything that's not super-embedded or a toy), will reliably cause const objects to end up in those segments. Sometimes objects that don't meet those conditions will also end up in read-only segments, but that's not a contradiction.

In your example, those objects are being initialized with a non-constexpr constructor. So yes, subject to optimization, initializing them will involve running code at runtime. What you are seeing is that MSVC's optimizer is smart enough to inline and fold the running of those constructors into constant initialization, but isn't quite smart enough to realize that, having completely eliminated the dynamic part of their initialization, it can subsequently put the objects into read-only segments. Clang's optimizer is now smart enough to do that, but it wasn't always, and I can understand why MSVC doesn't do it.

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
Are there any compilers that do things like "I'm going to put this object next to other things that are used at the same time for better memory locality, even though it's not the same section I'd normally use for it"?

Or is that sort of broad memory access thing generally considered beyond what the compiler can reason about, and it's up to the programmer to coerce the desired behaviour if they have a specific optimization they want?

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
Well, there are lots of papers about optimizing for locality. It’s a great scam: every five years, you can write a paper about some slightly novel form of locality optimization and show some nice wins on targeted benchmarks. I don’t think any common compilers actually do it, though, which is why the scam keeps working. You really need something like trace profiles to do it right, and even basic PGO comes with a lot of difficulties.

Also, linkers don’t usually promise to maintain object-file adjacency, so to do it you’d be fighting the rest of the toolchain a bit.

I know the Darwin linker has some code-locality optimizations, basically trying to put code that’ll be run during app launch on the same pages.

nielsm
Jun 1, 2009



roomforthetuna posted:

I think they *sometimes* compile static const things into a memory space that's read-only, right? Like it'll still compile, but then it'll segfault or similar when it reaches the code that tries to write to it?

After looking it up to make sure I'm not making this up, yeah, when the operating system and processor combination supports it, "protected" or "data section" memory, where the operating system and/or processor will deny writes and execution.

I'm pretty sure there are also some architectures where accessing true globals is troublesome, and it might actually make sense for the compiler to make multiple copies of the constant data. Is that allowed, letting the same named constant have different addresses depending on where in the program you are?

Qwertycoatl
Dec 31, 2008

nielsm posted:

I'm pretty sure there are also some architectures where accessing true globals is troublesome, and it might actually make sense for the compiler to make multiple copies of the constant data. Is that allowed, letting the same named constant have different addresses depending on where in the program you are?

It should be fine if nothing looks at the addresses

roomforthetuna
Mar 22, 2005

I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

rjmccall posted:

However, a lot of systems have been adding mitigations to put such objects in a special segment which will get remapped read-only after relocations are applied; in ELF, this is called RELRO, and I believe it's still generally opt-in. This covers a lot of very important cases like v-tables and the non-lazy part of the GOT.
That was interesting stuff I wasn't previously aware of, thanks!

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

Qwertycoatl posted:

It should be fine if nothing looks at the addresses

Yeah, that’s the critical limitation. Objects (and functions!) with external linkage have to behave like they have a consistent address when that address is observed. But note that (in C++ only) const variables at global or namespace scope have internal linkage unless explicitly declared extern.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
I've got some really basic Make questions, and realized that I have never really learned make, but rather just muddled through. It looks like I'm going to be spending a lot more time in C/C++ than I ever have over the last decade, so it's time to finally learn Make. Is there a better place to start than https://www.gnu.org/software/make/manual/make.html ?

I'm going to be dealing with a lot of scientific projects that probably only ever worked at a single university in their default computing environment, without any thought to portability. While my group is installing & using these projects I'm hoping that I can improve documentation and the build process a bit where it's easily possible. Here's a great example of a Makefile that I'm starting with: https://github.com/xqwen/fastenloc/blob/master/src/Makefile

code:
main: main.o controller.o sigCluster.o
	g++ -fopenmp -O3 main.o controller.o sigCluster.o  -lm -lgsl -lgslcblas -lboost_iostreams -lz  -o fastenloc
static: main.o controller.o sigCluster.o
	g++ -fopenmp -O3 main.o controller.o sigCluster.o  -lm -lgsl -lgslcblas -lboost_iostreams -lz  -static -o fastenloc.static
main.o: main.cc
	g++ -c  main.cc
controller.o: controller.cc controller.h
	g++ -fopenmp   -c  controller.cc
sigCluster.o: sigCluster.h sigCluster.cc
	g++ -c sigCluster.cc 
clean:
	rm *.o fastenloc
One thing that jumps out at me is that they forgot to -O3 where they actually compiled the code, meaning that the binary that this group uses is entirely unoptimized, right? Binary size changed a lot when I applied the -O3 flag to where we're actually compiling the .cc files, and it ran faster in some very small test data sets.

My goals to improve this are to:
  • Allow environment variables to extend or change compiler options
  • Let it build on GCC or Clang, and ideally on Mac or Linux
  • Turn on some warnings because the code quality is pretty awful

This is as far as I got before I realized I should read Make documentation, because I have no idea if what any of what I'm doing is following norms, nor do I have a good solution right now to be able to stick "-march=broadwell" or similar on there via an environment variable. I've also added the include and library paths for libraries installed via Homebrew on Mac, but I only want those to be present if we're building on a Mac. I'm also not sure how to handle clang vs gcc, because gcc just needs -fopenmp while clang needs that to be passed to the preprocessor with -Xpreprocessor.

code:
CXX=g++
CXXFLAGS=-O3 -Wall
INC=-isystem /opt/homebrew/include # Mac libraries installed via Homebrew
LIBS=-lm -lgsl -lgslcblas -lboost_iostreams -lz -lomp
LDFLAGS=-L/opt/homebrew/lib

all: main

main: main.o controller.o sigCluster.o
	$(CXX) $(CXXFLAGS) $(INC) -Xpreprocessor -fopenmp $(LDFLAGS) main.o controller.o sigCluster.o $(LIBS) -o fastenloc
static: main.o controller.o sigCluster.o
	$(CXX) $(CXXFLAGS) $(INC) -Xpreprocessor -fopenmp $(LDFLAGS) main.o controller.o sigCluster.o $(LIBS) -static -o fastenloc.static
main.o: main.cc
	$(CXX) $(CXXFLAGS) $(INC) -c  main.cc
controller.o: controller.cc controller.h
	$(CXX) $(CXXFLAGS) $(INC) -Xpreprocessor -fopenmp -c  controller.cc
sigCluster.o: sigCluster.h sigCluster.cc
	$(CXX) $(CXXFLAGS) $(INC) -c sigCluster.cc
clean:
	rm *.o fastenloc

Can I get some tips? Should I just read the Make documentation and then come back and try again? Did I miss something in the original Makefile as distributed where it was actually optimizing, and I just misunderstood?

If I get this one cleaned up, there's a related, even more widely used project that has -fpermissive all over the place that I'd like to clean up.

Edit: It looks like to allow CXXFLAGS to be extended by environment variables, I just need to do
code:
CXXFLAGS+=-O3 -Wall
and then call make with an environment variable set, like CXXFLAGS=-march=broadwell make

Twerk from Home fucked around with this message at 21:38 on Sep 16, 2021

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
I haven’t been closely following the high-level story for OpenMP support in clang, but that thing about -Xpreprocessor doesn’t sound right. I know it was changing a lot from release to release a few years ago.

Xarn
Jun 26, 2015
I would go to CMake and avoid Make like the plague, but if you end up with Make, look up how to get header dependencies from compiler.

This can get you started: https://codingnest.com/basic-makefiles/

pseudorandom name
May 6, 2007

I’d use meson instead of cmake if you don’t have to support Windows

leper khan
Dec 28, 2010
Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.
Since we're talking about build systems... What should I use to target multiple platforms each with different compilers/toolchains? Not all platforms are valid on all host machines (probably going to set up CI at some point to deal with that..)

I have some games I'm writing for GBA and wonderswan color, probably going to port to additional platforms later. Right now everything is forks with a makefile per target.

Is there some thing in cmake that makes this simple? Or at least not as manually hellish as it is now?

ultrafilter
Aug 23, 2007

It's okay if you have any questions.


CMake is a high-level scripting language that's used to generate a makefile. On any platform where it's supported, it is absolutely going to be easier to use than an actual platform-specific makefile if you have a project of any real complexity. The learning curve is not entirely gentle, but I think it's worth it.

chglcu
May 17, 2007

I'm so bored with the USA.
Yeah, CMake sucks, but it probably sucks the least out of available options. Probably want to make sure you follow a guide using modern CMake since it has a lot of cruft and "wrong" ways to do things.

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Might be an odd question, but are there any standards for C++ coding like PEP8 for Python? Things like standards for variable/class/function naming, code line length, etc.

Also, the Doxygen docs aren't much help for how to construct docstrings for C++, if those are a thing. I'm coming from Python, obviously, and there's something there like:

Python code:
def foo(bar: int):
    """
    Performs such and such operation.

    Rotates the loofah around the stick such that the notches
    line up and the loofah can be removed from the stick.

    :param int bar: Degrees of rotation needed.
    :return bool: True for success, false for failure.
    """
    pass

Brownie
Jul 21, 2007
The Croatian Sensation
Premake is also good, I personally prefer it to CMake.

Xerophyte
Mar 17, 2008

This space intentionally left blank
For the specific problem of making CMake work with a platform-specific toolchain, there's a specific set of toolchain options you define. Usually in a platform-specific toolchain.txt, then you launch cmake with cmake --toolchain path/to/toolchain.cmake. For a hobby project this will probably be quite straight-forward, you just set the paths to the compiler, linker, archiver, etc plus some default flags. For a real project that other people will use you might need to dive into specifying detailed c++ feature set support and things like that, but in this case you almost certainly don't need to.

With the toolchain defined you write normal CMake and it Should All Work for all your portable gaming device so-and-so.


Edit, re: quality of CMake vs other options, I have all sorts of problems with it as a language but it works to define whatever you need for your build without too much hassle and it's the de-facto standard for any cross-platform C/C++ project. I'd say it's the first build system you should learn, even though it is undeniably crufty.

Xerophyte fucked around with this message at 17:38 on Sep 17, 2021

leper khan
Dec 28, 2010
Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

Xerophyte posted:

For the specific problem of making CMake work with a platform-specific toolchain, there's a specific set of toolchain options you define. Usually in a platform-specific toolchain.txt, then you launch cmake with cmake --toolchain path/to/toolchain.cmake. For a hobby project this will probably be quite straight-forward, you just set the paths to the compiler, linker, archiver, etc plus some default flags. For a real project that other people will use you might need to dive into specifying detailed c++ feature set support and things like that, but in this case you almost certainly don't need to.

With the toolchain defined you write normal CMake and it Should All Work for all your portable gaming device so-and-so.


Edit, re: quality of CMake vs other options, I have all sorts of problems with it as a language but it works to define whatever you need for your build without too much hassle and it's the de-facto standard for any cross-platform C/C++ project. I'd say it's the first build system you should learn, even though it is undeniably crufty.

:tipshat:

Adbot
ADBOT LOVES YOU

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Why might g++ decide to just skip a null check? I am assigning a pointer with a helper function that instead returns NULL. I then test NULL with an if-statement. In the debugger, I can see the pointer is NULL. It then rockets right past the conditional and goes straight into trying to use it.

I've tried to disable optimizations with -O0 and then disable null pointer checks with -fno-delete-null-pointer-checks but it still skips over the check.

Edit: I figured out my big stupid. I was just making GBS threads out the return code without actually using the return statement so the compiler had determined it was a pointless fart even at O0 and skipped it.

Rocko Bonaparte fucked around with this message at 18:57 on Sep 17, 2021

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply