Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde
If Indexer and Robot are part of the same design and you can add a copy constructor to Indexer, then add it. If not, define a wrapper for Indexer that does shared reference counting the way shared_ptr does. Or if you're in a hurry and these classes are not for third-party use, just use shared_ptr.

KNITS MY FEEDS posted:

e: Making the constructor accept Indexer *index_service and made the member variable Indexer* index_service seems to have worked but I'm not sure how right that was.
It's OK as long as you also provide for the Indexer to be destroyed after the last Robot using it is destroyed. If you know for certain that the Indexer and all the Robots using it will always be created and destroyed in the same block, an auto_ptr will do that.

Gazpacho fucked around with this message at 05:26 on Sep 3, 2012

Adbot
ADBOT LOVES YOU

Acer Pilot
Feb 17, 2007
put the 'the' in therapist

:dukedog:

Gazpacho posted:

If Indexer and Robot are part of the same design and you can add a copy constructor to Indexer, then add it. If not, define a wrapper for Indexer that does shared reference counting the way shared_ptr does. Or if you're in a hurry and these classes are not for third-party use, just use shared_ptr.

It's OK as long as you also provide for the Indexer to be destroyed after the last Robot using it is destroyed. If you know for certain that the Indexer and all the Robots using it will always be created and destroyed in the same block, an auto_ptr will do that.

Thanks for the answer. Indexer should theoretically only be destroyed after all the Robots have called their destructors, they may call their destructors at different times. If I called delete index_service in Robot's desructor, would that delete the Indexer all the other instances of Robot are using? I don't want any memory leaks but I still want Indexer to run on the other instances of Robot.

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde

KNITS MY FEEDS posted:

If I called delete index_service in Robot's desructor, would that delete the Indexer all the other instances of Robot are using?
As the code currently stands (using a real pointer), yes it would and that would be bad.

Acer Pilot
Feb 17, 2007
put the 'the' in therapist

:dukedog:

Gazpacho posted:

As the code currently stands (using a real pointer), yes it would and that would be bad.

Does auto_ptr free up that memory after the program finally completes or is this going to leak?

(Robots run in threads if that matters)

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde
auto_ptr deletes its object when the auto_ptr's destructor runs, according to the same destruction rules as any other instance. As you've written it, the Robots and then the Indexer will be automatically destroyed at the end of the block that created them.

raminasi
Jan 25, 2005

a last drink with no ice

KNITS MY FEEDS posted:

Does auto_ptr free up that memory after the program finally completes or is this going to leak?

(Robots run in threads if that matters)

Most operating systems automatically free all of a program's memory when it completes (the exceptions are some embedded systems, I believe - this isn't my area of expertise). A memory "leak" is when a program permanently loses access to memory, not the whole system.

pseudorandom name
May 6, 2007

And note that thread termination isn't sufficient, the malloc arena is global to the process.

That Turkey Story
Mar 30, 2003

Does anyone *cough* rjmccall *cough* know what it could mean when I'm working with Clang and get:

my_cpp_filename_here-490634.s|6162|Error: missing or invalid immediate expression `72057594037927944'

I wish I could come up with a simple, reduced test-case, but it's a huge metaprogram.

Edit: I think I'm hitting some kind of compiler limit -- I'm working on a tensor library and when testing any tensors with a rank below 6, all my tests compile and work fine. Tensors of rank 6 and 7 cause the compiler to stop responding, and tensors of any rank greater than 7 cause an error like the one mentioned above. I was originally testing up to rank 15 in Clang and everything worked fine, but after some backend changes I've encountered the bugs.

That Turkey Story fucked around with this message at 02:24 on Sep 7, 2012

MrMoo
Sep 14, 2000

Presumably an overflow error when either generating or passing the intermediate assembler. Get Clang to only generate the assembler and see if that barfs or at least try and find the broken line.

This may be helpful:

http://stackoverflow.com/questions/10990018/how-to-generate-assembly-code-with-clang-in-intel-syntax

pseudorandom name
May 6, 2007

That's an error from gas, not clang/llvm.

ZombieApostate
Mar 13, 2011
Sorry, I didn't read your post.

I'm too busy replying to what I wish you said

:allears:
I'm working on a project that I want to be Unicode friendly, so I'm trying to follow the guidelines laid out here (basically, use UTF-8 everywhere you can, all char[]s/strings should be UTF-8, convert to something else immediately before/after you absolutely have to), but for some reason my conversion functions aren't working. At least I think that's the problem.

Conversion functions:
C++ code:
	std::string narrow(const wchar_t* _input)
	{
		int size = WideCharToMultiByte(CP_UTF8, 0, _input, -1, nullptr, 0, nullptr, false);
		//int size = WideCharToMultiByte(CP_ACP, WC_COMPOSITECHECK, _input, -1, nullptr, 0, nullptr, false);
		char* narrowString = new char[size];
		size = WideCharToMultiByte(CP_UTF8, 0, _input, -1, narrowString, size, nullptr, false);
		//size = WideCharToMultiByte(CP_ACP, WC_COMPOSITECHECK, _input, -1, narrowString, size, nullptr, false);
		DWORD error = GetLastError();
		std::string output(narrowString);
		delete [] narrowString;
		return output;
	}

	std::wstring widen(const char* _input)
	{
		int size = MultiByteToWideChar(CP_UTF8, 0, _input, -1, nullptr, 0);
		//int size = MultiByteToWideChar(CP_ACP, 0, _input, -1, nullptr, 0);
		wchar_t* wideString = new wchar_t[size];
		size = MultiByteToWideChar(CP_UTF8, 0, _input, -1, wideString, size);
		//size = MultiByteToWideChar(CP_ACP, 0, _input, -1, wideString, size);
		DWORD error = GetLastError();
		std::wstring output(wideString);
		delete [] wideString;
		return output;
	}
Test:
C++ code:
	//test1
	std::wstring wideStr = L"UNICODE goes here";
	std::string narrowStr = UnicodeHelper::narrow(wideStr.c_str());
	wideStr = UnicodeHelper::widen(narrowStr.c_str());

	//test2
	narrowStr = "UNICODE goes here";
	wideStr = UnicodeHelper::widen(narrowStr.c_str());
	narrowStr = UnicodeHelper::narrow(wideStr.c_str());
The test strings are "UNICODE, I hate you with a passion" machine translated to russian and hardcoded because I'm lazy. I've got a console and log file with the same sort of tests. If I feed CP_ACP to WideCharToMultiByte/MultiByteToWideChar (like the commented out lines), everything seems to work. If I use UTF8 like everything I've seen says you're supposed to, I get gibberish. What am I doing wrong? :qq:

Edit: Apparently you have to set the flags parameter to 0 if you're using CP_UTF8, which fixes the first test, sort of, but the second one still gives me gibberish.

The tests look like this as I step through the code:
Test 1 Start: "UNICODE, <russian>"
After narrow: "UNICODE, <random japanese characters>" (I assume that's just because Visual Studio doesn't like displaying UTF-8 and my language for non-unicode programs is set to japanese?)
After widen: "UNICODE, <russian>"

Test 2 Start: "UNICODE, <russian>"
After widen: "UNICODE, <mostly diamond question mark symbols with a few english characters>"
After narrow: "UNICODE, <random japanese characters>"

ZombieApostate fucked around with this message at 06:05 on Sep 7, 2012

Acer Pilot
Feb 17, 2007
put the 'the' in therapist

:dukedog:

ZombieApostate posted:

I'm working on a project that I want to be Unicode friendly, so I'm trying to follow the guidelines laid out here (basically, use UTF-8 everywhere you can, all char[]s/strings should be UTF-8, convert to something else immediately before/after you absolutely have to), but for some reason my conversion functions aren't working. At least I think that's the problem.

Conversion functions:
C++ code:
	std::string narrow(const wchar_t* _input)
	{
		int size = WideCharToMultiByte(CP_UTF8, 0, _input, -1, nullptr, 0, nullptr, false);
		//int size = WideCharToMultiByte(CP_ACP, WC_COMPOSITECHECK, _input, -1, nullptr, 0, nullptr, false);
		char* narrowString = new char[size];
		size = WideCharToMultiByte(CP_UTF8, 0, _input, -1, narrowString, size, nullptr, false);
		//size = WideCharToMultiByte(CP_ACP, WC_COMPOSITECHECK, _input, -1, narrowString, size, nullptr, false);
		DWORD error = GetLastError();
		std::string output(narrowString);
		delete [] narrowString;
		return output;
	}

	std::wstring widen(const char* _input)
	{
		int size = MultiByteToWideChar(CP_UTF8, 0, _input, -1, nullptr, 0);
		//int size = MultiByteToWideChar(CP_ACP, 0, _input, -1, nullptr, 0);
		wchar_t* wideString = new wchar_t[size];
		size = MultiByteToWideChar(CP_UTF8, 0, _input, -1, wideString, size);
		//size = MultiByteToWideChar(CP_ACP, 0, _input, -1, wideString, size);
		DWORD error = GetLastError();
		std::wstring output(wideString);
		delete [] wideString;
		return output;
	}
Test:
C++ code:
	//test1
	std::wstring wideStr = L"UNICODE goes here";
	std::string narrowStr = UnicodeHelper::narrow(wideStr.c_str());
	wideStr = UnicodeHelper::widen(narrowStr.c_str());

	//test2
	narrowStr = "UNICODE goes here";
	wideStr = UnicodeHelper::widen(narrowStr.c_str());
	narrowStr = UnicodeHelper::narrow(wideStr.c_str());
The test strings are "UNICODE, I hate you with a passion" machine translated to russian and hardcoded because I'm lazy. I've got a console and log file with the same sort of tests. If I feed CP_ACP to WideCharToMultiByte/MultiByteToWideChar (like the commented out lines), everything seems to work. If I use UTF8 like everything I've seen says you're supposed to, I get gibberish. What am I doing wrong? :qq:

Edit: Apparently you have to set the flags parameter to 0 if you're using CP_UTF8, which fixes the first test, sort of, but the second one still gives me gibberish.

The tests look like this as I step through the code:
Test 1 Start: "UNICODE, <russian>"
After narrow: "UNICODE, <random japanese characters>" (I assume that's just because Visual Studio doesn't like displaying UTF-8 and my language for non-unicode programs is set to japanese?)
After widen: "UNICODE, <russian>"

Test 2 Start: "UNICODE, <russian>"
After widen: "UNICODE, <mostly diamond question mark symbols with a few english characters>"
After narrow: "UNICODE, <random japanese characters>"

You could try to use UTF8-CPP to make your life a little simpler..

ZombieApostate
Mar 13, 2011
Sorry, I didn't read your post.

I'm too busy replying to what I wish you said

:allears:
I think the only case that isn't working now is if I try to start from a normal string literal that is using non-ASCII characters. I assume that gets encoded as whatever the current code page on your system is? It didn't seem to make a difference if I told it to save that file as UTF-8. I doubt there's a reliable way to go from that to a UTF-8 encoded string since I can't control what code page someone else is using and if it doesn't match whatever the string is encoded as, it won't convert properly. And it probably doesn't matter since I'm not going to hard code string literals outside of testing and all my input files will be UTF-8 anyways.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

That Turkey Story posted:

Does anyone *cough* rjmccall *cough* know what it could mean when I'm working with Clang and get:

my_cpp_filename_here-490634.s|6162|Error: missing or invalid immediate expression `72057594037927944'

I wish I could come up with a simple, reduced test-case, but it's a huge metaprogram.

Edit: I think I'm hitting some kind of compiler limit -- I'm working on a tensor library and when testing any tensors with a rank below 6, all my tests compile and work fine. Tensors of rank 6 and 7 cause the compiler to stop responding, and tensors of any rank greater than 7 cause an error like the one mentioned above. I was originally testing up to rank 15 in Clang and everything worked fine, but after some backend changes I've encountered the bugs.

As someone else mentioned, that's not a clang/llvm diagnostic. That doesn't mean it isn't clang/llvm bug — we're either trying to emit an invalid operand, or we're actually internally corrupted and spitting out garbage. The latter would probably trigger an assert if you tried it with an +Asserts build of clang.

OneEightHundred
Feb 28, 2008

Soon, we will be unstoppable!
Aliasing rules are bothering me. Long story short, I need a version of memcpy that is guaranteed to copy aligned pointers atomically (on any platform I care about, that means loading and storing as a pointer-sized value), but the data can contain anything. I'm pretty sure the stock memcpy will do this in most scenarios, but I can't be 100% sure it won't optimize to rep stos or something like that.

At the same time, I'm trying to stay strict aliasing clean and it looks like that can only be accomplished if I was copying char-sized data, or was using a union, but a union will only guarantee aliasing of types contained in the union.

Is there a way to do this? Would declaring the destination volatile enforce that potentially-aliased values are reloaded after?

OneEightHundred fucked around with this message at 21:12 on Sep 7, 2012

floWenoL
Oct 23, 2002

OneEightHundred posted:

Aliasing rules are bothering me. Long story short, I need a version of memcpy that is guaranteed to copy aligned pointers atomically (on any platform I care about, that means loading and storing as a pointer-sized value), but the data can contain anything. I'm pretty sure the stock memcpy will do this in most scenarios, but I can't be 100% sure it won't optimize to rep stos or something like that.

At the same time, I'm trying to stay strict aliasing clean and it looks like that can only be accomplished if I was copying char-sized data, or was using a union, but a union will only guarantee aliasing of types contained in the union.

Is there a way to do this? Would declaring the destination volatile enforce that potentially-aliased values are reloaded after?

What's wrong with casting your pointers to char pointers, which can alias anything?

OneEightHundred
Feb 28, 2008

Soon, we will be unstoppable!

floWenoL posted:

What's wrong with casting your pointers to char pointers, which can alias anything?
The problem is that if they're copied as chars, then the pointer-sized value copies may not be atomic, which is necessary in this case. I'm pretty sure recasting the pointers won't fix that since aliasing affects value load/stores, not pointer conversion.

pseudorandom name
May 6, 2007

Then cast them back to pointers to word sized values.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

pseudorandom name posted:

Then cast them back to pointers to word sized values.

That's not how aliasing works.

OneEightHundred: if you can use compiler-specific stuff, GCC has an attribute may_alias which grants char-like powers to an arbitrary type. Alternatively, if you put your memcpy in a file by itself, I promise you will not have optimization problems, even if that one function technically violates aliasing rules.

pseudorandom name
May 6, 2007

edit: wrong button

OneEightHundred
Feb 28, 2008

Soon, we will be unstoppable!

rjmccall posted:

Alternatively, if you put your memcpy in a file by itself, I promise you will not have optimization problems, even if that one function technically violates aliasing rules.
Does that still work even with whole program optimization?

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

OneEightHundred posted:

Does that still work even with whole program optimization?

No, if you have some sort of whole-program optimization enabled, and you can't exclude that file, it's going to be a problem. Usually there's some way to mark the function as not being inlinable, or if all else fails you can write it in assembly. What compiler are you using?

OneEightHundred
Feb 28, 2008

Soon, we will be unstoppable!
Visual Studio, but there'll invariably be a GCC port at some point.

e: Apparently VS took out the aliasing optimizations in 2005 and replaced them with a function-level keyword. :psyduck:

OneEightHundred fucked around with this message at 22:42 on Sep 7, 2012

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
Yeah, MSVC is now very conservative about alias analysis; I guess they got burned by being too aggressive. GCC's TBAA is way too aggressive, really, and it basically miscompiles so much obvious code that people just turn the option off. Clang has a strategy which would let us use TBAA but hopefully not miscompile the obvious cases that GCC miscompiles; unfortunately, we haven't had enough time to do the validation that would make us feel confident about enabling it by default.

rjmccall fucked around with this message at 22:53 on Sep 7, 2012

floWenoL
Oct 23, 2002

OneEightHundred posted:

The problem is that if they're copied as chars, then the pointer-sized value copies may not be atomic, which is necessary in this case. I'm pretty sure recasting the pointers won't fix that since aliasing affects value load/stores, not pointer conversion.

Wait, so you're relying on the value type for atomic copying? i.e., are you saying you think that if you treat the array as a word array instead of a char array, it'll be atomic? That seems dangerous. I don't think there's any guarantee as to atomicity of value copies based on type. You may have to use atomic instructions. If so, then you can just use char *.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
It's definitely not guaranteed (well, except in Java), but in practice, loads and stores of small, aligned, scalar, non-bitfield objects will always be emitted in a way that does not admit the possibility of partial writes. If that's all you need, you can skate by pretty effectively without a formal guarantee.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
It's actually quite similar to the use of volatile objects to do memory-mapped I/O, which relies for correctness on the same not-technically-guaranteed assumptions about which instructions are going to be used to perform the operation. Deep systems programmers seem very comfortable relying on this kind of thing. Well, right up until they run into something they didn't expect, at which point they start insisting that they know the language better than you.

OneEightHundred
Feb 28, 2008

Soon, we will be unstoppable!

floWenoL posted:

You may have to use atomic instructions. If so, then you can just use char *.
Like I said, char is out because char will emit as multiple single-byte copies, due to the possibility of the memory regions overlapping. I'm pretty sure copying it as a larger value forces the compiler to emit instructions that will fully read the value before writing it out.

Also I don't need them to be atomic in the sense of surviving load/store reordering, I just need them to be atomic in the sense that partially-written memory addresses will never be visible to another thread. In practice, every platform I can imagine caring about will guarantee write atomicity of pointer-sized values to aligned addresses, which is all I want. I just want to be able to do it without the compiler deciding that values it reads from the destination area haven't changed.

covener
Jan 10, 2004

You know, for kids!

OneEightHundred posted:

Like I said, char is out because char will emit as multiple single-byte copies, due to the possibility of the memory regions overlapping.

I have to confess to not following the topic, but memcpy does not tolerate overlapping arguments. Maybe that tainted some previous test?

OneEightHundred
Feb 28, 2008

Soon, we will be unstoppable!

covener posted:

I have to confess to not following the topic, but memcpy does not tolerate overlapping arguments. Maybe that tainted some previous test?
The problem is that I'm not using memcpy, I need something that will guarantee a particular behavior: That pointer values contained in the source data (which can be mixed data, but the pointer values will be aligned) will not be visible in a partially-written state at the destination to other threads.

memcpy does this on Windows at least and glibc probably does it too, but it's not guaranteed and intrinsics may cause other behavior (like emitting rep stos).

Implementing it using chars, i.e.
code:
while(numBytes--) *destChar++ = *srcChar++;
...will not work because the compiler can't be sure that destChar and srcChar don't overlap, so it will emit the copies as single-byte read/writes to ensure the correct behavior if, for example, srcChar is only offset 2 bytes from destChar.

Copying it as an array of pointers will work, but the problem is that compilers are allowed to assume that, say, a float value read from the same destination region was not changed from the operation.

OneEightHundred fucked around with this message at 16:54 on Sep 8, 2012

The_Franz
Aug 8, 2003

OneEightHundred posted:

Implementing it using chars, i.e.
code:
while(numBytes--) *destChar++ = *srcChar++;
...will not work because the compiler can't be sure that destChar and srcChar don't overlap, so it will emit the copies as single-byte read/writes to ensure the correct behavior if, for example, srcChar is only offset 2 bytes from destChar.

Would using the 'restrict' keyword on the pointer parameters achieve your desired behavior since the compiler will know that your memory regions don't overlap?

Citizen Erased
Mar 20, 2004

Wash Me Away
I'm looking to write a small program that crawls through a directory tree looking for files of a certain type and copies matching files to a new directory (and creates the directory if it doesn't already exist).

While I have an intermediate level of C++ experience, I've never had to do any real file manipulation and don't know where to start. Are there any good libraries for interrogating the contents of folders / getting file information / checking file extensions / copying files / creating new folders?

nielsm
Jun 1, 2009



Citizen Erased posted:

I'm looking to write a small program that crawls through a directory tree looking for files of a certain type and copies matching files to a new directory (and creates the directory if it doesn't already exist).

While I have an intermediate level of C++ experience, I've never had to do any real file manipulation and don't know where to start. Are there any good libraries for interrogating the contents of folders / getting file information / checking file extensions / copying files / creating new folders?

Do you intend to make it cross-platform, i.e. the same code compile and run on both Windows and Unix-like systems? If not, then by all means use the APIs for file manipulation given by your OS.

Citizen Erased
Mar 20, 2004

Wash Me Away
Windows only most likely - it'd be nice to have cross platform support but if it is significantly more work then it isn't necessary. What would be the best source for the Windows calls? MSDN?

Cat Plus Plus
Apr 8, 2011

:frogc00l:

Citizen Erased posted:

While I have an intermediate level of C++ experience, I've never had to do any real file manipulation and don't know where to start. Are there any good libraries for interrogating the contents of folders / getting file information / checking file extensions / copying files / creating new folders?

Boost.Filesystem.

Citizen Erased
Mar 20, 2004

Wash Me Away
Awesome, that looks about the same in terms of implementation as the native windows APIs so I'll give it a shot - thanks!

Papes
Apr 13, 2010

There's always something at the bottom of the bag.
Trying to optimize this factoring program and am getting absolutely nowhere, so I've come for help :).

What this program is supposed to do is find all prime factors of any number you input, and for extra credit it must work with 64 bit integers. I currently am using the following :

code:
long long minDivisor(long long n)
{
    if (n % 2 == 0)  // checks to see if n is even
     {
        cout << "2 ";
        return 2;
     }
   else
     {
        for (int factor = 3; factor <= n/2; factor += 2)
          {
             if(n % factor == 0)
              {
                 cout  <<  factor << " ";
                 return factor;

              }
           }
           cout << n << " ";
           return n;
       }
}

int main()
{
   long long x = 0;
   long long y;
   cout << "Enter an integer " << endl;
   cin >> x;
   while(x != 1)
     {
        y = minDivisor(x);
        x = x / y;
     }
}

While this works just fine, it is incredibly slow with actual 64 bit integers. Anyone have any ideas on how to make this faster? The only thing I can think of is to throw the first 1000 or so primes in an array and have those checked first, but I don't think that would make too much of a difference.

Papes fucked around with this message at 03:43 on Sep 10, 2012

Nippashish
Nov 2, 2005

Let me see you dance!

Papes posted:

Trying to optimize this factoring program and am getting absolutely nowhere, so I've come for help :).

Instead of checking all of the numbers from 2 to n/2 you only need to check the primes from 2 to floor(sqrt(n)).

Papes
Apr 13, 2010

There's always something at the bottom of the bag.

Nippashish posted:

Instead of checking all of the numbers from 2 to n/2 you only need to check the primes from 2 to floor(sqrt(n)).

Thanks so much dude, that cut runtime down by nearly half. Seriously awesome man, being a newbie can be so overwhelming at times. :yayclod:

Adbot
ADBOT LOVES YOU

shrughes
Oct 11, 2008

(call/cc call/cc)

Papes posted:

Thanks so much dude, that cut runtime down by nearly half. Seriously awesome man, being a newbie can be so overwhelming at times. :yayclod:

Half? Should cut it down a bit more than that.

Also you're outputting all the factors (or all the ones up to sqrt(n)) when you said you're only supposed to output the prime ones.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply