C/C++ Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›641 »

Xarn: Jun 26, 2015

Just casual 10k+ chars long symbols. This language is great :v:

# ? Nov 30, 2021 00:47

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 01:57

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

cheetah7071 posted:

What might cause a function to be so un-linked that the compiler doesn't even know its name? I'm getting this error, with only information about which project and which .lib file the error was in

e: I figured it out, complaining on the internet is always the fastest way to get something done. I forgot that pre-compiled .lib files shouldn't have templated functions

ee: wait no that wasn't it

eee:

I have code like this
code:
void Foo2() {
auto f = [&] {Foo();};
}
void Foo() {}
in my source file, with both Foo and Foo2 declared in a header. I thought it wouldn't matter what order I defined them in, but I guess the lambda function messed with it somehow? reversing them so that Foo was defined before Foo2, in the source file, resolved the issue

This either should work or shouldn�t compile, depending on what comes before, but MSVC is a very weird compiler.

# ? Nov 30, 2021 03:45

Zopotantor: Feb 24, 2013; ...und ist er drin dann lassen wir ihn niemals wieder raus...

Xarn posted:

Just casual 10k+ chars long symbols. This language is great

When I started using C, you had to make do with 8. You could still use longer names, but external symbols were silently(!) truncated. Also, most compilers would reduce the limit to 7 because they always inserted an underscore at the beginning for arcane reasons.

The limit was imposed by the object file format, not the language.

# ? Dec 1, 2021 02:10

cheetah7071: Oct 20, 2010; honk honk; College Slice

Is there a generally better way to read small amounts of information from a large number of files on the hard drive, than something like

code:

for (auto filename : filelist) {
	std::ifstream ifs{filename};
	ifs.seekg(positionWithHeaderInfoYouWant);
	ifs.read(...);
	//use that header info
	ifs.close();
}

This works but the speed is extremely variable. Like sometimes it'll handle ten thousand files per second and sometimes it takes multiple minutes to read 100 files, with the exact same code and the exact same set of files.

I'd think this is just how it always works, but when I use programs written by other people, they never seem to have this problem, which is what makes me think that maybe there's a trick I'm not aware of

# ? Dec 3, 2021 21:33

Xarn: Jun 26, 2015

Are you limited to standard C++?

# ? Dec 3, 2021 21:35

cheetah7071: Oct 20, 2010; honk honk; College Slice

Xarn posted:

Are you limited to standard C++?

I'm free to add a library dependency, if that's what you're asking

# ? Dec 3, 2021 21:35

OddObserver: Apr 3, 2009

Which OS you're on may matter a lot. Also if are you using spinning media or SSDs?

(Linux has bad file system performance. Windows and Mac are much worse).

# ? Dec 3, 2021 21:44

cheetah7071: Oct 20, 2010; honk honk; College Slice

This is Windows, on an SSD

to be honest the frustrating part is less the slowness and more the inconsistency. If it was just slow I could rewrite my code to not do a first-pass read of the headers. But when it randomly decides to be fast, getting the header info before doing a second-pass full read of the files is much, much faster. And again, when I run binaries written by other people that are doing very similar things under the hood (but I don't have access to the source code so I can't be completely sure), the header-reading step seems to be consistently very fast.

# ? Dec 3, 2021 21:47

cheetah7071: Oct 20, 2010; honk honk; College Slice

Never mind, I'm a big dummy. There's still some variability, but the insane thousand-fold slowdown I was getting today which caused me to write my post was because I was compiling in debug mode

If there's a better way than what I have posted I'd still love to hear it though, because I do get random 2- or 3-fold slowdowns even when I'm not being a dummy

# ? Dec 3, 2021 21:49

nielsm: Jun 1, 2009

Do the work in parallel and use the native Win32 functions for "overlapped IO". That's the Windows kernel term for asynchronous, event driven file access.
You'd first issue a large number of overlapped CreateFile calls, and for the completion of each of those trigger overlapped seek and read operations.

Theoretically, issuing more IO operations at once should allow the device drivers and storage controller combine and reorder requests into what's most efficient for the actual physical representation.

It's also a lot more work writing and ensuring the code is correct.

nielsm fucked around with this message at 05:07 on Dec 4, 2021

# ? Dec 3, 2021 23:42

Computer viking: May 30, 2011; Now with less breakage.

This seems somewhat relevant - and is kind of neat in general:

https://www.youtube.com/watch?v=qbKGw8MQ0i8

Their biggest simple upgrade doesn't really apply to you, though - a big difference between Windows and the others is that in Windows, closing a file only returns after syncing to disk, while in the other OSes it returns immediately and syncs in the background. Which means that if you were writing to a lot of files, it makes sense to have a lot of threads set aside just to wait for the close operations.

Computer viking fucked around with this message at 00:50 on Dec 4, 2021

# ? Dec 4, 2021 00:46

Xarn: Jun 26, 2015

nielsm posted:

Do the work in parallel and use the native Win32 functions for "overlapped IO". That's the Windows kernel term for asynchronous, event driven file access.
You'd first issue a large number of overlapped CreateFile calls, and for the completion of each of those trigger overlapped seek and read operations.

Theoretically, issuing more IO operations at once should allow the device drivers and storage controller combine and reorder requests into what's most efficient for the actual physical representation.

It's also a lot more work writing and ensuring the code is correct.

Beware, unless you are already familiar with MS's docs so you know how to read them, this is fuckton of work.

Depending on what you are gonna do with the files afterwards, you can try memory mapped IO and Boost to abstract over it if you plan to port the code to different platforms.

# ? Dec 5, 2021 18:12

Volguus: Mar 3, 2009

Xarn posted:

Beware, unless you are already familiar with MS's docs so you know how to read them, this is fuckton of work.

Depending on what you are gonna do with the files afterwards, you can try memory mapped IO and Boost to abstract over it if you plan to port the code to different platforms.

Wouldn't a library like libuv help in this situation? As in, provide the async IO without having to get dirty with the Win32 functions? And still have some hope that the IO code is at least somewhat correct?

# ? Dec 5, 2021 19:25

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

I am betting that online course or whatever calling for an observer-subject assignment didn't mention what to do about object lifetime management. I think that just gets overlooked in general with that pattern and they should go through some strategies with it. You can even screw yourself up with it in languages/runtimes that manage memory by keeping references to objects that you otherwise no longer cared about.

# ? Dec 14, 2021 19:40

Pennywise the Frown: May 10, 2010; Upset Trowel

Do you guys do C# in here? I'm just looking for some (hopefully free) online tutorials to help me continue my learning. I finished a semester in school and we have this stupid loving "book" where we have access to it virtually for 4 months (full price) and then we lose access at the end of the semester so I don't have that resource anymore. :patriot:

I'm going to check out W3 Schools I think first. Any other in depth guides?

# ? Dec 18, 2021 14:15

nielsm: Jun 1, 2009

Try the .NET thread for C# talk.

# ? Dec 18, 2021 14:32

Pennywise the Frown: May 10, 2010; Upset Trowel

nielsm posted:

Try the .NET thread for C# talk.

Thanks!

# ? Dec 18, 2021 17:26

Nalin: Sep 29, 2007; Hair Elf

I've been smacking my head against C++20 modules for a while now and I just haven't been able to get my code to compile. Is anybody able to explain to me why this tiny solution doesn't compile in Visual Studio 2022? If I can find out why it is failing, I would be one step closer to getting my project to compile.

I get the following error:
C7621: module partition 'Part' for module unit 'File' was not found.

The project:
https://drive.google.com/file/d/1-9Rotd3c2qSmr1esoYs-1YTQmCYL1oMk/view?usp=sharing

# ? Jan 3, 2022 11:00

Nalin: Sep 29, 2007; Hair Elf

Well, I think I've figured it out. Visual Studio 2022 17.0.4 cannot have a module implementation unit of a partition. I'm not sure if this is a limitation of modules or a bug with Visual Studio, but I can work around it by either not using partitions or only using partitions without an implementation unit.

# ? Jan 4, 2022 18:16

Twerk from Home: Jan 17, 2009; This avatar brought to you by the 'save our dead gay forums' foundation.

I'm trying to get an application that's using the simde library for portable SIMD to run happily on Apple Silicon. So far, the biggest tripping point is that ARM NEON bit-shifting operations can only be called with a compile-time constant offset. For example, I'm calling _mm_slli_epi64, which simde is mapping onto these NEON instructions like vshlq_n_s16. I can't find an elegant way to make sure that these always are called with compile-time constants, as we had wrapper functions around them and as I understand it, there's no way to guarantee that a function is called only with compile-time constants, so I changed the functions into macros.

The other problem that I had is a small number of places where these were called with runtime variables, which I could build a lookup table like this project did: https://github.com/VectorCamp/vectorscan/pull/81/files#diff-1f738fe3ab986614e926b3ce01fcd42dbf3e8ccb79337931af69450f56319554R178

Doing a lookup table like that feels incredibly repetitive and inelegant, is there some way with templates to create something like this over a range of numbers that I define?

code:

VecW vecw_slli_lookup(VecW vv, uint8_t ct) {
  switch(ct) {
    default: return _mm_slli_epi64(vv, 0);
    case 1: return _mm_slli_epi64(vv, 1);
    case 2: return _mm_slli_epi64(vv, 2);
    case 3: return _mm_slli_epi64(vv, 3);
    case 4: return _mm_slli_epi64(vv, 4);
    case 5: return _mm_slli_epi64(vv, 5);
    case 6: return _mm_slli_epi64(vv, 6);
    case 7: return _mm_slli_epi64(vv, 7);
  }
}

# ? Jan 14, 2022 23:53

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

NEON has "shift by a runtime value" operations (e.g. vshlq_s16 instead of vshlq_n_s16), it seems strange that your portability library is using the compile-time-constant-only variant of the intrinsic to emulate an SSE instruction that isn't restricted to compile-time constants.

# ? Jan 15, 2022 03:49

Nalin: Sep 29, 2007; Hair Elf

C++20 introduces consteval, which can force a function to be evaluated at compile-time.

code:

template <uint8_t N>
consteval auto vecw_slli_lookup(VecW vv)
{
    static_assert(N < 8, "vecw_slli_lookup may only be used with N < 8.");
    return _mm_slli_epi64(vv, N);
}

gcc supports it well, but clang does not (partial support in clang 14). And iirc Apple clang is behind normal clang.

EDIT: Some searching shows this C++17 based solution:

code:

template <auto V>
struct constant {
    constexpr static decltype(V) value = V;
};

template <uint8_t N>
constexpr auto vecw_slli_lookup(VecW vv)
{
    static_assert(N < 8, "vecw_slli_lookup may only be used with N < 8.");
    return _mm_slli_epi64(vv, N);
}

int main()
{
    auto result = constant<vecw_slli_lookup<5>(0)>::value;
    return 0;
}

Nalin fucked around with this message at 07:04 on Jan 15, 2022

# ? Jan 15, 2022 06:51

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Twerk from Home posted:

The other problem that I had is a small number of places where these were called with runtime variables, which I could build a lookup table like this project did: https://github.com/VectorCamp/vectorscan/pull/81/files#diff-1f738fe3ab986614e926b3ce01fcd42dbf3e8ccb79337931af69450f56319554R178

Doing a lookup table like that feels incredibly repetitive and inelegant, is there some way with templates to create something like this over a range of numbers that I define?
code:
VecW vecw_slli_lookup(VecW vv, uint8_t ct) {
  switch(ct) {
    default: return _mm_slli_epi64(vv, 0);
    case 1: return _mm_slli_epi64(vv, 1);
    case 2: return _mm_slli_epi64(vv, 2);
    case 3: return _mm_slli_epi64(vv, 3);
    case 4: return _mm_slli_epi64(vv, 4);
    case 5: return _mm_slli_epi64(vv, 5);
    case 6: return _mm_slli_epi64(vv, 6);
    case 7: return _mm_slli_epi64(vv, 7);
  }
}

code:

template<uint8_t N, typename F>
auto to_constexpr(uint8_t n, F&& func) {
    if (N == n) {
        return func(std::integral_constant<uint8_t, N>());
    }
    if constexpr (N == 0) {
        abort();
    }
    else {
        return to_constexpr<N - 1, F>(n, std::forward<F>(func));
    }
}

VecW vecw_slli_lookup(VecW vv, uint8_t ct) {
  return to_constexpr<7>(ct, [vv](auto ct) {
      return _mm_slli_epi64(vv, ct());
  });
}

Probably compiles to the same thing as the switch (with the addition of a default case that calls abort()) with optimizations enabled but I didn't bother to verify.

# ? Jan 15, 2022 07:18

Beef: Jul 26, 2004

Yep. All compilers will constant fold the switch variant. At least as long as the number is known at compile time and not the result of some atoi() or something.

# ? Jan 15, 2022 09:45

Twerk from Home: Jan 17, 2009; This avatar brought to you by the 'save our dead gay forums' foundation.

Jabor posted:

NEON has "shift by a runtime value" operations (e.g. vshlq_s16 instead of vshlq_n_s16), it seems strange that your portability library is using the compile-time-constant-only variant of the intrinsic to emulate an SSE instruction that isn't restricted to compile-time constants.

I looked at vshlq_s16, it looks like it takes a vector instead of an integer as the argument to shift by, so if I wanted to bit-shift a whole vector by 4, I'd need to create a second vector of equal length and fill it with 4s.

I appreciate all of the suggestions that people gave here! The project is using C++11, and I think that the lookup table will be easier to live with, given that it's only needed in one place. The value to shift by is actually determined at runtime, so maybe I should inline the function to keep overhead reasonably small.

# ? Jan 15, 2022 21:16

baby puzzle: Jun 3, 2011; I'll Sequence your Storm.

I'm trying to figure out how to write a const getter for this class. It is an object that gives you access to an object while it locks a mutex for the duration of the lifetime of a ProtectedObjectAccess object. I'm not sure if I need to include that code.

code:

template < typename T >
class ProtectedObject
{
	std::recursive_mutex m_Mutex;
	T m_Object;
public:

	ProtectedObjectAccess< T > Get()
	{
		return ProtectedObjectAccess( m_Object, m_Mutex );
	}
};

I'm trying to use it this way:

code:

bool MyClass::ConstFunction() const
{
	const auto access = m_SelectedObjects.Get();
}

I get an error because Get() isn't const, so I can't call it. I'm having a hard time figuring out how to write a const version of the Get() function.

# ? Jan 19, 2022 03:59

roomforthetuna: Mar 22, 2005; I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

baby puzzle posted:

I get an error because Get() isn't const, so I can't call it. I'm having a hard time figuring out how to write a const version of the Get() function.

I could be wrong, but I think what you need here is for the mutex to have the 'mutable' keyword (which means you can gently caress with it even when its container is const). Then your Get function can be const and still do its job (so long as ProtectedObjectAccess's constructor has an instance where the first parameter is a const T).

This whole thing looks like a horror though.

# ? Jan 19, 2022 04:43

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

A locking wrapper like that isn�t the craziest thing � Rust does something like it � but you should really take the locked operation as a lambda argument or something instead of trying to return some sort of guarded value. And yes, you will need to make the mutex mutable. Also, it probably shouldn�t be a recursive mutex, because any time you find yourself wanting a recursive mutex is a time when you need to back up and completely rethink what you�re doing.

# ? Jan 19, 2022 07:27

baby puzzle: Jun 3, 2011; I'll Sequence your Storm.

I think I'm just gonna const_cast when I need to..

e: i'm drunk and being gross. nvm.

baby puzzle fucked around with this message at 09:48 on Jan 19, 2022

# ? Jan 19, 2022 09:41

Xarn: Jun 26, 2015

rjmccall posted:

but you should really take the locked operation as a lambda argument or something instead of trying to return some sort of guarded value.

This is the correct solution.

# ? Jan 19, 2022 13:06

baby puzzle: Jun 3, 2011; I'll Sequence your Storm.

Probably. I'm always using in cases like this, at the top of a scope. Just to make it easy for me to see if I'm using it improperly. But of course I can still easily do the wrong thing with it.

code:

			{
				auto selectedObjects = m_SelectedObjects.Get();
				for ( auto& i : selectedObjects.get() )
				{
					auto obj = i.lock();
					if ( obj )
					{
						newGroup->AddObject( obj );
					}
				}
			}

So I guess a lambda would make more sense? And would look basically the same. But I'm not sure exactly how to do that. Well, if I really want to do it, I'll figure it out.

# ? Jan 21, 2022 07:22

Xarn: Jun 26, 2015

The idea is that you write something like this (add forwarding constructors and proper return types and so on as needed)

C++ code:

template <typename T>
class locking_wrapper {
    std::mutex m_mutex;
    T m_t;
public:

    template <typename Func>
    void do_locked(Func&& f) {
        std::unique_lock _(m_mutex);
        f(m_t);
    }    
};

And then the usage looks like this

C++ code:

locked_t.do_locked([](int i) {
    // everything here happens with the mutex on locked_t engaged
});

So that the users of the synchronized value don't have to fiddle around with mutexes and lock lifetimes.

# ? Jan 22, 2022 13:46

roomforthetuna: Mar 22, 2005; I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

Xarn posted:

So that the users of the synchronized value don't have to fiddle around with mutexes and lock lifetimes.

Though if you're using clang, absl::Mutex and friends with annotations (ABSL_GUARDED_BY etc.) are amazing and will prevent most footguns, in conjunction with -Wthread-safety. (The annotations are also compatible with not-clang but will no longer actively prevent footguns.)

Edit: you could also combine the annotations with this template idea, which would prevent you from doing a lock-mutex from within another lock on the same mutex, for example.

# ? Jan 22, 2022 17:31

Absurd Alhazred: Mar 27, 2010; by Athanatos

Xarn posted:

The idea is that you write something like this (add forwarding constructors and proper return types and so on as needed)
C++ code:
template <typename T>
class locking_wrapper {
    std::mutex m_mutex;
    T m_t;
public:

    template <typename Func>
    void do_locked(Func&& f) {
        std::unique_lock _(m_mutex);
        f(m_t);
    }    
};
And then the usage looks like this
C++ code:
locked_t.do_locked([](int i) {
    // everything here happens with the mutex on locked_t engaged
});
So that the users of the synchronized value don't have to fiddle around with mutexes and lock lifetimes.

Doesn't this get a bit unwieldy if you have more than one locked value? Although I guess you could call .do_locked() on other objects inside of lambdas.

# ? Jan 22, 2022 19:39

Xarn: Jun 26, 2015

Yes if they are independent (so they should logically get their own mutex) and yes.

# ? Jan 22, 2022 20:22

Foxfire_: Nov 8, 2010

If you're locking two mutexes at the same time, you are well on your way to deadlocktown and should do a bunch more thinking

# ? Jan 22, 2022 22:40

roomforthetuna: Mar 22, 2005; I don't need to know anything about virii! My CUSTOM PROGRAM keeps me protected! It's not like they'll try to come in through the Internet or something!

Foxfire_ posted:

If you're locking two mutexes at the same time, you are well on your way to deadlocktown and should do a bunch more thinking

And if it turns out you *must* do that for some reason, this is a really really great time to set up those thread safety annotations! ABSL_ACQUIRED_BEFORE and/or ABSL_ACQUIRED_AFTER is a good way to be confident you're not causing two-lock deadlocks.

# ? Jan 22, 2022 23:56

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

If they�re not independent, you can of course just make the locked value a struct.

As a general rule, some people struggle with just making new types when they�d be useful, as if writing the struct keyword was some enormously fraught undertaking. If you are one of them, that is a good instinct to drive out.

# ? Jan 23, 2022 00:15

Xarn: Jun 26, 2015

roomforthetuna posted:

And if it turns out you *must* do that for some reason, this is a really really great time to set up those thread safety annotations! ABSL_ACQUIRED_BEFORE and/or ABSL_ACQUIRED_AFTER is a good way to be confident you're not causing two-lock deadlocks.

Orrrr you can use std::lock, which afaik guarantees no deadlocks when acquiring multiple mutexes.

rjmccall posted:

If they�re not independent, you can of course just make the locked value a struct.

As a general rule, some people struggle with just making new types when they�d be useful, as if writing the struct keyword was some enormously fraught undertaking. If you are one of them, that is a good instinct to drive out.

I sometimes have this problem and my first idea was fo frankenstein together a variadic version of the wrapper :v:

# ? Jan 23, 2022 00:43

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 01:57

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

Xarn posted:

Orrrr you can use std::lock, which afaik guarantees no deadlocks when acquiring multiple mutexes.

This doesn't help if your usage pattern involves doing work while holding lock 1, then acquiring lock 2 without releasing lock 1, then doing more work.

# ? Jan 23, 2022 01:01

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > C/C++ Programming Questions Not Worth Their Own Thread

«‹›641 »