|
TOPS-420 posted:as a relocatable object format elf is great, but as an image format there are way too many bells and whistles that make dynamic linking complicated, insecure, and slow. loading pe and macho images is much simpler, though pe is dumb for not supporting PIC and brute force relocating images instead it's not a limitation of PE. before the linux subsystem, there was a thing called interix, which was a bsd-ish subsystem for windows. architecturally it went to heroic lengths to run on windows without dirty or undocumented tricks, and this included using pe/coff for all executables, including shared objects. softway systems (the original developer) first and microsoft next maintained for over a decade a bizarro gnu toolchain that could emit unix-style pes, complete with ld.so and pic. it's entirely possible that they achieved it through completely skipping the standard PE loader though, but I'm like 60% sure that you can do a got on win32 with a suitable compiler/linker hell, in fact, PE outright supports pic, on platforms where it's native and not a call $+5/pop ebx hack. I'm pretty sure there's a dedicated image directory for the global data pointer (only relevant on mips iirc. ia64 in typical ia64 fashion did something else and ad-hoc) it's all moot since ASLR though, because executables come out of the kernel loader pre-relocated (the relocations are actually applied by the memory manager, on page-in), so in user mode you'll never apply a relocation again elf is massively immensely easier on the kernel side compared to pe, and its 64 bit support is clean and not a hack. I really envy elf's two-level structure (program headers/sections) compared to pe's (sections/directories), too. plus elf uses the same format for object files, executables and core dumps, where windows uses two formats and a half (coff for object files, pe/coff for executables and the minidump format for dumps) because pe is so inflexible TOPS-420 posted:use objective-c++ and then you have c with classes with classes I've used objective-c++ and I loved it Deep Dish Fuckfest posted:statistically, there's some fucker out there using c++ but building their own vtable, possibly with macros, i just know it. i wish they were dead COM was designed to be "usable" from C in exactly that way and yes, the windows sdk has code generation tools and macros to do exactly that. then microsoft designed their c++ abi to be binary compatible with handwritten COM vtables Plorkyeran posted:the vague kernel of truth in "you shouldn't use a debugger for modern c++" is that a big part of "modern c++" is pushing more of your validation and checking to compile time, and a debugger is not a useful tool for diagnosing compile time errors. in theory, you should be spending a smaller proportion of your time in a debugger compared to older styles of c++ correct premise, wrong conclusion. what it really means - and i'm 100% dead serious on this - is that we need a compile-time debugger. especially if herb sutter gets his way (a man after my own heart ) eschaton posted:what this really means is that a debugger needs to be usable at compilation time so you can step through things like templates and examine their intermediate states VikingofRock posted:This would be super useful. And it would let compilers (optionally) cut down on the famously long template error messages, since they wouldn't need to provide the laundry list of "this instantiation didn't work for this reason" hints. which, you'll notice, is literally a stack backtrace Suspicious Dish posted:please tell me how i can compile-time check a network packet I did this. not literally compile-time check the packet, but encoding the result of the check in the type system. extremely useful, especially when you absolutely don't have the budget for tests or even CI and the code positively has to work right the first time hackbunny fucked around with this message at 05:29 on Jan 9, 2019 |
# ? Jan 9, 2019 05:25 |
|
|
# ? Jun 6, 2024 01:50 |
|
I'm skeptical of the cry from the Modern C++ people that I can move any of the meaningful inputs I deal with -- network packets, controller/keyboard input, artist-driven states and checkboxes -- to compile-time and find bugs that actually matter. Most of the bugs I deal with are not bugs in the core engine, they're in the gameplay logic, which is something that operates on a giant soup of objects in a world and "does stuff" to them.
|
# ? Jan 9, 2019 05:42 |
|
i don't have to work with videogames so i can have my cake and eat it too i use java and scala when i have a choice, so i can encode useful things into the type system, have errors caught by the compiler, and still also have a fully interactive repl this is the world i always dreamed of like lisp, except the type system is useful
|
# ? Jan 9, 2019 07:21 |
|
DONT THREAD ON ME posted:im chaotic gnutral Mahatma Goonsay posted:im lawful eval
|
# ? Jan 9, 2019 11:26 |
|
Suspicious Dish posted:I'm skeptical of the cry from the Modern C++ people that I can move any of the meaningful inputs I deal with -- network packets, controller/keyboard input, artist-driven states and checkboxes -- to compile-time and find bugs that actually matter. I guess your mileage may vary but on the game I work on we have a lot of gameplay systems made by many different people or even teams and a lot of bugs are caused by incorrect interactions between them. Our gameplay code is large and complex. There are many places where I wish we could have taken advantage of modern c++ to make more robust interfaces.
|
# ? Jan 9, 2019 14:38 |
|
hackbunny posted:it's not a limitation of PE. before the linux subsystem, there was a thing called interix, which was a bsd-ish subsystem for windows. architecturally it went to heroic lengths to run on windows without dirty or undocumented tricks, and this included using pe/coff for all executables, including shared objects. softway systems (the original developer) first and microsoft next maintained for over a decade a bizarro gnu toolchain that could emit unix-style pes, complete with ld.so and pic. it's entirely possible that they achieved it through completely skipping the standard PE loader though, but I'm like 60% sure that you can do a got on win32 with a suitable compiler/linker fair, I should’ve said “pe as used by windows on x64” quote:it's all moot since ASLR though, because executables come out of the kernel loader pre-relocated (the relocations are actually applied by the memory manager, on page-in), so in user mode you'll never apply a relocation again sliding on page-in is cool and good, and it sounds like apple is moving in that direction with dyld3 and its pre linked launch closures. it does still mean that you have to duplicate any pages with relocations on them if you load the same image with a different base address in multiple processes though quote:elf is massively immensely easier on the kernel side compared to pe, and its 64 bit support is clean and not a hack. I really envy elf's two-level structure (program headers/sections) compared to pe's (sections/directories), too. plus elf uses the same format for object files, executables and core dumps, where windows uses two formats and a half (coff for object files, pe/coff for executables and the minidump format for dumps) because pe is so inflexible otoh by using the same object format for everything, the format isn’t really optimized for anything. like rjmccall said you end up with ridiculously large relocation records in shared objects for instance, and while mapping elf images statically is easy enough for the kernel, the dynamic linker has a lot more work to do in user space. with pe, it seems more practical to do all the loading work in the kernel because it’s so limited
|
# ? Jan 9, 2019 16:23 |
|
Shaggar posted:y'all should just buy windows pcs and use vs + Xamarin. *smacks lips* mm, 10/10, vintage shaggar
|
# ? Jan 9, 2019 16:53 |
|
Suspicious Dish posted:I'm skeptical of the cry from the Modern C++ people that I can move any of the meaningful inputs I deal with -- network packets, controller/keyboard input, artist-driven states and checkboxes -- to compile-time and find bugs that actually matter. have you tried it, though? if you have invariants, literally any invariants at all, it can help to express them in a way the compiler can actually enforce Suspicious Dish posted:Most of the bugs I deal with are not bugs in the core engine, they're in the gameplay logic, which is something that operates on a giant soup of objects in a world and "does stuff" to them. I tried game development once and it's a really hard mindset for me to get into. The idea of letting something as "frivolous" as a collider change global state makes me scream internally. do you have any control over state and state changes? it seems a programming style specifically designed for glitches, out of which and with great effort developers manage to wring out some intended behavior. are there any patterns at all that you can fall back onto when you hit a snag, outside of entity-component-system? or is it just Unity that's like that? I guess I'd be one of those programmers with a single huge "system" object that implements all logic TOPS-420 posted:fair, I should’ve said “pe as used by windows on x64” what I meant is I'm pretty sure (less than 100% but more that 50%) it's a limitation of build tools and windows would load pie pes just fine TOPS-420 posted:sliding on page-in is cool and good, and it sounds like apple is moving in that direction with dyld3 and its pre linked launch closures. it does still mean that you have to duplicate any pages with relocations on them if you load the same image with a different base address in multiple processes though I might be wrong but iirc windows will still try to load a certain dll at the same address in all processes, so that relocated pages can still be shared. also I recently found out that x64 unlike x86 has pc-relative addressing (did I get that right?) which cuts down on relocated pages even more TOPS-420 posted:otoh by using the same object format for everything, the format isn’t really optimized for anything. like rjmccall said you end up with ridiculously large relocation records in shared objects for instance, and while mapping elf images statically is easy enough for the kernel, the dynamic linker has a lot more work to do in user space. I fully admit I noped the gently caress out at that part and just dealt with program headers TOPS-420 posted:with pe, it seems more practical to do all the loading work in the kernel because it’s so limited oh yeah some of the limitations of pe can actually be assets, like the fact that a pe image can't have holes, or that headers are automatically mapped as an implicit unnamed read-only section, instead of requiring that a program header explicitly maps them like on elf. and some of elf's strengths, like unlimited extensibility through named sections, and how sections can be non-loadable, so you don't have to arbitrarily append poo poo to the file but you have a proper descriptor for them, can be a liability for all the complexity they add
|
# ? Jan 9, 2019 21:29 |
|
hackbunny posted:also I recently found out that x64 unlike x86 has pc-relative addressing (did I get that right?) which cuts down on relocated pages even more x86-64 has pc-relative addressing but it doesn’t generally affect how many relocated pages you need because compilers avoid producing load-time relocations in text at all. if you’re building pic on x86 and the compiler needs a pic base it’ll materialize it by doing the call/pop sequence (which is special-cased by the processor) and addressing relatively from that; it doesn’t emit an absolute address and expect the loader to rebase it at least this is how it works in every compiler i know of
|
# ? Jan 9, 2019 21:59 |
|
hackbunny posted:have you tried it, though? if you have invariants, literally any invariants at all, it can help to express them in a way the compiler can actually enforce The most recent difficult bug I've had to fix that could plausibly be described as something that the compiler could fix would be a bug where we had a packet.SerializeInt(int Value, int MaxValue) function, and we got mixed up with whether MaxValue was inclusive or exclusive -- and some core engine code also got it wrong. And the runtime assert that checked Value also got it wrong. But we used it like this: packet.SerializeInt(player.PhysicsState, PhysState_MAX) We could probably have packet.SerializeEnum<T>(player.PhysicsState) and have the MAX implicitly interpreted, but player.PhysicsState is explicitly an uint8 (because "enum" in C/C++ will, by default on our platforms, be uint32 if it doesn't need 64-bit values and we want to save memory so we made it a uint8). hackbunny posted:I tried game development once and it's a really hard mindset for me to get into. The idea of letting something as "frivolous" as a collider change global state makes me scream internally. do you have any control over state and state changes? it seems a programming style specifically designed for glitches, out of which and with great effort developers manage to wring out some intended behavior. are there any patterns at all that you can fall back onto when you hit a snag, outside of entity-component-system? or is it just Unity that's like that? Part of game development is to trust others (artists, writers, designers, sound guys) to be able to set up scenes, do some basic scripting. Your goal as a core engineer is not to build a great game, it's to give others robust tools they can use to build the game.
|
# ? Jan 9, 2019 22:07 |
|
I think you're correct that the compiler isn't going to be able to catch the really tough game logic bugs. However, it eliminates a lot of mechanical bugs which frees you up to focus on the stuff that matters. Of course, if eliminating these bugs comes at the cost of indecipherable template errors and long compile times, it may not be worth it to you, especially if you're already really good at C/C++ and know how to write mechanical code that works.
|
# ? Jan 9, 2019 22:16 |
|
Suspicious Dish posted:We could probably have packet.SerializeEnum<T>(player.PhysicsState) and have the MAX implicitly interpreted, but player.PhysicsState is explicitly an uint8 (because "enum" in C/C++ will, by default on our platforms, be uint32 if it doesn't need 64-bit values and we want to save memory so we made it a uint8). you can specify the storage type for enums explicitely in modern c++: enum class butt : uint8_t { }; it also works for old style enums
|
# ? Jan 9, 2019 23:13 |
|
we adopted this wholesale into Objective-C and it has been a very good thing
|
# ? Jan 9, 2019 23:32 |
|
iirc it’ll be in the next c revision, it’s just really obvious goodness
|
# ? Jan 9, 2019 23:35 |
|
rjmccall posted:x86-64 has pc-relative addressing but it doesn’t generally affect how many relocated pages you need because compilers avoid producing load-time relocations in text at all. if you’re building pic on x86 and the compiler needs a pic base it’ll materialize it by doing the call/pop sequence (which is special-cased by the processor) and addressing relatively from that; it doesn’t emit an absolute address and expect the loader to rebase it compilers that target windows don't even try to do pic, though, that's what I was talking about. they do emit absolute addresses in text sections, and as a result, text sections do have relocations. ida pro even uses text relocations to disambiguate whether an operand is an immediate or an address, because they're so common storytime: openoffice.org is made of a handful of executables and a smorgasbord of dlls, and a long time ago their windows build naively built all dlls with the default preferred address. at the time, I was a regular on a newsgroup dedicated to debating the relative merits of operating systems (a sort of containment zone I guess), and I had a ton of free time, so when people tried to argue that microsoft word was faster than ooo writer because of anticompetitive microsoft secret sauce, it took me 5 minutes using dependency walker in profiling mode to figure out the truth (dependency walker's profiler helpfully highlights relocated dlls in bright red because relocation is expensive on windows). text relocations are a real, non-theoretical issue on windows I have no idea why no one ever tried pic on windows (except, as I mentioned, the gnu toolchain for opennt/interix). relocating a got should be even easier than relocating code, unless I'm missing some subtle but vital detail
|
# ? Jan 9, 2019 23:58 |
|
hackbunny posted:a newsgroup dedicated to debating the relative merits of operating systems comp.os.yours.piece-of-poo poo
|
# ? Jan 10, 2019 00:27 |
|
Zlodo posted:you can specify the storage type for enums explicitely in modern c++: what's the "class" bit about?
|
# ? Jan 10, 2019 00:46 |
|
Suspicious Dish posted:what's the "class" bit about? it makes it a scoped enum, which has very little in common with a class but c++ is allergic to new keywords
|
# ? Jan 10, 2019 00:49 |
|
Suspicious Dish posted:what's the "class" bit about? it's arbitrary
|
# ? Jan 10, 2019 01:14 |
|
DELETE CASCADE posted:comp.os.yours.piece-of-poo poo should be in the yospos.* hierarchy on the news server
|
# ? Jan 10, 2019 01:14 |
|
hackbunny posted:compilers that target windows don't even try to do pic, though, that's what I was talking about. they do emit absolute addresses in text sections, and as a result, text sections do have relocations. ida pro even uses text relocations to disambiguate whether an operand is an immediate or an address, because they're so common huh, thanks, i didn't realize that. that's terrible. does that mean that relocatable images in windows require text to be writable, or is it remapped read-only after the relocations are applied? hackbunny posted:I have no idea why no one ever tried pic on windows (except, as I mentioned, the gnu toolchain for opennt/interix). relocating a got should be even easier than relocating code, unless I'm missing some subtle but vital detail it's not so much that it's easier as that it's way way faster to load because the loader is touching (and dirtying) a contiguous array instead of random offsets in the instruction stream the mach-o linker does sort the got to separate the rebases from the binds so it can express them more compactly. you can easily see this with dyldinfo -rebase. as discussed, this is pointless with elf
|
# ? Jan 10, 2019 01:20 |
|
Suspicious Dish posted:what's the "class" bit about? it awkwardly bundles two unrelated behaviors: values of the enum type no longer implicitly convert to integer type (although you can still cast them), and enumerators are no longer injected into the enclosing scope (so you have to write myenum::foo, but otoh you don't have to name them things like me_foo to avoid conflicts). you still can't add methods to enums because bjarne is an idiot ...i know game development used to be held back in the dark ages because console sdks shipped with ancient, hacked-up toolchains, but i thought that got way better in the ps4 generation
|
# ? Jan 10, 2019 01:29 |
|
DELETE CASCADE posted:comp.os.yours.piece-of-poo poo a more polite it.comp.os.dibattiti ("comp.os.debates") actually but yes literally an unironic yospos, where I made my earliest windows internals effortposts and generally stanned for win32 (and fantasized about an improved replacement for windows scripting host that microsoft telepathically plagiarized from me to make powershell) rjmccall posted:huh, thanks, i didn't realize that. that's terrible. does that mean that relocatable images in windows require text to be writable, or is it remapped read-only after the relocations are applied? oh of course the original protection is restored after relocations are applied and imports resolved. come on, it's not that bad. writable text would be hilariously dangerous (it can happen if your sections are smaller than pages - you can set section alignment pretty low and I used to play with it a lot in my exe size fetish days. the kernel may then have to merge several sections in a single page. if it has to merge, say, .data with .text, you'll end up with a rwx page: writable code and executable data ). hell, a writable got is dangerous enough imo, what with all the function pointers, I really hope the thing is write-protected after relocation rjmccall posted:it's not so much that it's easier as that it's way way faster to load because the loader is touching (and dirtying) a contiguous array instead of random offsets in the instruction stream I meant in the sense of how many different kinds of relocations would be needed. the got only contains absolute addresses, right? I understand the principle but I have little practical experience with it hackbunny fucked around with this message at 02:08 on Jan 10, 2019 |
# ? Jan 10, 2019 02:06 |
|
yeah, the got always has absolute addresses. the whole point of relative addresses is that they’re constant after linking, so you’d never need a got for them swift makes heavy use of relative addresses in its metadata, partly because they’re smaller but mostly because they require zero eager overhead to load. this is also common in eh tables but there’s a lot more relative-address-to-got-entry stuff there because the target might not be local to the image
|
# ? Jan 10, 2019 02:25 |
|
rjmccall posted:...i know game development used to be held back in the dark ages because console sdks shipped with ancient, hacked-up toolchains, but i thought that got way better in the ps4 generation nah, this is just my own ignorance. c++ grew up much faster than i did. is there a way to get the max value of an enum at compile-time, then? the last time i tried to use C++ metaprogramming to solve an issue, i ended up 1. needing to use the C preprocessor anyway, which was a bit annoying, 2. it required some copy/paste foo and if you forgot to adjust it in both places you got a bug. I could have solved 2. using the C preprocessor as well but it hosed up intellisense so I decided not to do it.
|
# ? Jan 10, 2019 02:28 |
|
oh, and mach-o currently requires pages with rebases and binds to be mapped writable just for the loader’s benefit; the loader doesn’t remap them read-only after load. so it wasn’t totally crazy to me that that might be a restriction on windows, although yeah that would be hilariously bad for security. anyway that’s something we’re fixing
|
# ? Jan 10, 2019 02:28 |
|
Suspicious Dish posted:nah, this is just my own ignorance. c++ grew up much faster than i did. other than adding an enumerator explicitly for that purpose, no; that part of the model hasn’t changed at all
|
# ? Jan 10, 2019 02:30 |
|
Okay so that doesn't really help me, you still need to pass in a Max value, it's unclear whether this value is inclusive or exclusive, and it can't be checked at compile time whether you passed the right one. Modern C++: 0
|
# ? Jan 10, 2019 03:36 |
|
what exactly so you think an enum is for that it would be useful to dynamically extract a max value for it the point of a strongly-typed enum is that you assume the value is one of the enumerators. you can pretty easily validate that in the narrow set of places where you’re starting from an opaque int or whatever. it’s true that c++’s enum can be subverted somewhat easily but if you really want a harder guarantee you can pretty easily make a struct that does that
|
# ? Jan 10, 2019 06:29 |
|
My issue is that I'm serializing network packets. I want to serialize an enum (which I can guarantee is consecutive and starts at zero) using the lowest amount of bits. This is not a hypothetical problem, this is something I solved and ran into a bug with. Please help me use the C++ type system so that I don't make the same bug again.
|
# ? Jan 10, 2019 07:07 |
|
write a template that requires the enum to define a MaxValue enumerator. it’s not provided automatically in the language, but it should be obvious to maintainers how to keep it up-to-date when they’re editing the enum, certainly more so than needing to edit some separately-defined MyEnumBitCount constantC++ code:
if you need some separate glue code — like maybe you don’t want to switch to a scoped enum right now, so the enumerator has to be called MyEnumMax, so you need a separate macro or something to propagate that in a way the template will pick up on — that’s not a big deal, because the glue code is boilerplate that propagates correctness. the key is that the one thing you need to be explicitly written (and it’s too bad you need that, for sure) is trivially auditable, and everything else picks up on that rjmccall fucked around with this message at 07:53 on Jan 10, 2019 |
# ? Jan 10, 2019 07:38 |
|
Plorkyeran posted:it makes it a scoped enum, which has very little in common with a class but c++ is allergic to new keywords tbf adding new keywords is always risky, there's always the risk of picking something already widely used as an identifier in the wild unless you pick some super awkward keyword (like how they had to use co_yield for coroutines)
|
# ? Jan 10, 2019 08:08 |
|
Suspicious Dish posted:My issue is that I'm serializing network packets. I want to serialize an enum (which I can guarantee is consecutive and starts at zero) using the lowest amount of bits. This is not a hypothetical problem, this is something I solved and ran into a bug with. Please help me use the C++ type system so that I don't make the same bug again. the short answer is that it can't be done because of language limitations that still haven't been addressed the long answer is long. maybe tonight I'll give it a try
|
# ? Jan 10, 2019 10:29 |
|
Zlodo posted:tbf adding new keywords is always risky, there's always the risk of picking something already widely used as an identifier in the wild unless you pick some super awkward keyword (like how they had to use co_yield for coroutines) just continue extending static imo
|
# ? Jan 10, 2019 10:44 |
|
ynohtna posted:just continue extending static imo Scala's underscore says hi
|
# ? Jan 10, 2019 11:49 |
|
hackbunny posted:the short answer is that it can't be done because of language limitations that still haven't been addressed ah yes, more features to C++ will surely solve my problem
|
# ? Jan 10, 2019 21:08 |
|
rjmccall posted:write a template that requires the enum to define a MaxValue enumerator. it’s not provided automatically in the language, but it should be obvious to maintainers how to keep it up-to-date when they’re editing the enum, certainly more so than needing to edit some separately-defined MyEnumBitCount constant also just so I don't gloss over this with a sarcastic post above, I genuinely want to thank you for this post, since you did help me the right way to accomplish this. The only thing I'm worried about is the inclusive vs. exclusive issue, since that confusion can still happen when defining the enum.
|
# ? Jan 10, 2019 21:11 |
|
Suspicious Dish posted:also just so I don't gloss over this with a sarcastic post above, I genuinely want to thank you for this post, since you did help me the right way to accomplish this. The only thing I'm worried about is the inclusive vs. exclusive issue, since that confusion can still happen when defining the enum. the obvious* solution is to require both, and have a static assert in the template that verifies the exclusive one is one greater than the inclusive one * as opposed to good
|
# ? Jan 10, 2019 21:48 |
|
or just name it INCLUSIVE_MAX. that seems like the easiest part to solve unless I've totally misunderstood
|
# ? Jan 10, 2019 23:52 |
|
|
# ? Jun 6, 2024 01:50 |
|
Suspicious Dish posted:ah yes, more features to C++ will surely solve my problem I came here to work on an answer to your question and I'm greeted with this. zero regrets in telling you to gently caress off forever
|
# ? Jan 11, 2019 00:35 |