Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
lord funk
Feb 16, 2004

Didn't even throw my name in this year. :( All travel funds have been zeroed in my job.

Adbot
ADBOT LOVES YOU

Doh004
Apr 22, 2007

Mmmmm Donuts...

dazjw posted:

Anyone else going to WWDC? I'm going to be a first-timer.

Same here! No luck my first 4 years but this one was the charm :woop:

lord funk
Feb 16, 2004

Doh004 posted:

Same here! No luck my first 4 years but this one was the charm :woop:

Nice!

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av
Does @synchronize imply memory barriers before and after the block, and if not WHY. I'm seeing weird "impossible" behavior in multithreaded code that looks a lot like some stores made by a thread are not visible to a different thread. Why wouldn't @synchronize imply memory barriers? what conceivable use would it have, if it doesn't actually synchronize?

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

hackbunny posted:

Does @synchronize imply memory barriers before and after the block, and if not WHY. I'm seeing weird "impossible" behavior in multithreaded code that looks a lot like some stores made by a thread are not visible to a different thread. Why wouldn't @synchronize imply memory barriers? what conceivable use would it have, if it doesn't actually synchronize?

@synchronize acquires and releases a (recursive) lock, with all the normal memory-ordering semantics of locks. It's actually a pthread mutex, if you care.

Atomic properties don't imply any memory ordering, if it helps. They're essentially memory_order_relaxed.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

rjmccall posted:

@synchronize acquires and releases a (recursive) lock, with all the normal memory-ordering semantics of locks. It's actually a pthread mutex, if you care.

I've already had to write code like this at least once:

Objective-C code:
@synchronized(self) {
	OSMemoryBarrier();

	// ...

	OSMemoryBarrier();
}
because values I was setting in the @synchronized body (ivars, not properties, to be clear) weren't visible in other threads. Again to be absolutely clear, I'm talking about ivars that are only accessed in -init or within @synchronized(self), I'm not using double-checked locking. What gives? What is the point of @synchronized if it doesn't... synchronize? I'd be more understanding if we were talking about raw pthread mutexes (which may or may not be memory barriers depending on the implementation) or something equally low level, but this is a language construct, having to use a low level OS function to make it work seems a little backwards

When I stuck those calls to OSMemoryBarrier I was desperate to fix a bug (and it worked), what I want to know is: should I always do this when I use @synchronized, just to be safe?

e: efficiency is not a goal, correctness is. I'll bracket everything between two full barriers, no problem

hackbunny fucked around with this message at 19:41 on Apr 29, 2016

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
I mean, since you clearly don't believe me that @synchronized orders memory, you are welcome to go read the source code yourself. The implementation of enter/exit is here, and the implementation of recursive_mutex_lock is here. And yes, pthread_mutex_lock on Apple platforms does an acquire and pthread_mutex_unlock does a release.

If adding random unnecessary barriers looks like it's fixing your problem, then be my guest, but as far as I can tell your assumption is incorrect.

Doc Block
Apr 15, 2003
Fun Shoe
I'm writing a bitmap font generator to get some custom drawing effects. Most of it is pretty straightforward with Core Text and Core Graphics, like mapping characters to glyphs, figuring out where they should go on a texture, drawing them, etc.

But I'm wondering if anyone knows of a simpler way to get the kerning data than what I'm currently doing. The game engine supports kerning for bitmap fonts in the form of (leftChar, rightChar, offset) tuples, and so right now I'm:
  1. Mapping the characterss to their respective glyphs
  2. ...doing everything else
  3. Fetching the kern table
  4. Parsing the kern table, which is a pain since there are multiple versions, and various subtables for handling all kinds of esoteric stuff like vertical kerning that must be dealt with/ignored
  5. Trying to reverse the mapping of characters to glyphs, since the kern table only contains glyphs
Is there an easier way?

#5 is kind of a problem, since sometimes a single character will be made up of multiple glyphs, and could even contain kerning data for glyphs that don't map to characters at all.

Doc Block fucked around with this message at 22:31 on Apr 29, 2016

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

rjmccall posted:

I mean, since you clearly don't believe me that @synchronized orders memory, you are welcome to go read the source code yourself.

I believe you, have to because the alternative is too horrible to consider, but. Let's go back to my example:

Objective-C code:
@synchronized(self) {
	OSMemoryBarrier();

	// ...

	OSMemoryBarrier();
}
In the middle of the block, the application applies a simple obfuscation to a stream of bytes: I have a fixed size key, and I XOR it byte by byte with the stream. When I run out of key bytes, I roll back to the beginning of the key - a Vigenere cipher basically. The obfuscated stream is saved to a file, and multiple threads can and do write to it. Until I added the barriers, I was guaranteed to get a corrupted file, because at some point the code would start using the wrong offset into the key, and I even had to add a heuristic mode to our decoding tool to recover data from corrupted files because it had started happening in the wild. I had tried making the affected ivars volatile, but it didn't work, so it wasn't the compiler that was caching values but the CPU

I want you to be right, and I want to have overlooked something in diagnosing the bug I'm currently dealing with, but it's happened before that I had broken code, and only barriers fixed it. I'm now in a similar situation (ironically, again with a circular array), where I mutate state inside a @synchronized block in one thread, but another thread still sees the old values. I should probably just do away with locks and use dispatch_async(dispatch_get_main_queue(), ..., but that may not an option everywhere I use @synchronized

rjmccall posted:

The implementation of enter/exit is here, and the implementation of recursive_mutex_lock is here. And yes, pthread_mutex_lock on Apple platforms does an acquire and pthread_mutex_unlock does a release.

Oh nice, I didn't know the source was public

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
This is the sort of thing where the details really matter; I would need to see real code. If you're not comforting posting that here, you can send it to me specifically, or just file a radar and ping me with the number.

Carthag Tuek
Oct 15, 2005

Tider skal komme,
tider skal henrulle,
slægt skal følge slægters gang



FWIW, in my experience it's been a lot of grief to have too much @synchronize in my code. That includes in methods that are called from multiple places. I know you know your poo poo hackbunny, I'm just saying limit the @sync blocks to their absolute necessity. At best, you'll get a ton of code that just queues up waiting for the locks to get unlocked. @sync is a last-effort solution imo.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

rjmccall posted:

This is the sort of thing where the details really matter; I would need to see real code. If you're not comforting posting that here, you can send it to me specifically, or just file a radar and ping me with the number.

Monday, when I'm back at work. Naturally, when I'll try to reproduce the issue, it will have mysteriously disappeared

Snapchat A Titty posted:

FWIW, in my experience it's been a lot of grief to have too much @synchronize in my code. That includes in methods that are called from multiple places. I know you know your poo poo hackbunny, I'm just saying limit the @sync blocks to their absolute necessity. At best, you'll get a ton of code that just queues up waiting for the locks to get unlocked. @sync is a last-effort solution imo.

In this particular case, it's a retrofit on a hot mess of legacy code that I should have already rewritten. I don't know anyway, I find synchronized code much more readable and debuggable than lock-free code. Doesn't mean I don't prefer lock-free, and blocks and GCD queues help ease the pain a lot. It's not like I have a lot of contention anyway, I never have more than two, three active threads at the same time, tops

Simulated
Sep 28, 2001
Lowtax giveth, and Lowtax taketh away.
College Slice

hackbunny posted:

When I stuck those calls to OSMemoryBarrier I was desperate to fix a bug (and it worked), what I want to know is: should I always do this when I use @synchronized, just to be safe?

e: efficiency is not a goal, correctness is. I'll bracket everything between two full barriers, no problem

In a "correct" program inserting OSMemoryBarrier() calls inside an @synchronized block should be a complete waste of time and not affect the execution of the program whatsoever. POSIX spec 4.11 says pthread_mutex_lock is required to guarantee memory ordering and platforms must insert whatever read and/or write barriers required to make that happen.

Unfortunately you have data races or some other unsafe stuff going on. Your OSMemoryBarrier() fix is almost certainly not a 100% fix and it would be one new CPU design or OS update away from exploding.


hackbunny posted:

I want you to be right, and I want to have overlooked something in diagnosing the bug I'm currently dealing with, but it's happened before that I had broken code, and only barriers fixed it. I'm now in a similar situation (ironically, again with a circular array), where I mutate state inside a @synchronized block in one thread, but another thread still sees the old values. I should probably just do away with locks and use dispatch_async(dispatch_get_main_queue(), ..., but that may not an option everywhere I use @synchronized

Are both threads accessing the data behind @synchronized? Are you accidentally storing a pointer outside the @synchronized block?

What I usually do in this kind of situation is put the data structure inside the implementation of a class (or in Swift use a generic `SpinLockedValue` or `QueueProtectedValue` struct that forces all accesses through closures I control) and use a serial dispatch queue and dispatch_sync. Then no reference to the data structure can leak and all users of it are forced to go through the single well-reviewed and well-tested interface that I know is correct.

Obviously dealing with a legacy codebase is another story but as a general rule I try to make all my internal APIs as close to impossible to gently caress up as I can. Today me is much more clever than six-months-from-now me.

I also want to say I feel you pain. I recently redid our login/logout flow and holy crap... the race conditions.... so many race conditions. I think I hit at least 15 different asserts from various invariants being violated because some random thread was firing an NSNotification, which another object observed and changed its properties, which triggered KVO observers on multiple threads, which triggered their own KVO, which fired another NSNotification. And for force-logout due to session expiration a view controller started a race to tear down as it threw up an alert AND setup a dispatch_after that waited 2 seconds before calling the same completion block the alert OK button would trigger. The completion block tried to pop navigation controllers in a loop, sleep for a bit, then check if the root view controller was the one it wanted. Did I mention some horribly written view controllers that generate network requests as a side-effect of being dismissed, while the network subsystem is racing to shut itself down?

Thankfully our refactor project is to the point where we are starting to take a chainsaw to some of the old code. Nice clean Swift code with real test coverage. I'll be happy if I never see KVO again.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

Ender.uNF posted:

Unfortunately you have data races or some other unsafe stuff going on. Your OSMemoryBarrier() fix is almost certainly not a 100% fix and it would be one new CPU design or OS update away from exploding.

Eh, tomorrow you'll get to see the code and judge for yourself

Ender.uNF posted:

Are both threads accessing the data behind @synchronized? Are you accidentally storing a pointer outside the @synchronized block?

It will probably be something totally obvious like this I have overlooked

Ender.uNF posted:

Obviously dealing with a legacy codebase is another story but as a general rule I try to make all my internal APIs as close to impossible to gently caress up as I can.

It's not easy in Objective C. I've written nearly bug-free code in C++ through the power of strong typing, but Objective C lets too much poo poo slip through without a warning. Forget to return a value, or return a value from a void method, the compiler won't say poo poo. id, int and BOOL silently converting to each other, I hate Objective C sometimes

Ender.uNF posted:

I also want to say I feel you pain. I recently redid our login/logout flow and holy crap... the race conditions.... so many race conditions.

Doubt you can beat my codebase with its dozens of -[NSObject performSelectorInBackground:withObject:] (which I have removed. Almost all of them). I've had to write a wrapper around dispatch_sync(dispatch_get_main_queue(), ^{}) for all the places where I used that to "fix" race conditions (fingers crossed, I probably didn't create any deadlocks)

Ender.uNF posted:

I think I hit at least 15 different asserts from various invariants being violated because some random thread was firing an NSNotification, which another object observed and changed its properties, which triggered KVO observers on multiple threads, which triggered their own KVO, which fired another NSNotification. And for force-logout due to session expiration a view controller started a race to tear down as it threw up an alert AND setup a dispatch_after that waited 2 seconds before calling the same completion block the alert OK button would trigger. The completion block tried to pop navigation controllers in a loop, sleep for a bit, then check if the root view controller was the one it wanted. Did I mention some horribly written view controllers that generate network requests as a side-effect of being dismissed, while the network subsystem is racing to shut itself down?

I easily have you beat :newlol: How about instead of dispatch_after it was -[NSObject performSelector:withObject:afterDelay:] on arbitrary threads that may be about to die? you know, the reason dispatch_get_current_queue is deprecated. And you can't even replace them with dispatch_after because they are sometimes cancelled with +[NSObject cancelPreviousPerformRequestsWithTarget:selector:object:]

Doc Block
Apr 15, 2003
Fun Shoe
How do you not at least get a warning when writing a function in Objective-C that's supposed to return a value but doesn't?

Because as soon as I start implementing a function that returns a value, Xcode starts complaining that the implementation doesn't return a value when it should, right up until I type the return statement.

Doctor w-rw-rw-
Jun 24, 2008

Doc Block posted:

How do you not at least get a warning when writing a function in Objective-C that's supposed to return a value but doesn't?

Because as soon as I start implementing a function that returns a value, Xcode starts complaining that the implementation doesn't return a value when it should, right up until I type the return statement.

Just return nil or return 0 until you fill it in. If your methods are so huge that the delta between that and implementing it is > 15 minutes, you're writing a foolishly huge method.

Froist
Jun 6, 2004

Doctor w-rw-rw- posted:

Just return nil or return 0 until you fill it in. If your methods are so huge that the delta between that and implementing it is > 15 minutes, you're writing a foolishly huge method.

I think he was replying to hackbunny's suggestion that Xcode gives no warning in that situation, rather than complaining about the behaviour.

Doctor w-rw-rw-
Jun 24, 2008
Oh. Oops. I ignored that post since he claims to be good at C++ but doesn't set flags to make clang more strict with Objective-C. :confused:

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

Doctor w-rw-rw- posted:

Oh. Oops. I ignored that post since he claims to be good at C++ but doesn't set flags to make clang more strict with Objective-C. :confused:

When did I piss in your cereals? We have never even interacted :confused: And I'm pretty sure I'm just using the default Xcode settings, which for Objective C are for some reason even looser than plain C?

lord funk
Feb 16, 2004

Can someone enlighten me on why this compiles with only a precision loss warning? Why don't I have to declare a type? (ObjC)

code:
 __block filesToUpdateCount = files.count;

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av
Speaking of loose warning defaults:

lord funk posted:

Can someone enlighten me on why this compiles with only a precision loss warning? Why don't I have to declare a type? (ObjC)

code:
 __block filesToUpdateCount = files.count;

Variables in C default to int. It should warn you but :shrug:

Doctor w-rw-rw-
Jun 24, 2008
Are you compiling with a C standard of less than C99? That looks like implicit int declaration.

(Where iOS is concerned you should never compile with older than C99 or C++11)

lord funk
Feb 16, 2004

hackbunny posted:

Variables in C default to int. It should warn you but :shrug:

Thanks - good to know.

Doctor w-rw-rw- posted:

Are you compiling with a C standard of less than C99? That looks like implicit int declaration.

(Where iOS is concerned you should never compile with older than C99 or C++11)

I've never explicitly set this, and it was sitting on GNU99.

edit: another look and there was a warning about implicit typing to int

lord funk fucked around with this message at 22:51 on May 1, 2016

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

Doctor w-rw-rw- posted:

Are you compiling with a C standard of less than C99? That looks like implicit int declaration.

(Where iOS is concerned you should never compile with older than C99 or C++11)

Me, I compile in C99 mode (I have even used a VLA once 😱), but id still loves implicitly converting to and from int causing me a lot of headaches. I'll have to tweak the warning settings some time

e: white-knighting clang warning settings and Xcode defaults? you really should know better

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Doctor w-rw-rw- posted:

(Where iOS is concerned you should never compile with older than C99 or C++11 C11 or C++14)

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
The compiler will generally not make something an error if it traditionally hasn't been, and we really can't if the standard says it's officially well-formed, which is why most people strongly recommend always using aggressive warning settings and -Werror when coding in C.

We did at least fix the id-to-int thing in ARC, and if you're still not using ARC, you really only have yourself to blame.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

rjmccall posted:

We did at least fix the id-to-int thing in ARC, and if you're still not using ARC, you really only have yourself to blame.

Yourself or the five year old outsourced code that resisted automated conversion and defies human understanding

I don't need to be taught the basics of programming, please. I'm a one person team and I get very little downtime between new features to do "useless" work like fixing warnings, converting to ARC or experimenting with project settings. Or, god forbid, writing tests. I don't need the attitude, thanks

I'm not even complaining, programming tools are what they are and I'm used to working with flawed tools. Probably too used in fact

hackbunny fucked around with this message at 23:22 on May 1, 2016

Simulated
Sep 28, 2001
Lowtax giveth, and Lowtax taketh away.
College Slice

rjmccall posted:

The compiler will generally not make something an error if it traditionally hasn't been, and we really can't if the standard says it's officially well-formed, which is why most people strongly recommend always using aggressive warning settings and -Werror when coding in C.

We did at least fix the id-to-int thing in ARC, and if you're still not using ARC, you really only have yourself to blame.

Sometimes you find that a previous "engineer" stored object pointers on a struct, then sent that struct on a ride through some desolate broken hellscape of macros and header files and void pointers. You can't prove the code is conjuring demons but you're pretty sure. You know the correct path forward is to burn it all to the ground but you hear the voice of a small child calling from inside. It's definitely a trap but you can't bring yourself to light the match. It's just a few lines of code for one feature. You didn't create the abomination, you're just trying to contain it.

Some code bases are damned. They belong to one time and place, exploiting undefined behavior and swizzling the very marrow from your bones. ARC? Naught but a fevered dream in the depths of Turing's pit; a lament for life as you scrawl warnings on the walls for whichever unfortunate souls follow. There is nothing of beauty and elegance to be found here, only the rending of flesh and regrets.

stuffed crust punk
Oct 8, 2004

by LITERALLY AN ADMIN

hackbunny posted:

Yourself or the five year old outsourced code that resisted automated conversion and defies human understanding

I don't need to be taught the basics of programming, please. I'm a one person team and I get very little downtime between new features to do "useless" work like fixing warnings, converting to ARC or experimenting with project settings. Or, god forbid, writing tests. I don't need the attitude, thanks

I'm not even complaining, programming tools are what they are and I'm used to working with flawed tools. Probably too used in fact

Myfuckinglife.rtf

dc3k
Feb 18, 2003

what.

Doctor w-rw-rw- posted:

If your methods are so huge that the delta between that and implementing it is > 15 minutes, you're writing a foolishly huge method.
If you measure method hugeness on an arbitrary 15 minute implementation time limit, you're making foolish assumptions.

Doctor w-rw-rw-
Jun 24, 2008

status posted:

If you measure method hugeness on an arbitrary 15 minute implementation time limit, you're making foolish assumptions.

Yeah, I assumed a lot. But I think in general – assuming incremental compilation is working and you have errors-on-warnings turned on – it's generally not a bad idea to stub out methods while focusing on whatever critical path of implementation you're working on, for code clarity, then break those down or refactor as needed. I've been staring at a bunch of code lately with methods that are stupid huge, and it's been bugging me. vOv

kode54
Nov 26, 2007

aka kuroshi
Fun Shoe

rjmccall posted:

if you're still not using ARC, you really only have yourself to blame.

I'd love to fix this some day with Cog, but I have no idea how I would deal with that. There are probably 5 million places I'll have to add autorelease pools, and remove retain/release cycles. And even then, I'll probably be leaking memory like crazy, since I don't even know how to properly test that.

The only thing remotely modern in the entire codebase is that a few things may or may not require C++11, for which I make the minimum deployment target 10.7, but I have never actually tested it on anything that old.

Anyone want to help? Pretty please?

Doc Block
Apr 15, 2003
Fun Shoe

hackbunny posted:

Yourself or the five year old outsourced code that resisted automated conversion and defies human understanding

I don't need to be taught the basics of programming, please. I'm a one person team and I get very little downtime between new features to do "useless" work like fixing warnings, converting to ARC or experimenting with project settings. Or, god forbid, writing tests. I don't need the attitude, thanks

I'm not even complaining, programming tools are what they are and I'm used to working with flawed tools. Probably too used in fact

I don't mess with project settings hardly ever, and 99% of stuff is left at the default setting, and yet I still get warnings for a lot of the things you mentioned. Of course, nearly all the code in my projects uses ARC, so that's probably why.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

rjmccall posted:

This is the sort of thing where the details really matter; I would need to see real code. If you're not comforting posting that here, you can send it to me specifically, or just file a radar and ping me with the number.

You know what, I looked at the code and there's some dodgy as gently caress code that could cause two instances of a singleton to exist at the same time, I will probably have to look into that! It's a rare event though, that will only happen at specific times. I checked one of the affected files that was attached to the ticket, and that can't be the issue, because the whole file is out of whack, it only seems to print the right data by chance. This was the code (stripped down of course):

Objective-C code:
#import <Foundation/Foundation.h>
#include <libkern/OSAtomic.h>

@interface TurdFileManager : NSObject

- (void) append:(NSString*)logEntry;

@end

#define FILE_PATH @"turd"

@implementation TurdFileManager {
    NSFileHandle *fhWrite;
    unsigned long long fileSize;
    NSFileManager *fileManager;
    NSData *encodingKey;
    const char *encodingKeyBytes;
    NSUInteger encodingKeySize;
    NSUInteger encodingKeyOffset;
}

- (id) init
{
    self = [super init];
    if (self != nil) {
        static char key[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 };
        encodingKey = [[NSData dataWithBytesNoCopy:key length:sizeof(key) freeWhenDone:NO] retain];
        encodingKeyBytes = [encodingKey bytes];
        encodingKeySize = [encodingKey length];
        encodingKeyOffset = 0;

        fileManager = [[NSFileManager alloc] init];

        [self createTurdFile];
        [self handlers];
    }
    return self;
}

- (void) dealloc
{
    [fhWrite release];
    [fileManager release];
    [encodingKey release];
    [super dealloc];
}

- (void) createTurdFile
{
    NSString* aStr = @"";
    NSData* aData = [aStr dataUsingEncoding: NSUTF8StringEncoding];
    [fileManager createFileAtPath:FILE_PATH contents:aData attributes:nil];
}

- (void) handlers
{
    fhWrite = [[NSFileHandle fileHandleForWritingAtPath:FILE_PATH] retain];
    fileSize = [fhWrite seekToEndOfFile];
    encodingKeyOffset = fileSize % encodingKeySize;
}

- (void) append:(NSString*)toAppend
{
    NSUInteger size = [toAppend maximumLengthOfBytesUsingEncoding:NSUTF8StringEncoding];
    char *bytes = malloc(size);

    [toAppend getBytes:bytes
             maxLength:size
            usedLength:&size
              encoding:NSUTF8StringEncoding
               options:0
                 range:NSMakeRange(0, [toAppend length])
        remainingRange:NULL];
    
    @synchronized(self) {
        OSMemoryBarrier();

        for (NSUInteger i = 0; i < size; ++ i, encodingKeyOffset = (encodingKeyOffset + 1) % encodingKeySize) {
            bytes[i] ^= encodingKeyBytes[encodingKeyOffset];
        }

        [fhWrite writeData:[NSData dataWithBytesNoCopy:bytes length:size freeWhenDone:NO]];
        fileSize += size;

        OSMemoryBarrier();
    }

    free(bytes);
}

@end
Before I put the calls to OSMemoryBarrier out of sheer desperation, I was getting corrupted data even within a single write. Like, it printed the first 16 bytes OK, and then immediately corruption. You'd think that all corrupted lines were corrupted on encodingKeySize boundaries, but no, not even that. I couldn't reproduce it personally, but it happened often

hackbunny fucked around with this message at 10:54 on May 2, 2016

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av
The way the pre-barrier code corrupted files is nothing short of fascinating. There is a pattern but I can't see it:
  • the first call writes 157 bytes. Except, 158 bytes are actually written: an extra byte is written at offset 16
  • second call writes 92 bytes at file offset 158, but 93 bytes are written. Extra byte at block offset 50 (file offset 209)
  • third call writes 84 bytes at file offset 251
  • fourth call writes 255 bytes at file offset 335 but a whopping 3 extra bytes are written: offsets 51, 161, 226 (file offsets 386, 496, 561)
The extra bytes are invisible to the obfuscation code, because if you remove them, the file decodes fine

...

Holy gently caress the extra bytes are all \r :aaa: It had nothing to do with race conditions! What really fixed it was renaming the file from .txt to .dat!

e: one mystery solved, one more mystery to go. I added extra log messages to the new code that seems to be failing in an impossible way, and guess what, the issue disappeared

hackbunny fucked around with this message at 14:39 on May 2, 2016

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Doc Block posted:

I don't mess with project settings hardly ever, and 99% of stuff is left at the default setting, and yet I still get warnings for a lot of the things you mentioned. Of course, nearly all the code in my projects uses ARC, so that's probably why.

And your project files probably aren't five years old. I don't think the project updater enables all of the same warnings that are now on by default.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av
Disregard everything I said, I'm a connoisseur of fine turds. Can you spot the issue?

Objective-C code:
- (void) turdDidSucceed {
    @synchronized(self) {
        if (turds.count > 1 && currentTurd > 0) {
            // Rotate the list of turds so that the successful turd is in front
            NSMutableArray *newTurds = [turds mutableCopy];

            for (NSUInteger i = currentTurd; i < turds.count; ++ i) {
                [newTurds addObject:turds[i]];
            }

            for (NSUInteger i = 0; i < currentTurd; ++ i) {
                [newTurds addObject:turds[i]];
            }

            currentTurd = 0;
            turds = [newTurds copy];
        }
    }
}
This duplicates turds every time it's called. The right way to initialize newTurds is [NSMutableArray arrayWithCapacity:turds.count]. I guess my initial idea was to rotate the array in place and then I opted for rebuilding it. You can probably see where this is going: the array grows exponentially. I just had this conversation with my QA man:

QA: I can't send the logs by e-mail, it says the mail is too large, over 40 MB
me: Huh, isn't that funny, it's supposed to only keep the last two log files, and they're limited to 2 MB. Let me see

So I replace the beta build with a development build, and I download the app container from Xcode. Two log files, one of which is over 30 MB. What the gently caress... I look inside it, and I find a single log entry spanning over 1.5 million lines. I was logging the contents of the array, and the array had grown from 2 entries into a monster. I never rotate log files mid-line so I ended up with an oversized file


It's nice when the comments are right but the code is wrong

hackbunny fucked around with this message at 17:08 on May 2, 2016

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
Happy to help! :)

I wasn't trying to be obnoxious about ARC. It really is possible to adopt incrementally, though; you don't have to convert everything in one massive three-month-long push. New code, and old code as you find yourself fixing bugs in it. Adding a #if !ARC #error block to the top of each implementation file is a great way to document your assumptions/progress.

Same thing about warnings. It's good to get warning-clean across your project, but since that's probably unattainable in one go, just keep opting in files to -Werror, one file at a time.

hackbunny
Jul 22, 2007

I haven't been on SA for years but the person who gave me my previous av as a joke felt guilty for doing so and decided to get me a non-shitty av

rjmccall posted:

Same thing about warnings. It's good to get warning-clean across your project, but since that's probably unattainable in one go, just keep opting in files to -Werror, one file at a time.

Problem is, I have zero warnings ATM (well one, but it's a deprecated method with no easy replacement), after I have painfully squashed every single one of them. I'll have to start another campaign to enable all warnings one by one (like -Wreturn-type) to see if I find any bugs, hoping I won't get too many false positives

The Xcode defaults are a little weird. Why is _FORTIFY_SOURCE set, but not -fstack-protector-strong? -fstack-protector-strong isn't even in the UI, you have to add it manually

e: great, -Wint-conversion isn't in the defaults, either...

hackbunny fucked around with this message at 18:37 on May 2, 2016

Adbot
ADBOT LOVES YOU

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

hackbunny posted:

Problem is, I have zero warnings ATM (well one, but it's a deprecated method with no easy replacement)

There's a pragma to temporarily suppress this if you want.

The big thing is that a lot of warnings get rolled by default by enabling them in new projects rather than just flipping the default in the compiler.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply