p-lang thread: (now (have you (problems two)))

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

Malcolm XML posted:

We must journey to the ivory tower to question wadler himself

https://www.youtube.com/watch?v=KaqC5FnvAEc

# ? Apr 17, 2013 20:52

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 19:23

Opinion Haver: Apr 9, 2007

Malcolm XML posted:

Well all that means is that it isn't a free monad but I still very much doubt the monad laws hold which is a bigger problem

i don't actually know much about monads in a general category theory context, just haskell :shobon:

# ? Apr 17, 2013 20:53

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

yaoi prophet posted:

i don't actually know much about monads in a general category theory context, just haskell

actually i'm not entirely sure that the opposite of a restricted monad (like Ord a => Set a) is properly called "free" since that means something specific w.r.t. categories

but confusingly a bunch of monads are in fact the free monads on a particular functor (since the free monoid on a set is essentially a list, the free monad on a functor is structurally very similar to a list)

Categories for the Working Mathematician is really good, anything by Mac Lane is great since he both invented categories and writes very well

but the rabbit-hole is deep and treacherous

# ? Apr 17, 2013 21:04

MeruFM: Jul 27, 2010

for anyone looking to explore horrible pseudo-languages, try beanshell

we had this integrated somewhere for a long time because code injection via XML was a `great` idea except it was an XML file and 99% of developers apparently can't write proper java without a 500mb IDE.

muh squiggly red lines

there was an unresolved issue for 3 months because someone used arrayThing[5] instead of arrayThing.get(5)

MeruFM fucked around with this message at 21:14 on Apr 17, 2013

# ? Apr 17, 2013 21:11

Shaggar: Apr 26, 2006

lmao forever at retards who hate ides and the massively huge benefits they provide over garbage text editors

# ? Apr 17, 2013 21:13

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

rofl beanshell

# ? Apr 17, 2013 21:14

Jonny 290: May 5, 2005; [ASK] me about OS/2 Warp

Shaggar posted:

lmao forever at retards who hate ides and the massively huge benefits they provide over garbage text editors

pshh i can't call it ~code poetry~ if a MACHINE writes it

bespoke subs, artisanal callbacks

# ? Apr 17, 2013 21:15

MeruFM: Jul 27, 2010

IDEs are fine, they're great for refactoring

i just find the idea of changing to a scripting language lookalike because the IDE isn't working for the static language funny

# ? Apr 17, 2013 21:22

Gazpacho: Jun 18, 2004; by Fluffdaddy; Slippery Tilde

Shaggar posted:

lmao forever at retards who hate ides and the massively huge benefits they provide over garbage text editors

on the other side you get people who literally can't work on a file if the IDE hasn't been customized to hold their hand through it

notepad supremacy

# ? Apr 17, 2013 21:24

double sulk: Jul 2, 2010

funny compiler

# ? Apr 17, 2013 21:25

Condiv: May 7, 2008; Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod

quote:

Adventures in Undefined Behavior

I recently had to write my own malloc. While replacing important bits of the C standard library would normally be serious over-engineering, it turned out to be the only option.

The Backstory
We had taken over an embedded project with known stability issues, and I quickly determined that thread race conditions were involved. After fixing some low-hanging fruit by switching to mailboxes and queues for cross-thread communication, there was still strange behavior � in particular, certain actions could cause infinite loops after a few tries or during heavy network activity.

Once I was able to consistently reproduce the issue, I focused on the affected code: One thread was allocating memory to notify the other about user input, the other allocated a response and put it on the network stack�s outgoing queue, and then both were freed. Two threads were allocating memory to send messages to each other safely, then freeing the buffers at the same time. If I commented out the free(2), the instability went away (at least until memory was exhausted). Ack! There was a race condition in free(2) itself! After consulting the C99 standard (free draft) and Harbison and Steele, I confirmed that malloc/free/etc. are not required to be reentrant, they just are on most platforms because doing otherwise is a world of pain. Unfortunately, that doesn�t include embedded platforms.

The race condition was during the update of free(2)�s internal records. If one thread free�d memory, but suspended while it was being returned to the freelist, another thread could free memory and clobber the freelist in mid-update. Since the updates were neither locked nor atomic, this could cause a cycle, and subsequent calls to malloc could get stuck in it.

Like most embedded projects, most data structures were statically allocated, but some libraries used dynamic allocation. In particular, a proprietary network stack library directly called malloc and free, so I couldn�t just wrap all their calls in a mutex. The embedded platform�s compiler had an option to generate a reentrant version of the standard library, but (despite what its documentation strongly implied!) this never called the lock/unlock callbacks I had to provide. As a last resort, I could monkeypatch all the standard library memory management functions � malloc, free, realloc, and calloc were weak symbols, and if I defined them with proper types, mine would be linked instead.

Lessons Learned
1. Design for Discoverability, Even in a Hex Dump
I had enough space to log allocations, but only in a packed format. (Unless I logged to flash, but that modified timing enough to break network functionality.) Still, I could control the structure, and had a few bytes of padding. All of my allocations were wrapped by four bytes on each side � two �[� characters, a uint16_t size (not counting padding), the actual memory, then two �]�s and a uint16_t monotonic allocation ID. That way, I could detect most buffer overruns at runtime, but this also meant that memory dumps looked like this:

..[[..230p8.80aeuoa$u.80ahhte&au-theothsnth]]..[[..$#(*GU>JH.hej]]..

00000000 2e 2e 5b 5b 2e 2e 32 33 30 70 38 2e 38 30 61 65 |..[[..230p8.80ae|
00000010 75 6f 61 24 75 2e 38 30 61 68 68 74 65 26 61 75 |uoa$u.80ahhte&au|
00000020 2d 74 68 65 6f 74 68 73 6e 74 68 5d 5d 2e 2e 5b |-theothsnth]]..[|
00000030 5b 2e 2e 24 23 28 2a 47 55 3e 4a 48 2e 68 65 6a |[..$#(*GU>JH.hej|
00000040 5d 5d 2e 2e 0a |]]...|
00000045
and the relative allocation sizes stood out. Also, after internal bookkeeping, the 2N-sized linked lists for common allocation sizes were filled with �L�s (for �link�), any memory in the large-chunk freelist was set to �F�s, and otherwise un-initialized heap memory was initialized to �u�s. This meant I could see the general state of the heap at a glance, and some kinds of pointer corruption stood out: the network stack was freeing memory mis-aligned to its allocation, causing one allocation to nest in another.

Similarly, I logged allocation metadata to a ring buffer in memory. Each of the records had a few bytes of padding (� � characters) so lines wrapped at 16 bytes, and looked like a s1s2 PpppCccc where a was �m� (malloc), �f� (free), �r� (realloc), or �c� (calloc), s1 and s2 were sizes (pre- and post- for realloc, otherwise 0 for s2), P was the pointer, and C was the same ID that followed the closing brackets. I came to recognize certain patterns in the log at a glance. Later, I was able to chase down a memory leak by converting the SREC memory dump to a human readable format, then cancelling out all freed allocations with awk. (Yes, I did miss valgrind.)

2. Malloc & Co. Have Many Edge Cases
The contracts assumed by malloc, free, realloc and calloc are more complicated than they appear. I see now why Lua�s designers chose to wrap them all in one function, lua_Alloc (whose design is closest to realloc). I didn�t know that when realloc is called on a NULL, it�s equivalent to malloc. malloc(0), while pointless, is legal, but implementation defined. free(2) and realloc(2) are full of undefined behavior. (I�d also never seen anyone actually use calloc before.)

3. Many C Programs Are Too Cavalier about Error Conditions
People debate whether it�s worth checking malloc(2)�s result for NULL when virtual memory (or the [OOM killer]) will pick up slack on modern systems, but few embedded systems have such things. If malloc returned NULL, things went haywire. The �C/C++� implementation of msgpack was usually the first thing to fail. (Spot the bug.) Systems need to check input for security and stability, and dynamic memory allocation is another place where bad data can creep in.

4. Fail Early and Loudly
The spoonful of memory corruption tends to compound on itself, eventually leading to swarms of mind-boggling bugs whose root cause is distant in space and time. Instead of letting the system struggle on, creating issues that will seem too bizarre to reproduce, it�s best to just halt everything and alert developers while there�s still a chance for a meaningful autopsy.

Closing
While the code (�spaceman,� since it manages space) is not open source, I may write a less-platform-specific version if there is interest. The overall design is similar to my mpool project on github, except that was written for a VM for an APL dialect I�ve been working on, and it doesn�t have any of the concessions to embedded platforms� limitations (e.g. it assumes mmap(2)).

i really don't get C hipsters

while it really doesn't apply much to this article, since it's about embedded development, people prattling off about how they can keep a model of their program in their mind because of how simple c is piss me off.

# ? Apr 17, 2013 21:53

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

You can tell that guy didn't start out as a C programmer because he refers to playing linker games as monkeypatching

# ? Apr 17, 2013 22:22

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

it's also kind of funny that the Right Answer (eliminating dynamic allocation where it was causing the problem because it wasn't actually needed) was in front of him and he just about stumbled upon it before writing his own malloc anyways

# ? Apr 17, 2013 22:26

MeruFM: Jul 27, 2010

tl;dr
that's a lot of words for not much of anything

# ? Apr 17, 2013 22:31

Zaxxon: Feb 14, 2004; Wir Tanzen Mekanik

Condiv posted:

i really don't get C hipsters

it's cause it's vintage.

# ? Apr 17, 2013 23:01

uG: Apr 23, 2003; by Ralp

C is good cause its not C++

# ? Apr 17, 2013 23:44

Zaxxon: Feb 14, 2004; Wir Tanzen Mekanik

uG posted:

C is good cause its not C++

that is one of it's finer aspects.

# ? Apr 18, 2013 00:41

Nomnom Cookie: Aug 30, 2009

guess what platform mandates a thread safe allocator

if you guessed the jvm, congratulations

# ? Apr 18, 2013 02:01

skeevy achievements: Feb 25, 2008; by merry exmarx

Otto Skorzeny posted:

You can tell that guy didn't start out as a C programmer because he refers to playing linker games as monkeypatching

also because he says this:

"I recently had to write my own malloc."

followed by this:

"it turned out to be the only option."

complete loving nonsense

# ? Apr 18, 2013 02:20

Nomnom Cookie: Aug 30, 2009

real c programmers know theres always more than one way to fault a seg

# ? Apr 18, 2013 04:41

Police Academy III: Nov 4, 2011

my favourite way is when you free something you weren't supposed to and then the next call to malloc gives you the same address back again and you're trying to use the same area of memory for two different purposes and wondering why completely separate areas of your program are loving up each other's poo poo

# ? Apr 18, 2013 04:45

uG: Apr 23, 2003; by Ralp

same but not my favorite

# ? Apr 18, 2013 04:46

vapid cutlery

Apr 17, 2007

php:

<?
"it's george costanza" ?>

Condiv posted:

i really don't get C hipsters

while it really doesn't apply much to this article, since it's about embedded development, people prattling off about how they can keep a model of their program in their mind because of how simple c is piss me off.

people bragging about implementing their own malloc are loving amateur retards.

# ? Apr 18, 2013 05:50

vapid cutlery

Apr 17, 2007

php:

<?
"it's george costanza" ?>

dude, look how close i am to the metal #themetal #malloc #emm #epicmemorymanagement #pointers

# ? Apr 18, 2013 05:52

MeruFM: Jul 27, 2010

**ptr &(int*)(int)ptr;

brb haxxoring here

# ? Apr 18, 2013 06:50

PleasingFungus: Oct 10, 2012; idiot asshole bitch who should fuck off

on one of the handful of occasions that I worked with C, I ended up loving an index into a local variable and started stomping on the call stack & corrupting function parameters

good times

# ? Apr 18, 2013 07:08

Guido van Possum: Apr 7, 2012; by T. Finninho

C more like P

# ? Apr 18, 2013 07:17

Workaday Wizard: Oct 23, 2009; by Pragmatica

how would c with types on the right (a la Rust, Go, etc.) look like?

would you like it?

inquiring minds want to know

# ? Apr 18, 2013 07:37

tef: May 30, 2004; -> some l-system crap ->

welsh

# ? Apr 18, 2013 07:47

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

tom jones was welsh i think

# ? Apr 18, 2013 15:02

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

hacking for christ :xd:

# ? Apr 18, 2013 15:06

GameCube: Nov 21, 2006

honking for christ

# ? Apr 18, 2013 15:18

Notorious b.s.d.: Jan 25, 2003; by Reene

MononcQc posted:

AMQP wasn't open source community effort at its origins despite being an open standard. It was started by JPMorgan Chase people for the enterprise:

It's the typical big business standardization effort that leads to overspecified poo poo like ws-* and whatnot.

most open sores crazies work for huge businesses, how do you think they get funding?

ibm mqseries is from 1993
biztalk is from 2000

they did not need to reinvent that wheel.

# ? Apr 18, 2013 15:24

FamDav: Mar 29, 2008

http://gergo.erdi.hu/projects/metafun/

hannnnnnnnnnnn

# ? Apr 19, 2013 05:19

vapid cutlery

Apr 17, 2007

php:

<?
"it's george costanza" ?>

PleasingFungus posted:

on one of the handful of occasions that I worked with C, I ended up loving an index into a local variable and started stomping on the call stack & corrupting function parameters

good times

i have no idea how you managed to do this accidentally

# ? Apr 19, 2013 07:26

uG: Apr 23, 2003; by Ralp

ive done it before accidently its especially p sweet when it will pass its test suite on one machine but not another identical machine

# ? Apr 19, 2013 07:32

Condiv: May 7, 2008; Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod

i see this argument more and more recently, but is there anyone here except shaggar and maybe tbc who thinks null was not a huge mistake?

# ? Apr 19, 2013 14:48

Dr. Honked: Jan 9, 2011; eat it you slaaaaaaag

vapid cutlery posted:

i have no idea how you managed to do this accidentally

it's super easy to do in c, local arrays are on the stack, nothing to stop you indexing beyond the end or beginning , and then you're stomping on all kinds of poo poo

# ? Apr 19, 2013 14:52

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

Condiv posted:

i see this argument more and more recently, but is there anyone here except shaggar and maybe tbc who thinks null was not a huge mistake?

In what context?

# ? Apr 19, 2013 19:22

Adbot: ADBOT LOVES YOU

# ? May 25, 2024 19:23

uG: Apr 23, 2003; by Ralp

0 == NULL

# ? Apr 19, 2013 20:09

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »