GrumpyDoctor posted:

lol

Mr Dog posted:

i wish ms wouldn't refer to their libc as a crt though argh that's not what a crt is god drat it, a crt is something that sets up the stack and prepares argc and argv (ok and envp but who gives a gently caress about that) for main

suffix posted:

microsoft

chmods please posted:

i thought it did or is it just winmain or what.

GrumpyDoctor posted:

lol

hackbunny posted:

no, it's statically linked in the executable, an object file called crt0.obj

I forget what the actual entry point for Win32 is conventionally called (boo, shame, boo), but it's implemented by crt0.obj, which sets up stuff and calls the user-defined main (or WinMain. or wmain. or wWinMain)

main vs WinMain makes no difference whatsoever, the code is the same and it can detect which entry point to call. the only difference between main and WinMain is that, if the linker finds a symbol called _main, it sets the executable's subsystem to "Win32 console", if it finds a symbol called _WinMain@16 (stdcall calling convention) it sets it to "Win32". this is just the default and it can be overridden with linker option /SUBSYSTEM (so you can have a text-mode program that starts from a WinMain and vice-versa)

WinMain used to actually matter in the bad old days when Windows was if not incompatible, at least extremely hostile to ANSI C and the arguments to WinMain actually were vital information provided by the kernel itself, while argc and argv were a relatively alien concept. if you look at the source code of crt0, you'll see that the kernel no longer passes any argument to the entry point (lie, it passes a pointer to the process's PEB, the grand central data structure with pointers to the heap, environment variables, command line, etc., but its layout is technically a secret at the Win32 layer crt0 operates at), and the arguments to WinMain are queried with public APIs you can use yourself (the arguments to main come from the libc's __argc and __argv exports)

hackbunny posted:

Microsoft posted:

In order to unify these different CRTs, we have split the CRT into three pieces:

chmods please posted:

yeah that's it, i haven't done native windows programming in a while. i went and looked and i think it's (w(in))mainCRTStartup? idk enough about PEs to find out what the actual assigned entry point is but it looks like there's a different crt0.obj for every combination of unicode and console subsystem? i assume there's a completely different kind of fun for native mode programs

hackbunny posted:

** I was about to say that the PE format has no native concept of constructors to be run before the entry point, unlike ELF, but that's not strictly true, you have TLS constructors. they technically are for initializing __thread-storage variables with a non-trivial constructor, but in practice they are passed the same arguments as DllMain, which is extremely interesting as you can get DLL_THREAD_ATTACH/DLL_THREAD_DETACH events in the main EXE, making _beginthread/_endthread redundant ***. I wonder why Microsoft never leveraged TLS constructors to do this: the old loader limitation where you couldn't dynamically load a DLL with a TLS segment doesn't even apply to an EXE. TLS constructors were even available before FLS ****

Mr Dog posted:

Huh. I wrote an entire PE loader myself and never knew about this.

Mr Dog posted:

Mind you that's one of the things I like about PE: it's very easy to turn an on-disk image into an executing image. Though traversing in-memory IATs and such to populate them is kind of a pain.

Shaggar posted:

what happens if some clients don't get notification of the new host and how do you control that notification?

quote:

It is easy to see that HRW avoids replication, thus potentially giving a higher hit rate, as long as clients have a consistent list of servers for each cluster. There are two possible methods for achieving such consistently quickly.

First, we could require participation from servers themselves, and have them maintain consistent lists. In this case, HRW could be run at the servers, and one server could forward or redirect client requests to another server. This method, however, incurs additional latency.

Alternatively, we could require clients to arrive at consistent lists, and try to minimize the convergence time for clients (and providers in eager-provider domains) to reach consistency in their server lists. One option is for clients to periodically resolve the server list amongst themselves, in which case the convergence time is equal to the resolution period. In this case, the resolution period should be chosen such that the convergence time is acceptable. On the other hand, when server lists are obtained from a "push" name service, this convergence time is simply the time until all clients receive a new server list from the name service.

Shaggar posted:

yeah i was thinking removing a node or having a node die is pretty easy to handle by replicating its objects to the appropriate next node for that object based on the same alg the clients are using, but if your clients are out of sync with a new node entering the cluster you could have clients w/ old data looking for/updating objects at the wrong node.

i guess if the node could detect that its the wrong place for the object it could try to fix it. either by sending it to the right place, redirecting the client to the right place, or by telling the client to go pound sand and update its list of nodes. imo the detection is gonna add enough overhead already so just tell the client to fix it.

Shaggar posted:

the pro move would be to use SRV records cause then you could put the weights right in there which makes adding/removing nodes easy and lets you do traditional location based dns.

to exit the cluster gracefully, a node sets its weight to 0, stops accepting requests, and then makes sure everything it had is replicated off to the proper secondary before shutting down. if you do the replication regularly its probably already up to date. clients that see the new 0 weight stop using it immediately and get the next node, clients out of sync who try to hit it get rejected and use the next node.

to add a node, have it fire up, add itself to srv with 0 weight, ask clustermates for all objects it would own at the weight it plans to go in at, once all those objects are received, set weight to whatever it should be. then if a request is made to a node that no longer owns the object, punt the client back w/ an error that its node list is out of date. client updates its list and finds the new server.

hackbunny posted:

the real fun is in C++/CLI executables. they are mixed native/managed executables (what a terrible idea), so you have a raw entry point in native code that queries the executable's CLI metadata to locate the user-level entry point in managed code and call it. this is because they never made a managed CRT, so even C++/CLI programs have to use the native one and end up as an ungodly mix of native and managed code

quote:

** I was about to say that the PE format has no native concept of constructors to be run before the entry point, unlike ELF, but that's not strictly true, you have TLS constructors. they technically are for initializing __thread-storage variables with a non-trivial constructor, but in practice they are passed the same arguments as DllMain, which is extremely interesting as you can get DLL_THREAD_ATTACH/DLL_THREAD_DETACH events in the main EXE, making _beginthread/_endthread redundant ***. I wonder why Microsoft never leveraged TLS constructors to do this: the old loader limitation where you couldn't dynamically load a DLL with a TLS segment doesn't even apply to an EXE. TLS constructors were even available before FLS ****

MononcQc posted:

Yeah that all works. There's also use-case-specific stuff there that you can add. I.e. for a cache server, it doesn't matter if data is replicated in all kinds of wonky places because it will eventually time out and fix itself, as long as clients are trusted to eventually readjust. The authoritative data can always be found from somewhere else.

Using it as a scheme for a key/value store likely requires you to be more careful. I like the idea of telling clients to go pound sand, but the redirection-based one is pretty clever because it lets the clients update on a per-need basis; if you have very infrequent changes in topology or a relatively large number of nodes, you can minimize how often things need updating. If you're writing very-frequently, it might become cheaper to frequently update your list, though.

Also as much as I hate sticky sessions, it sounds like a decent algorithm to have load-balancing done that way, with automated migration from endpoint A to new endpoint D once it's added to the cluster, slowly propagating resource usage. There's still a shitload of stuff that can go wrong, but that's more based on sticky sessions being terrible than anything else.


That does sound pretty nifty as an approach, especially for planned replacements. It requires you to really have and want to control things for the entire stack at once (the client needs to know about that ad-hoc protocol, the people maintaining servers must be able to toy with DNS, TTLs have to be respected everywhere, etc.), and you still need to handle catastrophic out-of-protocol failures, but it lets you know between 'retirement' and 'crashes/netsplits' which is pretty cool.

Shaggar posted:

really the minimum client side protocol would be that if you get denied by the server, refresh your list of servers.

Shaggar posted:

continually failing because the data in the cache is wrong is worse

Otto Skorzeny posted:

cache invalidation blows goat dicks

quote:

Description:
------------
The test script I provided _should_ print out "1", "2", "3", "4", and "5" in order and then it should exit gracefully.

When I run the test script under PHP 5.6.0alpha1 (downloaded and built today from http://downloads.php.net/tyrael/php-5.6.0alpha1.tar.gz), it prints out "1", "2", and "3", and then it gets stuck in an infinite loop that keeps printing out "4" over and over again. It never prints out "5" and the php process never exits.

I can consistently reproduce this behavior on my machine. I'm running CentOS Linux 6.3 on a 64-bit Intel Xeon processor, but I think this bug should be easily reproducible on most other platforms and processors.

I attached gdb to the php process to investigate, and it appears that the Zend engine is getting stuck in an infinite loop that keeps jumping between in execute_ex(), ZEND_JMP_SPEC_HANDLER(), ZEND_FAST_RET_SPEC_HANDLER(), and ZEND_ECHO_SPEC_CONST_HANDLER() from "zend_vm_execute.h".

I can also reproduce this bug with various builds of PHP 5.5.0.

Test script:
---------------

PHP code:

function bar() {
  try {
    echo "1\n";
  } finally {
    try {
      throw new Exception("");
    } catch (Exception $ab) {
      echo "2\n";
    } finally {
      echo "3\n";
    }
    echo "4\n";
  }
  echo "5\n";
}
bar();

Expected result:
----------------
1
2
3
4
5

Actual result:
--------------
1
2
3
4
4
4
4
4
4
4
4
... the php process keeps printing "4" over and over forever ...

Kevin Mitnick P.E. posted:

tbf that's an alpha. to be unfair how did their test suite not catch this, try/catch/finally inside a try/finally isn't a super complicated construct

Kevin Mitnick P.E. posted:

to be unfair how did their test suite not catch this

Suspicious Dish posted:

http://gcov.php.net/viewer.php?version=PHP_HEAD&func=tests

Kevin Mitnick P.E. posted:

oh wait i just saw 5.5 has it too. lol

chmods please posted:

don't all managed executables have a native stub which starts up the managed code with ICLRRuntimeHost? or did that go away?

chmods please posted:

you mean DllMain as a TLS constructor? i wonder if it's related to initialization order, is that not guaranteed? possibly could interfere with COM as well since it has its own threading concerns

MononcQc posted:

In other JS news (crosspost from security thread):

MononcQc posted:

After tef linked to them on twitter, I've tried playing with LRW/HRW/Rendezvous Hashing that is a different way to get something similar to a circular hash (with consistent hashing), but based on IPs and a weighted hash.

In a regular hash ring, we use some hash algorithm and wrap it around a circle where bytes are bucketed to specific sets of nodes, which means needing to either neither resize the circle (or require to re-hash and redistribute everything), or use vnodes like Riak does (which allows to control the resizing and eliminating hot spots).

That quickly gets to be tricky, and LRW/HRW is a very nifty simpler algorithm that can yield similar results, without the complexity. The trick comes from hashing the key of an object, along with the IP of all nodes you can contact, and get a weight out of it. You then sort the nodes by weight, and can get the top N of them to replicate data there.

What's cool is that the hashing is stable, and if you take away one node, only keys that had it as the lowest (or highest) weight need to be moved around. If you had a replication factor, you may not need to move anything just yet (rewriting on further keys may eventually move everything fine). When you add a new node, you can let things go until they're replicated, or force a re-hashing of most items -- they'll mostly all re-hash to the same destination, but maybe 1/|Nodes| of them will require being moved around.

I've put together a short implementation in Erlang: https://github.com/ferd/lrw (and a more general one than in the paper)

This is another super simple yet surprisingly powerful algorithm. Need more of these.