|
Soricidus posted:for how long did you think anything else? there's "this is a terrible idea that will lead to unintended consequences" bad, and there's "this breaks the interface contract, can lead to broken states, and should be fixed even at the cost of backwards compatibility" bad sure, equals/hashCode breakage happens all the time in badly written third-party code but it shouldn't happen in the standard library imo i did a little testing, and this leads to duplicate items in hashmaps, but not of the exact same item since the hash code is cached. test code: Java code:code:
|
#
?
Oct 30, 2014 05:19
|
|
|
|
| # ? Jun 11, 2024 08:44 |
|
|
suffix posted:there's "this is a terrible idea that will lead to unintended consequences" bad, and there's "this breaks the interface contract, can lead to broken states, and should be fixed even at the cost of backwards compatibility" bad this worked so nicely for python3 and perl6 or you u could use URI you baby
|
|
#
?
Oct 30, 2014 10:24
|
|
|
suffix posted:there's "this is a terrible idea that will lead to unintended consequences" bad, and there's "this breaks the interface contract, can lead to broken states, and should be fixed even at the cost of backwards compatibility" bad url should be marked as deprecated
|
|
#
?
Oct 30, 2014 14:05
|
|
|
akadajet posted:http://builtwithember.io/ I have heard of all those identical base-template-looking emberjs sites, but idk who dell or godaddy are.
|
|
#
?
Oct 30, 2014 14:11
|
|
|
akadajet posted:it's like, angular's headliner product. Hahaha. I didn't use it much because it was unusable I don't think that's really a horror though. Why are they using a client oriented js to build what should be a native app.?
|
|
#
?
Oct 30, 2014 14:39
|
|
|
because ~web "developers"~
|
|
#
?
Oct 30, 2014 14:43
|
|
|
wonder if i can bait him into a probation-by-Silverlight-post
|
|
#
?
Oct 30, 2014 14:53
|
|
|
MALE SHOEGAZE posted:Hahaha. i was wondering why ps3 youtube was so goddamn slow
|
|
#
?
Oct 30, 2014 15:39
|
|
|
can we talk about erlang error messages for a second?code:code:
|
|
#
?
Oct 30, 2014 16:22
|
|
|
It's a formatted stack trace. What's not to understand about it?
|
|
#
?
Oct 30, 2014 16:31
|
|
|
maybe if it was actually formatted with indentation and line breaks it would be better, but not by much
|
|
#
?
Oct 30, 2014 16:44
|
|
|
Pie Colony posted:maybe if it was actually formatted with indentation and line breaks it would be better, but not by much If it was formatted that way, it'd confuse a bunch of line-based loggers, I believe. The default error logger shipping with Erlang prints it formatted, but lager (which you likely use here indirectly) forces them on a single line.
|
|
#
?
Oct 30, 2014 18:03
|
|
|
Malcolm XML posted:yeah i mean NT itself ("native") can fork processes and poo poo "{Nt,Zw}CreateProcess" <-- maybe ok, it's been a while since last time I did this. here we go superficially, it may look like win32 has almost everything needed to implement the UNIX core APIs except for a low-level fork primitive that creates a copy on write duplicate of the calling process. handles are pretty much the same thing as file descriptors, and everything revolves around handles and objects just like everything in unix revolves around file descriptors let's see how win32 could implement a hypothetical CloneProcess function. a rough outline of what CreateProcess does:
CloneProcess looks pretty simple to implement:
so easy! so straightforward! so broken! you may or you may not realize that the clone process is created with an empty handle table, except for the inheritable handles. all file handles (and other handles) are now invalid, the clone process is useless. making a handle inheritable is equivalent to clearing FD_CLOEXEC for a file descriptor, which is all fine and well since CreateProcess is a fork+exec equivalent and inheritable handles is what you need for that, but totally useless in the case where fork essentially behaves like exec. how do we clone the handle table, inheritable and non-inheritable handles alike? well, we can't, there is no way. except by deep, undocumented kernel fuckery, of course. DuplicateHandle in a loop would work, if it made any guarantee of giving the duplicate handle the same value (it doesn't) let's say we do resort to deep, undocumented kernel fuckery. we now have perfect duplicates of the original handles, with the same values and referring to the same objects. again, this is broken. file objects (most common object type after events, iirc) behave in a subtly different way from UNIX file descriptors. while multiple handle can refer to the same object, one file number refers to one file descriptor, no exceptions: duplicating a file descriptor doesn't create a new reference to the same descriptor like duplicating a handle does. multiple handles to the same file object compete for access to the same data, like the current read/write position and the synchronous I/O lock: we have created a mechanism to generate interesting deadlocks at best, or data corrupting race conditions at worst. we'd have to special case file objects, instead of duplicating their handles we'll outright reopen them (... and copy over metadata like the aforementioned current r/w position), and use even more fuckery to ensure the reopened files have the same handle values as the the originals assume we made it so far. still broken. reopening a pipe doesn't give us an independent descriptor of the same pipe, it tries to reconnect to the pipe, which is fine if the pipe is something like a RPC client end (if not for the fact that it will completely misalign the actual state of the connection with the state expected by the client code), but will outright fail for anonymous pipes, which are created with a connection limit of 1. god knows what happens with other very special files like sockets. these special files are probably safer to just duplicate instead of reopen ok, assume we successfully special-cased for pipes, sockets and the like (may not be actually possible). broken again. there's a very important object type that really, really doesn't like being shared across processes: they're called LPC (local procedure call) ports, and they're the single most important IPC mechanism in all of windows. nearly all built-in windows services use them as an RPC transport, and out-of-process COM components are invoked through them. some win32 APIs are secretly based on RPC calls. all of these things automatically stop working in the cloned process, or at the very least all current connections are interrupted (I don't know if the RPC client even has reconnection logic for LPC ports considering they're never supposed to disconnect) let's be optimistic, and assume that all RPC clients are well behaved and eventually reconnect (may not be true). well I hope you weren't using windows in your parent process, because the cloned process is going to be boarded up like a cellar in a storm. all window handles are now automatically invalid, because there's no way to duplicate windows, and even if there was, window handles share a namespace across all processes in the same session. nominally, the handles are still valid, but the windows are owned by the parent process: you can send messages to them, but not receive messages sent to them. maybe you think you don't have any windows in your process, but you'd be surprised, a lot of libraries create hidden windows to receive broadcasts or implement per-session IPC protocols. in fact I think OLE itself creates hidden message-only windows when you run an OLE message loop. all of these things stop working in the cloned process. again a lot of APIs and libraries may use hidden windows or OLE and you have no way of knowing ok, let's speculate wildly. let's say we define a new notification for DllMain, something like DLL_PROCESS_BEFORE_CLONE and DLL_PROCESS_AFTER_CLONE that would work like pthread_atfork. let's say all core system libraries use these notifications to properly clone their state to the child process (the simplest implementation would be to simply mark all state invalid, and reinitialize on first use, to add the least overhead to CloneProcess). let's say that we document the limitations this imposes on what you can do in a cloned process (like doing UI work). a regular, honest, blue-collar command line program will do just fine! it will be, after all, the most common use case for CloneProcess, and we only have to ensure that things like console I/O still work CloneProcess breaks console I/O specifically. in win32, consoles are not I/O devices. sure you open handles to them, you pass them to ReadFile and WriteFile, but a few things should have tipped you off that they are impostors. for one, console input handles are waitable: sure file/device handles are waitable, but they are signaled when a I/O operation completes, not when data is available to read. that select/poll-like behavior is unique to console input handles handles (in fact there is no select/poll primitive: you use select/WSAPoll for sockets, wait primitives for console input handles and redesign for asynchronous I/O for everything else). consoles are odd values, while actual handles are multiples of 4 (the lowest two bits are reserved as tag bits). so maybe consoles are of a special object type and their handles are tagged to save a roundtrip to kernel mode? no, consoles are an entirely user mode thing. all console APIs are actually RPC calls; when you pass a console handle to WriteFile, WriteFile punts to WriteConsoleA; when you pass a console handle to WaitForSingleObject, it's replaced with a handle to an event that's signaled when input is available; and so on. the implementation of consoles is not just any service, it's a critical system service that among other things implements the few win32 functionality that doesn't belong in kernel mode (initially: all of it. since NT 4: very little). of all the things that should have been implemented in kernel mode, consoles would top my list, but Microsoft thinks differently. whatever so what, just connect to this service (basesrv - "base" is the internal codename for win32 - running in csrss.exe - client-server runtime subsystem, once supposed to be the cornerstone of the NT microkernel which never was), ask to inherit the parent's console. can't be done, because we weren't registered as a win32 process. yeah, I lied, CreateProcess does one more thing, which is calling to basesrv to register the newly created process and thread. basesrv will reject any calls that don't come from a currently registered process (LPC ports reliably notify servers of the client's process and thread ids; throughout NT, the process id/thread id tuple is called a "client id", again something that was supposed to be far more important had NT actually been implemented as a microkernel), so this is a mandatory step. CloneProcess keeps getting fatter and fatter: while a lot of state cloning operations can be simply delayed for when the related feature is actually used, registering the clone process and thread is pretty much mandatory. still, not much more overhead than CreateThread, which is the point of a fork function speaking of threads. have you ever wondered what's inside user mode mutexes, i.e. RTL_CRITICAL_SECTION? it's by no means mandatory, but a mutex should ensure that only its current owner can release it. sure, properly written code will never do otherwise, but it's a nice feature to have, and as it turns out, RTL_CRITICAL_SECTIONs do keep track of the current owner. the current owner is identified by its thread id. the cloned thread has a different id (thread ids are globally unique, in fact they come from the same global handle table as process ids; this wouldn't be necessary if all thread manipulating APIs required both a thread and process id - i.e. the full client id - but that ship has sailed), so it won't be the owner of the cloned critical sections (oh and, luckily critical sections no longer contain a wait queue handle, or the cloned critical sections would share the wait queue with the original critical sections). the cloned thread won't be able to release locks it's supposed to own, and will deadlock when trying to re-acquire them. luckily, for easier debugging, critical sections are all listed in a process-wide doubly-linked list, so we can fix them in the cloned process (more overhead). we could redesign critical sections so that the owner is identified by some per-thread value that carries over to the clone, like the stack base address. the stack base address sadly is unusable, because it can change if the thread is used to run fibers. we could use the TEB, a large amalgamation of all sorts of internal thread-specific data (like the TLS values for the thread, or a copy of the thread id), but the clone thread is in no way guaranteed to have the same TEB address in fact, speaking of the TEB: we forgot to copy the TEB of the original thread to the clone's. we can't just copy everything over either, things like the thread id should obviously not be overwritten. the TEB is also extremely dense and complicated, and shared between a dozen components. it even contains interior pointers that refer to other parts of itself, which should be fixed up in the clone. and god help you if someone is keeping a pointer to some TEB field because that poo poo will break too. also pray that you aren't calling CloneProcess inside of some system function that uses the TEB, because when you return to that code, it may refer to the old TEB (which may or may not carry over to the cloned address space, I don't know). I don't know if the PEB (like the TEB, but for the whole process) is clone automatically either, it probably isn't and we'll have even more work to do assuming we fix all users of the TEB to avoid clone-unsafe operations, we can store some kind of local thread id in the TEB, just some integer incremented for every new thread, with the last value stored in some global variable that will be cloned with the address space, and use that as the owner of critical sections, so that we won't have to fix them up so, after a massive review of an ancient codebase with 30 years of legacy, we have at best a niche, semi-reliable API with an uneasy trade-off between massive overhead, worse than CreateThread and possibly even CreateProcess, and delayed reinitialization, with its associated out-of-context failures this is all going mostly from memory and I'm sure I'm missing even more problems, but you get the idea. fork for win32 is never going to happen ... by the way, CloneProcess only clones the calling thread, what about all those worker threads secretly spawned by third party libraries, that will now stop working in confusing ways? all DLLs will have to implement DLL_PROCESS_BEFORE_CLONE and DLL_PROCESS_AFTER_CLONE, because there is no high level construct that unequivocally identifies a worker thread as such. also hope that nobody is keeping process and thread ids cached somewhere! e: oh, actually I didn't answer re. message pumping. the cloned thread is going to have no message queue, which isn't that terrible as they are automatically created on demand. much worse is if you called CloneProcess while inside the message pump, which is altogether a very easy situation to get yourself into. user32 is probably not going to like that the entire message queue disappeared from under its rear end. oh and of course if you lose the message queue, you lose all the messages and this could cause deadlocks if you were waiting for them. non-window messages could be cloned I guess, but messages meant for internal hidden windows are going to be lost this way. it's all very delicate hackbunny fucked around with this message at 19:09 on Oct 30, 2014 |
|
#
?
Oct 30, 2014 19:03
|
|
|
so how the gently caress does cygwin work at all in such a hostile environment? is it all fuckery with threads pretending to be processes?
|
|
#
?
Oct 30, 2014 19:24
|
|
|
is there any actual reason to want to clone a process like that? also cygwin exists to solve a problem that doesn't exist on windows.
|
|
#
?
Oct 30, 2014 19:46
|
|
|
Notorious b.s.d. posted:so how the gently caress does cygwin work at all in such a hostile environment? cygwin fork is a gigantic hack, they actually documented what went into it but I can't remember the details cygwin has a huge advantage though, in that user applications and libraries by and large don't use Win32 directly, so their "kernel" layer can insulate user code from almost all of this mess. also they don't attempt in any way to fork Win32 state iirc, at the Win32 layer the child process is a regular process, they just copy over stuff from the parent process into it
|
|
#
?
Oct 30, 2014 19:49
|
|
|
how did http://en.wikipedia.org/wiki/Windows_Services_for_UNIX work? or UWIN ?
|
|
#
?
Oct 30, 2014 20:03
|
|
|
Shaggar posted:is there any actual reason to want to clone a process like that? not really because with threads and without exec, fork is pointless. you can do cute things like "going in background" in a console program, where you return to the shell while the program keeps running in background, but that's it exec is the main use case of fork: you fork, set up things in the child process (kernel things, like base priority, security token and so on) and then you exec to reinitialize the process from scratch with a new executable program and command line. that's why fork/exec has so few arguments and CreateProcess so many and so many variants (like CreateProcessAsUser): fork lets you make your own process creation options basically. it's like what I said about using objective c blocks to wrap existing code instead of refactoring everything into an operation packet that you submit and process: being able to pass code around, instead of packets, is so incredibly flexible let's call the Win32 exec ReCreateProcess. let's say the parent process creates a lot of children, always setting their stdin/stdout. now, the only way to pass stdin/stdout to a child process is by passing inheritable handles. but if you have two concurrent child process creations, both processes will get all currently inheritable handles, not just the ones meant for them! so. instead of calling CreateProcess, we change the parent process to use CloneProcess. in the clone, we use DuplicateHandle to make inheritable duplicates of the stdin/stdout handles to pass to the child process. we pass the inheritable duplicates to ReCreateProcess (which we can assume takes a STARTUPINFO argument, like CreateProcess). the clone process turns into a proper child process, running the program passed to ReCreateProcess with a new command line and the specified stdin/stdout handles another example: sanitizing environment variables. sure, you can make a copy of the current environment, and then gently caress with it to remove the offending variables, and pass the sanitized copy to CreateProcess. or you can use CloneProcess, sanitize the environment with regular GetEnvironmentVariable/SetEnvironmentVariable calls, and then call ReCreateProcess (which we can assume takes a lpEnvironment argument, that we'll set to NULL to mean "inherit current environment") to run the program with the sanitized environment also: in some applications, separate address spaces for threads are very useful, to ensure they can't easily corrupt each other's state fork/exec is a very flexible mechanism
|
|
#
?
Oct 30, 2014 20:08
|
|
|
fork/exec sounds like dumb Linux crap and a bad idea.
|
|
#
?
Oct 30, 2014 20:21
|
|
|
easy: no Win32 to worry about. Interix programs are run as clean, pure POSIX programs, with no Win32 dlls needed or supported, just ntdll.dll (which hates cloned processes and doesn't even attempt to do things like fix up critical sections; in fact it creates a new main heap and loader lock altogether). if you wanted Win32 functions on Interix you'd have to, like, port Wine to it to interact with the Win32 world somewhat, they have Win32 launchers (posix.exe) on one side, and on the other, Interix is a microkernel where all system calls are LPC calls to a subsystem process (psxss.exe), which is a regular Win32 program. very important because things like winsock require Win32 (in fact, the original Microsoft-provided POSIX subsystem didn't have sockets and it was useless and garbage). at least, this was Interix 1 in Interix 2 and later, they moved a lot of system calls to client-side posix.dll (which previously only contained libc and LPC stubs), especially things like I/O, if they could be implemented entirely in terms of NT system calls; this massively improved performance for things like, say, read or write which didn't affect global state or involved security checks. they also improved psxss.exe to better interact with Win32, in particular things like mapping Win32 consoles to POSIX ttys. also they ported gcc and binutils so that you could have abominable frankenstein mostrosities of executables that were PE on the outside and old school COFF on the inside, with modern amenities like dynamic loader support and GOT relocations (Windows DLLs are not exactly the same thing as UNIX shared objects, so they added UNIX shared objects to Windows) in Interix 3 they introduced mixed programs, where you could use both Win32 and POSIX APIs in the same program (apparently the use case for this was using Oracle's client DLL; similar bending over backwards for Oracle can be observed in ADO.NET). to sidestep all the known issues with this setup, mixed programs were compiled and run as Win32 programs that could call Interix system calls if necessary, and they were barred from using fork tef posted:or UWIN ? no idea. never used UWIN or MKS
|
|
#
?
Oct 30, 2014 20:27
|
|
|
Shaggar posted:fork/exec sounds like dumb Linux crap and a bad idea. shaggar getting swept away by the tsunami of actual windows internal knowledge being poured into this thread
|
|
#
?
Oct 30, 2014 22:02
|
|
|
posts like that make me amazed that windows not only functions, but functions so well that our world literally relies on it
|
|
#
?
Oct 31, 2014 00:48
|
|
|
you can tell i'm a webdev i can't comprehend most of hackbunny's posts
|
|
#
?
Oct 31, 2014 01:14
|
|
|
meanwhile, the Austin group has realized that fork is incompatible with threads and has amended the spec to make fork async-signal-unsafe and forbid the use of any async-signal-unsafe functions in the child after a fork
|
|
#
?
Oct 31, 2014 03:24
|
|
|
uncurable mlady posted:posts like that make me amazed that windows not only functions, but functions so well that our world literally relies on it to be fair to win32, CloneProcess was just a thought experiment where we stressed its assumptions pretty hard: when all you have is fork+exec, of course you're going to build over that and pile more and more code over the years that assumes that a new process always starts from a clean slate but yeah, it's not an elegant design by any stretch. still better than 80s relic VMM (Virtual Machine Monitor, the Windows 9x kernel) e: Virtual Machine Manager oh god I'm such a fraud  pseudorandom name posted:meanwhile, the Austin group has realized that fork is incompatible with threads and has amended the spec to make fork async-signal-unsafe and forbid the use of any async-signal-unsafe functions in the child after a fork  of course
hackbunny fucked around with this message at 10:20 on Oct 31, 2014 |
|
#
?
Oct 31, 2014 10:17
|
|
|
lmao Reporter field
|
|
#
?
Oct 31, 2014 10:26
|
|
|
hackbunny posted:lmao Reporter field you can't really blame a guy for being passionate about ornithology
|
|
#
?
Oct 31, 2014 11:47
|
|
|
akadajet posted:you can tell i'm a webdev i can't comprehend most of hackbunny's posts
|
|
#
?
Oct 31, 2014 12:09
|
|
|
hackbunny posted:but yeah, it's not an elegant design by any stretch. still better than 80s relic VMM (Virtual Machine Monitor, the Windows 9x kernel) e: Virtual Machine Manager oh god I'm such a fraud "virtual machine manager" is now a thing that's used to manage hyper-v vms
|
|
#
?
Oct 31, 2014 12:28
|
|
|
really sort of weird that microsoft had such a slow start at enterprise virtualization, since they did legit do more software virtualization work earlier than anyone else. all very special-purpose, but still recognizably the right sort of thing
|
|
#
?
Oct 31, 2014 15:01
|
|
|
hackbunny posted:cygwin fork is a gigantic hack, they actually documented what went into it but I can't remember the details they have FAQs and it is pretty nasty https://www.cygwin.com/faq.html#faq.api.fork quote:Cygwin fork() essentially works like a non-copy on write version of fork() (like old Unix versions used to do). Because of this it can be a little slow. In most cases, you are better off using the spawn family of calls if possible. the exec implementation is even worse. apparently exec is impossible on windows so cygwin tracks fake PIDs to provide the illusion of continuity https://cygwin.com/cygwin-ug-net/highlights.html quote:However, spawn and exec present their own set of difficulties. Because there is no way to do an actual exec under Win32, Cygwin has to invent its own Process IDs (PIDs). As a result, when a process performs multiple exec calls, there will be multiple Windows PIDs associated with a single Cygwin PID. In some cases, stubs of each of these Win32 processes may linger, waiting for their exec'd Cygwin process to exit. there is lots of "fun" stuff in the highlights document i'm not going to quote the whole thing
|
|
#
?
Oct 31, 2014 16:17
|
|
|
dont use fork
|
|
#
?
Oct 31, 2014 16:21
|
|
|
Notorious b.s.d. posted:the exec implementation is even worse. apparently exec is impossible on windows so cygwin tracks fake PIDs to provide the illusion of continuity Interix does this too. run Interix ps, and you'll see that all Interix processes have odd pids. as we have learned:
so, Interix has a virtualized process table hackbunny fucked around with this message at 16:50 on Oct 31, 2014 |
|
#
?
Oct 31, 2014 16:35
|
|
|
hackbunny posted:Interix does this too. run Interix ps, and you'll see that all Interix processes have odd pids. as we have learned: interix was a clusterfuck btw hackbunny have u considered writing for the Windows Internals people?
|
|
#
?
Oct 31, 2014 16:54
|
|
|
prefect posted:"virtual machine manager" is now a thing that's used to manage hyper-v vms I guess but it's a totally different codebase Notorious b.s.d. posted:so how the gently caress does cygwin work at all in such a hostile environment? this is how that one ksh for windows works
|
|
#
?
Oct 31, 2014 16:57
|
|
|
Malcolm XML posted:interix was a clusterfuck still the best UNIX implementation for Windows ever. they even ported Gentoo to it IIRC Malcolm XML posted:btw hackbunny have u considered writing for the Windows Internals people? I have, and I guess alex ionescu could even vouch for me, but
maybe it's the underachiever speaking, but these are "amateur computer historian" credentials which at best entitle me to give talks in a silly hat*
|
|
#
?
Oct 31, 2014 17:19
|
|
)
|
Malcolm XML posted:I guess but it's a totally different codebase i just wanted to contribute some low-value trivia to the discussion 
|
|
#
?
Oct 31, 2014 17:20
|
|
|
hackbunny posted:I don't feel like an "expert", it's more like a persistent strain of itchy herpes. I write all of my posts 99% from memory, I literally can't forget this stuff same, and most of the time i can't remember what day of the week or what date it is. i also get left and right mixed up. quote:[*]I'm not rigorous nor an expert on OS design/development, more of an excitable fangirl. I'm in awe at actual experts like Russinovich or (leaving the field of operating systems) MononcQc, tef, etc. (talk prolog to me tef i'm just a database janitor, i have no formal qualifications, i only did two years of computer science etc. i have read a lot of papers though and people think that counts, but i haven't really written anything or given any advice that i myself did not read or learn from another. i only became a full time programmer by accident, and almost every job bar one i've gotten was through referrals  i started in computer security and family stuff meant i had to move and get a new job and i ended up writing screen scrapers, and it was all downhill from there. quote:maybe it's the underachiever speaking, but these are "amateur computer historian" credentials which at best entitle me to give talks in a silly hat folklore is fine to tell, and you're pretty good at it. you manage to cram a lot of detail in without the prose disintegrating to default. maybe because you've told this story before.
|
|
#
?
Oct 31, 2014 17:49
|
|
|
i will be happier if i was referred to as a "youtube expert"[1]. as that is the only medium in which i have garnered any attention, outside of writing hn bait on tumblr [1] http://i.imgur.com/rhtx8kr.png tef fucked around with this message at 17:58 on Oct 31, 2014 |
|
#
?
Oct 31, 2014 17:55
|
|
|
|
| # ? Jun 11, 2024 08:44 |
|
|
tef posted:i will be happier if i was referred to as a "youtube expert"[1]. pretty sure you're a twitter thought leader
|
|
#
?
Oct 31, 2014 18:00
|
|