p-lang thread: (now (have you (problems two)))

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »

Shaggar: Apr 26, 2006

Kevin Mitnick P.E. posted:

every time i use a scala library it's like...the type system isn't expressive enough to do what the author wanted in a straightforward way. but if they do enough wankery with traits and macros and implicit classes then you can mostly get there. really that's what everything coming from typesafe is like. a tech demo

hobbyist languages are all tech demos

# ? Jul 3, 2014 13:36

Adbot: ADBOT LOVES YOU

# ? Jun 2, 2024 15:34

Sapozhnik: Jan 2, 2005; Nap Ghost

I think TCP lets you close one half of a connection.

Plan 9's design is just stupid, sorry. You're straight-jacketing everything through an abstraction of "hierarchy of named objects that you can exchange byte streams with". Sometimes you need something richer than a stream of bytes, and sometimes you want to group things in a manner other than an associative array. Down that road lies implied semantics and stringbashing and just general arbitrary sloppiness.

9p is also dumb as poo poo. Time and time again people decide that if you could just transparently marshall function calls and byte streams then you can magically sprinkle this pixie dust onto a non-distributed application to turn it into a distributed one. Marshalling is, like, the least difficult part of making a distributed system. Cost models for caching and buffering getting thrown completely the gently caress out of whack because your latency and throughput characteristics just got shifted by a few orders of magnitude would be the second least difficult part of making a distributed system to deal with, but nobody ever even thinks that far ahead. So you have borderline unusable abominations like network filesystems or X11.

For every difficult problem there is an answer that is simple, elegant, and wrong. And "everything is a file" is not even elegant.

# ? Jul 3, 2014 13:57

Sapozhnik: Jan 2, 2005; Nap Ghost

I'm just particularly angry about this issue because a contractor I'm working keeps having his license server for his embedded IDE go down so I can't debug stuff in his lab over RDP. I mean, that's painful enough, but the workaround I'm using is a remote USB connection from the JTAG in his lab to my laptop running an IDE about 4000 miles away.

Just flashing in a build takes literally five minutes. Setting a breakpoint or doing a quick-watch takes about ten seconds for the UI to respond. This isn't an intentional example of this "magically make it distributed" crap so much as it is a desperate last resort but it is a particularly extreme example of it

# ? Jul 3, 2014 14:00

MononcQc: May 29, 2007

Mr Dog posted:

I think TCP lets you close one half of a connection.

It lets you do it if you ACK a FIN, rather than doing your own FIN+ACK to it. That leaves you with a remote end that won't send you data, but by protocol design should still be accepting yours.

Some/many stacks however will do this temporarily, or up until you read from the closed connection, or time out, and then go and either RST it or close the other end entirely on your behalf when they can, because you don't have access to that protocol level of details up from your languages and libraries. You may need a specific mode or function call to say that you explicitly want to support half-opened/half-closed connections.

MononcQc fucked around with this message at 14:08 on Jul 3, 2014

# ? Jul 3, 2014 14:03

Sweeper: Nov 29, 2007; The Joe Buck of Posting; Dinosaur Gum

Damiya posted:

Can anyone recommend some good reading on distributed application architecture?

At work we're in a kind of lovely place where we have to keep piling new messages into a queue that a worker dyno consumes instead of being able to spin up services that listen for events.. The initial implementation was done this way because of a need for resiliency ('if our worker dyno crashes the message just sits in RabbitMQ') but this makes adding new stuff a lot harder to encapsulate.

For example, we have a set of tasks that we kick off whenever a user writes a comment on a page; we now need to add a new task, which involves setting up another message to get published onto the queue instead of just broadcasting out 'hey, a user commented' and writing code that listens for and responds to that.

Where's a good place to start growing my knowledge of this kind of thing? I'm familiar with the rough concepts around Service-Oriented/Event-Driven, but would like to dig into the meat of how people in the real world actually make these things go.

we wrote a scheduler which gets partitioned on different categories of work and it sends tasks to the workers which are also partitioned

our task dependency tree is pretty complicated so you may not need this, but like just have your worker implement all of the message types and just let any if them pull whatever off of any queue?

how complicated is your workflow

# ? Jul 3, 2014 14:34

jony neuemonic: Nov 13, 2009

double sulk posted:

how do i do this

also i can't really move because i am getting married next year and we live in a house. however we will be potentially moving away for one year while she does her matching program so if an online course exists, that would be great. i know there's that georgia tech one that they started up but it also requires a bachelors. maybe i'll try talking to them though.

https://intellij-support.jetbrains.com/entries/23455956-Selecting-the-JDK-version-the-IDE-will-run-under

just swap in 1.8. then go back to eclipse because you didn't know how good you had it before.

# ? Jul 3, 2014 15:56

Malcolm XML: Aug 8, 2009; I always knew it would end like ｔｈｉｓ．

pls stop ruining my fantasy of a world w/o ioctls

ioctls are awful and bad but too goddamn necessary

# ? Jul 3, 2014 16:24

Sweeper: Nov 29, 2007; The Joe Buck of Posting; Dinosaur Gum

fidel sarcastro posted:

https://intellij-support.jetbrains.com/entries/23455956-Selecting-the-JDK-version-the-IDE-will-run-under

just swap in 1.8. then go back to eclipse because you didn't know how good you had it before.

eclipse blows compared to IntelliJ

# ? Jul 3, 2014 16:41

Shaggar: Apr 26, 2006

wrong. intellij sucks compared to eclipse.

# ? Jul 3, 2014 16:43

Squinty Applebottom: Jan 1, 2013

Shaggar posted:

wrong. intellij sucks compared to eclipse.

java stymie

# ? Jul 3, 2014 16:47

AWWNAW: Dec 30, 2008

i had to do a pipeline of services that had to process poo poo linearly and i just stuffed relevant info into the messages that allowed each subscriber/service in the pipeline to subsequently queue the next message lol

# ? Jul 3, 2014 16:55

jony neuemonic: Nov 13, 2009

Shaggar posted:

wrong. intellij sucks compared to eclipse.

# ? Jul 3, 2014 17:27

Stubbs: Oct 24, 2005

Only human filth could've posted:

wrong. intellij sucks compared to eclipse.

# ? Jul 3, 2014 20:08

Nomnom Cookie: Aug 30, 2009

AWWNAW posted:

i had to do a pipeline of services that had to process poo poo linearly and i just stuffed relevant info into the messages that allowed each subscriber/service in the pipeline to subsequently queue the next message lol

we put the worker code in the message so the nodejs worker processes look like this

code:

while (message = getMessage()) {
    eval(message);
}

simple. extensible. elegant

# ? Jul 3, 2014 22:47

MononcQc: May 29, 2007

That looks absurdly risky whenever the code you eval depends on versions of libraries or runtimes that aren't there or uniform around the cluster (say mid-deploy).

# ? Jul 3, 2014 23:34

Nomnom Cookie: Aug 30, 2009

but it's elegant and minimal. not pictured: 250k lines of shell, go, and erlang primarily written to compensate for shortcomings of the continuation passing queue

# ? Jul 3, 2014 23:59

MononcQc: May 29, 2007

Sounds like what you implemented is a scheduler queue that would be provided to you if you had a language with proper support for concurrency via green threads in it already or something.

# ? Jul 4, 2014 00:01

Nomnom Cookie: Aug 30, 2009

nodejs has green threads its called nonblocking. read up and get on my level

# ? Jul 4, 2014 00:04

AWWNAW: Dec 30, 2008

one thing i was looking into at old job was breaking down big chunky process into smaller units, say instead of queuing one message to do thing to document, queue one message per page of document; so that big docs get distributed across subscribers and processed faster. i couldn't figure out any good way to know when the document would be done processing without having some sort of "supervisor" checking to see that all messages got processed before queuing next step. was i right or was i only partially right

# ? Jul 4, 2014 00:26

Shaggar: Apr 26, 2006

MononcQc posted:

That looks absurdly risky whenever the code you eval depends on versions of libraries or runtimes that aren't there or uniform around the cluster (say mid-deploy).

yeah its javascript alright

# ? Jul 4, 2014 00:27

syntaxrigger: Jul 7, 2011; Actually you owe me 6! But who's countin?

Shaggar posted:

wrong. intellij sucks compared to eclipse.

Squinty Applebottom posted:

p-lang stymie

i don't even have to use intellij to know this is wrong

loving christ shaggar

# ? Jul 4, 2014 00:39

MononcQc: May 29, 2007

AWWNAW posted:

one thing i was looking into at old job was breaking down big chunky process into smaller units, say instead of queuing one message to do thing to document, queue one message per page of document; so that big docs get distributed across subscribers and processed faster. i couldn't figure out any good way to know when the document would be done processing without having some sort of "supervisor" checking to see that all messages got processed before queuing next step. was i right or was i only partially right

Use the blockchain.

# ? Jul 4, 2014 00:50

MononcQc: May 29, 2007

More seriously though, it depends if what you want is to know when you're done, which is easier to distribute, or if what you want to know is if it's done OR still processing, which is a lot harder.

For wanting to know when you're done, you still need cooperation, but it can be as simple than incrementing a counter somewhere and knowing the expected total amount. When the value is full, you're done. You can then start wondering about "but what if I have duplicate pieces of work?!" and other shenanigans, where instead of just counters, you may need to start handing over work tokens that must be checked in.

The thing with these is that you don't need a proper supervisor: each process, upon checking in, can verify whether value is complete, and if so, send a signal about it.

The downside of that kind of approach is that you have no way to know if pieces have been lost -- if one process goes away, the canvas is never completed and nobody rings the alarm.

Having a supervisor to look at progress becomes useful at that point. That supervisor or agent sitting somewhere can look at pieces of data, and merge them on your behalf, re-scheduled failed pieces, and so on.

When you're playing into that realm, you're coming dangerously close to reimplementing an ad-hoc map-reduce, and looking at existing solutions (whether it's Hadoop, Disco, or whatever) becomes more interesting.

# ? Jul 4, 2014 00:50

MORE CURLY FRIES: Apr 8, 2004

AWWNAW posted:

one thing i was looking into at old job was breaking down big chunky process into smaller units, say instead of queuing one message to do thing to document, queue one message per page of document; so that big docs get distributed across subscribers and processed faster. i couldn't figure out any good way to know when the document would be done processing without having some sort of "supervisor" checking to see that all messages got processed before queuing next step. was i right or was i only partially right

create everything as a batch with a batch header and batch details and you know a batch is done when the details all say theyre done

you dont really need a supervisor because each child could interrogate its siblings to know whether it was the last piece

# ? Jul 4, 2014 00:52

MononcQc: May 29, 2007

MononcQc posted:

More seriously though, it depends if what you want is to know when you're done, which is easier to distribute, or if what you want to know is if it's done OR still processing, which is a lot harder.

For wanting to know when you're done, you still need cooperation, but it can be as simple than incrementing a counter somewhere and knowing the expected total amount. When the value is full, you're done. You can then start wondering about "but what if I have duplicate pieces of work?!" and other shenanigans, where instead of just counters, you may need to start handing over work tokens that must be checked in.

The thing with these is that you don't need a proper supervisor: each process, upon checking in, can verify whether value is complete, and if so, send a signal about it.

The downside of that kind of approach is that you have no way to know if pieces have been lost -- if one process goes away, the canvas is never completed and nobody rings the alarm.

Having a supervisor to look at progress becomes useful at that point. That supervisor or agent sitting somewhere can look at pieces of data, and merge them on your behalf, re-scheduled failed pieces, and so on.

When you're playing into that realm, you're coming dangerously close to reimplementing an ad-hoc map-reduce, and looking at existing solutions (whether it's Hadoop, Disco, or whatever) becomes more interesting.

storm is designed for handling this sort of poo poo too, and it's realtime

# ? Jul 4, 2014 02:05

AWWNAW: Dec 30, 2008

MononcQc posted:

The thing with these is that you don't need a proper supervisor: each process, upon checking in, can verify whether value is complete, and if so, send a signal about it.

this seemed like the easiest way to do what i wanted but something felt wrong about checking completeness once after each unit of work, potentially thousands of times for one document, and it also didn't cover the case when one of the operations failed

oh well gently caress it, on to the next job

# ? Jul 4, 2014 03:30

titaniumone: Jun 10, 2001

http://www.php.net/manual/en/datetimeimmutable.modify.php

lol

# ? Jul 4, 2014 15:38

Tiny Bug Child: Sep 11, 2004; Avoid Symmetry, Allow Complexity, Introduce Terror

titaniumone posted:

http://www.php.net/manual/en/datetimeimmutable.modify.php

lol

it doesn't modify the datetimeimmutable it just makes a new one.

code:

$dti = new DateTimeImmutable('january 1 1990');
$dt2 = $dti->modify('+1 month');

var_dump($dti->format('Y-m-d'));
var_dump($dt2->format('Y-m-d'));

code:

string(10) "1990-01-01"
string(10) "1990-02-01"

# ? Jul 4, 2014 15:45

titaniumone: Jun 10, 2001

Tiny Bug Child posted:

it doesn't modify the datetimeimmutable it just makes a new one.

code:

$dti = new DateTimeImmutable('january 1 1990');
$dt2 = $dti->modify('+1 month');

var_dump($dti->format('Y-m-d'));
var_dump($dt2->format('Y-m-d'));

code:

string(10) "1990-01-01"
string(10) "1990-02-01"

then it should be called "clone"

a class named "...Immutable" with a method called "modify" is idiocy

# ? Jul 4, 2014 15:59

Tiny Bug Child: Sep 11, 2004; Avoid Symmetry, Allow Complexity, Introduce Terror

no, because then the interface would be different than DateTime's. it's obvious what it does if you spend like a second looking at the documentation

# ? Jul 4, 2014 16:01

titaniumone: Jun 10, 2001

yea im really out of date on my php

thank god

# ? Jul 4, 2014 16:06

Bloody: Mar 3, 2013

that isnt really all that bothersome, sorry. you've done a poor job of pointing out how bad php is.

# ? Jul 4, 2014 16:16

raminasi: Jan 25, 2005; a last drink with no ice

tbc what causes it to return false

# ? Jul 4, 2014 16:20

Tiny Bug Child: Sep 11, 2004; Avoid Symmetry, Allow Complexity, Introduce Terror

failure

# ? Jul 4, 2014 16:25

Damiya: Jul 3, 2012

Tiny Bug Child posted:

failure

dsyp

# ? Jul 4, 2014 16:29

karms: Jan 22, 2006; by Nyc_Tattoo; Yam Slacker

titaniumone posted:

then it should be called "clone"

a class named "...Immutable" with a method called "modify" is idiocy

idiocy!! yeah no

# ? Jul 4, 2014 16:30

Notorious b.s.d.: Jan 25, 2003; by Reene

Kevin Mitnick P.E. posted:

really that's what everything coming from typesafe is: a tech demo

# ? Jul 4, 2014 16:35

Notorious b.s.d.: Jan 25, 2003; by Reene

Malcolm XML posted:

like many things bsd is only correct in the smallest sense; ivy leagues tend to prefer fulltime bachelors because of ~*reasons*~ mostly relating to "preserving the experience" but are totes cool with non-traditional backgrounds if you can justify them

in other words, the ivy leagues are not particularly welcoming to non-trad students.

you won't have to "justify" poo poo to your local state school extension campus. they don't care about some tree-lined brick building ideal undergrad experience and they don't have to sell that to alums debating what to do with millionaire brats

# ? Jul 4, 2014 16:37

Adbot: ADBOT LOVES YOU

# ? Jun 2, 2024 15:34

Notorious b.s.d.: Jan 25, 2003; by Reene

Mr Dog posted:

9p is also dumb as poo poo. Time and time again people decide that if you could just transparently marshall function calls and byte streams then you can magically sprinkle this pixie dust onto a non-distributed application to turn it into a distributed one. Marshalling is, like, the least difficult part of making a distributed system. Cost models for caching and buffering getting thrown completely the gently caress out of whack because your latency and throughput characteristics just got shifted by a few orders of magnitude would be the second least difficult part of making a distributed system to deal with, but nobody ever even thinks that far ahead. So you have borderline unusable abominations like network filesystems or X11.

these systems had much more modest goals for their distributed systems. your dumb as gently caress display terminal would call on one or more compute servers and one or more disk servers. this was feasible because the network was very nearly as fast as your system bus

x11 and nfs made a lot of sense in a world where your network connection was 25% or 50% or 150% of the speed of your path to memory, but compute resources were expensive

(teeny problem: that world only existed for like 18 months in the early 1980s)

# ? Jul 4, 2014 16:39

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »