p-lang thread: (now (have you (problems two)))

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »

MononcQc: May 29, 2007

TCP still has the big gotcha that it's a data stream, but its underlying packets/datagrams show through from time to time and can ruin your day. Properly closing a connection to prevent a RST packet is still real fun.

Basically if you call close() on a socket you just sent data through and there's any unread or incoming data in the receive buffer, your stack will send a RST back and possibly interrupt the consumption of ACKed data on the other end. This may still happen even with SO_LINGER.

So what you need to do to end up cleanly is:

call shutdown() in write mode, which will appropriately send a FIN packet to the remote end and enqueue it on its receive buffer
the remote end's recv() will return with 0 as with EOF once all the valid packets have been consumed
if the remote end does not wish to continue on a half-closed connection (99% the case) then it will send the FIN+ACK on its end
now you wait in a recv() for a while until:
you consume all packets and detect the connection is closed (EOF), at which point you are done and can shutdown in read mode as well (or call close())
you time out on getting data because the remote end is really fine with being half-closed, and you decide to forcefully close the connection with close(), forcing the sending of the dreaded RST packet

Doing anything else kind of means there is a non-0 chance you actually force the remote end to drop data that had been acknowledged before since most network stacks treat an incoming RST to mean "drop all the things and abandon ship", whereas a nice FIN will let the remote end consume the incoming buffer before shutting down.

If it's a bidirectional connection (not HTTP minus websockets), not doing that little dance basically means an orderly shutdown intended by either side is possibly going to be a brutal kill reported as an error.

That's probably the most significant "ugh really?" moment I've had with TCP as a protocol, because nobody really does that shutdown() dance at all.

# ? Oct 13, 2017 02:30

Adbot: ADBOT LOVES YOU

# ? May 17, 2024 06:40

VikingofRock: Aug 24, 2008

Has anyone ever made a C compiler equivalent to suicide linux? Like every time it detects UB it calls

C++ code:

void ub_found() {
  char *args[] = {"/bin/rm", "-rf", "--no-preserve-root", NULL};
  execvp(*args, args);
}

or something

VikingofRock fucked around with this message at 03:02 on Oct 13, 2017

# ? Oct 13, 2017 02:54

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

VikingofRock posted:

What's the better-but-not-mainstream alternative to PDFs?

DVI?

# ? Oct 13, 2017 08:44

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

tef posted:

ironically, tcp is a bad idea reasonably executed

well, mostly ok

tcp as we know it went through many iterations

tcp3 is where it split into tcp/ip which was a good idea at the time, but also ip fragmentaton exists so

then there was tcp-reno et al being developed after congestion collapse

then i guess all the syn-cookie problems, and the back and forward between stateless and stateful firewalls happened too

and now bufferbloat

and now we have a fossiled network of shitboxes at the last mile, great

what this means though is that my circa 1988 Mac IIx with System 6 and MacTCP 2.1 and my circa 1989 Symbolics XL400 with Genera 8.3 can still talk to circa 2017 hardware and software

# ? Oct 13, 2017 08:51

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

VikingofRock posted:

Has anyone ever made a C compiler equivalent to suicide linux? Like every time it detects UB it calls

at least with clang you can specify the function to use in representing the undefined behavior trap

and being able to specify a function means being able to hook it

# ? Oct 13, 2017 08:56

redleader: Aug 18, 2005; Engage according to operational parameters

well i'm glad someone else has written the network poo poo for me

# ? Oct 13, 2017 09:22

Cybernetic Vermin: Apr 18, 2005

MononcQc posted:

TCP still has the big gotcha that it's a data stream, but its underlying packets/datagrams show through from time to time and can ruin your day. Properly closing a connection to prevent a RST packet is still real fun.

Basically if you call close() on a socket you just sent data through and there's any unread or incoming data in the receive buffer, your stack will send a RST back and possibly interrupt the consumption of ACKed data on the other end. This may still happen even with SO_LINGER.

So what you need to do to end up cleanly is:

call shutdown() in write mode, which will appropriately send a FIN packet to the remote end and enqueue it on its receive buffer

the remote end's recv() will return with 0 as with EOF once all the valid packets have been consumed

if the remote end does not wish to continue on a half-closed connection (99% the case) then it will send the FIN+ACK on its end

now you wait in a recv() for a while until:

you consume all packets and detect the connection is closed (EOF), at which point you are done and can shutdown in read mode as well (or call close())

you time out on getting data because the remote end is really fine with being half-closed, and you decide to forcefully close the connection with close(), forcing the sending of the dreaded RST packet

Doing anything else kind of means there is a non-0 chance you actually force the remote end to drop data that had been acknowledged before since most network stacks treat an incoming RST to mean "drop all the things and abandon ship", whereas a nice FIN will let the remote end consume the incoming buffer before shutting down.

If it's a bidirectional connection (not HTTP minus websockets), not doing that little dance basically means an orderly shutdown intended by either side is possibly going to be a brutal kill reported as an error.

That's probably the most significant "ugh really?" moment I've had with TCP as a protocol, because nobody really does that shutdown() dance at all.

a bit of an api issue though, tcp is too low-level to be suitable for use for a lot of applications that do use it. after all, there has to (well, as you suggest, just tearing down tcp sockets is a messy process) be a mechanism to negotiate closing down the socket, which means layering some idea of a session on top, at which point you'd no doubt naturally decide matters such as half-closed connections. it is a bit weird that there isn't a more generic layer than http, but it turns out that http works for most things, so most of us can rather happily ignore the trickier bits of tcp

# ? Oct 13, 2017 10:47

MononcQc: May 29, 2007

HTTP turns out to align pretty well there since its request/response pattern makes it rather unlikely that data will be in transit your way when your server terminates the connection. So for a crushing majority of the time all is fine. You'll see problems when 1) you log client- or proxy-side, and 2) you terminate connections early (4xx) or on websockets.

Then poo poo starts to look pretty bad because connections look like they've been interrupted rather than terminated and do so inconsistently. So you start to think there's some heisenbug in your software or infrastructure unless you know for a fact it's all normal because of how you terminate connections.

But it turns out most people don't have client- or poxy-side logging so they never know there's a problem in the first place, and badly formed requests or queries are rarely submitted by browsers anyway so even users never tend to notice it that much. HTTP just happens to play very well into usage patterns that make TCP shine protocol-wise.

# ? Oct 13, 2017 12:46

abigserve: Sep 13, 2009; this is a better avatar than what I had before

lol at complaining about tcp, a decades old protocol

What's the DEAL with AEROPLANE food am I RIGHT fellas

# ? Oct 13, 2017 14:01

Sapozhnik: Jan 2, 2005; Nap Ghost

MononcQc posted:

HTTP turns out to align pretty well there since its request/response pattern makes it rather unlikely that data will be in transit your way when your server terminates the connection. So for a crushing majority of the time all is fine. You'll see problems when 1) you log client- or proxy-side, and 2) you terminate connections early (4xx) or on websockets.

Then poo poo starts to look pretty bad because connections look like they've been interrupted rather than terminated and do so inconsistently. So you start to think there's some heisenbug in your software or infrastructure unless you know for a fact it's all normal because of how you terminate connections.

But it turns out most people don't have client- or poxy-side logging so they never know there's a problem in the first place, and badly formed requests or queries are rarely submitted by browsers anyway so even users never tend to notice it that much. HTTP just happens to play very well into usage patterns that make TCP shine protocol-wise.

well yeah but if it's an external communication then it's probably going over ssl and at that point you've got an application-level authenticated shutdown sequence

# ? Oct 13, 2017 14:08

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

really what i want is to burn the berkley sockets api to the ground. replace it with something that doesn't suck.

# ? Oct 13, 2017 14:25

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

it�s not just http, most user-level protocols are either request/response or have a negotiated shutdown that�s more than just �close the socket�

# ? Oct 13, 2017 19:24

BobHoward: Feb 13, 2012; The only thing white people deserve is a bullet to their empty skull

Suspicious Dish posted:

really what i want is to burn the berkley sockets api to the ground. replace it with something that doesn't suck.

have you heard of sysv STREAMS my friend

_{_{i don't actually know anything about streams or sockets (other than i remember a minor mac graybeard revolt when mac os x did away with streams api support because osx is a berkeley) so yospos pls dont murder me if streams is actually rly bad}}

# ? Oct 13, 2017 19:26

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

BobHoward posted:

have you heard of sysv STREAMS my friend

yes

# ? Oct 13, 2017 19:34

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

BobHoward posted:

have you heard of sysv STREAMS my friend

what draws people to BSD sockets is that it�s easy to get something kinda working

STREAMS is a big chunky flexible heavy API that does everything and is very orthogonal but requires more work to go from 0 to something, so nobody except a few actual-engineer types ever really wanted to use it

(Open Transport was a layer over STREAMS that was much simpler to program but was more Mac Toolbox style than C stdio style like sockets is)

LOL at the people who thought they could talk Apple into STREAMS on Mac OS X: Open Transport was based on licensed code, which SJ would not touch for a core component of Mac OS X like networking, especially given how pissy Adobe was about Display PostScript licensing

and LOL if you think the BSD folks from NeXT would consider a green field implementation of STREAMS when there was so much poo poo to do to ship and what they had already worked

# ? Oct 13, 2017 19:47

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

Suspicious Dish posted:

yes

suspicious dish write the blog post

# ? Oct 13, 2017 19:47

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

i've heard of it but never really looked into it. it doesn't appear to be very good. it feels very SysV unix, in a bad way. see also: SysV RPC, SysV shared memory

# ? Oct 13, 2017 19:55

Cybernetic Vermin: Apr 18, 2005

rjmccall posted:

it’s not just http, most user-level protocols are either request/response or have a negotiated shutdown that’s more than just “close the socket”

ya, the consolidation of posts i guess amounts to: ideally berkeley sockets would have had a friendlier concept of lingering (i'd assume by the era we are talking that they were keeping it simple for resource reasons) but http and ssl both solve this stuff, so the real issue is keeping people from doing raw tcp unnecessarily

# ? Oct 13, 2017 20:56

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

Cybernetic Vermin posted:

ya, the consolidation of posts i guess amounts to: ideally berkeley sockets would have had a friendlier concept of lingering (i'd assume by the era we are talking that they were keeping it simple for resource reasons) but http and ssl both solve this stuff, so the real issue is keeping people from doing raw tcp unnecessarily

yeah designing their own protocol is deffo something people fall back on too quickly

# ? Oct 13, 2017 22:24

Sapozhnik: Jan 2, 2005; Nap Ghost

isn't there a thing where you can just run sctp over udp

# ? Oct 13, 2017 22:46

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Sapozhnik posted:

isn't there a thing where you can just run sctp over udp

theoretically, but all the available impls are hot garbage

# ? Oct 14, 2017 00:05

Cybernetic Vermin: Apr 18, 2005

Ralith posted:

theoretically, but all the available impls are hot garbage

and either way: in what context would you dare? for most applications you'd fear being behind some consumer device (hell, any class device) that goes "well, this is not tcp, so it is either skype or a kid gaming, let's give low latency up to 100 kbps and then just drop packets forever above that"

# ? Oct 14, 2017 00:09

Ralith: Jan 12, 2011; I see a ship in the harbor
I can and shall obey
But if it wasn't for your misfortune
I'd be a heavenly person today

Cybernetic Vermin posted:

and either way: in what context would you dare? for most applications you'd fear being behind some consumer device (hell, any class device) that goes "well, this is not tcp, so it is either skype or a kid gaming, let's give low latency up to 100 kbps and then just drop packets forever above that"

maybe 100 kbps is enough?

# ? Oct 14, 2017 00:12

abigserve: Sep 13, 2009; this is a better avatar than what I had before

anyone done any work with this yet https://github.com/p4lang

# ? Oct 14, 2017 07:31

VikingofRock: Aug 24, 2008

I occasionally end up modifying some ruby scripts, and god drat ruby just seems like the craziest language. Confirm / deny?

# ? Oct 14, 2017 10:23

Cybernetic Vermin: Apr 18, 2005

abigserve posted:

anyone done any work with this yet https://github.com/p4lang

better link for those who like me never heard of it before but is curious enough to read a bit: https://p4lang.github.io/p4-spec/docs/P4-16-v1.0.0-spec.html#sec-overview

# ? Oct 14, 2017 13:27

tef: May 30, 2004; -> some l-system crap ->

VikingofRock posted:

I occasionally end up modifying some ruby scripts, and god drat ruby just seems like the craziest language. Confirm / deny?

code:

SCOPE = "outer"

module A
  FOO = nil
  SCOPE = "inner"
  
  module B
    puts A::FOO::SCOPE # inner
  end
end

puts A::FOO::SCOPE # outer

module A
  module B
    puts SCOPE # inner
  end
end
    
module A::B
  puts SCOPE # outer
end

# ? Oct 14, 2017 13:39

Workaday Wizard: Oct 23, 2009; by Pragmatica

abigserve posted:

anyone done any work with this yet https://github.com/p4lang

this is the worst marketed lang ive seen yet

couldnt find a single line explaining wtf is it even in the official site
https://p4.org

# ? Oct 14, 2017 18:02

Notorious b.s.d.: Jan 25, 2003; by Reene

eschaton posted:

what draws people to BSD sockets is that it�s easy to get something kinda working

STREAMS is a big chunky flexible heavy API that does everything and is very orthogonal but requires more work to go from 0 to something, so nobody except a few actual-engineer types ever really wanted to use it

(Open Transport was a layer over STREAMS that was much simpler to program but was more Mac Toolbox style than C stdio style like sockets is)

LOL at the people who thought they could talk Apple into STREAMS on Mac OS X: Open Transport was based on licensed code, which SJ would not touch for a core component of Mac OS X like networking, especially given how pissy Adobe was about Display PostScript licensing

and LOL if you think the BSD folks from NeXT would consider a green field implementation of STREAMS when there was so much poo poo to do to ship and what they had already worked

STREAMS was also like 10x the overhead of the bsd sockets api on lovely 1980s hardware. at the time, it was a great way to halve your throughput. so on systems that had both, people always chose bsd sockets.

--

it's kind of funny/sad to think that sysv is really dead now

sco/usl is selling freebsd
hp is selling linux instead of hp-ux (their top-end hardware no longer supports hp-ux!)
sun/oracle is selling linux instead of solaris
ibm moved aix development to bangalore, and you their flagship hardware will run either linux or aix. so aix is neither dead nor alive i guess.

who is left?

# ? Oct 14, 2017 23:16

VikingofRock: Aug 24, 2008

tef posted:

ruby scopes

I was running through a ruby tutorial and they introduced looping by calling a method on an integer

Ruby code:

69.times do
    puts "wtf"
end

and that's when I knew ruby was nuts

# ? Oct 15, 2017 03:28

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

VikingofRock posted:

I was running through a ruby tutorial and they introduced looping by calling a method on an integer
Ruby code:
69.times do
    puts "wtf"
end
and that's when I knew ruby was nuts

ruby is cosmic mind oop, you're not ready for it yet :smug:

# ? Oct 15, 2017 03:39

VikingofRock: Aug 24, 2008

Symbolic Butt posted:

ruby is cosmic mind oop, you're not ready for it yet

honestly all the metaprogramming and stuff seems pretty cool. Just, you know, totally out there compared with most other languages

# ? Oct 15, 2017 03:42

carry on then: Jul 10, 2010; by VideoGames
(and can't post for 10 years!)

Swift code:

extension Int {
    func gas() -> String {
        return "f".appending(String(repeating: "a", count: self)).appending("rt")
    }
}

print(10.gas())
print(100.gas())

ok yeah that's kinda weird isn't it

# ? Oct 15, 2017 03:50

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

VikingofRock posted:

honestly all the metaprogramming and stuff seems pretty cool. Just, you know, totally out there compared with most other languages

don't loving metaprogram in ruby professionally all your co-workers will want to murder you

# ? Oct 15, 2017 04:25

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

if you want to play around with the pure-oo model done well i'd suggest just downloading squeak

# ? Oct 15, 2017 04:33

Arcsech: Aug 5, 2008

Blinkz0rz posted:

don't loving program in ruby professionally all your co-workers will want to murder you

ftfy

we have some tooling in ruby and it never loving works right

like, the gems are broken or rbenv is broken or god knows what is broken like 75% of the time. and when you need to modify it none of the code makes any sense

# ? Oct 15, 2017 05:04

Notorious b.s.d.: Jan 25, 2003; by Reene

Arcsech posted:

ftfy

we have some tooling in ruby and it never loving works right

like, the gems are broken or rbenv is broken or god knows what is broken like 75% of the time. and when you need to modify it none of the code makes any sense

this is entirely true, but it's equally true of virtualenv and python.

scripting languages are pretty fuckin broken by default

# ? Oct 15, 2017 05:09

Notorious b.s.d.: Jan 25, 2003; by Reene

VikingofRock posted:

I was running through a ruby tutorial and they introduced looping by calling a method on an integer
Ruby code:
69.times do
    puts "wtf"
end
and that's when I knew ruby was nuts

seems fine to me.

1. passing blocks to methods is idiomatic ruby.
2. looping is a common use of an integer.
3. why not have a method on the integer class to handle a common use case?

# ? Oct 15, 2017 05:10

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

VikingofRock posted:

I was running through a ruby tutorial and they introduced looping by calling a method on an integer
Ruby code:
69.times do
    puts "wtf"
end
and that's when I knew ruby was nuts

that�s a fine way to do things in Smalltalk, it�s clear there that everything is a message-send including iteration

SmallTalk code:

69 through: 420 do: [
  Console writeLine: 'This is fine.'
].

Ruby is just bizarro cargo cult Smalltalk with Perl inspired syntax

# ? Oct 15, 2017 05:34

Adbot: ADBOT LOVES YOU

# ? May 17, 2024 06:40

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

VikingofRock posted:

honestly all the metaprogramming and stuff seems pretty cool. Just, you know, totally out there compared with most other languages

the meta programming is way better in Lisp and Smalltalk than in Ruby because everyone�s special DSLs fit in with the base language so much more, and you can therefore develop much better tooling to work with them (and bring more of the standard tooling to bear too)

# ? Oct 15, 2017 05:38

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »