p-lang thread: (now (have you (problems two)))

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > p-lang thread: (now (have you (problems two)))

«‹›1784 »

redleader: Aug 18, 2005; Engage according to operational parameters

Cybernetic Vermin posted:

succ objects

pls txt me

# ? Apr 5, 2018 07:03

Adbot: ADBOT LOVES YOU

# ? Jun 4, 2024 14:17

MononcQc: May 29, 2007

I know some of my blog posts in the past had decent reception in here so here's a long talk transcript about system design and dealing with unexpected poo poo with a focus on erlang/elixir for maybe half of the thing https://ferd.ca/the-hitchhiker-s-guide-to-the-unexpected.html

# ? Apr 5, 2018 18:29

suffix: Jul 27, 2013; Wheeee!

MononcQc posted:

I know some of my blog posts in the past had decent reception in here so here's a long talk transcript about system design and dealing with unexpected poo poo with a focus on erlang/elixir for maybe half of the thing https://ferd.ca/the-hitchhiker-s-guide-to-the-unexpected.html

subtle

does anyone do this supervisor hierarchy style outside the erlang ecosystem? it sounds good but i tend to tune out a bit because i'm unlikely to ever use it

the bit on the end about microservices is interesting, would love to see a viable approach
in kubernetes we've found that over-eager liveness probes are a great way to turn a degradation into a complete outage,
e.g. checking if the service can handle a query sounds like a reasonable health check, but then the database has a hiccup and your entire cluster goes down in restart cascades
so now our liveness probes are a ping check, if anything, and the service-level health checks are only used for monitoring and alerting

feels like the trap in dealing with unknown errors is "oops we made it worse" - like if your errors can be load related, but your solution adds load by restarting heavy components, clearing caches, lots of extra logging, or spamming retried requests

# ? Apr 5, 2018 21:13

Arcsech: Aug 5, 2008

suffix posted:

does anyone do this supervisor hierarchy style outside the erlang ecosystem? it sounds good but i tend to tune out a bit because i'm unlikely to ever use it

akka does it on the jvm

# ? Apr 5, 2018 23:22

MononcQc: May 29, 2007

suffix posted:

subtle

does anyone do this supervisor hierarchy style outside the erlang ecosystem? it sounds good but i tend to tune out a bit because i'm unlikely to ever use it

Elixir and Akka (both for Scala and Akka.NET for C# and F#) come to mind.

You usually need: well-isolated units of processing (actors or processes), a way to detect faults between them (links and/or monitors), and a way for each unit of processing to start other ones. If you have that, you can duct tape your own system in place with various degrees of effectiveness (i.e. Go would have a hard time since goroutines share whatever memory they want making it tricky to effectively restart units of work without messing memory up, and there is no great way to detect the faults of goroutines)

suffix posted:

the bit on the end about microservices is interesting, would love to see a viable approach
in kubernetes we've found that over-eager liveness probes are a great way to turn a degradation into a complete outage,
e.g. checking if the service can handle a query sounds like a reasonable health check, but then the database has a hiccup and your entire cluster goes down in restart cascades
so now our liveness probes are a ping check, if anything, and the service-level health checks are only used for monitoring and alerting

feels like the trap in dealing with unknown errors is "oops we made it worse" - like if your errors can be load related, but your solution adds load by restarting heavy components, clearing caches, lots of extra logging, or spamming retried requests

The better pattern for such a case is a circuit-breaker. The circuit-breaker is entirely local, and can react to patterns such as "more than 5% failures over 30 seconds means the service is unhealthy" and only then will it trip. You can then get it to stop a part of your system (or do things like go to a failover and retry from a different pool member), or at least get them to prevent taking a costly action.

For example, if you have expensive DB queries that create overload and things grind to a halt, trying more queries is likely to just enqueue more heavy work and prevent recovery. A circuit breaker here should be located at the DB client's side: if the DB stops answering, fail the queries fast; don't even run them, return an error for a blown fuse. A few seconds later, let the fuse untrip itself, and try again; if performances are still bad, they re-trip and stop again. Yes, the error will cascade, but in theory (and in practice in my experience), the component causing problems will have an easier time recovering that way, and the outage should be shorter.

Usually, everything going down in a major restart cascades is due to properly detecting failures, but not being able to live with them. If you absolutely need the database to be around to function, no amount of breakers and supervision will allow you to just pretend it works. Queues and buffers can buy you some time and act as a dampener, but they can also hinder proper fault detection. You will want to monitor sojourn time within the queue/buffer as a proxy for processing rate there.

If you need the failing database for some functionality only though, then you can have a system that properly lives with partial failures. That's where Chaos engineering becomes interesting to tune the probes and circuit breakers, and to ensure the reaction is the right one (and why I have that section in the text about property-testing supervision trees).

One example I had at Heroku was one data pipeline going down and accidentally taking down the whole routing layer for the platform, which consumed the pipeline. The problem we had was that the data system dying had us shut down the node itself.

After having that incident, we made the data feed consumers 'temporary' (meaning if it crashed, it crashed, but left the rest of the system alive albeit with stale data). What would happen then on the next failure was that all the routing nodes would remain alive, we'd get paged, and there was a well-documented procedure to restart feed consumption once the outage was fixed -- I left before we found it necessary to automate retries of that subsystem.

# ? Apr 6, 2018 03:23

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

MononcQc posted:

You usually need: well-isolated units of processing (actors or processes), a way to detect faults between them (links and/or monitors), and a way for each unit of processing to start other ones. If you have that, you can duct tape your own system in place with various degrees of effectiveness (i.e. Go would have a hard time since goroutines share whatever memory they want making it tricky to effectively restart units of work without messing memory up, and there is no great way to detect the faults of goroutines)

I didn't really read your blog post because it sounded like something way outside my wheelhouse, but then I read this and maybe not?

i've built several small scale systems with celery on python and what you say in this quoted part reminds me of celery at least at a surface level

am i seeing similarities that aren't there or is celery the same sort of thing you're tlaking about?

# ? Apr 6, 2018 05:18

AWWNAW: Dec 30, 2008

Netflix has some library for this called hysterics or something

# ? Apr 6, 2018 12:46

MononcQc: May 29, 2007

Thermopyle posted:

I didn't really read your blog post because it sounded like something way outside my wheelhouse, but then I read this and maybe not?

i've built several small scale systems with celery on python and what you say in this quoted part reminds me of celery at least at a surface level

am i seeing similarities that aren't there or is celery the same sort of thing you're tlaking about?

I'm not really familiar with Celery. The thing I'm not too sure it supports is, for example, having that well-isolated unit of processing able to maintain some state, progress with it, and send or receive messages with it. I.e. see it as the difference between an AWS Lambda (just applies a function to some state) and an Orleans grains (which can load and maintain state and even be given a name between incantations, and contact other Orlean grains).

I'm not sure it'd be useful to implement supervision when your task is mostly stateless to begin with, for example, since you'd mostly be free to just retry however you want whenever you want on as many systems as you want.

AWWNAW posted:

Netflix has some library for this called hysterics or something

https://github.com/Netflix/Hystrix and yeah that seems to be in line with this

# ? Apr 6, 2018 19:49

CPColin: Sep 9, 2003; Big ol' smile.

c Groovy s: gently caress you, Groovy

code:

def test1() {
   return 1 + 2
}

println(test1()) // 3

def test2() {
   return 1 +
          2
}

println(test2()) // 3

def test3() {
   return 1
        + 2
}

println(test3()) // 1

Good thing those semicolons are optional!

# ? Apr 7, 2018 00:29

Carthag Tuek: Oct 15, 2005; Tider skal komme,
tider skal henrulle,
sl�gt skal f�lge sl�gters gang

iow, pseudocode:

pre:

';' in set('\n', '') == ༼ �ل͟� ༽

# ? Apr 7, 2018 00:38

HappyHippo: Nov 19, 2003; Do you have an Air Miles Card?

optional semicolons are the dumbest poo poo. either have them or dont

# ? Apr 7, 2018 00:40

Lutha Mahtin: Oct 10, 2010; Your brokebrain sin is absolved...go and shitpost no more!

my fav part of gradle is that one of the files you often have in a gradle-using project is a file called gradlew

"gradl ew"

# ? Apr 7, 2018 05:26

DELETE CASCADE: Oct 25, 2017; i haven't washed my penis since i jerked it to a phtotograph of george w. bush in 2003

my favorite part of gradle is never touching it and using maven instead because i'm not some anroid idiot

# ? Apr 7, 2018 05:34

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

CPColin posted:

Good thing those semicolons are optional!

fwiw swift addresses this in like four ways:

first, our parsing rule is sensitive to asymmetric whitespace. so "1+2", "1 + 2", and "1\n+ 2" are infix operators, "1\n+2" is a prefix operator, and "1+\n2" is a postfix operator. it seems complicated when you write it out but basically it matches up well with how people actually write code

second, operators have to be declared before they can be used, and most operators aren't both infix and postfix/prefix operators (this isn't special vs. what languages with hardcoded operator sets do, though)

third, you'll get a warning for something like +2 on a line by itself

fourth, you'll get another warning for unreachable code after a return statement

# ? Apr 7, 2018 05:49

DELETE CASCADE: Oct 25, 2017; i haven't washed my penis since i jerked it to a phtotograph of george w. bush in 2003

why not just use the fuckin semicolons

# ? Apr 7, 2018 05:53

CPColin: Sep 9, 2003; Big ol' smile.

rjmccall posted:

third, you'll get a warning for something like +2 on a line by itself

fourth, you'll get another warning for unreachable code after a return statement

These two are the key. I can kind of understand why, when I start the second line with +, Groovy's lexer sees the end of the first line and figures it's the end of the statement, but for the second line not to result in any kind of warning is just stupid.

Another one that keeps getting me is that formatting like one of these is an error, but the other isn't (can't remember which):

code:

if (foo
   || bar) { ... }

if (foo
   && bar) { ... }

DELETE CASCADE posted:

why not just use the fuckin semicolons

Every time I accidentally use one, Intellij whines about it! I just made it through my informal, three-month review, so it's about time to start taking control of this lovely language (by tossing it in the bin).

# ? Apr 7, 2018 05:53

Gazpacho: Jun 18, 2004; by Fluffdaddy; Slippery Tilde

more like great-el

cue shaggar telling me I "don't understand" maven's inability to do what i need it to do

Gazpacho fucked around with this message at 06:30 on Apr 7, 2018

# ? Apr 7, 2018 06:28

Fiedler: Jun 29, 2002; I, for one, welcome our new mouse overlords.

rjmccall posted:

fourth, you'll get another warning for unreachable code after a return statement

How does this conversation go?
"Hey should this obvious error be flagged as an error?"
"Nah... what if they want to have unreachable code?"

# ? Apr 7, 2018 06:49

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

there are a number of things that are obvious semantic mistakes that we've left as warnings because making them errors would prevent people from testing their code incrementally as they write it. basically if we can soundly run the code as written we'll usually let it by with a warning

# ? Apr 7, 2018 06:59

Fiedler: Jun 29, 2002; I, for one, welcome our new mouse overlords.

That's... well. Yeah, okay then.

# ? Apr 7, 2018 07:02

Xarn: Jun 26, 2015

rjmccall posted:

there are a number of things that are obvious semantic mistakes that we've left as warnings because making them errors would prevent people from testing their code incrementally as they write it. basically if we can soundly run the code as written we'll usually let it by with a warning

This is a cool and good decision. I cant count the number of times Java pissed me off when I wanted to temporarily force a certain return value (e.g. have heuristic always return 0) and it didn't let me run it until I commented out rest of the code.

OTOH optional semicolons are bad.

# ? Apr 7, 2018 07:11

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

i don't like it when languages force me to write if (true-expression-the-compiler-cant-prove-is-constant) { return; } when i just want to add an early exit while debugging something

# ? Apr 7, 2018 07:15

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

oh yeah, that's also a big deal for the unreachable-code warning specifically

of course, people can immediately turn around and subvert that by turning on warnings-as-errors

# ? Apr 7, 2018 08:00

Gazpacho: Jun 18, 2004; by Fluffdaddy; Slippery Tilde

serves you right for breaking the line before the operator tbh

# ? Apr 7, 2018 08:03

Notorious b.s.d.: Jan 25, 2003; by Reene

Lutha Mahtin posted:

my fav part of gradle is that one of the files you often have in a gradle-using project is a file called gradlew

"gradl ew"

gradle is so bad

the only thing worse is sbt

# ? Apr 7, 2018 08:03

Notorious b.s.d.: Jan 25, 2003; by Reene

rjmccall posted:

fwiw swift addresses this in like four ways:

first, our parsing rule is sensitive to asymmetric whitespace. so "1+2", "1 + 2", and "1\n+ 2" are infix operators, "1\n+2" is a prefix operator, and "1+\n2" is a postfix operator. it seems complicated when you write it out but basically it matches up well with how people actually write code

second, operators have to be declared before they can be used, and most operators aren't both infix and postfix/prefix operators (this isn't special vs. what languages with hardcoded operator sets do, though)

third, you'll get a warning for something like +2 on a line by itself

fourth, you'll get another warning for unreachable code after a return statement

this is scala-esque in its horror

more and more i am convinced infix operators are just not a good thing to generalize

# ? Apr 7, 2018 08:03

Fiedler: Jun 29, 2002; I, for one, welcome our new mouse overlords.

rjmccall posted:

of course, people can immediately turn around and ~~subvert~~ fix that by turning on warnings-as-errors

# ? Apr 7, 2018 09:14

Athas: Aug 6, 2007; fuck that joker

Warnings-as-errors is good for development and CI, but gently caress people who hardcode that in the Makefiles (or whatever) for their releases.

# ? Apr 7, 2018 09:46

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

Gazpacho posted:

more like great-el

cue shaggar telling me I "don't understand" maven's inability to do what i need it to do

no, it's "if maven can't do the thing you want, what you're trying to do is stupid and/or wrong."

# ? Apr 7, 2018 12:21

Workaday Wizard: Oct 23, 2009; by Pragmatica

Wheany posted:

no, it's "if maven can't do the thing you want, what you're trying to do is stupid and/or wrong."

i haven�t seen a good argument against this. have you?

# ? Apr 7, 2018 14:24

Asymmetrikon: Oct 30, 2009; I believe you're a big dork!

Notorious b.s.d. posted:

more and more i am convinced infix operators are just not a good thing to generalize

just get rid of operators totally and use methods like

code:

let x = y.plus(1).times(12)

so they're handled like every other thing in the language instead of special casing math

# ? Apr 7, 2018 15:27

carry on then: Jul 10, 2010; by VideoGames
(and can't post for 10 years!)

Asymmetrikon posted:

just get rid of operators totally and use methods like
code:
let x = y.plus(1).times(12)
so they're handled like every other thing in the language instead of special casing math

you mean x.setTo(y.plus(1).times(12)), right?

# ? Apr 7, 2018 15:30

comedyblissoption: Mar 15, 2006

Fiedler posted:

That's... well. Yeah, okay then.

an extreme example of interim code causing compilation errors in infuriating ways when in the middle of writing code is golang compilation erroring because you have...an unused import

# ? Apr 7, 2018 15:34

aardvaard: Mar 4, 2013; you belong in the bog of eternal stench

carry on then posted:

you mean x.setTo(y.plus(1).times(12)), right?

no, that's using the dot operator. it would end up something like (setTo x (times (plus y 1) 12))

wait a minute

# ? Apr 7, 2018 16:16

Fiedler: Jun 29, 2002; I, for one, welcome our new mouse overlords.

comedyblissoption posted:

an extreme example of interim code causing compilation errors in infuriating ways when in the middle of writing code is golang compilation erroring because you have...an unused import

I'd take that over Swift's "just guess what I meant by x\n+y" default any day.

# ? Apr 7, 2018 16:43