Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
cowboy beepboop
Feb 24, 2001

or you can use the ~advanced php cache~ which is free and somehow not part of the language

Adbot
ADBOT LOVES YOU

coaxmetal
Oct 21, 2010

I flamed me own dad
people bitch about pyc files?

Zombywuf
Mar 29, 2008

MononcQc posted:

So yes in production you'll probably want something that restarts software, reboots servers, or allows you to add or remove them, because otherwise you'll slowly end up with not enough machines to keep going. But you won't fall in the case where, say, the connection between servers falls in a bad state and a 3 hours timeout resolves it later on and all of a sudden one end thinks the other was crashed but the other just thought things were slow.

I think Paxos has the requirement that a machine that goes down can be brought back up in the same state it was in before it fell over. Faulty RAID controllers do not seem to be what it's designed to cope with.

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

Ronald Raiden posted:

people bitch about pyc files?everything

Nomnom Cookie
Aug 30, 2009



my stepdads beer posted:

or you can use the ~advanced php cache~ which is free and somehow not part of the language

its not part of the language because then zend would have a harder time selling PHP Enterprise Edition or whatever the gently caress its called

MeruFM
Jul 27, 2010

Ronald Raiden posted:

people bitch about pyc files?

If you change the name of a file in Flask, delete the previous files pyc because it'll gently caress stuff up.

MononcQc
May 29, 2007

Zombywuf posted:

I think Paxos has the requirement that a machine that goes down can be brought back up in the same state it was in before it fell over. Faulty RAID controllers do not seem to be what it's designed to cope with.

Paxos has implicit mechanisms to deal with new nodes joining the quorum (you can use Paxos to configure Paxos), where it can play catch-up. This was covered in the Google paper about using it in Chubby:

quote:

5.1 Handling disk corruption

Replicas witness disk corruption from time to time. A disk may be corrupted due to a media failure or due to an operator error (an operator may accidentally erase critical data). When a replica’s disk is corrupted and it loses its persistent state, it may renege on promises it has made to other replicas in the past. This violates a key assumption in the Paxos algorithm. We use the following mechanism to address this problem [14].

Disk corruptions manifest themselves in two ways. Either file(s) contents may change or file(s) may become inaccessible. To detect the former, we store the checksum of the contents of each file in the file2. The latter may be indistinguishable from a new replica with an empty disk – we detect this case by having a new replica leave a marker in GFS after start-up. If this replica ever starts again with an empty disk, it will discover the GFS marker and indicate that it has a corrupted disk.

A replica with a corrupted disk rebuilds its state as follows. It participates in Paxos as a non-voting member; meaning that it uses the catch-up mechanism to catch up but does not respond with promise or acknowledgment messages. It remains in this state until it observes one complete instance of Paxos that was started after the replica started rebuilding its state. By waiting for the extra instance of Paxos, we ensure that this replica could not have reneged on an earlier promise.

This mechanism enables the following optimization to improve the latency of the system. Since the system can now deal with occasional disk corruption, under some circumstances it may be acceptable not to flush writes to disk immediately3. While we have considered schemes to exploit this observation, we have not implemented them yet

The catch-up mechanism exists because nodes in multi-paxos may lag behind:

quote:

Each replica maintains a locally persistent log to record all Paxos actions. When a replica crashes and subsequently recovers, it replays the persistent log to reconstruct its state prior to crashing. Replicas also use this log when helping lagging replicas to catch up.

The mechanisms for this are not in the google paper, but something as simple as submitting a no-op write for an existing entry (I no-op write entry 1, members of the quorum should answer back with the value that was accepted already) are implicit to the protocol. This is likely slow as you may need to replay the entire log history, but Paxos doesn't directly care about speed.

Optimizations would likely have you handing off logs or a snapshot from another member on a side-channel, but I don't recall that stuff in either of the original papers (the most that is mentioned from memory is keeping a quick log of most recent transactions for network blips, not full on corruption).

Zombywuf
Mar 29, 2008

MononcQc posted:

Paxos has implicit mechanisms to deal with new nodes joining the quorum (you can use Paxos to configure Paxos), where it can play catch-up. This was covered in the Google paper about using it in Chubby:

Well this is the sort of buttressing I was talking about. It seems you need some sort of reliable database underlying Paxos to make it work operationally.

MononcQc
May 29, 2007

Yeah, it assumes reliable storage is there, can deal with failures if it can detect them locally, but does not necessarily need persistent storage. It's not always necessary if you decide to go for the crash-stop model: once the node knows it crashed, it wipes its state clean and recovers from peers. That's what Scalaris does, although I couldn't say how well it fares in production given how few people use it.

double sulk
Jul 2, 2010

@ID_AA_Carmack: The lack of strong typing in lisp is freaking me out, even in tiny programs. Writing an industrial program like this seems... unsound.

MORE CURLY FRIES
Apr 8, 2004

gucci void main posted:

@ID_AA_Carmack: The lack of strong typing in lisp is freaking me out, even in tiny programs. Writing an industrial program like this seems... unsound.

strong typing is the programming equivalent of big government

therefore carmack should be all over loose typing

MORE CURLY FRIES
Apr 8, 2004

the invisible bit of the loose type

Squinty Applebottom
Jan 1, 2013

youre mother seems to be a loose type

weird
Jun 4, 2012

by zen death robot

gucci void main posted:

@ID_AA_Carmack: The lack of strong typing in lisp is freaking me out, even in tiny programs. Writing an industrial program like this seems... unsound.

Lisp is strongly typed tho

JewKiller 3000
Nov 28, 2006

by Lowtax

VanillaKid posted:

Lisp is strongly typed tho

dynamic typing doesn't really count as typing

Deus Rex
Mar 5, 2005

Zombywuf posted:

buttressing

JewKiller 3000 posted:

dynamic typing doesn't really count as typing

yes it does. weak typing doesn't count tho

Posting Principle
Dec 10, 2011

by Ralp
scons seems like a nice build system if you're the kind of person who lives in a nightmare where you have to think about build systems. thats my programming story for the day

Brain Candy
May 18, 2006

http://www.electric-cloud.com/blog/2010/03/08/how-scalable-is-scons/

:toot:

JewKiller 3000
Nov 28, 2006

by Lowtax

Deus Rex posted:

yes it does. weak typing doesn't count tho

enjoy your p-lang hellscape

Deus Rex
Mar 5, 2005

JewKiller 3000 posted:

enjoy your p-lang hellscape

i didn't say it was goodor that i liked it, just that it counts as typing, which is true

Posting Principle
Dec 10, 2011

by Ralp

Decider('time stamp-match') fixed the worst thing I encountered today, but that was only ~3600 files :shrug:

Brain Candy
May 18, 2006

it's more of a ding on the maturity of scons. old mistakes are new mistakes are old mistakes. you fixed your issue by invoking 'do it like make, goddamn it'

the best thing about scons is that you can write things in python. the worst thing about scons is that you have to write things in python.

coaxmetal
Oct 21, 2010

I flamed me own dad
you can still have type errors in a dynamically typed language, its got typing

MeruFM
Jul 27, 2010
today, i conceptualized a threaded process and it worked as expected on the 1st pass

this what god is like being like is

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
but what about the 2nd 3rd 4th and 5th passes

Janitor Prime
Jan 22, 2004

PC LOAD LETTER

What da fuck does that mean

Fun Shoe

Suspicious Dish posted:

but what about the 2nd 3rd 4th and 5th passes

coaxmetal
Oct 21, 2010

I flamed me own dad

Suspicious Dish posted:

but what about the 2nd 4th 3rd and 5th passes

FamDav
Mar 29, 2008
if were talking about things we did that surprised us i was testing out a haskell memoization library today and its like wow i can just write out the recurrence relation i came up with and thats it?

felt good

ultramiraculous
Nov 12, 2003

"No..."
Grimey Drawer

MeruFM posted:

today, i conceptualized a threaded process and it worked as expected on the 1st pass

this what god is like being like is

what about when the device decides to check its email at the same time and the lead horse loses the race?

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

electric cloud makes some pretty impressive build software; they're really good at what they do (they're also really expensive)

tef
May 30, 2004

-> some l-system crap ->

Suspicious Dish posted:

but what about the 3rd 5th 4th and 2nd passes

coaxmetal
Oct 21, 2010

I flamed me own dad
I made that joke first tef :colbert:

PleasingFungus
Oct 10, 2012
idiot asshole bitch who should fuck off

Ronald Raiden posted:

I made that joke first tef :colbert:

if only we had some synchronization mechanism in place

then this tragic joke-after-joke incident would never have occurred.

Jonnty
Aug 2, 2007

The enemy has become a flaming star!

Jonnty posted:

enough of the synchronisation jokes

shut up, killjoy

Jonnty
Aug 2, 2007

The enemy has become a flaming star!

enough of the synchronisation jokes

MORE CURLY FRIES
Apr 8, 2004

:golfclap:

tef
May 30, 2004

-> some l-system crap ->

Ronald Raiden posted:

I made that joke first tef :colbert:

eventual consistency :argh:

Zombywuf
Mar 29, 2008

like ordering is the major prob

titaniumone
Jun 10, 2001

some group at my work which has a long history of making terrible decisions and insane "we'll just write it all ourselves from scratch" type poo poo just decided to start using scons

idgaf what they do or how much time they waste but just for my own amusement what are some better alternatives to scons??????????????

Adbot
ADBOT LOVES YOU

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

titaniumone posted:

some group at my work which has a long history of making terrible decisions and insane "we'll just write it all ourselves from scratch" type poo poo just decided to start using scons

idgaf what they do or how much time they waste but just for my own amusement what are some better alternatives to scons??????????????

maven?

and is scons particularly bad or something? i've never used it

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply