Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





could someone explain to a bad programmer why i shouldn't be concerned that like basically every distributed platform that isn't yarn (possibly even yarn) relies on zookeeper? isn't that a single point of failure and a potential bottleneck? is zookeeper that reliable?

Adbot
ADBOT LOVES YOU

Quebec Bagnet
Apr 28, 2009

mess with the honk
you get the bonk
Lipstick Apathy

~Coxy posted:

don't mock out your database. some people say that this means it's not proper Unit Testing anymore but who gives a gently caress

well, it's not if you have actual logic relating to your database. for example, if you have stored procedures of any kind, that's another piece of code that could have a problem, so you don't know if you're testing the object you want to test or how it interacts with your specific database implementation

Quebec Bagnet
Apr 28, 2009

mess with the honk
you get the bonk
Lipstick Apathy
btw Bloody you should look into bcp.exe for loading you're data quickly

DONT THREAD ON ME
Oct 1, 2002

by Nyc_Tattoo
Floss Finder

BONGHITZ posted:

c#f#r#

what is next im going crazy

B#

coffeetable
Feb 5, 2006

TELL ME AGAIN HOW GREAT BRITAIN WOULD BE IF IT WAS RULED BY THE MERCILESS JACKBOOT OF PRINCE CHARLES

YES I DO TALK TO PLANTS ACTUALLY

BONGHITZ posted:

c#f#r#

what is next im going crazy



http://research.microsoft.com/en-us/projects/fstar/

Soricidus
Oct 21, 2010
freedom-hating statist shill
F*** you

suffix
Jul 27, 2013

Wheeee!

the talent deficit posted:

could someone explain to a bad programmer why i shouldn't be concerned that like basically every distributed platform that isn't yarn (possibly even yarn) relies on zookeeper? isn't that a single point of failure and a potential bottleneck? is zookeeper that reliable?

1. zookeeper is very reliable

2. you can and should have several zookeeper servers, e.g. if you have 5 servers in a cluster you can lose any 2 of them

3. you should be more concerned about HA setups that don't use zookeeper, since that means they're writing their own consensus protocol, which is very tricky to get right

that said, yes a botched upgrade can take down everything, and zookeeper can become a performance bottleneck if you do a lot of writing
if you're really big and care about this, each service can have its own zookeeper cluster

zookeeper has some faults, but it's a known quantity and can prevent you from being a mongodb or an elasticsearch

MononcQc
May 29, 2007

the talent deficit posted:

could someone explain to a bad programmer why i shouldn't be concerned that like basically every distributed platform that isn't yarn (possibly even yarn) relies on zookeeper? isn't that a single point of failure and a potential bottleneck? is zookeeper that reliable?

So Zookeeper is that very solid place in your system where data can be safely serialized. Zookeeper has good consistency guarantees through its distribution protocol (ZABPDF), which ensures safe replication and implicit leader election.

This means Zookeeper can have 3 or 5 nodes (more than that and it starts being a lot slower to replicate, and therefore to write anything) and see a minority of them fail and keep being available. This is a quorum system, equivalent to Paxos or Raft in capacity; In your system, zookeeper is used as the main point of serialization for writes and only requires a majority to work, but has better availability properties than a SQL database by virtue of self-electing a new leader without data loss.

Also, given it works like other Quorum systems, you can have 'learner' or 'observer' nodes, which replicate state, but do not ever write to it -- they can be queried and used in a way to scale up by only using the voting nodes as writers, and the learners for reading, unloading the writers a bit.

So for systems that tend to:
  • Have writes that can fit a single node
  • Tend to have more reads than writes
  • Can afford to pay for some redundancy to cancel out individual node failure
  • Can deal with something closer to K/V stores than full blown SQL transactions
Zookeeper is a very good choice, and a shitload more mature than stuff like etcd (using Raft).

E: because Paxos is a quorum system, it still lies on the CP side of CAP -- it can just cope with a minority of losses, which is nicer than SQLs, also on CP side of CAP but unable to easily deal with the loss of the only master they have. Quorum is generally the most available side of CP you can get, but can still freeze everything if you lose a majority of your writers, or if they get stuck in netsplits that leaves no clear majority

MononcQc fucked around with this message at 13:24 on Sep 28, 2014

serious norman
Dec 13, 2007

im pickle rick!!!!
im a Delphi guru

~Coxy
Dec 9, 2003

R.I.P. Inter-OS Sass - b.2000AD d.2003AD

chmods please posted:

well, it's not if you have actual logic relating to your database. for example, if you have stored procedures of any kind, that's another piece of code that could have a problem, so you don't know if you're testing the object you want to test or how it interacts with your specific database implementation

who cares

i test it and it works, or it doesn't.

tef
May 30, 2004

-> some l-system crap ->

MononcQc posted:

The advantages of TDD are:

- You think about what your code should do and expose as an interface before jumping in (write it with the user in mind, not the programmer)
- Your tests are more about what the code should do than crystallizing what crappy code you wrote is doing right now
- It's a shitload more boring to write code after everything is done, than making it part of the design process

- You don't think about your code that much beyond can I double everything going in.
- Sometimes the tests you write are so divorced from the code you can write tests that pass even if the code doesn't exist
- A series of crappy tests give you a false sense of security, but in reality give you a maintenance burden to changing the api

The thing about TDD is you end up with some tests, which sometimes, is better than nothing. Some people find that test-first makes them think about design, or helps them focus. Other times you end up with a bunch of sleep(0.1) statements because life is suffering.

TDD without thinking is some of the test driven i've encountered. It's worth trying test first, along with various other strategies. It isn't worth doing test first if you're going to write lovely tests — TDD means you will have more tests but it does not guarantee better design for code. It isn't a magic wand and you should kill any buddha who claims otherwise.

If you're asking if you should do it, the answer is probably yes.

It is probably worth doing because you'll expose yourself to more mistakes quickly, and making lots of mistakes afaik is the best way to learn programming.

tef
May 30, 2004

-> some l-system crap ->
I'm not against testing, or strong typing, but they don't come for free, and don't guarantee good, performant, robust software.

However it is unlikely your code is good if you don't check your assumptions, and you're probably not checking them if you don't have code to check them for you. Testing does not give quality, but the quality of your tests will impact the quality of your software.

I kinda want more people to use things like dialyser and quickcheck :3:

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
you should almost certainly try tdd if you haven't yet

you don't have to stick with it after you've tried it tho

kitten emergency
Jan 13, 2008

get meow this wack-ass crystal prison
is there a good book or something on tdd

MononcQc
May 29, 2007

uncurable mlady posted:

is there a good book or something on tdd

Never even give Beck's book a read. I hated it and it's the cargo-cult TDD bullshit where you write tests instead of thinking about your problem. I much preferred Growing Object-Oriented Software, Guided by Tests, though it's not like it's a super entertaining read.

kitten emergency
Jan 13, 2008

get meow this wack-ass crystal prison
thanks

Malcolm XML
Aug 8, 2009

I always knew it would end like this.
tests are basically an executable spec, they limit the space in which your program can grow, but dont limit it to much or u end up straightjacketing urself

hth

qntm
Jun 17, 2009

Jabor posted:

you should almost certainly try tdd if you haven't yet

you don't have to stick with it after you've tried it tho

yeah absolutely, it is far from a silver bullet but everybody should know how to do it and when

fixing defects being the big one

Valeyard
Mar 30, 2012


Grimey Drawer

chmods please posted:

btw Bloody you should look into bcp.exe for loading you're data quickly

master of the sea
Apr 16, 2003

*skweeeeeee*
just found engineering mathematics 7th edition on ebay, soo great, i am finally going to conquer my god drat crippling math disability.

Squinty Applebottom
Jan 1, 2013

mathematics is a socially transmitted disease

fritz
Jul 26, 2003

Squinty Applebottom posted:

mathematics is a socially transmitted disease

not that socially

theadder
Dec 30, 2011


sexually

Notorious b.s.d.
Jan 25, 2003

by Reene

that thought makes erdos numbers chilling

jony neuemonic
Nov 13, 2009

uncurable mlady posted:

is there a good book or something on tdd

this one is pretty good if you work with python or on web stuff.

Symbolic Butt
Mar 22, 2009

(_!_)
Buglord
Obey the Testing Goat

jony neuemonic
Nov 13, 2009

Symbolic Butt posted:

Obey the Testing Goat

web developers won't write tests unless you make it so wacky!!

coffeetable
Feb 5, 2006

TELL ME AGAIN HOW GREAT BRITAIN WOULD BE IF IT WAS RULED BY THE MERCILESS JACKBOOT OF PRINCE CHARLES

YES I DO TALK TO PLANTS ACTUALLY

master of the sea posted:

just found engineering mathematics 7th edition on ebay, soo great, i am finally going to conquer my god drat crippling math disability.
:respek:

it wont be easy or quick but its mos def worth it

master of the sea
Apr 16, 2003

*skweeeeeee*

coffeetable posted:

:respek:

it wont be easy or quick but its mos def worth it

yes that's what I figure i might as well just grind this out once and for all

BONGHITZ
Jan 1, 1970

when you die another human will have to be trained in math to replace you

Kathleen
Feb 26, 2013

Grimey Drawer

oooh this is cool. it's good to see microsoft knows how important .net is to their long-term survival

Pittsburgh Fentanyl Cloud
Apr 7, 2003


Luigi Thirty posted:

i have only ever done this and web things which is why uis are stupid and confusing

don't sign your posts lmfao

gonadic io
Feb 16, 2011

>>=
I'm playing around with http://rise4fun.com/FStar/tutorial/guide and I'm trying to write a tail recursive reverse function on vectors.

I have:
code:
module ILIST0

type ilist : int => S => S = 
   | INil  : ilist 0 'a
   | ICons : x:'a -> n:int -> ilist n 'a -> ilist (n + 1) 'a

val reverse : n:int -> ilist n 'a -> ilist n 'a
let reverse n l =
  let rec reverse2
            (n1:int)
            (acc:ilist n1 'a)
            (l':ilist (n-n1) 'a)
             = match l' with
    | INil -> acc
    | ICons x ln_minus_1 ls -> 
      reverse2 (n1+1) (ICons x n1 acc) ln_minus_1 ls
 in reverse2 0 INil n l
and am having a number of problems: since I can't figure out how to write an explicit type sig for reverse2 and slip a {n1+n2=n} (where n2 is the length of the second list), I'm instead having to do it by pattern matching on the length of the second list as (n-n1) and hope it works out what I mean. The error I'm getting is:

input(13,15-17,3) : Error : Expected a pattern of type
ILIST0.ilist (Sub (x_8) (x_10_2)) 'x_7_2
But got pattern of type
ILIST0.ilist 0 '_U309
Type checking failed: ILIST0

which seems to me that it can't prove that the length of the list resulting from reverse2 will be n

gonadic io
Feb 16, 2011

>>=
nevermind, i got it working by specifying reverse2 on its own so i could give it a proper type sig. is there a way to do a typesig inside a let binding inside a function?

The Leck
Feb 27, 2001

gently caress them posted:

Leaving my comfortable little womb of kushiness with nonexistent standards or deadlines and no technical coworkers is the obvious first step. There is nothing to keep me here accountable except shame or a desire to improve. I could be the laziest fucker ever, and retire comfortably. gently caress that.
are you me? this is worryingly accurate.

Janitor Prime
Jan 22, 2004

PC LOAD LETTER

What da fuck does that mean

Fun Shoe

The Leck posted:

are you me? this is worryingly accurate.

god drat it same

Bloody
Mar 3, 2013

this is unrelated to my previous line of questions

say i have a c# program, butt, that produces objects, farts, and i have another (or several other) c# programs on my network that consume farts and return odors and i want the butt to produce farts and get back odors via these other programs over the network (because the odor of a fart takes a lot of cpu time to figure out)

how do i do this

Symbolic Butt
Mar 22, 2009

(_!_)
Buglord
bloody where you wanna go with this :geno:

b0lt
Apr 29, 2005

is this some sort of fascist movement

Adbot
ADBOT LOVES YOU

BONGHITZ
Jan 1, 1970

just have all the noses write odors to a text document and then the butt can smell that text document

or something

  • Locked thread