terrible programmers: my god, it's full of chars

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > terrible programmers: my god, it's full of chars

«‹›1446 »

30 TO 50 FERAL HOG: Mar 2, 2005

tef posted:

Yeah, and a lot of distributed system stuff is tied up with protocol design, along with topology, and well, handling recovery/ failure.

https://twitter.com/worrydream/status/866467925003128832

I am sure I could dig up a dijkstra quote, but in essence, this stuff is really hard: You have to know how your system works and that begins and ends with knowing all of the ways in which it fails.

Your job as a system engineer is to resolve ambiguity and uncertainty in a complex system.

Your job is to make sure causality holds.

Your job is deterministic operations in the face of failure and latency.

You should know what it does, especially when it doesn't.

You model lifecycles of things, you work out at which part is responsible for which point in the lifecycle

you work out your tradeoffs, where latency will always happen, where latency can't happen, and where the buck stops when things go wrong

not so much flow charts, but definitely state machines.

it's more about building responsibility in than working out what goes wrong precisely though.

but it is also about how global concerns trump local ones

i'd probably add some stonebreaker paper about why 'the one size fits all database' is a pipe dream: performance tradeoffs cut across the entirety of the system

any system designed around local convenience is almost certainly globally inconvenient. this is what happens when you use brokers

again, anyone who tells you that you don't need to handle scale in the application layer is lying to you

the end to end principle isn't so much about the edges of the network, but the top layers of the network.

if you have to handle errors in your application code, handling errors deep down inside the library is an optimization

if you have to handle errors at either end of your tcp link, well, handling errors deep down inside ip is an optimization

it isn't so much no-one knows how to do this stuff

no-one wants to care about it

the people who use brokers are never responsible for the brokers, or pretty much any operations

Part of managing a distributed system is experience with failure. Not necessarily a paranoia, or a lack of trust, but an understanding that everything is a risk, but risks can be weighed up, and damage can be mitigated.

anyway

- why do computers stop and what can be done about it
- the end to end argument in system design
- why complex systems fail
- systems for young bloods (the jeff hodges piece)
- making reliable distributed systems in the presence of software errors (this is about process supervision & fail fast)
- ivan sutherland's on the design of display co-processors,
- on the criteria of decomposing systems into modules
- some of the postel's principle stuff (including the langsec revisions),
- crash only software,
- the systems bible / systemantics
- maybe even a system r paper or two
- tcp/ip illustrated
- roy fielding's representation state transfer thesis (lol)
- challenger accident reports
- (bonus norbert weiner or buckminster fuller work on cybernetics or systems if you want to dig deep)

also this

https://www.amazon.com/dp/0671819100/_encoding=UTF8?coliid=I1APO59D04D8X6&colid=2CWLM9OCL7GUZ

# ? Jun 3, 2017 06:38

Adbot: ADBOT LOVES YOU

# ? May 23, 2024 09:58

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

yesterday evening whilst coding and drinking beer (ffffriday night! muthafucka) i hand encoded some html snippets in java.

today i replaced the code with jsoup

# ? Jun 3, 2017 09:32

gonadic io: Feb 16, 2011; >>=

MALE SHOEGAZE posted:

well today i found out that equality in scala is garbage

which part of it?

we've just had some fun with week-year vs actual year but that's java's datetimeformatter, can't blame scala for that one

# ? Jun 3, 2017 10:05

Powerful Two-Hander: Mar 10, 2004; Mods please change my name to "Tooter Skeleton" TIA.

god Dapper is so fuckin good i feel like a wizard binding up split objects in a single line of code instead of 25 lines wrapped into a reader

if id known about this a year ago i could have saved so much typing

# ? Jun 3, 2017 11:09

pokeyman: Nov 26, 2006; That elephant ate my entire platoon.

gonadic io posted:

which part of it?

we've just had some fun with week-year vs actual year but that's java's datetimeformatter, can't blame scala for that one

did someone miscapitalize a y

# ? Jun 3, 2017 13:20

gonadic io: Feb 16, 2011; >>=

pokeyman posted:

did someone miscapitalize a y

yuuuuuuuuup

# ? Jun 3, 2017 14:18

MononcQc: May 29, 2007

the talent deficit posted:

re: alternatives to message queues

check out aws step functions. the implementation is terrible and the interface is bad but the protocol is probably what you want if you are using a message queue to distribute work or do distributed transactional work

basically you have a central coordinator that accepts jobs (a state machine) and then handles informing the workers what they are supposed to do when. it handles all the retry/timeout logic for you and if you are a little bit clever you can even do things like saga transactions pretty easily

probably relevant reference for sagas: https://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf

# ? Jun 3, 2017 14:21

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

gonadic io posted:

yuuuuuuuuup

hmm, you seem to be trying to render the year multiple times. and what is up with that padding specifier?

# ? Jun 3, 2017 14:24

NewForumSoftware: Oct 8, 2016; by Lowtax

https://np.reddit.com/r/cscareerquestions/comments/6ez8ag/accidentally_destroyed_production_database_on/

quote:

Today was my first day on the job as a Junior Software Developer and was my first non-internship position after university. Unfortunately i screwed up badly.

I was basically given a document detailing how to setup my local development environment. Which involves run a small script to create my own personal DB instance from some test data. After running the command i was supposed to copy the database url/password/username outputted by the command and configure my dev environment to point to that database. Unfortunately instead of copying the values outputted by the tool, i instead for whatever reason used the values the document had.

Unfortunately apparently those values were actually for the production database (why they are documented in the dev setup guide i have no idea). Then from my understanding that the tests add fake data, and clear existing data between test runs which basically cleared all the data from the production database. Honestly i had no idea what i did and it wasn't about 30 or so minutes after did someone actually figure out/realize what i did.

While what i had done was sinking in. The CTO told me to leave and never come back. He also informed me that apparently legal would need to get involved due to severity of the data loss. I basically offered and pleaded to let me help in someway to redeem my self and i was told that i "completely hosed everything up".

So i left. I kept an eye on slack, and from what i can tell the backups were not restoring and it seemed like the entire dev team was on full on panic mode. I sent a slack message to our CTO explaining my screw up. Only to have my slack account immediately disabled not long after sending the message.

I haven't heard from HR, or anything and i am panicking to high heavens. I just moved across the country for this job, is there anything i can even remotely do to redeem my self in this situation? Can i possibly be sued for this? Should i contact HR directly? I am really confused, and terrified.

EDIT Just to make it even more embarrassing, i just realized that i took the laptop i was issued home with me (i have no idea why i did this at all).

# ? Jun 3, 2017 15:03

AggressivelyStupid: Jan 9, 2012

im the ability for a junior dev to delete the production database

# ? Jun 3, 2017 15:09

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

gonadic io posted:

which part of it?

we've just had some fun with week-year vs actual year but that's java's datetimeformatter, can't blame scala for that one

the part where the signature of equals is

code:

override def equals(o: Any): Boolean

Since case class structural equality is so great I've been blissfully unaware that the `==` operator is not type safe at all.

Since we use mongo (yaaay) all of our domain objects look like this:

code:

trait HasId { def id: String }
case class Butt(id: String) extends HasId
case class ButtHaver(id: String, butt: String) extends HasId

String is a UUID but we convert it to a string for unknown reasons. This is tolerable in obvious situations like the above, but it gets really confusing with less self documenting models

code:

case class GenericDomainObject(id: String, name: String, genericThing: String, otherThing: String, foo: String)

Are all of those strings references? What do they refer to? It's a mystery! (also to make things extra confusing nobody adopted the minimal convention of at least writing `otherThingId: String` so you know it's an id)

So I'm trying to add a simple reference wrapper to fix this problem:

code:

case class Ref[T <: HasId](val id: String)

So now I can do:

code:

case class GenericDomainObject(
  id: String,
  name: String,
  genericThing: Ref[GenericThing],
  otherThing: Ref[OtherThing],
  foo: Ref[ActuallyABar]
)

(I added custom jackson ser/deser so this is all transparent to mongo/our api).

So anyhow I thought I'd just be able to add my nifty type wrapper and then I'd get lots of useful compiler errors in all of the places the code was now wrong, but I didn't realize that equality works the way it does so now I have many instances of this:

code:

val listOfBars = Set("1, "2", "3") 
val barHaver = GenericDomainObject("456", "foo", Ref("42145"), Ref("21245124"), Ref("3"))
listOfBars.find(_ == barHaver.foo) // Compares string to Ref[Bar], compiler doesn't care!

My initial response was to hesitantly override equals for Ref, so when compared to a string it would use the underlying Id instead of the full class for comparison (and emit a backtrace so we can go fix the code). But then I dumbly realized that this only works when the Ref is on the left hand side of the operator. lol owned.

DONT THREAD ON ME fucked around with this message at 15:15 on Jun 3, 2017

# ? Jun 3, 2017 15:10

raminasi: Jan 25, 2005; a last drink with no ice

AggressivelyStupid posted:

im the ability for a junior dev to delete the production database

yeah that poor dev carries zero blame for that fuckup imo

# ? Jun 3, 2017 15:19

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

raminasi posted:

yeah that poor dev carries zero blame for that fuckup imo

yeah the horror there is the CTO for sure

# ? Jun 3, 2017 15:20

kitten emergency: Jan 13, 2008; get meow this wack-ass crystal prison

jesus christ

# ? Jun 3, 2017 15:28

gonadic io: Feb 16, 2011; >>=

MALE SHOEGAZE posted:

the part where the signature of equals is
code:
override def equals(o: Any): Boolean
Since case class structural equality is so great I've been blissfully unaware that the `==` operator is not type safe at all.

oh yeah i loving hate that poo poo. don't forget that pattern matching and even scalaTest's "shouldBe" are all non-typesafe too. what's the loving point of being in a static lang if you're going to throw away all of the benefits

at least the first two have warnings in intellij for using them with different types, but the last doesn't even have that. the number of times i've been writing tests and refactoring as i go and getting a "TEST FAILED: 5 was not equal to Some(5)" because i forgot and was given no assistance in editing my test cases

# ? Jun 3, 2017 16:11

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

gonadic io posted:

oh yeah i loving hate that poo poo. don't forget that pattern matching and even scalaTest's "shouldBe" are all non-typesafe too. what's the loving point of being in a static lang if you're going to throw away all of the benefits

at least the first two have warnings in intellij for using them with different types, but the last doesn't even have that. the number of times i've been writing tests and refactoring as i go and getting a "TEST FAILED: 5 was not equal to Some(5)" because i forgot and was given no assistance in editing my test cases

yeah this is the first time I've been really let down by the language. I was very excited to put my little documentation wrapper to use but instead of being trivial to refactor existing classes it is now a minefield so I guess i'll just have to settle for strings.

# ? Jun 3, 2017 16:30

MrMoo: Sep 14, 2000

Two or three times a day my production system is gobbling up memory and being killed by a supervisor process. I ask the team looking after the message queue library for some instrumentation on thei memory usage. I receive the following reply:

quote:

The dbcapi uses the heap from the call process so there�s no separate heap for the dbcapi itself.

Idk, I cannot use Valgrind in production and haven't reproduced the issue despite running the same system without restarts for weeks. The app does barely anything: pops a message off a queue, converts to JSON using a popular C++ header library that has a tonne of tests, then posts it via Boost ASIO to Beast WebSockets.

I also have fun every morning because there are two cables from NYSE to IDC and sometimes one is faster than the other and that breaks things for everyone. People scream because they hit a button and the message has to go through a dozen systems around the Americas and back to be displayed on the monitor above their head. My diagnostic input is basically no different from watching MSNBC or Fox News in the morning: I can see numbers flag on the TV but cannot get any additional detail. Awesome sauce.

No wonder no one else tendered for this project, lol

# ? Jun 3, 2017 16:34

Brain Candy: May 18, 2006

MALE SHOEGAZE posted:

yeah this is the first time I've been really let down by the language. I was very excited to put my little documentation wrapper to use but instead of being trivial to refactor existing classes it is now a minefield so I guess i'll just have to settle for strings.

but what if you some Set[A] with the same elements as some Set[B] and A and B were not related. how would you know that Set[A] == Set[B]?

# ? Jun 3, 2017 16:45

FamDav: Mar 29, 2008

the talent deficit posted:

re: alternatives to message queues

check out aws step functions. the implementation is terrible and the interface is bad but the protocol is probably what you want if you are using a message queue to distribute work or do distributed transactional work

why do you say the implementation is terrible?

# ? Jun 3, 2017 16:48

gonadic io: Feb 16, 2011; >>=

Brain Candy posted:

but what if you some Set[A] with the same elements as some Set[B] and A and B were not related. how would you know that Set[A] == Set[B]?

i hope you're being sarcastic

# ? Jun 3, 2017 16:48

Arcsech: Aug 5, 2008

NewForumSoftware posted:

https://np.reddit.com/r/cscareerquestions/comments/6ez8ag/accidentally_destroyed_production_database_on/

how much do you want to bet that the database in question is mongodb

# ? Jun 3, 2017 18:31

Brain Candy: May 18, 2006

gonadic io posted:

i hope you're being sarcastic

that's actually the reason it is the way it is, and yes i am

# ? Jun 3, 2017 19:06

silvergoose: Mar 18, 2006; IT IS SAID THE TEARS OF THE BWEENIX CAN HEAL ALL WOUNDS

I like the response that starts with "Hey so I'm the guy who made gitlab die, remember that?"

# ? Jun 3, 2017 19:27

MrMoo: Sep 14, 2000

Arcsech posted:

how much do you want to bet that the database in question is mongodb

Reading through comments on CouchDB I think that previously defaulted to no password at all, wonder how many systems were wide open on the Internets like that.

# ? Jun 3, 2017 19:44

necrotic: Aug 2, 2005; I owe my brother big time for this!

Arcsech posted:

how much do you want to bet that the database in question is mongodb

it was postgres based on a later comment. the type of database doesnt matter here (except that mongo would have deleted the data without his help)

# ? Jun 3, 2017 20:06

kugutsu: Dec 31, 2008

scala's equality is a lovely holdover from java. they're looking to improve the situation in a future version: http://www.scala-lang.org/blog/2016/05/06/multiversal-equality.html

in the meantime if you use scalaz or cats, both of them support an Equal/Eq typeclass and === operator to use instead of the default equality.

# ? Jun 3, 2017 20:58

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

kugutsu posted:

scala's equality is a lovely holdover from java. they're looking to improve the situation in a future version: http://www.scala-lang.org/blog/2016/05/06/multiversal-equality.html

in the meantime if you use scalaz or cats, both of them support an Equal/Eq typeclass and === operator to use instead of the default equality.

yeah i'm really tempted to just find+replace every '==' in the codebase with '===' and then figure things out.

but also i looked spent an hour last night trying to figure out how to use '===' and i couldn't. the cats documentation is total garbage.

# ? Jun 3, 2017 21:10

kugutsu: Dec 31, 2008

you can get at the === operator in cats by importing cats.implicits._
to actually use it for types that aren't built in you'll need to provide an instance of the Eq typeclass for that type. this would usually be done for a case class you define by placing an implicit in the companion object for that type, e.g:

code:

case class Butt(...)

object Butt {
  implicit val eq = Eq.fromUniversalEquals[Butt]
}

the fromUniversalEquals is one of the functions on the Eq object. since case classes already implement structural equality you can just delegate to the built in equals for those. for a normal class you'd wanna define your own Eq instance implementing the eqv method

# ? Jun 3, 2017 22:23

JewKiller 3000: Nov 28, 2006; by Lowtax

if you use scalaz then you are irredeemably hosed and there is nothing anyone can do to help you

# ? Jun 4, 2017 01:21

MononcQc: May 29, 2007

BiohazrD posted:

also this

https://www.amazon.com/dp/0671819100/_encoding=UTF8?coliid=I1APO59D04D8X6&colid=2CWLM9OCL7GUZ

found some samples of it and it looks p. good. Managed to find a not-too-expensive used version and ordered it. Thanks for the recommendation.

# ? Jun 4, 2017 02:21

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

kugutsu posted:

you can get at the === operator in cats by importing cats.implicits._
to actually use it for types that aren't built in you'll need to provide an instance of the Eq typeclass for that type. this would usually be done for a case class you define by placing an implicit in the companion object for that type, e.g:
code:
case class Butt(...)

object Butt {
  implicit val eq = Eq.fromUniversalEquals[Butt]
}
the fromUniversalEquals is one of the functions on the Eq object. since case classes already implement structural equality you can just delegate to the built in equals for those. for a normal class you'd wanna define your own Eq instance implementing the eqv method

Thanks a lot. I knew it would be something like that but I couldn't get there. I'm going to try this just for fun and maybe for real.

e: it works!

DONT THREAD ON ME fucked around with this message at 03:17 on Jun 4, 2017

# ? Jun 4, 2017 02:30

fart simpson: Jul 2, 2005; DEATH TO AMERICA

NewForumSoftware posted:

https://np.reddit.com/r/cscareerquestions/comments/6ez8ag/accidentally_destroyed_production_database_on/

mega lol

# ? Jun 4, 2017 06:21

tef: May 30, 2004; -> some l-system crap ->

the talent deficit posted:

all the scala programmers i work with were like 'this is amazing! let's rewrite everything!'

yep, you've got scala programmers alright

# ? Jun 4, 2017 06:54

tef: May 30, 2004; -> some l-system crap ->

BiohazrD posted:

also this

https://www.amazon.com/dp/0671819100/_encoding=UTF8?coliid=I1APO59D04D8X6&colid=2CWLM9OCL7GUZ

- the systems bible / systemantics

ugh i think the systems bible is the 3rd ed

https://www.amazon.com/Systems-Bible-Beginners-Guide-Large/dp/0961825170

https://en.wikipedia.org/wiki/Systemantics

'General Systemantics (retitled to Systemantics in its second edition and The Systems Bible in its third) is a systems engineering treatise by John Gall in which he offers practical principles of systems design based on experience and anecdotes.'

it's kinda wheezy in places and suffers from that 'everything is hard so it's time for big words and clever aphorisms' smug posting i do but worse

tef fucked around with this message at 06:58 on Jun 4, 2017

# ? Jun 4, 2017 06:56

Luigi Thirty: Apr 30, 2006; Emergency confection port.

hmm, I got CodeWarrior installed on my iMac. I can't seem to locate a C version of the Mac Toolbox reference though, just a mirrored copy of Apple's old references which are all in Pascal. it does not include a Pascal compiler.

it does have a dinky console that you can use as a framework for standard C applications though so that's something

# ? Jun 4, 2017 07:03

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

JewKiller 3000 posted:

if you use scalaz then you are irredeemably hosed and there is nothing anyone can do to help you

should that really be an if�then? I thought there was a custom operator for that

my reaction when I first saw someone propose a "swiftz" was "NOOOOOOOOoOoOoooo!"

# ? Jun 4, 2017 07:06

Hunter2 Thompson: Feb 3, 2005; Ramrod XTreme

tef posted:

- the systems bible / systemantics

ugh i think the systems bible is the 3rd ed

https://www.amazon.com/Systems-Bible-Beginners-Guide-Large/dp/0961825170

https://en.wikipedia.org/wiki/Systemantics

'General Systemantics (retitled to Systemantics in its second edition and The Systems Bible in its third) is a systems engineering treatise by John Gall in which he offers practical principles of systems design based on experience and anecdotes.'

it's kinda wheezy in places and suffers from that 'everything is hard so it's time for big words and clever aphorisms' smug posting i do but worse

Oh yeah I bought the "Systems Bible" on your recommendation without realizing that the other two Systemantics books are previous editions of the same book but with different titles :doh:

# ? Jun 4, 2017 07:49

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

Luigi Thirty posted:

hmm, I got CodeWarrior installed on my iMac. I can't seem to locate a C version of the Mac Toolbox reference though, just a mirrored copy of Apple's old references which are all in Pascal. it does not include a Pascal compiler.

it does have a dinky console that you can use as a framework for standard C applications though so that's something

what you're looking for is called "Macintosh Toolbox Programmer's Assistant" and it should actually be included with "newer" versions of CodeWarrior (look in an extras folder, MPW should be there too)

don't be put off by Pascal API descriptions though, systems Pascal on the Mac is isomorphic to C and it should be trivial to just pick up and use virtually any APIs, your main "difficulty" will just be including the right header (not library, almost all the APIs are in InterfaceLib on PowerPC) and CodeWarrior's batch find is super fast

also grab yourself ObiWan from Stairways Software (Peter N Lewis), it's primarily a handy API quick-reference that comes up at a keystroke, and only takes a couple MB of disk, and runs lightning fast even on a 68030

I should see if Peter will release the source

# ? Jun 4, 2017 08:19

Luigi Thirty: Apr 30, 2006; Emergency confection port.

I have CodeWarrior 7.1. I'll poke around the ISO and see what I can find

# ? Jun 4, 2017 09:21

Adbot: ADBOT LOVES YOU

# ? May 23, 2024 09:58

Mahatma Goonsay: Jun 6, 2007; Yum

whois john gall

# ? Jun 4, 2017 13:04

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > terrible programmers: my god, it's full of chars

«‹›1446 »