Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
ChickenWing
Jul 22, 2010

:v:

Fellatio del Toro posted:

If you do decide to help him the answer is probably not writing a generic JSON manipulation API but just giving him an Update Progress endpoint

Seconding this. Makes your overall validation a lot easier. PATCH has always seemed like more trouble than it's worth.

Adbot
ADBOT LOVES YOU

Doom Mathematic
Sep 2, 2008
We dodged a problem when we were designing our API - and wanted a PATCH method - by making it so that every field was mandatory, but some fields could be explicitly set to null. So if an object starts out as { a: 1, b: 2 }, you can

  • send a PATCH with { a: 35, b: 600 }, resulting in { a: 35, b: 600 }, or
  • send a PATCH with { a: 35, b: null }, resulting in { a: 35, b: null }, or
  • send a PATCH with { a: 35 }, resulting in { a: 35, b: 2 }.

And there's no way to get an object which is just { a: 35 }. That's an error.

spacebard
Jan 1, 2007

Football~
There’s always the rabbit hole of implementing your own CRDT. :v:

Do it. don’t do it

PhantomOfTheCopier
Aug 13, 2008

Pikabooze!

Munkeymon posted:

I don't see how PATCH fixes that in the case of a related or overlapping change...
It's much simpler: PUT always succeeds for a valid object, whereas PATCH can return an error when a valid update request nevertheless conflicts with an intermediate change.

EG, put(a=5,b=6) will happily overwrite anything, as in the above examples, but patch(a,from 4,to 5) can return a failure if something else has already changed a, after which the client can retry, refetch, etc.

FormatAmerica
Jun 3, 2005
Grimey Drawer
It's not *entirely* uncommon for REST APIs to include an Etag & require If-Match headers to match a hash of the current representation of the object in the database in order to update it. Return a 412 - Precondition Failed if it doesn't match, 428 - Precondition Required if it's missing in the request. Or use use a database with transaction locking. Or implement a distributed locking mechanism using something like redis. Even eventing every change out so you can un-fuckup stuff later is a valid strategy/backup. If you have any control over the backend do something to guard the update & deletion of persisted objects.

Pie Colony
Dec 8, 2006
I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED

poemdexter posted:

An object exists: {name: joe, sex: male, height: tall}
Client 1 wants to change height to short.
Client 2 wants to change sex to female.

Client 1 GET object {name: joe, sex: male, height: tall}
Client 2 GET object {name: joe, sex: male, height: tall}
Client 1 POST object {name: joe, sex: male, height: short}
Client 2 POST object {name: joe, sex: female, height: tall}

Result: {name: joe, sex: female, height: tall}

-- or --

Client 1 PATCH object {height: short}
Client 2 PATCH object {sex: female}

Result: {name: joe, sex: female, height: short}

What I meant to say was HTTP verbs have absolutely zero relation to update sequencing. The first way is just a lovely way to structure an application, the second still has race conditions. The OP made it seem like just doing partial updates via PATCH solved race conditions.

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



PhantomOfTheCopier posted:

It's much simpler: PUT always succeeds for a valid object, whereas PATCH can return an error when a valid update request nevertheless conflicts with an intermediate change.

EG, put(a=5,b=6) will happily overwrite anything, as in the above examples, but patch(a,from 4,to 5) can return a failure if something else has already changed a, after which the client can retry, refetch, etc.

Yeah, but the handler thread can't know about an intermediate change on some other random thread unless there's a versioning scheme like I mentioned in the part you cut out or something like

FormatAmerica posted:

It's not *entirely* uncommon for REST APIs to include an Etag & require If-Match headers to match a hash of the current representation of the object in the database in order to update it. Return a 412 - Precondition Failed if it doesn't match, 428 - Precondition Required if it's missing in the request. Or use use a database with transaction locking. Or implement a distributed locking mechanism using something like redis. Even eventing every change out so you can un-fuckup stuff later is a valid strategy/backup. If you have any control over the backend do something to guard the update & deletion of persisted objects.

Which does make some sense! However, it sounds like it'd be more complicated for the OP to implement this because they would have to build a 'current version' of the object from DB queries to check its hash which introduces a race condition. I guess you could have, say, Redis storing a 'last seen' copy and check against that. It'd probably also help GET performance :v:

Space Gopher
Jul 31, 2006

BLITHERING IDIOT AND HARDCORE DURIAN APOLOGIST. LET ME TELL YOU WHY THIS SHIT DON'T STINK EVEN THOUGH WE ALL KNOW IT DOES BECAUSE I'M SUPER CULTURED.

Munkeymon posted:

Which does make some sense! However, it sounds like it'd be more complicated for the OP to implement this because they would have to build a 'current version' of the object from DB queries to check its hash which introduces a race condition. I guess you could have, say, Redis storing a 'last seen' copy and check against that. It'd probably also help GET performance :v:

You can't get around a potential race condition here without some way to do an atomic "compare and only run the update if unchanged" operation on the backing data store - locks, transactions, or built-in support for an etag-type mechanism in the DB.

My Rhythmic Crotch
Jan 13, 2011

sunaurus posted:

Client 1 - GET
Client 2 - GET
Client 1 - POST based on GET results
Client 2 - POST based on GET results

Client 2 overwrites Client 1's changes (even if they were completely unrelated to Client 2's changes)

Our architecture has been susceptible to this, but only JUST NOW that we were chatting about am I having to work on it... arrghghuughhhh

Fellatio del Toro posted:

If you do decide to help him the answer is probably not writing a generic JSON manipulation API but just giving him an Update Progress endpoint

Soooo

This is probably not going to work. Now that we have moved on to the topic of race conditions here, he's convinced that we need pretty wide patch support, and that patch support would also "fix" the race condition issues.

Blarghhhhh

Edit, gently caress me, this is his proposed idea:

User edits a value --> mark that value as dirty --> right before saving refresh the value (why?) --> PATCH all dirty variables

Now, imagine how crusty my code will get. My front-end code will need clean/dirty marking for every value, and somewhere (front or back) will need to map the sent variable to the corresponding column in the DB. And I would need PATCH support for all those columns. Fuuuuuu

My Rhythmic Crotch fucked around with this message at 21:35 on Aug 26, 2019

FormatAmerica
Jun 3, 2005
Grimey Drawer

Space Gopher posted:

You can't get around a potential race condition here without some way to do an atomic "compare and only run the update if unchanged" operation on the backing data store - locks, transactions, or built-in support for an etag-type mechanism in the DB.

Dependent on the actual use case the impact to the system could vary from "huge loving showstopper" to "unnoticed by users". If it's a big problem, that's probably indicative of sub-optimal design through the stack but mainly the persisted data model.

Supporting PATCH may reduce the likelihood of the problem occurring simply due to smaller change sets but if you're blasting all your PATCH operations out when the user submits a form that seems, if anything, more complicated than the conflicts you'd be creating with fewer requests.

Given the limited options, I would not keep track of clean/dirty state but instead keep in memory & diff the initial, form, and new/refreshed representations of the objects as-needed and decide what to do based on the diff content between initial & form, initial & new before submitting the changes with a single PUT/POST. It seems like you don't have a race condition as much as a "persisted-state-awareness" problem (made that up, idk if there's a real term for this). If the object in question is updated so frequently that you risk maybe a couple hundred milliseconds on the refresh+update, I'd go back to questioning the validity of the database design.

e: also json patch is good http://jsonpatch.com/

spiritual bypass
Feb 19, 2008

Grimey Drawer
Sounds like the stateful design is the real problem here?

My Rhythmic Crotch
Jan 13, 2011

FormatAmerica posted:

Supporting PATCH may reduce the likelihood of the problem occurring

I think this is his position, and I mean, while I agree... this system has been growing and evolving for 20+ years at this point, and just now they got bit by their first race condition bug (well, I'm sure it's occurred many times, but it was the first time he bothered to look). To me it's a huge loving overreaction on the part of my coworker.

rt4 posted:

Sounds like the stateful design is the real problem here?

I suppose so, yes. Imagine Jenkins (that's basically what we do here, although with embedded firmware so "the same only different"). Now make Jenkins without maintaining state anywhere. I know it's possible, but is it practical? Interesting to think about if nothing else.

My Rhythmic Crotch fucked around with this message at 23:10 on Aug 26, 2019

PhantomOfTheCopier
Aug 13, 2008

Pikabooze!
Won't bother quoting, but wth you've never heard of a lock? That would be the entire point of 'patch'. By all means convince yourself that you dislike locks and want to build a stateless, seamless eventually consistent API and data layer... but don't say you weren't warned.

https://en.m.wikipedia.org/wiki/KISS_principle

https://en.m.wikipedia.org/wiki/You_aren%27t_gonna_need_it

Keetron
Sep 26, 2008

Check out my enormous testicles in my TFLC log!

FormatAmerica posted:

e: also json patch is good http://jsonpatch.com/

We use this and it is the best way I have encountered doing something that doesn't feel right in the beginning but turns out fine in the end. This is mostly due to the standardisation of it.

My Rhythmic Crotch
Jan 13, 2011

PhantomOfTheCopier posted:

Won't bother quoting, but wth you've never heard of a lock? That would be the entire point of 'patch'. By all means convince yourself that you dislike locks and want to build a stateless, seamless eventually consistent API and data layer... but don't say you weren't warned.

https://en.m.wikipedia.org/wiki/KISS_principle

https://en.m.wikipedia.org/wiki/You_aren%27t_gonna_need_it

I would love to use locks but I don't think I can sell that idea.

Cuntpunch
Oct 3, 2003

A monkey in a long line of kings

Keetron posted:

We use this and it is the best way I have encountered doing something that doesn't feel right in the beginning but turns out fine in the end. This is mostly due to the standardisation of it.

I have a glimmer of a suspicion that in 10 years we are all going to look back on shoehorning adhoc extensions to json in a similar light to the late 90s "... But Online!" days.

Keetron
Sep 26, 2008

Check out my enormous testicles in my TFLC log!

Cuntpunch posted:

I have a glimmer of a suspicion that in 10 years we are all going to look back on shoehorning adhoc extensions to json in a similar light to the late 90s "... But Online!" days.

Not disagreeing, it is weird to have a perfectly fine object that you deserialize to text, edit that text and then serialize it back to the object while at the same time saying "I am fine, this is fine."

PhantomOfTheCopier
Aug 13, 2008

Pikabooze!

My Rhythmic Crotch posted:

I would love to use locks but I don't think I can sell that idea.
Don't sell, tell.

Lord Of Texas
Dec 26, 2006

PhantomOfTheCopier posted:

Won't bother quoting, but wth you've never heard of a lock? That would be the entire point of 'patch'. By all means convince yourself that you dislike locks and want to build a stateless, seamless eventually consistent API and data layer... but don't say you weren't warned.

https://en.m.wikipedia.org/wiki/KISS_principle

https://en.m.wikipedia.org/wiki/You_aren%27t_gonna_need_it

I assume you're referring to pessimistic locking, the suggestions being made here are generally referring to optimistic locking, which IMO is a common and easy enough pattern to not fall under YAGNI. Pessimistic locking is obnoxious to deal with as a user and developer.

ChickenWing
Jul 22, 2010

:v:

Ugh god yes there's nothing quite like backlog grooming and removing like 15 stories that have long since been duplicated or descoped


our backlog is a goddamn mess and every step closer to it being an actual informative and useful look at our future fills me with bliss

PhantomOfTheCopier
Aug 13, 2008

Pikabooze!
That you can actually see light at the end of that train wreck... is enviable.

lifg
Dec 4, 2000
<this tag left blank>
Muldoon
Why does every company use Jive for their intranet? And why does it always have some quirky name like, "The Park", or, "The Landing"?

Achmed Jones
Oct 16, 2004



I think my old company used Igloo

smackfu
Jun 7, 2004

And every intranet tries to be Facebook-esque with followers and blogs and crap,

Mr Shiny Pants
Nov 12, 2012
And no one is interested, because the interesting people you want to follow are not your coworkers.

lifg
Dec 4, 2000
<this tag left blank>
Muldoon
Jive even tries to gamify it. You can earn badges for participating.

Queen Victorian
Feb 21, 2018

So I finished the first week at my new job and, uh, wow... it’s so different when you and your skills/knowledge are actually valued and your teammates are cool and friendly and not patronizing douchebags or process-obsessed surly robots.

So yeah, everyone here is lovely, and the lead dev (my direct supervisor) is great and actually respects what front-end/UI/design people do. In general, everyone here is really into my design background and so far CEO is pleased with my very thorough design analysis of the product (my project for most of the week was to learn/assess the application and also learn the codebase).

I think this job is going to be pretty good. :kiddo:

Carbon dioxide
Oct 9, 2012

Queen Victorian posted:

So I finished the first week at my new job and, uh, wow... it’s so different when you and your skills/knowledge are actually valued and your teammates are cool and friendly and not patronizing douchebags or process-obsessed surly robots.

So yeah, everyone here is lovely, and the lead dev (my direct supervisor) is great and actually respects what front-end/UI/design people do. In general, everyone here is really into my design background and so far CEO is pleased with my very thorough design analysis of the product (my project for most of the week was to learn/assess the application and also learn the codebase).

I think this job is going to be pretty good. :kiddo:

Congrats! In my experience and from what I hear by actually Talking To People In Real Life, there's plenty of these great jobs around, it's just, there's not much to Discuss On The Internet if everything goes well, so the people complaining in this thread day in day out are actually the small minority that have something to complain about.

Although I do have a suspicion that there's more bad jobs around in countries that have a tradition of not valuing their employees, such as the USA.

Wibla
Feb 16, 2011

Carbon dioxide posted:

Congrats! In my experience and from what I hear by actually Talking To People In Real Life, there's plenty of these great jobs around, it's just, there's not much to Discuss On The Internet if everything goes well, so the people complaining in this thread day in day out are actually the small minority that have something to complain about.

Although I do have a suspicion that there's more bad jobs around in countries that have a tradition of not valuing their employees, such as the USA.

The bit about USA seems fairly accurate from pretty much everyone I've talked to who work and live in the US :v:

Keetron
Sep 26, 2008

Check out my enormous testicles in my TFLC log!

Wibla posted:

The bit about USA seems fairly accurate from pretty much everyone I've talked to who work and live in the US :v:

Weirdest thing tho is that the best dev manager I had so far was from Brazil working in a US company while we are all located in socialist Europe.

Wibla
Feb 16, 2011

Keetron posted:

Weirdest thing tho is that the best dev manager I had so far was from Brazil working in a US company while we are all located in socialist Europe.

Now that is weird :v:

Volmarias
Dec 31, 2002

EMAIL... THE INTERNET... SEARCH ENGINES...

Queen Victorian posted:

So I finished the first week at my new job and, uh, wow... it’s so different when you and your skills/knowledge are actually valued and your teammates are cool and friendly and not patronizing douchebags or process-obsessed surly robots.

So yeah, everyone here is lovely, and the lead dev (my direct supervisor) is great and actually respects what front-end/UI/design people do. In general, everyone here is really into my design background and so far CEO is pleased with my very thorough design analysis of the product (my project for most of the week was to learn/assess the application and also learn the codebase).

I think this job is going to be pretty good. :kiddo:

Congrats! :unsmith:

You'll find plenty of reasons to hate it in a year but at least you're not working with absolute garbage people.

Queen Victorian
Feb 21, 2018

Thanks guys.

Carbon dioxide posted:

Although I do have a suspicion that there's more bad jobs around in countries that have a tradition of not valuing their employees, such as the USA.

At my old job the C-levels were European, but they managed to not bring over any of those good European workplace philosophies and instead embraced the American ones.

At my new job, the CEO is American and female, which might have something to do with the much better gender balance and attention to team happiness/harmony here.

Volmarias posted:

Congrats! :unsmith:

You'll find plenty of reasons to hate it in a year but at least you're not working with absolute garbage people.

Oh, I’ll be sticking around to bitch about the codebase because oh my lord, but yeah, don’t have any complaints about my coworkers. Taking on a hosed up codebase with rad people at your side is infinitely preferable to working on a good codebase with lovely people as far as I’m concerned.

As for the codebase, this application is in a less mature state than the one at my last job, and some very.... interesting early decisions were made in the front end that I will get to unfuck. Though it does help that they’re acutely aware of the issues and are pretty much giving me free reign to figure out how to restructure both the workflows/design and the front-end codebase (I guess that’s another thing - new job is definitely a step up in terms of developer rank and influence).

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
My current CEO is ex military and an ex VC but is nicer in every way than most of the private sector only folks I’ve worked under. Take people for whom they are and want to be rather than what you’d think from their background. Maybe he was an rear end in a top hat before but he certainly isn’t now after making his millions elsewhere. Most of the founding executives had already made their money so the company is more like a passion project to atone for their mistakes. A lot of current employees (that are not assholes either) came with them which is another important sign of “not rear end in a top hat” status to me.

Queen Victorian
Feb 21, 2018

I think another important thing to look at in companies and their leadership is the aspects of the work environment they choose to prioritize. Leadership at my new company strongly prioritizes having a chill office full of nice not-rear end in a top hat people and good work/life balance, because the belief is that they’ll get the most productivity out of a happy, well-rested team. They were very forthcoming about this during the hiring process.

On the other hand, leadership at my old company eventually fell into the trap of thinking that the way to optimum productivity was to push everyone to work more/harder and be all “[company] is my life and so is coding in general!”. I even got a talking to about not being a team player because I wasn’t dragging my work laptop home every single night. gently caress that. All it did was make me resentful. I even asked my new boss if I should be taking my work laptop home every night night and he was like “lol no” and then told us all to go home at 5.

Gildiss
Aug 24, 2010

Grimey Drawer
But is it an open office workspace anyway? Despite all the caring about other factors?

Queen Victorian
Feb 21, 2018

Open/shared offices are unavoidable in startups. However, if set up right, I don’t mind them, personally. I think I’m actually more productive in an environment with other stuff going on in the background than in silence and isolation. When I freelanced, my favorite spots to go work were coffee shops. Library was too quiet.

Even so, new office does have multiple rooms - admin/sales, dev, etc. There is also a totally separate quiet/solitude room (something my old office didn’t have). So I guess having at least some division with walls and doors is better than a full-blown “open concept” office where you don’t even have your own designated desk or some poo poo. Also the WFH policy is a lot more generous.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
My open-plan office is great until about 2 PM, when my floor gets messed up with chatty folks who are all trying to socialize their way through various after-lunch productivity plummets. I get it, and that's why I have all my 1:1s scheduled in that same time window, but it's really frustrating when I need to read notes.

Queen Victorian
Feb 21, 2018

I tend to head out for lunch around 1:30-2, so I miss other folks’ chatty unproductive post-lunch periods.

At my old office, it was dreary and awkwardly quiet all the time, so lunch timing didn’t matter (outside of hitting lunch places before they closed). No one chatted about anything (definitely not out loud, and not even a lot on Slack). I always felt like I was imposing if I tried to talk to anyone, so I generally didn’t. It sucked a lot.

Adbot
ADBOT LOVES YOU

Carbon dioxide
Oct 9, 2012

After lunch, we always go for a walk around the block.

Recently, a coworker even bought those trash grabber thingies, so we bring a baggie and a couple of those and clean up any trash we see on the way. The residents appreciate it, and some fresh air after lunch clears the mind right up. And no, it's not causing people to throw more trash on the streets in front of us. If anything, it's becoming less.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply