Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
pokeyman
Nov 26, 2006

That elephant ate my entire platoon.
aww I wanted to hear what happened to my dipshit toddler

Adbot
ADBOT LOVES YOU

Sapozhnik
Jan 2, 2005

Nap Ghost
yeah i was on the edge of my seat although i think the examples could have been even more condescending than they already were, which kind of detracted from the experience

raminasi
Jan 25, 2005

a last drink with no ice
one of the most astonishing things I learned after getting a job in fintech was how much post-facto patching up of speculative transactions there is in finance.

jeffery
Jan 1, 2013
go on...

MononcQc
May 29, 2007

Sapozhnik posted:

yeah i was on the edge of my seat although i think the examples could have been even more condescending than they already were, which kind of detracted from the experience

yeah the condescension is why I stopped. This thread is supposed to be a safe space anyway and I was going over the limit there.

Workaday Wizard
Oct 23, 2009

by Pragmatica

MononcQc posted:

yeah the condescension is why I stopped. This thread is supposed to be a safe space anyway and I was going over the limit there.

don't think :justpost:

VikingofRock
Aug 24, 2008




I read the story and found it to be illuminating, but also I kept getting distracted by thinking about what it would be like to live in a country where, if I go to a random ER, my doctor there automatically has access to my medical records

gonadic io
Feb 16, 2011

>>=

VikingofRock posted:

I read the story and found it to be illuminating, but also I kept getting distracted by thinking about what it would be like to live in a country where, if I go to a random ER, my doctor there automatically has access to my medical records

All the effort of getting my lawyer to sign my dnr with an ink gun wasted

mystes
May 31, 2006

VikingofRock posted:

I read the story and found it to be illuminating, but also I kept getting distracted by thinking about what it would be like to live in a country where, if I go to a random ER, my doctor there automatically has access to my medical records
As an American, I imagine that if I have a medical emergency I'll die before even getting to the ER because I'll have to spend hours trying to figure out which ER is in-network on my insurance plan using my insurance company's useless website.

Fiedler
Jun 29, 2002

I, for one, welcome our new mouse overlords.

mystes posted:

As an American, I imagine that if I have a medical emergency I'll die before even getting to the ER because I'll have to spend hours trying to figure out which ER is in-network on my insurance plan using my insurance company's useless website.

don't forget to also verify for in-network providers you might need -- surgeon, anesthetist, radiologist...

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

Fiedler posted:

don't forget to also verify for in-network providers you might need -- surgeon, anesthetist, radiologist...
and also make sure that if any in-network providers are off work or on vacation, their replacement is also in-network

DaTroof
Nov 16, 2000

CC LIMERICK CONTEST GRAND CHAMPION
There once was a poster named Troof
Who was getting quite long in the toof
if all else fails leave a note for your next of kin to look for an in-network mortician

Shaggar
Apr 26, 2006

MononcQc posted:

Yes, it's a thing called data replication. People can work with local redundant copies, which get synced, repaired, and updated across regions once the network heals.

If you have ever been to a pharmacy like CVS to get a prescription and the pharmacy had their own independent system instead of using the same exact records as general practitioner you have, you may inadvertently used one such system where the pharmacists work on their own copy of the data rather than updating your doctor's records directly in their computers.

they absolutely do not have everyones records and in fact that would be a hipaa violation. The only way CVS would have your records is if they've been authorized to look at them and then that would require an active internet connection. In your scenario where you've taken out the internet for the provider, be it hospital, pharmacy, or whatever, unless you have previously interacted with them they are not going to have access to your records if their access to the HIE over the internet is down. Even if they have previously interacted with you theres no guarantee they will have up to date records.

If the us were a tiny monoculture then maybe pushing records to everyone provider everywhere might be viable but in the US its absolutely not nor would I want it to be. Its a massive security risk.

HoboMan
Nov 4, 2010

i am just saying most users would be pretty pissed if like twitter dropped a tweet

Chalks
Sep 30, 2009

Dropping a tweet vs not everyone being able to see it for a period of time when the system is under high load. I'd imagine twitter may well have systems to prioritise "trending" content over messages posted by brand new accounts with zero followers or something.

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

HoboMan posted:

i am just saying most users would be pretty pissed if like twitter dropped a tweet
lots of people are extremely pissed off whenever their follower counts fluctuate due to a twitter bug

MononcQc
May 29, 2007

Shaggar posted:

they absolutely do not have everyones records and in fact that would be a hipaa violation. The only way CVS would have your records is if they've been authorized to look at them and then that would require an active internet connection. In your scenario where you've taken out the internet for the provider, be it hospital, pharmacy, or whatever, unless you have previously interacted with them they are not going to have access to your records if their access to the HIE over the internet is down. Even if they have previously interacted with you theres no guarantee they will have up to date records.

If the us were a tiny monoculture then maybe pushing records to everyone provider everywhere might be viable but in the US its absolutely not nor would I want it to be. Its a massive security risk.

yeah what I'm saying is that if your pharmacist has some data about your prescriptions and so does your doctor, you already live in a system with inconsistent data sets about your healthcare. It happens at a systemic level above any single database or program implementation. The gap between the systems is bridged by humans and paperwork. The consistent data is not useful for the computer unless it is also useful to the practitioners that use the computers.

Even in hospitals here, or in geriatric residences, sometimes they will call in a pharmacist to go over the health files of a patient, and then re-organize their prescriptions -- over time, prescriptions for various illnesses get to be added up over each other, and sometimes prescriptions are then re-added just to cover other effects of other medication. A pharmacist going over the whole medical history and current precriptions can drop lots of medication and negative side-effects by just being able to take a holistic view of things and going "if we switch these two, then the following side-effects are controlled and we can drop the following 2 other prescriptions as well." So you go for patients taking 23 pills a day down to 11-12 maybe.

That holistic view about the system is where you often find out that yes, your database is fully consistent and transactions are well done, but the data it contains is still out-of-sync with the real world and measures are already in place to cover for these. Once that repair mechanism exists, you find out that you can also readjust things on the tech side and in fact make for easier software because it better represents the real world.

The thing I've been telling people I've taught distributed systems practice to is that it is generally much easier to rework the problem and the approaches taken to match an inconsistent real world and expose actionable options to operators than it is to make the system bulletproof to begin with by constraining things.

For example, if you're reading sensor data and generating alarms and delays or bad connectivity makes you lose events or delay events in some control panel for devices in the field, you can try to patch up the timeline, delay the ordering, and provide one super clean timeline with only valid events. You can invest in better networking and so on. But then you may have delays in error reporting that could be more urgent because you were trying to prevent bad reporting and provide a good consistent view.

If you build a super consistent database to store the events and the alarms themselves, you're using a fully consistent system to store totally inconsistent data! The only gain you have is some attempt at an upper boundary on the data being made weirder, but you've got garbage in already, you'll have a hard time never getting garbage out.

Alternatively, you could embrace the chaos: relay the info you get and raise alarms when you can; go "events lost for period xyz" or if events are just late, raise an alarm with a timestamp in the past that links to the event data in full sorted order so that they can go look at the "historical alarm data" and validate it themselves.

You can even go hybrid and provide a slider that lets people adjust for "speed of detection" vs. "accuracy of detection" that just controls how long you buffer to reorder events before handling for alarms. Humans will probably like that better than the super clever accurate solution that sometimes breaks in unexpected ways because operators can't easily form a reliable mental model for the system.

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

MononcQc posted:

yeah what I'm saying is that if your pharmacist has some data about your prescriptions and so does your doctor, you already live in a system with inconsistent data sets about your healthcare. It happens at a systemic level above any single database or program implementation. The gap between the systems is bridged by humans and paperwork. The consistent data is not useful for the computer unless it is also useful to the practitioners that use the computers.

Even in hospitals here, or in geriatric residences, sometimes they will call in a pharmacist to go over the health files of a patient, and then re-organize their prescriptions -- over time, prescriptions for various illnesses get to be added up over each other, and sometimes prescriptions are then re-added just to cover other effects of other medication. A pharmacist going over the whole medical history and current precriptions can drop lots of medication and negative side-effects by just being able to take a holistic view of things and going "if we switch these two, then the following side-effects are controlled and we can drop the following 2 other prescriptions as well." So you go for patients taking 23 pills a day down to 11-12 maybe.

That holistic view about the system is where you often find out that yes, your database is fully consistent and transactions are well done, but the data it contains is still out-of-sync with the real world and measures are already in place to cover for these. Once that repair mechanism exists, you find out that you can also readjust things on the tech side and in fact make for easier software because it better represents the real world.

The thing I've been telling people I've taught distributed systems practice to is that it is generally much easier to rework the problem and the approaches taken to match an inconsistent real world and expose actionable options to operators than it is to make the system bulletproof to begin with by constraining things.

For example, if you're reading sensor data and generating alarms and delays or bad connectivity makes you lose events or delay events in some control panel for devices in the field, you can try to patch up the timeline, delay the ordering, and provide one super clean timeline with only valid events. You can invest in better networking and so on. But then you may have delays in error reporting that could be more urgent because you were trying to prevent bad reporting and provide a good consistent view.

If you build a super consistent database to store the events and the alarms themselves, you're using a fully consistent system to store totally inconsistent data! The only gain you have is some attempt at an upper boundary on the data being made weirder, but you've got garbage in already, you'll have a hard time never getting garbage out.

Alternatively, you could embrace the chaos: relay the info you get and raise alarms when you can; go "events lost for period xyz" or if events are just late, raise an alarm with a timestamp in the past that links to the event data in full sorted order so that they can go look at the "historical alarm data" and validate it themselves.

You can even go hybrid and provide a slider that lets people adjust for "speed of detection" vs. "accuracy of detection" that just controls how long you buffer to reorder events before handling for alarms. Humans will probably like that better than the super clever accurate solution that sometimes breaks in unexpected ways because operators can't easily form a reliable mental model for the system.
the problems you are discussing go far beyond the difference between sql and nosql

MononcQc
May 29, 2007

anthonypants posted:

the problems you are discussing go far beyond the difference between sql and nosql

They do. But the idea that "lol nosql you can't have inconsistencies" I think is reductive.

I think the best example for that is source control -- git, svn or whatever.

They do not provide a consistent view. Everyone works on a local version of the code, all copies and visions diverge, and only once they are committed and merged in the same servers does the tool expose the underlying data conflicts to the user (if it can't fix them), and you, the operator, are called to the rescue. The storage model is not transactional, it is not based on two-phase commits or on quorums. It is a historical log of edits and decisions, with complete support for users to alter and add more fixes on top of it.

Can you imagine building a source control tool on a relational SQL model only? Chances are you'd have to just retrofit the log-append structure onto the relational model and you'd lose all useful guarantees anyway.

I find it kind of baffling that so many people dismiss anything not-SQL so willingly when their own tools do not implement or use that model for the most part. Adopt a solution that fits the problem space.

Shaggar
Apr 26, 2006
In modern healthcare the way it works with HIEs is your PCP has pretty much everything going on with you and then if you visit elsewhere the new provider can pull the consistent records for you from the HIE and then push their results back to anyone interested in your data (your pcp). This results in accurate and up to date information. Everything is basically immediately consistent or not used.

The whole reason you're calling the pharmacist to confirm an order is because you need immediate consistency in the view of the patient's history. if you're stuck waiting for eventual consistency you'll be too late to help the patient. Theres also no real way to say that in the event of a network issue some data is more important than other data so you should sync the important data first. Its a clinical decision the computer cant make.

Once the data is consistent then the computer can help and can do things like point out medication errors. That is, if the humans using the system don't ignore it

And when you're talking about the data within the local emr, consistency is always more important than performance and if performance is a problem in your local instance it means you've hosed something up. like maybe you used a javascript based back end.

Even in the overall HIE performance shouldn't really be an issue since there really isn't much going on in there. Certainly not like what youd see in a financial or social network.

Your alarm scenario is totally different. If dropping alarms is ok then you never needed a consistent view to begin. Theres no such thing as data you can drop for later in healthcare.

Fiedler
Jun 29, 2002

I, for one, welcome our new mouse overlords.

MononcQc posted:

Can you imagine building a source control tool on a relational SQL model only? Chances are you'd have to just retrofit the log-append structure onto the relational model and you'd lose all useful guarantees anyway.
I see you're unfamiliar with TFVC.

Shaggar
Apr 26, 2006

MononcQc posted:

They do. But the idea that "lol nosql you can't have inconsistencies" I think is reductive.

I think the best example for that is source control -- git, svn or whatever.

They do not provide a consistent view. Everyone works on a local version of the code, all copies and visions diverge, and only once they are committed and merged in the same servers does the tool expose the underlying data conflicts to the user (if it can't fix them), and you, the operator, are called to the rescue. The storage model is not transactional, it is not based on two-phase commits or on quorums. It is a historical log of edits and decisions, with complete support for users to alter and add more fixes on top of it.

Can you imagine building a source control tool on a relational SQL model only? Chances are you'd have to just retrofit the log-append structure onto the relational model and you'd lose all useful guarantees anyway.

I find it kind of baffling that so many people dismiss anything not-SQL so willingly when their own tools do not implement or use that model for the most part. Adopt a solution that fits the problem space.

Git allows you to throw away consistency so its a bad example, but you could definitely implement something like svn in sql. The server is always consistent and the clients are either up to date or out of sync. Theres no situation where a client is ever consistent and the server is not.

MononcQc
May 29, 2007

Shaggar posted:

Git allows you to throw away consistency so its a bad example, but you could definitely implement something like svn in sql. The server is always consistent and the clients are either up to date or out of sync. Theres no situation where a client is ever consistent and the server is not.

when you have written code that is not yet committed, that is an inconsistent situation. You might not be able to pull in server changes without corrupting, ditching, or adapting local state.

Google docs could give you a fully consistent view, since all edits to the document happen live in there. Source control does not let you do that.

Shaggar
Apr 26, 2006
code that is uncommitted is effectively non-existent and should be treated as such. If you treat your uncommitted code as a valid state you start running into problems caused by people getting way out of sync without realizing it.

MononcQc
May 29, 2007

if you don’t consider clients part of the distributed system, you have an incomplete view of your distributed system, and end up pushing that complexity back onto the users.

Mao Zedong Thot
Oct 16, 2008


MononcQc posted:


I find it kind of baffling that so many people dismiss anything not-SQL so willingly when their own tools do not implement or use that model for the most part. Adopt a solution that fits the problem space.

In fairness you were replying to shaggar who is literally a gimmick poster.

Youre totally correct, but 'Adopt a solution that fits the problem space' only comes from lots of hard won experience, usually.

DaTroof
Nov 16, 2000

CC LIMERICK CONTEST GRAND CHAMPION
There once was a poster named Troof
Who was getting quite long in the toof

MononcQc posted:

yeah what I'm saying is that if your pharmacist has some data about your prescriptions and so does your doctor, you already live in a system with inconsistent data sets about your healthcare. It happens at a systemic level above any single database or program implementation. The gap between the systems is bridged by humans and paperwork. The consistent data is not useful for the computer unless it is also useful to the practitioners that use the computers.

Even in hospitals here, or in geriatric residences, sometimes they will call in a pharmacist to go over the health files of a patient, and then re-organize their prescriptions -- over time, prescriptions for various illnesses get to be added up over each other, and sometimes prescriptions are then re-added just to cover other effects of other medication. A pharmacist going over the whole medical history and current precriptions can drop lots of medication and negative side-effects by just being able to take a holistic view of things and going "if we switch these two, then the following side-effects are controlled and we can drop the following 2 other prescriptions as well." So you go for patients taking 23 pills a day down to 11-12 maybe.

That holistic view about the system is where you often find out that yes, your database is fully consistent and transactions are well done, but the data it contains is still out-of-sync with the real world and measures are already in place to cover for these. Once that repair mechanism exists, you find out that you can also readjust things on the tech side and in fact make for easier software because it better represents the real world.

The thing I've been telling people I've taught distributed systems practice to is that it is generally much easier to rework the problem and the approaches taken to match an inconsistent real world and expose actionable options to operators than it is to make the system bulletproof to begin with by constraining things.

For example, if you're reading sensor data and generating alarms and delays or bad connectivity makes you lose events or delay events in some control panel for devices in the field, you can try to patch up the timeline, delay the ordering, and provide one super clean timeline with only valid events. You can invest in better networking and so on. But then you may have delays in error reporting that could be more urgent because you were trying to prevent bad reporting and provide a good consistent view.

If you build a super consistent database to store the events and the alarms themselves, you're using a fully consistent system to store totally inconsistent data! The only gain you have is some attempt at an upper boundary on the data being made weirder, but you've got garbage in already, you'll have a hard time never getting garbage out.

Alternatively, you could embrace the chaos: relay the info you get and raise alarms when you can; go "events lost for period xyz" or if events are just late, raise an alarm with a timestamp in the past that links to the event data in full sorted order so that they can go look at the "historical alarm data" and validate it themselves.

You can even go hybrid and provide a slider that lets people adjust for "speed of detection" vs. "accuracy of detection" that just controls how long you buffer to reorder events before handling for alarms. Humans will probably like that better than the super clever accurate solution that sometimes breaks in unexpected ways because operators can't easily form a reliable mental model for the system.

maybe two whole people will finish reading this bullshit

Corla Plankun
May 8, 2007

improve the lives of everyone
when i was a kid i made a bot that tagged people back in a text based game when they tagged me, and then someone else made the another bot that did the same thing and oops all tagging

im having flashbacks

Shaggar
Apr 26, 2006

MononcQc posted:

if you don’t consider clients part of the distributed system, you have an incomplete view of your distributed system, and end up pushing that complexity back onto the users.

source control is not a distributed system

bob dobbs is dead
Oct 8, 2017

I love peeps
Nap Ghost
there are a million ways to get shaggared and we hit upon a very unexpected one today

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

MononcQc posted:

if you don’t consider clients part of the distributed system, you have an incomplete view of your distributed system, and end up pushing that complexity back onto the users.
yes, this is why your argument is loving stupid when you're trying to compare it to sql vs nosql. if the user can't decide whether they want the green one or the orange one, or if the nurse hasn't sent the prescription from the doctor's office to the pharmacy yet, that's not what you'd call an inconsistency as it relates to databases.

Progressive JPEG
Feb 19, 2003

anthonypants posted:

the problems you are discussing go far beyond the difference between sql and nosql

anthonypants posted:

yes, this is why your argument is loving stupid when you're trying to compare it to sql vs nosql.

you seem weirdly angry about this

the issues theyre describing are way more important than whether the query engine follows sql semantics

cinci zoo sniper
Mar 15, 2013




Progressive JPEG posted:

you seem weirdly angry about this

the issues theyre describing are way more important than whether the query engine follows sql semantics

i think the problem is monoqc chose to condescendingly write 5000 words when it can be done in a couple dry sentences to get general idea across, so no one bothers to read them through and pierce the story together

Progressive JPEG
Feb 19, 2003

Shaggar posted:

code that is uncommitted is effectively non-existent and should be treated as such. If you treat your uncommitted code as a valid state you start running into problems caused by people getting way out of sync without realizing it.

whats branching precious

HoboMan
Nov 4, 2010

Progressive JPEG posted:

whats branching precious

to be fair he is talking about svn

Progressive JPEG
Feb 19, 2003

cinci zoo sniper posted:

i think the problem is monoqc chose to condescendingly write 5000 words when it can be done in a couple dry sentences to get general idea across, so no one bothers to read them through and pierce the story together

eh that stuff is v complicated so i think the length is warranted, and its good material despite the condescending start

HoboMan posted:

to be fair he is talking about svn

oic. given how busted merging anything is in svn, i'd have assumed its users would be even more knowledgeable than git users about the perils of inconsistent state

cinci zoo sniper
Mar 15, 2013




Progressive JPEG posted:

eh that stuff is v complicated so i think the length is warranted, and its good material despite the condescending start
this is internet not some academic institute, not everyone is going to work through condescending start to see if the last paragraph is okay

jony neuemonic
Nov 13, 2009

i read the posts. they’re good and you should work on your attention span.

cinci zoo sniper
Mar 15, 2013




jony neuemonic posted:

i read the posts. they’re good and you should work on your attention span.

my attention span is perfectly fine, the posts just didn’t sell themselves

Adbot
ADBOT LOVES YOU

Chopstick Dystopia
Jun 16, 2010


lowest high and highest low loser of: WEED WEE
k

cinci zoo sniper posted:

my attention span is perfectly fine, the posts just didn’t sell themselves

To each their own I suppose, entertaining and thought provoking works for me.

  • Locked thread