Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
smoke sumthin bitch
Dec 14, 2009

by FactsAreUseless

SimonChris posted:

Also, the IBD/TIPP tracking poll really is the most accurate poll ever!

no poo poo that was the only unrigged poll out there even nates was rigged as hell and yall hated on him

Adbot
ADBOT LOVES YOU

ErIog
Jul 11, 2001

:nsacloud:
I ate the only probation in the history of this pre-9/11 regged account defending Nate Silver's uncertainty and I now feel less bad about doing so.

On the other hand.. literally loving weeping in the arms of my wife over the America I knew being dead and buried by a Cheeto Fascist hatemonger. Was shook for weeks over this poo poo. Nearly lost friends who told me I was overreacting. Suddenly, today, for some reason they're not telling me that. Wonder why.

Literally not an ounce of hyperbole, this feels like 9/11 happened again while my dad died on the same day. Every time I get a 10 minute reprieve not thinking about it, it comes flooding back like trauma. Gotta get used to the fact that things are very different now. Gotta get used to the fact that thousands will most likely die when they repeal Obamacare. 9/11 killed 3,000. Repealing Obamacare kills 80,000 people, by conservative estimates, if repealed first thing during a Trump presidency.

Oh wait, there's more!

They control all levels of government now. So any good thing Obama ever did never loving happened. Obama's post-presidency "lets skullfuck gerrymandering" tour is also now completely dead in the water.

ErIog has issued a correction as of 17:25 on Nov 9, 2016

Peel
Dec 3, 2007

the ~90% predictions were pretty reasonable and can semi-plausibly claim to have been unlucky (even wang suggested he could get to such a number with slightly different assumptions) but lol at the 99% crew

Kudaros
Jun 23, 2006
I'm trying to find a poll that asked about "financial situation compared to four years ago". I think it was done by CNN. Kind of vague, but if anyone can link me to that it would be great. Having a hard time searching for it.

Vox Nihili
May 28, 2008

http://election.princeton.edu/2016/11/09/aftermath/

let those who stood with nate throughout now find choice posts to mock ITT

Xandu
Feb 19, 2006


It's hard to be humble when you're as great as I am.

Fangz posted:

Nate's disgraced in the eyes of idiots like you, maybe, but correctly calling out uncertainty is still important. PEC and Cohn have some deep explaining to do, and 538 might be downsized, but he should be fine.

If not polling, general data journalism will have an audience as we chronicle the full Trumponomics clusterfuck over the next few years.

He called out uncertainty in the final days, but still wildly missed it. If the data is trash to begin with, what use does a data aggregator have?

Vox Nihili
May 28, 2008

Xandu posted:

He called out uncertainty in the final days, but still wildly missed it. If the data is trash to begin with, what use does a data aggregator have?

He reminded us that the data actually was trash and that we couldn't really draw a 95% conclusion. That's something, at least.

Patter Song
Mar 26, 2010

Hereby it is manifest that during the time men live without a common power to keep them all in awe, they are in that condition which is called war; and such a war as is of every man against every man.
Fun Shoe
He kept insisting that people mocking Trump for campaining in MI, MN, WI, and PA were wrong and that Trump had a good shot in those states. Trump won 3 of them.

Old Kentucky Shark
May 25, 2012

If you think you're gonna get sympathy from the shark, well then, you won't.


Xandu posted:

He called out uncertainty in the final days, but still wildly missed it. If the data is trash to begin with, what use does a data aggregator have?

He pointed out for pretty much the entire general election that bias was more likely to be systematic than not, hence the cross-state correlations that Wang poopooed as creating unnecessary noise but which ended up telegraphing accurate uncertainty. He also pointed out numerous times that the Blue Wall was mythical and fragile.

Nate actually ended up doing s pretty good job with data that was bad.

dwarf74
Sep 2, 2012



Buglord

Old Kentucky Shark posted:

Nate actually ended up doing s pretty good job with data that was bad.
Yeah. I can't give him too much crap for being more confident than deserved, given the broad, systemic polling failures nationwide. Those aren't his fault.

His model was pointing at a deeper story that the polls weren't catching, and his model was right.

frankenfreak
Feb 16, 2007

I SCORED 85% ON A QUIZ ABOUT MONDAY NIGHT RAW AND ALL I GOT WAS THIS LOUSY TEXT

#bastionboogerbrigade

Vox Nihili posted:

http://election.princeton.edu/2016/11/09/aftermath/

let those who stood with nate throughout now find choice posts to mock ITT
Sh- Shook Sam?

FuturePastNow
May 19, 2014


Nate was right

Hail Nate

Schwarzwald
Jul 27, 2004

Don't Blink

Vox Nihili posted:

http://election.princeton.edu/2016/11/09/aftermath/

let those who stood with nate throughout now find choice posts to mock ITT

This is a small thing, but I respect Sam Wang quite a bit more for owning his failure.

Eat the bug, Sam.

Xandu
Feb 19, 2006


It's hard to be humble when you're as great as I am.

Old Kentucky Shark posted:

He pointed out for pretty much the entire general election that bias was more likely to be systematic than not, hence the cross-state correlations that Wang poopooed as creating unnecessary noise but which ended up telegraphing accurate uncertainty. He also pointed out numerous times that the Blue Wall was mythical and fragile.

Nate actually ended up doing s pretty good job with data that was bad.

He may have known that, but his model gave her a 75% win chance or so through 10pm last night. Don't publish poo poo models if you know better.

Peel
Dec 3, 2007

Xandu posted:

He may have known that, but his model gave her a 75% win chance or so through 10pm last night. Don't publish poo poo models if you know better.

That's not a wild miss. He predicted that Trump had a 25% chance of winning the election, and that 25% chance came to pass. Well within plausibility.

People make fun of him for hedging, but he was correct to hedge, because the evidence was inconclusive. A good model will return uncertainty given inconclusive data, and his did.

Schwarzwald
Jul 27, 2004

Don't Blink

Xandu posted:

He may have known that, but his model gave her a 75% win chance or so through 10pm last night. Don't publish poo poo models if you know better.

If you're looking for a model that can predict the future, you're not going to find it.

Trabisnikof
Dec 24, 2005


This proves politics == sports I'm pretty sure

joepinetree
Apr 5, 2012
He didn't know better. Models set to be underconfident will underperform when the favorite wins and overperform when it loses. Overconfident models will do the opposite. Hence Sam Wang did much better in 2012 and in the Republican primaries and Nate did less worse in the general. Wang's model performed worse because of his overconfidence, but he was still closer on the popular vote than Nate.

Once again, in terms of real winners this cycle, its the economic fundamentals models. Ray Fair's and Alan Abramowitz's models picked Trump.

Peel
Dec 3, 2007

I think I'd say most of all that we just don't have enough data on the performance of the models themselves to make a good judgement, and likely never will, except for the overconfident ones being overconfident.

Like maybe 538 is systematically underconfident but we wouldn't be able to tell without dozens of trials, which is a lot of elections to sit through, in which time there will be new factors in the models and old ones will have become less important.

joepinetree
Apr 5, 2012
We may not know about overconfidence and underconfidence in relation to the objective reality, but we can know in relation to each other. There was very little difference in terms of who was favored to win in each state across models. My point is if we have that, but differences in confidence, then when the favorite wins the overconfident model will look good and vice versa. But being the underconfident model doesn't mean it is a good model when the underdog wins.

Progressive JPEG
Feb 19, 2003


Remember it was the same story with the Red Sox and Bush in 2004.

PerniciousKnid
Sep 13, 2006
I feel like a lot of people are swinging from "polls can tell us who will win in three months with uncanny accuracy" to "polls are worthless garbage" as though there's no in-between.

Pomplamoose
Jun 28, 2008

joepinetree posted:

Once again, in terms of real winners this cycle, its the economic fundamentals models. Ray Fair's and Alan Abramowitz's models picked Trump.

What economic fundamentals tilted the race in Trump's favor? I thought the most recent jobs report was positive but that might just be the NPR talking.

Schwarzwald
Jul 27, 2004

Don't Blink

PerniciousKnid posted:

I feel like a lot of people are swinging from "polls can tell us who will win in three months with uncanny accuracy" to "polls are worthless garbage" as though there's no in-between.

It's the same general line of thinking that keeps casino's in business.

joepinetree
Apr 5, 2012

Sebadoh Gigante posted:

What economic fundamentals tilted the race in Trump's favor? I thought the most recent jobs report was positive but that might just be the NPR talking.

GDP per capita growth was sluggish for a party seeking a 3rd straight term.

Vox Nihili
May 28, 2008

joepinetree posted:

GDP per capita growth was sluggish for a party seeking a 3rd straight term.

Lol.

joepinetree
Apr 5, 2012

https://fairmodel.econ.yale.edu/vote2016/index2.htm

joepinetree has issued a correction as of 03:56 on Nov 10, 2016

Carmant
Nov 23, 2015


Treadmill? What's that? Is that some kind of cake?


ErIog posted:

On the other hand.. literally loving weeping in the arms of my wife over the America I knew being dead and buried by a Cheeto Fascist hatemonger.

:smugdon:

ErIog
Jul 11, 2001

:nsacloud:

PerniciousKnid posted:

I feel like a lot of people are swinging from "polls can tell us who will win in three months with uncanny accuracy" to "polls are worthless garbage" as though there's no in-between.

The thing I think is most loving toxic about what happened here is saying, "wow gosh all these polls were pretty hosed.." and then like 2 seconds later, "well black people didn't show up and white women voted for Trump." Too many Dems are spending a lot of time parsing bullshit polls to figure out who they can blame for this happening. Not shockingly the blame game ends with the arrow pointing in the general direction of "minorities."

If they directed some of the energy outward at the GOP stooges for abetting him, media for elevating him like he was a real candidate from the second he came down the escalator, and literally all of the bad actors who conspired to get him elected then maybe we have a shot of stopping him.

Instead everyone's pickin' up their BB guns and ready for the circular firing squad.

NYT loving partnered with Clinton Cash guy over bullshit claims about the Clinton Foundation. Their behavior here is the same kind of behavior that led us into the Iraq war.

ErIog has issued a correction as of 04:53 on Nov 10, 2016

Golden Bee
Dec 24, 2009

I came here to chew bubblegum and quote 'They Live', and I'm... at an impasse.
Hillary lost Obama voters and the buck stops with the (theoretical) president.

Fangz
Jul 5, 2007

Oh I see! This must be the Bad Opinion Zone!

I wait for you to embarrass yourself in the next election then. Edit: Your conclusion for next election in that model will probably be that the democrats don't have to do *anything*, any candidate will win, full stop.


quote:

The thing I think is most loving toxic about what happened here is saying, "wow gosh all these polls were pretty hosed.." and then like 2 seconds later, "well black people didn't show up and white women voted for Trump." Too many Dems are spending a lot of time parsing bullshit polls to figure out who they can blame for this happening. Not shockingly the blame game ends with the arrow pointing in the general direction of "minorities."

Nate has a pretty good post up on that:

http://fivethirtyeight.com/features/what-a-difference-2-percentage-points-makes/

In any case what Nate basically shows us is the value of proper statistics. His model wasn't 'underconfident' - it was correctly calibrated. As he (and I) warned, polls do consistently miss by a few percent with each round, and the electoral college situation this time round was not safe for Hillary this year. He never fudged the data, he didn't add uncertainty for clicks. He modeled correctly the large number of undecided voters and the correlated relationship of state errors.

One point Nate made that everyone missed is that in theory, you should campaign on not the states close to flipping, but states that *most match the national polling average*. This is the choice that is robust to a systematic polling miss. Because those states might not look like swing states in the average, but under a lot of error scenarios they could be.

Fangz has issued a correction as of 13:41 on Nov 10, 2016

Fangz
Jul 5, 2007

Oh I see! This must be the Bad Opinion Zone!
In the next few weeks and months everyone is going to blame everyone except themselves. The polling data will be used as ammunition for that fight when really they don't contain any real information in that issue. Sigh - that's the problem with data science: interpretation is everything in the end, and maths doesn't help.

I'm totally blameless in this whole affair btw. :)

Fangz has issued a correction as of 13:37 on Nov 10, 2016

ErIog
Jul 11, 2001

:nsacloud:

Yeah.. well aware and I like Nate, but like.. 2% is not the story of this loving election. Turnout is and the polls got the turnout way wrong, and exit polls have always been problematic. So.. yeah he did a better job parsing the polls and pointing out flaws in them. It's a big loving leap to start making conclusions about that stuff less than the margin of error, but a lot of people seem very motivated to do so.

People want a blame narrative. Blaming Trump or blaming whites who tacitly endorsed Trump by staying home isn't enough for some people. We gotta start shaming women voters or minority voters or other voters who bothered to loving show up.

I'm sorry, but I can't blame them. They showed up. They did their part. This is on white people who voted FOR Trump and for people who didn't find the racist fascist horrifying enough to go loving vote against him.

mastershakeman
Oct 28, 2008

by vyelkin
I don't have a stats background but my issue with what Nate did is essentially what Bill Mitchell did which is to go "the polls are bad go Trump". how can that even be modeled? Wangs model seems to have trusted the inputs because that's really all that can be done with a poll driven forecast.

joepinetree
Apr 5, 2012

Fangz posted:

I wait for you to embarrass yourself in the next election then. Edit: Your conclusion for next election in that model will probably be that the democrats don't have to do *anything*, any candidate will win, full stop.


Nate has a pretty good post up on that:

http://fivethirtyeight.com/features/what-a-difference-2-percentage-points-makes/

In any case what Nate basically shows us is the value of proper statistics. His model wasn't 'underconfident' - it was correctly calibrated. As he (and I) warned, polls do consistently miss by a few percent with each round, and the electoral college situation this time round was not safe for Hillary this year. He never fudged the data, he didn't add uncertainty for clicks. He modeled correctly the large number of undecided voters and the correlated relationship of state errors.

One point Nate made that everyone missed is that in theory, you should campaign on not the states close to flipping, but states that *most match the national polling average*. This is the choice that is robust to a systematic polling miss. Because those states might not look like swing states in the average, but under a lot of error scenarios they could be.

Are you stupid? You realize that i am not the author of that article, and that those types of models have been around forever. Im not "arguing" anything. Besides, jt is funny to see you misinterpret basic linear algebra even as you try to argue statistics. Go back and read that link, and if you are able to understand linear algebra tell if the conclusion is "any Democrat will win."

Fangz
Jul 5, 2007

Oh I see! This must be the Bad Opinion Zone!

mastershakeman posted:

I don't have a stats background but my issue with what Nate did is essentially what Bill Mitchell did which is to go "the polls are bad go Trump". how can that even be modeled? Wangs model seems to have trusted the inputs because that's really all that can be done with a poll driven forecast.

That's not what Nate did. As I understand it: He empirically estimated the level of correlation in state poll errors from past data. He modeled the variability in how undecided voters break for one candidate or another. If you observe that (I) you did have polls with Trump in the lead (II) polls miss by a few percent on average (III) the undecideds were very high then you can easily recreate his result.

Wang made a lot of mistakes. The biggest one is that he did not actually 'trust the inputs'. His model used the median of the polls. Which means that he implicitly and arbitrarily removed a bunch of polls from the model as outliers. He also made very strong assumptions on what the undecideds meant, and a stupid assumption on the uncorrelatedness of errors.

The funny thing is that as a neuroscientist, Wang really ought to have known better. We had a recent paper out about misuse of statistical methods in neuroscience, particularly pointing out mistreatment of correlation.

Fangz
Jul 5, 2007

Oh I see! This must be the Bad Opinion Zone!

joepinetree posted:

Are you stupid? You realize that i am not the author of that article, and that those types of models have been around forever. Im not "arguing" anything. Besides, jt is funny to see you misinterpret basic linear algebra even as you try to argue statistics. Go back and read that link, and if you are able to understand linear algebra tell if the conclusion is "any Democrat will win."

These types of model have been around forever and are always garbage that demonstrates the problem of overfitting and the statistical incompetence of political science departments.

Look at the variables in table 1 of the manuscript. NONE of the covariates consider anything about the candidates except their party status. If a Democrat is due to win by that model, ANY democrat will win.

Every year the media picks one new dude whose 'model predicts every election since 1940!!!!' not realising that overfitting makes them no better than a coin flip. Well it came up heads this time so I guess everyone thinks they are awesome now!!!

Fangz has issued a correction as of 14:54 on Nov 10, 2016

Nocturtle
Mar 17, 2007

mastershakeman posted:

I don't have a stats background but my issue with what Nate did is essentially what Bill Mitchell did which is to go "the polls are bad go Trump". how can that even be modeled? Wangs model seems to have trusted the inputs because that's really all that can be done with a poll driven forecast.

There are various ways to estimate systematic biases in data samples, it comes up a lot in analyses involving measurement data or that require comparisons between simulated and real datasets. One obvious method is historical comparisons of poll predictions and results in similar contests (for reference the 2016 primary polls weren't very accurate with a ~2-3% bias, although they did generally predict the winners correctly. I don't know how the 2012 general polls compared.).

However it's a dark art and will often rely on making assumptions that can only be defended in terms of plausibility as opposed to following a well-defined protocol. In principle this is where the forecasters provide some value, if it was trivial to evaluate systematic biases in data then there'd be no need to hire hot shot analysts/pundits. One lesson from this stupid election is that you can't trust poll results at face value. Everyone needs to internalize that on top of the sampling error there are likely very large systematic errors. Nate stands out for trying to evaluate these biases using data-driven methods and standing by them despite heavy criticism.

Typo
Aug 19, 2009

Chernigov Military Aviation Lyceum
The Fighting Slowpokes
538 is literally one of the most relevant websites in the US:

Adbot
ADBOT LOVES YOU

FMguru
Sep 10, 2003

peed on;
sexually
Nate is interesting because he was right about how there was a much larger chance of a massive systematic polling failure than anyone else was estimating, but that really undercuts the entire raison d'etre of his website.. It's great that he predicted that all this poll stuff had a good likelihood of being garbage, but I'm not sure that's going to be much of a selling point to his bosses at ESPN/ABC/Disney - "I was the only poll aggregator in the media who was right about how worthless all this polling stuff you pay us to do actually is" isn't a very strong pitch.

  • Locked thread