|
I dont think you guys understand how handicapping works.
|
#
?
Oct 1, 2016 04:23
|
|
|
|
| # ? Jun 8, 2024 19:31 |
|
|
If the Vegas line on a football game is even does that mean they don't know anything about the football teams?
|
|
#
?
Oct 1, 2016 04:23
|
|
|
Bip Roberts posted:If the Vegas line on a football game is even does that mean they don't know anything about the football teams? If your one job is that line, you failed.
|
|
#
?
Oct 1, 2016 04:25
|
|
|
Despera posted:If your one job is that line, you failed. I can no longer sense if irony is real.
|
|
#
?
Oct 1, 2016 04:26
|
|
|
Problem with a football game analogy is there are far fewer inputs, many more possible scores etc. Two football teams are nearly never equal (there like 3 pickems a year) two candidates less.
|
|
#
?
Oct 1, 2016 04:28
|
|
|
Hell, one could argue that a fault of 538's model in 08 and 12 was that it called too many states correctly. Iirc it had a few states as pretty close, and getting all but one of those near-coin flips right suggests to me that it was too conservative -- those weren't really coinflips.
|
|
#
?
Oct 1, 2016 04:33
|
|
|
Tayter Swift posted:Hell, one could argue that a fault of 538's model in 08 and 12 was that it called too many states correctly. Iirc it had a few states as pretty close, and getting all but one of those near-coin flips right suggests to me that it was too conservative -- those weren't really coinflips. Well for a perfectly constructed model getting all the states right should be the most likely single outcome even if it might be exceedingly unlikely overall.
|
|
#
?
Oct 1, 2016 04:35
|
|
|
Hm. Yeah I see what you mean although my intuition is fighting it
|
|
#
?
Oct 1, 2016 04:42
|
|
|
Bip Roberts posted:I dont think you guys understand how handicapping works. Bip Roberts posted:If the Vegas line on a football game is even does that mean they don't know anything about the football teams? These are fundamentally different things. They are not comparable.
|
|
#
?
Oct 1, 2016 05:19
|
|
|
Concerned Citizen posted:These are fundamentally different things. They are not comparable. Explain to me why a model shouldn't aim to predict the chances of potential outcomes?
|
|
#
?
Oct 1, 2016 05:32
|
|
|
Bip Roberts posted:Explain to me why a model shouldn't aim to predict the chances of potential outcomes? And a 50/50 outcome makes no prediction.
|
|
#
?
Oct 1, 2016 05:54
|
|
|
Despera posted:And a 50/50 outcome makes no prediction. wtf
|
|
#
?
Oct 1, 2016 06:07
|
|
|
Bip Roberts posted:Explain to me why a model shouldn't aim to predict the chances of potential outcomes? A model's input is a direct measurement of what it is aiming to predict (i.e. vote share.) It's fundamentally different from predicting a sports game. I also want to make a point re: "50% = tie vote." I will point out that if you rank order 538's states by % chance of a Democratic win, they will not match up with how close the actual race was. For example, Nate gave Obama a 90.6% chance of winning Ohio and a 79.4% chance of winning Virginia. But Ohio was almost a full percentage point closer than Virginia in the actual final result. But that doesn't mean Nate was wrong - it just means that the inputs he had gave a stronger level of confidence in him winning Virginia than Ohio. Because the percentage doesn't measure the closeness of the race, it measures the likelihood of a win.
|
|
#
?
Oct 1, 2016 06:10
|
|
|
Yes, they're right. If you say there's a 50% chance of something occurring with only two possible outcomes, it means your model doesn't know what is going to happen. It's the model equivalent of throwing your hands up in the air and saying "who knows?" The theoretical perfect model would only assess either a 0% or 100% chance to any outcome, and it would be correct every time.
|
|
#
?
Oct 1, 2016 06:12
|
|
|
Concerned Citizen posted:Yes, they're right. If you say there's a 50% chance of something occurring with only two possible outcomes, it means your model doesn't know what is going to happen. It's the model equivalent of throwing your hands up in the air and saying "who knows?" The theoretical perfect model would only assess either a 0% or 100% chance to any outcome, and it would be correct every time. I don't think you understand how data works.
|
|
#
?
Oct 1, 2016 06:15
|
|
|
Despera posted:And a 50/50 outcome makes no prediction. It predicts people staying up all night rocking themselves to sleep, crying.
|
|
#
?
Oct 1, 2016 06:16
|
|
|
Bip Roberts posted:I don't think you understand how data works. Darn, don't tell my boss.
|
|
#
?
Oct 1, 2016 06:16
|
|
|
Despera posted:And a 50/50 outcome makes no prediction. Concerned Citizen posted:Yes, they're right. If you say there's a 50% chance of something occurring with only two possible outcomes, it means your model doesn't know what is going to happen. It's the model equivalent of throwing your hands up in the air and saying "who knows?" The theoretical perfect model would only assess either a 0% or 100% chance to any outcome, and it would be correct every time.  Are these guys trolling?
|
|
#
?
Oct 1, 2016 06:17
|
|
|
Concerned Citizen posted:Darn, don't tell my boss. You make a prediction from finite noisy data. There is an inherent information limit from that.
|
|
#
?
Oct 1, 2016 06:18
|
|
|
Bip Roberts posted:You make a prediction from finite noisy data. There is an inherent information limit from that. Again, as I mentioned before - there is better, more precise data that is available in private analytics world. I do not criticize Nate Silver for making the best possible prediction with the data he has, but rather point out that other models make more useful predictions in this particular cycle. We originally got here because I pointed out that Nate Silver's model turned Florida blue because it was 50.3% and PEC went red because they had almost the same projection, but on the other side of 50. Both of them could have gone either way. In effect, both PEC and 538 performed identically in 2012 because neither made an actual solid prediction of what would happen in Florida. I think, too, that 538's model is going to be left in the dust soon. Many of the tools common in the private analytics world are making their way to the press. The TargetSmart/W&M and the Upshot/Siena polls are good examples of this. It's only a matter of time until modelers start utilizing the vast treasure trove of data from commercially-available enhanced voter files to make more accurate vote share and turnout predictions that don't rely on shoddy public polls. edit: from TargetSmart, this is an actual real-world partisanship model in Ohio. Notice that there are huge jumps at both ends of the model - that's by design! The model does not measure intensity of support (i.e. an Obama supporter with a score of 10 is not necessarily more conservative than an Obama supporter with a score of 90) but rather measures likelihood of support - if you talk to 100 people with a score of 90, 10 of them will not be supporters. People in the middle are often fierce partisans (although there are, statistically, more persuadable voters in that range), but we simply lack the data to tell for certain. Once, say, the Democratic Party speaks to them and finds out their thoughts on the election, a refresh would move their score in one direction or the other. This is analogous to 538's projections. 
Concerned Citizen has issued a correction as of 06:38 on Oct 1, 2016 |
|
#
?
Oct 1, 2016 06:25
|
|
|
Concerned Citizen posted:I think this is a good post, so I'd like to respond more thoroughly when I'm back on the computer but I'd point out several things: When Clinton was south of 60% in the odds, she was also a point or two max above Trump in the polls and on a downward trajectory, a trajectory that she had been on for nearly a month by that point. If the election were a football game, Clinton would have been up by three touchdowns in the middle of the second quarter (the end of August after the convention afterglow wore off), only to see Trump close to within a touchdown by the end of the third quarter (right before the debate but after all the horrible crap from September). Clinton started the fourth quarter (where we are now in this already strained analogy) by scoring a touchdown (winning the debate) and now she's just intercepted the ball and is driving for another score (getting Trump into a Twitter feud with a Latina immigrant over Trump's misogynistic comments). The point where it was 50-60% for Clinton was the point in the game where she was still up by a little bit, but the momentum was with Trump. Just like in my football analogy, anyone who could tell you that, from the data available at that moment, that they could tell you with certainty who is going to win is either an idiot or trying to sell you something. There are situations that, because of their complexity, a deterministic prediction cannot be made. While we all saw the reasons that Hillary could pull herself out of that downward spiral, there's no reason to think that it would necessarily happen, or that she wouldn't end up down by a touchdown or two before she did. As for other models, including private ones, I'm sure that you're right. However, there will always be some cases where it just won't be able to call something with certainty because it's just too close. Concerned Citizen posted:A model's input is a direct measurement of what it is aiming to predict (i.e. vote share.) It's fundamentally different from predicting a sports game. However, I've written at least once about it in this thread, but 538 uses percentages in a really baffling way. Sometimes they treat them like odds, and even use the field goal kicker analogy, but on a per-state basis, there's some really goofy numbers. For example, Hillary has a 5.9% chance of winning North Dakota right now in the Now-cast. This should mean that, if we could run the election multiple times in an infinite number of parallel universes, that Hillary would somehow come out ahead about one in twenty times. The closest Hillary has been in the polls is down 11 points. On the other side, Trump currently has a 10% chance in Delaware. I'm not sure how these numbers aren't literally <0.1%, with the <0.1% chance of some kind of freak thing, like Peter Thiel/George Soros going full super-villain and infecting the water in Democrat/Republican-heavy areas to drive down turnout enough for their candidates of choice to win. Azathoth has issued a correction as of 06:40 on Oct 1, 2016 |
|
#
?
Oct 1, 2016 06:33
|
|
|
Concerned Citizen posted:I think, too, that 538's model is going to be left in the dust soon. Many of the tools common in the private analytics world are making their way to the press. The TargetSmart/W&M and the Upshot/Siena polls are good examples of this. It's only a matter of time until modelers start utilizing the vast treasure trove of data from commercially-available enhanced voter files to make more accurate vote share and turnout predictions that don't rely on shoddy public polls. Can you expand on this a bit more, what makes those polls superior? W&M is my alma mater so I wanna know why my school is better 
|
|
#
?
Oct 1, 2016 06:37
|
|
|
Azathoth posted:However, I've written at least once about it in this thread, but 538 uses percentages in a really baffling way. Sometimes they treat them like odds, and even use the field goal kicker analogy, but on a per-state basis, there's some really goofy numbers. For example, Hillary has a 5.9% chance of winning North Dakota right now in the Now-cast. This should mean that, if we could run the election multiple times in an infinite number of parallel universes, that Hillary would somehow come out ahead about one in twenty times. The closest Hillary has been in the polls is down 11 points. On the other side, Trump currently has a 10% chance in Delaware. I'm not sure how these numbers aren't literally <0.1%, with the <0.1% chance of some kind of freak thing, like Peter Thiel/George Soros going full super-villain and infecting the water in Democrat/Republican-heavy areas to drive down turnout enough for their candidates of choice to win. I think part of the 538 model uncertainty that causes these wild swings is both the possibility of a large national swing between now and election day as well as the large number of undecideds/third parties and unknowns based on election day turnout. Nate clearly overestimates these compared to the field. On the other hand if Trump fully melts down leading to a real landslide it might not look too bad.
|
|
#
?
Oct 1, 2016 06:42
|
|
|
my bony fealty posted:Can you expand on this a bit more, what makes those polls superior? W&M is my alma mater so I wanna know why my school is better Public polling suffers from three major issues: 1. Non-response bias - Certain campaign events (like conventions) make supporters of one candidate more likely to answer a poll. This skews polling compositions. 2. Sampling bias - It's hard to get adequate samples of certain demographics, meaning that pollsters need to weigh the answers they get more heavily. This is a notorious issue in states like Nevada, where English-speaking Latinos that answer polls skew more Republican than the overall Latino vote. 3. Likely voter screens - Most public polls fundamentally use the idea that every likely voter (i.e. person who self-identify as a voter) will vote and every person who isn't a likely voter won't vote. But many people who say they will vote won't, and many people who say they won't vote actually will. (And, surprisingly, studies show that both sides actually lie at about the same rate!) To correct these issues, private pollsters use list-based samples (as opposed to randomly dialing digits in certain area codes) derived from voter files enhanced with purchased cell phone data and other information. If a private pollster can't get enough young voters, they can simply call more young people until enough answer to get an adequate sample. Moreover, they can use vote history combined with other data to create a turnout model and project a likely electorate. That means that, instead of weighting by demographics (as traditional pollsters do) they can weight their responses by how likely someone is to vote. This is what W&M and TargetSmart did in their Ohio poll: http://targetsmart.com/tswmp-ohio/ - TargetSmart is a major Democratic warehouse for data. They just buy up massive piles of private data. W&M used their lists to poll and thus eliminated these problems to create a more accurate poll.
|
|
#
?
Oct 1, 2016 06:51
|
|
|
Azathoth posted:Just because a model is less volatile does not mean that it's inherently more accurate. Sometimes there's volatility in a model because...well, the election is volatile or the available data doesn't allow an accurate prediction. Sure, I agree that a lack of volatility doesn't mean it's more accurate. Obviously, we could achieve 0 volatility by just running the model once in September and calling it a day. But modeling is meant to help us step back and see the big picture without getting consumed by the day-to-day details. But take the Democratic convention - we went from the poll-plus model projecting 60% to nearly 80% a week later. I don't actually think Hillary's chances of winning the election increased by that much, especially since we know from history that convention bounces eventually fade. And if we look back at 2012, one thing that is notable is that the 538 model showed a significant tightening of the race through October. We know from interviews with Obama's analytics staff in 2012 that they didn't see any such tightening - in fact, their massive data machine saw the race as fairly constant even after the disastrous first debate. They even knew they were going to win Florida in late September after accounting for their field staff's voter registration efforts. Here's one article that talks about it: quote:By Election Day, Wagner’s analytic tables turned into predictions. Before the polls opened in Ohio, authorities in Hamilton County, the state’s third-largest and home to Cincinnati, released the names of 103,508 voters who had cast early ballots over the previous month. Wagner sorted them by microtargeting projections and found that 58,379 had individual support scores over 50.1—that is, the campaign’s models predicted that they were more likely than not to have voted for Obama. That amounted to 56.4 percent of the county’s votes, or a raw lead of 13,249 votes over Romney. Early ballots were the first to be counted after Ohio’s polls closed, and Obama’s senior staff gathered around screens in the boiler room to see the initial tally. The numbers settled almost exactly where Wagner had said they would: Obama got 56.6 percent of the votes in Hamilton County. In Florida, he was as close to the mark; Obama’s margin was only two-tenths of a percent off. “After those first two numbers, we knew,” says Bird. “It was dead-on.” The point isn't that 538 can hope to compete with Obama's analytics - of course it can't. Their modeling cost tens of millions of dollars. But my point IS that the actual modeling saw very little volatility in the race while 538 moved around a lot. That means that 538 is prone to move when the actual electorate isn't moving - the model is supposed to remove the noise, but that noise is still there and it's affecting the picture of the race. Now, let's say the other models aren't moving as much. There's reasonable disagreement there as to whether the electorate is moving or 538 is simply more sensitive to noise. I object, however, to Nate's attitude toward pollsters and other modelers - oftentimes dismissing them when they commit the crime of, say, pointing out that he aggregates a ton of very low quality polls. It's one thing to say "I think the electorate is less stable in 2016 than 2012" and another to dismiss pollsters as dumb unskewers. quote:This is an excellent point. Well, I guess Republicans won Delaware in 1980, 1984, and 1988.. But really, I think the issue is that 538 should be saying "I have 90% confidence that Hillary wins Delaware" rather than "Hillary has a 90% chance of winning." That seems to be a lot closer to how their model works. Concerned Citizen has issued a correction as of 07:27 on Oct 1, 2016 |
|
#
?
Oct 1, 2016 07:25
|
|
|
Concerned Citizen posted:Public polling suffers from three major issues: If public polling is bad doesn't that mean a good model should have high uncertainty?
|
|
#
?
Oct 1, 2016 07:29
|
|
|
Bip Roberts posted:If public polling is bad doesn't that mean a good model should have high uncertainty? If you solely use public polling, sure. That would require first acknowledging that public polling is often flawed, which I think some data journalists are not excited about. To 538's credit, they do try to weigh better polls more heavily. Now, if you add in more inputs (for example, past election results) you can often (but not always) start increasing your level of certainty. 538's polls-plus is somewhat more stable than their polls-only model for that reason. But voter-level data is available from vendors like L3, so it's high time that data journalists (and public pollsters, for that matter) start incorporating that stuff into their models.
|
|
#
?
Oct 1, 2016 07:38
|
|
|
Concerned Citizen posted:Yes, they're right. If you say there's a 50% chance of something occurring with only two possible outcomes, it means your model doesn't know what is going to happen. It's the model equivalent of throwing your hands up in the air and saying "who knows?" The theoretical perfect model would only assess either a 0% or 100% chance to any outcome, and it would be correct every time. Nate did know what was going to happen in Florida, though. In the final week of the election, his model moved to a very slight Obama advantage, so he made a guess that Obama would win the state. Obama did win Florida and it was the closest state of 2012, with Romney having actually led in the RCP aggregate by 1.5% and Obama winning by .6%. This is what you expect good models to do, make close calls even in tight situations where simply looking at the polls will not give you the full answer. It picked up Obama's late movement there in a way that PEC and the others did not. You seem to be caught up that Silver wasn't 100% confident in his prediction because he didn't say Obama will win Florida hands-down, no questions asked, but to a statistician that kind of stance is pointless. You're always going to talk about things in terms of probabilities because predictive models give you varied results with the same exact inputs.
|
|
#
?
Oct 1, 2016 07:44
|
|
|
Azathoth posted:Er, I mean NAET SUKKS!!!! GET A HAIRCUT YOU HIPPIE! Oh, Nate is way ahead of you there.
|
|
#
?
Oct 1, 2016 08:13
|
|
|
Despera posted:If your model isnt any more useful than a random number generator, I'm going to call your model poo poo. You're a moron, dude.
|
|
#
?
Oct 1, 2016 09:48
|
|
|
Concerned Citizen posted:I don't think there's a significant issue with the day-to-day predictions of 538, but rather the insane twists and turns. A model is only useful if it allows you to predict what will happen on election day. If it changes every two days, it's not useful - you might as well just follow RCP's polling average. Part of the issue is one of garbage in, garbage out - the increased expenses of high quality, live polling means that we are being flooded with large numbers of poorly conducted polls. Emerson, for example, has done dozens of polls that are strictly IVR, landline-only. They just add "+2 points to liberal opinion" to account for cell phones. There's very good reason to think that private, proprietary polling, which is conducted in a very different (more expensive, higher quality, and taking advantage of non-public data) manner are just not seeing Hillary's booms and busts. Much of the movement can be attributed either to artifacts of LV models or to non-response bias. While private polling is able to effectively deal with those two issues (by using list-based polls and using that to get correctly sized samples across turnout ranges), public polling simply isn't. Your posts ITT are awesome fyi I haven't seen more than 1-2 private polls leak, though, and definitely nothing that's been as stable as you're saying. Do you have any publicly available sources that can confirm this? It's odd because I agree with the theory but thought I was out on a ledge by saying that. Is there a quantifiable estimate of how big(ly) non-response bias can theoretically get over the past few cycles?
|
|
#
?
Oct 1, 2016 09:54
|
|
|
I'm actually really interested how all the various models deal with the relatively large number of undecideds this election, it seems one of the most interesting questions.
|
|
#
?
Oct 1, 2016 13:20
|
|
|
Concerned Citizen posted:Sure, I agree that a lack of volatility doesn't mean it's more accurate. Obviously, we could achieve 0 volatility by just running the model once in September and calling it a day. But modeling is meant to help us step back and see the big picture without getting consumed by the day-to-day details. But take the Democratic convention - we went from the poll-plus model projecting 60% to nearly 80% a week later. I don't actually think Hillary's chances of winning the election increased by that much, especially since we know from history that convention bounces eventually fade. And if we look back at 2012, one thing that is notable is that the 538 model showed a significant tightening of the race through October. We know from interviews with Obama's analytics staff in 2012 that they didn't see any such tightening - in fact, their massive data machine saw the race as fairly constant even after the disastrous first debate. They even knew they were going to win Florida in late September after accounting for their field staff's voter registration efforts. Here's one article that talks about it: Points well taken, though how much of the lack of volatility in Obama's analytics was because of all the private polling and proprietary data collected and painstakingly curated by his campaign and the DNC? From a glance at RCP from 2012, it looks like the polling showed a whole lot of tightening in the aftermath of that debate. Obama's team had some information that said differently, and it was accurate, but how would any model that relies on public polling only get that kind of certainty? With the Democrat's internal stuff, aside from the ability to do polls as needed, on what they need to know exactly, there's the mountain of volunteers who collect and input data that can be integrated with the polling data and used to validate/invalidate the data. As for Nate's attitude, I'm not sure why it matters. I follow 538 via RSS, and don't look at Nate's (or anyone else's) twitter account except when it gets posted here or linked in some news article. I'm sure that I'm missing out on the hottest of the hot takes for doing this, along with a million other internet slapfights, not to mention Harry's never-ending quest to find true love, but I'm okay with that. There's plenty of writers out there, not just on 538, who put out good stuff when they've got an editor keeping them in check, but are terrible when that filter is off. Nate is a raging rear end in a top hat towards anyone who disagrees with him and Nate has a bad/sub-optimal/crazy model are two entirely separate things, and they have no bearing on each other.
|
|
#
?
Oct 1, 2016 18:03
|
|
|
Why doesn't Nate just shave his head? He'd look like various psychic pop culture characters instead of a loving disaster.
|
|
#
?
Oct 1, 2016 18:19
|
|
|
exquisite tea posted:Nate did know what was going to happen in Florida, though. In the final week of the election, his model moved to a very slight Obama advantage, so he made a guess that Obama would win the state. Obama did win Florida and it was the closest state of 2012, with Romney having actually led in the RCP aggregate by 1.5% and Obama winning by .6%. This is what you expect good models to do, make close calls even in tight situations where simply looking at the polls will not give you the full answer. It picked up Obama's late movement there in a way that PEC and the others did not. You seem to be caught up that Silver wasn't 100% confident in his prediction because he didn't say Obama will win Florida hands-down, no questions asked, but to a statistician that kind of stance is pointless. You're always going to talk about things in terms of probabilities because predictive models give you varied results with the same exact inputs. I'm not "caught up" in the fact that Nate wasn't 100% confident. In fact, I think he did a good job in 2012. He called 49 states pretty confidently. Ohio and NC were 3 and 2 point margins, respectively, and he managed to call them with relatively high levels of certainty. My point is that the Florida call is the least impressive "call" he made, since it was hedged so badly that you learned nothing more from its call than you would from a Sabato's Crystal Ball "toss-up" designation. When Nate ran his model 1000 times, Obama won 503 of the simulations and Romney won 497. That's a coin-toss. Adar posted:Your posts ITT are awesome fyi I'll point out one tweet from Nick Gourevitch - part of GSG, one of the top Democratic pollsters in the country, and a guy who sees far, far more polls than the rest of us do. https://twitter.com/nickgourevitch/status/779699800346427392 https://twitter.com/nickgourevitch/status/780055284379254784 https://twitter.com/jhagner/status/778572207714893824 When you see stuff like that you surely get the sense that it would be difficult to hold those positions if the polling they see every day were actually pretty volatile. That combined with the decisions being made by the Clinton campaign and Priorities USA is a fairly strong indication (without being able to see the actual polling) that they don't see the need to change their game plan in response to changing conditions. Now, I think we can't really know for sure (we're kind of reading tea leaves) if the polling is really volatile or not, but the fact that some very well-informed and smart people people see little volatility means that there's at least pretty good justification for hypothesis that the electorate is just not nearly as volatile as 538 implies. That said, it's totally reasonable to disagree with that - my beef with Silver is that he very aggressively pushes back on people who say, well, maybe the race isn't changing as much and as often as you are claiming. As far as non-response bias, you can check out this research paper that was pivotal in documenting the phenomenon. There were swings of 5-6 points that seemed to be entirely phantom in 2012. Cthulhu Dreams posted:I'm actually really interested how all the various models deal with the relatively large number of undecideds this election, it seems one of the most interesting questions. This is why I don't really blame 538 for consistently giving lower scores to Hillary. There are a fairly large number of undecideds and third party voters. We don't really know exactly how they'll break, and that is good reason for looking at this like a 65% race for Hillary instead of a 75% race. But that's sort of my problem with the level of volatility in the Silver model - if your model thinks there are a lot of undecideds and they are influencing the outcome of the race, surely it means you should consistently see a probability of (for example) a 55-60% Hillary win rather than a model that says 55% one day and 69% a few days later. A lower probability implies a greater uncertainty, which makes more sense and is generally more useful.
|
|
#
?
Oct 1, 2016 18:19
|
|
|
Azathoth posted:Points well taken, though how much of the lack of volatility in Obama's analytics was because of all the private polling and proprietary data collected and painstakingly curated by his campaign and the DNC? From a glance at RCP from 2012, it looks like the polling showed a whole lot of tightening in the aftermath of that debate. Obama's team had some information that said differently, and it was accurate, but how would any model that relies on public polling only get that kind of certainty? With the Democrat's internal stuff, aside from the ability to do polls as needed, on what they need to know exactly, there's the mountain of volunteers who collect and input data that can be integrated with the polling data and used to validate/invalidate the data. So, this is my overall point on 538: 1. 538's performance compared to PEC was almost identical. It was not perfect, but it did very well on the presidential race and succeeded in calling many of the Senate races (although it missed North Dakota and Montana badly). So I think we cannot give Nate a unique amount of credit in terms of his election predicting prowess. 2. Nate's model is now up against new competition, and they've made different decisions when building their model vs. Nate Silver. The fundamental issue is not what any of the models rate Hillary's chance of winning - we won't really have a great idea of which model performed the best until Nov 8, and even then since they are all largely in agreement it will be hard to pick it out. However, we can judge whether a model's choices make it more or less useful in terms of predicting what will happen on election day. 3. That's where the rub lies. Nate's competition all think this race is very static. Their predictions have not moved much even as some of the public polling has shifted. 538 has moved quite a bit, which makes it less useful. Now, Nate's defense is "well, of course it has moved! Because the actual electorate is moving, so my model is just properly scoring the shift!" His competition, on the other hand, thinks that the electorate is not moving with largely the same data. if Hillary wins, which I strongly suspect she does, the end result is that we'll have had a 538 model that moved around quite a bit and we'll have two other models that solidly predicted Hillary would win from the conventions onward. Should Hillary win, it will be hard to argue that those models did not do a better job of predicting November outcomes than Nate Silver. They will have made a call and consistently stuck with it, and in the end they were accurate. 538 hedged and hemmed and haw'ed until it arrived at their same conclusion weeks later. You might say, "well, that's a bit unfair. If the electorate moves back and forth, the percentage likelihood of Hillary winning should change." I would disagree and point out that a good model ought to take into account the inherent uncertainty caused by potential future moves in polling. It should slowly trend in one direction or the other, perhaps, but it should not go back and forth. That is supposed to be the difference between the now-cast and the polls-only/polls-plus. In other words, a good model would have already built polling volatility into its predictions. This is what Upshot does, for example - ignoring "momentum" changes in favor of a more stable projection. 4. Given that Nate's model is in the minority, his insistence on dismissing his perceived opponents is rather absurd. Concerned Citizen has issued a correction as of 20:46 on Oct 1, 2016 |
|
#
?
Oct 1, 2016 20:40
|
|
|
It's pretty odd that a model wouldn't move at all when the race went from Clinton +6 nationally and leading in every swing state to Clinton +2 nationally and losing in Iowa, Ohio, and briefly Florida and Nevada, which is exactly what happened a couple weeks ago. I don't think it's explained by changes in voter screens, etc., either. Why do you think a model should remain static in that case?
|
|
#
?
Oct 1, 2016 20:50
|
|
|
Is there any code for what 538 is doing?
|
|
#
?
Oct 1, 2016 21:15
|
|
|
Concerned Citizen posted:So, this is my overall point on 538: As I've been running some Electoral College scenarios, I think I am starting to see your point about odds of overall victory, though I will still stand by my previous points. If I'm following, you're saying that, even as polling cut 4-5 points off of Hillary's lead, she always maintained a solid lead in these states:  Trump needs to run the table in the six undecided states to win, and a Hillary win in just one of the six would mean that she wins, and the odds of Trump running the table in those states is roughly the same now as it was back at the end of August. If you're saying that, because of this, that her odds of winning hadn't decreased meaningfully despite her overall drop in the polls, I can at least see the logic of the argument. Am I close or am I missing the point you're making?
|
|
#
?
Oct 1, 2016 21:35
|
|
|
|
| # ? Jun 8, 2024 19:31 |
|
|
Vox Nihili posted:It's pretty odd that a model wouldn't move at all when the race went from Clinton +6 nationally and leading in every swing state to Clinton +2 nationally and losing in Iowa, Ohio, and briefly Florida and Nevada, which is exactly what happened a couple weeks ago. I don't think it's explained by changes in voter screens, etc., either. Why do you think a model should remain static in that case? There's a significant difference "some movement" and "not moving at all." The now-cast, which is a snapshot of the polls today, should move around a lot - that's what it's designed to do. The polls-only and polls-plus models should be more conservative - in effect they ought to have "priced in" that the polls will tighten or expand prior to that actually happening. PEC does this well due to the peculiarities of their model. Upshot tries to do this, but it's between PEC and 538 in terms of its stability. 538 is the most sensitive to change, to the point where it moved the election to near-even odds at a couple points (in essence saying the race is too difficult to confidently predict one way or another but that Hillary is slightly more likely to win). That actually speaks more to the limitations of the model than to whether or not the race had become an actual toss-up. Of course, if Trump actually wins, we'll all have to say that while all of the models were wrong, 538 at least was less certain that Hillary would win than the others for what that's worth (not a lot).
|
|
#
?
Oct 1, 2016 21:38
|
|