Tuesday, November 8, 2016

Cheat Sheet for Election Night

You and me, we're friends, right? You know I get you the good stuff. The right stuff. The stuff that lets you feel smart.

So this post is for you, for tonight. You're going to be at a party, at a friend's house, at a bar, maybe sitting in your living room holding a knife and waiting for an understanding of which way to cut. I know, it's okay. You want to know before anyone else knows how tonight's going to go.

So here's a cheat sheet. I'll break it down, hour by hour, of which states you should be watching.

Before 8:00 PM:

Polls fully closed in: VT, VA, OH, WV, VA, NC, SC, GA, KY, IN

Your guide:


Before 9:00 PM:

Polls fully closed in: ME, MA, NH, CT, RI, PA, NJ, MD, DE, IL, MO, OK, TN, AR, MS, AL, FL

Your guide:

Before 10:00 PM:

Polls fully closed in: MI, WI, MN, NE, WY, CO, KS, AZ, NM, TX, AR, LA

After 10:00 PM:

I plan to be extremely drunk.

Monday, November 7, 2016

Reassurances, part 3 - why Hillary will do better than anyone expects

Okay, so thus far I've been explaining why the fundamentals are solid: why the overall scenario is terrible for Trump and the GOP (again, a theme I'll explore more post-election), and why Nate Silver's numbers don't seem as on-point as we've built them up to be.

Now let me talk about why the polls are wrong.

Specific egg points: Georgia, Arizona, Ohio, Iowa
This is dangerous territory, fraught with the errors of 2012. (For people new to my blog, yes, I do eventually link to things other than previous posts I've made. But this is kind of a culmination of themes, so bear with me). Romney's polling team followed the same logic chain I'm following, just from the opposite perspective, making assumptions like I do that the universe skews their way. I think I'm better at balancing facts vs. hope, but I'm well aware that I'm precariously balancing some egg above my face and waiting for the universe to decide what happens. But I made a prediction at the end of September - before both the Access Hollywood video and the FBI "she's still a crook/okay maybe she isn't, but now you're thinking about it, so that's good" thing - and it's probably overly aggressive, but I'm sticking with it.

So why do I think things are better for Hillary than the polls show?

First: because the polls were wrong. In 2012, final polling average had Obama at +0.7; the actual results were Obama +3.9. While a 3 point swing is within margin of error, it's pretty big, and what's more worrying is that it's a consistent error: of the polls listed on that RCP page, not a single one over-predicted for Obama, and most of them called it a tie or slight Romney lead. The assumption in poll aggregation is that margin of error will drive polls to fall on both sides of the true point, and that aggregating and averaging will lead to a result - so when literally every national poll is on the wrong side of the true results, it points to foundational problems in the polling.

What are those problems? Well, here's where I'm speculating - but the consistent problems called out in 2012 were poor LV screens, and poor inclusion of cell phones. LV screens consistently undervalued whether minorities or millennials were planning to vote, a trend which continues today - for example, the CNN-ORC polls consistently include too few people under 35 to be able to statistically provide a breakdown for (see pages 25+ of this pdf). Yes, poll aggregators like 538 adjusted to reflect the inherent bias polling firms showed (which has led to this article claiming that Silver is skewing the polls, leading Silver to lash out in response), but they're reacting to past events and not adjusting for the directional trend: i.e., they'd be fine if we re-run 2012 again, but now it's 2016 and those same factors are only more aggravated, so they're behind again.

Second: because polls don't take GOTV into account. "GOTV" means "Get Out The Vote", which is all of the facilitation a party does locally to get you into the voting booth: tell you where to go, tell you how to get there, let you know when to go, help you get there if you need assistance, make it easy to vote early or vote by mail if you need it, and generally play the nagging mother to remind you that you have a civic duty and they need you and vote goddammit.

Speaking of which: vote, goddammit. If we as a country are incredibly lucky, no election after this will ever equal this in importance.

Anyways. So, GOTV - occasionally called "ground game"- identifies voters and gets them to the polls. We don't usually care about that, because usually both parties are running GOTV and they're both doing the same things in the same way, and unless you're Mitt Romney and decide to eliminate the local structure and replace it with a nationalized private enterprise solution, and you happen to be up against one of the most tech-and-stat-savvy organizations in years, it's hard to say that GOTV did anything. But, again, Obama outdid polls by 3 points. How much of that was polling, and how much was GOTV?

And here's the thing: Hillary has Obama's machine. Trump has... well, the RNC never made Orca work that great, and Trump's reaction seems to have been "eh, fuck it", which has led to a raft of articles from commentators on both sides kind of worried / kind of fascinated to see what happens. Which has quite a few political scientists watching carefully because we might actually get more of an idea on whether GOTV matters or if it's just something campaigns waste time and money on because it used to be important.

So, to sum up: Obama outdid his polls by about 3 points in 2012, and my argument is that all of the factors that seem to have driven that are only more prevalent in 2016, so I think Clinton will outdo her polls by about as much.

We won't know if I'm right until some time later this week, when it's all counted, but consider this: the Clinton campaign is saying that overall early voting by Hispanics in Florida is up 139%.



Thursday, November 3, 2016

Part 2: Nate Silver is not the Data Messiah

Okay, so I'd originally planned to write a Part 2 that was about something else, and to do it on Monday, but I've had people ask me to address 538, or more specifically, how everything on 538 is tanking and it's down to nearly a coin flip and oh God we're all going to die in the Trump Camps.

Okay, to begin with: 538 isn't the only poll aggregator/modeler out there. Princeton has one. I've heard Princeton is kind of a reputable joint. The New York Times, the 'grey lady of journalism', has one. Perhaps less reliable journalists have some. Now, knowing this: watch this great video.

Image result for nate silver hair
There is no way I should be in a position to comment
on someone else's hairline, and yet.
Note that, from about the first week of September forward, 538 has consistently given Clinton the lowest chance of success. Nate's model is always more bullish for Trump. To the point where just over a week ago he wrote an article specifically addressing that point, using the word "bullish" because I'm not the only one who uses that term, god dammit.

Anyways.

I want to address one of the specific points he brings up, because to me it's pretty obvious where things are wrong. His second point in that article is that he includes polling data going back to 1972 in order to make his models, and he inherently implies that by including more data, he is naturally more correct. But that's a fallacy. Let me put it this way: if a doctor told you that he was using all of the tools at his disposal, including ones they stopped using in the '70's, would you trust him more or less?
Image result for 1980 there you go again
Back when people who despised each other and everything
they stood for respected the process enough to feign civility.

In 1980, polling organizations consistently called the presidential race "too close to call", and admitted complete surprise when Reagan shellacked Carter 51 - 41. The polling data in here is in Nate Silver's model, and not in other people's models. Silver admits that if you take the pre-2000 data out, his model immediately jumps to be much more in line with everyone else.

538 gets a lot of press, and Nate Silver somehow got stuck in the public consciousness as "the guy who got it all right"  even though Sam Wang did it earlier and with more consistently correct results since 2004Wang and Silver have even had public back-and-forths over how the other guy's model sucks.

If you want to take it in a much larger picture, consider this: Nate Silver aggregates polls, adding them all together and averaging it out* on the assumption that if one polling agency is off in one direction, standard statistical deviation makes it likely that another polling agency is off in the other, and once you have enough in the pile, you can get a true average. So throw 538's chances for Hillary on the pile, average it with the other polls trackers, and she's not at 538's dire sub-70, but at a much more respectable near 90s.




*Yes, yes, it's more complex than that, but being specific that he applied weights and corrections means I have to explain more before I get to the analogy which is really unaffected by the additional facts.