NORTON META TAG

04 September 2014

Daily Kos Elections Poll Explorer: Kansas switcheroo blows up our model (in a good way) & Daily Kos Elections Poll Explorer says Democrats have 47% chance of holding the Senate (right now) 4SEP & 25AUG14

FOR those who love politics.....
Thu Sep 04, 2014 at 06:29 AM PDT

Daily Kos Elections Poll Explorer: Kansas switcheroo blows up our model (in a good way)


Distribution of likely number of Democratic Senate seats
I'm ordinarily not a big believer in "game changes;" they may be good at selling superficial books about presidential elections, but if you actually drill down into, say, the polling data from the 2012 election, all of those supposed "game changes" barely moved the needle at all when they happened. But we may have had a genuinely game changing event on Wednesday, in a previously sleepy race that was on few people's radars as competitive; it changed not only the trajectory of one race, but the overall likelihood of Democratic control of the Senate. On Wednesday, the Democratic nominee in the Kansas Senate race, little-known and underfunded Shawnee Co. District Attorney Chad Taylor, pulled the plug on his candidacy right before the deadline. Now ordinarily, you'd think that would reduce Democratic odds in Kansas from almost zero (5 percent, according to our model on Wednesday) to absolute zero. That's not the case at all, though. There is also a much better-funded and more appealing independent candidate in the race who might caucus with the Democrats if he wins: Greg Orman. Taylor and Orman were splitting the vote against the Republican three-term incumbent, Pat Roberts, who was already weakened by a mediocre primary win over Milton Wolf and accusations that he doesn't actually reside in Kansas.
Previous polls of a three-way race showed Taylor and Orman, added together, outpolling Roberts ... but combined vote shares are of no value in a first-past-the-post system. However, the most recent Public Policy Polling look at Kansas tried out permutations with just Taylor vs. Roberts, and Orman vs. Roberts. Roberts would beat Taylor 43-39, but Orman (thanks to crossover votes from moderate Republicans) would beat Roberts 43-33. (In a three-way race, it was Roberts 32, Taylor 25, and Orman 23.) And in a remarkable bit of team-player-dom, Taylor seems to have taken those polling results to heart and pulled the plug on his sputtering campaign.
Put visually, here's the trendline from the old version of the Kansas race:
And here's the trendline from the new version, as of Wednesday:
The switch to a two-way race between Roberts and Orman, where Orman has a 10-point lead, dramatically changes the trajectory of the Kansas race. Where the Democrats previously had a 5 percent shot of winning the race, they now have a 54 percent shot! That same shift also has a big impact on the overall Senate picture, suddenly making the Democrats more likely than not to retain control of the Senate. In our last full-length look at the Poll Explorer last Thursday, Democratic odds of retaining the Senate were 45 percent, and it had stayed parked there at 45 for most of the week, ticking up to 46 percent on Wednesday. However, the Kansas news boosts Democratic odds of controlling the Senate up to 56 percent overall.
Why such a big boost? Last week, I'd used the analogy of a plate-spinning contest. Although the Democrats had enough races, where they had even odds or better, to enable them to hold 50 seats (which would give them control, thanks to Joe Biden's tiebreaking vote), they were spinning a lot more wobbly plates than the Republicans. If only one of those plates fell, they'd lose the whole game. Well, with the Kansas news, the Republicans have essentially been handed their own wobbly plate. The Kansas race is still hardly a sure thing, in itself, but it gives the Democrats a much bigger cushion against failure somewhere else.
Shouldn't a 10-point lead, though (which would be considered to be outside the margin of error) give Orman better odds than 54 percent, though? There's a whole other wrinkle to the Orman candidacy, which we'll explain further over the fold:
While the shift from 5 percent odds to 54 percent odds is a huge jump anyway, it would be much higher if Orman were to simply confirm he would caucus with the Democrats. On the surface, it's a reasonable assumption that he'd caucus with the Democrats. Orman ran for the Senate in the Democratic primary in 2008, and his issues page — where he says, in rather oblique fashion, that he's pro-choice and supports immigration reform — makes it clear he'd be very uncomfortable as a Republican. However, Orman isn't saying that. (Nor would it be wise for him to do so; he'd probably shed a number of moderate Republican votes as soon as he put on a blue shirt, in a state where he can't afford to lose any of those votes.) Instead, he tiptoes around the issue, saying that he'd seek to work with "work with the other independent Senators to caucus with the party that is most willing to face our country’s difficult problems head on and advance our problem-solving, non-partisan agenda." But more importantly, he also says:
With that said, if one party is clearly in the majority, he will seek to caucus with the party that was in the majority as that would be in the best interest for the state of Kansas.
Here's where the modeling gets tricky, and it starts to cross over from mere poll aggregation into game theory. In order for the Democrats to "win" the Kansas race, they have to not only see Orman defeat Roberts, but they also have to garner a majority of the other seats anyway. If Orman wins, but the Republicans already have a majority with or without Orman, then Orman just caucuses with the Republicans anyway, and the Democrats have "lost" that race too. In other words, here are the potential scenarios:
Democrats win 48 seats, Republicans win 51 seats: Orman caucuses with the Republicans, pushing that total to 48 D, 52 R. Democrats win 50 seats, Republicans wins 49 seats: The Democrats will have a majority anyway because of Joe Biden's tiebreaking vote, so Orman caucuses with the Democrats, pushing that total to 51 D, 49 R.
Democrats win 51 seats, Republicans win 48 seats: Orman caucuses with the Democrats, pushing that total to 52 D, 48 R.
Democrats win 49 seats, Republicans win 50 seats: Here's where the real uncertainty enters the picture. There's no "clear" majority (in other words, if Orman throws in with the Democrats, the Dems will have a majority because of Biden; if he throws in with the Republicans; the Republicans will have a majority), and Orman needs to go one way or the other, unless fellow indie Angus King decides to leave the Dem caucus and join Orman on a two-person magical unicorn ride. Our model, awesome as it is, still can't predict Orman's state of mind, so in all 49 D/50 R scenarios, we're allowing Orman a 50 percent chance of caucusing with the Dems and 50 percent chance of caucusing with the GOP.
So that's why Democratic odds of "winning" the race are 54 percent, even though Orman's personal odds of winning the race are much higher. In all scenarios where the GOP ends up with 51 seats or more regardless of what happens in Kansas, and half of the ones where they end up with 50, an Orman win just pads the Republican total further. And that's why our histogram no longer resembles a neat, tidy bell curve any more (if you want to see what it looked like, check our post from last week). Scenarios where the Democrats win exactly 50 seats, which used to be the modal result, are now pretty rare, now mostly involving permutations where Roberts wins the race instead. Most of those 50-seat scenarios instead get turned into 51-seat scenarios, since at that point Orman gets turned into a bonus Dem. Fifty-one seats is the new mode, while fifty seats is the new median. Either 50 or 51 is enough for control, so now the Democrats retain control in the majority of all Monte Carlo simulations.
While Kansas is largely responsible for the Dems' move upward, if you look at the totem pole, you'll also notice that Mary Landrieu's odds in Louisiana have increased significantly (perhaps too much to be truly realistic, from 47 percent to 76 percent). That's thanks to one poll that didn't attract much attention, a poll released by the Senate Conservatives Fund that had Landrieu leading her likely runoff opponent, Republican Bill Cassidy, 49-43. Why, you might be asking, would a right-wing group leak a poll showing a Republican in such dire straits? Well, it was to show that their preferred candidate, Rob Maness, trailed Landrieu by slightly less (48-44), as well as improving his share in the jungle primary. Whatever their purposes, though, the model counts it as something of an own-goal, boosting Landrieu's averages even further because it's a Republican partisan poll and the model downweights Cassidy even further!
In addition, two polls that put Michelle Nunn in a small lead (after a long series of ones showing her losing) boosted her odds somewhat, from 14 percent last week up to 23 percent now. That's balanced, however, by several polls in Kentucky showing Mitch McConnell expanding his lead a bit, moving Alison Lundergan Grimes's odds there from 22 percent down to 18 percent.
Let me repeat these caveats until I'm blue in the face: As happy as this shift made me, the fact that the Democrats have a 56 percent chance of controlling the Senate this week is in no way a prediction that they will control the Senate, any more than, when we were talking about this last week, the fact that the Democrats had a 45 percent chance of controlling the Senate was a prediction that they will lose the Senate. The overall proposition is still, entirely, a coin flip. The coin has simply gone from being very slightly weighted in one direction to very slightly weighted in the other direction. Finally, let's take a look at the gubernatorial races. For one brief day, the median number of Dem-held state houses ticked up to 23, which would be a two-seat improvement over what they currently have. However, that ticked back down to 22 today, which is a one-seat gain. That's largely due to polling fluctuations in Florida, where a SurveyUSA poll on Tuesday gave Charlie Crist a 45-43 lead, but then a Univ. of Florida poll on Wednesday gave Rick Scott a 41-36 lead.
Overall Democratic odds of gaining gubernatorial seats continued to move up, though; it was at 52 percent last week, and now it's up to a 60 percent chance. That's in part due to improved odds in Georgia and Arkansas (which the Democratic candidates are still on track to lose, though), but more than anything due to Alaska, where there was an entirely separate switcheroo in a three-way race where the Dem and indie had been splitting the anti-incumbent vote.
There, Democratic nominee Byron Mallott dropped out to become the Lt. Governor on a ticket with independent candidate Bill Walker. What few polls we'd previously seen of a head-to-head matchup between Walker and GOP incumbent Sean Parnell showed a very close race, and even a Walker lead in one case, so that boosted Dem odds here from 1 percent to 37 percent. (To make things simpler, we're counting Walker as a de facto Democrat here, since policy-wise he'd be a big improvement on Parnell, even though there will be an "I" next to his name; there's no need to ask about who he's going to caucus with, seeing as how governors don't caucus with anybody, so Walker's individual odds are what's important.)
We'll be continuing to post updates like this several times a week, as we see if the Democrats can keep building on this momentum or not. Our permanent Poll Explorer Senate and gubernatorial pages are updated every day, so please check them frequently; you can bookmark them, or find them in the "Blogroll" list in the right column on the main Daily Kos Elections page.

Originally posted to Daily Kos Elections on Thu Sep 04, 2014 at 06:29 AM PDT.

Also republished by Kansas & Missouri Kossacks and Daily Kos.

Mon Aug 25, 2014 at 06:30 AM PDT

Daily Kos Elections Poll Explorer says Democrats have 47% chance of holding the Senate (right now)


Distribution of likely number of Democratic Senate seats
Daily Kos Elections is pleased to announce the introduction of its Poll Explorer! The Poll Explorer is a statistical model that allows us to track public opinion in each Senate and gubernatorial race this year, and from there, predict the likely outcome of each race and the number of seats that each party will control after the election is held. This is an ongoing feature that will be available from now until the November election. We'll be doing posts like this one, describing the current state of the election, several times a week from now until Nov. 4; however, there will also be permanent pages for all the Senate and gubernatorial data which will be updated every day, and which you can bookmark and access at any time. As the headline states, today's version of the model shows that if 100 elections were held today, the Democrats would control the Senate in 47 of those elections. That means that it's slightly likelier than not that the Republicans will win the necessary 50 seats for control. The median number of seats that the Democrats end up controlling, in the hundreds of thousands of simulations that we run, is 49. (In other words, if you lined up every single result in order, 49 seats is the point where 50 percent of the results would be higher than that, and 50 percent would be lower.)
However, there's an interesting quirk here: the likeliest distribution of seats would be 50 Democratic-controlled seats and 50 Republican-controlled seats, which, thanks to Joe Biden's tie-breaking vote, would allow the Democrats to retain control. That result is only very slightly likelier than the second-likeliest outcome, though, which is 49 Democratic-controlled seats and 51 Republican-controlled seats, and the more outlying results are more likely to be Republican-friendly (for instance, it's much likelier that the Democrats will end up controlling 46 seats than they will control 53 seats), which explains why the Republicans have slightly better than 50-50 odds overall. (In other words, 49 Democratic seats is the median, while 50 is the mode.) You can see the full distribution of the range of likely numbers of Democratic seats in the bar chart, known as a histogram, shown above.
And there's one more quirk here, too. If you look closely at the specific races, you'll notice that the Democrats have greater than 50 percent odds of winning 50 individual races. The Democrat-held seats that they are, individually, on track to lose are South Dakota (1 percent odds), Montana (1 percent), West Virginia (1 percent), Arkansas (37 percent), and Louisiana (46 percent). The Democrats are currently on track to win Iowa (51 percent), North Carolina (58 percent), Colorado (63 percent), Michigan (72 percent), Alaska (82 percent), and New Hampshire (82 percent). However, you have to take the races into consideration as a whole. Think of the fight for the Senate as a plate-spinning contest: the Democrats are currently spinning a lot more plates than the Republicans, and it only takes one more plate falling for them to lose control. The Democrats have a lot more wobbly plates (North Carolina, Iowa, Colorado) than do the Republicans (Louisiana). In a majority of simulations, one of them still falls, even though individually each plate has a better-than-even chance of not falling; such is the nature of probability.
Before you start running around in circles screaming that "even the liberal Daily Kos" says the Democrats are going to lose the Senate, please bear in mind this most important caveat: this is not a prediction that the Democrats will lose the Senate, any more than if we gave the Democrats a 53 percent chance of controlling the Senate, it would be a prediction that the Democrats will win the Senate. It's merely a reflection of the fact that control of the Senate is truly a coin flip, and if there's any contention between us and 538 or the New York Times over what specific percentage should be applied, it's purely an argument about whether that coin, as of today, is ever-so-slightly weighted toward the Democrats or Republicans.
Moreover, the model is merely a description of the polls. If the polls are simply wrong, then the model will be wrong, too. We aren't terribly concerned about that, as poll averaging produced highly reliable results in the 2008 and 2012 presidential races. Nevertheless, statistical modeling is only one potential approach to forecasting an election, and Daily Kos Elections will continue to offer more subjective and gestalt-based Senate and gubernatorial race ratings as well (in much the same way that baseball teams that have incorporated a more sabremetric approach continue to employ scouts, as well).
We'll delve more thoroughly into how the model works and why we're doing this, over the fold:
The Daily Kos Elections Poll Explorer is an adaptation of Votamatic, which was a site developed for 2012 by Drew Linzer that correctly predicted the 332-206 outcome in that year’s presidential election. We've repurposed the Votamatic model to work in the Senate and gubernatorial context as well. It's a Bayesian model that, first, creates smoothed trendlines for each individual race and then runs thousands of Monte Carlo simulations to see how likely Democratic candidates are to win each race, and how likely Democrats are to win the Senate and gubernatorial playing fields as a whole. If you would like to read a much more detailed explanation of each step in the process, and how to interpret the various charts, that can be found at our How It Works page.
The model draws its data from the Daily Kos Elections Polling Database, which is scrupulously maintained by our own Steve Singiser, and which is a great resource if you want to zoom in on the complete polling history of a particular race. We cast as wide a net as possible including all public pollsters (and yes, that includes Rasmussen), under the assumption that averaging all polls will smooth out each individual pollster's house effects or other quirks. We also include polls from partisan pollsters, such as those leaked by campaigns, although we uniformly adjust partisan polls to make them less favorable to the candidates who commissioned them.
We’ve stress-tested the model by retroactively applying it to the 2012 election, which finds it correctly predicting all but two Senate races. Those races are Montana and North Dakota; those two races, it's worth noting, also stumped other prominent aggregators. (And that's simply because there wasn't a robust selection of polls of the Montana and North Dakota races, and the majority of those polls were, at the end of the day, simply wrong; more gestalt-based predictors who weren't bound to models were able to get one or both those races right.)
Now that you know how it works, you might be wondering: why do we need another model? After all, there are perfectly serviceable predictive models courtesy of sites like 538, the New York Times' Upshot, and the Washington Post's Election Lab. Part of the problem, though, is that every model is built on its own set of assumptions; in fact, the next step for some enterprising person might not be to build a model that aggregates the polls, but one that aggregates the aggregators. Models may not just have, for instance, different ways of accounting for polls from partisan pollsters or polls that are too old, but also give different emphases to other factors, like polls of the generic congressional ballot, economic data, or historical trends in midterm elections.
Part of what sets our model apart is that it doesn't account for those economic or historical factors; it's just the polls. Historical factors are relevant earlier in the cycle, when there simply isn't adequate polling data and you need something to fill the gap, but they aren't (as) necessary now that we have a suitable number of polls in all the key races. In other words, this is intentionally set up to be an all-meat, no-special sauce model. Omitting those factors, however, means that this isn't a "predictive" model; we don't add in expectations that voters will swing toward the Democrats or Republicans between now and Election Day. The range of possible outcomes that we calculate for the Senate and gubernatorial races simply starts from where we estimate public opinion to be today, then adds in uncertainty about how voters' preferences might change over the next two-and-a-half months.
To the extent that the models are poll-driven, though, the end results aren't too different; our expectation that the Republicans have a 53 percent chance of taking the Senate is quite close to the NYT's prediction that they have a 65 percent chance of doing so, or the Washington Post's prediction that the GOP has a 63 percent chance of taking the Senate. They see the same polls as us, and there's only so much you can do differently in interpreting them.
The slightly more pessimistic result that these other aggregators arrive at has largely to do with the way that their models rely less on polls and more heavily on economic and fundraising data, which is something that we consider, at this point, already be showing up in the results pollsters are finding. If anything, the Democratic candidates are currently overperforming in the polls compared with what economic and historical fundamentals would lead you to expect. However, if a late-breaking wave or black swan event suddenly causes that Democratic overperformance to head south, that too would be rapidly reflected in the polls.
And there's one other key difference that separates us from the other aggregators: we are also looking at the gubernatorial races. It's not as sexy a topic, since you don't get a prize for holding the majority of gubernatorial seats, and at any rate the number of likely Dem-held gubernatorial seats isn't hovering right between 24 and 25. But gubernatorial races are hugely important, both in terms of establishing progressive policies in individual states and building a presidential bench, so we want to give a similar focus to gubernatorial races using the same method.
As you can see in our gubernatorial histogram, the median outcome would be that, after the election, the Democrats will hold 21 gubernatorial seats. As you can see on the histogram, the modal outcome would be for the Democrats to hold 20 seats. Twenty-one is the current number that the Democrats hold (which is why that column is gray), so that means the likeliest outcome would be either no net change or a net change of one in the GOP's direction.
That may sound surprising, when most pundits would say the Dems are on track to pick up a seat or two. However, the most current round of polls do not support that conclusion. Take a look at the totem pole of races to see the specifics: as most pundits expect, the Democrats have overwhelming odds in picking up the Pennsylvania (99 percent) and Maine (91 percent) races. In addition, they're on track to pick up Kansas (surprising a few months ago but not surprising now, at 63 percent) and Wisconsin (that is a surprise, but it's a total coinflip at 52 percent). However, they're also on track to lose not just Arkansas (19 percent) and Illinois (10 percent), but also Connecticut (11 percent) and Hawaii (15 percent). Many people assume that the dark blue lean of Connecticut and Hawaii will save those races in the end (and that's particularly the case in Hawaii, where David Ige beat unpopular Neil Abercrombie in the primary—bear in mind, though, that the model does reflect polling of an Ige/Duke Aiona race). For the moment, though, the polls—which are all we're relying on here—do not support that idea.
To reiterate, none of this—not the percentages for the individual races, nor the percentages for the Dems' overall odds of holding the Senate—are set in stone. The percentages can, and will, fluctuate quite a bit over the coming months. That's why we'll be updating the model with new data every day, which you can see by visiting the permanent pages. I'll also be writing a full post like this one several times a week, discussing what has changed in the model since the last post, and what individual polls may have changed the numbers. So please follow along as we track the stretch run for Election '14.

Originally posted to Daily Kos on Mon Aug 25, 2014 at 06:30 AM PDT.

Also republished by Daily Kos Elections.

No comments:

Post a Comment