By NATE SILVER
Mitt
Romney gained further ground in the FiveThirtyEight forecast on Monday,
with his chances of winning the Electoral College increasing to 25.2
percent from 21.6 percent on Sunday.
The change represents a continuation of the recent trend: Mr. Romney’s chances were down to just 13.9 percent immediately in advance of last week’s debate in Denver. He has nearly doubled his chances since then.
But the gains that he made on Monday in particular were all because of a single poll.
We’ll talk about that poll — a Pew Research poll that gave Mr. Romney a 4-point lead among likely voters — in a moment. But let’s first consider the day’s worth of polling without it, which was pretty mixed for Mr. Romney.
The most unfavorable numbers for Mr. Romney came in the national tracking polls published by Gallup and Rasmussen Reports. Both showed the race trending slightly toward President Obama, who increased his lead from 3 points to 5 points in the Gallup poll, and pulled into a tie after having trailed by 2 points in the Rasmussen survey.
In both cases, the numbers looked more like pre-debate data than the stronger numbers that Mr. Romney has been receiving since then. On average between the Democratic convention and the debate, the Rasmussen poll showed Mr. Obama with a 0.7-point lead (the Rasmussen poll is Republican-leaning relative to the consensus), while the Gallup poll had Mr. Obama ahead by an average of 3.4 points.
A third national tracking poll, an online tracking poll published by the RAND Corporation, showed essentially no change from Sunday. All of this seemed to be consistent with a story in which Mr. Romney’s debate bounce was receding some. (A fourth tracking poll, from Ipsos, had not been published as of the time we ran our forecast on Monday.)
The swing state polls published on Monday might best be described as being OK for Mr. Obama. He led in polls of Colorado, Iowa, Pennsylvania and Virginia, and in two polls of Michigan. In all cases, Mr. Obama’s lead was small. However, this particular group of pollsters had shown reasonably unfavorable numbers for Mr. Obama in the same states before. Three of the polls actually moved toward Mr. Obama from the numbers that the same polling firms had published before the debates.
There
are a few polls that I’ve left out of the discussion here — a George
Washington University survey for Politico, for example, showed just a
1-point lead for Mr. Obama, even though most of its interviews were
conducted before the debate. That was a strong poll for Mr. Romney.
But were it not for the Pew poll, our forecast would have been unchanged from Monday, with Mr. Romney’s chances holding at 21.6 percent.
The Pew poll, however, may well be the single best polling result that Mr. Romney has seen all year. It comes from a strong polling firm, and had a reasonably large sample size. Just as important is the trendline. Pew’s polls have been Democratic-leaning relative to the consensus this year; its last poll, for instance, had Mr. Obama 8 points ahead among likely voters. So this represents a very sharp reversal.
One line of complaint about the poll has come from Democrats, who noted that the poll showed more Republicans than Democrats in its sample — unlike most other recent surveys.
I feel the same way about this critique that I do when it comes from Republicans — which is to say I don’t think very much of it. As The Washington Post’s Jon Cohen notes, party identification is fluid rather than fixed, and can change in reaction to political and news events. If voters are feeling better about Mr. Romney after the debates, they might also be inclined to identify themselves to pollsters as Republicans.
It is probably also the case that Republicans won’t actually have a 5-point party identification advantage in the exit poll on Election Day. But it isn’t the pollster’s job to project what will happen on Nov. 6. (That’s my job, instead!) Rather, the pollster’s job is to take the most accurate snapshot of the electorate at the time the poll is conducted. Note that the Rasmussen Reports polls, which (improperly, in my view) adjust for party identification, show very little bounce for Mr. Romney. The party-identification adjustment is causing them to miss the story of the election — just as they were largely missing the story of Mr. Obama’s bounce following his convention.
There are two smarter questions to ask about the Pew poll. First, is it really likely that Mr. Romney leads the race by 4 points right now? The consensus of the evidence, particularly the national tracking polls, would suggest otherwise. Instead, the forecast model’s conclusion is that the whole of the data is still consistent with a very narrow lead for Mr. Obama, albeit one that is considerably diminished since Denver.
It might be granted that the situation is more ambiguous than usual right now. But our forecast model looks at literally all of the polls; it estimates Mr. Romney’s post-debate bounce as being 2.5 percentage points, not quite enough to erase Mr. Obama’s pre-debate advantage.
The other valid line of inquiry concerns the timing of the poll. The Pew poll was conducted from Thursday through Sunday, although more of the interviews were conducted in the earlier part of that period. There’s nothing in the poll that really refutes the story that Mr. Romney initially received a very large bounce after the debate (perhaps somewhere on the order of 4 or 5 points, if not quite as large as Pew shows it), which has since faded some between the news cycle turning over and the favorable jobs report on Friday.
The evidence that Mr. Romney’s bounce is receding some is only modestly strong — as opposed to the evidence that he got a significant bounce in the first place, which is very strong. Still, the order in which polls are published does not exactly match the order in which they were actually conducted — and at turning points in the race, these details can make a difference.
The last thing to consider is that the fundamentals of the race aren’t consistent with a 4-point lead for Mr. Romney. Instead, the most recent economic numbers, and Mr. Obama’s approval ratings, would seem to point to an election in which he is the slight favorite. We don’t use approval ratings in our forecast, but we do use the economic data, and both the monthly payrolls report and the broader FiveThirtyEight economic index would point toward an election in which Mr. Obama is favored in the popular vote by around 2.5 percentage points.
There is a fair amount of uncertainty in this calculation: models that claim to achieve exceptionally precise results based on economic data alone have not always lived up to their billing — so a forecast based on the economic index alone would have Mr. Obama as only a 60 or 65 percent favorite, hardly a sure thing. (In Mr. Obama’s case, much of the downside risk would come from the potential for a poor Democratic turnout.) And our forecast model weights the economic factors less and less as time goes on.
Still, the economic component of the model still has some influence on the model (it accounts for about 25 percent of the forecast for the time being). Just as the economic component was causing the model to adjust Mr. Obama’s numbers downward during the height of his convention bounce, the same consideration will help Mr. Obama slightly if he starts to see more results like the Pew poll.
This technique has produced a very stable forecast over the whole of the year: since we began to publish the model in the spring, the projected Nov. 6 result has varied only between a 1.6-point win for Mr. Obama in the national popular vote and a 4.3-point edge.
That’s not to say the model weights all the polls equally: high-quality national polls have an especially big influence on the trendlines that we estimate. Hence, Mr. Romney’s odds of winning the Electoral College increased by more than 3 percent on the basis of the Pew poll alone; I doubt that any other individual poll has had as much influence on the forecast.
But it’s one thing to give a poll a lot of weight, and another to become so enthralled with it that you dismiss all other evidence. If you can trust yourself to take the polls in stride, then I would encourage you to do so. If your impression of the race is changing radically every few minutes, however, then you’re best off looking at the forecasts and projections that we and our competitors publish, along with Vegas betting lines and prediction markets.
All of these methods have slightly different ways of accounting for new information, but they do involve people who are risking either money or reputation to get it right, and who have systematic ways to weigh the evidence rather than doing so on an ad hoc basis.
http://fivethirtyeight.blogs.nytimes.com/2012/10/09/oct-8-a-great-poll-for-romney-in-perspective/
The change represents a continuation of the recent trend: Mr. Romney’s chances were down to just 13.9 percent immediately in advance of last week’s debate in Denver. He has nearly doubled his chances since then.
But the gains that he made on Monday in particular were all because of a single poll.
We’ll talk about that poll — a Pew Research poll that gave Mr. Romney a 4-point lead among likely voters — in a moment. But let’s first consider the day’s worth of polling without it, which was pretty mixed for Mr. Romney.
The most unfavorable numbers for Mr. Romney came in the national tracking polls published by Gallup and Rasmussen Reports. Both showed the race trending slightly toward President Obama, who increased his lead from 3 points to 5 points in the Gallup poll, and pulled into a tie after having trailed by 2 points in the Rasmussen survey.
In both cases, the numbers looked more like pre-debate data than the stronger numbers that Mr. Romney has been receiving since then. On average between the Democratic convention and the debate, the Rasmussen poll showed Mr. Obama with a 0.7-point lead (the Rasmussen poll is Republican-leaning relative to the consensus), while the Gallup poll had Mr. Obama ahead by an average of 3.4 points.
A third national tracking poll, an online tracking poll published by the RAND Corporation, showed essentially no change from Sunday. All of this seemed to be consistent with a story in which Mr. Romney’s debate bounce was receding some. (A fourth tracking poll, from Ipsos, had not been published as of the time we ran our forecast on Monday.)
The swing state polls published on Monday might best be described as being OK for Mr. Obama. He led in polls of Colorado, Iowa, Pennsylvania and Virginia, and in two polls of Michigan. In all cases, Mr. Obama’s lead was small. However, this particular group of pollsters had shown reasonably unfavorable numbers for Mr. Obama in the same states before. Three of the polls actually moved toward Mr. Obama from the numbers that the same polling firms had published before the debates.
But were it not for the Pew poll, our forecast would have been unchanged from Monday, with Mr. Romney’s chances holding at 21.6 percent.
The Pew poll, however, may well be the single best polling result that Mr. Romney has seen all year. It comes from a strong polling firm, and had a reasonably large sample size. Just as important is the trendline. Pew’s polls have been Democratic-leaning relative to the consensus this year; its last poll, for instance, had Mr. Obama 8 points ahead among likely voters. So this represents a very sharp reversal.
One line of complaint about the poll has come from Democrats, who noted that the poll showed more Republicans than Democrats in its sample — unlike most other recent surveys.
I feel the same way about this critique that I do when it comes from Republicans — which is to say I don’t think very much of it. As The Washington Post’s Jon Cohen notes, party identification is fluid rather than fixed, and can change in reaction to political and news events. If voters are feeling better about Mr. Romney after the debates, they might also be inclined to identify themselves to pollsters as Republicans.
It is probably also the case that Republicans won’t actually have a 5-point party identification advantage in the exit poll on Election Day. But it isn’t the pollster’s job to project what will happen on Nov. 6. (That’s my job, instead!) Rather, the pollster’s job is to take the most accurate snapshot of the electorate at the time the poll is conducted. Note that the Rasmussen Reports polls, which (improperly, in my view) adjust for party identification, show very little bounce for Mr. Romney. The party-identification adjustment is causing them to miss the story of the election — just as they were largely missing the story of Mr. Obama’s bounce following his convention.
There are two smarter questions to ask about the Pew poll. First, is it really likely that Mr. Romney leads the race by 4 points right now? The consensus of the evidence, particularly the national tracking polls, would suggest otherwise. Instead, the forecast model’s conclusion is that the whole of the data is still consistent with a very narrow lead for Mr. Obama, albeit one that is considerably diminished since Denver.
It might be granted that the situation is more ambiguous than usual right now. But our forecast model looks at literally all of the polls; it estimates Mr. Romney’s post-debate bounce as being 2.5 percentage points, not quite enough to erase Mr. Obama’s pre-debate advantage.
The other valid line of inquiry concerns the timing of the poll. The Pew poll was conducted from Thursday through Sunday, although more of the interviews were conducted in the earlier part of that period. There’s nothing in the poll that really refutes the story that Mr. Romney initially received a very large bounce after the debate (perhaps somewhere on the order of 4 or 5 points, if not quite as large as Pew shows it), which has since faded some between the news cycle turning over and the favorable jobs report on Friday.
The evidence that Mr. Romney’s bounce is receding some is only modestly strong — as opposed to the evidence that he got a significant bounce in the first place, which is very strong. Still, the order in which polls are published does not exactly match the order in which they were actually conducted — and at turning points in the race, these details can make a difference.
The last thing to consider is that the fundamentals of the race aren’t consistent with a 4-point lead for Mr. Romney. Instead, the most recent economic numbers, and Mr. Obama’s approval ratings, would seem to point to an election in which he is the slight favorite. We don’t use approval ratings in our forecast, but we do use the economic data, and both the monthly payrolls report and the broader FiveThirtyEight economic index would point toward an election in which Mr. Obama is favored in the popular vote by around 2.5 percentage points.
There is a fair amount of uncertainty in this calculation: models that claim to achieve exceptionally precise results based on economic data alone have not always lived up to their billing — so a forecast based on the economic index alone would have Mr. Obama as only a 60 or 65 percent favorite, hardly a sure thing. (In Mr. Obama’s case, much of the downside risk would come from the potential for a poor Democratic turnout.) And our forecast model weights the economic factors less and less as time goes on.
Still, the economic component of the model still has some influence on the model (it accounts for about 25 percent of the forecast for the time being). Just as the economic component was causing the model to adjust Mr. Obama’s numbers downward during the height of his convention bounce, the same consideration will help Mr. Obama slightly if he starts to see more results like the Pew poll.
This technique has produced a very stable forecast over the whole of the year: since we began to publish the model in the spring, the projected Nov. 6 result has varied only between a 1.6-point win for Mr. Obama in the national popular vote and a 4.3-point edge.
That’s not to say the model weights all the polls equally: high-quality national polls have an especially big influence on the trendlines that we estimate. Hence, Mr. Romney’s odds of winning the Electoral College increased by more than 3 percent on the basis of the Pew poll alone; I doubt that any other individual poll has had as much influence on the forecast.
But it’s one thing to give a poll a lot of weight, and another to become so enthralled with it that you dismiss all other evidence. If you can trust yourself to take the polls in stride, then I would encourage you to do so. If your impression of the race is changing radically every few minutes, however, then you’re best off looking at the forecasts and projections that we and our competitors publish, along with Vegas betting lines and prediction markets.
All of these methods have slightly different ways of accounting for new information, but they do involve people who are risking either money or reputation to get it right, and who have systematic ways to weigh the evidence rather than doing so on an ad hoc basis.
http://fivethirtyeight.blogs.nytimes.com/2012/10/09/oct-8-a-great-poll-for-romney-in-perspective/
No comments:
Post a Comment