- Email: [email protected]
From polls to votes to seats: Forecasting the 2015 British general election
Electoral Studies 41 (2016) 244e249
Contents lists available at ScienceDirect
Electoral Studies journal homepage: www.elsevier.com/locate/electstud
From polls to votes to seats: Forecasting the 2015 British general election Robert Ford a, Will Jennings b, Mark Pickup c, *, Christopher Wlezien d a
University of Manchester, UK University of Southampton, UK c Simon Fraser University, Canada d University of Texas at Austin, USA b
a r t i c l e i n f o
a b s t r a c t
Article history: Available online 30 November 2015
This paper develops a three-stage method to forecast parliamentary election results from vote preferences in British opinion polls: (1) adjusting and aggregating vote-intentions from different polling organizations; (2) forecasting how public support for parties will change in the period before election day; and (3) translating, through simulations, the forecast of election day vote shares into seat totals while incorporating constituency-level information, including local vote-intention polls. Overall, this approach seeks to combine relevant national, regional and local information, and uncertainty about that information, to better reflect the fragmentation and diversity of political contexts found in the new era of five/ six-party British politics. © 2015 Elsevier Ltd. All rights reserved.
Keywords: Forecasting British general election Vote intention Seat prediction
Our goal at the outset, building on the method we developed in 2010 (see Fisher et al., 2011), was to produce an election forecast that includes a meaningful estimate of (un)certainty. There are three parts. 1. Part 1: noise and the polls Since early in the parliament, the Polling Observatory (pollob.ca) team have been keeping track of movements in the polls on a regular basis. The key to tracking public opinion is to be cautious about short-term fluctuations e often the product of random noise or pollsters' methodological choicese and focus on the underlying trends. There have been notable shifts in vote intentions, such as the sharp decline in Liberal Democrat support after their decision to go into coalition with the Conservatives in 2010, the substantial drop in support for the Conservatives after the infamous ‘omnishambles’ budget in Spring 2012, and the slow but steady decline in Labour support, and rise in UKIP support, since late 2012. The first stage of our method produces estimated ranges of support for the
* Corresponding author. Department of Political Science, Simon Fraser University, 8888 University Drive, Burnaby, BC, V5A 1S6, Canada. E-mail addresses: [email protected] (R. Ford), [email protected]. uk (W. Jennings), [email protected] (M. Pickup), [email protected] (C. Wlezien). http://dx.doi.org/10.1016/j.electstud.2015.11.013 0261-3794/© 2015 Elsevier Ltd. All rights reserved.
parties that take into account the uncertainty inherent in each poll (for previous applications see Jackman, 2005; Pickup and Johnston, 2007; Fisher et al., 2011). The result is an aggregation of the polls that is preferable to a simple poll-of-polls, since our estimates do not vary as a function of the mix of polling firms that have been in the field recently or by which pollsters poll more frequently (Erikson and Wlezien, 1999; Jackman, 2005; Pickup and Johnston, 2008). As of April 30th, our estimates show the race as very close, with Labour on 33.1% and the Conservatives on 34.2% – but we cannot say with confidence that this Conservative lead is greater than zero.
2. Part 2: vote forecast: lessons from polling history The vote component of our forecast uses the historical relationship between polls and the election day vote from past elections to project forward from the current polling. A key idea here is ‘regression to the mean’: that if a party is polling above its historical equilibrium, it will tend to underperform the polls on Election Day. If a party is below the equilibrium, it will tend to do better. We model this relationship as follows:
v:sharei N m:v:sharei;t ; s2v;t
(1)
R. Ford et al. / Electoral Studies 41 (2016) 244e249
m:v:sharei;t ¼ at þ bt *p:sharei;t v.sharei is the share of the vote received by the party in election i; and p.sharei,t is the share of the vote intention received by the party in election i on day t; m.v.sharei,t is the expected value of the share of the vote received by the party in election i conditioning on the polls on day t; s2v;t is the standard deviation of the residuals from the daily cross-sectional regressions. Using the results from 3293 polls across 17 elections we can estimate at, bt and s2v;t (Wlezien and Erikson, 2002; Erikson and Wlezien, 2012; also see Wlezien et al., 2013; Jennings and Wlezien, 2015). However, the precision of these estimates is reasonably low, as each at, bt and s2v;t for a given day t is based on only 17 (or less) data points. In order to increase the precision, we link the slope and intercept coefficients by a random walk, which allows information to be pooled across daily regressions. The result is smoothed slope and intercept estimates with narrower distributions.
v:sharei N m:v:sharei;t ; s2v;t
(2)
at N at1 ; s2a bt N bt1 ; s2b
3. Part 3: from votes to seats
s2a;t is the standard deviation of the intercepts, at, from the daily cross-sectional regressions; s2b;t is the standard deviation of the slopes, bt, from the daily cross-sectional regressions. The prior for at¼ED is U (0.6, 0.6) for the Liberal Democrats, and U (0.7, 0.7) for Labour and Conservatives; the prior for bt¼ED is U (0.98, 1.02) for the Liberal Democrats, and U (0.82, 1.18) for Labour and Conservatives. These priors assume that the election day slope coefficients are close to 1 and the election day intercept coefficients are close to 0. In other words, we assume there would be little regression to the mean from a hypothetical poll conducted on election day to the result on election day. With smoothed slope and intercept estimates and an estimate of the standard deviation of the residuals (s2v;t ), we can use our estimate of current poll shares (from Part 1) for a party to forecast election day vote shares. This is done in a single model as follows:
ft N m:ft ; s2v;t m:ft ¼ at þ bt *pollt (3)
m:v:sharei;t ¼ at þ bt *p:sharei;t at N at1 ; s2a bt N
bt1 ; s2b
Even with smoothed coefficients, the forecast confidence intervals are still relatively large, and so we pool forecasts on a weekly basis. For day t¼t, we average the forecasts from t¼t to t¼t6, which we denote ft¼tjt6:t, using Bayesian averaging to account for the precision of each forecast. Fig. 2 plots the pooled forecasts for Labour, Liberal Democrats and Conservatives up to the last day for which we have information (April 30th). Recall that if a party is polling above its historical equilibrium, it will tend to underperform the polls on election day. If a party is below the equilibrium, it will tend to over-perform. Of course, this election cycle has been unusual. To date, support for both Labour and the Conservatives has remained below the equilibrium history suggests, as voters have turned instead to alternative options - UKIP, the SNP, and more recently the Greens. The longer this structure of preferences remains in place, the less and less we expect Conservatives and Labour shares to recover towards their historic equilibrium.1 The final forecast is listed below with credible intervals to indicate our estimate about the degree of uncertainty that remains with a week left to election day. Con 35.0 (32.9, 37.0) Lab 32.6 (29.8, 35.4) LD 8.8 (6.7, 11.0)
m:v:sharei;t ¼ at þ bt *p:sharei;t
v:sharei N m:v:sharei;t ; s2v;t
245
Fig. 1 plots the smoothed slope and intercept estimates for the Conservatives, Liberal Democrats and Labour. As can be seen, the forecasted regression to the mean declines the closer one gets to election day, i.e., current poll results matter more and more.
The next step is to use the vote forecast to simulate electoral outcomes in each of the 631 constituencies in Great Britain (excluding the Speaker's seat). This allows us to calculate win probabilities for each party in each constituency and probabilities for various electoral outcomes, e.g., the probability that the Conservatives win more seats than Labour. In each simulation, the constituency level vote outcomes are estimated using a ‘deviation from uniform swing’ model. The forecasted vote for party p in constituency j based on information we have at the time of the forecast t¼current is modelled as:
fp;j ¼ v:sharep;j;2010 þ S v:S:swingp;t¼ED þ ð1 SÞ v:EW:swingp;t¼ED þ d:v:swingp;j;t¼ED ;
(4)
Where v.sharep,j,2010 is the share of the vote that party p received in constituency j in 2010. v.S.swingp,t¼ED is the forecasted uniform swing on election day (t ¼ ED) for party p for Scottish constituencies, v.EW.swingp,t¼ED is the same for constituencies in England and Wales (E&W), and S takes on the value 1 for Scottish constituencies and 0 for others, and d.v.swingp,j,t¼ED represents the forecasted constituency-specific deviations from uniform swing. The region-specific uniform swing forecasts (v.S.swingp,t¼ED and v.EW.swingp,t¼ED) are calculated in a three-step procedure in each simulation. First, the forecasted Great Britain swings (v.swingp,t¼ED) for Labour, the Conservatives, the Liberal Democrats and ‘other’ parties are drawn from the distributions:
N m fp v:share2010 ; s2ðf
p v:share2010
Þ
(5)
where fp for each party is a vector of an arbitrarily large (40,000) number of draws from the ft¼current from the Eq. (3) above. For each
1 In other words, the underlying equilibria really are not constant over time but change across election cycles and within cycles as well.
246
R. Ford et al. / Electoral Studies 41 (2016) 244e249
Fig. 1. Smoothed coefficients.
Fig. 2. Vote forecast by date, May 1st 2015.
party, this is the distribution of differences between the 2010 national vote result and the distribution of forecasted 2015 national vote results, based on current information. Second, the most recent vote intention polls in Scotland and Great Britain are used to estimate the current (time t) Scotland and Great Britain vote swings (swings from 2010 to the most recent polls), v.S.swingp,t v.GB.swingp,t for each party. These then are used to estimate the current vote swing for each party in E&W.
v:EW:swingp;t ¼
v:GB:swingp;t 0:088 v:S:swingp;t 0:912
swing, v.EW.swingp,t, for each party. Third, we make the assumption that any change in any swing for Scotland and E&W between now and election day will be proportionately the same. For example, if the swing in Scotland for Labour increases by 1 percent (not percentage point), the swing in E&W for Labour will also increase by 1 percent. Using this assumption, we can use the forecasted 2010/2015 Great Britain swing and the current Great Britain, Scotland and E&W swings to calculate forecasted 2010/2015 Scotland and E&W swings (for each party):
(6)
Here we take the vote swings from the latest Scottish polls and subtract them from the swings for the corresponding period for Great Britain, adjusting for population size, to calculate the E&W
v:S:swingp;t¼ED ¼ v:GB:swingp;t¼ED
v:S:swingp;t v:GB:swingp;t
R. Ford et al. / Electoral Studies 41 (2016) 244e249
v:EW:swingp;t¼ED ¼ v:GB:swingp;t¼ED
v:EW:swingp;t v:GB:swingp;t
(7)
The precision of the estimates of the current Great Britain, Scotland and E&W swings and the precision of the forecasted Great Britain swing used in these calculations provide us with the precision of the forecasted Scotland and E&W swings. This gives us distributions for the forecasted Scotland and E&W swings. For each simulation, we draw a Great Britain swing for each party (Conservatives, Liberal Democrats, Labour and ‘other’) from the distributions described in Eq. (5). The swings are scaled to add to zero and then used in (6) and (7) to calculate the forecasted Scotland and E&W swing for each party. The last element of our model (4) allocates constituency specific deviations from uniform swing, d.v.swingp,j,f. For constituencies where we have no additional information since 2010, the constituency-specific deviations from uniform swing are drawn from a distribution determined by the distribution of the deviations from uniform swing between 2005 and 2010.2 In other words, we use the range of actual swings that occurred between 2005 and 2010 to inform the range of possible swings in 2015. Each simulation randomly draws one possible swing from this distribution for each party in each constituency. So the overall range of swings in each simulation follows the distribution seen in previous elections, but the projected swing in each constituency varies - each simulation draws a new swing from the distribution for each constituency. In some (136) constituencies, we have constituency-specific polling. We use that information to update our expectation for the deviation from uniform swing using the constituency poll (point estimate and standard error) to specify normal distributions for the deviations from uniform swing.3 See the Online Appendix for the formula used. For UKIP and the SNP, we follow the same procedure used for the Conservatives, Labour and the Liberal Democrats, except we do not have a forecasted 2015 national vote and forecasted swing. What we do have is a forecasted 2015 national vote for ‘other’ parties, i.e. parties other than the Conservatives, Labour and the Liberal Democrats. We therefore need to determine the proportion of the ‘other’ uniform swing (both the other swing for E&W and for Scotland) that will go to UKIP and the SNP. We do this on a constituency by constituency basis. The default for the SNP is that 0.95 of the other uniform swing will go to SNP in Scotland constituencies and 0 in E&W. The default for UKIP is that 0.7 of the other uniform swing goes to UKIP in constituencies in England & Wales and 0 in Scotland. Therefore, Eq. (4) is adjusted as follows for UKIP and SNP:
fp;j ¼ v:sharep;j;2010 þ 1 oth:oth:shp;t¼ED S v:S:swingp;t¼ED þ 1 oth:oth:shp;t¼ED ð1 SÞ v:EW:swingp;t¼ED þ d:v:swingp;j;t¼ED
247
swing for UKIP we used the distribution of swings to UKIP in the 2014 European Parliament election.4 This is not a perfect proxy for the general election for several reasons, but we think it gives a better idea as to the likely distribution of UKIP swings than the last general election - not least because the overall rise in UKIP support between 2009 and 2014 in European Parliament elections (10.6 percentage points) is fairly similar to the rise in UKIP support seen in current general election polling. The current SNP surge also makes 2010 a poor choice for defining the distribution of SNP deviations from uniform swing. Here we use the distribution of deviations from the 2011 Scottish Parliament election, when the SNP vote rose by 12.5 percentage points.5 This is a rather smaller swing than current polls suggest, but was a major surge in support and should prove a more useful yardstick than the 2010 general election, when SNP support barely rose. Note also that we are not using this data to estimate the SNP swing but rather the distribution of the deviations from uniform swing.6 Also note that for UKIP we use the sum of votes for radical right parties in 2010 as our estimate of the 2010 UKIP vote e as we expect that UKIP, as the dominant representative of the radical right, will in 2015 win most of the support that previously went to the BNP, the National Front and the England Democrats. Even if such support does not flow directly to UKIP, we expect that the success of these smaller fringe right parties provides a useful proxy measure of UKIP potential in individual seats. For the remaining ‘other’ parties, we follow the same procedure as for UKIP and the SNP, with these parties receiving the proportion of the Scotland and E&W uniform swings not allocated to UKIP and SNP. It was not necessary to include a deviation from uniform swing, as this is largely already accounted for by the way we specified a different proportion of the other uniform swing going to remaining ‘others’ in constituencies where such parties are relevant e most notably seats with significant Plaid Cymru presence in Wales, and Green and Respect presence in England. For example, we allocate proportions of 'other' swing to Plaid Cymru and UKIP in seats where Plaid were strong in 2010 on the basis of the ratio between Plaid Cymru and UKIP support in the 2010 election. For example, in Monmouth in 2010, Plaid won 2.7% of the vote and UKIP 2.4%, and so we allocate 0.47 of the other swing to UKIP and the remainder (0.53) to Plaid. The simulation was repeated an arbitrarily large number of times (15,000 in total), and for each simulation we have an estimate of seat wins for each party. Aggregating these simulations gives us a range of possible outcomes. We can also calculate the percentage of times each party wins each constituency across all simulations. We interpret these as the probability that each party will win each constituency. 4. The forecast
(8)
oth.oth.shp,t¼ED is the proportion of the other vote that we assign to other ‘other’ parties e not UKIP or SNP. To calculate the distribution for the deviations from uniform
2 Conservatives Mean ¼ 0.077 Variance ¼ 11.90; Labour Mean ¼ 0.222 Variance ¼ 39.97; Lib Dem Mean ¼ 0.13 Variance ¼ 20.76. See the Online Appendix for the formula used. 3 In a couple of cases, specifically Clacton and Rochester & Strood, we use the result of the by-election as equivalent to a constituency poll with a large sample, since these provide useful information on deviation from uniform swing (this is not least important because the sitting MPs in these two seats may benefit from incumbency advantages in a way that other UKIP candidates may not).
Our forecast draws on national, Scottish and constituency polling up to April 30th, as described in the foregoing text. Inputting the vote forecasts into our constituency-level model, the seat forecast (excluding Northern Ireland) produces the following: CON 274 (251, 305) LAB 272 (244, 295) LD 24 (18, 29)
4
UKIP Mean ¼ 1.62 Variance ¼ 24.01. SNP Mean ¼ 0.12 Variance ¼ 16.56. 6 As before, if we have a constituency poll we use that information to update our expectation for the deviation from uniform swing for that constituency. 5
248
R. Ford et al. / Electoral Studies 41 (2016) 244e249
UKIP 2 (1, 4) SNP 53 (46, 58) OTH 7 (5, 9) Northern Ireland (not forecasted): 18 seats
To begin with, consider our original seat forecast:
This suggests that the race for seats is on a knife-edge and that the Conservatives are slightly (53%e47%) more likely to be the largest party. They are not at all likely (less than 0.2%) to have a majority, however, and, their paths to a governing coalition are even more winding. They cannot reach a majority with the backing of the Liberal Democrats (only a 6 percent chance that the Conservatives and Liberal Democrats will have enough seats between them to form a majority) or with both the Liberal Democrats and the Northern Irish DUP (only an 8 percent chance that the Conservatives, Liberal Democrats and DUP will have enough seats between them to form a majority), or even by adding UKIP to that two party combination (only a 10 percent chance). It would be very hard, with this seat outcome, for the Conservatives to sustain a government without some form of acquiescence from the Scottish Nationalists, who have ruled it out. It will also be very difficult for Labour to form a majority government with the backing of the Liberal Democrats (only a 1.5 percent chance that the two parties will have a majority between them). By contrast, there is a 60 percent chance that Labour and the Scottish National Party will be able command a majority between them. This increases to over 95 percent if you add the Liberal Democrats into the coalition mix. If our forecast proves accurate, the 2015 hung parliament may remain hung well beyond election day, as bargaining among parties plays out. (For a forecast based on poll results as of May 5th 2015, please see the Online Appendix.) 5. From polls to votes to seats: after the 2015 British general election, post mortem Our pre-election expectations did not pan out, as we forecasted a hung parliament with a high level eover 99% – of certainty. Although the final outcome was close to being hung, the Conservatives Party managed a small majority of 12. Where did our forecast go wrong? Here we consider both our estimates of the national vote and then the translation of vote shares into seats. As it turns out, both mattered, though the former a little more than the latter, and the polls had meaningful consequences at both stages. Consider the performance of our vote forecast, as follows:
CON LAB LD
Vote forecast
Actual vote
Error
35.0 32.6 8.8
37.8 31.2 8.1
2.8 1.4 0.7
On average the forecasted shares here look pretty good, off on average about 1.6%. This is better than the raw poll averages just before the election, which put the Conservatives at 34.2% and Labour at 33.1%, further from the final vote tallies. Our translation of polls shares into vote shares thus helped bridge the gap - the Conservatives did outperform the polls, and Labour underperformed them, just as the forecast predicted. It still left us quite a bit off, and since the errors for the Conservatives and Labour were in different directions our forecast of the Conservatives' winning margin over Labour was off by even more - 4.2%. Let us see how much difference this made for our forecast of seats.
CON LAB LD UKIP SNP OTH
Seat forecast
Seats won
Error
274 272 24 2 53 7
331 232 8 1 56 4
57 40 16 1 3 3
(251, 305) (244, 295) (18, 19) (1, 4) (46, 58) (5, 9)
The seat projections clearly were off for the two major parties – by a total of 97 seats – and also the Liberal Democrats, who won 16 seats less than we forecast. We were much more successful with the other parties, particularly the SNP. Let us consider how much difference the polls made, keeping in mind that we relied on both national polls to forecast vote shares and constituency polls to forecast seats. For our analysis, we first rerun our final pre-election seat forecast excluding the constituency polls. The results are shown in the second column in the table just below (the first column displays our original forecast).
CON LAB LD SNP UKIP OTH Mean Abs Error (Con/Lab/LD)
Original forecast
Without constituency polls
With Actual vote share
274 (251, 305) 272 (244, 295) 24 (18, 29) 53 (46, 58) 2 (1, 4) 7 (5, 9) 37.7
285 (258, 313) 272 (249, 296) 17 (11, 24) 50 (38, 57) 0 (0, 1) 6 (4, 9) 31.7
321 (293, 350) 246 (223, 270) 13 (8, 20) 43 (29, 54) 0 (0, 2) 7 (4, 9) 9.6
Here we can see that including constituency polls in our forecast, which we expected to make things better given the emerging multipartyism, made things worse. Our estimates for both the Conservatives and the Liberal Democrats are more accurate with the constituency polls excluded, and in the pack of other political science forecasts. Using constituency polls did not make a huge difference, however, and accounts for only 16% of the mean absolute error in our original seat forecast, which leaves most of the error yet unexplained. National polls are a big part of the story. This is clear from the third column of the table (see just above), which shows the predicted seats using the election day vote shares in place of those we forecasted based on the polls.7 Using the actual vote shares e and not using constituency polls e produces seat forecasts which fairly closely match the final outcome, though this still leaves the Conservative Party without a clear majority (of 321 seats). The Labour Party share also is a bit too high and Liberal Democrats too, though now much closer to the final result. Further, the result is now within our 95 percent credible intervals. What are modest poll errors by historical standards, both in the UK (see Wlezien et al., 2013) and elsewhere (see Jennings and Wlezien, 2015), had substantial effects on the distribution of seats.
7 In running the simulations, we allow the degree of error in the forecasted vote to be the same as in our original forecast, even though we correct the forecasted vote to be the same as the election outcome.
R. Ford et al. / Electoral Studies 41 (2016) 244e249
All told, our analysis suggest that the incorrect vote forecast e due to errors in the national polls and our failure to fully correct for those errors e accounts for about 60% of the mean absolute seat forecast error. The remainder has to do with the translation of votes into seats. Some of that error reflects the inclusion of constituency polls, as we have seen, but other aspects of our seat-level model matter more, accounting for about 25% of the total forecast error, i.e., the mean absolute error of 9.6 for the seat forecast using the actual vote share as a proportion of the error of 37.7 from the original forecast. These aspects are a worthy subject for future research as they, unlike the polls, are within our control when forecasting. Acknowledgements Earlier versions of the forecast were presented at the London School of Economics and Political Science, the University of Southampton, and at the National Centre for Research Methods conference on Predicting and Understanding the 2015 General Election. We would like to thank those in attendance for their helpful comments. We would also like to acknowledge that this work builds on our previous work conducted with our colleague Stephen Fisher.
249
Appendix A. Supplementary data Supplementary data related to this article can be found at http:// dx.doi.org/10.1016/j.electstud.2015.11.013. References Erikson, Robert S., Wlezien, Christopher, 2012. The Timeline of Presidential Elections. University of Chicago Press, Chicago. Erikson, Robert S., Wlezien, Christopher, 1999. Presidential polls as a time series: the case of 1996. Public Opin. Q. 63, 163e177. Fisher, Stephen, Ford, Robert, Jennings, Will, Pickup, Mark, Wlezien, Christopher, 2011. From polls to votes to seats: forecasting the 2010 British general election. Elect. Stud. 30 (2), 250e257. Jackman, Simon, 2005. Pooling the polls over an election campaign. Aust. J. Political Sci. 40 (4), 499e516. Jennings, Will, Wlezien, Christopher, 2015. The timeline of elections: a comparative perspective. Am. J. Political Sci. http://dx.doi.org/10.1111/ajps.12189. Advance online publication. Pickup, Mark, Johnston, Richard, 2008. Campaign trial heats as election forecasts: measurement error and Bias in 2004 presidential campaign polls. Int. J. Forecast. 24, 270e282. Pickup, Mark, Johnston, Richard, 2007. Campaign trial heats as election forecasts: evidence from the 2004 and 2006 Canadian elections. Elect. Stud. 26, 460e476. Wlezien, Christopher, Erikson, Robert S., 2002. The timeline of presidential election campaigns. J. Polit. 64, 969e993. Wlezien, Christopher, Jennings, Will, Fisher, Stephen, Ford, Robert, Pickup, Mark, 2013. Polls and the vote in Britain. Polit. Stud. 61 (S1), 129e154.
Contents lists available at ScienceDirect
Electoral Studies journal homepage: www.elsevier.com/locate/electstud
From polls to votes to seats: Forecasting the 2015 British general election Robert Ford a, Will Jennings b, Mark Pickup c, *, Christopher Wlezien d a
University of Manchester, UK University of Southampton, UK c Simon Fraser University, Canada d University of Texas at Austin, USA b
a r t i c l e i n f o
a b s t r a c t
Article history: Available online 30 November 2015
This paper develops a three-stage method to forecast parliamentary election results from vote preferences in British opinion polls: (1) adjusting and aggregating vote-intentions from different polling organizations; (2) forecasting how public support for parties will change in the period before election day; and (3) translating, through simulations, the forecast of election day vote shares into seat totals while incorporating constituency-level information, including local vote-intention polls. Overall, this approach seeks to combine relevant national, regional and local information, and uncertainty about that information, to better reflect the fragmentation and diversity of political contexts found in the new era of five/ six-party British politics. © 2015 Elsevier Ltd. All rights reserved.
Keywords: Forecasting British general election Vote intention Seat prediction
Our goal at the outset, building on the method we developed in 2010 (see Fisher et al., 2011), was to produce an election forecast that includes a meaningful estimate of (un)certainty. There are three parts. 1. Part 1: noise and the polls Since early in the parliament, the Polling Observatory (pollob.ca) team have been keeping track of movements in the polls on a regular basis. The key to tracking public opinion is to be cautious about short-term fluctuations e often the product of random noise or pollsters' methodological choicese and focus on the underlying trends. There have been notable shifts in vote intentions, such as the sharp decline in Liberal Democrat support after their decision to go into coalition with the Conservatives in 2010, the substantial drop in support for the Conservatives after the infamous ‘omnishambles’ budget in Spring 2012, and the slow but steady decline in Labour support, and rise in UKIP support, since late 2012. The first stage of our method produces estimated ranges of support for the
* Corresponding author. Department of Political Science, Simon Fraser University, 8888 University Drive, Burnaby, BC, V5A 1S6, Canada. E-mail addresses: [email protected] (R. Ford), [email protected]. uk (W. Jennings), [email protected] (M. Pickup), [email protected] (C. Wlezien). http://dx.doi.org/10.1016/j.electstud.2015.11.013 0261-3794/© 2015 Elsevier Ltd. All rights reserved.
parties that take into account the uncertainty inherent in each poll (for previous applications see Jackman, 2005; Pickup and Johnston, 2007; Fisher et al., 2011). The result is an aggregation of the polls that is preferable to a simple poll-of-polls, since our estimates do not vary as a function of the mix of polling firms that have been in the field recently or by which pollsters poll more frequently (Erikson and Wlezien, 1999; Jackman, 2005; Pickup and Johnston, 2008). As of April 30th, our estimates show the race as very close, with Labour on 33.1% and the Conservatives on 34.2% – but we cannot say with confidence that this Conservative lead is greater than zero.
2. Part 2: vote forecast: lessons from polling history The vote component of our forecast uses the historical relationship between polls and the election day vote from past elections to project forward from the current polling. A key idea here is ‘regression to the mean’: that if a party is polling above its historical equilibrium, it will tend to underperform the polls on Election Day. If a party is below the equilibrium, it will tend to do better. We model this relationship as follows:
v:sharei N m:v:sharei;t ; s2v;t
(1)
R. Ford et al. / Electoral Studies 41 (2016) 244e249
m:v:sharei;t ¼ at þ bt *p:sharei;t v.sharei is the share of the vote received by the party in election i; and p.sharei,t is the share of the vote intention received by the party in election i on day t; m.v.sharei,t is the expected value of the share of the vote received by the party in election i conditioning on the polls on day t; s2v;t is the standard deviation of the residuals from the daily cross-sectional regressions. Using the results from 3293 polls across 17 elections we can estimate at, bt and s2v;t (Wlezien and Erikson, 2002; Erikson and Wlezien, 2012; also see Wlezien et al., 2013; Jennings and Wlezien, 2015). However, the precision of these estimates is reasonably low, as each at, bt and s2v;t for a given day t is based on only 17 (or less) data points. In order to increase the precision, we link the slope and intercept coefficients by a random walk, which allows information to be pooled across daily regressions. The result is smoothed slope and intercept estimates with narrower distributions.
v:sharei N m:v:sharei;t ; s2v;t
(2)
at N at1 ; s2a bt N bt1 ; s2b
3. Part 3: from votes to seats
s2a;t is the standard deviation of the intercepts, at, from the daily cross-sectional regressions; s2b;t is the standard deviation of the slopes, bt, from the daily cross-sectional regressions. The prior for at¼ED is U (0.6, 0.6) for the Liberal Democrats, and U (0.7, 0.7) for Labour and Conservatives; the prior for bt¼ED is U (0.98, 1.02) for the Liberal Democrats, and U (0.82, 1.18) for Labour and Conservatives. These priors assume that the election day slope coefficients are close to 1 and the election day intercept coefficients are close to 0. In other words, we assume there would be little regression to the mean from a hypothetical poll conducted on election day to the result on election day. With smoothed slope and intercept estimates and an estimate of the standard deviation of the residuals (s2v;t ), we can use our estimate of current poll shares (from Part 1) for a party to forecast election day vote shares. This is done in a single model as follows:
ft N m:ft ; s2v;t m:ft ¼ at þ bt *pollt (3)
m:v:sharei;t ¼ at þ bt *p:sharei;t at N at1 ; s2a bt N
bt1 ; s2b
Even with smoothed coefficients, the forecast confidence intervals are still relatively large, and so we pool forecasts on a weekly basis. For day t¼t, we average the forecasts from t¼t to t¼t6, which we denote ft¼tjt6:t, using Bayesian averaging to account for the precision of each forecast. Fig. 2 plots the pooled forecasts for Labour, Liberal Democrats and Conservatives up to the last day for which we have information (April 30th). Recall that if a party is polling above its historical equilibrium, it will tend to underperform the polls on election day. If a party is below the equilibrium, it will tend to over-perform. Of course, this election cycle has been unusual. To date, support for both Labour and the Conservatives has remained below the equilibrium history suggests, as voters have turned instead to alternative options - UKIP, the SNP, and more recently the Greens. The longer this structure of preferences remains in place, the less and less we expect Conservatives and Labour shares to recover towards their historic equilibrium.1 The final forecast is listed below with credible intervals to indicate our estimate about the degree of uncertainty that remains with a week left to election day. Con 35.0 (32.9, 37.0) Lab 32.6 (29.8, 35.4) LD 8.8 (6.7, 11.0)
m:v:sharei;t ¼ at þ bt *p:sharei;t
v:sharei N m:v:sharei;t ; s2v;t
245
Fig. 1 plots the smoothed slope and intercept estimates for the Conservatives, Liberal Democrats and Labour. As can be seen, the forecasted regression to the mean declines the closer one gets to election day, i.e., current poll results matter more and more.
The next step is to use the vote forecast to simulate electoral outcomes in each of the 631 constituencies in Great Britain (excluding the Speaker's seat). This allows us to calculate win probabilities for each party in each constituency and probabilities for various electoral outcomes, e.g., the probability that the Conservatives win more seats than Labour. In each simulation, the constituency level vote outcomes are estimated using a ‘deviation from uniform swing’ model. The forecasted vote for party p in constituency j based on information we have at the time of the forecast t¼current is modelled as:
fp;j ¼ v:sharep;j;2010 þ S v:S:swingp;t¼ED þ ð1 SÞ v:EW:swingp;t¼ED þ d:v:swingp;j;t¼ED ;
(4)
Where v.sharep,j,2010 is the share of the vote that party p received in constituency j in 2010. v.S.swingp,t¼ED is the forecasted uniform swing on election day (t ¼ ED) for party p for Scottish constituencies, v.EW.swingp,t¼ED is the same for constituencies in England and Wales (E&W), and S takes on the value 1 for Scottish constituencies and 0 for others, and d.v.swingp,j,t¼ED represents the forecasted constituency-specific deviations from uniform swing. The region-specific uniform swing forecasts (v.S.swingp,t¼ED and v.EW.swingp,t¼ED) are calculated in a three-step procedure in each simulation. First, the forecasted Great Britain swings (v.swingp,t¼ED) for Labour, the Conservatives, the Liberal Democrats and ‘other’ parties are drawn from the distributions:
N m fp v:share2010 ; s2ðf
p v:share2010
Þ
(5)
where fp for each party is a vector of an arbitrarily large (40,000) number of draws from the ft¼current from the Eq. (3) above. For each
1 In other words, the underlying equilibria really are not constant over time but change across election cycles and within cycles as well.
246
R. Ford et al. / Electoral Studies 41 (2016) 244e249
Fig. 1. Smoothed coefficients.
Fig. 2. Vote forecast by date, May 1st 2015.
party, this is the distribution of differences between the 2010 national vote result and the distribution of forecasted 2015 national vote results, based on current information. Second, the most recent vote intention polls in Scotland and Great Britain are used to estimate the current (time t) Scotland and Great Britain vote swings (swings from 2010 to the most recent polls), v.S.swingp,t v.GB.swingp,t for each party. These then are used to estimate the current vote swing for each party in E&W.
v:EW:swingp;t ¼
v:GB:swingp;t 0:088 v:S:swingp;t 0:912
swing, v.EW.swingp,t, for each party. Third, we make the assumption that any change in any swing for Scotland and E&W between now and election day will be proportionately the same. For example, if the swing in Scotland for Labour increases by 1 percent (not percentage point), the swing in E&W for Labour will also increase by 1 percent. Using this assumption, we can use the forecasted 2010/2015 Great Britain swing and the current Great Britain, Scotland and E&W swings to calculate forecasted 2010/2015 Scotland and E&W swings (for each party):
(6)
Here we take the vote swings from the latest Scottish polls and subtract them from the swings for the corresponding period for Great Britain, adjusting for population size, to calculate the E&W
v:S:swingp;t¼ED ¼ v:GB:swingp;t¼ED
v:S:swingp;t v:GB:swingp;t
R. Ford et al. / Electoral Studies 41 (2016) 244e249
v:EW:swingp;t¼ED ¼ v:GB:swingp;t¼ED
v:EW:swingp;t v:GB:swingp;t
(7)
The precision of the estimates of the current Great Britain, Scotland and E&W swings and the precision of the forecasted Great Britain swing used in these calculations provide us with the precision of the forecasted Scotland and E&W swings. This gives us distributions for the forecasted Scotland and E&W swings. For each simulation, we draw a Great Britain swing for each party (Conservatives, Liberal Democrats, Labour and ‘other’) from the distributions described in Eq. (5). The swings are scaled to add to zero and then used in (6) and (7) to calculate the forecasted Scotland and E&W swing for each party. The last element of our model (4) allocates constituency specific deviations from uniform swing, d.v.swingp,j,f. For constituencies where we have no additional information since 2010, the constituency-specific deviations from uniform swing are drawn from a distribution determined by the distribution of the deviations from uniform swing between 2005 and 2010.2 In other words, we use the range of actual swings that occurred between 2005 and 2010 to inform the range of possible swings in 2015. Each simulation randomly draws one possible swing from this distribution for each party in each constituency. So the overall range of swings in each simulation follows the distribution seen in previous elections, but the projected swing in each constituency varies - each simulation draws a new swing from the distribution for each constituency. In some (136) constituencies, we have constituency-specific polling. We use that information to update our expectation for the deviation from uniform swing using the constituency poll (point estimate and standard error) to specify normal distributions for the deviations from uniform swing.3 See the Online Appendix for the formula used. For UKIP and the SNP, we follow the same procedure used for the Conservatives, Labour and the Liberal Democrats, except we do not have a forecasted 2015 national vote and forecasted swing. What we do have is a forecasted 2015 national vote for ‘other’ parties, i.e. parties other than the Conservatives, Labour and the Liberal Democrats. We therefore need to determine the proportion of the ‘other’ uniform swing (both the other swing for E&W and for Scotland) that will go to UKIP and the SNP. We do this on a constituency by constituency basis. The default for the SNP is that 0.95 of the other uniform swing will go to SNP in Scotland constituencies and 0 in E&W. The default for UKIP is that 0.7 of the other uniform swing goes to UKIP in constituencies in England & Wales and 0 in Scotland. Therefore, Eq. (4) is adjusted as follows for UKIP and SNP:
fp;j ¼ v:sharep;j;2010 þ 1 oth:oth:shp;t¼ED S v:S:swingp;t¼ED þ 1 oth:oth:shp;t¼ED ð1 SÞ v:EW:swingp;t¼ED þ d:v:swingp;j;t¼ED
247
swing for UKIP we used the distribution of swings to UKIP in the 2014 European Parliament election.4 This is not a perfect proxy for the general election for several reasons, but we think it gives a better idea as to the likely distribution of UKIP swings than the last general election - not least because the overall rise in UKIP support between 2009 and 2014 in European Parliament elections (10.6 percentage points) is fairly similar to the rise in UKIP support seen in current general election polling. The current SNP surge also makes 2010 a poor choice for defining the distribution of SNP deviations from uniform swing. Here we use the distribution of deviations from the 2011 Scottish Parliament election, when the SNP vote rose by 12.5 percentage points.5 This is a rather smaller swing than current polls suggest, but was a major surge in support and should prove a more useful yardstick than the 2010 general election, when SNP support barely rose. Note also that we are not using this data to estimate the SNP swing but rather the distribution of the deviations from uniform swing.6 Also note that for UKIP we use the sum of votes for radical right parties in 2010 as our estimate of the 2010 UKIP vote e as we expect that UKIP, as the dominant representative of the radical right, will in 2015 win most of the support that previously went to the BNP, the National Front and the England Democrats. Even if such support does not flow directly to UKIP, we expect that the success of these smaller fringe right parties provides a useful proxy measure of UKIP potential in individual seats. For the remaining ‘other’ parties, we follow the same procedure as for UKIP and the SNP, with these parties receiving the proportion of the Scotland and E&W uniform swings not allocated to UKIP and SNP. It was not necessary to include a deviation from uniform swing, as this is largely already accounted for by the way we specified a different proportion of the other uniform swing going to remaining ‘others’ in constituencies where such parties are relevant e most notably seats with significant Plaid Cymru presence in Wales, and Green and Respect presence in England. For example, we allocate proportions of 'other' swing to Plaid Cymru and UKIP in seats where Plaid were strong in 2010 on the basis of the ratio between Plaid Cymru and UKIP support in the 2010 election. For example, in Monmouth in 2010, Plaid won 2.7% of the vote and UKIP 2.4%, and so we allocate 0.47 of the other swing to UKIP and the remainder (0.53) to Plaid. The simulation was repeated an arbitrarily large number of times (15,000 in total), and for each simulation we have an estimate of seat wins for each party. Aggregating these simulations gives us a range of possible outcomes. We can also calculate the percentage of times each party wins each constituency across all simulations. We interpret these as the probability that each party will win each constituency. 4. The forecast
(8)
oth.oth.shp,t¼ED is the proportion of the other vote that we assign to other ‘other’ parties e not UKIP or SNP. To calculate the distribution for the deviations from uniform
2 Conservatives Mean ¼ 0.077 Variance ¼ 11.90; Labour Mean ¼ 0.222 Variance ¼ 39.97; Lib Dem Mean ¼ 0.13 Variance ¼ 20.76. See the Online Appendix for the formula used. 3 In a couple of cases, specifically Clacton and Rochester & Strood, we use the result of the by-election as equivalent to a constituency poll with a large sample, since these provide useful information on deviation from uniform swing (this is not least important because the sitting MPs in these two seats may benefit from incumbency advantages in a way that other UKIP candidates may not).
Our forecast draws on national, Scottish and constituency polling up to April 30th, as described in the foregoing text. Inputting the vote forecasts into our constituency-level model, the seat forecast (excluding Northern Ireland) produces the following: CON 274 (251, 305) LAB 272 (244, 295) LD 24 (18, 29)
4
UKIP Mean ¼ 1.62 Variance ¼ 24.01. SNP Mean ¼ 0.12 Variance ¼ 16.56. 6 As before, if we have a constituency poll we use that information to update our expectation for the deviation from uniform swing for that constituency. 5
248
R. Ford et al. / Electoral Studies 41 (2016) 244e249
UKIP 2 (1, 4) SNP 53 (46, 58) OTH 7 (5, 9) Northern Ireland (not forecasted): 18 seats
To begin with, consider our original seat forecast:
This suggests that the race for seats is on a knife-edge and that the Conservatives are slightly (53%e47%) more likely to be the largest party. They are not at all likely (less than 0.2%) to have a majority, however, and, their paths to a governing coalition are even more winding. They cannot reach a majority with the backing of the Liberal Democrats (only a 6 percent chance that the Conservatives and Liberal Democrats will have enough seats between them to form a majority) or with both the Liberal Democrats and the Northern Irish DUP (only an 8 percent chance that the Conservatives, Liberal Democrats and DUP will have enough seats between them to form a majority), or even by adding UKIP to that two party combination (only a 10 percent chance). It would be very hard, with this seat outcome, for the Conservatives to sustain a government without some form of acquiescence from the Scottish Nationalists, who have ruled it out. It will also be very difficult for Labour to form a majority government with the backing of the Liberal Democrats (only a 1.5 percent chance that the two parties will have a majority between them). By contrast, there is a 60 percent chance that Labour and the Scottish National Party will be able command a majority between them. This increases to over 95 percent if you add the Liberal Democrats into the coalition mix. If our forecast proves accurate, the 2015 hung parliament may remain hung well beyond election day, as bargaining among parties plays out. (For a forecast based on poll results as of May 5th 2015, please see the Online Appendix.) 5. From polls to votes to seats: after the 2015 British general election, post mortem Our pre-election expectations did not pan out, as we forecasted a hung parliament with a high level eover 99% – of certainty. Although the final outcome was close to being hung, the Conservatives Party managed a small majority of 12. Where did our forecast go wrong? Here we consider both our estimates of the national vote and then the translation of vote shares into seats. As it turns out, both mattered, though the former a little more than the latter, and the polls had meaningful consequences at both stages. Consider the performance of our vote forecast, as follows:
CON LAB LD
Vote forecast
Actual vote
Error
35.0 32.6 8.8
37.8 31.2 8.1
2.8 1.4 0.7
On average the forecasted shares here look pretty good, off on average about 1.6%. This is better than the raw poll averages just before the election, which put the Conservatives at 34.2% and Labour at 33.1%, further from the final vote tallies. Our translation of polls shares into vote shares thus helped bridge the gap - the Conservatives did outperform the polls, and Labour underperformed them, just as the forecast predicted. It still left us quite a bit off, and since the errors for the Conservatives and Labour were in different directions our forecast of the Conservatives' winning margin over Labour was off by even more - 4.2%. Let us see how much difference this made for our forecast of seats.
CON LAB LD UKIP SNP OTH
Seat forecast
Seats won
Error
274 272 24 2 53 7
331 232 8 1 56 4
57 40 16 1 3 3
(251, 305) (244, 295) (18, 19) (1, 4) (46, 58) (5, 9)
The seat projections clearly were off for the two major parties – by a total of 97 seats – and also the Liberal Democrats, who won 16 seats less than we forecast. We were much more successful with the other parties, particularly the SNP. Let us consider how much difference the polls made, keeping in mind that we relied on both national polls to forecast vote shares and constituency polls to forecast seats. For our analysis, we first rerun our final pre-election seat forecast excluding the constituency polls. The results are shown in the second column in the table just below (the first column displays our original forecast).
CON LAB LD SNP UKIP OTH Mean Abs Error (Con/Lab/LD)
Original forecast
Without constituency polls
With Actual vote share
274 (251, 305) 272 (244, 295) 24 (18, 29) 53 (46, 58) 2 (1, 4) 7 (5, 9) 37.7
285 (258, 313) 272 (249, 296) 17 (11, 24) 50 (38, 57) 0 (0, 1) 6 (4, 9) 31.7
321 (293, 350) 246 (223, 270) 13 (8, 20) 43 (29, 54) 0 (0, 2) 7 (4, 9) 9.6
Here we can see that including constituency polls in our forecast, which we expected to make things better given the emerging multipartyism, made things worse. Our estimates for both the Conservatives and the Liberal Democrats are more accurate with the constituency polls excluded, and in the pack of other political science forecasts. Using constituency polls did not make a huge difference, however, and accounts for only 16% of the mean absolute error in our original seat forecast, which leaves most of the error yet unexplained. National polls are a big part of the story. This is clear from the third column of the table (see just above), which shows the predicted seats using the election day vote shares in place of those we forecasted based on the polls.7 Using the actual vote shares e and not using constituency polls e produces seat forecasts which fairly closely match the final outcome, though this still leaves the Conservative Party without a clear majority (of 321 seats). The Labour Party share also is a bit too high and Liberal Democrats too, though now much closer to the final result. Further, the result is now within our 95 percent credible intervals. What are modest poll errors by historical standards, both in the UK (see Wlezien et al., 2013) and elsewhere (see Jennings and Wlezien, 2015), had substantial effects on the distribution of seats.
7 In running the simulations, we allow the degree of error in the forecasted vote to be the same as in our original forecast, even though we correct the forecasted vote to be the same as the election outcome.
R. Ford et al. / Electoral Studies 41 (2016) 244e249
All told, our analysis suggest that the incorrect vote forecast e due to errors in the national polls and our failure to fully correct for those errors e accounts for about 60% of the mean absolute seat forecast error. The remainder has to do with the translation of votes into seats. Some of that error reflects the inclusion of constituency polls, as we have seen, but other aspects of our seat-level model matter more, accounting for about 25% of the total forecast error, i.e., the mean absolute error of 9.6 for the seat forecast using the actual vote share as a proportion of the error of 37.7 from the original forecast. These aspects are a worthy subject for future research as they, unlike the polls, are within our control when forecasting. Acknowledgements Earlier versions of the forecast were presented at the London School of Economics and Political Science, the University of Southampton, and at the National Centre for Research Methods conference on Predicting and Understanding the 2015 General Election. We would like to thank those in attendance for their helpful comments. We would also like to acknowledge that this work builds on our previous work conducted with our colleague Stephen Fisher.
249
Appendix A. Supplementary data Supplementary data related to this article can be found at http:// dx.doi.org/10.1016/j.electstud.2015.11.013. References Erikson, Robert S., Wlezien, Christopher, 2012. The Timeline of Presidential Elections. University of Chicago Press, Chicago. Erikson, Robert S., Wlezien, Christopher, 1999. Presidential polls as a time series: the case of 1996. Public Opin. Q. 63, 163e177. Fisher, Stephen, Ford, Robert, Jennings, Will, Pickup, Mark, Wlezien, Christopher, 2011. From polls to votes to seats: forecasting the 2010 British general election. Elect. Stud. 30 (2), 250e257. Jackman, Simon, 2005. Pooling the polls over an election campaign. Aust. J. Political Sci. 40 (4), 499e516. Jennings, Will, Wlezien, Christopher, 2015. The timeline of elections: a comparative perspective. Am. J. Political Sci. http://dx.doi.org/10.1111/ajps.12189. Advance online publication. Pickup, Mark, Johnston, Richard, 2008. Campaign trial heats as election forecasts: measurement error and Bias in 2004 presidential campaign polls. Int. J. Forecast. 24, 270e282. Pickup, Mark, Johnston, Richard, 2007. Campaign trial heats as election forecasts: evidence from the 2004 and 2006 Canadian elections. Elect. Stud. 26, 460e476. Wlezien, Christopher, Erikson, Robert S., 2002. The timeline of presidential election campaigns. J. Polit. 64, 969e993. Wlezien, Christopher, Jennings, Will, Fisher, Stephen, Ford, Robert, Pickup, Mark, 2013. Polls and the vote in Britain. Polit. Stud. 61 (S1), 129e154.