- Email: [email protected]
Methodologies of election forecasting: Calling the 2010 UK “hung parliament”
Electoral Studies 30 (2011) 247–249
Contents lists available at ScienceDirect
Electoral Studies journal homepage: www.elsevier.com/locate/electstud
Methodologies of election forecasting: Calling the 2010 UK “hung parliament” Rachel Gibson a, *, Michael S. Lewis-Beck b a b
Department of Political Science, University of Manchester, Manchester M13 9PL, UK Department of Political Science, University of Iowa, Iowa, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 17 September 2010 Accepted 22 September 2010
Election forecasting has experienced considerable growth as a subfield within political science. Research work on United Kingdom elections has been cutting edge here. The recent 2010 general election afforded the opportunity for a trial of different forecasting methodologies. These efforts are showcased in this volume, and include standard, and notso-standard, statistical models. Overall, these models perform well, foreseeing the unprecedented outcome of a “hung parliament”, as most pollsters and pundits did not. Moreover, they achieved this accuracy with forecasts well in advance of the election itself. Ó 2010 Published by Elsevier Ltd.
Keywords: Election forecasting UK elections Hung parliament Prediction models
Election forecasting in the United Kingdom has enjoyed an increasing profile since the general election of 2001, and by 2010 the enterprise vibrates from the competition among several different forecasting teams. Furthermore, their efforts have met with considerable success. As this volume reveals, as a group, they were near consensus in predicting that no party would achieve the seat majority necessary to govern after the election of 2010, with the majority forecasting a “hung parliament”. Such an outcome is no mean feat, for three reasons. First, this forecast is a rare event. A hung parliament, has occurred only twice in the last century. Second, the forecasts were made in March 2010, well before the May 6 election date, at a specially convened conference held at the University of Manchester. Thirdly, at the time of the forecast pundits and pollsters were generally highly skeptical about the prospect of a hung parliament. While the recent UK election saw the most widespread forecasting effort by academic teams, their success was built on efforts begun in earnest about ten years ago. Beginning with the 2001 national contest, UK election analysts started to seriously engage in forecasting, i.e., putting forward ex ante models predicting the votes/seats * Corresponding author. E-mail address: [email protected] (R. Gibson). 0261-3794/$ – see front matter Ó 2010 Published by Elsevier Ltd. doi:10.1016/j.electstud.2010.09.003
outcome. [(For a current review of this literature, see Nadeau et al., 2009)]. Further application of the practice followed in the general election of 2005, and now 2010. On each occasion, a forecasting symposium was convened with the results published in a major journal. Electoral Studies hosted the first collection of papers, and now again serves as host, with the publication of this special issue. As editors we are pleased to present such a rich collection of methodologically sophisticated and diverse approaches to the topic. In particular we would seek to draw reader attention to the fact that all papers were submitted to Electoral Studies before the May election date, and therefore appear here in their original form. Thus, they represent true forecasts, taking place prior to the actual event itself. It is within the context of this important methodological constraint, then, that we seek to summarize the contributions of these works to the wider literature. Traditionally, the leading forecasting method has been polling, where vote intentions are gauged in national surveys by a question such as “If the general election were held tomorrow, which party would you vote for?” In Table 1 we report results from the Poll of Polls (an aggregation of available polls) about two months before the 2010 election. As can be seen, as an estimator of the vote share, it is off two points for Labour, one point for the Conservatives, and
248
R. Gibson, M.S. Lewis-Beck / Electoral Studies 30 (2011) 247–249
Table 1 Evaluating rival forecasts of the 2010 UK contest.a Teamb
Vote share (%)c (L, C, LD)
Seat share (%)c (L, C, LD)
Actual POP BES FFJPW LN LNB LBS RTBL WSSC
(29, 36, 23) (31, 37, 19) (28, 36, 27) (32, 39, 20) (–, –, –) (36, 42, –) (–, –, –) (27, 36, 27) (–, –, –)
(258, 306, 57)
Rank (L, C)d (Seats prediction)
Leade (months) 2 .03
(275, 299, 46) (287, 285, –) (303, 293, –) (–, 358, –) (237, 299, 83) (273, 282, 41)
(2, 1.5) (4,4) (5, 3) (6,6) (3, 1.5) (1, 5)
1 2 3 1 1 6
a
If an entry is blank (–), that signifies no forecast was reported in that instance. Initials (first letter of each last name) for each forecasting team or organization: Poll of Polls; British Election Study; Fisher, Ford, Jennings, Pickup, Wlezien; Lebo and Norpoth; Lewis-Beck, Nadeau, and Bélanger; Lewis-Beck and Stegmaier; Rallings, Thrasher, Borisyuk, Long; Whiteley, Sanders, Stewart, Clark. c The percentage of the total vote share received by the party (L ¼ Labour, C ¼ Conservative), LD ¼ Liberal Democrat) at the national level. d The accuracy ranking of the seats forecast for the team, from most accurate (rank ¼ 1.0) to least accurate (rank ¼ 6) for Labour (L) and the Conservatives (C). e Lead time before the election in months. b
four points for the Liberal Democrats. Compare this to the estimates from another survey, that of the British Election Study (BES), the day before the election. It does somewhat better than the Poll of Polls (POP). The BES error is one point for Labour, zero points for the Conservatives, and four points for the Liberal Democrats. Either of these polling instruments –the POP or the BES – shows an overwhelming vote lead for the Conservatives. It is on the basis of such results that pre-election commentators easily believed that the Conservatives would secure a majority. Indeed in March, the editorial of a leading national paper ventured it was virtually impossible for the Conservatives to lose based on its analysis of over 130 polls. The problem that these poll-based forecasts encounter the discrepancy between the vote share won and the parliamentary seats won - is of course not unique to the 2010 contest. Here, however, is the first area where the modelers profiled in this volume offer a significant advance over the methods relying on the polls – in that they explicitly aim to forecast seats from votes. Indeed of the six papers at hand, there is only one that also attends to vote intention - that of Fisher and his colleagues. Their model, new to this election, employs sophisticated statistical techniques with a statespace model. This is used to first to model vote intentions, and from that vote share, and then from that seat share. Lewis-Beck, Nadeau, and Bélanger also offer a new “nowcasting” approach. Eschewing the modelling of vote intentions, they choose instead to model vote share, in a unique two-equation system. They then calculate a “swing ratio” for votes into seats, for their final “nowcast”. In contrast, Whiteley and his colleagues step directly to the modelling of seat shares itself, staying with a method they successfully employed in a 2005 UK general election forecast. Lebo and Norpoth follow suit, in that they stay with their 2005 model with the same seats dependent variable, driven in part by a cyclical component over a long time series. The other two papers differ, in that they are not based on aggregate time series. Lewis-Beck and Stegmaier derive “citizen forecasts” from national surveys that simply ask voters “which party do you think will win?” This method, previously applied to United States presidential elections, is applied for the first
time to the United Kingdom case. Finally, Rallings et al. try for something completely different, in the use of local byelection voting data as a barometer for the coming national voting pattern. This method has been applied to UK general election forecasting longer than any of the others offered, having been fielded in 1997, 2001, and 2005. How accurate were these models? First, with one exception, they all forecast a “hung parliament”, meaning that no one party had the necessary majority of 326 seats to govern alone. Further, they also reach a close approximation of the relative magnitude of Labour and Conservative seat shares. Only one paper, that of Rallings et al. predicted a greater gap than the 48 seats that finally separated the two parties. Which model was the most precise? Consider the forecast for the incumbent party, the usual dependent variable focus for these models. The ruling Labour party received 258 seats. Closest to this figure is the Whitely et al. model, predicting 273 Labour seats, an overestimation of only 15 seats. The opposition Conservative party received 306 seats. Closest here were the Fisher et al. and Rawlings et al. models, both predicting 299 Conservative seats, an underestimation of only 7 seats. Finally, the Liberal Democrats received 57 seats. The most accurate of the models in this case was that of Fisher et al., which predicted 46 Liberal Democratic seats, an underestimation of only 11 seats. Accuracy, as important as it is, is not the whole story. What makes a forecast a forecast is that it occurs before the event. In other words, there must be some lead time. In general, the longer the lead time, the more impressive an accurate forecast will be. For example, a precise forecast on election eve is not of much interest. In contrast, a precise forecast six months in advance gets our attention. In Table 1, the lead time for each forecasting model is reported. If the team made more than one forecast for the same variable, then we simply reported the one with the greatest lead time. This maximizes opportunity for forecasting at an impressive distance, while at the same time preventing the violation of the parsimony principle. [On these and other principles of forecasting evaluation, see Lewis-Beck (2005)]. In examining the last column of Table 1, we see the shortest lead-in of any of the predictions is that of the BES,
R. Gibson, M.S. Lewis-Beck / Electoral Studies 30 (2011) 247–249
which occurred the day before the election. The longest leadin, by contrast, is that of the Whiteley et al. model, at six months before the election. As well, the Whiteley et al. model can be judged the most accurate, in terms of forecasting the incumbent Labour share (ranking first, see Table 1). The fact that the most accurate model, with respect to incumbency forecasting, has the longest lead-in time is no fluke. The specific lead-in time of six months has previously been identified as the optimal lead-in time for forecasts in other democracies, such as the United States and France (LewisBeck and Rice, 1992, 123). In the words of those authors, a model “assessed at a certain distance, before the battle heats
249
up, predicts more accurately”. (Lewis-Beck and Rice, 1992, 123). Given the robustness of results that have emerged from the latest forecasting trials in the UK, we look forward to the application of these models in the next “battle” to come. References Lewis-Beck, M.S., 2005. Election forecasting: principles and practice. British Journal of Political and International Relations 7 (2), 145–164. Lewis-Beck, M.S., Rice, T.W., 1992. Forecasting Elections. Congressional Quarterly Press, Washington, D.C. Nadeau, Richard, Lewis-Beck, M.S., Bèlanger, E., 2009. Election forecasting in the United Kingdom: a two-step model. Journal of Elections, Public Opinion, and Parties 19 (3), 333–335.
Contents lists available at ScienceDirect
Electoral Studies journal homepage: www.elsevier.com/locate/electstud
Methodologies of election forecasting: Calling the 2010 UK “hung parliament” Rachel Gibson a, *, Michael S. Lewis-Beck b a b
Department of Political Science, University of Manchester, Manchester M13 9PL, UK Department of Political Science, University of Iowa, Iowa, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 17 September 2010 Accepted 22 September 2010
Election forecasting has experienced considerable growth as a subfield within political science. Research work on United Kingdom elections has been cutting edge here. The recent 2010 general election afforded the opportunity for a trial of different forecasting methodologies. These efforts are showcased in this volume, and include standard, and notso-standard, statistical models. Overall, these models perform well, foreseeing the unprecedented outcome of a “hung parliament”, as most pollsters and pundits did not. Moreover, they achieved this accuracy with forecasts well in advance of the election itself. Ó 2010 Published by Elsevier Ltd.
Keywords: Election forecasting UK elections Hung parliament Prediction models
Election forecasting in the United Kingdom has enjoyed an increasing profile since the general election of 2001, and by 2010 the enterprise vibrates from the competition among several different forecasting teams. Furthermore, their efforts have met with considerable success. As this volume reveals, as a group, they were near consensus in predicting that no party would achieve the seat majority necessary to govern after the election of 2010, with the majority forecasting a “hung parliament”. Such an outcome is no mean feat, for three reasons. First, this forecast is a rare event. A hung parliament, has occurred only twice in the last century. Second, the forecasts were made in March 2010, well before the May 6 election date, at a specially convened conference held at the University of Manchester. Thirdly, at the time of the forecast pundits and pollsters were generally highly skeptical about the prospect of a hung parliament. While the recent UK election saw the most widespread forecasting effort by academic teams, their success was built on efforts begun in earnest about ten years ago. Beginning with the 2001 national contest, UK election analysts started to seriously engage in forecasting, i.e., putting forward ex ante models predicting the votes/seats * Corresponding author. E-mail address: [email protected] (R. Gibson). 0261-3794/$ – see front matter Ó 2010 Published by Elsevier Ltd. doi:10.1016/j.electstud.2010.09.003
outcome. [(For a current review of this literature, see Nadeau et al., 2009)]. Further application of the practice followed in the general election of 2005, and now 2010. On each occasion, a forecasting symposium was convened with the results published in a major journal. Electoral Studies hosted the first collection of papers, and now again serves as host, with the publication of this special issue. As editors we are pleased to present such a rich collection of methodologically sophisticated and diverse approaches to the topic. In particular we would seek to draw reader attention to the fact that all papers were submitted to Electoral Studies before the May election date, and therefore appear here in their original form. Thus, they represent true forecasts, taking place prior to the actual event itself. It is within the context of this important methodological constraint, then, that we seek to summarize the contributions of these works to the wider literature. Traditionally, the leading forecasting method has been polling, where vote intentions are gauged in national surveys by a question such as “If the general election were held tomorrow, which party would you vote for?” In Table 1 we report results from the Poll of Polls (an aggregation of available polls) about two months before the 2010 election. As can be seen, as an estimator of the vote share, it is off two points for Labour, one point for the Conservatives, and
248
R. Gibson, M.S. Lewis-Beck / Electoral Studies 30 (2011) 247–249
Table 1 Evaluating rival forecasts of the 2010 UK contest.a Teamb
Vote share (%)c (L, C, LD)
Seat share (%)c (L, C, LD)
Actual POP BES FFJPW LN LNB LBS RTBL WSSC
(29, 36, 23) (31, 37, 19) (28, 36, 27) (32, 39, 20) (–, –, –) (36, 42, –) (–, –, –) (27, 36, 27) (–, –, –)
(258, 306, 57)
Rank (L, C)d (Seats prediction)
Leade (months) 2 .03
(275, 299, 46) (287, 285, –) (303, 293, –) (–, 358, –) (237, 299, 83) (273, 282, 41)
(2, 1.5) (4,4) (5, 3) (6,6) (3, 1.5) (1, 5)
1 2 3 1 1 6
a
If an entry is blank (–), that signifies no forecast was reported in that instance. Initials (first letter of each last name) for each forecasting team or organization: Poll of Polls; British Election Study; Fisher, Ford, Jennings, Pickup, Wlezien; Lebo and Norpoth; Lewis-Beck, Nadeau, and Bélanger; Lewis-Beck and Stegmaier; Rallings, Thrasher, Borisyuk, Long; Whiteley, Sanders, Stewart, Clark. c The percentage of the total vote share received by the party (L ¼ Labour, C ¼ Conservative), LD ¼ Liberal Democrat) at the national level. d The accuracy ranking of the seats forecast for the team, from most accurate (rank ¼ 1.0) to least accurate (rank ¼ 6) for Labour (L) and the Conservatives (C). e Lead time before the election in months. b
four points for the Liberal Democrats. Compare this to the estimates from another survey, that of the British Election Study (BES), the day before the election. It does somewhat better than the Poll of Polls (POP). The BES error is one point for Labour, zero points for the Conservatives, and four points for the Liberal Democrats. Either of these polling instruments –the POP or the BES – shows an overwhelming vote lead for the Conservatives. It is on the basis of such results that pre-election commentators easily believed that the Conservatives would secure a majority. Indeed in March, the editorial of a leading national paper ventured it was virtually impossible for the Conservatives to lose based on its analysis of over 130 polls. The problem that these poll-based forecasts encounter the discrepancy between the vote share won and the parliamentary seats won - is of course not unique to the 2010 contest. Here, however, is the first area where the modelers profiled in this volume offer a significant advance over the methods relying on the polls – in that they explicitly aim to forecast seats from votes. Indeed of the six papers at hand, there is only one that also attends to vote intention - that of Fisher and his colleagues. Their model, new to this election, employs sophisticated statistical techniques with a statespace model. This is used to first to model vote intentions, and from that vote share, and then from that seat share. Lewis-Beck, Nadeau, and Bélanger also offer a new “nowcasting” approach. Eschewing the modelling of vote intentions, they choose instead to model vote share, in a unique two-equation system. They then calculate a “swing ratio” for votes into seats, for their final “nowcast”. In contrast, Whiteley and his colleagues step directly to the modelling of seat shares itself, staying with a method they successfully employed in a 2005 UK general election forecast. Lebo and Norpoth follow suit, in that they stay with their 2005 model with the same seats dependent variable, driven in part by a cyclical component over a long time series. The other two papers differ, in that they are not based on aggregate time series. Lewis-Beck and Stegmaier derive “citizen forecasts” from national surveys that simply ask voters “which party do you think will win?” This method, previously applied to United States presidential elections, is applied for the first
time to the United Kingdom case. Finally, Rallings et al. try for something completely different, in the use of local byelection voting data as a barometer for the coming national voting pattern. This method has been applied to UK general election forecasting longer than any of the others offered, having been fielded in 1997, 2001, and 2005. How accurate were these models? First, with one exception, they all forecast a “hung parliament”, meaning that no one party had the necessary majority of 326 seats to govern alone. Further, they also reach a close approximation of the relative magnitude of Labour and Conservative seat shares. Only one paper, that of Rallings et al. predicted a greater gap than the 48 seats that finally separated the two parties. Which model was the most precise? Consider the forecast for the incumbent party, the usual dependent variable focus for these models. The ruling Labour party received 258 seats. Closest to this figure is the Whitely et al. model, predicting 273 Labour seats, an overestimation of only 15 seats. The opposition Conservative party received 306 seats. Closest here were the Fisher et al. and Rawlings et al. models, both predicting 299 Conservative seats, an underestimation of only 7 seats. Finally, the Liberal Democrats received 57 seats. The most accurate of the models in this case was that of Fisher et al., which predicted 46 Liberal Democratic seats, an underestimation of only 11 seats. Accuracy, as important as it is, is not the whole story. What makes a forecast a forecast is that it occurs before the event. In other words, there must be some lead time. In general, the longer the lead time, the more impressive an accurate forecast will be. For example, a precise forecast on election eve is not of much interest. In contrast, a precise forecast six months in advance gets our attention. In Table 1, the lead time for each forecasting model is reported. If the team made more than one forecast for the same variable, then we simply reported the one with the greatest lead time. This maximizes opportunity for forecasting at an impressive distance, while at the same time preventing the violation of the parsimony principle. [On these and other principles of forecasting evaluation, see Lewis-Beck (2005)]. In examining the last column of Table 1, we see the shortest lead-in of any of the predictions is that of the BES,
R. Gibson, M.S. Lewis-Beck / Electoral Studies 30 (2011) 247–249
which occurred the day before the election. The longest leadin, by contrast, is that of the Whiteley et al. model, at six months before the election. As well, the Whiteley et al. model can be judged the most accurate, in terms of forecasting the incumbent Labour share (ranking first, see Table 1). The fact that the most accurate model, with respect to incumbency forecasting, has the longest lead-in time is no fluke. The specific lead-in time of six months has previously been identified as the optimal lead-in time for forecasts in other democracies, such as the United States and France (LewisBeck and Rice, 1992, 123). In the words of those authors, a model “assessed at a certain distance, before the battle heats
249
up, predicts more accurately”. (Lewis-Beck and Rice, 1992, 123). Given the robustness of results that have emerged from the latest forecasting trials in the UK, we look forward to the application of these models in the next “battle” to come. References Lewis-Beck, M.S., 2005. Election forecasting: principles and practice. British Journal of Political and International Relations 7 (2), 145–164. Lewis-Beck, M.S., Rice, T.W., 1992. Forecasting Elections. Congressional Quarterly Press, Washington, D.C. Nadeau, Richard, Lewis-Beck, M.S., Bèlanger, E., 2009. Election forecasting in the United Kingdom: a two-step model. Journal of Elections, Public Opinion, and Parties 19 (3), 333–335.