Evaluating the effect of IMF lending to low-income countries

Journal of Development Economics Vol. 61 Ž2000. 495–526 www.elsevier.comrlocatereconbase

Evaluating the effect of IMF lending to low-income countries Louis Dicks-Mireaux ) , Mauro Mecagni, Susan Schadler International Monetary Fund, Policy DeÕelopment and ReÕiew Department, 700 19th Street, NW, Washington, DC 20431, USA

Abstract The purpose of this paper is twofold: to apply to a group of low-income borrowers from the IMF, the most commonly used technique for measuring the independent effects on economic developments of IMF support; and to develop a minimum set of diagnostic tests for determining whether necessary conditions for using the methodology exist. The modified control-group methodology is used to measure the effect of IMF support on three key variables — output growth, inflation, and the external debtrservice ratio. The sample comprises adjustment programs begun during 1986–1991 supported by the IMF’s Enhanced Structural Adjustment Facility ŽESAF.. The distinguishing feature of the modified controlgroup approach is the estimation of a policy counterfactual — policies that would have been followed in the absence of IMF support against which to compare actual policies and resulting outcomes. Using this approach for the ESAF, the sample reveals statistically significant beneficial effects of IMF support on output growth and the debtrservice ratio but no effects on inflation. Diagnostic tests of these results, rarely if ever reported in the literature, are shown to be critical in interpreting the validity of the results of assessments of adjustment lending. For this sample, at least, the diagnostic tests cast doubt on the reliability of estimates of the effects of IMF-supported programs using panel data in a modified control-group model. The most obvious and manageable modifications to the

)

Corresponding author. Tel.: q1-202-623-5699. E-mail address: [email protected] ŽL. Dicks-Mireaux..

0304-3878r00r$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S 0 3 0 4 - 3 8 7 8 Ž 0 0 . 0 0 0 6 6 - 3

496

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

model do not overcome its basic weaknesses. q 2000 Elsevier Science B.V. All rights reserved. JEL classification: E65; F33 Keywords: International Monetary Fund; Conditionality; Stabilization programs

1. Introduction Evaluations of macroeconomic programs supported by international financial institutions ŽIFIs. do not all address the same question. Some look at the design of such programs to see if they represent ‘‘best practices’’ for correcting countries’ macroeconomic problems. Others examine whether programs are effectively implemented. Another question that has attracted attention recently is whether IFI support has significant independent effects; i.e., does it bring about developments significantly different from those that would have occurred in the absence of support from the IFI in question? This is a difficult question to address because it requires the construction of a counterfactual indicating what policies and outcomes would have been in the absence of IFI support. The independent effects are then calculated as the difference between outcomes that would have occurred in the absence of IFI support and actual outcomes. Since the mid-1980s, several papers have considered how to construct a counterfactual for such exercises and how to address other problems in identifying independent effects of IFI-supported programs. In particular, differentiating the effects of the counterfactual policies from exogenous developments, initial conditions and IFI support. The methodology that has been most widely applied was developed by Goldstein and Montiel Ž1986. by adapting techniques from the literature on labor training evaluation. Essentially, this technique, referred to as the General Evaluation Estimator ŽGEE. or modified control group, involves using policy reaction functions estimated for countries that did not have support from a particular IFI to approximate the counterfactual for countries that did have IFI backing for their program. 1 The GEE is a potentially powerful technique, although, as Goldstein and Montiel point out, it entails many restrictive assumptions; e.g., it must be possible to characterize macroeconomic policy choices in a relatively simple reaction function based on quantifiable data, and it must be credible that the reaction functions estimated for countries that do not receive IFI support describe the counterfactual for countries that do receive such support. The purpose of this paper is both to apply the GEE methodology to data for low-income countries eligible for the IMF’s Enhanced Structural Adjustment 1

Applications of the GEE can be found in Greene Ž1989., Khan Ž1990., Faini et al. Ž1991., Corbo and Rojas Ž1992. and Conway Ž1994..

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

497

Facility ŽESAF. 2 and to examine in greater detail than previous studies have done the conditions that must be met for the GEE to give robust results. To do this, we calculate the effects of ESAF-supported programs during the first 6 years of the facility’s existence Ž1986–1991. on three macroeconomic variables that are typically the main objectives of the programs: output growth, inflation and, a key indicator of progress toward external viability, the external debtrservice ratio. We then perform several diagnostic tests to answer the question, ‘‘Are the restrictive assumptions underlying the standard GEE consistent with the sample?’’. Three main issues are addressed in these tests: does the single, relatively simple macroeconomic model used in applications of the GEE capture the interaction between macroeconomic policies and outcomes for a large number of countries over time; can a robust policy reaction function be estimated for periods when, and in countries where, IMF support is not in place; is it possible to address sample selection bias that is likely to characterize applications of the GEE to date? The results, like others for different data sets, point to significant positive effects of IMF-supported programs on growth and the debtrservice ratio. The diagnostic tests, however, cast doubt on the appropriateness of the restrictive assumptions underlying the GEE and accordingly about the reliability of the results. This finding raises questions about whether there are inherent problems in estimating GEE models with panel data. At a minimum, it strongly indicates that future applications of the GEE on other data sets need to incorporate standard diagnostic tests to ascertain whether the GEE methodology is valid for the sample under study.

2. Specification of the model The GEE is geared toward answering the question, ‘‘Did the involvement of the IMF through a lending arrangement significantly improve the outcomes for important macroeconomic variables relative to what they would have been in the absence of ESAF support?’’. To answer this question, the macroeconomic outcomes or target variables in countries are described as a function of: Ži. policies that would have been observed in the absence of an IMF-supported program; Žii.

2 The ESAF is the window through which the IMF can make highly concessional long-term loans available to low-income countries. This facility succeeded the Structural Adjustment Facility ŽSAF. which had similar characteristics but with somewhat less rigorous conditions on economic policies than the ESAF. During the period covered in this study, 74 countries were eligible for loans through the ESAF.

498

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

exogenous external factors; Žiii. the existence or otherwise of an IMF-supported program; and Živ. unobservable random shocks: yi j s b o j q b jk x i k q a jh wi h q b jIMFd i q e i j ,

Ž 1.

where yi j is the jth target variable in country i, x i k is a k-element vector of policy variables that would be observed in country i in the absence of IMF support, wi h is an h-element vector of exogenous external variables for each country i, d i is a dummy variable equal to 1 if an IMF program is in effect and zero otherwise, and e i j is a zero mean, fixed variance, serially uncorrelated error. For the jth target variable, b jk and a jh are kxl and hxl vectors, respectively, of fixed parameters. After postulating a rule for policies in the absence of an IMF-supported program Ž x i k ., the model is estimated using pooled cross-section and time-series data drawn from countries and periods in which IMF support was in place and those in which IMF support was absent. The aim is to get consistent estimates for b jIMF, the ‘‘independent effect’’ of IMF support on each target variable. If these are statistically significant at a reasonable confidence level, IMF support is found to have significant effects. Policies adopted in the absence of an IMF-supported program Ž x i k . are directly observable only for nonprogram periods, and thus, a key element of the GEE is the construction of a counterfactual for policies during programs. In Goldstein and Montiel Ž1986. and subsequent empirical applications, this counterfactual is based upon a policy reaction function that links changes in policy instruments to the deviation of the observed lagged value for each target from its desired value, yidj .3 Specifically, the policy reaction function is described by: D x i k sg k j yidj y yi jŽy1. q hi k ,

Ž 2.

where yi j is a j-element vector of target variables, hi k is a zero mean, fixed variance, serially uncorrelated error term assumed to be uncorrelated with e i j , and D is the first difference operator.4 The kxj parameter matrix g k j indicates the extent to which policy instruments are adjusted in response to disequilibria in the target variables.

3

In the empirical exercise, we also experimented with other reaction functions. One, derived explicitly from an optimizing exercise, produced a final reduced form equation in which past policies do not enter Žsee Eq. 3.. A second included lagged values of exogenous influences wi hŽy 1. , which would be important, e.g., if lagged changes in export or import prices affected the current fiscal position. Both of these performed less well than Eq. 2 in estimation and are not, therefore, reported here. See Dicks-Mireaux et al. Ž1995. for a fuller discussion of selecting a reaction function. 4 The lack of mutual correlation between hi k and e i j implies that changes in policy instruments Ž D x i k . are not influenced by contemporaneous exogenous shocks to the target variables.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

yidj

499

Substituting Eq. 2 into Eq. 1 and subsuming yidj in the constant Ži.e., assuming is invariant across countries and over time., we obtain: D yi j s b jo y Ž b jkg k j q 1 . yi jŽy1. q b jk x i kŽy1. q a jh wi h q b jIMFd i q Ž e i j q b jk hi k . .

Ž 3.

Eq. 3 constitutes the basic GEE reduced form model as applied in most earlier studies.5 Its operational usefulness depends on the validity of several restrictive assumptions, discussed in the remainder of this section.6 First, an important question for the empirical application is whether individual country behavior can be sensibly aggregated in a uniform model that is stable across countries and over time. Specifically, differing institutional characteristics Že.g., the degree of policy discipline inherent in specific exchange rate arrangements or the relationship with a major donor., changing political conditions, or varying severities of economic distress are likely to result in countries formulating policies with respect to different or changing objective functions, or subject to different or changing constraints. Another question is whether it is appropriate to assume that the policy reaction function of a program country, had it not received IMF support, is identical to that of a nonprogram country that did not seek IMF support. For example, the counterfactual for a country receiving IMF support may involve the imposition of trade or exchange restrictions, while countries that do not seek IMF support may constrain themselves to ‘‘IMF-type’’ policies; i.e., avoiding the use of trade or exchange restrictions. Second, the constant additive term b jIMFd i is meant to capture four separate channels through which IMF support could affect macroeconomic targets: Ži. changes in the state of confidence in the economy; Žii. changes in the desired value of targets, e.g., through structural reforms aimed at raising the rate of potential growth; Žiii. policies different from what they would have been in the absence of a program; and Živ. changes in the effectiveness of any given stance of policies. While an additive term in a reduced form equation like Eq. 3 can capture the first two of these channels, a more complex specification requiring the explicit estimation of the policy reaction functions is needed to capture the third and fourth channels of influence.7 The intuition is straightforward. Both the third and fourth channels involve effects that are proportional to the size of the difference between actual and counterfactual policies: strictly, therefore, they require the different sizes of effects of actual and counterfactual policies to be measured directly. In sum, a simple invariant additive term in the GEE may not do full justice to the 5 A potential source of bias in Eq. 3 because of nonzero correlation between the error terms and explanatory variables is ruled out by assuming a stochastic structure whereby shocks are transitory. 6 Some of these restrictive assumptions characterize other, especially cross-section, estimators. 7 See Dicks-Mireaux et al. Ž1995. and Goldstein and Montiel Ž1986. for a discussion of the specification needed to accurately capture these channels and the operational constraints in doing so.

500

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

range of potential program effects. In practice, the possibility of a more informative decomposition of program effects that allows variation across countries requires a large sample and the ability to identify empirically a stable policy reaction function as examined in Section 4. Third, Eq. 3 contains only the most simple dynamics. The principal dynamic influence comes from the lagged target variable term in the policy reaction equation — policies are assumed to adjust to deviations of actual from desired values of the target variables in the preceding period. Lags in the effects of policies on target variables or inertial effects in the targets themselves are typically not considered.8 Indeed, in Eqs. 1–3, the stochastic terms e i j and hi k are assumed to be serially uncorrelated: all shocks are assumed to be transitory and to cause one-period changes in target variables that are fully reversed in the following period. This restrictive assumption requires that each target variable be stationary, but it also rules out a wide range of stationary stochastic processes for which the impact of temporary shocks persists over time.9 If, in fact, significant inertia exists; imposing full one-period reversion to mean will understate the positive effects of an IMF-supported program after a negative shock. Empirically, this issue could not be explored for ESAF countries because of the short sample time span. While a more general dynamic form can be easily specified, in practice, the data set for ESAF countries is not long enough to apply it meaningfully.10

3. Estimation procedures 3.1. The sample The model, as specified in Eq. 3, was estimated with data from 1986 to 1991 for 61 of the 74 countries ŽTable 1. eligible to use ESAF resources as of 1992 Žexclusions are noted below.. Nineteen of these countries had ESAF arrangements at some time during the sample period.11 The sample was restricted to ESAF-eligible countries, rather than a larger set of developing countries: including only 8 Inertial effects may arise for a variety of reasons, such as backward-looking indexation, slowly adjusting expectations, staggered contracts, and transaction costs. 9 In the absence of stationarity, the concept of reversion to mean is not well-defined because the mean of the stochastic process is not time-invariant, and the series will tend to move continuously away from a given level as a result of past and current shocks. See Harvey Ž1981. and Priestley Ž1981.. 10 See Dicks-Mireaux et al. Ž1995. for a fuller discussion of the dynamic properties of the model and the full dynamic specification. 11 For these countries, program years are those when either a SAF or ESAF arrangement was in place. SAF arrangements typically had less stringent conditionality than ESAF arrangements. For most countries that had ESAF arrangements, however, prior SAF arrangements were used to establish commitment to adjustment and were close in nature to ESAF-supported programs.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

501

low-income countries reduced the scope for parameter instability owing to differences in structural and institutional conditions between low-income countries and other developing countries. Even the sample restricted to ESAF-eligible countries is quite diverse. The program countries are dominated by heavily indebted African countries with a narrow range of exports and relatively simple market structures. The nonprogram countries are more diverse, including, in about equal proportions, indebted African countries and small Caribbean or Pacific island countries with close institutional and financial links to particular industrial countries. There are also a few South and Southeast Asian countries. For estimation, a number of data points are dropped from the full sample of ESAF-eligible countries.12 Some observations are excluded because of inadequate data owing to civil strife wAfghanistan Ž1990–1991. and Angola, Liberia, and Nicaragua Žall years.x; major discontinuities which could not be corrected wDjibouti, Sao Tome and Principe, and Zaire Žall years.x; extreme isolation wAlbania, Cambodia and Mongolia Žall years.x; or political discontinuities wDemocratic Republic of Yemen Ž1990–1991. and Yemen Arab Republic Ž1989–1991.x. Years in which countries had a SAF arrangement Žexcept when followed immediately by an ESAF arrangement., standby, or extended arrangement are omitted because they were considered invalid as nonprogram counterfactuals; this excludes Cote d’Ivoire, Philippines and, together with lack of data in 1991, Somalia, entirely from the sample.13 Even for the limited sample covered, the quality of data is poor. In many instances, the accuracy of the measures of macroeconomic variables, such as GDP, is likely to very weak. Also, ad hoc correction for breaks in the series was frequently needed. These fundamental weaknesses qualify the inferences or judgements that can be drawn from the data. 3.2. Definitions: targets, policies, exogenous influences and period of IMF support This study considers three target variables Ž yi j . that reflect the objectives of the ESAF: Ži. the growth rate of real GDP; Žii. consumer price inflation; and Žiii. one 12 As a result, the structure of the panel data is incomplete. Techniques for analyzing panel data in which missing observations are random or regular Žor ‘‘rotating’’. are not applicable because the sample data exclusions do not conform to either of these patterns. Instead, for estimation, the panel data are handled as a pooling of annual observations, with the number of program and nonprogram years varying from country to country. 13 In principle, years when countries had a SAF, standby, or extended arrangement could have been included as neither ESAF years nor nonprogram years, but rather as additional categories. In this case, the selection of the facility through which a country borrowed Žwith its attendant conditionality. would have had to be modeled as the outcome of strategic negotiations; see, e.g., Knight and Santaella Ž1994.. The correction for sample selection bias Ždiscussed below. would also have had to be differentiated for each facility.

502

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Table 1 Sample of countries and programrnonprogram years a Nonprogram countries

Nonprogram years

SAFrESAF program countries

Program years

Afghanistan

Bangladesh

Benin Bhutan

1986r1987– 1988r1989 1986–1988 1986–1991

Burkina Faso

1986–1991

Gambia, The

Cape Verde Central African Republic Chad China Comoros

1986–1991 1991

Ghana Guinea

1986r1987– 1991r1992 1987–1991 1986–1989, 1991 1986r1987– 1990r1991 1987–1991 1987–1991

1986, 1991 1988–1991 1986–1990

Guyana Kenya Lesotho

1990–1991 1986–1990

Dominica Dominican Republic Egypt Equatorial Guinea Ethiopia Grenada Guinea Bissau Haiti

Honduras India Kiribati Lao, PDR Maldives Mali Myanmar Nepal Nigeria Pakistan Rwanda St. Kitts and Nevis St. Lucia

Bolivia Burundi

Nonprogram years

1990 1991r1992

1986 1986–1989 1986–1987 1986r1987– 1987r1988

Madagascar Malawi

1990–1991 1988–1991 1988r1989– 1991r1992 1987–1991 1988–1991

1989r1990 1987–1988, 1990–1991 1986–1991 1986–1991

Mauritania Mozambique

1986–1990 1987–1991

1991 1986

Niger Senegal

1986, 1988, 1991 1985r1986, 1987r1988– 1988r1989, 1990r1991 1986–1989 1986r1987– 1989r1990 1986–1991 1986–1988 1986–1991 1987, 1991 1986–1991 1990r1991 1986, 1988, 1990 1986r1987– 1987r1988 1986–1990 1986–1991 1986r1987– 1991r1992

Sri Lanka

1987–1991 1986r1987– 1991r1992 1988–1991

1986–1987

1986–1987

Tanzania

1987r1988– 1991r1992

1986r1987

Togo Uganda

1988–1990 1987r1988– 1991r1992

1991 1986r1987

Number of countries: 19 Number of annual program observations: 88 Number of annual nonprogram observationsb : 20

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

503

Table 1 Ž continued . Nonprogram countries

Nonprogram years

SAFrESAF program countries

Program years

Nonprogram years

St. Vincent Sierra Leone

1986–1991 1987r1988– 1990r1991 Solomon Islands 1986–1991 Sudan 1986–1991 Tonga 1986–1991 Vanuatu 1986–1991 Vietnam 1986–1991 Western Samoa 1986–1991 Yemen, AR 1986–1988 Yemen, PDR 1986–1989 Zambia 1988–1991 Zimbabwe 1986–1991 Number of countries: 42 Number of annual nonprogram observations:b 183 a Fiscal years tr t q1 were considered to correspond to calendar year t if the fiscal year started on or before July 1st. b Includes observations in which a SAF, ESAF, SBA or EFF arrangement was not in place.

of the key measures by which the IMF measures progress toward external viability —the ratio of external debtrservice to exports. The last target is preferred to other indicators of the external position, such as the current account, the overall balance of payments, or the level of international reserves for several reasons.14 Most countries entered ESAF arrangements with large debt overhangs, and reducing the debtrservice ratio to manageable levels was the primary external objective of the program. For other external variables, however, even the direction of targeted and actual changes varied depending on initial conditions and prospects for attracting concessional inflows. Moreover, for nonprogram periods, developments in these other variables sometimes reflected the imposition of formal or informal trade and exchange restrictions rather than changes in the viability of the external position. Increases in reserves, probably the next most general indication of external sector developments, were at times associated with the accumulation of arrears. Also, a reserves target did not exist for many countries ŽCFA and ECCB members. that did not directly own international reserves.

14

The debtrservice ratio is not, however, an infallible indicator of progress toward external viability. For example, changes in this ratio may reflect only a temporary change in export prices.

504

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Three policy instruments Ž x i k . are considered: Ži. the deficit of the central government in relation to GDP; Žii. the growth of net domestic assets of the banking system ŽNDA.;15 and Žiii. the change in the nominal effective exchange rate ŽNEER.. Ideally, the vector of policy instruments would also include indicators of structural reforms and conditions and institutional arrangements Žsuch as flexible or fixed exchange rate regimes.. However, these variables cannot easily be objectively quantified or reduced to an index. The external environment indicators Ž wi h . comprise changes in the terms of trade and the growth of export markets.16 Whenever possible, the data were taken from documents for the Executive Board of the IMF. Several questions arise in defining the variable denoting the presence of an IMF-supported program Ž d i .. First, the distinction between program and nonprogram years is blurred when IMF support starts in the middle of the year. For this study, any year in which a SAFrESAF-supported program was in effect for 6 months or more was considered a program year. Even this rule, however, does not clearly delineate the period during which IMF support influenced policies and outcomes. Usually, substantive negotiations and policy actions occurred in anticipation of IMF support in the year preceding the formal program. In some cases, the IMF influenced policies even after an ESAF arrangement. For example, The Gambia’s SAFrESAF arrangements stretched from FY1986 ŽJuly–June. to FY1990, but even in FY1991, the IMF monitored macroeconomic developments vis a` vis quantified targets agreed with the authorities. A second question is whether the influence of the IMF should be measured only through a 0–1 dummy Žone when the arrangement is in effect, zero when it is not. or also through the proportion of purchases made relative to total access under the arrangement as a measure of the completeness of implementation of agreed policies. Purchases are likely to be an imperfect indicator of implementation, however, because purchases in SAFrESAF-supported programs are scheduled at 6-month frequencies and waivers may be granted to permit purchases even when implementation slips. This study uses a binary, one-zero index of IMF involvement for the dummy variable Ž d i .: a period when a country has an agreed program with the IMF but fails to implement it or meet the targets is treated as a program year in which the effect of the IMF is low. Conway Ž1994. used both these approaches as well as the proportion of the year covered by the program as proxies

15

Although the controllable monetary policy instrument of the authorities is net domestic assets of the central bank, data on a comparable basis across countries were not available. 16 In principle, only one of market price and volume indicators should be used. In practice, the suitability of each varies among the countries: the terms of trade are relevant for small primary producers, but world market growth is relevant for large primary producers and countries with differentiated exports.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

505

for participation, but obtained similar estimates of the effects of IMF-supported programs for all three proxies. A third point to recognize is that estimates of IMF-supported programs will likely include the effects of parallel World Bank programs. As distinct from the IMF’s standby and extended facilities, SAFrESAF arrangements require explicit collaboration among country authorities, the World Bank and the IMF.

4. Results 4.1. The generalized eÕaluation estimator Estimation results were obtained for the basic GEE Eq. 3. Several modifications of the basic specification were also estimated but produced worse results: apart from those involving richer specifications of the reaction function,17 two other efforts to reduce the restrictiveness of the basic GEE model were tried: Ži. country and time dummies were introduced to help account for some of the cross-country differences in economic structures and time-specific exogenous developments not captured in the terms of trade and market growth variables; and Žii. a correction procedure developed by Heckman Ž1979. Žexplained in detail in Appendix A. was tried to correct for possible sample selection bias. Regression estimates based on pooled time-series, cross-country data are prone to heteroschedasticity, and even after the inclusion of country-specific and time dummies in the estimated equations, regression residuals display this characteristic.18 Without information on the form of heteroschedasticity, however, the primary form of one weighting scheme over another is unknown. Therefore, the reported t-statistics were computed from heteroschedastic-consistent estimates of the standard errors based on White’s variance–covariance estimator that provides consistent estimates even when the exact form of heteroschedasticity is not known.19 Table 2 presents the preferred estimates from these exercises. These estimates exclude the time dummies mentioned above because their coefficients were generally not significant and had little effect on Žor worsened. the fit of the equations. The impact of IMF-supported programs is found to be sizeable and statistically significant with respect to growth Žat the 5% level. and the external debtrservice ratio Žat the 10% level., but not inflation. On average, growth rates are found to be more than 1 percentage point per annum higher during program 17

See footnote 3 and Dicks-Mireaux et al. Ž1995.. Statistically significant values of the Breusch–Pagan test for heteroschedasticity ŽBreusch and Pagan, 1979. were observed in the estimated GEE equation for inflation Žat the 5% significance level. and the external debtrservice ratio Žat the 1% level.. 19 See White Ž1980.. 18

506

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Table 2 Estimates of the GEE a Target variable

Real GDP growth rate

Constant Lagged real GDP growth rate Lagged inflation rate Lagged external debtr service ratio Lagged fiscal balancer GDP Lagged net domestic asset growth Lagged percentage change in NEER Current percentage change in terms of trade Current export market growth IMF program dummy R2 SEE Number of observations Breusch– Pagan test for heteroschedasticity Jarque–Bera test for normality of residuals

y6.619 Žy1.71. y1.107UU Žy17.96.

0.0005 Ž0.13.

Inflation rate 10.248 Ž1.08. y0.764U Žy2.18.

y0.687UU Žy4.76.

External debtr service ratio 22.258UU Ž3.98. 0.022 Ž0.09.

0.027 Ž1.09. y0.376UU Žy3.09.

0.013 Ž0.74.

0.106Ž1.14.

y0.042 Žy1.37.

y0.467 Žy1.31.

0.097 Ž0.76.

0.004 Ž1.82.

y0.088 Žy1.47.

y0.020 Žy1.78.

y0.009 Žy1.03.

0.436U Ž2.12.

0.058 Ž1.05.

0.002 Ž0.21.

y0.104 Žy0.78.

y0.104UU Ž3.44.

0.090 Ž1.78.

0.293 Ž1.26.

y0.059 Žy0.30.

1.374U Ž2.18.

y3.330 Žy0.35.

y5.552 Žy1.75.

0.537 3.259 291

0.398 29.612 291

0.069 15.734 291

1.35

10.83U

26.57UU

28,231.00UU

23.71UU

7086.90UU

a The regression estimates were obtained using an ordinary least squares procedure, with countryspecific dummies included in the specification. Standard errors and t-statistics of coefficients are computed using White’s heteroschedasticity-consistent variance–covariance estimator. The figures in parentheses are t-statistics; R 2 is the adjusted coefficient of determination; SEE is the standard error of the regression. A single asterisk indicates statistical significance at the 5% level; two asterisks indicate statistical significance at the 1% level.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

507

periods than they would have been in the absence of an IMF-supported program. Debtrservice ratios are found on average to be more than 5 percentage points lower during program periods than they would have been in the absence of an IMF-supported program.20 On the other right-side variables, very few coefficients are significant at the 5% or 1% confidence level. The exceptions are own lagged levels of the target variables, which enter with coefficients significantly different from zero at the 1% level; the lagged level of the real GDP growth rate and the change in the nominal effective exchange rate Žwhich has an implausible positive sign. in the inflation equation; and the terms of trade in the debtrservice ratio equation.21 The lagged fiscal balance and lagged net domestic asset growth are not found to have a significant impact on the outcome of any target variable.22 4.2. EÕaluating the underlying assumptions of the GEE We now turn to examining the robustness of the results of the GEE estimates. Specifically, we go beyond the standard presentation of GEE results, which focuses on the estimates of b jIMF only, to examine whether the regression estimates are stable and unbiased — evidence that would support the underlying assumptions of the model. The tests reported, which should be standard in all presentations of GEE applications, cut to the core issue of this paper — are estimates of the effects of ESAF programs using panel data in the GEE framework meaningful? 4.2.1. The policy reaction function The first window on this issue is the policy reaction function. Identifying the coefficients and measuring the standard errors in the policy reaction function from

20 This effect on the debtrservice ratio is not attributable to the link between Paris and London Club agreements and IMF support, because the debtrservice ratio is measured before debt relief. The effect of stock of debt reduction operations associated with IMF-supported programs would be reflected in measures of the debtrservice ratio before debt relief in years subsequent to the debt reduction. However, in the sample of program countries considered in this study, no stock of debt operations linked to IMF-supported programs was undertaken. 21 It is possible that this link reflects the presence of export prices in both the terms of trade and debtrservice ratio. However, in light of the many other variables affecting these ratios, this seems unlikely. 22 The residuals of most of the estimated equations fail to pass the Jarque–Bera test for normality. The t-tests should, therefore, be interpreted cautiously as they may be sensitive to nonnormality in a fashion that is determined by the numerical value of the regressors. This cautionary note applies also to other regression diagnostic tests Žsee Jarque and Bera, 1987..

508

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

the reduced form GEE are not possible nor is it necessary for obtaining estimates of the b jIMF terms.23 Nevertheless, obtaining estimates by directly estimating the counterfactual policy reaction functions with data from the Žobservable. nonprogram periods only, provides a means of evaluating the validity of the reaction functions for the sample over which they can be estimated. Direct estimates of the policy reaction function also allow one to test and correct for sample selection bias arising when unobservable factors that influence countries’ decisions to receive IMF support also influence their policy reactions.24 Regression estimates of the policy reaction functions ŽEq. 2. consistent with the basic GEE ŽEq. 3. are reported in Table 3.25 These estimates are poor in several respects. The R 2 statistics are negative or very close to zero; t-statistics for individual coefficients are insignificant Žexcept on the debtrservice ratio in the equation for the nominal effective exchange rate.; F-tests cannot reject the null joint hypothesis of zero slopes; and the regression residuals exhibit statistically significant heteroschedasticity and nonnormality. In short, these estimates provide a weak basis for deriving estimates of the unobservable counterfactual policies for program periods. The inverse Mills ratio ŽIMR. was included as a regressor in the policy reaction functions to correct for possible sample selection bias, following a two-step procedure proposed by Heckman Ž1979..26 Appendix A describes in detail and reports estimates of this procedure. However, the estimated coefficients of the IMR are statistically insignificant, suggesting that sample selection bias is not present. Sample selection bias may arise in estimates of the reduced form GEE ŽEq. 3. even when it is not present in the reaction function. This would occur if the choice of having a program depends on expectations of better performance in the target variable. To test for this, the IMR was included in the estimated reduced form

23

The g k j parameters cannot be identified from the single equation estimates because the number of structural parameters exceeds the number of reduced form coefficients. However, by pooling the parameter estimates from the three equations, the g k j parameters can be identified if policy instruments and macroeconomic targets are equal in number. When the number of instruments is less than that of targets, the g k j parameters cannot be identified, and when they exceed the number of targets, multiple solutions exist. It is not possible to measure the standard errors of the estimates. 24 Goldstein and Montiel Ž1986., Greene Ž1989. and Khan Ž1990. do not attempt to correct for this potential source of bias, but several studies of World Bank adjustment lending do Žsee World Bank, 1990; Faini et al., 1991; Corbo and Rojas, 1992.. 25 Country and time dummies were introduced in the policy reaction function, but most had coefficients insignificantly different from zero and had little effect on, or worsened, the overall fit of the equation Žreduced the R 2 . and therefore were not retained. 26 The IMR is a monotone decreasing function Žranging from 0 to `. of the probability that an observation is selected into the sample of nonprogram countries.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

509

Table 3 Estimates of the policy reaction functiona Policy variable Constant Lagged real GDP growth rate Lagged inflation rate Lagged external debtr service ratio IMRb R2 SEE Number of observations F-statistics Žzero slopes. Breusch–Pagan test for heteroschedasticity Jarque–Bera test for normality ofresiduals

D Fiscal balancer GDP

D Net domestic asset growth

D Percentage change in NEER

1.643 Ž0.67. 0.024 Ž0.19. 0.006 Ž1.02. y0.0007 Žy0.04.

y2.209 Žy0.11. y1.090 Žy1.11. y0.081 Žy0.12. y0.097 Žy0.40.

y2.607 Žy0.35. y0.204 Žy0.69. 0.017 Ž0.29. y0.152U Žy2.22.

y3.911 Žy0.65. y0.013 7.064 203 0.36 0.70

16.271 Ž0.24. y0.016 106.339 203 0.22 62.27UU

13.070 Ž0.76. 0.019 23.239 203 1.96 21.88UU

4875.06UU

17,986.30UU

495.36UU

a

The regression estimates were obtained from the sample of nonprogram observations using an ordinary least squares procedure. Standard errors and t-statistics of coefficients are computed using White’s heteroschedasticity-consistent variance–covariance estimator. The figures in parentheses are t-statistics; R 2 is the adjusted coefficient of determination; SEE is the standard error of the regression. A single asterisk indicates statistical significance of the 5% level; two asterisks indicate statistical significance at the 1% level. b Values of the IMR were computed using the estimated probit equation reported in Table 8.

GEE. The estimated coefficients of the IMR were statistically insignificant, again suggesting the absence of sample selection bias Žsee also Appendix A..27 4.2.2. Significance and stability of the estimates A second window on the robustness of the GEE estimates is to test for the significance and stability of the parameter estimates. The question at issue is whether the GEE results reported here or in other studies are robust to changes in the size and composition of the sample. There are many ways to evaluate the regression estimates, and the approach taken here is not intended to be exhaustive. As measured by the R 2 statistics, the overall fit of the estimated equations is modest Žalmost nil for the external 27 The IMR was highly collinear with the indicator of the presence of an IMF-supported program Ž d i . in Eq. 3, and the statistically significant impact of a program on growth and the external debtrservice ratio was statistically insignificant. The coefficient estimates of the other regressors in the GEE were not significantly altered by the inclusion of the IMR. An alternative procedure using the predicted probability of undertaking a program as an instrument for d i in Eq. 3 did not significantly alter the coefficient of the other regressors. These results are not reported here, but are available from the authors.

510

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

debtrservice ratio.. This, together with the evidence of heteroschedastic residuals Ževen after the inclusion of country and time dummies. and the large number of coefficients insignificantly different from zero or with counterintuitive signs, suggests the possibility of biased coefficient estimates due to misspecification. For instance, one potentially important omission is structural conditions and reforms that figured prominently in ESAF-supported programs. A second concern is the possibility of heterogeneity bias ŽHsiao, 1986..28 A third concern is the risk of invalid inferential statements, even in large samples ŽJarque and Bera, 1987. given that the regression residuals fail to pass the Jarque–Bera test for normality. The reliability of the parameter estimates is also revealed by the stability properties of the model. Many GEE applications investigate changes in the effectiveness of IFI support as measured by the b jIMF coefficients between two sub-periods of the sample used; Greene Ž1989. and Corbo and Rojas Ž1992. also report changes in estimates of the other coefficients of the reduced form GEE. Yet, in each study, the evidence of changes in point estimates of the b jIMF and other coefficients Žat times statistically significant. is not seen as an indication of instability in the underlying model. The best approach to exploring instability would be to estimate a varying parameters model in which estimated parameters are allowed to vary across countries 29 and over time and therefore, also between program and nonprogram periods.30 This general approach would nest tests of stability and uniformity restrictions within a less restrictive framework, e.g., taking into account heteroschedastic errors in estimation and testing of coefficient variation; this property is particularly appealing in pooled cross-section, time-series data. However, such an approach requires a data set considerably larger than that available in this study: to explore inter-country instability, the number of data points for each country must exceed the number of regressors; and to explore instability between program and nonprogram periods, the number of data points for each country in each regime Žprogram and nonprogram. must exceed the number of regressors.31 For our sample Žwhich is small and has an unequal number of annual observations for each country., the stability of the individual parameters b jk and

28

The key problem in heterogeneity bias is that the imposition of identical parameters leads to an averaging of coefficients that differ greatly across countries Žor time. and therefore produces nonsensical results. In effect, the assumption of a ‘‘representative country’’ that can be described by an average is not valid. 29 See Swamy Ž1970.. 30 See Hsiao Ž1986.. 31 Extending the sample, however, would have required dropping several countries owing to the lack of consistent data for earlier periods. Also, the number of additional useable observations Ži.e., years in which countries did not have a standby or extended arrangement in place. would have been a rather small proportion of the total.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

511

b jIMF has to be examined by recursive regression methods.32 Two types of recursive exercises were done: in the first, the reduced form GEE was estimated recursively, starting with a baseline sample Žcomprising all nonprogram observations plus one program observation. and then adding in program observations one at a time Žreported as ‘‘program recursions’’.; 33 in the second, the recursive procedure was carried out by starting with the full sample and then subtracting nonprogram observations one-by-one Žreported as ‘‘nonprogram recursions’’.. The share of estimates in the recursions that differ significantly from the pooled sample estimates or from zero indicates the sensitivity of the estimated parameters to variations in the sample.34 For the estimates of the effectiveness of IMF support — the b jIMF coefficients — four of the six recursive exercises Žtwo for each of the three equations; Figs. 1–6. produce numerous estimates of b jIMF that are significantly different from the whole sample estimates. In all the recursive exercises, estimates were frequently not significantly different from zero. Thus, the finding of significant effects of IMF support on growth and the debtrservice ratio cannot be considered robust to variations in the sample. For the coefficients on the lagged policy variables, the recursive exercises suggest that the full sample estimates of policy effects are relatively more robust ŽTables 4 and 5.. The general pattern is for lagged policy variables to have little effect on target variables. The stability of the policy reaction functions parameters Ž g k j . was tested in a simplified recursive exercise over the sample of nonprogram observations. The recursions began with an initial sample of 25 observations, and observations were added one by one.35 The results, reported in Tables 6 and 7, indicate that the coefficient estimates from the full sample of nonprogram observations are generally robust across recursions, and Žexcept for the lagged external debtrservice ratio in the equation, the nominal effective exchange rate policy. are insignifi-

32 Stability is assessed in terms of the point estimates and standard errors of individual parameters. An alternative approach would be to conduct Chow or Wald tests for the joint stability of the coefficients of interest on the recursive estimates of the equations. However, this was not possible because the fitted equation does not meet the requirement of both tests that the set of regressors remains constant over recursive estimates: the country dummies, which entered with statistically significant coefficients, change across the recursions. 33 The additional program country observation was required to avoid singularity in the presence of the IMF dummy variable Ž d i .. 34 These recursive procedures consider only a small subset Žarbitrarily chosen. of all subsamples that could be drawn from the data set. Thus, the range of coefficient estimates reported in Tables 4–7 does not necessarily encompass global maximum and minimum values for all permutations of the sample. 35 The size of the initial block of observations was chosen so as to start with a reasonable number of degrees of freedom Ž20..

512

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Fig. 1. Recursive point estimates and confidence intervals for IMF program dummy variable Žreal GDP growth rates..

cantly different from zero. Sign reversals are not widespread and most of the estimates are neither significantly different from zero nor from the full sample estimate.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Fig. 2. Recursive point estimates and confidence intervals for IMF program dummy variable Žreal GDP growth rates.. 513

514

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Fig. 3. Recursive point estimates and confidence intervals for IMF program dummy variable Žrate of inflation..

4.2.3. Dynamics and initial conditions The empirical results call into question the adequacy of the simple dynamic specification of the GEE commonly used in the literature. Specifically, the product

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Fig. 4. Recursive point estimates and confidence intervals for IMF program dummy variable Žrate of inflation.. 515

516

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Fig. 5. Recursive point estimates and confidence intervals for IMF program dummy variable Žexternal debtrservice ratio..

of independent estimates of g k j from the policy reaction function for nonprogram periods and estimates of b jk from the GEE reduced form for each equation is close to zero. If this is true, the coefficient on the lagged target variables in the

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Fig. 6. Recursive point estimates and confidence intervals for IMF program dummy variable Žexternal debtrservice ratio.. 517

518

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Table 4 Share of statistically significant t-statistics at the 5% level in recursive estimates of the GEE a Target variable

Lagged fiscal balancer GDP, b 1

Lagged NDA growth, b2

Lagged NEER percent Change, b 3

H o : b1 s 0

H o : b2 s 0

H o : b3 s 0

D Real GDP growth rate Share in total 0.0 program year recursionsb Share in total 45.0 nonprogram year recursions c D Inflation rate Share in total program year recursionsb Share in total nonprogram year recursions c

H o : b1 s full sample value

H o : b2 s full sample value

H o : b3 s full sample value

0.0

0.0

0.0

0.0

0.0

0.0

47.0

0.0

0.0

4.5

0.0

0.0

0.0

0.0

3.4

0.0

0.0

0.0

10.9

13.9

85.6

0.0

36.8

0.0

0.0

0.0

0.0

49.5

57.4

0.0

0.0

2.5

D External debtr serÕice ratio Share in total 0.0 program year recursionsb Share in total 54.0 nonprogram year recursions c

a Excluding full sample estimates, the total number of recursive estimates was equal to 87 for program year observations, and to 202 for nonprogram year observations. b Recursive procedure starts with the baseline sample Žall nonprogram observations plus one program observation. and adds program observations one by one. c Recursive procedure starts with the entire sample and subtracts out one by one nonprogram observations.

reduced form equation  yŽ b jkg k j q 1.4 should be close to y1. But the data strongly reject this hypothesis, except in the growth equation.36 This finding,

36 The null hypothesis of an estimated coefficient equal to y1 on the lagged dependent variable is rejected at the 10% level for real GDP growth, 5% level for the inflation rate, and 1% level for the debtrservice ratio.

Table 5 Generalized evaluation estimates of policy parameters Ž b . a Target variable

Policy variable

b 1 y2 standard errors

b1 Ž t-statistics.

b 1 q2 standard errors

y0.042 Žy1.37.

0.019

y0.180 Ž1.55. y0.033 Žy1.08.

0.051 0.028

y1.181

y0.467 Žy1.31.

y6.101 y0.954

y2.592 Žy1.48. 1.358 Ž1.17.

D Real GDP growth rate Full sample y0.103 estimates Recursive estimates Ø Minimum y0.411 Ø Maximum y0.094 D Inflation rate Full sample estimates Recursive estimates Ø Minimum Ø Maximum

Lagged net domestic asset growth

D External debtr serÕice ratio Full sample y0.157 0.097 Ž0.76. estimates Recursive estimates Ø Minimum y0.291 y0.116 Žy1.33. Ø Maximum 1.095 1.790 Ž5.149.UU

b 2 y2 standard errors

b 2 q2 standard errors

b 3 y2 standard errors

b3 Ž t-statistics.

b 3 q2 standard errors

0.004 Ž1.82.

0.009

y0.027

y0.009 Ž1.03.

0.009

y0.001 y0.004

0.003 Ž1.46. 0.013 Ž1.51.

0.006 0.030

y0.055 y0.005

y0.022 Žy1.30. 0.010 Ž1.32.

0.012 0.026

0.247

y0.207

y0.088 Žy1.47.

0.031

0.025

0.917 3.669

y0.216 y0.032

y0.116U Žy2.31. 0.162 Ž1.67.

y0.015 0.356

0.351

y0.043

y0.020 Žy1.78.

2.486

y0.068 y0.067

y0.041UU Žy2.99. 0.004 Ž0.11.

0.0004

b2 Ž t-statistics.

Lagged NEER Žpercentage change.

0.436U Ž2.12.

0.846

y0.190 0.193

0.170 Ž0.95. 0.873U Ž2.567.

0.530 1.553

0.002

y0.052

0.058 Ž1.05.

0.168

0.076

y0.174 y0.018

y0.057 Žy0.97. 0.128 Ž1.76.

0.060 0.275

519

a Standard errors and t-statistics are computed using White’s heteroschedasticity-consistent variance–covariance estimator. A single asterisk indicates statistical significance at the 5% level; two asterisks indicate statistical significance at the 1% level.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Lagged fiscal balancerGDP

0.775

y0.691U Žy2.27 . 0.277 Ž1.11 .

y1.300 y0.222

0.277

y0.708 y0.065

y0.289

y0.768 0.192

y0.097

y1.861 y0.355

y0.795

gy2 standard errors

y0.397 U Žy2.55 . 0.211 Ž1.53 .

y0.152 U Žy2.22 .

y0.439UU Žy2.67 . 0.511UU Ž3.20.

y0.017 Ž0.29 .

y0.685 Žy1.16 . 0.510 Ž1.18 .

y0.204 Žy0.69 .

g Ž t-statistics.

D NEER Žpercentage change .

y0.085 0.487

y0.015

y0.110 0.830

0.131

0.491 1.375

0.388

gq2 standard errors

degrees of freedom ..

a Standard errors and t-statistics are completed using White’s heteroschedasticity-consistent variance–covariance estimator. A single asterisk indicates statistical significance at the 5% level; two asterisks indicate statistical significance at the 1% level. b Recursive least squares estimates obtained by adding observations one-by-one to an initial sample of 25 nonprogram years Žcorresponding to a minimum of 20

0.389

y0.160 1.861

y0.097 Žy0.40 .

y1.398U Žy2.26 . 0.504 Ž0.74 .

1.312

0.431 1.482

0.882

y0.584

y2.636 y0.851

0.300 0.356

y0.448 Žy1.20 . 0.104 Ž0.82 .

y0.081 Žy0.12 .

y1.181 Žy1.46 . 0.149 Ž0.22 .

y1.090 Žy1.11 .

gq2 standard errors

0.039

y1.474

0.016

0.006 Ž1.02 .

y2.793 y1.184

0.394 0.516

y0.159 Žy0.57 . 0.156 Ž0.86 .

Lagged external debt r serÕice ratio Full sample y0.041 y0.0007 Žy0.04 . estimates Recursive estimates b Ø Minimum y0.102 y0.029 Žy0.79 . Ø Maximum y0.127 0.075 Ž0.74 .

Lagged inflation rate Full sample y0.005 estimates Recursive estimates b Ø Minimum y1.196 Ø Maximum y0.148

y3.063

0.269

0.0024 Ž0.19 .

g Ž t-statistics.

gy2 standard errors

gq2 standard errors

gy2 standard errors

g Ž t-statistics.

D Net domestic asset growth

Policy variable D Fiscal balancerGDP

Lagged GDP growth rate Full sample y0.222 estimates Recursive estimates b Ø Minimum y0.712 Ø Maximum y0.205

Target variable

Table 6 Sensitivity of estimates of policy reaction function parameters Žg . a

520 L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

521

Table 7 Share of statistically significant t-statistics at the 5% level in recursive estimates of the policy reaction functiona Žin %. Policy variable

Lagged GDP growth rate Lagged inflation rate Lagged external debtr service ratio

D Fiscal balancer GDP

D Net domestic assets growth

D NEER percentage change

H o: g s0

H o: g s full sample value

H o: g s0

H o: g s full sample value

H o: g s0

0.0 Ž0.0.

0.0 Ž0.0.

0.0 Ž0.0.

0.0 Ž3.4.

0.0 Ž0.0.

0.0 Ž0.6.

39.3 Ž57.9.

0.0 Ž38.8.

0.6 Ž3.4.

9.0 Ž9.6.

3.4 Ž7.3.

2.8 Ž7.3.

0.0 Ž0.0.

0.0 Ž0.0.

9.0 Ž14.0.

0.0 Ž7.9.

70.8 Ž73.6.

5.6 Ž14.0.

H o: g s full sample value

a

Excluding full nonprogram sample estimates, the total number of recursive estimates was equal to 178 for nonprogram years observations. Shares of statistically significant t-statistics at the 10% level are reported in parentheses.

together with the fact that the reduced form coefficients for the lagged target variables are significantly different from zero, calls into question the GEE assumption that all shocks to target variables are fully reversed within one period. Indeed, if the reaction function parameters cannot account for the statistical significance of the lagged target variable, partial inertial effects in the target variables may explain why initial conditions do influence subsequent macroeconomic performance. However, the logical consequence of interpreting the coefficients of the lagged target variable as reflecting inertial effects is that all other coefficient estimates should also be interpreted in light of a richer dynamic specification. In this case, the b jIMF estimates cannot be interpreted as the full contemporaneous effect of IMF support in a given period.

5. Conclusions With respect to the central objectives of this paper — to use the GEE framework to identify the independent effects of ESAF support during 1986–1991 on key macroeconomic variables and to assess whether the assumptions underlying the GEE are applicable to the ESAF-eligible countries — conclusions can be summarized as follows. For output growth and the debtrservice ratio, sizable beneficial effects that are statistically significantly different from zero are identified.37 The effects on inflation are not significantly different from zero. These 37

For the external debtrservice ratio, statistical significance was at the 10% level.

522

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

results are similar to those found in other studies, albeit for samples of countries that differed from those used here. The present paper, however, goes beyond others applying the GEE and asks the questions: ‘‘Are the assumptions underlying the model valid for the cross-section of ESAF countries and, by extension, are GEE results for these countries reliable?’’ In fact, a battery of diagnostic tests casts doubt on the applicability of the GEE framework at least to the ESAF-eligible countries: the overall fit of the model is poor; estimates of the coefficients on many variables are insignificantly different from zero; regression residuals are heteroschedastic and nonnormally distributed; and the estimates of the coefficient on the dummy for ESAF support are quite sensitive to variations in the sample. A striking finding is that the counterfactual policy reaction function does not have any significant explanatory power for the sample of nonprogram observations. These results raise questions about the validity of other applications of the GEE that do not rigorously test underlying assumptions. The GEE is a rigorous framework, conceptually superior to before–after and simple control-group comparisons for identifying the independent effects of IFI financial support. It is, however, based on many restrictive assumptions that are necessary to define the counterfactual and to specify in a simple framework the main determinants of important endogenous macroeconomic variables. A major shortcoming of most applications of the GEE is their focus on the bottom line — the estimates of the effects of IMF support — with little or no evaluation of the validity of the underlying model; indeed, some studies have reported only estimated coefficients on the dummy variables and their standard errors, without diagnostic statistics or the estimates for other coefficients. One important lesson to be drawn from this study is that the validity for any given sample of the premises of the GEE methodology must be investigated before reliable conclusions about the independent effects of IMF support can be drawn from it. Indeed, on the basis of this study, it cannot be ruled out that the inherent limitations of panel data covering countries facing highly diverse circumstances render it impossible to obtain reliable estimates of the independent effects of IMF-supported programs.

Acknowledgements The authors are indebted to Sharmini Coorey for her considerable input during the early stages of the project and to Kirsten Fitchett for her invaluable research assistance. We also thank Adam Bennett, David Burton, Benedicte Vibe Christensen, David Goldsbrough, Morris Goldstein, Mohsin Khan, Peter Montiel, Roger Nord, and Alessandro Zanello and two anonymous referees for their comments on an earlier draft of this paper. The views expressed in this paper are those of the authors and do not necessarily represent those of the International Monetary Fund.

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

523

Appendix A. Sample selection bias and a model of program participation Sample selection bias in the parameter estimates of the policy reaction function ŽEq. 2. fitted over nonprogram observations can arise when unobservable factors that make a country more likely to seek IMF assistance also make a country more likely to have adopted a different policy package in the absence of a program than another country facing similar circumstances. To correct for this sample selection bias, following Heckman Ž1979., the estimated policy reaction function is augmented by an additional regressor which accounts for the bias due to nonrandom sampling. The correction requires the estimation of a model of the probability of program status. Program status is characterized by a dummy variable d i that equals one if the country has a IMF-supported program; otherwise, a zero: Ii s d X D q p i ,

Ž 4.

d i s 0 if Ii ) I U , d i s 1 if Ii F I U , where I is a random variable, which is an index of country-specific characteristics that determines the probability of country I not having a program; and D is an n-element vector of variables that determines program status; d is a nxl vector of parameters; p i is a zero mean fixed variance error term. The bias in the reaction function stems from a correlation between p i and the error term Žhi k . in the reaction function ŽEq. 2..38 Estimates of the probability of nonprogram status were obtained using the probit model Žfitted over the full sample.: Prob Ž d i s 1 . s f Ž yd X D . ,

Ž 5.

Prob Ž d i s 0 . s 1 y F Ž yd X D . ,

Ž 6.

where f and F denote the density and distribution function for a standard normal variable, respectively. The estimated parameters of the probit equation are used to compute the ŽInverse Mills Ratio. IMR, i.e., the ratio f Žyd X D .rw1 y F Žyd X D .x,39 which is included in the reaction function fitted over nonprogram countries to obtain unbiased estimates of the policy response coefficients. Assuming that hi k and p i 38 In terms of the reduced form GEE, a correlation between the p i and the error term in Ž e i j q b jk hi k . in Eq. 3 will lead to biased coefficient estimates. 39 This ratio is a monotone decreasing function Žbounded by 0 and `. of the probability that an X X observation is selected into the sample of nonprogram countries, f Ž d D . or identically w1yF Žy d D .x.

524

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

have a bivariate normal density, the IMR takes into account the separate effect of not having a program on policy responses. Fitted values of the estimated policy reaction function provide the counterfactual set of policies Ž x i k . for program countries, which can be used in the estimation of the GEE ŽEq. 1. to obtain an unbiased estimate of b jIMF. An alternative procedure would be to include the IMR as a regressor in the reduced form GEE ŽEq. 3.. This would also take into account sample selection bias arising from the possibility that the choice of having a program is influenced by expectations of better performance in the target variable.40 Several explanatory variables were considered for the probit model of program status: the ratio of the overall balance of payments to exports;41 the external debtrservice ratio; the ratio of the flow of external payments arrears to exports;42 the real GDP growth rate; the inflation rate; the percentage change in the terms of trade; the growth rate in export markets; a dummy variable with a value of 1 for persistent arrears with the IMF Žwhich would preclude a program., 0 otherwise; and a dummy variable with a value of 1 if a country had previously had an IMF-supported program, 0 otherwise countries familiar with IMF program operations may be more likely to adopt a program. The explanatory power of these regressors was weak, and at times was associated with unexpected or counterintuitive signs. In part, the difficulty of explaining the nonprogram–program status of a country reflects the fact that the program periods under review are several years in duration: there can be substantial changes Žimprovements. in the macroeconomic variables that prompted countries to implement an IMF-supported program during the program period. Also, while ‘‘economic need’’ may turn a country towards adopting an IMF-supported program, variables such as external arrears may not explain the precise timing of the decision to start a program. The best fit probit regression that was used to correct for sample selection bias in the policy reaction function is reported in Table 8. It should be noted that the residuals of the probit model exhibited evidence of nonnormality, in which case the equation estimates and the calculated IMR are likely to be inconsistent Žsee

40

Yet another procedure to obtain a consistent estimate of b jIM F would be to use the predicted probability of undertaking a program as an instrument for d i in Eq. 3; in this case, the policy reaction function need not be estimated separately. This procedure would also correct for the possible simultaneity of program choice and performance in the target variable. See World Bank Ž1990. and Corbo and Rojas Ž1992. for applications of this procedure. 41 The overall balance of payments was measured including scheduled debtrservice payments and excluding exceptional financing, in order to provide a measure of underlying pressures on the external position. An alternative indicator of an external need to seek an IMF-supported program would be the leÕel of international reserves. However, for several countries in the sample ŽCFA and ECCB country members., data on reserves are not available. 42 Two versions of this variable were tried: one in which both positive and negative changes in arrears were recorded, and one in which only positive changes were included. The second version isolates the buildup of arrears Žtypically prior to programs. and excludes the repayment of arrears Žtypically during programs..

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

525

Table 8 Probit model of nonprogram statusa Variable Constant Lagged ratio of balance of payments to exports Lagged external debtr service ratio Lagged change in terms of trade Log likelihood Pseudo-R 2 b Percent correct predictions c Number of observations Jarque–Bera test for normality of residuals in auxiliary regression

Coefficient

Partial effects at the means Žpercentage points.

0.756 Ž7.06. y0.0002 Žy0.32.

0.25336 y0.00007

y0.005 Žy2.95.

y0.00181

0.007 Ž1.46.

0.00249

y172.041 0.035 0.70 291 47.29UU

a

The figures in parentheses are t-statistics. The pseudo-R 2 measure is equal to 1yŽlog L UR rlog L R ., where log L UR is the maximum of the likelihood function when maximized with respect to all parameters and log L R is the maximum when maximized with respect to the constant term only. c A predicted probability greater than or equal to 0.50 is associated with nonprogram status. b

Greene, 1993..43 Therefore, the correction for sample selection bias, which relies on the consistency of the IMR should be interpreted with caution.

References Breusch, T., Pagan, A., 1979. A simple test for heteroschedasticity and random coefficient variation. Econometrica 47, 1287–1294. Conway, P., 1994. IMF lending programs: participation and impact. Journal of Development Economics 45, 365–391. Corbo, V., Rojas, P., 1992. World Bank supported adjustment programs: country performance and effectiveness. In: Corbo, V., Fischer, S., Webb, S.B. ŽEds.., Adjustment Lending Revisited. Policies to Restore Growth. The World, WA, p. 5. Faini, R., de Melo, J., Senhadji, A., Stanton, J., 1991. Growth-oriented adjustment programs: a statistical analysis. World Development 19 Ž8., 957–967. Goldstein, M., Montiel, P.J., 1986. Evaluating fund stabilization programs with multicountry data: some methodological pitfalls. In: Staff Papers 33 International Monetary Fund, WA, pp. 304–344.

43 The probit residuals Žp i . were regressed upon a constant, and the residuals of this auxiliary regression revealed evidence of nonnormality; the Jarque–Bera test for nonnormality had a test statistic of 47.29, statistically significant at the 1% level.

526

L. Dicks-Mireaux et al.r Journal of DeÕelopment Economics 61 (2000) 495–526

Greene, J., 1989.The effects of fund-supported adjustment programs in African countries, 1973–86. IMF Working Paper No. WPr89r38, International Monetary Fund, WA. Greene, W.H., 1993. Econometric Analysis. 2nd edn. Macmillan, New York. Harvey, A.C., 1981. Time Series Models. Wiley, New York. Heckman, J., 1979. Sample selection bias as a specification error. Econometrica 47 Ž1., 153–161. Hsiao, C., 1986. Analysis of Panel Data. Cambridge Univ. Press, New York. Jarque, C.M., Bera, A.K., 1987. A test for normality of observations and regression residuals. International Statistical Review 55 Ž2., 163–172. Khan, M.S., 1990. The macroeconomic effects of fund-supported adjustment programs. In: Staff Papers 37 International Monetary Fund, WA, pp. 195–231. Knight, M., Santaella, J.A., 1994. Economic determinants of fund financial arrangements. IMF Working Paper No. WPr94r36, International Monetary Fund, WA. Priestley, M.B., 1981. Spectral Analysis and Time Series. Academic Press, London. Swamy, P.A.V.B., 1970. Efficient inference in a random coefficient regression model. Econometrica 38 Ž2., 311–323. White, H., 1980. A heteroschedasticity-consistent covariance matrix estimator and a direct test for heteroschedasticity. Econometrica 48, 817–838. World Bank, 1990. Adjustment lending policies for sustainable growth. Policy and Research Series, Paper 14, WA.