Environmental management decisions: A paramorphic analysis of planning permission for mineral development

Journal of Environmental Management (1995) 43, 249-264

Environmental Management Decisions: A Paramorphic Analysis of Planning Permission for Mineral Development Kenneth G. Willis

Department of Town and Country Planning, University of Newcastle, Newcastle upon Tyne NE1 7RU a n d w i t h a s s i s t a n c e f r o m R i c h a r d K. H a r d y

Law School, University of Hull Received 19 January 1994

Many decisions on the management of the environment are made within the Town and Country Planning Acts by way of development control decisions. The cognitive continuum and lens models are used as frameworks for describing how people make judgements in practice. A logit decision model is developed to analyse the recommendation of the planning officer and the decision of the planning committee, with respect to applications to develop mineral deposits. The problem of a "gold standard" against which decisions can be judged is discussed. The logit model of planning decisions has a high hit rate in predicting development control judgements. The results demonstrate that cue utilization by subjects using intuitive methods is poor. Planning officers and committee members engage in differential dimension focus: whilst they believe in complexity, in reality their judgements depend on just a few major variables.

Keywords: decision-making, environmental impacts, logit models, planning permission, minerals. "The essence of ultimate decisions remains impenetrable to the observer--often, indeed the decider himself... There will always be the dark and tangled stretches in the decision making process--mysterious even to those who may be most intimately involved". President John F. Kennedy

1. Introduction The Town and Country Planning Act 1947 empowered local authorities to prepare development plans for their areas. The economic rational of planning was to overcome problems o f market failure (Willis, 1980): 1. To ensure that every agent was a "price taker" and not a price setter: i.e. 249 0301-4797/95/030249+ 16 $08.00/0

© 1995 Academic Press Limited

250

Environmental management decisions

to overcome monopolistic control of land and avoid excess profits (through compulsory purchase and betterment tax); and to realize increasing returns to scale in the provision of local authority services (through key settlement policies, green belts and new towns); 2. To ensure that the prices taken by every agent are relevant: to minimize enviroumental externalities (through the separation of land uses, and by other planning conditions on developments); and by the provision of "public goods" where the free market failed to provide them (e.g. the conservation of buildings of architectural and historic importance by planning control and grants; provision of public open spaces; etc.); 3. To avoid problems arising from the presence of uncertainty (although planning blight actually created some!); 4. To ensure a more just and equitable distribution of economic welfare benefits (although some authors have argued that planning exacerbates inequalities--see Pearce and Nash (1973) and Reade (1982)). Subsequent Acts, such as the Town and Country Planning Act 1971; the Local Government, Planning and Land Act 1980; and the current Town and Country Planning Act 1990, and the Planning and Compensation Act 1991 (see Heap, 1991), all continued these philosophical aims. Many policies and planning decisions within these Acts are discretionary. The amount of land designated for housing, industry, mineral development, and so on, is discretionary to the extent that: 1. The local authority can determine the need for different kinds of development in the future and initiate a development plan to control developments in accordance with perceived needs; 2. The local authority can determine the perceived extent of externalities and designate the separation of land uses accordingly to minimize these externalities, within the framework of the use classes order. The discretionary nature of planning law and policy poses considerable problems in analysing individual applications for planning permission: for example, to develop mineral deposits. To appreciate this difficulty consider the law as it is generally formulated in criminal and civil cases, based on the property rights of individuals. For example, if individual A takes property from individual B, without B's consent, then it is termed theft. This is a 24 carat gold standard by which the law judges whether A is a thief or not. The actual tests employed to make this judgement such as identification by a witness, fingerprints, possession of the goods, intent to steal, circumstantial evidence, etc., may be less than perfect, that is, each test may be less than perfectly discriminatory (i.e. subject to error), and hence the evidence may be less than 24 carat in purity. But what is a 24 carat gold standard in planning? Is the gold standard whether the application for development is in conformity with the statutory development plan? Or that of an "expert", namely the planning officer's recommendation? Or the decision of the planning committee, that is the elected representatives (a jury)? Or, if permission is refused, and the developer appeals, is the gold standard the inquiry inspector's recommendation; or the Secretary of State's decision, which may or may not be in accordance with the inspector's recommendation? And who is it to say that the statutory development plan was "right" in the first place; or that there hasn't been a change in

K. G. Willis

251

material considerations, affecting the plan and its legitimacy since the plan was formulated? Clearly it is extremely difficult to determine the gold standard in discretionary legal cases such as these, in which planning permission is sought; and where permission is granted if the proposed development is in accordance with the development plan and there are no other material considerations; but where planning permission may also be granted if the development represents a departure from the plan, given that there are other material considerations. This problem of a gold standard in town planning contrasts with other professions such as medicine where: I. Decisions of practising clinicians can be compared with a gold standard verdict. Some studies have revealed a surprising divergence between clinical judgements and a "gold standard" verdict. For example, McGoogan (1984) found that autopsy (here regarded as the gold standard) failed to confirm 39% of the main diagnosis of clinicians and 66% of other conditions considered by hospital doctors and consultants to have contributed to death; 2. The accuracy of clinicial tests are known from controlled and laboratory experiments (see Sox et al. (1988) who list the true positive rate and false positive rate for many clinical tests). In discretionary law cases, the primary problem is determining whether a decision is "right" or "wrong" or "good" or "bad", given the lack of a gold standard against which to judge the decision. Hence this paper does not attempt to evaluate the accuracy of these (planning permission) decisions, given the lack of a gold standard on each case: we are simply not 100% certain that we can say that a decision is right or wrong. Arguably it is more appropriate in any case to judge discretionary decisions in terms of outcomes: whether welfare to society as a whole is higher with the development or without it. Such judgements need to be made with the aid of cost-benefit analysis. Rather this paper attempts to: 1. Determine what factors are instrumental in the planning officer's decision to recommend permission being granted or denied; 2. Determine what factors are instrumental in the planning committee's decision to grant planning permission or reject the application; 3. Assess the relative importance of factors determining the granting or rejection of planning permission by the planning officer and the planning committee, and any differences between them; 4. Develop a policy capturing equation of decisions on mineral applications; 5. Investigate whether bootstrapping can improve the hit rates and consistency of judgements between "experts", namely the planning officer, and other decisionmakers viz. the planning committee in determining planning applications.

2. Methodology Cognitive continuum theory serves as a framework for describing tasks people are capable of performing, and modes of cognition employed on these tasks (Hammond, 1978). Professionals in practice, whether physicians, estate agents, or lawyers, typically

Environmental management decisions

252 Well

1

structured

High

Scientific experiment 2 Co~.~lled

3

Quasi experiment Task

Possibility of manipulation; visibility of process; time required

4 Systemaided ~dgement

structure

5

Peer-aided judgement

Ill

6 Intuitive judgement Low

structured Intuition <

COGNITIVE MODE

> Analysis

Figure I. The cognitive continuum: the six modes of enquiry (after Hammond, 1978).

make intuitive judgements (at mode 6, see Figure 1). Intuitive thought involves rapid, unconscious data processing that combines the available information by averaging it, has low consistency, and is moderately accurate. At the other end of the cognitive continuum, analytical thought is slow, conscious, and consistent, and usually quite accurate (Hamm, 1988). A large number of empirical studies in different professions have revealed that intuitive judgements are subject to considerable error. The study by McGoogan has already been mentioned, de Dombal (1984) found the accuracy of diagnosis of cases in an accident and emergency department of a hospital varied from 42% for admitting doctors; 71% for house surgeons; 79% for registrars; 82% for senior clinicians; and 91% for a computer aided system. Renwick et al. (1991) found that radiographers disagreed with radiologists in 9.4% of examinations: there were 7% false positives and 14% false negatives. Dawes (1980) argued, in a comparison of university selection systems for admissions, that statistical methods of choice outperformed intuitive judgements by being more consistent, less discriminatory, and judged people on the basis of what they did, not how they impressed. People have limited information processing abilities, and this (bounded rationality) affects how people interpret values and make choices. In planning decisions, the planning officer predicts (recommends) an outcome y from a given data set (x), based on causal or correlational reasoning. Hammond (1975) conceptualized this task as a set of independent variables xl . . . . x~ determining the value of some dependent variable Ye; and a person's prediction, denoted by y,. The independent variables are information cues that the person uses to predict the true outcome y,, whereas y, denotes the person's

K. G. Willis

253

Variable of interest:

Subject's response:

Ye

Ys

or actual state

or judged state

Predicted level: A

Y.

Predicted response: A

Ys

Figure 2. Diagram of the Lens model. Note ¢/@and ~/, are prediction models.

prediction of this outcome (i.e. it is subjective) (see Figure 2). Because the diagram resembles light rays in a convergent lens, the conceptualization in Figure 2 is known as the Lens model. It is assumed that there is some actual, underlying, hidden condition or state--planning permission--which the planning officer (or planning committee) is trying to identify and classify, from observable signs, clues, cues, or indicants it produces. In the lens diagram a proposed development is thought of as emitting signals, differing in character, frequency and strength. The particular pattern of cues, signs, or indicants which decision-makers see, or assume will arise, is what they have to work back from to arrive at the underlying object/condition. Decision-makers succeed to the extent that they bring the cues back together in the same way as the actual state produced (or might produce) them. Since there is no way of knowing what the actual state (the "right answer") is, then the decision analyst can only work out how the planning officer/ committee might be using the cues--not how well they could be using them. The task for the decision theorist is to discover (1) the relative weight the person (or committee) assigns to each of the cues, (2) the functional form of each cue in relation to the individual's (or organization's)judgement, (3) the principle by which the data from the cues is organized, and (4) the consistency with which the judgement is exercised (Hammond, 1975). The decision to grant or refuse planning permission can be modelled as a discrete choice. The county planning officer either recommends that planning permission is refused (Y = 1) or granted CY= 0); and the county planning committee, as the planning authority, either refuses ( Y = I ) or grants (Y=0) the proposed development, on the

Environmental management decisions

254

basis of a set of factors (x) (outlined in Appendix 1) considered instrumental in determining the decision. Thus Prob[Y= 1] =F(x, B) Prob[Y = 0] = 1 - F(x, B) The set of parameters B reflect the impact of changes in x on the probability of planning permission being refused. For example, among the factors of interest is the marginal effect of decreasing distance from houses of the proposed mineral site; and the landscape impact of the development. An OLS regression model of the problem to refuse or grant planning permission has a number of short-comings: the error term in the model is heteroscedastic; but more important there is nothing constraining the predictions to the zero-one interval. Therefore, the logistic function e Bx

Prob[Y = 1] =1 + e Bx was adopted, and a linear logistic model employed for the dependence of p,. on the values of k explanatory variables associated with the observation: logit(pi) = log(p/(1 -- p~))= bo + b~x~,+ b~x2;+ . . . + bkXk/ where Pi = the probability of not obtaining planning permission. 3. Intuitive decision-making and data Before applying this discrete choice model, the intuitive method for deciding upon each mineral application is outlined, as the decision-making framework currently employed. Concomitantly, the data and information upon which intuitive decisions are based is also reviewed, since this forms an integral part of the process. Moreover, the data employed in arriving at each intuitive decision is the same data subsequently employed to estimate the discrete choice model. Recommendations on mineral development applications by the county planning officer, and decisions by the planning committee, are made with reference to: 1. The statutory development plan; 2. Central government guide-lines, such as the Mineral and Planning Guidance (MPG) Notes; 3. Any other material considerations. Central government recommendations such as those outlined by the Department of the Environment (1988) (MPG3) provide guide-lines to the local planning authority. However, local authorities can also have their own policies with respect to mineral developments, which form part of County Structure Plans, or Unitary Development Plans, and mineral applications are also judged in relation to these. The groups of factors influencing a mineral planning decision can be thought of as those associated with the: 1. Mineral itself: whether the application is to extend a site, which may be easier

K. G. Willis

255

to grant; or for a completely new site, which may generate considerable opposition; the size of the site; main mineral to be extracted; the estimated size of reserves; and the number of minerals which can be extracted simultaneously; 2. Conformity of the proposal to the development plan and MPG guidelines (e.g. distance from the nearest 10 dwellings); the effect on the landscape; whether the development has a cumulative adverse environmental impact on local communities; whether the development is piecemeal, or part of a strategy etc.; 3. Market and economic considerations: whether a market exists for the mineral; the impact of the development on the labour market (the number of jobs to be created); 4. Other environmental and economic effects: whether objections to the development are received; whether proposed subsequent building development on the site will sterilize the mineral; whether the development will allow an existing derelict site to be reclaimed; transport externalities associated with the site, and so on. All of these different factors are considered by the planning officer and planning committee in deciding to refuse or grant planning permission. The factors are judged with reference to various policies that comprise the Development Plan for the area, to ensure that mineral proposals conform with the Development Plan, unless there are other material considerations. The Development Plan outlines uses and future uses of land including land not designated for mineral development, and other areas zoned for future mineral development. The significance of these factors varies not only with respect to land designation identified in the Development Plan; but also in relation to other material considerations, such as the creation of possible hazards for other road users, congestion, the effect of blasting and noise on local residents etc. These other material considerations can vary significantly with respect to each site subject to a mineral application, so that the intuitive decisionmaker has to weigh up these disparate factors and assess the importance or the magnitude of each factor at different sites. A priori expectations can indicate a particular relationship between some factors and the development decision. For example, sterilization of a mineral may arise due to the approval of another planning application for an alternative land-use, and this should increase the likelihood of obtaining planning permission (see MPG Note 3). Similarly, increased employment as a consequence of a mineral development may be expected to increase the probability of approval, although the influence of this factor may vary depending upon the unemployment rate in the local area, and whether the employment opportunities are likely to be filled by local people. These are standard factors which are assessed by the planning officer (CPO) for each planning application. However, the values of these factors can vary extremely widely between sites, and need to be weighted in the intuitive decision. Moreover, the factors listed (Appendix i) upon which the intuitive decision is based are not wholly objective or definitive; for example, the cumulative impact of mining on local communities, or the landscape impact of a proposed development, are only subjectively quantified on a nominal or ordinal scale. Whereas other factors such as tonnage of reserves to be extracted, number of minerals, and employment to be created are assessed on a cardinal scale. There are other reservations about the data. The data records whether the application conforms to the Development Plan, or whether it constitutes a material departure, or non-material departure from the Plan, and also particular attributes in the environment

256

Environmental management decisions

with respect to employment etc. that council policies aim to promote or restrict. However, difficulties can arise in individual cases, e.g. where the information in the CPO's report differs from the standard set with respect to a policy. For example, dwelling distance in the data assesses the distance of 10 or more dwellings from the development. This conforms with Policy 88 of the County Structure Plan which states that, as a rule, mineral extraction should not be permitted within 250 m (500 m if blasting is required) of the nearest group of 10 or more dwellings. However, the data provided by the CPO, for some mineral applications, refers to less than 10 dwellings, or to more dwellings more than 250 m from the site. In other cases such data is omitted, or the distance is only given to the nearest dwelling, say at 25 m distant, and not of the nearest group of dwellings. How such discretionary variations in data are incorporated in the intuitive decision is unknown; but it also creates difficulties in discrete choice modelling since the definition of the variable has changed, and hence it must be regarded as a missing observation. Particular mineral development applications show that mineral development policies can be applied in a discretionary manner. It is possible to cite particular cases that on occasion contravene development plan policies, or MPG advice, in order to allow a development, presumably where it is merited on other material considerations, such as the need for local employment. For example, a number of cases contravene Policy 88 of Durham County Council's Statutory Development Plan, but planning permission was still granted. Mineral development decisions are made with a considerable degree of freedom. Indeed, the planning permission process actually involves tenuous negotiations between the mineral developer (the extractor) and the planning officer who provides information, advice, and a recommendation in his final report to the development control committee. Thus it is the planning officer who makes the recommendation to the local authority planning committee, and who has an influence on the planning committee's decision. The ultimate decision by the planning committee may also be subject to a substantial degree of discretion and influence as the political expression of councillors' and their constituents are aired. Again, this poses data problems. Planning recommendations made by the CPO, are based on technical rationality, but are also influenced by the CPO's perception of the CPC's likely thinking. Similarly, decisions by the CPC are influenced by the CPO's overall perception of the case, comprising a combined consideration of the material facts, policies relevant to the application, and political realism. The data for this study refers to all mineral applications to Durham County Council from 1988 to 1992. The planning officer's report, to aid an intuitive (mode 6) decision, on each application, contains consideration of a great number of factors in the context of the development plan and M P G Notes. All factors documented in the mineral development control officer's report, perceived as influencing the planning decision were recorded and a data base established of these factors. These various factors, instrumental in determining planning permission, are reported in Appendix 1, and were recorded for each individual planning application. Because of the relatively small number of mineral applications per year, it was necessary to record all applications back to 1988. In all, 99 applications for mineral extractions were received by Durham County Council between 1988 and 1992, for which a planning decision was subsequently determined. The data on each application was collected from the reports submitted by the planning officer to the development control committee. The final decision of the planning committee on each planning application produced a 50-5% refusal rate, and conversely a 49.5% approval rate.

K. G. Willis

257

4. Results of the discrete choice model

Of the 99 cases in the sample, the CPO recommended that planning permission should be refused in 38 cases and granted in 44, whilst, for the remainder, he made no recommendation. The CPC refused planning permission in 50 cases and granted permission in 49. There were 82 cases where the CPO made a recommendation and the CPC a decision. In 34 of these cases the CPO recommended refusal and planning permission was refused by the CPC; however, the CPO recommended refusal and the CPO granted planning permission in 4 cases. In another 40 cases, the CPO recommended planning permission be granted and the CPC granted permission; whilst, in the remaining 4 cases, the CPO also recommended planning permission be granted, but it was refused by the CPC. A simple linear functional form of the logit model was used. Lens model research suggests that humans tend to think about the environment linearly, whether the environment is linear or not (see Brehmer and Brehmer, 1988). Since the mind of a human "expert" is typically simpler (i.e. more linear) than the environment he or she is trying to predict, it seems appropriate to employ a linear (logit) functional form to model decisions, rather than a more sophisticated non-linear model. The results of the logit models of CPO and CPC decisions are reported in Table 1. Only those variables significant at a 10% level or more were included in the models. Clearly, detrimental landscape impacts increase the probability that the CPO will recommend refusal of planning permission; as does the cumulative impact of mineral developments in an area (Table 1). Similarly, objections to the development by the district council also increased the probability of a refusal. The CPC based their decision on some different and additional variables to those used by the CPO. Thus, the discrete choice model indicated that, for the CPC, the type of development, and the question of sterilization of the land, was also significant in influencing a decision. However, the cumulative impact of mineral development on an area was not significant in the CPC's decision. Again in line with expectations, a new development proposal was associated with an increase in the probability of refusal (and conversely an extension to an existing site increased the probability of obtaining planning permission); while the prospect of the site being sterilized by another proposed development on it decreased the likelihood of refusal for mineral extraction, in line with advice in MPG3 and elsewhere that sterilization arguments might outweigh objections to a proposed development. Landscape impact and district council views both influenced CPO and CPC decisions in the same way. However, as Table 1 reveals, the CPO and the CPC placed different weights on these variables. Landscape impacts were much more likely to increase the probability that the CPC would refuse planning permission for the mineral development than for the CPO. Similarly if the district council opposed the development, the CPC was much more likely to refuse planning permission than the CPO. These models of the decisions by the CPO and CPC perform quite well, with overall "hit rates" of 81-4% and 82.9% respectively. As a test, the results of both models are highly indicative of whether planning permission will be refused or granted: i.e. the sensitivity and specificity rates are extremely high for these two policy capturing equations, compared with similar policy capturing equations in other professions. In terms of being able to predict whether a planning proposal will be refused or granted prior to the recommendation of the CPO or the decision by the CPC, again both models have high predictive value positive and high predictive value negative rates.

258

Environmental management decisions

•,-,

t:~ " 0 0 !~I

:o ¢~

,~ :~o = ~ I D . - ~ ".~

I=I

©

r~ o6.=

o I

I

i 9 9 9 9 0 0 0 0

9 0

0

,.0

o

9 ~ o

I

I

I

,.0 "i=

O0 ',,ID ,.--~ ID.,

6 6 6 I

0

o

o

o A

c~a

o

"1=

o

6 6 6

0o oo

,.-~ ¢",1 ~"~ o o t"~- ¢',,I

I

I

0

K. G. Willis

259

What can be concluded from Table 1, and the observations above, about the ability of professionals to make decisions? The results demonstrate that cue utilization by subjects using intuitive methods is poor. Out of all the variables considered by the CPO and CPC (see Appendix 1), only 3 or 4 appear to be instrumental in determining a planning decision. Thus, planning officers and committee members engage in "differential dimension focus": they look for just a few variables than they are aware of themselves. Whilst they believe in complexity, in reality their judgements depend on just a few major variables. This means they ask for information they hardly use. In addition, comparing the two logit regression equations, it can also be seen that the CPO and the CPC place different weights on the limited cues. Of course this comparison is only strictly legitimate if the "true" environmental model determining planning decisions is indeed known. This is seldom the case: as argued earlier, a 24 carat gold standard test for a planning application has yet to be determined! Nevertheless, the CPO, and CPC members, appear to exhibit poor insight into how much weight they place on different cues. These are fairly common findings by decision analysts in terms of the lens model: many studies and experiments in a number of different professions have shown that subjects have a poor insight into how they make decisions, that intuitive decisions have low reliability, and are inconsistent between one case and the next (see, for example, Kirwan et al., 1984; Willis and Garrod, 1993). 5. Discussion

Given these findings on planning judgements, would it be better to rely on objectively validated models? Many studies, in a number of different disciplines, have compared expert's intuitive predictions with expert's own regression model. Invariably a linear model of the decision-maker outperforms the intuitive predictions of the expert himself (see Dawes and Corrigan, 1974; Dawes et al., 1989). By estimating the implicit weights used by the human judge, and eliminating any noise or inconsistencies, such bootstrapping models, so-called because an individual can improve his judgement or pull himself up by his own mental bootstraps, typically outperform the human judge. However, in the case of mineral development applications in Durham, the results are not clear cut. Table 2a reveals that there is a high degree of correspondence between the CPO's recommendation and the CPC's decision: with the CPO correctly estimating the outcome in 90.2% of all cases where both reached a clear decision. The policy capturing (bootstrapping) equation of the CPO correctly predicted 81"4% of his recommendations, based upon three variables (Table 2b). The policy capturing equation of the CPC correctly captured 82"9% of its decisions, based upon four variables (Table 2c). A test of the policy capturing equation model of the CPO's decision is how well it can predict the outcome of the CPC's decision. Table 2e reveals that it can predict 82"9% of the CPC's decisions. This is obviously less than that of the CPO using his own intuitive judgement. In other words, the CPO's decision process appears to outperform his bootstrapping equation. It is not easy to understand how this is achieved: adding more variables into the CPO's bootstrapping model, by lowering the statistical significance for inclusion, does not improve the model's "hit rate"; in fact when all the variables are included the hit rate of the model actually decreases marginally. Out of the 12 incorrectly predicted development applications by the CPO model (Table 2c), 8 were border line cases where the probability of refusal or acceptance was extremely close to the cut-off value: that is, the data did not indicate a clear cut decision either

260

Environmental management decisions TABLE 2. Policy capturing equations (PCE)

(a) Cases where both CPO & CPC reach clear decision ,Event •~ Planning permission by CPC •~ o O O

Grant Refusal

Grant 40 4

Refusal 4 34

(b) PCE of CPO recommendation Event Planning recommendation by CPO "~ ~ ~.

"~ .o Grant Refusal

Grant 34 4

Refusal 8 24

Grant 31 7

Refusal 6 26

Hit rate: 81-4% Sensitivity: 81.6% Specificity: 81.3% PVP: 83.8% PVN: 78.8%

Hit rate: 90"2% Sensitivity: 90"9% Specificity: 89"5% PVP: 90.9% PVN: 89.5%

(c) PCE of CPC decisions for cases where CPO makes clear recommendations Event Planning permission by CPC

Grant Refusal

(d) Cases where CPC reaches decision for all cases of CPO recommendation or indifference = Event •,~ Planning permission by CPC ,~ o

Grant Refusal

Grant 34 ?

Hit rate:

74.7%

Refusal ? 40

O Hit rate: Sensitivity: PVP:

82.9% 89"5% Specificity: 75-0% 81.0% PVN: 85-7%

(e) CPO PCE applied to CPC decisions where both CPO & CPC reached clear decisions Event Planning permission by CPC

~,

Grant Refusal

Grant 37 1

Refusal 11 21

Hit rate: 82.9% Sensitivity: 97.4% Specificity: 65"6% PVP: 77"1% PVN: 95.5%

(t) CPO PCE applied to all CPC decisions Event Planning permission by CPC .~ "~

Grant Refusal

Grant 47 2

Refusal 18 32

Hit rate: Sensitivity: PVP:

79.8% 95-9% Specificity: 64-0% 72"3% PVN: 94.1%

K. G. Willis

261

way. Out of these 8 cases, the CPC followed the CPO recommendation in 4 cases, and reached the opposite decision in the other 4 cases. However, in the remaining 4 cases, the bootstrapping model indicated an extremely high probability that planning permission should be granted, yet in all of these cases the CPO recommended refusal and the CPC refused the application. In these cases one can only presume that the decision was made on other criteria, for example visiting the site on a bright sunny morning or allowing particular features of the site to unduly influence the decision, than those criteria contained in the CPO's report to the CPC; if the CPC's decision is accepted as the relevant "gold standard". Alternatively, the CPO can be seen as an extremely good judge of the CPC's thinking, picking up nuances and subtleties of elected members feelings; which the model is unable to do. Another reason may be that the CPC's decision is not a good "gold standard" i.e. an error of judgement has been made: in at least one case, an open cast coal site at Firtree, both the CPO's and CPC's decision was to refuse the planning application; however, the developer was granted planning permission on appeal. Perhaps a decision on an individual case was unduly influenced by political factors in the CPC. There is an important question about how good a "gold standard" CPC decisions actually represent. Clearly, it is not of the same standard as an autopsy, nor of the high accuracy of some other diagnostic tests in medicine and other disciplines, e.g. in engineering and the physical sciences. Thus, the CPC's decision is less than a 24 carat standard, and may only be a silver standard at best. Finally, and perhaps the most important reason for the close correspondence between CPO and CPC decisions is the fact that these two decision-making units interact with each other: the CPC uses the recommendations of the CPO in informing its own decisions; but also because the CPO gauges the CPC feelings and policy, and makes recommendations to the CPC on this basis. Hence, a close correspondence should be expected between CPO and CPC decisions. Where contact between decisionmakers does not occur, some studies have reported agreements as low as 52% to 59%, with disagreements in 25% of cases occurring even over wide discrepancies in value (Elffers et al., 1991). There are 17 cases where the CPO did not make an overt recommendation, so as not to influence the CPC unduly nor preclude the CPC's decision nor cause undue embarrassment to the CPC. This merely recognizes that there are degrees of recommendation: some decisions are more black and white than others, while for some applications there is a greater degree of uncertainty or indifference about which way the planning decision should go. If these cases are included in the analysis, then the correspondence between CPO and CPC decisions declines to 74.7% (Table 2d). Obviously this paints something of a false picture: if the CPO had to make a decision and simply tossed a coin he would have predicted half of these 17 cases correctly! This would have increased the hit rate to 83"8%. This still marginally outperforms the CPO's policy capturing equation when applied to all 99 cases (Table 2f). When applied to all 99 applications, the CPO's bootstrapping equation predicts those applications which will be granted planning permission extremely well; but tends to over predict refusals, so that the false positive rate is very high. 6. C o n c l u s i o n s

Paramorphic models try to mimic or predict the outcomes of the underlying decision process without claiming to emulate the sequence and manner in which information is processed by the decision-maker. Paramorphic models in medicine, education, or in

262

Environmental management decisions

specific social questions about discrimination (Maniscalco et al., 1980), or legal questions about parole (Glaser, 1985), etc., invariably outperform the judge whose decisions they seek to capture. The main finding of this study is that the CPO (for County Durham at least!) is a good judge of the CPC's decision on whether to refuse or grant planning permission for a mineral development; and that his intuitive judgement slightly outperforms his own policy capturing equation. Perhaps this should not be surprising, for two reasons. First, the decisions reached by the CPO and the CPC are not independent of one another: the recommendation by the CPO influences the CPC's decision; while the CPO in making a recommendation tries to decide in a "politically correct" way, by mimicking the feelings of the CPC, i.e. the CPO doesn't want to be seen to be making too many "wrong" decisions from the CPC's point of view! Second, CPC decisions cannot be regarded as a 24 carat gold standard, but rather as a silver standard. Whether an application should be "truly" refused or permitted is unknown: there is no test which will indicate this outcome. Thus predicting the "true" refusal or a "true" approval of planning permission from policy capturing equations is problematic, since intuitive decision-makers are less than perfectly discriminatory between cases, and also less than perfectly calibrated. This study has explored one area (mineral development) and one issue (the accuracy of paramorphic models) in decision making. Other issues could be explored. For example, does the CPO have a good idea of what variables he regards as being important in reaching a decision? How do these variables and the weights that the CPO attaches to them compare with the variables and weights determined by the CPO's own policy capturing equation? Why do the variables upon which the CPO makes a recommendation, differ from those employed by the CPC? Could statistical decision making techniques like those employed here improve decision-making within a discretionary legal framework? Probably: by making the actors involved in the process more aware of how they reach decisions, and the weights they attach to the different factors involved in refusing or granting planning permission. Whether statistical decision-making is likely to be employed, however, is another question. There is a ubiquitous recalcitrance to adopting decision models, despite their success in many disciplines. The reasons for this stem from the defensive position of professions, and also moral concerns which still favour intuitive judgements (Meehl, 1986). Can planners be regarded as good decision-makers with respect to discretionary environmental law? It seems to me that the jury is still out. We don't know whether the CPO or the CPC are good decision-makers with respect to a 24 carat gold standard, because we can't define the standard! However, we do know that the CPO and the CPC are well calibrated with each other, for Durham CC at least, with respect to mineral applications. What would be interesting to research is whether this also holds for other counties and metropolitan districts, and whether it holds across all types of planning decisions. Is the planning officer for minerals in Durham a "hot shot"? Are there other "hot shots" around? And conversely how many planning officers have "hit rates" below average? I am grateful to Richard Hird, who heads the minerals section in the planning department of Durham County Council, for access to mineral application records, and for advice. All views, and any errors in this paper, are, however, those of the authors.

K. G. Willis

263

References Brehmer, A. and Brehmer, B. (1988). What have we learned about human judgement from thirty years of policy capturing. In Human Judgement: the SJT I.qew(B. Brehmer and C. R. B. Joyce, eds). Amsterdam: North-Holland. Dawes, R. M. (1980). You can't systematise human judgement: dyslexia. New Directionsfor Methodology of Social and Behavioral Science 4, 67-78. Dawes, R. M. and Corrigan, B. (1974). Linear models in decision making. PsychologicalBulletin 81, 95-106. Dawes, R. M., Faust, D. and Mehhl, P. E. (1989). Clinical vs. Actuarial Judgement. Science 243, 1668-1673. de Dombal, F. T. (1984). Computer-aided diagnosis of acute abdominal pain: the British experience. Revue d'Epidemiologie et de Santk Publique 32, 50-56. Department of the Environment (1988). Minerals Planning Guidance: Opencast Coal Mining. MPG 3. DoE, London. Elffers, H., Robben, S. J. and Hessing, D. J. (1991). Under-reporting income: who is the best judge--taxpayer or tax-inspector? Journal of the Royal Statistical Society A154, 125--127. Glaser, D. (1985). Who gets probation and parole: case study versus actuarial decision making. Crime and Delinquency 31,367-378. Hamm, R. M. (1988). Clinical intuition and clinical analysis: expertise and the cognitive continuum. In ProfessionalJudgement: a reader in clinicaldecision making (J. A. Dowie and A. S. Elstein, eds). Cambridge: Cambridge University Press. Hammond, K. R. (1975). Social judgement theory: its use in the study of psychoactive drugs. In Psychoactive Drugs and Social Judgement: theory and research (K. R. Hammond and C. R. B. Joyce, eds). New York: Wiley. Hammond, K. R. (1978). Towards increasing competence of thought in public policy formation. In Judgement and Decision in Public Policy Formation (K. R. Hammond, ed.). Boulder CO, Westview Press, pp. 11-32. Heap, D. (1991). An Outline of Planning Law. Tenth edition. London: Sweet and Maxwell. Kirwan, J. R., Chaput de Saintonge, D. M., Joyce, C. R. B. and Currey, H. L. E (1984). Clinical judgement in rheumatoid arthritis. III. British rheumatologists' judgements of "change in response to therapy". Annals of the Rheumatic Diseases 43, 686-694. McGoogan, E. (1984). The autopsy and clinical diagnosis. Journal of the Royal College of Physicians of London 18, 240-243. Maniscalco, C. I., Doherty, M. E. and Ullman, D. G. (1980). Assessing discrimination: an application of social judgement technology. Journal of Applied Psychology 65, 1196-1207. Meehl, P. E. (1986). Causes and effects of my disturbing little book. Journal of Personality Assessment 50, 370-375. Pearce, D. W. and Nash, C. (1973). The evaluation of urban motorway schemes: a case study---Southampton. Urban Studies 10, 129-143. Reade, E. (1982). The effects of town and country planning. D202 Urban Change and Conflict: Unit 23. Milton Keynes: The Open University Press. Renwick, I. G. H., Butt, W. P. and Steele, B. (1991). How well can radiographers triage X-ray films in accident and emergency departments. The Radiographer 30(3), 112-114. Sox, H. C., Blatt, M. A., Higgins, M. C. and Marton, K. I. (1988). Medical Decision Making. Boston: Butterworths. Willis, K. G. (1980). The Economics of Town and Country Planning. London: Collins (Granada). Willis, K. G. and Garrod, G. D. (1993). Not from experience: a comparison of experts' opinions and hedonic price estimates of the incremental value of property attributable to an environmental feature. Journal of Property Research 10(3), 193-216.

Appendix 1 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Year o f a p p l i c a t i o n Location P r o p o s a l t y p e ( n e w site o r a n e x t e n s i o n ) Area M a i n m i n e r a l to be e x t r a c t e d E s t i m a t e d reserves to b e e x t r a c t e d District Council decision C o u n t y P l a n n i n g Officers r e c o m m e n d a t i o n Departure from Development Plan County Council decision

264

11. 12. 13. 14. 15. 16. 17. 18. 19. 20.

Environmental management decisions

Distance from nearest group of 10 dwellings Effect on landscape Existence of a cumulative impact Whether the development is piecemeal Whether a market exists for the mineral Employment to be created by extraction The possibility of sterilization Would reclamation occur Access problems to the site Number of private objections