- Email: [email protected]
The Delphi method and urbanization
The Delphi Method and Urbanization B. Marley-Clark, B.Comm. (Econ.), M.A. Senior Policy Advisor, Environment Canada, Ottawa
In this article the author puts forward his reasons for wishing to use the Delphi method for a future forecasting programme in Canada. He argues that the results of such a survey should be of great assistance in the field of urban affairs.
A
DELPHI STUDY IS AN ATTEMPT TO
bring the knowledge and intuition of a group of qualified individuals to bear upon the future possibilities in a given subject area. T h e Delphi technique or method as it is sometimes referred to, is a systematic means for developing and improving consensus on any numerical estimate. Although it is most often used in forecasting future technologies, it can be used in testing and developing group response to contemporary questions of almost any nature. Most readers will be aware that the Delphi method was developed at the R A N D Corporation by Theodore Gordon, Olaf Helmer and N o r m a n Dalkey. Dr. Dalkey has stayed on at R A N D and has done some testing of the Delphi method. Other workers in Delphi have done some testing of historical data on graduate college students' predictions of events that have already occurred. The results to date tend to give the technique increasing credibility for its use in predicting future events. The original R A N D Delphi called for 4 rounds of questions. Some Delphis have used as many as 13 rounds. It is the opinion of some of the experts that 3 rounds are sufficient if the panel of experts chosen are the best available in the field under examination. 1 agree with those experimenters who support the use of only 3 rounds. The other significant methodological issue in a Delphi study is the selection of the
DECEMBER, 1974
experts or what I prefer to call the 'blue ribbon' panel. By this expression, I mean the selection of recognized authorities in the field under examination. It is my belief that unless you are able to attract some of the best 'experts' available along with those who are more broadly qualified in a given area, the Delphi will not be as successful a prediction tool as it is possible of being. One might very well raise the question of why the Delphi method has recently received so much attention from forecasters in the management field. Why are not committees, task forces, brainstorming sessions and other group activities equally effective in accomplishing the task of predicting the future? The answer is thought to be in the psychology of groups: all of these traditional structures involve either hierarchical relationships, or at least some awareness of such relationships, which tend to affect adversely the outcome of the consensus. This phenomenon is referred to as the 'contamination of results', e.g., the boss very often influences his subordinates in a brainstorming session, even though all concerned are supposed to be on an equal footing, by the mere awareness of his being 'boss' in any other discussion or working situation. The further into the future, from the present, a forecaster tries to predict, the less valuable the base tends to be from which he is making the forecast. That is to say, other factors, not included in the present state of affairs (because they are unknown at the time the base is being formulated), will exert unknown influences on the prediction. Therefore, judgment, intuition and insight (all difficult to measure other than on the basis of individuals' 'track-records'), become increasingly significant in assessing the likely validity of any particular prediction or forecast. The lack of +hard data' is very often the
reason for using a Delphi study rather than such well established practices as trend extrapolation. As regards urbanization, one must ask: What are the indicators for urbanization? Numbers of commuters outside a certain radius of a city which have to come in and out of the city daily? The rate of crime? The degree of invasion of individual privacy ? There is a degree of 'hardness' to the first two 'indicators' but very little to the latter. When faced with the decision of allowing large cities (say, those over 2m. in population) to grow or whether to deconcentrate population to smaller cities (less than 2m.) some basis for making that decision must exist or be formulated. In this area a Delphi study can be of immense assistance, as the 'hard data' to base the decision on are so often not readily available. Delphi is an important tool also in reaching a consensus in an area where jurisdictions are interdependent. For example, in dealing with urban issues in Canada, the fact is that municipalities are constitutionally 'creatures' of the provinces. At the same time, however, provinces are structurally dependent on federal revenues through fiscal sharing agreements and transfer payments. Provinces, however, wish to exercise their autonomy in federalprovincial relations. Urbanization is no exception. The provinces' constitutional authority cover 'local works and undertakings' and 'municipal institutions', coupled with the rather wide federal influence on individual Canadians (76 per cent of whom live in urban areas) results in a delicate relationship between all levels of government as regards urbanization and the future of cities. In this context, however, Delphi can confront the crucial questions common to all cities in the country, registering the results anonymously and feeding back
81
these anonymous results to the respondents in the normal way, hopefully achieving consensus on the way or at least indicating that consensus is not possible without some 'loss of face' being suffered by the participants. Nor is there 'contamination of results' because one or another provincial government is seen to be more powerful than another provincial government; or one federal agent has more say in a jurisdictional area than does another federal agent. Joseph Martino in his book Technological Forecastingfor Decision-Making makes some very important points regarding the disadvantages of committees in dealing with futures. One of these is very important-namely the possible 'contamination' of results by certain members of the committee. 'A group can exert strong social pressure on its members--pressure for instance, to agree with the majority, even when the individual feels that the majority view is wrong. This is especially true in the production of group forecasts, since these are not really matters of fact but only of informed opinion. One member of the group may well give up presenting certain relevant factors if the remainder of the group persists in taking a contrary view'. Similarly 'brainstorming' has some very unfertunate side-effects; mainly because in groups, even where no leader is assigned, a dominant individual will emerge and exert pressure on the rest of the group, whether by means of putting across his ideas with a great deal of conviction, or through his very persuasive personality or just by overbearance. Task forces also have similar disadvantages to committees in that the behaviour of small groups generally has shown that it is often not the validity but the number of arguments for or against a specific position which determines the consensus on a given issue. In addition to dealing with the problem of 'contamination of results' effectively, Delphi has two other outstanding virtues: the nature and precision of the documentation both for future reference and for cross-referencing purposes; and the degree to which the influence of misinformation is reduced compared with committees. As the group interaction is carried out through responses to questionnaires, each round and its outcome are statistically collected and tabulated. All iterations (through the 3 rounds) and the feedback are carefully documented as each phase or round is completed. The advantage of this is that if there is concern, at any stage during the Delphi, about a particular result, the whole iterative process producing that particular result can be examined in detail. By contrast, minutes of committees are not structurally as rigorous 82
as reports on the succeeding rounds of a Delphi and often not as consistent in the use of language. Furthermore, the Delphi procedure presents a statistical response which embraces the opinions of all members of the panel rather than just representing majority opinion. Finally, Delphi is superior to committees or other groups because its results tend to be less open to misinterpretation. Very often the minutes of a meeting will record dissenting opinions only in form rather than substance indicating disenchantment, etc., leading the reader to form his own opinions, secondhand. Delphi information is structural in its questionnaire format, the results are tabulated with the response rate clearly indicated for all participants without identifying the participants by name. The anonymity, lack of contamination of results, establishment of statistical probabilities of certain future events occurring, all contribute to a solid information base upon which to make decisions. The lack of these qualities in committees and other group structures contribute largely to what I have called 'misinformation'. The above arguments and contentions are sufficient, from my point of view, to experiment with the Delphi methodology in trying to focus on the urban-related issues which can assist the federal government in Canada in arriving at a national urban policy. Furthermore, the Ministry of State for Urban Affairs* is largely responsible for ensuring that policy planning is carried out, at the federal government level, dealing with most urbanrelated problems in Canada. The problem areas which I will concentrate on in this article are four in number (there are many others). They are: urban transportation; environmental considerations; urban fiscal policy or municipal finance; and food production, marketing and distribution in the cities. Urban transportation. There has been considerable controversy over freeways vs. urban rapid transit and how these modes should be paid for. The Spadina expressway in Toronto was stopped 18 months after building had started. Public opinion wavered backwards and forwards for over 2 years before the decision to stop the expressway was made. Many experts were called in, 2 to 12 months, before the decision was taken. Similar situations exist in other parts of the country at this time, e.g., Ottawa and Vancouver. My contention here is that if a 'blue ribbon panel' on
*The author was formerly Strategic Planner for the Ministry of State for Urban Affairs and much of the experience gained in that position assisted him in writing this article.
the future of urban transportation had been assembled in these three cases and a Delphi constructed on the pertinent issues, policy alternatives for decisions could have been arrived at in a logical, ordered manner instead of by committee hearings or other group gatherings as has been the rule particularly in the Spadina expressway issue, with all the disadvantages that those procedures encounter. An urban transportation Delphi could also deal with problems of downtown parking, emission controls, traffic patterns, etc. Port development and airport location sites have also been considered by some to be part and parcel of urban transportation problems. Delphi might assist in deciding whether or not these latter two issues are part of the overall urban transportation problem by structuring the Delphi appropriately to include these concepts. Urban environmental issues in Canada encompass everything from solid waste disposal to urban parks as well as whether or not there should be greenbelts or concrete playgrounds for children. A Delphi can be constructed so that policy guidelines can be forthcoming on the degree of built environment a city should have and when it should have it, i.e., in what time horizons? e.g., 1975-1980, 1980-1985, etc. In the broadest meaning of environment the Canadian media have even carried stories linking crime with high-rise living and blaming ghetto life and poverty for stress in urban life. All of these issues can be examined in a Delphi thus providing information on which to base policy plans and decisions. However, it is not the purpose of Delphi to make the actual decisions, but rather to assist decision-making by procuring essential information from the decisions. With regard to urban fiscal policy and municipal finance a Delphi would be particularly helpful because of the anonymity factor of the numerous participants and levels of government involved in an area requiring great delicateness and tact. This is an area of federal-provicincial relations that requires as near to 'complete' information (both constitutional and financial) as possible in order to be useful to all parties concerned. Governments, i.e., federal and provincial, wish to be on an equal level of negotiation in talking about fiscal matters, especially regarding the cities. Naturally, unconditional grants, such as those administered under the federal Municipal Grants Act which pay a municipality a grant-inlieu of real property taxes due on Crown property, are the most welcome. Other grants, however, are not quite as popular because of their quid pro quo nature. These issues, because of their fragility, may be better handled in a Delphi than through committee or other group structures.
LONG RANGE PLANNING
The problem of food production, marketing and distribution in large Canadian cities is a particularly acute one. In the fiscal year June 1972 to June 1973 food prices have risen by an average of 16.9 per cent across Canada, and somewhere in the vicinity of 27 per cent in the major cities of Vancouver, Montreal and Toronto. To gain insight as to why food prices are higher in the cities, I find it especially helpful to reflect on Jane Jacobs' thesis (as postulated in her book, The Economy of Cities) about agricultural productivity. Ms. Jacobs makes the point that most labour and capital specialization take place in the city and then are exported, through skilled individuals, to the surrounding environs including the rural areas. The demand for packaging, quality, etc. often comes from the city-dweller. This means that the cost of increased technology, particularly research and development costs, are often, in the first instance, borne by the city-dweller. Gradually the technology is transferred outside the city and 'improves' food production, marketing and distribution in other areas outside the city proper. The questions of: how much of the capital costs are borne by the city-dweller ? How much of the benefits of the total development are passed on at very marginal costs to rural people ?; and other related questions can be expeditiously and accurately dealt with by using a Delphi procedure. Since the problem area of food production, marketing and distribution has its effects on individuals' disposable income and life-styles it is a good potential use for the Delphi method. In Canada, two major studies sponsored by Bell Canada, the major telephone utility, have been conducted on medical technology and edu-
DECEMBER, 1974
cational technology. One of the authors, Frank Doyle, has had much experience in using Delphi on futures technologies and I think he is in agreement that for a complicated area like food technology, Delphi is a very useful tool. Now, I would like to turn my attention very briefly to the technical questions of how one might structure a Delphi on the subject areas I have dealt with so far. Let us start with the question of rounds. Although the original RAND Delphi started with 4 rounds, in retrospect it was agreed by the authors of that study that the fourth round did not serve much purpose in achieving either a better consensus than that which existed after the third round, nor was the quality of the responses improved by a fourth round. That is why I would propose that the 3 rounds of carefully prepared questionnaires are sufficient and it is my intention to use only 3 rounds in this proposed Delphi on the cities in Canada. The nature of the questions in the questionnaire is obviously critical to the quality of the response that one can expect to achieve in a Delphi procedure. I hope to achieve this quality by following a few simple guidelines, which are: (a) Define several broad areas of the study--the number of areas being dictated by the simplicity or complexity of the subject area under examination--in this case there are four areas. (b) Carefully research each of the broad areas before drafting the questions in the questionnaire. (c) 'Kid-test' the questionnaire on a few knowledgeable colleagues to 'get the
bugs out'. In other words, make sure the questions are not: ambiguous, overly simplistic and verbose. (d) Design a thorough system of analysis for the data collected after each round. (e) Having completed items (a) to (d) select carefully the 'blue ribbon panel'--this is a very critical step and as I have already mentioned can severely bias results if not pursued prudently. In the interest of fairnessi it should be pointed out that if the decision-makers are included in the 'blue ribbon panel' there is a tendency to what is called the 'selffulfilling prophecy'. This can be viewed as either advantageous Or disadvantageous. In my opinion, if you can attract the best decisionmakers the 'self-fulfilling prophecy' aspect is a price that I, for one, would be willing to pay for the other benefits that Delphi has over any other group response. (f) Finally, one must work out a schedule for each of the 3 rounds and try and stick to the schedule so as not to exasperate yourself or your 'blue ribbon panel'. The results of the Delphi I want to conduct should be of great assistance to policy developers in the field of urban affairs in Canada. The four subject areas will collectively throw some light on the direction in which a national urban policy should be moving. Normative statements of this kind are not easy to achieve and if the Delphi method can assist us in so doing, we should use this tool in achieving the goal.
83
In this article the author puts forward his reasons for wishing to use the Delphi method for a future forecasting programme in Canada. He argues that the results of such a survey should be of great assistance in the field of urban affairs.
A
DELPHI STUDY IS AN ATTEMPT TO
bring the knowledge and intuition of a group of qualified individuals to bear upon the future possibilities in a given subject area. T h e Delphi technique or method as it is sometimes referred to, is a systematic means for developing and improving consensus on any numerical estimate. Although it is most often used in forecasting future technologies, it can be used in testing and developing group response to contemporary questions of almost any nature. Most readers will be aware that the Delphi method was developed at the R A N D Corporation by Theodore Gordon, Olaf Helmer and N o r m a n Dalkey. Dr. Dalkey has stayed on at R A N D and has done some testing of the Delphi method. Other workers in Delphi have done some testing of historical data on graduate college students' predictions of events that have already occurred. The results to date tend to give the technique increasing credibility for its use in predicting future events. The original R A N D Delphi called for 4 rounds of questions. Some Delphis have used as many as 13 rounds. It is the opinion of some of the experts that 3 rounds are sufficient if the panel of experts chosen are the best available in the field under examination. 1 agree with those experimenters who support the use of only 3 rounds. The other significant methodological issue in a Delphi study is the selection of the
DECEMBER, 1974
experts or what I prefer to call the 'blue ribbon' panel. By this expression, I mean the selection of recognized authorities in the field under examination. It is my belief that unless you are able to attract some of the best 'experts' available along with those who are more broadly qualified in a given area, the Delphi will not be as successful a prediction tool as it is possible of being. One might very well raise the question of why the Delphi method has recently received so much attention from forecasters in the management field. Why are not committees, task forces, brainstorming sessions and other group activities equally effective in accomplishing the task of predicting the future? The answer is thought to be in the psychology of groups: all of these traditional structures involve either hierarchical relationships, or at least some awareness of such relationships, which tend to affect adversely the outcome of the consensus. This phenomenon is referred to as the 'contamination of results', e.g., the boss very often influences his subordinates in a brainstorming session, even though all concerned are supposed to be on an equal footing, by the mere awareness of his being 'boss' in any other discussion or working situation. The further into the future, from the present, a forecaster tries to predict, the less valuable the base tends to be from which he is making the forecast. That is to say, other factors, not included in the present state of affairs (because they are unknown at the time the base is being formulated), will exert unknown influences on the prediction. Therefore, judgment, intuition and insight (all difficult to measure other than on the basis of individuals' 'track-records'), become increasingly significant in assessing the likely validity of any particular prediction or forecast. The lack of +hard data' is very often the
reason for using a Delphi study rather than such well established practices as trend extrapolation. As regards urbanization, one must ask: What are the indicators for urbanization? Numbers of commuters outside a certain radius of a city which have to come in and out of the city daily? The rate of crime? The degree of invasion of individual privacy ? There is a degree of 'hardness' to the first two 'indicators' but very little to the latter. When faced with the decision of allowing large cities (say, those over 2m. in population) to grow or whether to deconcentrate population to smaller cities (less than 2m.) some basis for making that decision must exist or be formulated. In this area a Delphi study can be of immense assistance, as the 'hard data' to base the decision on are so often not readily available. Delphi is an important tool also in reaching a consensus in an area where jurisdictions are interdependent. For example, in dealing with urban issues in Canada, the fact is that municipalities are constitutionally 'creatures' of the provinces. At the same time, however, provinces are structurally dependent on federal revenues through fiscal sharing agreements and transfer payments. Provinces, however, wish to exercise their autonomy in federalprovincial relations. Urbanization is no exception. The provinces' constitutional authority cover 'local works and undertakings' and 'municipal institutions', coupled with the rather wide federal influence on individual Canadians (76 per cent of whom live in urban areas) results in a delicate relationship between all levels of government as regards urbanization and the future of cities. In this context, however, Delphi can confront the crucial questions common to all cities in the country, registering the results anonymously and feeding back
81
these anonymous results to the respondents in the normal way, hopefully achieving consensus on the way or at least indicating that consensus is not possible without some 'loss of face' being suffered by the participants. Nor is there 'contamination of results' because one or another provincial government is seen to be more powerful than another provincial government; or one federal agent has more say in a jurisdictional area than does another federal agent. Joseph Martino in his book Technological Forecastingfor Decision-Making makes some very important points regarding the disadvantages of committees in dealing with futures. One of these is very important-namely the possible 'contamination' of results by certain members of the committee. 'A group can exert strong social pressure on its members--pressure for instance, to agree with the majority, even when the individual feels that the majority view is wrong. This is especially true in the production of group forecasts, since these are not really matters of fact but only of informed opinion. One member of the group may well give up presenting certain relevant factors if the remainder of the group persists in taking a contrary view'. Similarly 'brainstorming' has some very unfertunate side-effects; mainly because in groups, even where no leader is assigned, a dominant individual will emerge and exert pressure on the rest of the group, whether by means of putting across his ideas with a great deal of conviction, or through his very persuasive personality or just by overbearance. Task forces also have similar disadvantages to committees in that the behaviour of small groups generally has shown that it is often not the validity but the number of arguments for or against a specific position which determines the consensus on a given issue. In addition to dealing with the problem of 'contamination of results' effectively, Delphi has two other outstanding virtues: the nature and precision of the documentation both for future reference and for cross-referencing purposes; and the degree to which the influence of misinformation is reduced compared with committees. As the group interaction is carried out through responses to questionnaires, each round and its outcome are statistically collected and tabulated. All iterations (through the 3 rounds) and the feedback are carefully documented as each phase or round is completed. The advantage of this is that if there is concern, at any stage during the Delphi, about a particular result, the whole iterative process producing that particular result can be examined in detail. By contrast, minutes of committees are not structurally as rigorous 82
as reports on the succeeding rounds of a Delphi and often not as consistent in the use of language. Furthermore, the Delphi procedure presents a statistical response which embraces the opinions of all members of the panel rather than just representing majority opinion. Finally, Delphi is superior to committees or other groups because its results tend to be less open to misinterpretation. Very often the minutes of a meeting will record dissenting opinions only in form rather than substance indicating disenchantment, etc., leading the reader to form his own opinions, secondhand. Delphi information is structural in its questionnaire format, the results are tabulated with the response rate clearly indicated for all participants without identifying the participants by name. The anonymity, lack of contamination of results, establishment of statistical probabilities of certain future events occurring, all contribute to a solid information base upon which to make decisions. The lack of these qualities in committees and other group structures contribute largely to what I have called 'misinformation'. The above arguments and contentions are sufficient, from my point of view, to experiment with the Delphi methodology in trying to focus on the urban-related issues which can assist the federal government in Canada in arriving at a national urban policy. Furthermore, the Ministry of State for Urban Affairs* is largely responsible for ensuring that policy planning is carried out, at the federal government level, dealing with most urbanrelated problems in Canada. The problem areas which I will concentrate on in this article are four in number (there are many others). They are: urban transportation; environmental considerations; urban fiscal policy or municipal finance; and food production, marketing and distribution in the cities. Urban transportation. There has been considerable controversy over freeways vs. urban rapid transit and how these modes should be paid for. The Spadina expressway in Toronto was stopped 18 months after building had started. Public opinion wavered backwards and forwards for over 2 years before the decision to stop the expressway was made. Many experts were called in, 2 to 12 months, before the decision was taken. Similar situations exist in other parts of the country at this time, e.g., Ottawa and Vancouver. My contention here is that if a 'blue ribbon panel' on
*The author was formerly Strategic Planner for the Ministry of State for Urban Affairs and much of the experience gained in that position assisted him in writing this article.
the future of urban transportation had been assembled in these three cases and a Delphi constructed on the pertinent issues, policy alternatives for decisions could have been arrived at in a logical, ordered manner instead of by committee hearings or other group gatherings as has been the rule particularly in the Spadina expressway issue, with all the disadvantages that those procedures encounter. An urban transportation Delphi could also deal with problems of downtown parking, emission controls, traffic patterns, etc. Port development and airport location sites have also been considered by some to be part and parcel of urban transportation problems. Delphi might assist in deciding whether or not these latter two issues are part of the overall urban transportation problem by structuring the Delphi appropriately to include these concepts. Urban environmental issues in Canada encompass everything from solid waste disposal to urban parks as well as whether or not there should be greenbelts or concrete playgrounds for children. A Delphi can be constructed so that policy guidelines can be forthcoming on the degree of built environment a city should have and when it should have it, i.e., in what time horizons? e.g., 1975-1980, 1980-1985, etc. In the broadest meaning of environment the Canadian media have even carried stories linking crime with high-rise living and blaming ghetto life and poverty for stress in urban life. All of these issues can be examined in a Delphi thus providing information on which to base policy plans and decisions. However, it is not the purpose of Delphi to make the actual decisions, but rather to assist decision-making by procuring essential information from the decisions. With regard to urban fiscal policy and municipal finance a Delphi would be particularly helpful because of the anonymity factor of the numerous participants and levels of government involved in an area requiring great delicateness and tact. This is an area of federal-provicincial relations that requires as near to 'complete' information (both constitutional and financial) as possible in order to be useful to all parties concerned. Governments, i.e., federal and provincial, wish to be on an equal level of negotiation in talking about fiscal matters, especially regarding the cities. Naturally, unconditional grants, such as those administered under the federal Municipal Grants Act which pay a municipality a grant-inlieu of real property taxes due on Crown property, are the most welcome. Other grants, however, are not quite as popular because of their quid pro quo nature. These issues, because of their fragility, may be better handled in a Delphi than through committee or other group structures.
LONG RANGE PLANNING
The problem of food production, marketing and distribution in large Canadian cities is a particularly acute one. In the fiscal year June 1972 to June 1973 food prices have risen by an average of 16.9 per cent across Canada, and somewhere in the vicinity of 27 per cent in the major cities of Vancouver, Montreal and Toronto. To gain insight as to why food prices are higher in the cities, I find it especially helpful to reflect on Jane Jacobs' thesis (as postulated in her book, The Economy of Cities) about agricultural productivity. Ms. Jacobs makes the point that most labour and capital specialization take place in the city and then are exported, through skilled individuals, to the surrounding environs including the rural areas. The demand for packaging, quality, etc. often comes from the city-dweller. This means that the cost of increased technology, particularly research and development costs, are often, in the first instance, borne by the city-dweller. Gradually the technology is transferred outside the city and 'improves' food production, marketing and distribution in other areas outside the city proper. The questions of: how much of the capital costs are borne by the city-dweller ? How much of the benefits of the total development are passed on at very marginal costs to rural people ?; and other related questions can be expeditiously and accurately dealt with by using a Delphi procedure. Since the problem area of food production, marketing and distribution has its effects on individuals' disposable income and life-styles it is a good potential use for the Delphi method. In Canada, two major studies sponsored by Bell Canada, the major telephone utility, have been conducted on medical technology and edu-
DECEMBER, 1974
cational technology. One of the authors, Frank Doyle, has had much experience in using Delphi on futures technologies and I think he is in agreement that for a complicated area like food technology, Delphi is a very useful tool. Now, I would like to turn my attention very briefly to the technical questions of how one might structure a Delphi on the subject areas I have dealt with so far. Let us start with the question of rounds. Although the original RAND Delphi started with 4 rounds, in retrospect it was agreed by the authors of that study that the fourth round did not serve much purpose in achieving either a better consensus than that which existed after the third round, nor was the quality of the responses improved by a fourth round. That is why I would propose that the 3 rounds of carefully prepared questionnaires are sufficient and it is my intention to use only 3 rounds in this proposed Delphi on the cities in Canada. The nature of the questions in the questionnaire is obviously critical to the quality of the response that one can expect to achieve in a Delphi procedure. I hope to achieve this quality by following a few simple guidelines, which are: (a) Define several broad areas of the study--the number of areas being dictated by the simplicity or complexity of the subject area under examination--in this case there are four areas. (b) Carefully research each of the broad areas before drafting the questions in the questionnaire. (c) 'Kid-test' the questionnaire on a few knowledgeable colleagues to 'get the
bugs out'. In other words, make sure the questions are not: ambiguous, overly simplistic and verbose. (d) Design a thorough system of analysis for the data collected after each round. (e) Having completed items (a) to (d) select carefully the 'blue ribbon panel'--this is a very critical step and as I have already mentioned can severely bias results if not pursued prudently. In the interest of fairnessi it should be pointed out that if the decision-makers are included in the 'blue ribbon panel' there is a tendency to what is called the 'selffulfilling prophecy'. This can be viewed as either advantageous Or disadvantageous. In my opinion, if you can attract the best decisionmakers the 'self-fulfilling prophecy' aspect is a price that I, for one, would be willing to pay for the other benefits that Delphi has over any other group response. (f) Finally, one must work out a schedule for each of the 3 rounds and try and stick to the schedule so as not to exasperate yourself or your 'blue ribbon panel'. The results of the Delphi I want to conduct should be of great assistance to policy developers in the field of urban affairs in Canada. The four subject areas will collectively throw some light on the direction in which a national urban policy should be moving. Normative statements of this kind are not easy to achieve and if the Delphi method can assist us in so doing, we should use this tool in achieving the goal.
83