Policy indicators: Links between social science and public debate

Policy indicators: Links between social science and public debate

Evaluation and Program Planning, Vol. 10, pp. 297-302, Printed in the USA. All rights reserved. 1987 Copyright 0 0149-7189187 1987 Pergamon $3.00 ...

607KB Sizes 3 Downloads 90 Views

Evaluation and Program Planning, Vol. 10, pp. 297-302, Printed in the USA. All rights reserved.

1987 Copyright

0

0149-7189187 1987 Pergamon

$3.00 + .OO Journals Ltd

BOOK REVIEWS Policy Indicators: Links Between Social Science and Public Debate by Duncan MacRae, Jr. Chapel Hill: The University of North Carolina Press, 1985, xvi + 414 pp. Reviewer: Alex C. Michalos tematic ethics, to provide both better education and more career opportunities for members of these technical communities. Practical questions relevant to public debate need to be given equal importance with the theoretical concerns of basic disciplines. More importance must be given to values, especially general end-values, and to knowledge that aids decisions affecting them. This knowledge must be cultivated within organized technical communities, inside or outside existing disciplines, and must be part of a citizen’s general education. (pp. 357-358)

This is the best general introduction to policy and social indicators that has ever been written. The range of issues presented and the sophistication of the discussions of them is truly remarkable. All serious scholars in the fields of policy analysis, political science, planning and development, and social indicators should read this book. Teachers of these subjects in search of a text for graduate and advanced undergraduate courses should also consider this book. The very richness of the analyses probably precludes anyone’s agreeing with everything the author has to say, but the ability of a book to elicit agreement is neither necessary nor sufficient for excellence. What is required is that readers are taken to new frontiers and are shown a new vision that can provoke progressive growth and development. MacRae has satisfied this requirement admirably. Hence, I hope that readers of the critical remarks to follow will not draw the unwarranted conclusion that the book is bad. The book is good, but, like a good book, the essence of a good critical review is that it also provides a new vision. After a brief summary of the contents of the book, I will proceed immediately to a critique of a few fundamental issues. MacRae regards this book as a “a continuation of [his] two earlier works: The Social Function of Social Science, and (with James Wilde) Policy Analysis for Public Decisions.” (p. xiii) He asks generally, “What sort of statistics should be provided to citizens for making policy judgments?” (p. xiii), and his general answer is “that a major emphasis in development of policy indicators should be to provide actual measures of notions of the public interest, in the form of end-value indicator statistics.” (p. xv) Near the very end of the volume he provides a succinct statement of

The full development of this argument involves a review of such issues as the nature and relation of statistics, democracy and values, experts and ordinary citizens, causal models, experimental versus nonexperi-

mental research, economics as a model for other disciplines, subjective versus objective well-being, equity, temporal discounting functions, social integration, policy implementation, and technical communities. Each of these topics receives detailed treatment from the point of view of finding indicators that are useful measures of progress toward the American Constitutional goals of promoting the general welfare and establishing justice. For example, he approaches the technical topic of causal models with the question: How can causal models “best be developed to aid our choices of policies and indicators?” (p. 81). His answers include recommendations like the “development of models in particular social goal areas and lower priority to overall social system models” and the use of indicator concepts involving ethical values “as the dependent variables in policy models” (p. 82). To take another example, his review of some nonexperimental social policy research leads to such recommendations as that “In order to guide intelligent public discussion, communities of policy analysts might center their work about the choices available for policy and a broader range of criteria that need to be brought to bear on them. . . .The empirical and atheoretical nature of these studies to be supplemented not only by the framework of a general policy-related model, but also by the theoretical contributions of the various disciplines relevant to that model” (pp. 128-129). Finally, he offers the following

The major argument of this book: that expert communities as presently constituted are inadequate for guiding the design and use of policy indicators. Basic scientific communities have important contributions to make to policy models; but technical communities should be developed on

an equal footing with them. Institutional innovations are thus required within the universities, emphasizing the decision sciences and professional schools together with sys297

298

BOOK REVIEWS

good advice to those who want “to provide guidance” to the developers of policy indicators: The work “is both political and technical at the same time”; “A period of decades may be required to institutionalize an indicator at the national level.“; “In government decision making about indicators, the valuative aspect of an indicator variable is often subordinated to consensus and continuity.“; and “Different political communities, such as states and cities, have different goals and means of realizing them, and thus different information needs.” (p. 324). Broadly speaking, MacRae’s ideas about democracy, responsible citizenship and scientific research strike me as liberal and progressive. However, a more detailed examination suggests that the positions taken are so liberal that they cannot be very progressive. For example, the author asserts that there are scientific and technical communities, and that the former deal with basic or applied science while the latter deal with the attainment of social goals. The principles of division are apparently intentional in the sense that basic scientists intend to pursue knowledge for its own sake, applied scientists intend to apply science to serve any ends whatsoever, and experts in technical communities intend to solve social problems and pursue social goals (pp. 338346). But these intentions are only reasonable if one assumes a fairly traditional and positivistic view regarding the logical and ontological distinction between facts and values, one assumes that what might be logically possible is psychologically, sociologically, and historically not only possible but actually the case, or one assumes that good citizenship is consistent with being a hired gun. I think MacRae makes all three assumptions and that all three are mistaken. There are implicit and explicit references to these assumptions throughout the book. For example, he “reminds us that an information system is not simply good or bad in itself but must be judged in terms of its use” (p. lo), he refers to choices “based on valuation rather than science” (p. 228), “categories of law and ethics more than of social science” (p. 245), “in a more consensual scientific role [social scientists] may try to restrict their participation to presenting ‘facts”’ (p. 2491, “The choice of policy indicators is a politicai question and not simply a scientific one.” (p. 293), “The reconciliation of these two types of criteria of work-scientific rigor and significance, on the one hand, and valuation and policy relevance, on the other-is the problem of this chapter.” (p. 332), basic sciences are value-free where “To be value-free does not mean ignoring the internal scientific values of empirical verification, logical rigor, and generality, or the norms of the scientific community concerning modes of publication and criticism. It does require that scientists ignore the effects of their discoveries outside the realm of science (such as the changes in world-view occasioned by Copernicus, Newton, or

Darwin), and the effects of application of basic science (such as the development of new goods or weapons) . . . In basic social science, to be value-free means to avoid explicit espousal of social or political values in one’s contributions to the scientific community. Implicit incorporation of values in ‘secondarily evaluative terms,’ however, is common.” (pp. 346-347). I have argued against the fact-value distinction on theoretical and practical grounds in Michalos (1980a, 1986), explained the evaluative and ethical implications of allegedly purely scientific notions of such things as a scientific explanation in Michalos (1980b), explained why social reports, which are information systems, cannot be neutral in Michalos (198Oc), argued against the basic-applied science distinction in Michalos (1981) and in favour of the moral appraisal of scientific theories in Michalos (1978). Rather than repeat myself here, I refer readers to those studies. For good sociological and historical accounts of real scientific researchers versus the mythical schotars of too many textbooks, one might have a look at the excellent reviews in Durbin (1980) and Cohen (1981). In case anyone still believes that Darwin’s theory was nol mainly a reflection of a widely shared sexist, racist, capitalist, and metaphysically Hobbesian competitivist worldview, I would strongly recommend Bleier (I984), and Lewontin, Rose, and Kamin (1984). In the light of these sorts of critiques of MacRae’s fairly generous views of these issues, I think it is fair to say that the latter are not as progressive as one might have expected them to be. Although the author says that “From the viewpoint of values and citizenship, a new element in the present book is its concern with reconciliation of values in democracies.” (p. xiv}, he never explains how that reconciiiation is going to be accomphshed. It is clear, as aiready indicated, that he is committed to public debate by well-informed citizens aided by technical communities. But these commitments are not translated into concrete machinery for reconciling conflicting values. Indeed, I believe his last word on such reconciliation is virtually the same as his first, namely, there must be broad-based, informed public discussion in a democratic forum. Furthermore, I suspect that the main reason he could not get beyond this point is that he lacks a general theory of value. His discussions of value and values frequently involve distinctions between means and ends, instrumental and intrinsic, contributory and end-values. As one would expect, end-values are valued for their own sake and instrumental values are valued as means to obtaining the former (pp. 6-7, 32-33). But we are never told what “value” means, what, if anything, value is, or what, if anything, makes something more or less valuable than something else. He explicitly asserts that “valuative discourse is meaningful” (p. xiv), but his treatment of it is consistent with noncognitive views. As explained in

BOOK REVIEWS

Michalos (1986), noncognitivists (emotivists or prescriptivists) believe that sentences involving evaluative terms are meaningful, but they do not believe that such sentences are descriptive, declarative or assertive. The emotivists think that sentences which are formally declarative and apparently affirming some property of some subject are functionally exclamations, and the prescriptivists think that the formally declarative sentences are functionally imperatives. Exclamations and imperatives are meaningful. If someone cries “Ouch!” or says “Close the door,” we all know what is meant; but we also know that such sentences are not descriptions of anything and logically cannot be true or false. When positivistic noncognitivists assert that questions of value must be answered by political debate rather than by scientific research, they are affirming their belief that there is nothing, no thing, in virtue of which valued things have value. They admit that different people value different things; so they grant that there are valued things. But they deny that there is any property or set of properties in things in virtue of which they may correctly or incorrectly be said to be valuable or to have value. Since there are no such properties, there are none to be investigated scientifically. All one can do is discuss them and try to reach agreement about them in a democratic forum. I think that is what is behind MacRae’s views on the difference between evaluation and science, and behind the absence of even a suggestion of a general theory of value. Some attention to the great naturalistic (but incomplete) general theories of value of Perry (1926) or Lewis (1946) might have strengthened this part of policy indicators. The idea of a policy indicator is so loosely formulated that virtually any indicator can be or not be a policy indicator. The author says in a footnote that “We shall define policy indicators. . . as measures of those variables that are to be included in a broadly policy-relevant system of public statistics” (p. 5). Again, “A policy indicator is a measure on which we can justifiably base statistics that are included in the policyrelated part of a public statistical system; and a social indicator is a measure of social conditions that can be justifiably used in the system” (p. 31). The operative terms in these definitions are “broadly policy-relevant” and “policy-related part.” Policy indicators are any indicators that anyone can plausibly relate or find relevant to any public policy. So what is or is not a policy indicator depends to a large extent on the wits or imagination of anyone who wants to use the term to designate any particular indicator. Although the definition of a policy indicator will not allow one to designate a particular subset of social indicators as always and everywhere policy indicators, the definition seemed acceptable until I reached MacRae’s critique of the three Coleman reports on education (pp. 118-131). In the presence of that critique, the deficiency

299

in MacRae’s definition became clear. He claims that “Coleman’s 1966 report did not. . . deal directly with policy alternatives,” and that “teacher characteristics are not themselves a direct policy variable any more than student body characteristics; the relevant policy variables are policies for educating, attracting, and retaining teachers who have the desired characteristics” (p. 122). The key words in the quoted passages are “deal directly, ” “direct policy variable,” and “relevant policy variables.” According to MacRae’s definition, teacher and student body characteristics are certainly policy indicators. They can easily be related or relevant to a variety of public policies, and they are typically included in national statistical reports of education in the USA. So Coleman’s reports concerned policy indicators, but not the right ones. The right policy indicators would have been somehow more directly related or more relevant to some specified policy. But Coleman’s study was obviously undertaken in the interest of developing policies related to the racial mix in America’s schools. The Supreme Court’s 1954 decision implied that the schools ought to be desegregated. Whether one thought that desegregation was good for the sake of fairness or for the sake of pedagogy, or both, anyone responsible for education in America would have been interested in knowing exactly what consequences the Court’s decision actually had on the educational achievement of those most affected by the decision. After all, such achievement is still one of the most important functions of schools. Coleman’s first aim was to find out which input variables had what kind of impact on student achievement. That was an important aim, and the information was necessary before one could plan any kind of intervention in the system. MacRae’s objection seems to be that a broader model, with a more judicious selection of policy indicators, might have spared Coleman and others the problems following the 1966 report (p. 126). In particular, he thinks we might have been able to predict and prevent the “white flight” from Court-enforced desegregation. (pp. 124-127) But practically everyone predicted the “white flight,” and with good reason. The history of residential “block busting” in America is a history of “white flight,” and virtually all Americans are aware of that history. Besides, what typically happened with housing happened everywhere else in the country. When a certain racial mix was reached in parks, pools, theatres, etc., the facilities rapidly became black people’s parks, pools, and so on. Social scientists had even carefully documented the crucial percentages tipping the scales one way or another. Of course we “might” have had even better information. But what more could anyone have wanted and what difference would it have made? Like MacRae and other concerned citizen-scholars, I agree that we must continue to press for better

300

BOOK REVIEWS

information and better decision making in the light of that information, but I do not think his critique of the Coleman reports provides much support for our case. On the contrary, I think the story of the fate of those reports probably undermines our position. There are definite limits to what anyone can do about human cussedness. The author’s definition of a “social problem’” is “a contrast between an observed state of affairs and a valued expectation shared by a group (p. 21). In Michalos (1980d) I reviewed some of the literature regarding perceived gaps between what one has and expects versus perceived gaps between what one has and wants to have. Because the latter perceived aspiration-achievement discrepancy is a very powerful predictor of satisfaction and happiness, while the former perceived expectation-achievement discrepancy is a very weak predictor, reference to the aspiration-achievement gap in a definition of a “social problem” would be preferable. A “valued expectation” is not a want or even a need, and it is wants in particular that are required for the identification of problems. For example, the facts that one may be poor, ignorant and fat, and hate one’s condition will not be perceived as a problem until one wants to get from where one is to somewhere else. Philosophers and psychologists traditionally have distinguished cognition, from conation and affect, and although they are related in a variety of ways, logically and psychoIogicaIIy one can not get desire simply out of knowledge and feeling. That is one reason why some people insist that human agency is not reducible to material causality. Granted that there are some fundamental philosophical issues involved here and that one should therefore try to remain open-minded and care-

fully critical regarding one’s views, it is nevertheless important for policy analysts not to neglect voluntarism, wants, and aspirations. Whatever they are, they are fundamental features of human beings. Since I have spent the past decade or so developing a socio-psychologic~ theory of satisfaction and happiness to be used in social indicators or quality of life research (e.g., see Michalos, 1985), I was particularly interested in MacRae’s discussions of subjective and objective welfare, and subjective and objective indicators. He recommends the construction of causal models and theories that have human well-being (measured by objective and subjective indicators) as their output. (pp. 190-218) He assumes that there is some kind of a real world beyond this or that person’s perceptions which has reciprocal relations with the latter, and adequate models should accommodate these assumptions. I agree with all of this and the theory that I have developed, called “multiple discrepancies theory” or “MDT” for short, does accommodate these assumptions. It includes variables related to wants, needs, equity, expectations, social comparisons, self-esteem, social support, income, and some other variables that have been shown to be relevant to perceived well-being. The theory has been fairly successful in tests in Ontario the past few years, and is currently being tested with good results in 43 countries involving about 20,000 university undergraduates in 68 universities, speaking at least 22 different languages. Having reached an obvious thin end of the wedge leading to a lengthy discourse on my own research, I think it is time to stop. MacRae’s book is a mine of provocative ideas, and I strongly recommend it even though I do not agree with everything in it.

REFERENCES BLEIER, R. (1984). Science and gender: A critique of biology and ifs theories on women. New York: Pergamon Press.

science, technology, and medicine (pp. 197-281). New York: The Free Press.

COHEN, R. S. (Ed.). (1981). Thesocial implications of !he scientific and technological revolution: A Unesca symposium. Paris: Unesco.

MICHALOS, A. C. (198Oc). North American social report, VoL I. Foundations, population and health. Dordrecht: D. Reidel Pub. Co.

DURBIN, P. T. (Ed.). (1980). A guide to the culture of science, rechnology, and medicine. New York: The Free Press. LEWIS, C. 1. (1946). An analysis of knowledge and va~ua~jon. LaSalle: The Open Court Publishing Co. LEWONTIN, R. C., ROSE, S., & KAMIN L. J. (1984). Not in our genes. New York: Pantheon Books. MICHALOS, A. C. (1978). Foundations of decision making. Ottawa: Canadian Library of Philosophy.

MICHALOS, A. C. (198Gd). Satisfaction and happiness. Social fndicafors Research, 8, 385-422. MICHALOS, A. C. (1981). North American sociaf report, Vol. 3: Science, educufion and recreation. Dordrecht: D. Reidel Pub. Co. MICHALOS, A. C. (1985). Multiple discrepancies theory (MDT). Social Indicators Research, 16, 347-413.

MICHALOS, A. C. (198Oa). A reconsideration of the idea of a science court. Research in Philosophy and Technology, 3, 10-28.

MICHALOS, A. C. (1986). Technology assessment, facts and values, In W. N. Dunn (Ed.), Policy anaiysis: Perspectives, concepts and merhods (pp. 69-92). Greenwich: JAI Press Inc.

MICHALOS, A. C. (198Ob). Philosophy of science: Historical, social and value aspects. In P. T. Durbin (Ed.), A guide to the culfure of

PERRY R. B. (1926). General theory of value. Cambridge: Harvard University Press.