Program evaluation in education: When? How? To what ends?

Book this book is woefully inadequate. Similarly. the justification for government finance on the basis of spillovers is mentioned but not developed. For instance. there is no reference to Peston’s (1966) article. the theory of spillovers Jnd its connection with education. Also missing are references to Blaug‘s (1970) discussion of spillovers or the rather full bibliography provided by Cohn (1979). Probably, the discussion of the financing dilemma could have been enhanced by a complete examination of the question of tinancing education through property taxes. Some of the issues are highlighted but not explored. and the work would have been more helpful from a public policy view had the exploration occurred. It is offputting to read a book where the background, .tcadrmic qualification or affiliations of the author are not conveyed to the reader. Furthermore. the book lacks a bibliography and index. both of which arc sertous flaws. On the other hand, the type is large and easily read and the footnotes are on the bottom of the pages, which is highly .tppropriate. There is not a great deal in this book to warrant a recommendation to the readers of this journal that the book be read. On the other hand, for those who wish a basic understanding of the current financing of education III Canada, this is about as clear a book as there can be for that purpose. Thus. this reviewer feels that the effort and re:ources spent in compiling the book are not wasted. DOUGLAS J. MCCREADY

Wilfrid Laurier University

REFERENCES (1970) An fnrrodaclio,l to (he Economics of London: Penguin. CottN. E. (1979) The Economics of Educarlon. Cambridge, MA: Ballinger. I’ESTOS. XI. (1966) The theory of spillovers and its connection with education. Publ. Fin. XXI, 181-205. ~~LALIG, hf.

Educafion.

Program Evaluation in Education: When? How? To What Ends? Edited by SENTA A. KAIZEN and PETER H. ROSSI. Washington, DC: National Academy Press, 1981. pp. xi + 276. $8.00 (paper). WHILE the explicit topic of this book is evaluation in education, much of it is relevant to the larger field of evaluation across numerous programs. Evaluation is a field of endeavor beset by a multitude of conceptions and purposes. We evaluate programs all the time, mostly informally in our roles as citizens, program clients, and organization participants. Problems arise when we seek to elevate the art to a professional endeavor as practitioners or clients of evaluation. The problems are even more serious when we seek to assess evaluation as a topic of social scientific analysis. This volume comes from a unit in the Washington oftialdom with a three-line address: the Commiffee on

Reviews

163

Program Evaluation in Education of the Assembly of Behavioral and Social Sciences of the Narronal Research Council. It is tempting to blame the commtttee character of the authorship for the instances of contorted prose and elegant efforts to balance issues that are essentially irreconcilable. In fairness to the countless people who contributed to this volume, however. the faults just as clearly lie in the muddy field they are trying to clarify.. lIthere is a whole chapter devoted to the issue of defmmg evaluation. In an appendix, one contributor expresses what is by then a reader’s feeling, that there is no commonly accepted definition (p. 218). That wahich cannot be defined clearly cannot be measured without equivocation. Two irreconcilable estimates of the problem’s federal investment in evaluation of size are: “current education programs totals some $40 million a year, not including federal funds spent for evaluation at the state and and “about $100 million in local levels” (pp. 26-27). federal funds were spent for education evaluation in 1977 by extramural performers” (p. 223). If evaluation cannot be defined or measured precisely, it is helpful to describe and assess different kinds of evaluation. One table (p. -!I) portrays 10 varieties. The editors strive to elicit clarity from their analysis of evaluation, and to enchance practice by recommendations to Congress and to the Department of Education. If their recommendations cannot succeed beyond the point of modest improvements, their own analysis offers the explanations. Politics is one problem of evaluation. Policy makers are led to exaggerate and to obfuscate program goals. They exaggerate in order to recruit supporters for their proposals. They obfuscate in order to confuse or to buy off opponents and doubters. Both exaggeration and obfuscation confound evaluators by tampering with one of their basic standards: the goals that are said (often with tongue in cheek) to guide program administrators. Also. policy makers have discovered that evaluation has manipulative uses. The concept of evaluation that is misused pertains to work that is intentionally manipulated to show a program in a good-or bad - light and to work that produces data of no apparent utility to those who would judge program activities. The variety of potential audiences for evaluation is another problem. In the case of education, evaluations can be directed at the different perspectives of the U.S. Department of Education, members of Congress and their staffs, education agencies of state and local governments, teachers, students, parents and local taxpayers. plus the various interest groups with a concern for the contents of educational programs. It is “baffling and unrealistic” (p. 38) to expect evaluators to speak to the needs of all these audiences. Other problems derive from the federal nature of American government, and the traditional emphasis on local control of education. On the one side are pressures to designate evaluation goals and formats centrally, in order to obtain nationally comparable data and to control the quality of the work done. On the other side are the pressures for local authorities to evaluate programs according to their own needs. Related to the issue of centralization vs decentralization are disputes between standardiz-

164

Econonzics

of Edrccatiorz Rer,ierr

atlon of evaluation vs flexible procedures to favor inno\ation. Related to all of this is the issue of who does the evaluation. A great deal of federally funded evaluation IS done by a handful of large contractors (e.g. Rand. Abt Associates. Applied klanagsment Sciences. ETS). They can meet the increasingly complex btddins forms and reporting requirements created by the Department of Education in the interests of standardization and quallty control. Left out by the ueight of administrative detail are indlwduals or small groups of academic researchers and minoritv firms who might bring to evaluation a variety of psrspeciives and provide avenues to innovation. The contributors return several times to the issue of the disadvantaged. hlinorities and the disabled make up a disproportionately large share of the beneficiaries of federally funded programs of education. but are greatI! unrepresented among professional evaluators. The volume includes 3 recommendations. and a great deal of prose to explicate them. Occasionally the contributors cannot extricate themselves from the problems of evaluation while formulating their recommend Itions. The volume urges flexible procedures to allow the easier cntr) of evaluators with different perspectives. Yet lhe recommendations also include several points that wou d increase the participants in decision-making about evalu.ition. add reporting requirements and lengthen the stages of preevaluation plannin_e and post-evaluation respclnses. The contributors recogntze what has happened to tnem. They arc caught by the tangle of contrary prrssurcs that the) describe so aptly. Their readers must rely on .I knowledge of particular ctrcumstances and good judgement to decide exactly where to put the emphasis. Thus, the value of this volume may rest in its charting of the complexities of education evaluation, even more than the specifics of its recommendations. IRA SHARKANSKY

Hebrew’ University of Jerzualanz

Perspectives in State School Support Programs. Edited by K. FORBIS JORDAN and NELDA H. CAMBRON-MCCABE. Cambridge, MA: Ballinger, 1981. pp. 372. No price listed (cloth). ,4s A result

of t& school finance reform efforts of the 1970s. numerous states enacted legislation that attempted to provide more equitable school finance programs. Following the phenomenal strides of this era, the challenge of the 1980s appears to be that of refining these state support programs. Where once the pressures were for equalization of expenditures and property tax relief. pressures are currently mounting for recognition in the state school support program of the educational needs of special student populations and differences among districts in the cost of providing equivalent educational programs and services. In addition. efforts are being made to develop better measures of local school district fiscal capacity and to ensure adequate fiscal and educational planning in local school districts.

In this book. the second annual yearbook of the American Education Finance Association. the editors have compiled a collection of essays which focus on the emerging problems. considerations and components involved in refining school support programs. The Introduction to the volume is written by Roe L. Johns. professor emeritus at the University of Florida and first president of the American Education Finance Association. Johns presents a historical perspective for viewing the remainder of the volume by tracing the historical development of state school support programs and placing current concepts within the context of this hlstorical development. Part I contains five chapters that focus on educational needs and fiscal capacity. In Chapter I. Michael Hedge presents an ovewiew of the need in the areas of special education. compensatory education and bilingual-bicultural education. and reviews existing state programs which address these special populations. Some of the potential fiscal and programmatic effects of the various state-funding formulae are discussed. In the second chapter. Jay Chambers provides a comprehensive discussion of both the theoretical and empirical aspects of education cost and price level adjustments to state aid programs. Primary attention is given to the methodolo+ll and procedural issues involved in developing a cost-of-education index. Contrary to many who conceptualize the reform of school finance systems and the adjustment for special programs and populations independently of one another. Chambers recognizes and emphasizes the importance of integrating any adjustments of the state aid formula for the cost of school resources with adjustments being considered for such special populations as those discussed in the preceding chapter. The third chapter deals with the concern of the large city interveners in Lerirrorrn v. N~~ubr - municipal overburden. Municipal overburden refers to the fiscal stress experienced by large cities in attempting to meet the high demands for non-education services. These high noneducation burdens mean that education must compete with many other areas for fiscal resources. This factor, together with the high concentration of special populations which are normally found in central cities, results in these districts being at a fiscal disadvantage in supporting school services. The author, Jane Sjogren, discusses many of the dimensions of municipal overburden, including empirical attempts to measure municipal overburden. The weakness of the treatment lies in the paucity of information related to state formula provisions for municipal overburden. Chapter 4 focuses on school finance reform and the cities. Dealing with many of the same issues as discussed in the preceding chapter. Margaret Goertz examines the manner in which school finance reform has affected the inner cities and identifies a series of problems or issues that should be addressed by state school finance fiscal planners. The final chapter in Part I, by E. Kathleen Adams and Allan Odden, discusses alternative school district wealth measures that would broaden the concept of school district capacity beyond simply a measure of property wealth per pupil. They also discuss ways such measures might be used in the allocation of state school support. Part II, current state school support programs. includes three chapters that cover various aspects of current state