- Email: [email protected]
Cost-benefit analysis: theory and application
Book Reviews
288
evaluation literature, specifically on issues that concern the utility of project findings and their potential to inform program decision making. Part IV, the final section of Kvale’s book, is “apologetic” in the philosophical sense of that term. Here Kvale first raises and then attempts to answer a series of ten claims that interview research is not scientific, objective, trustworthy, generalizable, and so forth. Whether readers are willing to accept the answers intended to counter such claims will depend largely on their willingness to accept Kvale’s own definitions for terms such as “scientific,” “objective,” and “trustworthy.” In this concluding section, Kvale also restates his belief that: “The very strength of the interview is its privileged access to the common understanding of subjects, the understanding that provides their worldview and the basis for their actions” (p. 291). Perhaps this overstates my earlier point that the limitations of interview data can often be turned to one’s advantage. Granted, interviewing is not redundant to other methods of gathering information. To claim “privileged access,” however, unnecessarily implies a type of methodological favoritism. Interview researchers should be cautious of such favoritism because they themselves have suffered under the privileged status of other forms of research. This brief overview still leaves the questions of how this book contributes to cross-disciplinary fields or research, and for whom the book is most useful. One of the most distinctive features of Znnterk’iews is its effort to fully integrate what Kvale describes as “the practical issues of the interview method and the theoretical issues of the nature of interview knowledge” (p. 14). Kvale
Cost-Benefit Analysis: Theory and Application, $42.00. Paper $18.95. _ -Reviewer: Henry
not only takes recent theoretical debates seriously, but also struggles to specify what these debates mean for methodological practice. In this sense, Inter Views can be read pragmatically as a type of experiment in thinking about research. That is, if we conceptualize an interview as an inter view,- a perspective co-constructured between two or more peoplewhat difference does this postmodern understanding make in how researchers collect, analyze, and report interview data? In other words, does this understanding lead to better research? If the proof is in the pudding, it seems far from conclusive whether Kvale’s research is any better or any worse than interview studies that stem from our more conventional views of social science. This conclusion, however, is not entirely fair because Kvale’s efforts to integrate theory and practice are impeded by the broad and diffuse nature of postmodern conceptions of knowledge. In this respect, the epistemological side of Inter Views will limit its appeal for those who are specifically interested in learning how to do interview research. Grand theories are useful for the range of purposes (e.g., as perspectives from which to critique or organize a body of knowledge). Methodological decisions require a more middle-range level of thinking. For this reason, an evaluator’s love of theory is probably inclined to follow the aesthetic principle that “less is more.” In support of a research tradition, we look for well-developed lines of reasoning rather than the broad surveys of philosophical thought that now seem in fashion. Yet, readers still have much to gain from Interviews in the way of practical lessons, most of which are accessible regardless of one’s faith (or lack of faith) in postmodern conceptions of inquiry.
by Tevfik F. Nas. Thousand _
M. Levin
Evaluators and decision-makers are often faced with program alternatives among which they must recommend or choose for policy. Cost-benefit analysis and its close relation, cost-effectiveness analysis, represent evaluation tools designed to help make economic choices by comparing the outcomes for each alternative with its costs. When both the values of costs and benefits can be assessed in terms of their monetary magnitudes, costs can be compared directly with benefits for each alternative. In general, one would not consider an alternative unless the benefits were greater than the costs and would only choose the alternative whose benefits exceeded the costs by the widest margin.
Oaks, CA: Sage, 1996, 220 pp., HC
While this all makes common sense, evaluators encounter serious problems in using the tool. Often, the benefits and costs of an alternative are not easily measured and can assume widely different values under a range of plausible assumptions. Further, even with good measurements, there are alternative ways of comparing benefits and costs, sometimes leading to different rankings among alternatives. Far from following a recipe for a cost-benefit study as one might for a variety of standard statistical techniques, the evaluator must follow a set of steps in analyzing the problem and implementing the technique, each requiring fairly sophisticated judgments. It is little wonder that cost-benefit analysis is more often found in evaluation texts and among the parlance of evaluators in terms of its hortatory importance rather
Book Reviews
than its actual use. The use of cost-benefit analysis presents a formidable challenge to the evaluator because of the technical background in economics that is necessary to master and apply it. It is also a difficult tool for the intelligent decision-maker to understand in terms of its arcane procedures and ways of doing things. Costs are measured for purposes of cost-benefit analysis (CBA) in ways that are often quite different than the conventional way that laypersons and administrators think of them. For example, even when resources are donated to a project, they are considered to have a social cost. Further, costs determined by standard accounting procedures and project budgets may overstate or understate the true economic costs. The calculation of benefits is even harder to understand for the uninitiated where even civic behaviour or an aesthetic vista or the option of using a public facility in the future may be given benefit values in pecuniary terms. Finally, such matters as discount rates are not only difficult for the noneconomist to understand, but they are the stealth calculation in the benefit-cost equation. Even with the best cost and benefit measures, subtle differences in discount rates can transform dramatically the cost-benefit rankings among alternatives. This is the challenging context in which we must consider this volume on cost-benefit analysis by Tevfik F. Nas. Nas has written this well-organized book to appeal to two audiences, those with an undergraduate foundation in economics and those with some analytic skills and interest in public administration and public policy who lack the economics. In order to accommodate the latter, he has provided several short chapters (about 50 pages in total). Even though these are written at a very basic level, my guess is that they are too compacted to be accessible to most persons without economics training. In a few cases, the treatment of such sophisticated concepts as Pareto efficiency is handled so glibly that inadvertent errors are introduced: For example, if a resource allocation is such that no additional investment in information infrastructure is possible without reducing investment in other areas, such as defense, education, or health care, then Pareto efficiency has been attained (p. 11). This is obviously incorrect since the marginal social benefits of reallocation to one sector from another can exceed the loss or sacrifice of marginal social benefits from the sector where the resources are taken. Shifting resources from one area to another may lead more closely to a Pareto-efficient allocation rather than detracting from it. Following this introductory section, the book provides a series of brief chapters on cost-benefit analysis. The thrust of the presentation is weighted towards listing concepts that are used in cost-benefit analysis and
289
providing a brief description of each rather than integrating them into detailed applications. The focus is on steering a midcourse between rigorous definitions and theory and applied attempts to measure costs and benefits in the real world. Thus, those with a penchant for theoretical understanding and those who seek to understand how to apply the concepts may both feel hungry for more nourishment. Of course, this is a decision that any author must make in an area that has dimensions of technical complexity and practical applications. Occasionally, there is a very brief review of an article using cost-benefit analysis, but no detailed presentation on how one might conduct a study. In this respect the work reads more like an undergraduate textbook designed to meet a course requirement than an attempt to teach readers how to apply the concepts. This challenge might be overcome if the instructor complements the use of this text by providing guided projects and detailed case studies to develop proficiency in the use of the tool. But, by avoiding both abstract theory and applications, some of the concepts that are presented may be difficult to fathom for the uninitiated reader. For example, Chapter 9 addresses “The Choice of Discount Rate” and shows the formula for calculating net present value. But, nowhere in the chapter is there an adequate exposition for those without economics training of the rationale for and explanation of the discount rate and net present value. A discount rate is used to equate a given magnitude of benefits or costs where derived in the future with a similar magnitude when derived at present. It is based upon the well-known fact that individuals and societies are not indifferent about the time pattern over which they receive benefits or incur costs. A given quantity of benefits received at present (e.g., income) has greater value than when deferred to the future. Costs when deferred to the future represent less sacrifice than when incurred at present. The discount rate is used to reduce the “value” of future costs and benefits versus present ones to reflect this time preference so that all alternatives are compared in terms of their “net present values”. Although the chapter points out considerations in choosing discount rates and the different discount rates used, it does not stipulate the profound consequences of choosing different rates. In one paragraph on pages 142-143 the author identifies rates used by different government agencies that vary from 2.1 to 7%. Differences of this magnitude can vastly change both the calculated profitability and ranking of potential alternatives. Indeed, by using a low discount rate, advocates of particular projects can and have made it appear that benefits greatly exceed costs of alternatives, when the same projects look like losing propositions with a more acceptable, but higher, discount rate. The author might have demonstrated to the reader
290
Book Reviews
how the choice of discount rate can have a profound affect on the estimated profitability and ranking of projects, even overwhelming other benefit and cost calculations. This could be used to caution readers on how progenitors of projects have used unduly low interest rates to justify projects that were not economically viable when assessed by more reasonable calculations. Indeed, this would have been an ideal opportunity to show the vulnerability of cost-benefit analysis and reveal its susceptibility to the “political arithmetic” of project advocates in a way that is unlikely to be detected by decision-makers who lack technical knowledge on the approach. This represents the most vulnerable aspect of the book for those who will rely upon it to understand costbenefit analysis. By not developing applications from the ground up and reviewing the judgments that go into the myriad decisions at each stage, the reader will not gain a full appreciation of both the promise and limitations of cost-benefit analysis. Such detailed applications can examine the robustness of cost-benefit estimates and rankings of alternatives under different assumptions governing definitions and measurements of costs and benefits and different discount rates. In fact, though the book mentions doing a sensitivity analysis to
Evaluation Strategies for Communicating Hallie S. Preskill, and Mary E. Piontek,
get lower and upper boundaries of costs and benefits under different assumptions, it never applies this lesson. It is this need to walk the reader through detailed applications of the cost-benefit methodology that is sorely missing from the book. Although the last chapters try to apply some of the concepts to the environment, health, and developing countries, details are sketchy. As an introduction to the concepts underlying costbenefit analysis, this volume is well-written and comprehensive for a book of its length. It could be used profitably in an undergraduate course or in courses on evaluation where the students have at least some understanding of economics, if supplemented by detailed case studies. If accompanied by studies in applied social areas (health, education, environment, criminal justice, and so on), it would allow the student to participate in grappling with the various steps and judgments used for deriving cost-benefit calculations as well as implementing sensitivity analyses of the results under different, but plausible, assumptions. This type of hands-on activity will enable the student to obtain an understanding of the cost-benefit tool as a consumer or user of such studies in conjunction with the treatment provided by this textbook.
and Reporting: Enhancing Learning in Organizations, Thousand Oaks, CA: Sage, 1996, 274 pp.
Reviewer: Lyn M. Shulha At last! A practical guide that puts learning at the heart of the reporting enterprise and makes reporting a central element in evaluation design. The authors’ premise is that reporting can make significant contributions to organizational learning; that is, learning that occurs when “organizational members are developing new understandings and perspectives on themselves as they seek to improve” (p. 30). For this to happen, they argue, information concerning evaluation processes and findings must get infused throughout the organization in ways that will promote collective reflective practice. This complex process requires considered attention to the activities of communicating and reporting. Several features set this work apart from traditional discussions on reporting. First, as a preliminary step to writing the book, the authors surveyed the membership of the American Evaluation Association’ (AEA) about ’ The experiences of AEA members span the fields of education, social programming, policy development, international development and private consulting.
by Rosalie T. Torres,
their experiences and attitudes related to reporting. A summary of their findings, presented early, allows the reader to compare her/his experiences in this area with those of the broader professional community. Those of us who are only moderately satisfied by the effectiveness of our communicating and reporting plans, who feel challenged and often constrained by the complexity of each new evaluation context, who have serious doubts about the utility of summative documents we produce, but who still find themselves, usually due to time constraints, focusing on final reports with executive summaries, discover that we are in good company. For us, this book offers a fresh look at how to transform reporting from a symbolic act that denotes the conclusion of an evaluation, into an integrated plan to inject information at appropriate times both within and around an organization. The key, according to the authors, is not to focus on the techniques of information dissemination, but on the strategies and formats that will facilitate information processing given the program context and the target audience. This orientation is meant to help in the transformation of evaluative findings into functional knowledge.
288
evaluation literature, specifically on issues that concern the utility of project findings and their potential to inform program decision making. Part IV, the final section of Kvale’s book, is “apologetic” in the philosophical sense of that term. Here Kvale first raises and then attempts to answer a series of ten claims that interview research is not scientific, objective, trustworthy, generalizable, and so forth. Whether readers are willing to accept the answers intended to counter such claims will depend largely on their willingness to accept Kvale’s own definitions for terms such as “scientific,” “objective,” and “trustworthy.” In this concluding section, Kvale also restates his belief that: “The very strength of the interview is its privileged access to the common understanding of subjects, the understanding that provides their worldview and the basis for their actions” (p. 291). Perhaps this overstates my earlier point that the limitations of interview data can often be turned to one’s advantage. Granted, interviewing is not redundant to other methods of gathering information. To claim “privileged access,” however, unnecessarily implies a type of methodological favoritism. Interview researchers should be cautious of such favoritism because they themselves have suffered under the privileged status of other forms of research. This brief overview still leaves the questions of how this book contributes to cross-disciplinary fields or research, and for whom the book is most useful. One of the most distinctive features of Znnterk’iews is its effort to fully integrate what Kvale describes as “the practical issues of the interview method and the theoretical issues of the nature of interview knowledge” (p. 14). Kvale
Cost-Benefit Analysis: Theory and Application, $42.00. Paper $18.95. _ -Reviewer: Henry
not only takes recent theoretical debates seriously, but also struggles to specify what these debates mean for methodological practice. In this sense, Inter Views can be read pragmatically as a type of experiment in thinking about research. That is, if we conceptualize an interview as an inter view,- a perspective co-constructured between two or more peoplewhat difference does this postmodern understanding make in how researchers collect, analyze, and report interview data? In other words, does this understanding lead to better research? If the proof is in the pudding, it seems far from conclusive whether Kvale’s research is any better or any worse than interview studies that stem from our more conventional views of social science. This conclusion, however, is not entirely fair because Kvale’s efforts to integrate theory and practice are impeded by the broad and diffuse nature of postmodern conceptions of knowledge. In this respect, the epistemological side of Inter Views will limit its appeal for those who are specifically interested in learning how to do interview research. Grand theories are useful for the range of purposes (e.g., as perspectives from which to critique or organize a body of knowledge). Methodological decisions require a more middle-range level of thinking. For this reason, an evaluator’s love of theory is probably inclined to follow the aesthetic principle that “less is more.” In support of a research tradition, we look for well-developed lines of reasoning rather than the broad surveys of philosophical thought that now seem in fashion. Yet, readers still have much to gain from Interviews in the way of practical lessons, most of which are accessible regardless of one’s faith (or lack of faith) in postmodern conceptions of inquiry.
by Tevfik F. Nas. Thousand _
M. Levin
Evaluators and decision-makers are often faced with program alternatives among which they must recommend or choose for policy. Cost-benefit analysis and its close relation, cost-effectiveness analysis, represent evaluation tools designed to help make economic choices by comparing the outcomes for each alternative with its costs. When both the values of costs and benefits can be assessed in terms of their monetary magnitudes, costs can be compared directly with benefits for each alternative. In general, one would not consider an alternative unless the benefits were greater than the costs and would only choose the alternative whose benefits exceeded the costs by the widest margin.
Oaks, CA: Sage, 1996, 220 pp., HC
While this all makes common sense, evaluators encounter serious problems in using the tool. Often, the benefits and costs of an alternative are not easily measured and can assume widely different values under a range of plausible assumptions. Further, even with good measurements, there are alternative ways of comparing benefits and costs, sometimes leading to different rankings among alternatives. Far from following a recipe for a cost-benefit study as one might for a variety of standard statistical techniques, the evaluator must follow a set of steps in analyzing the problem and implementing the technique, each requiring fairly sophisticated judgments. It is little wonder that cost-benefit analysis is more often found in evaluation texts and among the parlance of evaluators in terms of its hortatory importance rather
Book Reviews
than its actual use. The use of cost-benefit analysis presents a formidable challenge to the evaluator because of the technical background in economics that is necessary to master and apply it. It is also a difficult tool for the intelligent decision-maker to understand in terms of its arcane procedures and ways of doing things. Costs are measured for purposes of cost-benefit analysis (CBA) in ways that are often quite different than the conventional way that laypersons and administrators think of them. For example, even when resources are donated to a project, they are considered to have a social cost. Further, costs determined by standard accounting procedures and project budgets may overstate or understate the true economic costs. The calculation of benefits is even harder to understand for the uninitiated where even civic behaviour or an aesthetic vista or the option of using a public facility in the future may be given benefit values in pecuniary terms. Finally, such matters as discount rates are not only difficult for the noneconomist to understand, but they are the stealth calculation in the benefit-cost equation. Even with the best cost and benefit measures, subtle differences in discount rates can transform dramatically the cost-benefit rankings among alternatives. This is the challenging context in which we must consider this volume on cost-benefit analysis by Tevfik F. Nas. Nas has written this well-organized book to appeal to two audiences, those with an undergraduate foundation in economics and those with some analytic skills and interest in public administration and public policy who lack the economics. In order to accommodate the latter, he has provided several short chapters (about 50 pages in total). Even though these are written at a very basic level, my guess is that they are too compacted to be accessible to most persons without economics training. In a few cases, the treatment of such sophisticated concepts as Pareto efficiency is handled so glibly that inadvertent errors are introduced: For example, if a resource allocation is such that no additional investment in information infrastructure is possible without reducing investment in other areas, such as defense, education, or health care, then Pareto efficiency has been attained (p. 11). This is obviously incorrect since the marginal social benefits of reallocation to one sector from another can exceed the loss or sacrifice of marginal social benefits from the sector where the resources are taken. Shifting resources from one area to another may lead more closely to a Pareto-efficient allocation rather than detracting from it. Following this introductory section, the book provides a series of brief chapters on cost-benefit analysis. The thrust of the presentation is weighted towards listing concepts that are used in cost-benefit analysis and
289
providing a brief description of each rather than integrating them into detailed applications. The focus is on steering a midcourse between rigorous definitions and theory and applied attempts to measure costs and benefits in the real world. Thus, those with a penchant for theoretical understanding and those who seek to understand how to apply the concepts may both feel hungry for more nourishment. Of course, this is a decision that any author must make in an area that has dimensions of technical complexity and practical applications. Occasionally, there is a very brief review of an article using cost-benefit analysis, but no detailed presentation on how one might conduct a study. In this respect the work reads more like an undergraduate textbook designed to meet a course requirement than an attempt to teach readers how to apply the concepts. This challenge might be overcome if the instructor complements the use of this text by providing guided projects and detailed case studies to develop proficiency in the use of the tool. But, by avoiding both abstract theory and applications, some of the concepts that are presented may be difficult to fathom for the uninitiated reader. For example, Chapter 9 addresses “The Choice of Discount Rate” and shows the formula for calculating net present value. But, nowhere in the chapter is there an adequate exposition for those without economics training of the rationale for and explanation of the discount rate and net present value. A discount rate is used to equate a given magnitude of benefits or costs where derived in the future with a similar magnitude when derived at present. It is based upon the well-known fact that individuals and societies are not indifferent about the time pattern over which they receive benefits or incur costs. A given quantity of benefits received at present (e.g., income) has greater value than when deferred to the future. Costs when deferred to the future represent less sacrifice than when incurred at present. The discount rate is used to reduce the “value” of future costs and benefits versus present ones to reflect this time preference so that all alternatives are compared in terms of their “net present values”. Although the chapter points out considerations in choosing discount rates and the different discount rates used, it does not stipulate the profound consequences of choosing different rates. In one paragraph on pages 142-143 the author identifies rates used by different government agencies that vary from 2.1 to 7%. Differences of this magnitude can vastly change both the calculated profitability and ranking of potential alternatives. Indeed, by using a low discount rate, advocates of particular projects can and have made it appear that benefits greatly exceed costs of alternatives, when the same projects look like losing propositions with a more acceptable, but higher, discount rate. The author might have demonstrated to the reader
290
Book Reviews
how the choice of discount rate can have a profound affect on the estimated profitability and ranking of projects, even overwhelming other benefit and cost calculations. This could be used to caution readers on how progenitors of projects have used unduly low interest rates to justify projects that were not economically viable when assessed by more reasonable calculations. Indeed, this would have been an ideal opportunity to show the vulnerability of cost-benefit analysis and reveal its susceptibility to the “political arithmetic” of project advocates in a way that is unlikely to be detected by decision-makers who lack technical knowledge on the approach. This represents the most vulnerable aspect of the book for those who will rely upon it to understand costbenefit analysis. By not developing applications from the ground up and reviewing the judgments that go into the myriad decisions at each stage, the reader will not gain a full appreciation of both the promise and limitations of cost-benefit analysis. Such detailed applications can examine the robustness of cost-benefit estimates and rankings of alternatives under different assumptions governing definitions and measurements of costs and benefits and different discount rates. In fact, though the book mentions doing a sensitivity analysis to
Evaluation Strategies for Communicating Hallie S. Preskill, and Mary E. Piontek,
get lower and upper boundaries of costs and benefits under different assumptions, it never applies this lesson. It is this need to walk the reader through detailed applications of the cost-benefit methodology that is sorely missing from the book. Although the last chapters try to apply some of the concepts to the environment, health, and developing countries, details are sketchy. As an introduction to the concepts underlying costbenefit analysis, this volume is well-written and comprehensive for a book of its length. It could be used profitably in an undergraduate course or in courses on evaluation where the students have at least some understanding of economics, if supplemented by detailed case studies. If accompanied by studies in applied social areas (health, education, environment, criminal justice, and so on), it would allow the student to participate in grappling with the various steps and judgments used for deriving cost-benefit calculations as well as implementing sensitivity analyses of the results under different, but plausible, assumptions. This type of hands-on activity will enable the student to obtain an understanding of the cost-benefit tool as a consumer or user of such studies in conjunction with the treatment provided by this textbook.
and Reporting: Enhancing Learning in Organizations, Thousand Oaks, CA: Sage, 1996, 274 pp.
Reviewer: Lyn M. Shulha At last! A practical guide that puts learning at the heart of the reporting enterprise and makes reporting a central element in evaluation design. The authors’ premise is that reporting can make significant contributions to organizational learning; that is, learning that occurs when “organizational members are developing new understandings and perspectives on themselves as they seek to improve” (p. 30). For this to happen, they argue, information concerning evaluation processes and findings must get infused throughout the organization in ways that will promote collective reflective practice. This complex process requires considered attention to the activities of communicating and reporting. Several features set this work apart from traditional discussions on reporting. First, as a preliminary step to writing the book, the authors surveyed the membership of the American Evaluation Association’ (AEA) about ’ The experiences of AEA members span the fields of education, social programming, policy development, international development and private consulting.
by Rosalie T. Torres,
their experiences and attitudes related to reporting. A summary of their findings, presented early, allows the reader to compare her/his experiences in this area with those of the broader professional community. Those of us who are only moderately satisfied by the effectiveness of our communicating and reporting plans, who feel challenged and often constrained by the complexity of each new evaluation context, who have serious doubts about the utility of summative documents we produce, but who still find themselves, usually due to time constraints, focusing on final reports with executive summaries, discover that we are in good company. For us, this book offers a fresh look at how to transform reporting from a symbolic act that denotes the conclusion of an evaluation, into an integrated plan to inject information at appropriate times both within and around an organization. The key, according to the authors, is not to focus on the techniques of information dissemination, but on the strategies and formats that will facilitate information processing given the program context and the target audience. This orientation is meant to help in the transformation of evaluative findings into functional knowledge.