The leap from ROI to SROI: Farther than expected?

Accepted Manuscript Title: The Leap from ROI to SROI: Farther than Expected? Author: John Gargani PII: DOI: Reference:

S0149-7189(17)30007-1 http://dx.doi.org/doi:10.1016/j.evalprogplan.2017.01.005 EPP 1413

To appear in: Received date: Accepted date:

13-1-2017 25-1-2017

Please cite this article as: & Gargani, John., The Leap from ROI to SROI: Farther than Expected?.Evaluation and Program Planning http://dx.doi.org/10.1016/j.evalprogplan.2017.01.005 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

The Leap from ROI to SROI: Farther than Expected? John Gargani, Ph.D. Gargani + Company, Inc.

2625 Alcatraz Avenue Number 508 Berkeley, CA 94705

[email protected]

The author received no funding for this research.

final.docx

Page 1 of 45

Highlights



A critical examination of social return on investment (SROI).



Describes how SROI is based on the general approach to financial analysis.



Discusses the strengths and weaknesses of adapting a financial method to evaluate impact.



Introduces a conceptual model of value that may help guide SROI analysis.



Describes how SROI ratios may be used to give greater voice to stakeholders and incorporate multiple perspectives.



Uses examples to illustrate concepts and methods.



Includes a technical appendix that identifies potential sources of statistical bias in SROI estimates.

final.docx

Page 2 of 45

The Leap from ROI to SROI: Farther than Expected?

Investors routinely decide how to allocate limited capital to potential investments. Although judgment and luck play roles in determining the wisdom of their decisions, the proper application of financial analysis has proven indispensable, allowing investors to make wiser choices more often (see, for example, Bernstein, 1996). Not surprisingly, those who are socially minded and financially savvy have begun applying the tools of financial analysis to the evaluation of social and environmental programs. Social return on investment, SROI, is one example. Over the past ten years, it has made the leap from a tool used to build private wealth to one that promises to advance the public good. Has it landed us in a better place? To answer the question, we need to be clear about what constitutes a program and how it relates to organizations and commerce. I use program to mean the systematic application of resources by one or more organizations to activities that are intended to improve the lives of people beyond what is strictly necessary for commerce. This definition is consistent with the broad scope suggested by The Program Evaluation Standards (Yarbrough, Shulha, Hopson, & Caruthers, 2011). It recognizes that commerce by itself often benefits people, and that programs and commerce may be intertwined. Moreover, it allows for the rich variation in programs we are now witnessing in the public and private sectors. A program may be a product or service provided by a nonprofit organization, for-profit company, or social enterprise; a public health campaign implemented by a consortium of pharmaceutical, social service, and education organizations; the investment portfolio of an impact investor seeking above-market, market, or concessionary returns; a policy implemented by a governmental, international, or multinational

final.docx

Page 3 of 45

agency; or the management of a corporation’s value chain as part of a social responsibility initiative. With this broad definition in mind, I consider the strengths and weaknesses of using SROI to evaluate programs. First, I describe what I call the general approach to financial analysis upon which SROI is based. The approach facilitates the evaluation and comparison of investment options by controlling for the time value of money, the amount invested, and a variety of other factors, such as risk. When done well, it successfully places investments on an equal footing, reducing complex, multidimensional information to a unidimensional metric— value—that can be evaluated with a simple decision rule—more is better. If you can read a thermometer, you can choose the best investment, at least in concept. Building an accurate thermometer is far more complicated. There are a number of established methods for doing so (see, for example, Thamhain, 2013). I consider two—net present value analysis and return on investment. Using a simple example, I describe their procedures, how they incorporate counterfactuals in discounted cash flows, and how their results may be interpreted. Then I make the leap to SROI. Using a similar example, I connect financial analysis to impact analysis. I argue that if the translation of return on investment from financial to social contexts is too literal, it weakens evaluative conclusions. We give up too much by idealizing a thermometer that simplifies decision making. The complexity of multiple contexts, cultures, and criteria should moderate expectations that a single, pre-established, more-is-better evaluative criterion can be consistently established. Dismissing SROI for this reason, however, is not justified because we can apply it in other ways that have the potential to produce stronger, richer evaluative conclusions.

final.docx

Page 4 of 45

Although the leap from financial analysis to program evaluation may be farther than it first appears, it is necessary. As recent economic conditions demonstrate, the pool of capital underwriting programs is not dependable, and as the Sustainable Development Goals advanced by the United Nations (2016) illustrate, our expectations for social and environmental change have grown more ambitious. This places great pressure on social investors—the broad collection of governments, philanthropists, ethical investment funds, socially-minded corporations, and other organizations that use their capital to promote the public good in addition to or instead of private financial gain. They need to get the most from the resources they have, and as our collective body of evaluation evidence accumulates, they will be better positioned to do so. Rather than searching for an effective program in one setting, investors will be able choose among many programs within and across settings, all with evidence of effectiveness of various kinds. As in medicine, we should expect this to lead social investors to base their decisions increasingly on measures of efficiency, along with those of effectiveness and other evaluative criteria (Krummenauer & Landwehr, 2005; Marshall & Hux, 2009). SROI is a practical tool for understanding efficiency. If evaluators approach it with a critical eye and apply it in inclusive ways that reflect multiple perspectives, we may be surprised by how far it can take us. The General Approach to Financial Analysis A Simple Example Imagine a traditional financial investor is presented with four investment options, labeled A, B, C, and D in Table 1. The first three require her to put $1,000 at risk, and the fourth $1,300. The investments return $1,100 after one year, $1,200 after one year, $1,200 after two years, and $1,875 after three years, respectively. Which investment should she choose? Which is best?

final.docx

Page 5 of 45

In finance, two principles distinguish better from worse—more money is better, and a dollar today is worth more than a dollar tomorrow (Scholten & Read, 2014). These principles are axiomatic, but not universal. Some cultures do not value more money sooner. In those that do, people frequently act as if they don’t. Many choose to work at jobs they enjoy for less than they could earn elsewhere, make interest-free loans to family and friends, and donate to charity. The two principles do not explain every human action related to money; in traditional financial analysis, they explain everything. -----------------------------------------------------------------------------------------------------------Table 1 -----------------------------------------------------------------------------------------------------------We can use the two principles to evaluate the investment options. Investment B is better than A because it provides a greater return for same investment in the same amount of time (principle 1), and investment B is better than C because it provides the same return for the same investment in less time (principle 2). But how do we compare investments B and D? Investment B pays the investor sooner, which is better according to principle 2, but D pays more, which is better according to principle 1. At this intuitive level, the principles appear to be in conflict. The way to bring them into agreement is to make the decision systematically on the basis of value. A Definition of Value for Finance Some have criticized the concept of value applied to SROI as being underdeveloped, especially in comparison to other concepts favored by economists, such as social welfare (Mertens, Xhauflair, & Marée, 2015; Mulgan, 2010). As Fujiwara (2015, p. 7) put it, “the term ‘social value’ in SROI as it currently stands is hollow." In response, I introduce an explicit conceptual model of value that has two dimensions. In this section, I focus on one dimension that

final.docx

Page 6 of 45

reflects the interests of strictly financial investors (Figure 1). I incorporate the second dimension when I introduce SROI. -----------------------------------------------------------------------------------------------------------Figure 1 -----------------------------------------------------------------------------------------------------------The overarching concept is perceived value, which is an individual-level evaluative judgment of something’s merit, worth, importance, or significance. It may be arrived at systematically or intuitively (Fournier, 1995; Scriven, 1980; Stake et al., 1997). It is a psychological construct, invisible to the eye, that affects other judgments, choices, and behaviors, and it is comprised of many possible dimensions. Of interest to investors is the dimension that entails judgments of pecuniary value. These judgments are related to “matters of the wallet” in the sense that they affect choices and behaviors that have practical financial consequences. Pecuniary value depends on the existence of real markets in which an individual’s perceived value can be expressed as a price at which an object of interest is bought or sold. If a real market for the object does not exist, it has no pecuniary value. This is not as limiting as it may seem because a market requires nothing more than two people willing to make an exchange (Hibbard, 1921). Investors may be interested in the pecuniary value of real transactions, such as the price someone paid for a particular house, which are exclusively past events. Investors may also be interested in the pecuniary value of transactions that have not taken place, such as potential future transactions (the price that someone might pay next year) or hypothetical present or past transactions (the price that someone would imagine paying now or a year ago). Altogether, the model considers four manifestations of pecuniary value because the past may be

final.docx

Page 7 of 45

viewed through two lenses, the historical and the hypothetical. The general approach to financial analysis can be applied to all. The measurement of something’s merit, worth, importance, or significance in monetary units is called valuation. In the case of pecuniary value, this entails creating a record of real, potential, or hypothetical transactions in real markets. Because real transactions typically generate accounting records, their valuation is often trivial. Valuations of potential or hypothetical transactions pose a greater challenge, but one that may be addressed by surveying consumers or statistically modeling prices using historical data. Because the markets are real, data are often available, and for many financial markets the data are abundant. Under the best of circumstances, the price someone paid, might pay, or would have paid for a home, car, or can of beans reflects the item’s perceived value imperfectly. Price is affected by the wealth of the buyer, desperation of the seller, substitute products, market competition, and many other factors. Furthermore, if we measure value with a single market price, as opposed to the idiosyncratic prices individuals would pay at personalized auctions, we ignore individual and group variation. From a strictly financial perspective, none of this matters. Investors are not attempting to measure perceived value, rather maximize the pecuniary value they can capture from transactions that are influenced by it. A transaction takes the form of a cash flow, which is the movement of money from one party to another. Thus, the discrepancy between the investor’s object of interest—the cash paid to or by her—and a measure of its value is typically negligible. In fact, cash flows can be described as self-instrumenting because the act of making or receiving a payment is also a measure of how much was paid. Of primary interest to investors is how much more or less an investment is worth compared to the next best alternative. Does this sound familiar? An evaluator might call it the

final.docx

Page 8 of 45

financial impact of the investment. An economist might call it the marginal value of the investment. Let us call it value added to emphasize that our interest is a (hopefully) positive change in value (defined in accordance with the conceptual model being introduced here). From a purely financial perspective, the best investment is the one that adds the greatest pecuniary value compared to an alternative. It is what analysts seek to maximize when making investment decisions (Koller, Goedhart, & Wessels, 2013). Applying the Concept of Value Added to Investments Let’s assume our investor currently has her money in a bank earning 10% interest per year. In this simple example, the bank deposit represents the next best alternative investment because she will keep her money where it is unless she has a compelling reason to move it. After one year, a $1,000 bank deposit will be worth $1,100, and after two years $1,210. A deposit of $1,300 after three years will be worth $1,730. This is easy enough to compute using

FVt  P 1  r  , t

(1)

where FV is the future value of the deposit at the end of year t, P is the principal deposited in the bank at the end of year 0, and r is the interest rate. -----------------------------------------------------------------------------------------------------------Table 2 -----------------------------------------------------------------------------------------------------------The future value is an estimate of the pecuniary value of the deposit at the end of year t, not the value added. On its own, it provides no information about the best investment. However, we might consider estimating the value added by comparing the future value of each investment to that of a bank deposit of corresponding size and duration (Table 2). Doing so, we find that the investor has no incentive to move her money to investment A because it returns the same amount

final.docx

Page 9 of 45

as the bank (the difference is 0), or C because it returns less (the difference is negative). She is, however, tempted to move her money to B or D because they return more than the bank. Which is better? We again find ourselves comparing different amounts of money at different points in time, so we cannot answer the question in a manner that is consistent with both principles of finance. To do that, we need to use 10% not as an interest rate but as a discount rate. Discount Rates as Counterfactuals Discounting is interest the other way round. With interest, we inflate the investor’s principal over time at 10% per year to estimate the value of the investment at a specific point in the future. With a discount rate, we deflate the future value backward in time to estimate the principal that would be required to generate it. An evaluator might call this hypothetical amount the counterfactual deposit. In finance it is called the present value, and it allows us to compare investment options according to their value at the same point in time. The present value, future value, and discount rate are related mathematically this way:

PV 

FV

1  d 

t

,

(2)

where PV is the present value, and d is the yearly discount rate. Equation (2) is nothing more than Equation (1) after a little algebraic manipulation, but it systematically controls for the time value of money described by the second principle of finance. Using Equation (2), we find that the present values of the potential investments are $1,000, $1,091, $992, and $1,409, respectively. The discount rate plays a central role in financial analysis. It represents the next best investment competing for the same resources, but it may not be a real opportunity. Investors construct discount rates for imagined counterfactual investments that they believe have relevant features in common with the proposed investment. They may be based on historical data or

final.docx

Page 10 of 45

judgment, typically both. By considering different bundles of potentially relevant features, investors may construct different types of discount rate. The risk-free rate is the return on extremely low risk investments, such as government bonds. Nothing is without risk, but these investments are as close to riskless as possible, and their returns place a lower bound on potential discount rates. A risk-adjusted discount rate takes into account that riskier investments are less likely to pay an investor what is promised. This rate is higher than the risk-free rate in order to compensate investors for those occasions when riskier investments fail to provide their full return. A fully risk-adjusted discount rate takes additional factors into account that an investor believes warrant consideration. These may include the underlying rate of inflation, taxes, and commissions, which reduce the value investors can realize and increase the premium they demand as compensation for it. In our simple example, we applied the same discount rate to each investment option. More often, each would have its own discount rate reflecting its particular level of risk and other relevant characteristics. Frequently, no single discount rate emerges for an investment. Different investors may favor different rates, and each investor may consider a range of plausible rates, each attending to a different set of factors and assumptions. This might lead one to conclude that discount rates are arbitrary. They are not. Because the investor is the sole party at financial risk, only the investor needs to be satisfied that the rate represents a reasonable alternative. Equal Footing Our analysis is still incomplete because it does not take into account the different amounts of principal each investment requires. We can control for different principal amounts several ways, and we will consider two—sums and ratios. When we use sums, our decision rule is based on the total value added. To estimate the sum, we treat any amount paid by the investor

final.docx

Page 11 of 45

as a negative cash flow and any amount paid to the investor as a positive cash flow, discount each cash flow to estimate its present value, and add all the resulting positive and negative present values. This is called net present value (NPV) analysis, and it uses the following formula T

NPV   t 0

CFt

1  d 

t

,

(3)

where CF represents the positive or negative cash flow at year t. NPV analysis controls for differences in the timing, number, and magnitude of cash flows, including the principal payment. This means we can make a decision with only one rule—more money is better. Applying Equation (3) to the example yields NPVs of $0, $91, -$8, and $109, respectively. These amounts can be interpreted as the value added (positive) or forfeited (negative) by the investment compared to the next best alternative. They can be placed on a single dimension, like a thermometer (top of Figure 2). The investment that adds the greatest value—regardless of the timing, number, and amount of its cash flows—is best. That is investment D. Even though it pays the investor later than investment B, it is worth waiting because its cash flows are of greater total value. -----------------------------------------------------------------------------------------------------------Figure 2 -----------------------------------------------------------------------------------------------------------Another method uses ratios. In this case, the decision rule is based on the efficiency with which value is added. Payments by the investor are treated as costs, and payments to the investor are treated as benefits. The ratio of the net present value of benefits to the net present value of costs is used to estimate the return on investment (ROI) using the formula

final.docx

Page 12 of 45

Bt  T  t   t  0 1  d   ROI  T  1  100% ,   Ct    t  t 0 1  d  

(4)

where B and C are the cash flows associated with benefits and costs. By convention, ROI in financial contexts is usually reported as the percentage change: 1 is subtracted from the ratio and the difference multiplied by 100%. On the other hand, SROI is often presented as a simple ratio of benefits to costs, making the direct comparison of ROI and SROI somewhat complicated. To facilitate comparison, let us define financial return on investment (FROI) as the simple ratio T

FROI 

Bt

 1  d  t 0 T

.

Ct

 1  d  t 0

t

(5)

t

There is no practical difference between Equations (4) and (5) because one is a linear transformation of the other. An investment that adds precisely no value has an ROI of 0% and an FROI of 1. Another with an ROI of 100% and an FROI of 2 would yield discounted benefits twice as great as the principal invested. Both ROI and FROI can be placed on a single dimension like NPVs (bottom of Figure 2). For our example, the FROI ratios are 1.00, 1.09, 0.99, and 1.08, respectively, and may be interpreted this way: every dollar invested in each option will return $1.00, $1.09, $0.99, and $1.08 of value, respectively. The tense of return is important—it should either be past or future, never present. Otherwise, prospective and retrospective analyses cannot be distinguished. Note that FROI identifies investment B as the best investment, while the NPV analysis identifies investment D. Which is right? That depends on how the investor’s capital can be distributed among investments. If the investment options are mutually exclusive, then NPV

final.docx

Page 13 of 45

analysis should be used. If this were the case, the investor should put the required $1,300 into investment D because it adds the greatest value (NPV of $109). On the other hand, if the investor may divide her $1,300 among any of the options, then ROI should be used. If this were the case, she should first invest as much as possible in the most efficient investment, which would be $1,000 in investment B. Then she should invest as much as possible in the next most efficient investment, which would be $300 in investment D. Doing so yields an NPV of $116 dollars ($1,000 x 1.09 + $300 x 1.08, allowing for rounding error), which adds more total value than any option on its own. Maximizing the total value added is typically the objective of investors; more efficiently adding less value is not. A measure of efficiency, therefore, may not lead to better decisions if it is used as the sole investment criterion. When investment options require the same principal, the results of an NPV and ROI analysis should agree. Otherwise, the thermometer should be constructed with care, taking into account the way in which funds may be divided among options. From ROI to SROI Another Example A social investor with no interest in receiving a financial return can support one of four education programs. The programs, labeled E through H in Table 3, serve the same number of students but have different costs. The investor must cover the full cost of the program he supports, which would be $1,000 for programs E, F, and G, and $1,300 for H—the same levels of investment required in the previous example. Similarly, programs E and F achieve their impacts in year 1, G in year 2, and H in year 3. All seek to improve three student outcomes— health, academic achievement, and cultural awareness—and the improvements last one year. The

final.docx

Page 14 of 45

magnitudes of the impacts are presented as standardized effect sizes (Borenstein, Hedges, Higgins, & Rothstein, 2009, pp. 21-32; Cohen, 1988, pp. 20-21). This means that they are reported in the same units, called standard deviation units, which provide a basis for comparison (see Appendix A). The “total” impacts of the programs are 0.75, 1.00, 1.00, and 1.25 standard deviations, respectively. The word total is in quotation marks because summing the magnitudes of qualitatively distinct impacts is a questionable way to gauge overall success because it ignores what may be important differences in patterns of results. Given this limited information, which program should the social investor support? Which is best? -----------------------------------------------------------------------------------------------------------Table 3 -----------------------------------------------------------------------------------------------------------As in the previous example, choosing the best is problematic. Variation in the cost, timing, magnitude, and pattern of impacts muddies comparisons. The problem is similar to that of comparing financial investments that vary in the timing, magnitude, and pattern of their cash flows. This suggests we may be able to improve our decisions about programs by adapting principles and methods from the general approach to financial analysis. Let’s start by adapting the two principles of finance. Principle 1 might be restated as more positive impact is better than less, all else being equal. Based on this, program F is better than E because it achieves the same or greater impact on all outcomes in the same amount of time with the same level of funding. In this case, both its total impact and pattern of impacts are preferable. Principle 2 might be restated as achieving a positive impact sooner is better, all else being equal. Program F is better than G because it achieves the same “total” impact in less time, but this is an imperfect comparison because neither pattern of impacts is unambiguously better.

final.docx

Page 15 of 45

That leaves us to compare programs H and F. Similar to the financial example, F achieves less total impact sooner at a lower cost. H achieves a greater total impact and a more desirable pattern of impacts later at a higher cost. The adapted principles appear to be in conflict. Now let’s consider adapting financial methods. We resolved a similar conflict previously by basing our decision, at least in part, on FROI. So perhaps we can apply an analogous method, like SROI, to evaluate how well financial resources will be (or have been) allocated to programs. The SROI ratio is defined as T

SROI 

Bt

 1  d  t 0 T

,

Ct

 1  d  t 0

t

(6)

t

which is mathematically identical to the definition of FROI given in Equation (5). However, benefits and costs are interpreted, measured, and aggregated differently. Here, they represent the value of a program’s impacts and inputs, some or all of which may be nonpecuniary. Value is measured in monetary units, which may not be the natural units of impacts and inputs. And it is aggregated not only over time, but potentially over outcome, group, and type of value. The nature of the calculation and its interpretation are more challenging. To make the leap from FROI to SROI successfully, we need to expand our definition of value, establish a trustworthy approach to valuation, and incorporate multiple, possibly competing perspectives. A Definition of Value for Impact I present an expanded model of value in Figure 3. Traditional financial investors are concerned with pecuniary value, matters of the wallet, which is the dimension of perceived value on the left of the figure. Social investors are also concerned with nonpecuniary value, matters of the heart, which is the complementary dimension on the right. Wellbeing, happiness, fulfillment,

final.docx

Page 16 of 45

satisfaction, purpose, engagement, and meaning are examples of program outcomes that have value to people in the absence of real markets in which they can be bought and sold. -----------------------------------------------------------------------------------------------------------Figure 3 -----------------------------------------------------------------------------------------------------------Some program outcomes, like educational attainment, may have both pecuniary and nonpecuniary value. My education has nonpecuniary value, at least to me, because I appreciate how it opens my mind to the world, other people, and ideas, all of which give my life meaning. However, I cannot directly buy or sell meaning in a market. My education also has pecuniary value because the skills I learned are demanded in the labor market. Although the meaning I derive from education may be among the many factors that determined the price I paid for it, the extent to which I value meaning does not depend on the existence of a market in which educational services are bought and sold. Likewise, the salary I can demand in the labor market because of my education has pecuniary value whether or not I value the meaning education has afforded me. Measuring the Value of Impacts There are many ways to estimate nonpecuniary value and many units in which it can be expressed (Scriven, 2007). SROI situates nonpecuniary and pecuniary value within the general approach to financial analysis, measuring both in monetary units. Dollars are not the natural units of most program outcomes, so we often face the problem of representing nonpecuniary value in monetary terms. The general solution is to invoke hypothetical markets. In essence, we record the hypothetical transactions that we believe people would make in the past, present, or future in markets that can be imagined but do not exist.

final.docx

Page 17 of 45

For example, we could use surveys to determine the amount of money stakeholders would hypothetically be willing to pay for improved health—not additional health services, but improvements in health per se. This approach is called willingness to pay. Or we could ask them for the price they would hypothetically accept to give up an outcome, like the enjoyment they receive from reading. This is called willingness to accept. Using large amounts of data and statistical methods, we could estimate the causal impact of wealth on wellbeing, happiness, or another outcome of interest, thereby establishing a hypothetical market price for various levels of the outcome. Or instead of framing market exchanges in terms of money, we could ask stakeholders to make hypothetical swaps between program outcomes and goods with known market values. These methods have been used by economists for some time, and many reviews have been published, including those by Carson and Hanemann (2005), Fujiwara and Campbell (2011), Fujiwara (2013), and Venkatachalam (2004). Using these methods to measure the nonpecuniary value of impacts in monetary units is akin to measuring their relative importance in Likert scale units. Both units provide stakeholders with the means of expressing perceived value; neither yields a result that could be easily interpreted as an exchange rate that should be applied in the real world. Having estimated the value of wellbeing in dollars, would it be right to pay poor, vulnerable people to forgo some level of it? We need not believe we should, or more generally endorse the literal cash equivalence of impacts, for valuations to be useful. We only need to view valuations as we would any other psychological measure. As such, they can help social investors calibrate the relative magnitude of their investments across multiple domains, enabling them invest more in what stakeholders value. Additionally, by defining valuations as psychological measures, it emphasizes the responsibility of the analyst to provide evidence that they are sufficiently

final.docx

Page 18 of 45

accurate and reliable to support their intended purposes and avoid harm (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014). Aggregation The concept of total impact and the comparison of impact patterns are problematic. If aggregating impacts is to be useful, reasonable requirements should be met that improve conceptual clarity and practical interpretation. One requirement is construct validity. When qualitatively distinct impact measures are combined, the resulting aggregate measure reflects a new, broader construct. Adding apples and oranges yields a measure, not of one or the other, but of fruit. If we combine an impact’s present-day pecuniary and nonpecuniary value, then the result is a measure of perceived value, or some facet of it, at the present moment. If we combine the pecuniary and nonpecuniary value of multiple impacts at various points in time, then the result is a measure of a different, more complex form of perceived value. The analyst should define the broader construct and provide evidence that it has been measured well. This entails more than ensuring that impacts or their value are measured in the same units, monetary or otherwise. An equivalence of units may belie a diversity of meaning. In our example, all impacts were measured in standard deviation units, yet each is qualitatively distinct and their sum is not constrained to a single interpretation. Some might construe the aggregation of health, academic achievement, and cultural awareness as a measure of student wellbeing, others job readiness or social integration. The same problem arises when trying to combine measures of pecuniary and nonpecuniary value. The conceptual model of value in Figure 3 may help. The model describes seven types of value reflecting different combinations of construct, setting, timing, and

final.docx

Page 19 of 45

transaction. They span past pecuniary value revealed in an actual market transaction to future nonpecuniary value expressed in a hypothetical market transaction. Even though they are measured in the same units, they may in some contexts function as apples and oranges, and in others only as apples. The analyst may be able to explore how they function by moving up or across the figure, placing various boundaries on the construct of interest in order to discover its meaning. Another requirement is incorporating stakeholder preferences into aggregations. In our example, would stakeholders be willing to trade a one unit increase in academic achievement for a one unit decrease in health or cultural awareness? If not, the impacts are not fungible from their point of view. Thus, we cannot conclude that more total impact is better, as we can with money, because stakeholders have a preference for certain patterns or arrangements of impact. A common strategy for addressing this problem is to weight impact measures according to their relative importance to stakeholders. For this to work, there must be a uniformity of preferences among all the stakeholder groups under consideration. If groups have different preferences, then analysts should estimate the total impact using appropriate weights for each group. Otherwise, a single measure may disadvantage some groups or lead to conclusions that no group would endorse. Applying SROI from the Investor’s Perspective Let’s assume that the social investor in our example is concerned only with the pecuniary value of program impacts in the form of savings to government, and that there is prior research to suggest that a one standard deviation increase in the health, academic achievement, and cultural awareness saves government an average of $4,000, $1,400, and $100, respectively. Multiplying the value of these savings by the impact estimates in Table 3 yields the valuations of the impacts

final.docx

Page 20 of 45

presented in the third column of Table 4. Multiplying averages and summing up the products is a common feature of SROI, intended to weight the sums of benefits and costs to reflect preferences. For technical reasons, this seemingly straightforward approach can produce misleading results under certain conditions, so it should be used and interpreted with caution (see Appendix B). In Table 4, the total value of the impacts is adjusted for the time value of money. Three discount rates are used, 0% (no discounting), 5%, and 10%. Similar to financial investments, discounting helps social investors judge whether capital allocated to one program adds more value than it would if allocated to the next best alternative. However, the interpretation of the resulting ratio requires more care. Consider the ratio for program H with no discounting. It indicates that every dollar invested in H will return (or returned) $2.10 of value on average at its proposed (or prior) level of operation. Not only does the tense of return matter, the analyst should remind decision makers that the ratio is an average (not marginal) rate and the program may become more or less efficient as it scales. In other words, should the investor like H so much he doubles his investment in order to expand its reach, his social return on investment may vary in ways he cannot anticipate from the ratio alone. In the example there is even greater complication because the results of the SROI analysis are sensitive to the choice of discount rate. Without discounting, program H has the largest SROI ratio. After discounting by 5% or 10%, G has the largest ratio. The reason for this is that discount rates favor earlier impacts (or more precisely, the earlier realization of value), and larger discount rates favor earlier impacts more. As compared to G, program H realizes more value later. As the discount rate increases, the value of later impacts deflates more quickly than that of earlier impacts, eventually making G’s discounted valuation greater. A consequence of

final.docx

Page 21 of 45

this property of discount rates is that when the value of impacts is realized in the very long-term, such as environmental benefits that will be bestowed on the next generation, they are deflated to the point of having no present value. Discounting has a values stance that emphasizes the near term, stakeholders may not. It is the responsibility of analysts to ensure that their analysis reflects the values of people, not methods. -----------------------------------------------------------------------------------------------------------Table 4 -----------------------------------------------------------------------------------------------------------NPV analysis reaches a different conclusion. It identifies program H as the one that adds the most value within the range of discount rates considered. As noted, the investment option that most efficiently adds value may not be the one that adds the most value. This is the case here, and it illustrates why SROI should not be used as the sole criterion for judging programs. This, of course, can be said of almost any single piece of evaluative evidence. Applying SROI from the Stakeholders’ Perspective The interests of stakeholders may differ from those of social investors, leading to different valuations. Parents in some communities may prefer increases in cultural awareness to increases in academic achievement or health. Moreover, they may value the intrinsic importance of cultural awareness—its nonpecuniary value—and care little for whether it saves government money. Let’s assume we used a valuation technique, such as willingness to pay, to measure how parents value program impacts, and found that they take a very strong view—a one standard deviation increase in health is valued at $0, academic achievement at $0, and cultural awareness at $5,500. In practice, it would be unusual for a community to believe that an impact adds precisely no value, as opposed to adding or subtracting some value, but it simplifies the example.

final.docx

Page 22 of 45

Multiplying the valuations by the impact estimates in Table 3 and applying three discount rates yields the values in Table 5. -----------------------------------------------------------------------------------------------------------Table 5 -----------------------------------------------------------------------------------------------------------Based on this analysis, parents prefer programs E and F, and they are indifferent as to which one is implemented (in the absence of additional evaluative criteria). The results are not sensitive to the range of discount rates considered. However, the social investor prefers program G or H. Either parents or the investor will not get the program they want. How should a choice be made? One tempting solution is to combine the two analyses by aggregating pecuniary and nonpecuniary value across the two stakeholder groups. If this were done, it would present a new problem—how should the valuations of the two stakeholder groups be weighted? Should they be given equal weight? Should one be privileged over another? Tempting as it may be, a single ratio may not be useful in this case, and multiple ratios describing group variation may be. Variation as Information, Not Nuisance Cash flows are a special case of value flows. For social investments, there are four aspects of value flow that take on greater importance than they do for financial investments: the types of value that flow, from whom it flows, to whom it flows, and who does the valuing. Financial investments (top of Figure 4) are comprised of cash flows between an investor and an investment. The cash flows from an investor ( C$I ) and to an investor ( B$I ) have pecuniary value (designated by $). The investor chooses the discount rate, judges its appropriateness, evaluates the potential investment, makes the investment, takes the risk, and receives the benefit—only the investor needs to be satisfied with this process and its result.

final.docx

Page 23 of 45

-----------------------------------------------------------------------------------------------------------Figure 4 -----------------------------------------------------------------------------------------------------------By definition, a social investment (bottom of Figure 4) is intended to benefit someone in addition to or instead of the investor. An analysis of a social investment takes into account at least one additional stakeholder group, ideally more. For simplicity, let’s limit our discussion to investors and the stakeholders (S) who may benefit. Both nonpecuniary (V) and pecuniary value may flow to or from investors and stakeholders. The flow of value from these groups to the program may include money from investments, donations, or purchases; emotional costs, such as stress, loneliness, or anxiety; time as volunteers or participants; and ancillary expenses, such as professional fees or childcare.

B$I Using this framework, we can see there is one way to calculate FROI, which is $ . On CI the other hand, there are 225 ways to calculate SROI when we consider two groups: 15 combinations of costs x 15 combinations of benefits = 225 possible ratios. The number of ratios becomes large quickly as we consider more groups (there are 3,960 possible ratios for three groups and over 65,000 for four groups). To view SROI as a single ratio is overly simplistic. It may be better to consider families of ratios that provide different perspectives on success. Perhaps the most commonly reported family of SROI ratios include

BS$ BSV , , and C$I C$I

BS$  BSV . They measure how efficiently investors use their resources to produce benefits for C$I B$I  BS$ B$I  BSV stakeholders. Another family of ratios that is sometimes reported includes , , or C$I C$I

final.docx

Page 24 of 45

B$I  BS$  BSV . Here, the numerators represent different interpretations of the concept of blended C$I value (Bonini & Emerson, 2005), a criterion for gauging the success of organizations with double or triple bottom lines. The ratios measure how efficiently investors use their resources to create financial value for themselves and social and environmental value for others. Like all aggregate measures, it hides as well as reveals. An organization that generates a large financial return and a small amount of social harm may create the same blended value as another that generates a small financial loss and a large social benefit. The remaining 218 possible ratios are not reported often. Perhaps they should be. For example, the ratio

BS$  BSV estimates the return to stakeholders on their investment, which CS$  CSV

provides insight into how stakeholders may judge the success of a program;

BS$ reflects the CS$

B$I economic incentive for stakeholders to participate in a program; and $ measures the CS  CSV investor’s financial return on stakeholders’ investment. We can look beyond this set of 225 ratios by allowing benefits and costs to be placed in

BS$ either the denominator or the numerator. That would allow us to consider $ , the proportion BI  BS$ of financial benefits that accrue to stakeholders;

BS$ C$I , which is the relative efficiency with CS$ B$I

which a program creates pecuniary value for stakeholders using their financial resources compared to investors using their financial resources; and

final.docx

CSV , which is the relative C VI

Page 25 of 45

nonpecuniary value of burdens a program imposes on stakeholders compared to those it imposes on investors. There is a wonderful richness in these ratios that would allow evaluators to tell evidence-based stories of programs, people, perspective, and purpose. It is a pity they are not used more. Conclusion It is remarkable that investments can be evaluated with a single criterion that consistently yields better decisions. It violates many of the principles of systems thinking, complexity theory, and a host of research traditions. Yet it works. If the same simple approach could be used to evaluate programs with a similar level of success, it would be transformative. The question I posed at the start is whether one method based on that approach, SROI, has successfully made the transition from evaluating financial investments to evaluating social and environmental programs. If we hold too tightly to the financial analogy upon which SROI is based, I believe the answer is no. The burden of accommodating diverse perspectives on multiple impacts, each of which may be measured, valued, and inferred in myriad ways, is more than a single number can bear. Better, I have suggested, to consider multiple ratios, alongside other forms of evidence, to better understand the full range of perspectives on program success. With so many possible ratios, decisions about resource allocation become more complicated. Decision makers are no longer able to maximize value, they must optimize it. Optimization is characterized by tradeoffs among multiple criteria, and it raises fundamental evaluative questions: Who decides? Whose values are considered? Which of their values should be included? How should they be weighted? How should they be operationalized as evaluative criteria? Evaluators have long answered these questions by applying explicit evaluative reasoning (Fournier, 1995). There have been recent calls for economic evaluations, like those

final.docx

Page 26 of 45

based upon the general approach to financial analysis, to do the same because they have implicit evaluative criteria embedded in them that may not be appropriate (King, 2016). For example, an SROI ratio greater than one indicates that a program produces more value than it consumes. This is sometimes presented as a natural, perhaps universal criterion for success. However, it does not indicate what stakeholders believe constitutes sufficient value creation to warrant success. They may set the threshold higher or lower, and where they set it may be contingent on other factors. For example, stakeholders might accept a ratio less than one in order to promote fairness, a more equitable distribution of value in a community, or some other end not fully captured by the SROI analysis. Left unexamined, this built-in criterion may lead decision makers to promote value at the expense of values. Even with explicit evaluative reasoning, there are potential risks to applying the general approach to financial analysis to program evaluation. One is commodification (with an f), which is the act of making something salable in real markets that previously was not. By applying the general approach to financial analysis, it becomes possible to buy and sell an impact because its value to someone has been established. We are witnessing this with social impact bonds, social success notes, and pay-for-success financing in general. They are funding arrangements in which programs are paid a greater amount of money for producing larger, better, or more sustainable impacts. They have two potential advantages. First, they may align the incentives of program managers with the values of stakeholders by rewarding investors for larger program impacts. Second, they may decrease the financial risk to the public because initial funding of the program is provided by private investors, and later payments of public funds to investors is contingent on the program’s success. Many pay-for-success arrangements have focused on pecuniary value, especially savings to government, because by freeing real dollars investors create the means for

final.docx

Page 27 of 45

their compensation. If the impacts that stakeholders value most are not pecuniary, commodification may create a perverse incentive that privileges the government’s wallet over the public’s heart. Another is commoditization (with a z). In commerce, a commodity is a product or service that has no differentiating characteristics except price. Gasoline is a commodity because oil companies refine it to the same specifications, so drivers are able choose between two neighboring gas stations solely on price. By reducing multiple impacts to an aggregate monetary estimate of their value, analysts suggest (intentionally or not) that the underlying collection of impacts has no other differentiating features. That would be a poor way to buy and sell impacts given the multiple purposes and constituencies programs serve. In spite of the risks, it is essential that we situate questions of efficiency at the center of our profession. In many sectors and geographies, there is an overabundance of nonprofit organizations, for-profit companies, and social enterprises implementing programs that have been developed over decades. In these settings, we may have a great deal of evidence to suggest that our current actions are beneficial. We do not need another study to demonstrate that a particular action is better than no action. We need to know whether one effective action is better than another. The explicit criteria we use to operationalize the concept of better may include effectiveness, distribution of benefits, scalability, feasibility, and other qualities. Shouldn’t efficiency be among them? SROI is one method for understanding efficiency, and it can be implemented in ways that illuminate the diverse values of stakeholders. We should be using it more.

final.docx

Page 28 of 45

Appendix A: Standardized Effect Sizes A standardized effect size indicates the number of standard deviations that, on average, a program shifts those it serves on the distribution of outcomes. For example, a program that produces an impact on academic achievement with a standardized effect size of 0.25 advances the achievement of students a quarter of a standard deviation on average. Students who otherwise would have been in the middle of the distribution—the 50th percentile in academic achievement—would progress to the 60th percentile on average. Those otherwise at the 75th percentile would progress to the 82nd percentile on average.

Appendix B: Potential Sources of Statistical Bias When Valuing Impacts The value of a program’s impact is estimated in three steps—measuring outcomes, valuing outcomes, and inferring the value added—that make use of at least two counterfactuals. One counterfactual describes the alternative use of financial (and other) resources that investors (and others) contribute to the program. This is addressed by discounting. The other describes one or more alternative experiences to that of program participation. This is addressed with a research design. Gargani, Fagergren, and Tangonan (2014) conducted a review of 107 publicly available SROI reports and identified four common approaches to valuation that incorporate these counterfactuals into the three-step process. I describe them here using the Rubin causal model (Holland, 1986; Rubin, 1974, 2004), which provides an explicit definition of impact. According to the model, a program’s impact on an individual is defined as the difference in potential outcomes

Di  Y(1)i  Y(0)i ,

final.docx

(B.1)

Page 29 of 45

where Y(1)i is the outcome measure for person i after experiencing the program, and Y(0)i is the outcome measure for the same person at the same point in time after an alternative experience. This is a conceptual definition, not a practical analysis strategy, because the same person cannot engage in mutually exclusive experiences. However, I use it because it describes what we would ideally like to know about program impacts, reveals how different approaches to valuation affect the quality of SROI estimates, and avoids the need to consider the complex interaction of research design and valuation method. Two of the three steps are incorporated in Equation (B.1)—the measurement of outcomes denoted by Y (step 1) and the inference of value added by taking a difference (step 3). The intermediate step of valuation is missing, through which the pecuniary or nonpecuniary value of an outcome is expressed in monetary units. There are four ways valuations are commonly calculated, which I refer to as cases. Each case introduces additional sources of potential statistical bias. A statistically biased estimate is one that, on average, provides a result that is too high or low. Another term for statistical bias is inaccuracy. For Case 1, outcomes, like wages or savings, are measured directly in monetary units at the individual level, and the average value of a program’s impact is

V1 

1  Y(1)i  Y(0)i , N i

(B.2)

where N is the number of participants. V1 is unbiased if either the outcome measure is unbiased (what analysts strive for) or the outcome measures are equally biased in both conditions (a lucky accident). These are the only sources of bias because the outcome measure is the valuation of the outcome, and the impact is inferred in an ideal way as described by the Rubin causal model. In the real world, the features of the implemented research design would also play a critical role in determining whether estimates were biased. final.docx

Page 30 of 45

When an outcome is measured in non-monetary units, its valuation depends on a financial proxy. The proxy is a rate, like hourly wage, that converts an outcome, like hours of work, to monetary units. Proxies are often used to express nonpecuniary value in monetary units. For example, we might measure the wellbeing of participants using a survey scale and construct a rate—$1,000 per scale point—that reflects the value stakeholders place on their wellbeing. For Case 2, the outcome is measured in nonmonetary units, and both the outcome and proxy are measured at the individual level. The average value of the impact in this case can be computed as

V2 

1  P(1)i Y(1)i  P(0)i Y(0)i , N i

(B.3)

where P(1) and P(0) are the proxies under the program and the alternative experience. A unique proxy is associated with every combination of person and experience because a person’s proxy may be affected by his or her experience. For example, a job training program may increase hours of employment and increase the hourly wage participants earn. If both the outcome and proxy measures are unbiased, then the resulting estimate is unbiased (given the ideal form of causal inference). Thus, Case 2 requires one more assumption than Case 1, for which the analysts should provide validity evidence. Case 2 helps us see that an analyst may conclude that a program added value even though it did not improve the outcome. Consider a program that educates the public about the health benefits of diet and exercise. If it provides precisely zero health benefits ( Y(1)i  Y(0)i for every participant), but caused the public to value their health more ( P(1)i  P(0)i on average), then V2 would be positive, suggesting the program added value by improving health. In reality, it changed the way people valued their current level of health.

final.docx

Page 31 of 45

For Case 3, only an average proxy, outcome measure, or both are available for the study sample. Gargani et al. (2014) found this to be a common situation. Sometimes averages were used because program staff only tracked aggregate metrics, the outcome and proxy were estimated from different samples of participants, or there was a desire to reduce the burden of data collection and analysis. In this case, the average value of the program impact is estimated as V3  P(1)Y(1)  P(0)Y(0) ,

(B.4)

where P and Y are the average proxy and outcome. This approach introduces a new source of potential bias—the covariance of the outcome and the proxy. Using the definition of covariance (Hogg & Craig, 1995, p. 92) and Equation (B.3), we get V3  P(1)Y(1)  COV P(1) , Y(1)   P(0)Y(0)  COV P(0) , Y(0)  ,  i i  i i

(B.5)

where COV is the population covariance of the outcome and proxy in square brackets. If the two covariance terms are equal, they drop out of the equation and we are left with Equation (B.4). When this is the case, V2 = V3 and Cases 2 and 3 provide the same result. If the covariance terms are not equal, Case 3 adds bias into the estimate equal to COV  P(0) , Y(0)   COV P(1) , Y(1)  .  i i  i i

(B.6)

For Case 3 to produce an unbiased estimate, not only must the outcome and proxy measures be unbiased, their covariances in the program and alternative conditions must be equal. That is one more assumption than Case 2. For Case 4, the average proxy comes from an external source, such as an unrelated research study or government records, and the average value of the impact is estimated as V4  P* (1)Y(1)  P* (0)Y(0) ,

final.docx

(B.7)

Page 32 of 45

where the asterisk indicates the external proxy. External proxies are common because they save effort and are widely available (Social Value International, 2016). Their use introduces an additional assumption regarding outside-in validity. Internal validity refers to the credibility of an impact estimate in a particular study, and external validity to whether the estimate generalizes to other studies (Gargani & Donaldson, 2011). Outside-in validity refers to whether proxies from another study, at another time, with another sample, in another context can be used in the current study to support its internal validity in a way that serves the intended purposes of the analysis without causing undue harm. Analysts tend to provide little validity evidence regarding the suitability of external proxies. Sometimes analysts take a shortcut with Cases 3 and 4, which introduces another potential source of bias. They assume the value of the proxy is unchanged by the program, that is

P  P(1)  P(0) . When this condition is met, we can arrive at the same answer by first estimating the impact and then multiplying it by the proxy, like this P  Y(1)  Y(0)  .

(B.8)

When the assumption of equal proxies is not met, it introduces bias equal to

 P  P(1)  Y(1)   P(0)  P  Y(0) .

(B.9)

As we move from Case 1 to Case 4, and when we introduce additional shortcuts, the credibility of impact valuations decrease because additional assumptions must be met to produce at an unbiased estimate. At the same time, the feasibility of the analysis increases because it requires fewer data, simpler calculations, and lower organizational capacity for evaluation. Currently, there is no professional guidance about how to assess the tradeoff between credibility and feasibility, which makes the role of the analyst critical.

final.docx

Page 33 of 45

References American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Psychological Association. Bernstein, P. L. (1996). Against the gods: The remarkable story of risk. New York: John Wiley & Sons. Bonini, S., & Emerson, J. (2005). Maximizing blended value: Building beyond the blended value map to sustainable investing, philanthropy and organizations. Retrieved from http://www.blendedvalue.org/wp-content/uploads/2004/02/pdf-max-blendedvalue.pdf Borenstein, M., Hedges, L. V, Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to metaanalysis. Chichester, UK: Wiley. Carson, R. T., & Hanemann, W. M. (2005). Contingent Valuation. In R.-G. Mler & J. R. Vincent (Eds.), Handbook of Environmental Economics (Vol. 2, pp. 821–936). San Francisco: Elsevier. doi:10.1016/S1574-0099(05)02017-6 Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Fournier, D. M. (1995). Establishing evaluative conclusions: A distinction between general and working logic. New Directions for Evaluation, 1995(68), 15–32. doi:10.1002/ev.1017 Fujiwara, D. (2013). A general method for valuing non-market goods using wellbeing data: Three-stage wellbeing valuation. London. Fujiwara, D. (2015). The seven principle problems of SROI. London. Fujiwara, D., & Campbell, R. (2011). Valuation techniques for social cost-benefit analysis : Stated preference, revealed preference and subjective well-being approaches: A discussion of current issues. Current (pp. 1–74). London. Gargani, J., & Donaldson, S. I. (2011). What works for whom, where, why, for what, and when?: Using evaluation evidence to take action in local contexts. In H.-T. Chen, S. I. Donaldson, & M. M. Mark (Eds.), Advancing validity in outcome evaluation: Theory and practice. New Directions for Evaluation (pp. 17–30). Gargani, J., Fagergren, A., & Tangonan, C. (2014). Findings from a new review of social return on investment analyses. In Annual Conference of the Canadian Evaluation Society. Ottawa. Hibbard, B. H. (1921). Marketing agricultural products. New York: D. Appleton and Company.

final.docx

Page 34 of 45

Hogg, R. V, & Craig, A. T. (1995). Introduction to mathematical statistics (Fifth.). Upper Saddle River, NJ: Prentice Hall. Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. Retrieved from http://www.jstor.org/stable/2289064 King, J. (2016). Using Economic Methods Evaluatively. American Journal of Evaluation, 1–13. doi:10.1177/1098214016641211 Koller, T., Goedhart, M., & Wessels, D. (2013). Valuation: Measuring and managing the value of companies. Journal of Chemical Information and Modeling (Vol. 53, p. 839). doi:10.1017/CBO9781107415324.004 Krummenauer, F., & Landwehr, I. (2005). Incremental cost effectiveness evaluation in clinical research. European Journal of Medical Research, 10(22), 18–22. Marshall, D. A., & Hux, M. (2009). Design and analysis issues for economic analysis alongside clinical trials. Medical Care, 47(Supplement), S14–S20. doi:10.1097/MLR.0b013e3181a31971 Mertens, S., Xhauflair, V., & Marée, M. (2015). Questioning the social return on investment (SROI). Leige, Belgium. Mulgan, G. (2010). Measuring social value. Stanford Social Innovation Review, Summer. Retrieved from https://ssir.org/articles/entry/measuring_social_value Rubin, D. B. (1974). Estimating the effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. Rubin, D. B. (2004). Teaching statistical inference for causal effects in experiments and observational studies. Journal of Educational and Behavioral Statistics, 29(3), 343–367. doi:10.3102/10769986029003343 Scholten, M., & Read, D. (2014). Better is worse, worse is better: Violations of dominance in intertemporal choice. Decision, 1(2). Retrieved from http://psycnet.apa.org/journals/dec/1/3/215/ Scriven, M. (1980). The Logic of Evaluation. Invemess, CA: Edgepress. Scriven, M. (2007). The logic of evaluation. (H. V. Hansen, C. W. Tindale, J. A. Blair, R. H. Johnson, & D. M. Godden, Eds.)Dissensus and the Search for Common Ground (pp. 1–16). Windsor, ON: OSSA. Social Value International. (2016). Databases & Financial Proxies. Retrieved August 26, 2016, from http://socialvalueportal.com/databases-and-financial-proxies/

final.docx

Page 35 of 45

Stake, R., Migotsky, C., Davis, R., Cisneros, E. J., Depaul, G., Dunbar, C., … Chaves, I. (1997). The evolving syntheses of program value. Evaluation Practice, 18(2), 89–103. doi:10.1016/S0886-1633(97)90015-5 Thamhain, H. J. (2013). Contemporary methods for evaluating complex project proposals. Journal of Industrial Engineering International, 9(1), 9–34. doi:10.1186/2251-712X-9-34 United Nations. (2016). Sustainable development goals. Retrieved from http://www.un.org/sustainabledevelopment/sustainable-development-goals/ Venkatachalam, L. (2004). The contingent valuation method: A review. Environmental Impact Assessment Review, 24, 89–124. doi:10.1016/S0195-9255(03)00138-0 Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage.

final.docx

Page 36 of 45

Constructs

Perceived Value

Pecuniary Value

Setting Real Market Timing Past Type of Transaction

Real

Future

Potential

Present

Past

Hypothetical

Financial Value Added: Marginal increase in the

Figure 1: A conceptual model ofcombination value for offinancial investors value of some these transactions.

final.docx

Page 37 of 45

C A -$8 $0

0.99 1.00 C A

B $91

D $109

1.08 1.09 D B

Figure 2: NPVs (top) and FROI ratios (bottom) placed on single scales

final.docx

Page 38 of 45

Constructs

Perceived Value

Pecuniary Value

Nonpecuniary Value

Real Market

Hypothetical Market

Setting

Timing of Transactions Past Type of Transaction

Real

Future

Present

Past Past

Potential

Present

Future

Hypothetical

Financial Value Added: Marginal increase in the

Figure 3:some A conceptual model of value for social investors value of combination of these transactions. Social Value Added: Marginal increase in the value of some combination of these transactions.

final.docx

Page 39 of 45

Figure 4: The structure of financial investments (A) and social investments (B)

final.docx

Page 40 of 45

Table 1: Four investment options

final.docx

Investment

Amount Invested

Years Invested

Amount Returned

A

$1,000

1

$1,100

B

$1,000

1

$1,200

C

$1,000

2

$1,200

D

$1,300

3

$1,875

Page 41 of 45

Table 2: Four investment options compared to a bank deposit Amount Returned (Future Value) Investment

Years Invested

Investments

Bank

Difference

A

Amount Invested (Present Value) $1,000

1

$1,100

$1,100

0

B

$1,000

1

$1,200

$1,100

$100

C

$1,000

2

$1,200

$1,210

-10

D

$1,300

3

$1,875

$1,730

$145

final.docx

Page 42 of 45

Table 3: Four programs, only one of which may be funded Impacts as Standardized Effect Sizes Program

Cost per School

E

$1,000

1

0.25

0.25

0.25

0.75

F

$1,000

1

0.25

0.50

0.25

1.00

G

$1,000

2

0.25

0.75

0.00

1.00

H

$1,300

3

0.50

0.50

0.25

1.25

final.docx

Year of Impact

Health Academic Cultural "Total"

Page 43 of 45

Table 4: Value from the investor’s perspective Table 4: Year of

Total Value of Impact

Program Impact d=0% d=5%

d=10%

SROI

NPV

d=0% d=5%

d=10%

d=0%

d=5%

d=10%

E

1

$1,375 $1,310

$1,250

1.38

1.31

1.25

$375

$310

$250

F

1

$1,725 $1,643

$1,568

1.73

1.64

1.57

$725

$643

$568

G

2

$2,050 $1,859

$1,694

2.05

1.86

1.69

$1,050

$859

$694

H

3

$2,725 $2,354

$2,047

2.10

1.81

1.57

$1,425

$1,054

$747

final.docx

Page 44 of 45

Table 5: Value from the stakeholders’ perspective Table 5: Year of Program Impact

Total Value of Impact d=0%

d=5%

SROI

d=10%

NPV

d=0% d=5%

d=10%

d=0%

d=5%

d=10%

E

1

$1,375

$1,310

$1,250

1.38

1.31

1.25

$375

$310

$250

F

1

$1,375

$1,310

$1,250

1.38

1.31

1.25

$375

$310

$250

G

2

$0

$0

$0

0.00

0.00

0.00

-$1,000

-$1,000

-$1,000

H

3

$1,375

$1,188

$1,033

1.06

0.91

0.79

$75

-$112

-$267

final.docx

Page 45 of 45