The role of financial incentives in utility-sponsored residential conservation programs: A review of customer surveys

Evaluation andProgram Planning, Vol. 7, pp. 131-141, 1984 Printed

Copyright

in the USA. All rights reserved.

0149-7189/84 $3.00 + .OO 0 1984 Pergamon Press Ltd

THE ROLE OF FINANCIAL INCENTIVES IN UTILITY-SPONSORED RESIDENTIAL CONSERVATION PROGRAMS: A Review of Customer Surveys

LINDA BERRY Oak Ridge National

Laboratory

ABSTRACT A namber of utility programs are encouraging residential customers to invest in energyefficient eqaipme~t by providi~g~nancial incentives for these actions. Subsidized loans are the most common type of incentive offered by utility programs, although discounts, rebates, lower rates, and free materiak or labor also have been provided. Given the large sums involved in utility loan programs, an understanding of the impact of these financial incentives on retrofit investment decisions has a high potential policy value. The purpose of this paper is to review the available evidence on this issue and to suggest ways that additional evidence can be obtained.

INTRODUCTION Investment in equipment that improves the energy efficiency of the existing U.S. housing stock can have significant effects on energy demand. It has been estimated that savings of 40%-50% are technically possible in the average residence. These savings can be achieved with existing technology, with no change in lifestyle or comfort, and with substantial cost savings to homeowners. (Office of Technology Assessment, 1979). Given an aggressive national retrofit program, savings equivalent to between 19 billion and 29 billion barrels of oil could be achieved by the year 2000 (Office of Technology Assessment, 1979). In spite of the cost effectiveness of conservation retrofit investments, most homeowners make few investments and the investments made are usually suboptimal. A number of explanations for these less than optimal investment levels are reviewed elsewhere (Crossley, 1983; Office of Technology Assessment, 1979; Rosenberg, 1981; Solar Energy Research Institute, 1981). One frequently identified barrier to increasing the energy efficiency of existing homes is the lack of access to capital. Because the cost of a complete home retrofit

may be as high as several thousand dollars, lack of funds, or financing, is clearly a major barrier for many homeowners. Obviously, a homeowner who does not have, and cannot borrow, the funds for retrofit will not invest. Because lack of capital is a recognized barrier to conservation, utilities are spending large sums on financial incentive programs designed to encourage investment. The Tennessee Valley Authority, for example, has provided over $250 million at zero interest for loans to its customers (Tennesseee Valley Authority, 1982). Given the large sums involved in utility-sponsored loan programs, an understanding of the impact of these incentives on retrofit investment decisions has a high potential policy value. The purposes of this paper are to review evidence on the impacts of utilitysponsored subsidized loan programs, to discuss some methodological problems with the studies reviewed, and to suggest ways that better evidence may be obtained. A focus on utility-sponsored loan programs was selected for three reasons. First, because there are a

Research sponsored by the Energy Analysis Department, Electric Power Research Institute under Interagency Agreement Union Carbide Corporation contract W-74OS-eng-26 with the U.S. Department of Energy. Address reprint requests to Linda Berry, Energy Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830. 131

11033CtMl under

LINDA

132

large number of utility loan programs, several studies of these programs have been completed. Second, although a variety of financial incentives are available (including discounts, rebates, grants, tax credits, lower rates, and free materials and/or labor), subsidized loans seem to be the most common type of incentive offered by utility programs. Third, variations in the design and coverage of utility loan programs make it possible to consider a variety of experimental and quasi-experimental approaches to loan impact studies. Clearly, other types of government and utilitysponsored incentive programs (e.g., tax credits, rebates, grants, and so on) need examination too; but, a review of these other incentive types is beyond the scope of this paper. It is important to recognize, METHODS

however, that information about the impacts of loans is probably not generalizable to other types of incentives. Three types of studies of the impacts of loan programs are reviewed: (a) prospective surveys of customer intentions concerning retrofit under various loan conditions, (b) retrospective surveys of customer beliefs about the influence of loan availability on their actions, (c) studies of the behavior of loan program participants versus nonparticipants. After the findings of these impact studies are reviewed (Impacts of Loan Programs), their methodological weaknesses are discussed and suggestions for future research efforts are presented (Methodological Issues and Questions for Future Evaluations).

AND SOURCES

There is little easily accessible literature on the impact of financial incentives in utility-sponsored residential conservation programs. Energy policy, social science, and utility trade journals present little information on the subject. The issue has not been completely neglected, however. A number of utilities have conducted studies of customer responses to subsidized loan programs. Some state energy offices and public IMPACTS

BERRY

service commissions (PUCs) also have conducted studies or hearings dealing with the issue. To locate materials for this review, over 50 utilities and all state energy offices and PUCs were contacted. In addition, some related studies that were sponsored by private foundations (such as the John A. Hartford Foundation and the Consumers Union Foundation) and by the Department of Energy are included in this review.

OF LOAN

Prospective Surveys Most evidence on the impact of loan availability on investment decisions is based on prospective survey questions that ask consumers what they wouId do in hypothetical situations. Responses to such questions may or may not be related to actual behavior (Archer, Aronson, & Pettigrew, 1983). In general, responses to such survey questions show that loans are a minor influence on decisionmaking. Most respondents indicate that the availability/nonavailability of loans has little influence on their decisions. In a Seattle City Light Survey (Olsen & Cluett, 1979), for example, respondents were asked to rank eight factors that might affect their decisions to invest in home retrofit. The choice of “being able to obtain a loan on good terms” was ranked last of the eight factors presented. Fully half of the respondents said that this factor would have no influence on their investment decision (Olsen & Cluett, 1979). A New York State survey (Baumann, Cianciulli, Izzo, Rizzuto, Anderson, & Schultz, 1977) found that among homeowners who did not plan to add insulation, only about 1% said that an inability to obtain financing was a reason for their decision. A survey conducted for Northeast Utilities in November 1981 suggests that the main impact of the availability of discounted interest rate loans is to shift financing of improvements from savings and salary to the loan program. This study also suggests, however,

PROGRAMS

that a loan program would encourage some new investment, and that zero-interest loans will produce higher response levels than low-interest loans. The overall purpose of the Northeast Utilities customer survey (1981) was to determine potential interest in a possible utility funded conservation loan program. Mail surveys were sent to a random sample of 1,873 customers. About one third of the less than 50% who responded were renters (214) and about two thirds were homeowners (650). Before any mention was made of subsidized interest rates, respondents were asked to choose (for improvements they planned to make) among four financing methods: current salary or income, savings, a loan (with no interest rate specified), and a credit card. A response category of “do not plan to make any improvements” also was presented. Seventy percent of homeowners and 33% of renters stated that they planned to make improvements. Of those homeowners who planned to make improvements, 21% said they would take out a loan to pay for the retrofits. Among renters planning improvements, 9% intended to use loans. The majority of both groups said they would pay for improvements with current income or savings. When the option of a low-interest’ loan was in-

‘in the Northeast Utilities study, the low or discounted interest rate presented on the questionnaires as a 9%-14% interest rate.

133

Financial Incentives in Conservation Programs TABLE 1 PERCENTAGE OF CUSTOMERS ALREADY PLANNING IMPROVEMENTS WHO WOULD BORROW AT UNSPECIFIED INTEREST RATE VS. LOW INTEREST RATE Homeowners (Percentage) Unspecified Low-interest

interest

Nore. From Northeast

Renters (Percentage)

21 66 Utilities,

9 53

1981.

traduced, the percentages that would borrow increased to 66% of the owners and 53% of the renters who had previously planned to make improvements (Table 1). This suggests that when low-interest loans are available, retrofits that would have been financed from salary or savings will be financed through the program instead (Table 1). The Northeast Utilities survey (1981) also suggests that loan availability will encourage some customers to make investments they otherwise would not have made. When the option of low-interest loans was added to the previously presented financing options (i.e., current income, savings, loan with an unspecified interest rate, and credit card), some of those who had said they did not plan to make improvements switched to saying they would make improvements. If a low-interest loan program were available, 20% of the homeowners and 24% of the renters who had previously said they planned no improvements would take advantage of such loans. The respondents who switched from not planning to planning investments represented 6% of the homeowners and 16% of the renters in the total sample. For all renters and all homeowners combined, then, an increase of 7% occurred in the proportion of respondents who said they would make improvements (Table 2). That is, without subsidized loans, 62% of the total sample said they would make improvements; but, with the availability of low-interest loans, 69% said they would make improvements (Table 3). An additional shift occurred when the option of zero-interest loans was presented. This shift was 4 percentage

INCREASES

points. With zero-interest loans available, 73% would choose to make improvements (Table 3). There is considerable evidence that zero-interest loans will be used more extensively than loans with low interest charges. The Northeast Utilities survey (1981), for example, included several questions on the effect of varying interest rates on borrowing intentions. Respondents were asked to indicate the amount they would feel comfortable about borrowing at different interest rates. Two versions of the questionnaire were randomly distributed. One version had a 9% interest rate question and the other a 14%. Both versions included a zero-interest rate question. The amount of funds customers said they would borrow increases as interest rates decline. But, the demand for borrowed money was inelastic at all income levels. That is, there was a proportionately smaller change in the amount of money that would be borrowed than there was a decline in interest rate (Northeast Utilities, 1981). Stern, Black, and Elworth (1981) also reported findings that support the conclusion that zero-interest loans would be used by more customers than lowinterest loans. In the Northeast Package Program they studied, loans at 10.5%-12.5% interest were offered. About 30% of the Northeast Package Program users took out these loans. The rest paid cash. When survey respondents who had not contracted with the program were asked if they would use zero-interest loans repayable on the sale of the house, however, 49% said yes. Programs that have offered zero-interest loans, such as those of Pacific Power and Light, Portland General Electric, and the Tennessee Valley Authority, have had 40%-50% of audited customers choosing to use loans (Scherer, 1981; Hirst, Bronfman, Goeltz, Trimble, & Lerman, 1983). This is a higher rate of use than is typically found for low-interest loan programs. There is more to these programs than zero-interest loans, however. They also have convenience and consumer protection features. As discussed in the section on studies of actual behavior, identical loan offers produce widely varying responses when program charac-

TABLE 2 IN PERCENTAGE PLANNING TO MAKE IMPROVEMENTS LOW-INTEREST LOANS ARE AVAILABLE Homeowners

Percentage of those households previously planning no improvements that switched to planning improvements Percentage of all households switching to plans for improvements Note.

From Northeast

Utilities,

1981.

Renters

Total

WHEN

Sample

20

24

21

6

16

7

134

LINDA BERRY TABLE 3 OF TOTAL SAMPLE IMPROVEMENTS

PERCENTAGE

Sample

PLANNING

Percentage

Group

No subsidized financing Low-interest loans Zero-interest loans Note.

From Northeast

62 69 73 Utilities,

1981.

Behavioral

teristics such as consumer protection, convenience features differ. Retrospective

marketing,

and

Surveys

Retrospective surveys give a higher estimate of loan effects than prospective surveys. In general, responses to retrospective survey questions suggest that loan availability did have a substantial impact on investment decisions. A Pacific Gas and Electric (PC&E) Company study (1982) found, for example, that 56% of respondents said they would nof have installed the same retrofits without the loan. (Table 4) Similarly, a Bonneville Power Administration pilot program survey (Hirst et al., 1983) found that almost half (45%) of the loan progr;m participants said they probably or definitely would not have installed the same retrofit measures without the zero-interest loan. The phrasing of the loan impact question was especially interesting in the PC&E study. Most surveys ask, “Would you have installed the same equipment if the loan had not been available?” Typically, no mention is made of the time period involved. When no time period is mentioned, about half of the respondents usually say they would not have made the same retrofits without the loan. When asked, in the PG&E study, if the purchases would have been made in the same time period, in contrast, 75% of the respondents said they would not have been (Table 4). This suggests that

TABLE 4 IMPACT OF ZERO-INTEREST LOAN ON INVESTMENT DECISIONS

Investment

Loan Program Participants N = 308 (Percentage)

Decision

Would have installed at same time without loan Would not have installed at same time but would have installed within 12 months without loan Would not have installed at all without loan Don’t know

23

19 56 2 100

Note.

From Pacific

Gas and Electric,

loan programs significantly compress the time scale over which improvements are made. Installing a large number of improvements sooner, and simultaneously, may result in significant increases in the amount of savings realized in, say, a 5-year time period. Even if households would reach approximately the same level of energy efficiency at the end of such a 5-year period, earlier installation will lead to less overall energy use.

1982.

Studies

Two types of behavioral evidence on loan impacts are reviewed in this section, The first is data on participation rates in loan programs. The second is a study that compares the retrofit actions of loan program participants versus nonparticipants. In general, data on participation rates in loan programs suggest that loans produce higher levels of retrofit activity. Programs that offer loans have, in the aggregate, higher participation rates than programs that do not offer loans. Of course, these higher participation rates are not necessarily due to loan impacts. Programs offering loans usually have more aggressive marketing, and more consumer protection and convenience features as well. In addition, utilities that offer loans are likely to have a stronger commitment to conservation programs. As a result, the higher average p~ticipation rates of loan programs are a necessary but not sufficient condition for concluding that the availability of loans produces significant impacts (Scherer, 1981). Program evaluations sometimes conclude that high levels of usage are equivalent to high levels of impact. This is an incorrect conclusion, simply because it has not been determined what actions households would have taken in the absence of the program (Scherer, 1981). Similarly, it is incorrect to conclude that low usage of a financial subsidy necessarily indicates that it is not an effective incentive. As Scherer (1981) states: “Low usage of a particular subsidy may be a reflection of poor program design (a lot of ‘red tape’ is required to use it), or poor implementation (mismanagement) or a lack of advertising” (p. 15). The most important conclusion shown by an examination of program participation rates is that other elements of program design and implementation have larger impacts than those of the financial incentives alone. The same incentive offer will produce very different results when implemented by different utilities in different ways. The New York State Home Insulation and Conservation Program, for example, requires all regulated utilities in the state to provide financing to customers at interest rates of 9% and 11% (Scherer, 198 1). The individual utility programs operating under the plan show a wide divergence in the percentage of audited customers using the financing option; these percentages range from less than 1% to over 40% (Scherer, 1981). These differences seem to result from

Financial

CONSERVATION

ITEMS

Items

Incentives in Conservation

INSTALLED

TABLE 5 BETWEEN

APRIL

135

Programs

1981 AND JANUARY ZIP Participants N = 308 (Percentage)

Installed

Ceiling insulation Weatherstripped windows and outside doors Water heater insulation blanket Insulated heating ducts Water saving shower head Wall insulation Caulked windows and outside doors Converting incandescent or regular lights to fluorescent Insulated hot water pipes Heat absorbent/reflective material on windows Thermostat that automatically reduces temperature at night Ignition device to replace pilot Floor insulation Windows with double paned glass Storm doors Mean number installed

93’ 27’ 33’ 4 11* 5 14’ 9* 5 4 4 2 2 2 2.2

Note. From Pacific Gas and Electric, 1982, pp. VII-7. Responses were aided; multiple possible. “Statistically significant difference between participants and nonparticipants.

variations in the way the utilities run their programs (Scherer, 1981). Utilities in the state of Oregon, which also are required to provide financing, show a similar divergence of loan usage rates. Portland General Electric has provided loans to about 15% of all customers eligible for audits,* Pacific Power and Light to about 12770, but California Pacific National only to 0.1% (Scherer, 1981). Similarly, a process evaluation of a Bonneville Power Administration pilot residential weatherization program implemented in 11 participating utilities found that the number of audits completed per 100 eligible customers ranged from a low of 10 to a high of 58, and that the proportion of weatherization jobs financed at zero-interest ranged from 6 to 89 of each 100 homes audited (Lerman, Bronfman, & Tonn, 1983). These variations in penetration rates were due in part to differences in the size of utility conservation staffs, the nature of the utility service area and customer base, and the interest of management in promoting conservation (Lerman, Bronfman, & Tonn, 1983). These wide ranges of response to identical loan offers show that the impact of a financial incentive cannot be separated from the total context in which it is embedded. Although it is clear that the same incentive may have a significant effect in some program contexts and little or no effect in others, at least one study of the

‘Of those who choose to have audits, over 40% take out loans.

1982 NonParticipants N = 553 (Percentage) 7 6 8 4 7 4 4 4 3 2 1 2 2 1 1 0.6 responses

were

behavior of loan program participants versus nonparticipants shows that loan users are much more likely to install conservation equipment. Nearly all participants (93%) in a Pacific Gas and Electric (PG&E) Company zero-interest loan program (ZIP) installed ceiling insulation between April 1981 and January 1982. Only 7% of nonparticipants installed insulation in the same time period (Table 5). Although ceiling insulation was by far the most likely item to be installed by participants, they also were more likely to install nearly all of the items listed in Table 5. One reason that ZIP participants were more likely to install conservation equipment during the April 1981January 1982 time period is that they were much less likely to have such equipment before the program began. Before April 1981, when ZIP was introduced, for example, only 4% of households that later participated had ceiling insulation. Among nonparticipants, in contrast, 60% had ceiling insulation. The authors of the study state that these low levels of preprogram equipment ownership among participants suggest that ZIP disproportionately attracted those who were not already actively engaged in installing conservation measures and that ZIP encouraged acquisition of conservation equipment among those who had previously been least motivated to purchase these items. This conclusion seems only partially correct. The low level of preprogram ownership among participants would result in their being more attracted to the program. It is not clear, however, that participants are necessarily less motivated to purchase equipment. It may be, for example, that participants tend to have recently moved

136

LINDA BERRY

CONSERVATION

ITEMS

INSTALLED

TABLE 6 BETWEEN

APRIL

1981 AND JANUARY

1982

Based on Those Who Did Not Have Conservation Item as of April 1982

Items

ZIP Participants

Installed

Ceiling insulation Weatherstripped windows and outside doors Water heater insulation blanket Insulated heating ducts Water saving shower head Wall insulation Caulked windows and outside doors Converting incandescent or regular lights to flourescent Insulated hot water pipes Heat absorbent/reflective material on windows Thermostat that automatically reduces temperature at night Ignition device to replace pilot Floor insulation Windows with double paned glass Storm doors Note.

From Pacific

Gas and Electric,

1982, pp. VII-g.

Nonparticipants

96% 43 42 7 16 8 18

Estimated Installations per 100 Non-owners Resulting from ZIP 78 32 32 2 6 -

18% 11 10 5 10 8 7

10 6 4

11

5 4 2

5 2 2 2

4 -

-

-

2 2 2 Responses

into unimproved homes. If this is the case, the unimproved condition of the homes would not mean that ZIP participants are inherently less motivated to purchase equipment than nonparticipants. Although it is not entirely clear how to interpret the lower preprogram level of improvements in participant households, it is clear that estimates of ZIP effectiveness should adjust for the fact that participants owned less before. Without such an adjustment, a self-selection bias is likely to influence the results. That is, because households that have few improvements choose to participate in ZIP and only households that lack improvements can install them, higher unadjusted rates of installation among participants do not necessarily indicate an effect due to the program. Households that lack improvements are likely to have higher rates of installation whether or not a program is offered. The authors of the PG&E study recognize the possibility of this self-selection bias and have made some adjustment for it. They calculate an estimate of ZIP effectiveness by comparing rates of installation among program participants to rates among nonparticipants who did not already own the item (Table 6). For example, between April 1981 and January 1982, 96 out of

were aided;

1 multiple

responses

were possible.

every 100 ZIP participants who did not already own ceiling insulation installed it. Among nonparticipants who did not already own ceiling insulation, 18 out of 100 installed it. Thus, the authors conclude, if participants were like nonparticipants, it would be expected that 18 out of 100 ZIP participants who did not have insulation would have installed it even if they had not taken out a loan. The difference of 78 (96 minus 18) is, then, attributed to the influence of ZIP participation. As the authors point out, this analysis assumes that the use of ZIP loans was the cause of the installation of a conservation item and not the other way around. In other words, it assumes that ZIP motivated the conservation activity and not that households that were going to install the items anyway chose to participate in ZIP. Unfortunately, given the design of the PG&E study, there is no way of verifying this assumption. Assuming the converse, i.e., that participants choose ZIP because they had already decided to install conservation items, is equally plausible. Thus, while this study is to be commended for recognizing the problem of self-selection, its method of dealing with the problem is insufficient.

METHODOLOGICAL ISSUES The review in the preceding section shows the importance of two major validity threats in loan impact studies. First, some self-selection process is undoubtedly

occurring and better methods of controlling its effects are required. Second, other elements of program design and implementation (such as marketing, con-

Financial Incentives in Conservation Programs sumer protection features, and management commitment to the success of conservation programs) have effects that overwhelm the effects of the incentives alone. Because the effectiveness of incentives is clearly dependent on these other program characteristics, systematic observation or experimental manipulation of these factors is essential. Several approaches to dealing with these problems are discussed here. The problem of self-selection is ubiquitous in evaluations of conservation programs. It is clear, for example, that customers who choose to obtain home energy audits differ from those who do not (Berry, Soderstrom, Hirst, Newman, & Weaver, 1981). Audit program participants have higher incomes and education, and a greater awareness/concern with energy conservation. They also own larger than average homes. As a result, people who choose to obtain audits are more likely to take conservation actions than the general population even when no programs are available. Thus, it is incorrect to attribute all actions taken by audited customers to the influence of the audit. Similarly, because persons who choose to obtain loans are a subset of audited customers, they almost certainly are not a cross section of all customers. In addition, loan users will probably differ from the population of audited customers as well. The percentage of audited customers who go on to obtain loans may vary from 2% to over 50%. Self-selection processes are undoubtedly operating in ways that make loan users different from nonusers. That is, loan users probably have characteristics that would cause them to behave differently from nonusers even if no loans were available. Loan users have, for example, more energy inefficient dwellings, higher incomes, and a greater need for retrofit measures (Pacific Gas and Electric, 1982). They also are likely to have larger homes that require higher expenditures for weatherization. They may plan to stay in their homes for longer time periods, or they may find longer payback periods more acceptable. All of the ways in which loan users differ from nonusers have yet to be documented. It is reasonable to assume, however, that users do differ in ways that would affect their retrofit decisions even if no loans were available. As a result, it is incorrect to attribute all differences in the retrofit actions of loan users versus nonusers to the influence of the incentive. Even if loan users and nonusers differed only in the strength of their intention to invest, higher levels of retrofit actions among loan users would be due, in part, to this difference between the groups. Thus, customers who are offered loans but choose not to use them cannot be a suitable comparison group. Controlling for self-selection requires a design that provides measurements on a comparison group that is not offered the incentive. If only groups offered the in-

137

centive are examined, it is extremely difficult to determine to what extent the actions of loan recipients are due to the effects of the loan and to what extent they are due to the composition of the group that chooses to obtain loans. One easily implemented approach to controlling for self-selection is to offer loans to customers in some service areas but not to those in others. For example, some parts of a utility service area might be included in a pilot program that offered loans and other parts excluded. The PG&E ZIP program did, in fact, follow this pattern. Only one division in the PG&E service area was offered loans in 1981. In 1982 the ZIP program was offered to seven more divisions (Pacific Gas and Electric, 1982). Unfortunately, no comparisons were made between groups in divisions that were and were not offered loans in 1981. The purpose of first offering loans to only one PG&E division was to test the program on a small scale before expanding it to a larger area. The potential value of this pilot program strategy for evaluation purposes was not utilized. It should be noted that the PG&E pilot loan program situation is not unique. There are several examples of programs that have been or will be offered to some groups of customers and not to others. In the Northern States Power service area, customers living in the city of St. Paul are eligible for loans that are not offered to customers outside of the St. Paul area. In the Bonneville Power Administration (BPA) area, a pilot weatherization audit and loan program began in some BPA utility service areas 3 years before a similar program was offered to all BPA utilities. In Oregon, all utilities must offer 6.5% loans while some have voluntarily offered zero-interest loans. A comparison of a group offered zero-interest loans with a group offered only 6.5% loans should yield some information on the effects of varying interest rates. In the TVA region, (beginning on October 1, 1982) distributors had the option of choosing which, if any, retrofit measures to offer to finance with zero and which with market interest rates. Some distributors are offering more comprehensive packages than others. Customer response to these differing types of retrofit financing offers may suggest the relative effectiveness of various incentive packages. Although natural experiments, in which some service areas offer incentives and others do not, occur frequently and offer an easily implemented evaluation design that controls for self-selection, they have some important drawbacks. The potentially fatal flaw of designs that compare responses in two or more service areas offering different incentives is that implementation differences may affect response levels more strongly than incentive differences. If the service areas use different marketing techniques, offer different convenience or protection features, or show varying

LINDA BERRY

138

levels of commitment to the program, response will vary independently of any incentive effects. Designs that create comparison groups within the same service areas are needed to avoid this difficulty. Two approaches to creating a suitable comparison group within the same service area are: true random assignment of individuals to treatments and a waitinglist control procedure. These approaches are more difficult to implement than the multiple service area designs, but they will yield more valid results. The feasibility of implementing a design that randomly assigns individual customers to different incentive treatments is supported by the fact that such a procedure has been used in previous evaluations of residential conservation programs. Both a Southern California Edison Company (Johnston, 1982) and a Lawrence Berkeley Laboratory (Meier, 1980) evaluation of the relative effectiveness of audit types used random assignment of customers to audit treatments. A Pacific Gas and Electric Company evaluation (1979) of a pilot program also used random assignments of QUESTIONS

customers to audit types and to a no-treatment control group. A waiting list control procedure also has been successfully implemented in evaluations of conservation programs (Price-Waterhouse, 1980). Detailed discussion of how to implement these approaches and of their advantages and disadvantages can be found in Soderstrom, Berry, and Hirst (1981); Berry, Tsao, and Hirst (1983); and Archer, Aronson, and Pettigrew (1983). The greatest strength of designs with random assignment or waiting-list control procedures is their high internal validity. They may have low external validity or generalizability of results, however. A number of replications are needed to increase the generalizability of results. Factorial designs that explore the interaction of incentive effects with other program components (e.g., marketing strategies, consumer protection features, and so on) are essential to determine the conditions under which loan offers increase program effectiveness.

FOR FUTURE

Assuming that a design in which some groups are offered a financial incentive and some are not has been implemented, let us now consider what questions the evaluation should address. Because loan eligibility is usually contingent on receiving an audit, it will be assumed that this is the case in the rest of this discussion. (It is important to note that the PG&E program did not require customers to receive an audit before applying for a loan. This is unusual, as most utility loan programs do require a prior audit.) Assuming that audits are required before obtaining loans or other financial incentives, three major evaluation issues seem to be important: 1. How does the offer of a financial incentive affect program participation (as measured by requests for audits)? That is, are financial incentives useful marketing tools? Which ones work best in motivating additional audit requests? By what types of households? 2. How do incentive users differ from audit program participants who do not use the financial incentives? 3. How effective are financial incentives in stimulating additional retrofit and subsequent energy savings (relative to audit programs without financial incentives)? More specifically: Do incentive users take more and/or different conservation actions, do they choose more cost-effective actions, take more auditor-recommended actions, invest more in retrofit, and/or make more improvements in a shorter time period?

EVALUATIONS

To address these issues, data must be collected on at least three customer groups (Groups 1, 2, and 3) as shown in Table 7. Group 1, the comparison group, consists of audited customers not offered an incentive. The two treatment groups (Groups 2 and 3) consist of customers who were offered an incentive and used it and of customers who did not. If one wishes to study the relative effectiveness of more than one type of incentive (e.g., loans with different terms or interest rates, or loans versus rebates), then additional treatment groups (such as Groups 4 and 5 in Table 7) can be added. It is important to examine both incentive users and nonusers so that the nature of the self-selection process can be studied. In addition, if pooled data for both incentive users and nonusers (Group 2 + Group 3) are compared to data on Group 1, self-selection is eliminated as a plausible explanation for outcome dif-

TABLE

CUSTOMER

GROUPS

7

FOR WHICH

DATA WERE COLLECTED

Comparison Group Group 1 = Audited customers not offered an incentive. Treatment Groups Group 2 = Audited customers offered an incentive who choose to use it. Group 3 = Audited customers offered an incentive who choose not to use it. Group 4 = Audited customers offered a second type of incentive who choose to use it. Group 5 = Audited customers offered a second type of incentive who choose not to use it.

Financial Incentives in Conservation Programs ferences between the comparison and treatment groups. If, for example, Groups 2 and 3 combined show a higher rate of retrofit actions than Group 1, this is unlikely to be due to a greater preprogram need for retrofit improvements among Groups 2 and 3 combined. In contrast, if Group 2 is compared only to Group 3, higher rates of retrofit in Group 2 could be due to the poorer initial conditions of their dwellings. Of course, to the extent that the offer of an incentive affects audit request decisions, the composition of Group 1 may differ from the composition of Groups 2 and 3 combined. That is, the offer of an incentive may both increase the number of audit requests and change the composition of the group that requests audits. Thus, the first analysis problem is to determine if the type of people who request audits differs with and without incentive offers. For example, one might compare the income, education, dwelling size and characteristics, and prior conservation activity of Group 1 versus Groups 2 + 3. Differences between these groups in terms of mean values and distributions (and the statistical significance of these differences as determined by t tests or chi-square tests) should be computed and compared. If these groups are not significantly different, one can conclude that the incentive offer has little effect on what type of customer responds. If the groups do differ, the nature of the differences will show how the incentive affects customer response. A comparison of Group 2 with Group 3 along the same kinds of dimensions (i.e., income, education, dwelling characteristics, and so on) will allow one to address the issue of how incentive users and nonusers differ. Again, comparisons of mean values and distributions are appropriate. Analysis of the third major issue (the effects of the incentive on retrofit patterns and energy savings) should take into account the findings from analysis of the first two issues. That is, if the incentive offer does change the composition of the group that requests audits, then these compositional differences may be an important cause of observed differences in retrofit patterns and energy savings. When an incentive is offered, for example, more households that lack many weatherization measures may choose to request audits. If this is the case, higher retrofit rates among groups offered incentives may be largely due to this effect on audit request decisions. Similarly, if the dwellings of incentive users have fewer weatherization features before program participation, this may largely explain why incentive users have higher retrofit rates than nonusers. Data on the preprogram weatherization status of dwellings for Groups 1, 2, and 3 can, of course, be assembled from audit records. Data on actions recommended also can be assembled from audit records.

139 TABLE

OVERALL

8

EVALUATION

DESIGN Time 2

Time 1

Group 1 2 3

U, A U, A U, A

XP Xnp

U, S U, S U, S

Audited customers not offered an incentive. Audited customers offered an incentive who chose to use it. offered an incentive who Group 3 = Audited customers chose not to use it. xp = Loan, rebate or other incentive offered and chosen. Xnp = Loan, rebate or other incentive offered and not chosen. Net consumption data from utility billing u= records. A= Audit data on dwelling characteristics and actions recommended. Survey data on retrofit actions taken and on s= demographic and attitudinal characteristics. Group Group

1 = 2 =

Another essential data set is records of what recommended actions were actually taken. For loan users, utility records probably will provide this data. For nonusers, self-reports of what actions were taken can be collected in telephone surveys.3 Data on the number and types of retrofit actions taken should be supplemented, if possible, with information on the costs and cost effectiveness of the retrofits chosen. Information on the costs and cost effectiveness of the actions chosen can be used to determine if customers who are offered financial incentives are more likely to take more costly and more cost effective actions. Finally, estimates of the energy savings due to programs (both with and without incentives) can be obtained with a variety of techniques that differ markedly in their analytical complexity. Discussions of a number of estimation approaches can be found in an evaluation plan prepared for the Bonneville Power Administration (Hirst, Berry, Bronfman, Johnson, Tepel, & Trimble, 1982). Some of the methodological issues that are considered in the plan (Hirst et al., 1982) include adjustments for weather and price effects and the selection of appropriate comparison groups. A summary of the overall evaluation design discussed in this section is given in Table 8.

3As Dillman (1978) has shown, telephone surveys can yield similar information and acceptable response rates (over 70%) at much less cost than face-to-face interviews. Because self-reports are likely to be inaccurate, however, observations of actual installations should be made on at least a sample of no-loan households.

140

LINDA BERRY CONCLUSIONS

The literature on the role of financial incentives in home retrofit decisions offers a number of valuable insights on the issue. It does not, however, allow one to reach any definite conclusions about the impact of incentives on levels of investment and energy savings. Determining the impacts of financial incentives is a complex research problem because of the need to understand the effects of a number of interactive factors. The availability of financial incentives is only one component of a residential conservation program. Advertising effectiveness, management commitment, consumer protection, and convenience features often have more important effects on customer response. Because incentive effects are clearly dependent upon other program characteristics, systematic observation or experimental manipulation of such nonfinancial factors is required if loan effects are to be understood. In general, households do borrow at higher frequencies when zero-interest loans are available. In addition, households that take out loans do make larger investments in retrofit. But, these larger levels of investment could result, in part, from self-selection. That is, those households that have already decided to make more extensive installations also may be those that decide to obtain loans. Some method of estimating what actions households would have taken in the absence of a loan offer is required if the loan’s impact is to be determined. A comparison group of households that are not offered loans is needed for conclusive research on loan impacts. Determination of the impact of financial incentives

on retrofit investment is important for several reasons. First, the amount of funds involved is substantial. The TVA program, for example, has provided over $250 million at zero interest for loans to its customers (Tennessee Valley Authority, 1982). Some assurance that such large subsidies actually produce results is needed. Second, there are serious equity issues involved. If mainly upper income households use the subsidies, poorer customers who are paying for, but not benefitting from the incentives, may be unfairly burdened. Of course, some utilities, with large differences between the average and marginal cost of power, can justify such a distribution of costs and benefits because conservation will result in reduced rates for all customers. In other words, all customers will benefit even if some benefit more than others. For many utilities, however, the cost effectiveness of conservation programs to all customers is uncertain. Participants in a program will always benefit because most retrofit investments are cost effective even without subsidies. Determination of program cost effectiveness from a nonparticipant, or a utility, perspective however, requires an understanding of the incremental effects of subsidies on the levels of investment and energy savings achieved as well as an understanding of demand reduction effects on overall costs for power. Thus, decisions about how extensive a subsidy should be adopted by a given utility need to be based on information about the incremental effects of financial incentives. This information is not yet available. Obtaining it is an important task for future research.

REFERENCES ARCHER, D., ARONSON, E., & PETTIGREW, T. (1983). An evaluation of the energy conservation research of Caltfornia’s major energy utility companies, 1977-1980. Santa Cruz: University of California, Energy Conservation Group. BAUMANN, E. N., CIANCIULLI, L. R., IZZO, A. M., RIZZUTO, J., ANDERSON, S., & SCHULTZ, M. (1977). New York residential insulation survey, final report. Albany: New York State Electric Utilities, New York State Public Service Commission and New York State Energy Office.

HIRST, E., BERRY, L., BRONFMAN, B., JOHNSON, K-E., TEPEL, R., & TRIMBLE, J. (1982). Evaluation plan for the Bonneville Power Administration residential energy conservation programs, Vols. I and II. (ORNL/CON-94). Oak Ridge, TN: Oak Ridge National Laboratory. HIRST, E., BRONFMAN, B. H., GOELTZ, R., TRIMBLE, J., & LERMAN, D. (1983). Evaluation of the BPA Residential Weatherization Pilot Program (ORNL/CON-124). Oak Ridge, TN: Oak Ridge National Laboratory.

BERRY, L., SODERSTROM, J., HIRST, E., NEWMAN, B., & WEAVER, R. (1981). Review of evaluations of utility home energy audit programs (ORNL/CON-58). Oak Ridge, TN: Oak Ridge National Laboratory.

JOHNSTON, R. (1982). Residential conservation service audits: A behavioral evaluation of four types. Rosemead, CA: Southern California Edison Company.

BERRY, L., TSAO, H., & HIRST, E. (1983). Design options to test the effects of financial incentives in a utility conservation program: TVA’s heat pump water heater program. (ORNL/CON-125). Oak Ridge, TN: Oak Ridge National Laboratory.

LERMAN, D., BRONFMAN, B. H., & TONN, B. (1983, October). Process evaluation of the Bonneville Power Administration Residential Weatherization Power Program (ORNL/CGN-138). Oak Ridge, TN: Oak Ridge National Laboratory.

CROSSLEY, D. J. (1983). Identifying barriers to the success of consumer energy conservation policies. Energy, 8(7), 533-546.

MEIER, A. (1980). The final report of the energy conservation inspection service. (LBL-10739) Berkeley, CA: Lawrence Berkeley Laboratory.

DILLMAN, D. A. (1978). Mail and telephone surveys: design method. New York: John Wiley and Sons.

The total NORTHEAST

UTILITIES.

(1981).

Consumer

survey

of potential

Financial interest in a conservation Policy Decisions.

loan program.

Incentives

in Conservation

Hartford, CT: Research for

OFFICE OF TECHNOLOGY ASSESSMENT. (1979). Residential energy conservation. Washington, DC: U.S. Government Printing Office. OLSEN, M., & CLUETT, C. (1979). Evaluation of the Seattle City Light neighborhood energy conservation program. Seattle, WA: Battelle Human Affairs Research Center. PACIFIC GAS AND ELECTRIC COMPANY. (1979). Final Report: Residential energy utilization analysis- Concord Pilot Study program evaluation. (MR-797). San Francisco, CA: Author. PACIFIC GAS AND ELECTRIC COMPANY. (1982). ZIP for/o wup survey (MR-82-1.) San Francisco, CA: Marylander Marketing Research. PRICE-WATERHOUSE. (1980). Final report on technical assistance provided to the states and territories in evaluating state energy conservation programs. Washington, DC: U.S. Department of Energy.

Programs

141

ROSENBERG, M. (1981). Making RCS work: Analysis of the residential conservation market and its implications for the operation of RCS. Boston, MA: Technical Development Corporation. SCHERER, J. B. (1981). Residential energy conservation andfinancial incentives. Richland, WA: Pacific Northwest Laboratory. SODERSTROM, J., BERRY, L., & HIRST, E. (1981). The use of metaevaluation to plan evaluations of conservation programs. Evaluation and Program Planning, 4, 113- 122. SOLAR ENERGY RESEARCH INSTITUTE. (1981). A new prosperity: Building a sustainable energy future. Boston, MA: Brick House Publishing. STERN, P. C., BLACK, J. S., & ELWORTH, J. T. (1981). Home energy conservation: Programs and strategies for the 1980s. Mount Vernon, NY: Consumer’s Union Foundation. TENNESSEE VALLEY AUTHORITY (1982, September) personal communication with staff.