The stability of PRISM estimates

Energy and Buildings, 9 (1986) 149 - 157

149

The Stability of PRISM Estimates JAMES RACHLIN*, MARGARET F. FELS and ROBERT H. SOCOLOW

Center for Energy and Environmental Studies, Princeton University, Princeton, NJ 08544 (U.S.A.) (Received September 1985)

SUMMARY

The stability o f the PRISM parameters against changes in time periods is studied, for a better understanding of the problems likely to emerge when meter readings are missing or when only bimonthly data are available. Consecutive monthly readings spanning intervals from 7 to 24 months, and oneyear data sets with gaps, are examined. The results indicate that 12 monthly readings are optimal for the most reliable PRISM results. The NA C index is far less sensitive to missing or insufficient data than are the individual parameters, whose stability is affected by the seasons included or omitted in the estimation period. Results from less than ten consecutive readings, or from less than nine readings spanning a full year but with large gaps, should be scrutinized carefully. Reliable savings estimates are available from bimonthly data, as long as missing readings are not prevalent. INTRODUCTION

In this study, we address some of the practical aspects o f applying the Princeton Scorekeeping Method (PRISM) to whole-house data. In particular, we examine the stability of the parameter estimates: the importance o f the choice of estimation period when more than one year of data is available, and the limits of the model's ability to characterize a house if less than one year of data has been collected or if one year of data is spanned b y fewer than 12 meter readings. As the model's primary purpose is to monitor energy c o n s u m p t i o n and savings, the analyses are tailored to that objective. (PRISM is d e ~ cribed in the introductory paper to this issue.) *Present addren: Booz-Allen and Hamilton, Inc., New York, NY 10178, U.S.A.

Starting with periods of greater than, and proceeding to periods of less than, one year, we examine the effects of varying the length of the estimation period. In the former case, there i s a conflict between statistical and practical considerations. From a statistical point of view, one generally wants to use all the data available in a regression. With regard to scorekeeping, however, an important argument for limiting the estimation period to 12 months is that consumption patterns change from year to year. Moreover, the inclusion of t w o winters (for example) in the estimation period gives undue weight to the winter periods in determining the parameter estimates. When less than one year of data is available, a different question is central: at what point is the insufficiency of data so severe that little meaning can be derived from the three-parameter approach (PRISM)? Of particular interest is the exclusion of winter or summer from the estimation period. A different problem plaguing scorekeepers is gaps in m o n t h l y data, wherein a single reading summarizes more than one m o n t h o f consumption. With an emphasis on bimonthly data, we explore the problem in detail for one house, and redo an entire savings analysis for a group of houses as if only bimonthly data were available, in order to compare it with the original scorekeeping results obtained from monthly data. S o m e stability analyses of the Minneapolis scorekeeping group, whose work is represented in this volume, lend support to many of the findings reported here [I]. In specific cases we quote their results. More generally, we want to emphasize the importance of companion studies such as theirs, which are essential for assessing the generalizability of results. Ultimately, treatment of missing data should be incorporated into the PRISM softElsevier Sequoia/Printed in The Netherlands

150 76--

75 74~

LENGTH OF I ESTIMATION PERIOD [~ 12 MONTHS [ ..... [8MONTHS 24MONTHS .

.

.

.

.

-- 21000

//

/_

.

/ 731 ~_

tO

.....

',,, .° .

t

\ .

.

.

.

.

...........

651-

I

I OI JUL 77

\ I

I OI JAN 7S

-

,ooo

yJ

68

"

,

/

\\

~_

I

I OI JUL 78

I

I OI JAN 79

,L

I 18000 OI JUL 79

PERIOD BEGINNING DATE

Fig. I. Estimates of NAC for estimation periods of lengths 12, 18 and 24 months, starting on the date shown. A house from Denver is used. Arrow on right-hand scale indicates median 12-month value used in Fig. 2.

ware. This will be done as experience grows, so that screening criteria to get rid of 'bad' houses do not lose many 'good' houses as well. CONSECUTIVE MONTHLY DATA OF VARIABLE LENGTH - - M O R E THAN 12 MONTHS

Our first stability analysis explores the issue of whether to use more than 12 months of data when they are available. Using a Denver data base which contains 42 months of meter readings for a number of houses [ 2 ] , we apply a 'month-swapping' technique to one particular house*. We derive PRISM estimates for the first 12, 18 and 24 months of data, then move each estimation period forward b y one m o n t h at a time, generating new PRISM estimates at each step until all the data are covered. (In the longer version of this paper [3], results for 12, 13 . . . . , 18 as well as 24 months are shown.) Figure 1 summarizes the resulting estimates o f Normalized Annual Consumption (NAC), as a function of the period's starting date. The 12-month data show t w o distinct levels, roughly 10% apart, over the course of the study period. The 24-month estimates *The data are from a retrofit experiment sponsored by the Electric Power Research Institute. A house with similar pre- and post-retrofit period estimates was selected. Generalizability o f the single-house results is discussed in ref. 3.

happen always to contain both consumption levels at once and remain relatively flat. The 18-month estimates lie, generally, between the 12- and 24-month estimates. All this is reasonable: the variability observed in NAC (and similarly in the other estimates; see [3] ) is reduced by increasing the length of the estimation period, primarily because more consumption information is being incorporated in the estimation, and because changes in consumption over time are thereby averaged. Figure 2 shows the standard error of NAC, se{NAC), for each estimation period and period length used in Fig. 1. We have scaled the y-axis by the median 12-month estimate of NAC { 1 9 6 5 0 kWh/year, or 70.7 GJelec/ year). The standard errors from the 24-month estimation periods remain relatively constant, always between 2.0% and 2.7%, while the 12-month standard errors range from 1.6% to 3.4%. The 18-month standard errors cover approximately the same range of values, and show the same overall trends, as their 12m o n t h counterparts. However, t h e y are not bracketed by the 12- and 24-month estimates, nor are they necessarily smaller than the 12m o n t h errors. One interesting feature of shifting the estimation period is the importance o f individual m o n t h l y data points. As October 1978 is picked up by the 12-month and 18-month periods, for example, the standard errors increase from 1.7% to 3.1% and from 1.2%

151 O.O~

LENGTH OF I ESTIMATION PERIOD I - 12 M O N T H S I ..... 18MONTHS / ....... 24MONTHS| / I

"C

x X 5%

0+0~ / /--/

~

/

....... ~ \ , ~

/ / /

'~ z eJ m

0.02

(~ "-......-'./..............

•*

\

/

\\

'X

I

J

I !

0.01 OI JAN77

I

I OI JUL77

I

I OI JAN 7 8

I

I OI J U L 7 8

PERIOD

I

I OI JAN 7 9

I

I OI J U L 7 9

I

I

I ,

OI JAN 8 0

BEGINNING DATE

Fig. 2. Estimates of the standard error o f NAC as a fraction of the median 12-month estimate for estimation periods o f lengths 12, 18, and 24 months, starting on the date indicated. See caption to Fig. 1.

to 2.0%, respectively. With t w o years of data, the standard errors are more constant because individual points no longer carry such power. This analysis strongly suggests that, despite the greater stability of the estimates gained b y lengthening the estimation period from one year to t w o years, interesting details are lost. In most scorekeeping applications it will be desirable to forfeit the stability offered b y longer estimation periods in order to reveal changes in consumption over approximately annual periods. When the estimation period is lengthened from one year to 18 months, we find no systematic reduction in the error, and only moderate improvements in the stability of the estimates. We therefore recommend that a 12-month estimation period be chosen even when more than 12 months of data are available. One exception might be when there are substantial gaps in the data for one year. Because of the presence of gaps, the Minneapolis study was able to increase the fraction of 'good' houses from 62% to 75% when the m a x i m u m length of estimation period was extended from 12 to 16 months [1]. (Their selection criteria, applied to PRISM results, rejected houses with R 2 < 0.95 or with s e ( N A C ) > 5% of NAC.) The relative insensitivity of our NAC results to the inclusion of a few extra months is reassuring confirmation of their approach.

CONSECUTIVE MONTHLY DATA -- LESS THAN 12 MONTHS N o w suppose that less than 12 months of consecutive data are available for parameter estimates. When key months for the determination of individual parameters are missing from the interval, the determination of the PRISM estimates will become problematic. To investigate this, we proceed with a monthswapping analysis analogous to the previous one, but with the size of the estimation period reduced rather than lengthened. Estimates of NAC derived from much less than 12 months of data depart substantially from the 12-month estimates, as illustrated in Fig. 3, which shows results from estimation periods of 7, 9 and 12 months as months are swapped across the 42 months of data. The 7-month estimates are seen to be as much as 14% away from the corresponding 12-month estimates. The 9-month estimates are never more than 5% different from the 12-month estimates, nor are the 10-month estimates, and the l l - m o n t h estimates differ from the 12-month estimates by less than 2% (see ref. 3 for complete results for 7, 8 . . . . . 12 months). Figure 4 shows se(NAC) for analogous estimation periods, with the axis scaled as in Fig. 2. In a b o u t half the cases, the 7-month error estimate is small, in some cases smaller than

152 24000

85'

LENGTH OF ] ESTIMATION PERIOD

80:

.......

.....

(D

~::,..

®

75

12 q MONTHSI n ~r - M-NTH- / 7MONTH~J

,•, • •

'

::... : : :

. • •

::

,':

^

"' :

"~ 7o

~

....."

.: 2.

I

:

.\

•

:

L

/ /

19-"

i

.:-~

:

""

"<:..

i

22000

/

-

/S

:

.

2oooo

---"

•

o .~

;

:

o

~ s5 ;:

~

:~../

•

/

.".. ."

."

.-

18000

, : -:

...." 16000 55 I

I

I

OI JAN 7 7

I

OI J U L 7 7

I

I

I

OI JAN 7 8

I

OI J U L 7 8

I

I

I

OI JAN 79

I

OI JUL79

I

OI JAN 80

ESTIMATION STARTING DATE

Fig• 3. Estimates o f N A C for periods o f lengths 7, 9 and 12 m o n t h s , starting o n t h e date s h o w n . The 1 2 - m o n t h estimates are the s a m e as in Fig. 1, w i t h this scale e x p a n d e d b y a factor o f 3. A r r o w o n right-hand scale indicates m e d i a n 1 2 - m o n t h value u s e d in Fig. 4.

0.16,

LENGTH OF ESTIMATION pERIOD I 12 MONTHS

0.I~

..... .......

9MONTHS

." !!

7MONTH~

:!

0.12

:! :

~

0J¢

.2

.o ,i •°

ODe

u z

O.O6 o• -

O.O~

"

"

i

"/---, LI

-,

J

i P".:

:

0.O~

-~: ....

-~\ \ •~ "' i W

".. :

'.

.•'..

" :

:

"

\--,':

/

/\

"

/.•:

•

~

:

" "•."•••- . . . . ~ - - - 3 %

•'"

'..."

O

i

OI JAN 7?

I

I,

OI J U L 7 7

i

I

I

I

I

I

OI JAN 78 OI JUL78 OI JAN 7 9 E S T I M A T I O N STARTING DATE

I

I

OI JUL 79,

I

I

I

OI JAN SO

Fig. 4. Standard error o f N A C as a percentage o f the m e d i a n 1 2 - m o n t h estimate for e s t i m a t i o n periods o f lengths 7, 9 and 12 m o n t h s , starting o n the date indicated• See c a p t i o n to Fig. 3. The 1 2 - m o n t h errors are the s a m e as in Fig. 2, w i t h this scale e x p a n d e d b y a factor o f 4.

the 12-month error, which is generally less than 3% of NAC. However, when the estimation period starts in certain months, the error can be very much larger: its largest value is 15% of NAC for a 7-month estimation period. This is in contrast to the 12-month estimation periods, for which the maximum error is 3.5% and the month-to-month variation is not sharply peaked• For intermediate estimation periods, the highest spike is between the 7-

and 12-month values: for a 9-month estimation period, it is 6%, while for 10 months it is 4% of NAC. These seasonal spikes and valleys are a straightforward reflection of the fact that both winter and summer months are required for an accurate determination of NAC {and, even more strongly, for a, ~, and r). When the 7-month estimation period starts between February and April, the consumption during

153

when no winter months are included. The 7m o n t h estimate of the error is generally small, although almost always larger than the corresponding 12-month error. When the estimation period starts in February, March or April, however, the standard error is as much as 10 times larger than it is at other times of the year. If fewer than 7 months of data are used (not shown), the standard errors become larger and the spikes in the errors even more pronounced. The reason for the structure of Figs. 5 and 6 is clear. The heating slope can only be well determined when months representing a wide range of average heating degree-days are present. A 7-month estimation period, starting in spring and ending in fall, is thus deficient. As expected, when estimation periods of 8, 9, 10 and 11 months are explored, periods with spring starts extend successively further into the winter months and the peak standard errors diminish. Analogous plots for T and ~ show similar instability, b u t in different months (see ref. 3). For ~, the standard errors peak when the estimation period omits summer months, as would a 7-month period starting in September or October. As verified in our seasonality study reported in this issue [4], the ~ estimate from 12 months of data accurately reflects summer consumption. Therefore,

the following winter is always missing from the estimation, and when it starts from August to October, summer consumption is missing. These are precisely the starting points for which the spikes in the standard errors occur. On the other hand, seven months starting in either mid-winter or mid-summer include b o t h winter and summer, and this leads to a seemingly well determined NAC. For these cases, se(NAC) may be even smaller for the 7m o n t h than for the 12-month estimate, a reflection of a good fit using fewer points. The 7-month estimates of NAC, however, m a y differ b y considerably more than one standard error from the corresponding 12m o n t h estimate. The winter starting date of January 1978 provides a good example (see Figs. 3 and 4): se(NAC) is only 1% of NAC when 7 months are used, vs. 3.5% for 12 months, b u t the t w o corresponding NAC estimates differ by 8%. Because of the instability of the 7-month estimates, we view these periodic small standard errors as deceptive rather than reassuring. We turn now to a similar examination of the individual parameters, ~, ~, and a. Figures 5 and 6 show, respectively, the heating slope j~ estimated from 7, 9 and 12 months of data and the corresponding standard errors. The 7-month estimates sometimes oscillate wildly

LENGTH OF " ESTIMATION PERIOD

40O

- ..... ....... 5O0

":

I2MONTHS 9 MONTHS 7MONTHS

~

® %..I :

x":

A

" "' " ~

~ ~

.

;'.

~

'

:

:/~

:

:

•

~

,,..

•

U o '~. 2 0 0 _=

:...~

@

<~

:..:"~,, ...: \

.1~

\

:

.

.

•

..__

.

I00

0

I OI JAN 77

I

I

o, J u c 7 ?

I

I

O, +AN Te

I

I

m JULTB

I

I

O, +AN T9

I

I

O, JUL ;'9

+ -

'

O, J ~

SO

E S T I M A T I O N STARTING DATE

Fig. 5. Estimates of heating slope (j~) for estimation periods of lengths 7, 9 and 12 months, starting on the date indicated. Arrow on right-hand scale indicates median 12-month value used in Fig.,6.

154 1.6 LENGTH OF ESTIMATION PERIOD 1.4,

i

12 MONTHS 9MONTHS 7MONTHS

..... .......

::

1.2, o I.C

£ ==

0.8

0.6 • .

".

:" 0.4

: LU

•

II

"

"

~

" ....

\- - - -':::,'"" . . . . ~ 0

i OI JAN 77

/'

:..

I

I OI J U L F 7

i

I OI JAN TEl

:

-. I

I OI JUL T8

".I

."'..

..

",3~ I

I OI JAN 7 9

... I

I OI J U L 7 9

I

I OI JAN 8 0

L

ESTIMATION STARTING DATE

Fig. 6. Standard error o f h e a t i n g s l o p e / ~ as a percentage o f the median 1 2 - m o n t h estimate for estimation periods o f lengths 7, 9 and 12 m o n t h s , starting o n the date indicated.

the requirement of summer information f o r a reliable ~ estimate is not surprising. For ~, the error peaks occur when e i t h e r summer or winter is n o t part of the estimation period, as is the case for a 7-month period starting in fall or spring. This is because the transition between the heating and non-heating seasons is needed to define a house's break-even temperature. As with /3, the standard-error peaks and the instability for both ~ and T diminish as more data are added, almost disappearing with 9 or 10 months of data. Note that the NAC results (Figs. 3 and 4) show similar peaks for short estimation periods starting in the transition m o n t h s (in spring or fall), reflecting the instability of all three parameters. From these results, it appears that 9 or 10 months of consecutive data are sufficient for reliable PRISM results. Shorter data sets, particularly with fall or spring starting dates, should be avoided. The Minneapolis study strongly supports these conclusions. GAPS WITHIN THE PERIOD OF ESTIMATION

Frequently, one has a full year of data spanned by less than 12 meter readings, because some of the readings summarize more than one m o n t h of consumption information. This happens whenever on a particular m o n t h a meter is not actually read and is estimated by the utility, and also when a utility regu-

larly uses a bimonthly billing cycle. Because of the prevalence of the latter, bimonthly data are emphasized here. The longer version of this paper [3] contains details on seven other combinations of periods spanning one year. With the month-swapping technique used in the last section, the estimations for 6 bimonthly, for 12 m o n t h l y , and for 6 m o n t h l y readings have been compared as the estimation period is moved through time. The same sample house from Denver is used. The results indicate that six bimonthly readings are only a little worse than 12 m o n t h l y readings and are far superior to the same number of consecutive readings. Over the 42 months studied, the NAC values from bimontly data differ from the 12-month estimates by a m a x i m u m of 9%, the middle 50% of the differences fall between --3% and +5%, and the median difference is 0%; in contrast, the m a x i m u m difference between bimonthly NAC and NAC from six consecutive readings is 38%, the middle 50% are between --8% and +5%, and the median is --2.5%. Unlike the previous cases spanning less than one year, the standard errors in bimonthly NAC are not seasonally sensitive, but they are considerably more erratic than the standard errors in 12-month NAC. Nevertheless, the former remain small, with a maximum value less than 6% and a median of about 3% of NAC, vs. the latter with a m a x i m u m of 3% and a median of 2%.

155 4

--

--6

-4 -2 f,D (}

O

":o e,i

W

•q

F., I

g

0

__2 5

I...

,-----,

-2 -- - 4

-3 ~-6

-4

-5

I =0.30

i - 0 1.5

0.00

I 0.15

I 0.30

I 0.~.5.

I 0-60

o.~5

0.9O

[~ (rno)- ~ (bi)] / ~(mo) Fig. 7. Plot demonstrating the negative relationship between heating slope (~) and reference temperature (r) observed when bimonthly (bi) estimates are compared to monthly (too) estimates.

As usual, the individual parameters fare worse than NAC. Nevertheless, the bimonthly and 12-month results are not very different: for ~, the middle 50% of the differences fall b e t w e e n --7% a n d +8%, with a median at --3%, and, for r, the middle 50% o f the differences are between --1 °C and +1 °C, with a median at --0.2 °C. These medians are much less than the median standard errors, which for ~ are 15% of the estimate for the bim o n t h l y case and 9% for the 12-month case and, for r, are respectively 2.3 °C and 1.4 °C. Moreover, the differences in the combined heating estimate, ~Ho(r), are even smaller than the differences in ~ and in r separately. Figure 7 shows the reason: the differences in the estimates of fl and r covary negatively, so that, if a bimontly estimate of ~ is larger than its 12-month c o u n t e r p a r t , then the corresponding estimate of r is smaller. The resulting estimate of heating consumption is relatively stable: the middle 50% of the differences between bimonthly and m o n t h l y estimates for ~Ho fall between --6% and +4%, and the median difference is --2%. In our studies of other combinations of missing data, we found that the presence of

uneven gaps in the data lead systematically to larger standard errors of all the parameters (see ref. 3). If complete bimonthly readings can produce reliable PRISM results, then the main disadvantage of bimonthly data may be the seriousness of missed readings, in that a single one produces a four-month interval in one consumption period.

SAVINGS ESTIMATES READINGS

FROM

BIMONTHLY

An important question for scorekeeping is h o w savings estimates are affected when only bimonthly rather than monthly readings are available, b u t when the data are otherwise complete (without gaps). Encouraged by the bimonthly vs. m o n t h l y comparison in the previous section, we have run the entire savings analysis twice for all 72 houses in the Denver study, using first (12) m o n t h l y and then (6) bimonthly readings. Figure 8 compares the pre-period NAC estimates from monthly vs. bimonthly readings. The t w o sets of results are strikingly similar, as indicated by an R 2 of 0.99 for the

156 ( k w h / yeor) I0 0 O0

160

~2o

~ o

30000

20000

'

40000

' /

'

~ / ~ [,oooo

8C

~

Ol

I

I 40

I NAC (bL)

I 80

I

I 120

I 160

( Gdeia¢/yeor)

Fig. 8. Monthly (mo) versus bimonthly (bi)estimates o f NAC for the 72 Denver houses during the preperiod of the Denver study. Shown is the line of equality between estimates.

the plot. Therefore, the accuracy of the bimonthly estimates of NAC observed previously for the single house is typical of the entire data set. As expected from the single-house analysis, the stability of the individual parameters, especially /3 and r, is considerably less. The estimates combine, however, to give more stable estimates of flHo(r), with an R 2 for the correlation across houses of 0.91. An R 2 of 0.91 is also obtained when the two values of are compared across houses.

Figure 9 dramatizes the stability of the total consumption estimates relative to the estimates of the individual parameters by showing the median and the middle 50% for the differences between the corresponding monthly and bimonthly estimates. Compared with ~, fl and r, or even flHo(7), NAC from bimonthly data is hardly at all different from the monthly estimate. The estimate flHo(r) is more consistent than the individual parameters, and ~ is more consistent than either orr. The Denver study was carried out to test the effectiveness of air infiltration measures. Two control groups w~re used for comparison with the treatment group. Table 1 compares the savings estimates for the three groups when PRISM is applied to monthly and to bimonthly data. For each group, the median change in NAC is seen to be very similar for the two cases. Savings in heating consumption, ~Ho, and changes in ~, fl, and r as well, are fairly similar. For this conservation project, the scorekeeping conclusions would be no different using monthly or bimonthly data. CONCLUSIONS

Several lessons concerning the data requirements of PRISM emerge from these analyses. The most important is that twelve consecutive monthly readings lead to optimal PRISM results. Missing readings will be troublesome,

10%

(*C)

L5 1.0 5%,

Q5

O~

0

i ,

-0.5J -I.0

-I.5

-

-5%

I ± T

-10%

/3

(Z

/~H0

NAC

Fig. 9. Comparison between the m o n t h l y (mo) and bimonthly (bi) estimates for 72 Denver houses. The middle 50% for each estimate is shown: " x " marks the median. Scale for 7 is chosen to be approximately two typical standard errors wide; absolute difference between m o n t h l y and bimonthly estimates, i.e., r(mo) - - 7(bi), is shown. A single percent scale is used for the other parameters, for which the difference relative to the monthly estimate is shown, i.e., [0(mo) - - 0 ( b i ) ] / 0 ( m o ) for 0 = 0~, 8,/~/o and NAC.

157 TABLE 1 Comparison of monthly and bimonthly estimates* Group (# houses)

Blind control (27)

Conscious control (22)

Air inf. retrofit (23)

Data type

Monthly

Monthly

Monthly

Bimonthly

+2.1 °C (3.8 OF) --8.5% --27.6% --7.1% --5.6%

+1.9 °C (3.5 OF) --9.5% --24.1% --7.3% --4.6%

A~** A~ A~ A~-[o ANAC

+2.8 °C (5.1 °F) --7.3% --25.5% --2.0% --3.5%

Bimonthly +3.4 °C (6.1 °F) --5.6% --27.9% --2.0% --4.8%~"

+2.1 °C (3.8 °F) --13.3%" --17.4% --2.2% --6.8%

Bimonthly +3.3 °C (6.0 OF) --7.4% --27.7% --2.4% --7.0%

*Changes in PRISM estimates from the pre- to the post-retrofit period are shown for the entire Denver data set, using monthly and bimonthly consumption data. Median values are given. **The anomalous increase in reference temperature, observed for all three groups and in both the monthly and bimonthly estimates, is explored in ref. 3. ~'Next higher value is --3.7%, almost exactly the median NAC value from monthly data. This sensitivity is the result of the small data set. and th e longer the gap t he greater t he problem. As t o period length, t he data should span as close to one year as possible: results f r om less t h a n ten consecutive m o n t h l y readings, or f r o m less th an nine readings spanning a full y e ar b u t with gaps o f greater t ha n t hr ee m o n th s , should be scrutinized carefully. Under any shift f r om t he optimal case of twelve m o n t h l y readings, t he NAC index emerges as the m o s t stable and well determ i ned PRISM parameter, by far. With fewer t h an twelve points, t he seasons defining t h e end points have a strong ef f ect on the stability o f th e individual parameters. A reliable d e t e r m i n a t i o n o f t he base-level estim a te requires s u m m e r data, the heating slope requires winter data, and the reference temperature requires i n f o r m a t i o n f r om b o t h seasons. In general, reliable results m a y be obtained f r o m b i m o n t h l y readings. The main problem with b i m o n t h l y data seems t o be the seriousness o f just one or t w o missed readings, which are easily to ler a t e d in m o n t h l y readings but which for b i m o n t h l y readings push PRISM t o its limits. The experiences of Rodberg with th e New Y or k City study [ 5 ] . and Hirst with the BPA study [ 6 ] , b o t h r e p o r t e d in this issue, bear t e s t i m o n y t o this conclusion. These findings underscore the i m p o r t a n c e o f planning, in any r e t r o f i t program, to allow time for collection o f sufficient data in t he pre- and post-retrofit periods. Further, t h e y p o in t to the need for a sensitive and cautious i n t er p r etatio n o f t he results when limited data are available.

ACKNOWLEDGEMENTS

The authors would like to acknowledge support from the Electric Power Research Institute, under Research Project RP2034-4. They are grateful to Martha Hewett and her co-workers for sharing the results from their own stability analyses, and to Jack Collins for supplying the Denver data set.

REFERENCES

1 T. Dunsworth, T. Miller and M. Hewett, Empirical Guidelines for Using the Princeton Scorekeeping Model, Technical Report #84-7 (draft), Minneapolis Energy Office, Minneapolis, MN, October 1984. 2 J. Collins et al., Demonstration of Energy Conservation through Reduction of Air Infiltration in Electrically Heated Houses, Final Report for EPRI Research Project 135-I, Manville Service Corp., Littleton, CO, 1983. 3 J. Rachlin, Finding Signatures of Physical Effects on Residential Energy Consumption Data, MS Thesis, Department of Civil Engineering, Princeton University, Princeton, NJ, October 1984, Ch. IV (longer version of this paper). 4 M. Fels, J. Rachlin and R. Socolow, Seasonality of non-heating consumption and its effect on PRISM results, Energy Build., this issue, pp. 139 148. 5 L. Rodberg, Energy conservation in low-income homes in New York City: the effectiveness of house doctoring, Energy Build., this issue, pp. 55 - 64. 6 E. Hi~t, Electricity savings one, two and three years after participation in the BPA Residential Weatherization Pilot Program, Energy Build., this issue, pp. 45 - 53.