Optimized reaction of large electrical consumers in response to spot-price tariffs

Optimized reaction of large electrical consumers in response to spot-price tariffs J R McDonald, P A Whiting and K L Lo Centre for Electrical Power Engineering, University of Strathclyde. Glasgow. Scotland. UK

Novel tariff structures have been introduced by power utilities over the past two decades in order to induce load shifts to achieve the aims of load management strategbs. As a result of the availability of reliable communications between suppliers and consumers, new 'smart' meterin 9 equipment and the re-organization of the UK power supply industo,, the potential Jbr dynamic- or spot-price based tar(ffs has been recognized. h7 order to ensure the successful introduction of such a prichT9 scheme, it is important to make use of accurate consumer process models as well as spot-price behavioural models to provide the means by which a consumer would opthnize his production schedules in the light of rapidly changing tar(ffs. This paper builds up consumer models from fimdarnental process characteristics and applies optimizing techniques hworporathzg spot-price models in order to develop akjorithms which would minimize production costs and contribute towards the philosophy of joint suppl#r/eonsumer opthnalio' throuoh spot-pricing. Kevwords: Spot.prichT9, load management, modelling

consumer

I. I n t r o d u c t i o n Given the experience of power utilities which offer flexible time-of-day tariff structures such as Economy 7, White Meter and a host of variable industrial tariffs in the UK, it is reasonable to make the observation that when a consumer is faced with fluctuating electricity prices, he will attempt to adjust his energy consumption appropriately over time. The most obvious example of this being where, under time-of-day tariffs, consumers will be likely to use less electricity when prices are high than they would use at that same period if the prices had been constant. However, it is important to note that the consumer may not respond at all if the cost of response is greater

Received 1 August 1990; revised 11 October 1993

Volume 16 Number 1 1994

than the potential savings. This may also be the case if the consumer does not have sufficient information about present and expected price levels to enable him/her to make decisions concerning his/her level of consumption. Equally as important, and more positive in terms of possible implementation of a spot-price scheme, is that if the electricity price fluctuations are large and if the consumer can anticipate price behaviour as well as being able to respond quickly and cheaply, the potential load response may be significantly large 1. A critical element required for the successful introduction of spot-price schemes is the ability to model the consumer process adequately in terms of technical and economic issues. These consumer models, coupled with spot-price models z, lead to the assessment of the likely response of consumers to spot-prices and to develop techniques which a particular consumer could employ in order to optimize his production process operation and costs in the light of these prices. Consumers who use little electrical energy will probably have benefits that are no greater than the costs of installing the necessary communications and control equipment. Therefore, this paper concentrates on large, profit maximizing consumers such as industrial companies or large commercial concerns. By developing methods to minimize the consumers' energy costs while satisfying production constraints, the behaviour of that particular demand can be found. This gives insight into the general nature of these sections of the energy demand sector and can help to evaluate the most successful strategies for both tariff setting and consumer response alike. Another important reason for concentrating our analysis on large industrial and commercial consumers is that large industrial consumers take over 40% of the annual kilowatt-hours utilities in the UK but comprise less than 1% of consumers by number. Thus, a spot pricing scheme providing significant effects on the load, could be rapidly introduced via this particular sector of the load. Similarly, the scheme could be extended to the large commercial users who represent 3% of consumer

0142-0615/94/01035-14 © 1994 Butterworth-Heinemann Ltd

35

Optimized reaction to spot-price tariffs." J. R. McDonald et al 100

--

parameters of the model are as follows:

I SPINNING RESERVE

2000

/

./

.,¢'

% t o t a l load on DP

PERCENTAGE %

,,'""

SPINNING RESERVE MW

./"

',,..,,A.je, "S t" % Customers t" i"

on DP

I !

,/ "1

It should be noted at this point that the various delays indicated in the generalized model are empirical elements• To reduce unnecessary complexity in the following analysis these shutdown and start-up delays can be neglected without affecting the inference which can be drawn from the analysis. Thus the consumer can be considered to be able to respond instantaneously when making his/her decisions on coming off or going back on.

i

I

I I I' I'

PLANNING MARGIN

i f'

i

I

I

I

2

4

6

8

10

YEARS

Figure 1. Possible scenario and timetable for the introduction of spot-pricing numbers while taking approximately 20% of the annual energy sales bringing the total potential energy levels supplied under such a pricing scheme to 60%. If no one else participated in the scheme this level would be more than adequate for technical and economic control purposes. Figure 1 shows a possible scenario and timetable for the introduction of spot-pricing 3. The potential for using spot-prices in a load management regime and the expected variation in electricity prices in response to the prevailing supplydemand conditions has been an area of interest for some time 4. In order to identify the general characteristics of demand response, two basic types of consumer are considered, i.e. the 'instantaneous' and the 'shut-down' consumer. It is reasonable to simplify the load models like this because of the complexity of actual consumers and their electricity consuming processes.

II. T h e i n s t a n t a n e o u s

consumer type

The first type of consumer process to consider is one which does not have any storage element. Furthermore, this process type can only have two states of behaviour, it either is on and consuming electrical energy at full-load or it is offand not consuming any electrical energy. Thus, in responding to variable prices, such a consumer would have to decide to shutdown that process completely or to continue to operate fiat out. However, some practical technical and/or economic constraints would have to be addressed before the decision could be made. For example, there may be a requirement for a finite time delay on shut-down and on restarting. A generalized model for this type of consumer is shown in Figure 2. In this model there is a need for lead times before plant response is possible after a 'switching' decision either to come off or to reconnect. The

36

forr = time at which decision was taken to shutdown to. = time at which decision was taken to reconnect ToEr = time required to shutdown To. = time required to reconnect Tdow,= duration of plant disconnection d(t) = condition of the plant ={01 ifplantisshutdown if plant is operating B = consumer's fixed level of demand for this process type

I1.1 Economic aspects of instantaneous consumer As spot-pricing tariffs apply principally to short-term operation, the consumer's stock of capital equipment can be thought of as fixed in that time frame. Similarly, running costs (excluding cost of electrical energy) can also be considered as constant for a particular period under study. A further assumption is that the ratio of the consumer's electrical energy consumption per unit of output is fixed, which is a reasonable assumption in the short-term. Whenever the plant is in operation, costs are incurred by the payment of labour force wages, electricity costs and other costs such as raw materials. It is also convenient to assume that the process output can be sold at a constant price regardless of the scale of production. In order for the consumer to make sensible decisions as to whether to shut-down or not, he must quantify the costs/savings associated with such an action. Two very important elements which complete the consumer's cost function are: K = consumer's fixed cost of shutting down and

BP(t) = avoided costs of electrical energy where

P(t) = spot-price during period of shutdown OemanO

run ,~Ilull (apac,ty

It q

shuldown

IL ~

tun al lull (apa(dy

i I I I

To*

41"--'Ioif -II

I I I m

ii--

! ao.~

~

i I I I

i toll

~ ,,no

l~

Figure 2. Instantaneous consumer model

Electrical Power & Energy Systems

Optimized reaction to spot-price tariffs: J. R, M c D o n a l d et al

These various cost elements will be used to form a cost function for inclusion in an optimization procedure to allow the consumer to make an informed decision on his production state, i.e. either shutdown or running fully.

Upstream process

Downstream process

Constant output )

input St

11.2 Representative instantaneous consumer processes Within any industrial process or commercial building environment, there will be elements of the electrical energy consuming equipment which could be shut-down by the consumer. Obvious examples of this type of process are as follows. •

•

•

Completely shutting down an assembly line within a production process. However, the effects on the rest of the process would have to be taken into account, i.e. hidden implicit costs may be incurred. Within a chemical process plant, there will be elements of the scheme which consume electrical energy but which could be shut-down temporarily without adversely affecting the whole production process. Non-essential plant and services could be shut-down. This type of equipment would include some heating, ventilating and air conditioning plant.

Storage bullet

Figure 3. Characteristic model for storage consumer run to supply the downstream unit and/or a storage buffer which in turn can be used to supply intermediate product to the downstream unit to allow continuous production output. Mathematically, we have the following operating constraints: I(t) = l(t - 1) -t- Cu(t) - Cd

(1)

the constant downstream output rate constraint.

III. T h e s t o r a g e c o n s u m e r t y p e A more general type of consumer model is one which incorporates a storage element where intermediate products are held after being produced by an electrically-powered process before being transferred to the next stage of the production. In terms ofdecision making, the consumer finds himself in a more complex situation. This is due to the decision being dependent upon past, present and expected future conditions such as the state of the plant (i.e. on or off), availability of storage space, limits on operating capacity of process elements and, naturally, the behaviour of the spot-price based tariffs. The storage type of consumer would have to employ a more sophisticated approach to decision making, e.g. given his requirement to consider expected events, he may actually increase his production rates (those consuming electricity) filling storage units when the spot-price increases tfhe then expects the spot-prices to increase even further. Another feature of this consumer type is his requirement for information on factors affecting the spot-price behaviour. This could take the form of modelling of the price structure itself. Thus, as identified by the MIT, Energy Laboratory, 'Information Brokers' providing analysis of market futures are likely to emerge 5. However, the larger consumers would be in a position to perform their own forecasts. II1.1 Technical and economic modelling aspects of storage consumers For ease of exposition a single production/storage element will be considered. Figure 3 presents the characteristic model for the storage consumer. The model shows a downstream process element which has to be operated to give a constant product output rate. This element is fed by an upstream process element with operating capacity limits. The upstream unit can be

Volume 16 Number 1 1994

0 <~ Cu(t) ~< C....

(2)

the upstream output bounds. 0 ~ l(t) 4,%Ira,"

(3)

the storage buffer bounds. Where

I(t) = storage buffer level at time t (output units) Cu(t) = upstream operating rate (output units/unit time) Cd = fixed downstream operating rate (output units/unit time) C.... = maximum upstream operating rate l,,a, = maximum storage level Given that the consumer will be presented with spot-price updates on a half-hourly basis, a half-hour would be the most appropriate unit of time for analytical purposes. For the consumer therefore, the basic state of the system to be controlled would consist of the time of day, current storage level and spot-price, i.e. X(t) = {t, I(t), P(t)}

(4)

where X(t) is the system state at time t. The economic aspects of the storage model differs from the instantaneous type of consumer. The costs of electrical energy are assumed to be the only variable costs. This implies a certain structure of the overall storage type consumer's process, i.e. the rescheduling flexibility lies within the intermediate process operating rates and storage buffers and not on complete shifting of times of production. Thus, the storage consumers can be one, two or three shift companies with increasing potential for rescheduling without loss of applicability of the generalized model developed earlier. 111.2 Representative storage consumer processes The most obvious type of large industrial processes with storage elements are petro/chemical producers, steel

37

O p t i m i z e d r e a c t i o n to s p o t - p r i c e tariffs: J. R. M c D o n a l d

manufacturers and those firms with several interdependent production lines. In particular, consider the following types of consumer processes. •

A chemical production which is required to output different products on a continuous basis. In this case the storage elements are represented by intermediatestage chemicals which, when not being immediately put to use, are stored in surge tanks. Metal manufacturers can be considered in two ways. First, as for chemical producers, there will be several production stages in which rolled steel for example would be drawn, rolled and pressed. Each stage would ultimately end in some form of storage holding of the material until it is required for the next production step. Secondly, electrical arc furnaces used to melt metal ores would effectively be storage elements while the material is being heated (i.e. heat is being input to the material and molten material is drawn off for steel/aluminium etc. production). The remaining material in the furnace is effectively being stored and reflects the electrical energy input to it up to that point 6'7,

•

IV. Analytical solution for instantaneous consumer behaviour Initially, consider a tariffstructure in which the spot-price can take a relatively high or low value with related probabilities in every period. The number of particular spot-price levels will be taken as three sequential pairs of low and high prices. The proposed price structure is as shown in Figure 4. In order to arrive at a measure of the expected savings and consequently the analytical form for the operating policy we need to make the following assumptions. •

The process starts in a type '1' price period with a high price level, P.,. The high and low prices are such that their actual level is immaterial and we are interested only in their respective contributions to the average saving function as will be seen. Savings can only be made when the price level is high and the customer is shut-down. This is an effective infinite planning horizon problem. The high prices are such that it does not pay to be off for one period alone.

•

• • •

Spot - price

P.I,P~

et al

Therefore the expected savings to be made by the consumer when shutting down can be evaluated by calculating every possible combination of a string of high prices followed by a low price level. This will model the conditions where the consumer will come off-load in response to high prices and then start-up again when the prices fall. The consumer expected savings are as follows: G = qz(BPH, - Cp) - K + p2q3[(BPn, - Cp) + (BPH." -- Cp)] + p2p3qt[(BPH, -- Cp) + (BPH2 - Cp) + (BPn3 - Cp)] + p,pzp3qz[2(BPH, -- Cp) + (BPn= - C,)

+ (BPH~ - Cp) + ptp~p3qz[2(BPH, -- Cp) + 2(BPM2 - Cp) + ( B P H ~ - C,)] + - - (5) where G = expected consumer savings to be made by shutting down an instantaneous process Cp = consumer hourly operational revenue

By considering each term in turn in equation (5) we see that the terms represent the savings to be made by shutting-down for a (PH,, PL.,), (Pn,, Pn_,, PL3), (P,,, PH,., Pn~, PL,), (PH,, Pn~,, Pn~, Pn,, PL.,) etc. spot-price sequence respectively. Grouping terms and simplifying by using geometric series leads us to the following expression for the savings function: [AI'(1

--

PlP2P3) +

G= (l

--

1 - - PlP2P3) + P2paA3( 1 -- PlP2P3)] p2A2(

-K

plp2P3) 2

(6) where Az = BPn, - Cp,

i = 1, 2, 3

This expression, therefore, represents the expected savings to be made by the consumer if the evaluation starts in a high price, PH,, state. The consumer will decide to shutdown if this expression is greater than zero. That is A 1 "4- p2A2 + p2PaA3 > K(1

--

PlP2P3)

(7)

In particular we now have a means by which the consumer can choose the optimal decisions. At the start of each period, the consumer will evaluate an inequality function based on his/her consumption level, revenue rates, shutdown costs as well as the individual price levels and their respective probabilities. For this three-pair price tariff the consumer has the following decision criteria.

P~z.P~

Pthqt

(1) If current price is PH,, shutdown if (BPn, - Cp) + p2(BPn2 - C n) + p2p3(BP m - Cp)

7-

P.I,Pl

> K(1 - PlP2P3)

P;q~

(2) If current price is Pn._, shutdown if (BPH. -- Cp) + ps(BPH3 - Cp) + p~p3(BPH, -- Cp)

Pt ;.q]

> K(1 - PlP2P3) 4

S

6

7

8

9

(periods)

Figure 4. Multi-block stochastic price structure

38

time

(3) If current price is Pro, shutdown if (BP m - Cp) + p x ( B P . , - Cp) + P2P3(BP.2 - C.)

> K(1 - PxP2P3)

Electrical Power & Energy Systems

Optimized reaction to spot-price tariffs: J. R. McDonald et al

V. Storage consumer cost function and conditions for optimal response to varying tariffs The consumer can be assumed to have access to knowledge about the tariff structure either in the form of pre-published, multi-level STOD tariffs or in terms of a model of expected spot-price behaviour vis-~i-vis Markov chains, time-series, discrete probability distribution models, etc. as developed by the authors in Reference 2. Another obvious way for the consumer to follow price behaviour would be to generate a quasi-static price structure via the forecasting process using a time-series approach. In this case the forecast produces a daily, or even weekly, set of expected half-hourly price levels. Thus the consumer can set a finite horizon type problem in which he has an estimate of the day ahead prices (48 half-hourly prices) and minimizes his electricity costs in the light of his production constraints. The cost function to be minimized in this case would be, for a daily production schedgle H = Cu,~R

P,'Cu,

(9)

i

where H = operating cost function C,, = decision variable, i.e. upstream rate at period i R = decision space (set of discrete operating levels) In order to simplify the numerical calculations for the solution of this problem the various parameters of the storage consumer will be normalized giving an effective 'per-unit' measure of consumption and output. Using the earlier defined variables we have the following constraints and bounds Cd = 1; fixed downstream rate constraint 0 ~< / ~< 1; storage bounds 0 ~< C~ ~< 1.5; upstream bounds Considering the plant equation as being that which generates the next system state, we have the storage level produced as follows I i = li+ , + C o , - 1

(10)

The normalization arises from the reasonable assumption that the upstream process electrical operation rate is proportional to the rate of intermediate product production. The downstream rate will be measured in the same units. Similarly, the storage buffer is measured in the same units and it is noted that this, in turn, is equivalent to an electrical consumption rate and is thus an effective, decentralized storage of electrical energy. In the 24-hour period over which the optimization is carried out, there are 96 variables to be evaluated. These variables are the upstream rate and storage level in each period. The solution of typical problems are presented in the following tables and figures. All of the prices have been generated by using 'the time-series based spot-price models developed in Reference 2. A typical summer day and winter day were selected as the basis for the cost function to be minimized giving the optimal storage consumer operating strategy. The summer day data and results are given in Table 1 and Figure 5. The winter day data and results are given in Table 2 and Figure 6.

Volume 16 Number 1 1994

Table 1. Optimal operating strategy for summer day

Half-hour period (h)

Day ahead forecast of spot price (p/kW h)

Upstream operation rate

Storage level

00.00 00.30 01.00 01.30 02.00 02.30 03.00 03.30 04.00 04.30 05.00 05.30 06.00 06.30 07.00 07.30 08.00 08.30 09.00 09.30 10.00 10.30 11.00 11.30 12.00 12.30 13.00 13.30 14.00 ! 4.30 15.00 15.30 16.00 16.30 17.00 17.30 18.00 18.30 19.00 19.30 20.00 20.30 21.00 21.30 22.00 22.30 23.00 23.30

1.65 1.61 1.60 1.62 1.63 1.63 1.62 1.62 1.62 1.64 1.62 1.65 1.74 1.86 2.00 2.09 2.14 2.16 2.18 2.18 2.18 2.19 2.20 2.21 2.23 2.21 2.20 2.21 2.23 2.24 2.24 2.26 2.30 2.33 2.33 2.31 2.27 2.24 2.23 2.21 2.21 2.21 2.25 2.26 2.28 2.24 2.17 2.08

1.0 1.5 1.5 1.0 0.0 1.0 1.0 1.0 1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.5 1.5 1.0 1.0 1.0 0.0 1.0 1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.5 1.0 1.5 1.0 1.0 0.0 1.0 1.0 1.0

0.0 0.5 1.0 1.0 0.0 0.0 0.0 0.0 0.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.5 1.0 1.0 1.0 1.0 0.0 0.0 0.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.5 1.0 1.0 1.0 0.0 0.0 0.0 0.0

There are several points to observe as follows. • •

•

The forecasts will progressively lose accuracy as the lead time increases from the forecasting point. This would introduce possible errors in the choice of optimal operating conditions thus incurring greater costs than necessary. One way to overcome this is to be able to change scheduling to a more dynamic 'on-line' basis allowing adaptive, more accurate forecasts to be made.

39

Optimized reaction to spot-price tariffs: J. R. McDonald et al OPSTR,AMOPERA.,.GRA,E []

Table 2. Optimal operating strategy for winter day

STORAGE LEVEL

O0 O0

time of day (periods)

Figure 5. Summer data and results UPSTREAMOPERATINGRATE [ ] STORAGE LEVEL

~,

i ~0

,~ ~ I . N O0 O0

time of day (periods)

24 O0

Figure 6. Winter data and results

The winter behaviour can be seen to take account of the daily peak times by reducing consumption at those times as would intuitively be expected. The technique used to solve these problems would be readily implemented in the types of automatic load management computers already in wide use within industry.

Vl. Optimal consumer operating strategies incorporating detailed price models for practical implementation In this section the stochastic variations of the spot-prices are explicitly dealt with by using the spot-price models such as Markov processes and time-series, etc. They can be directly incorporated into the optimizing algorithms by augmenting the state knowledge of the system through the forecasting of expected price trajectories. The techniques developed here have been chosen so that the optimization algorithms used to identify optimal strategies have accelerated convergence. This is important as the algorithms would be expected to be implemented on process control computers and as such, efficient computational methods are necessary. Various time frames can be considered which define the horizon over which the consumers would require to make production scheduling decisions. This planning horizon is subjective in nature and in this section can be considered in a general sense via the definition of an infinite horizon (which effectively could represent several days ahead) through to a finite horizon (several hours or a day). In an attempt to make the developed algorithms broadly applicable to a range of consumers, the concept of discounted pricing is utilized. This permits

40

Half-hour period (h)

Day ahead forecast of spot price (p/kWh)

Upstream operation rate

Storage level

00.00 00.30 01.00 01.30 02.00 02.30 03.00 03.30 04.00 04.30 05.00 05.30 06.00 06.30 07.00 07.30 08.00 08.30 09.00 09.30 10.00 10.30 11.00 11.30 12.00 12.30 13.00 13.30 14.00 14.30 15.00 15.30 16.00 16.30 17.00 17.30 18.00 18.30 19.00 19.30 2O.O0 20.30 21.00 21,30 22.00 22,30 23,00 23.30

1.89 1.97 2.01 2.01 2.00 1.99 1.97 1.96 1.94 1.93 1.94 1.96 2.04 2.15 2.35 2.51 2.63 2.64 2.70 2.72 2.69 2.69 2.69 2.69 2.69 2.64 2.64 2.65 2.64 2.61 2.59 2.81 3.99 4.43 3.56 3.06 2.87 2.79 2.76 2.70 2.64 2.60 2.60 2.53 2.43 2.31 2.22 2.17

1.5 1.5 1.0 0.0 1.0 .0 .0 .0 .0 .5 .5 .0 .0 .0 .0 .0 .0 .0 .0 0.0 1.5 0.5 1.5 1.5 0.0 1.5 1.5 0.0 1.0 1.5 1.5 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

0.5 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.5 0.0 0.5 1.0 0.0 0.5 1.0 0.0 0.0 0.5 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

the modelling of subjective consumer perception of the number of pricing periods or section of a forecasted price trajectory of relevance to their particular process. For example, those consumers manufacturing goods to supply an immediate market will be mainly interested in the prices expected over a relatively short lead-time giving a low value discount factor. The discount factor progressively reduces the importance of prices outside the desired lead-time. However, those consumers whose

Electrical Power & Energy Systems

Optimized reaction to spot-price tariffs: J. R. McDonald et al process has to be operated on a continuous basis or for long periods, e.g. a whole shift, would use a high value discount factor which gives a significant weighting to prices over a much longer time-frame. Vl.1 Use of Markov price structures With the proposition that the spot-price sequence conforms to a Markov chain, we can consider the incorporation of these models into a dynamic programming algorithm to improve the operating costs. Within our defined system, that is, a storage type consumer with operational and commercial constraints exposed to a highly variable electricity price over a planning horizon, we can readily assume that the only stochastic element in our structure is the spot-price itself. This stochastic behaviour can be shown by considering the spot-price as belonging to a finite set of discrete values with an observable relationship between the various price levels across the half-hour periods, see Figure 7. As mentioned earlier, different consumers will place different subjective weighting on the expected cost projection which is dependent upon the nature of their process and the demand and production conditions. The discount factor, ~, is a scalar lying in the interval [0, 1]. VI.2 Problem formulation Throughout the following development, we will assume that all the spaces, that is the state, control and disturbance spaces underlying the problem, are finite spaces. This allows us to develop tractable algorithms which lend themselves to numerical solutions via implementation on a digital computer. We can begin by assuming that the state space, S, consists of n states: S = {1,2,3 . . . . . n}

(11)

In our problem, the system state will be defined by a combination of storage level, period number and spot-price level. Given that we have defined all of these as belonging to finite sets, this leads to the sum of all permutations as being a finite number, i.e.n. Let us define the following transition probability: Plj(u) "--- P(x~+l =Jlx,+,,,.~),

V;4eS,

ueUi0 (12)

This represents plj(u) as the probability that the next system state will bej given that the current state is i and we apply the control u which belongs to the finite set of permissible controls given the state i, u(i). In our case, the transition probabilities will be assumed given a priori.

v

-'-I Pl v,.. ' l h V... 'T h

P2~ "I1 J

Periot

p.

/_

2\~

"11

2~'~

l ll) l÷ 1

t,,2

4- -

Finite set of prices per period (n-price levels)

t÷3

Figure 7. Markov structure for spot-prices. The probability of changing from one price level to another is time-inhomogeneous, therefore the transition probability matrix is different for each time period

Volume 16 Number 1 1994

The determining factor which introduces these probabilities is the measured spot-price transitions, Thus we can consider the spot-prices as being tne input disturbance to the system, wK which has a known probability distribution p(. Ix,,). Therefore we have:

pij(u) = P(Wij(u)li.,)

(13)

where Wit(u)-- (w~DI/, .... ~=j), a finite set D = disturbance space, the spot-price levels f(i, u, w) = the system plant equation The average cost of being in a particular state and taking a particular admissible decision, with the expectation taken over the spot-price distribution, is given by

O(i,u) =

E

(g(i,u, w)),

VieS,

u~u(i)

(14)

W

Thus the following expression

O(i, u) + otE{J[f(i, u, w)]}

(15)

represents the general form of the cost function to be minimized in the optimization and this can be stated in terms of the Markov structure for the spot-prices as follows: & (t(i, u) + ct ~ pq(u)J(j), i~S (16) j=l

This cost function can be minimized by employing contraction mapping theory and a policy iteration approach. Appendix I gives the theoretical background to this method. VI.3 S u m m e r behaviour of storage consumer By considering the matrices of the optimality equation in (A.9), the 'probability' matrix is initially set up by assuming a policy of run as low as possible for all spot prices. Based on the detailed spot-price simulation and modelling carried out in Reference 2, a finite set of discrete price bands were identified, i.e. 1.45, 1.58, 1.83, 1.98, 2.16 p / k W h. This permits the construction of the probability matrix. The elements of the matrix are generated from the Markov transition probability matrix developed in Reference 2. The policy also directly establishes the elements of the period cost vector [g~]. The matrix equation [J.] = {[I] - ~[P.]}-

,[g]

is then solved. This gives the cost-to-go vector [Ju] which then allows the application of the contraction mapping T[Ju](i) to identify any deviations from the initial proposed optimum strategy. These changes are then made to the matrices and the process carried out iteratively until the minimum cost-to-go vector is found, thus identifying the optimum policy. Studies were carried out for a value of the discount factor of 0.9. This value models the consumer's perception of the future (or expected) spot-price values as of increasing importance to his process as the discount factor increases. The solution for the storage consumers using the policy iteration method for the discounted cost problem leads to an optimal operation table. The results for a discount factor of 0.9 are shown in Table 3. The

41

Optimized reaction to spot-price tariffs. J. R. McDonald et al Table 3. Storage consumer response to summer spot-prices. (This solution arrived at after four iterations of policy iteration algorithm with a discount factor = 0.9. Initial policy was: run as low as possible for all prices)

Price (p/kW h) Storage

1.45

1.58

1.83

1.98

0

0.5

1.0

0

0.5

1.0

0

0.5

1.0

1.0 1.5 1.5 1.0 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.0 1.5 1.5 1.0 1.0 1.0 1.5 1.5 1.0 1.5 1.5 1.5 1.0

0.5 1.0 1.5 0.5 1.0 1.5 1.5 1.5 1.0 1.5 1.5 0.5 1.0 1.5 0.5 0.5 0.5 1.0 1.5 0.5 1.0 1.5 1.5 0.5

0.0 0.0 1.0 0.0 0.5 1.0 1.0 1.0 0.5 1.0 1.0 0.0 0.5 1.0 0.0 0.0 0.0 0.5 1.0 0.0 0.5 1.0 1.0 0.0

1.0 1.5 1.5 1.0 1.0 1.5 1.5 1.0 1.5 1.5 1.5 1.5 1.0 1.0 1.0 1.0 1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0

0.5 1.0 1.5 0.5 0.5 1.0 1.5 1.5 1.5 1.0 1.5 1.5 0.5 0.5 0.5 0.5 1.5 1.5 0.5 0.5 0.5 0.5 0.5 0.5

0.0 0.0 1.0 0.0 0.0 0.5 1.0 1.0 1.0 0.5 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0

1.0

0.5 NA NA 0.5 0.5 0.5 0.5 0.5 1.5 1.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 NA NA

0.0

0

2.16

0.5

1.0

0

0.5

1.0

Period 1 2 3 4-9 10 11 12 13 14 15 16 17 18 19 20-27 28-31 32 33 34 35-38 39 40 41-43 44-48

15

1.0 1.0 1.0 1.0 1.0 1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

NA NA NA NA NA NA NA NA NA 0.5 NA 0.5 0.5 0.5 NA 0.5 0.5 0.5 0.5 NA NA NA NA NA

1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA 0.5 NA NA NA NA NA

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

1.0

0.0

mniuF teorecast

UPSTREAM OPERATINGRATE [ ]

Actual 30

STORAGE LEVEL

Ooilg

~

Forecast

3.00 2.75 10

2.50 2.25 2.00

03

k 02 O0

1.75 1.50 1.25

O0

O0 O0

24 O0

1.00

time of day (periods)

Figure 8. Consumer behaviour in response to spotprices

0.75 0.50

I

0.25 0.00 0

behaviour of the consumer in response to the spot-prices is shown in Figure 8. In order to simulate the response of the storage consumer to spot-prices, the identified policy was applied to the consumer model along with a typical summer day's price sequence. The prices used in this context are those shown in Figure 9 for a summer spot-price sequence.

VI.4 Winter behaviour of storage consumer As in the previous section, the matrices can be set-up by using the winter Markov transition probability matrix, and an initial assumed operating policy. The initial policy

42

2

4

6

fl

I0

12

Hours

in

14

16

18

20

22

24

dG~/

Figure 9. Spot-pricing for summer day is run as low as possible for all spot prices, i.e. 1.9, 2.24, 2.61, 3.2, 4.1 p / k W h. Consequently, the initial structure of the probability matrix will be similar to that of the summer structure. The optimal operating policy was identified using the policy iteration technique for a discount factor of 0.9. Following the same procedure as in the previous section, the storage consumer optimal policy tables for a discount

Electrical Power & Energy Systems

Optimized reaction to spot-price tariffs." J. R. McDonald et al Table 4. Storage consumer response to winter spot-prices. (This solution arrived at after seven iterations of policy iteration algorithm with a discount factor -- 0.9. Initial policy was: run as low as possible for all prices.)

Operating rate of upstream process Price (p/kW h) Storage

1.89

2.24

2.61

0

0.5

1.0

0

0.5

1.0

1.5 1.0 1.5 1.5 1.5 1.5 1.5 1.0 1.0 1.0 1.5 1.5 1.5 1.5 1.0 1.0 1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0 1.5

1.5 0.5 1.0 1.5 1.5 i.5 1.5 0.5 0.5 0.5 1.0 1.5 1.0 1.5 0.5 0.5 1.0 1.5 0.5 0.5 0.5 0.5 0.5 0.5 1.0

1.0 0.0 0.5 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.5 1.0 0.5 1.0 0.0 0.0 0.5 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5

1.0 1.0 1.0 1.0 1.5 1.5 1.0 1.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.5 1.5 1.5 1.5 1.5 1.0 1.0 1.0 1.0 1.0

0.5 0.5 0.5 0.5 1.0 1.5 0.5 1.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 1.0 1.5 1.5 1.5 1.5 0.5 0.5 0.5 0.5 0.5

0.0 0.0 0.0 0.0 0.5 1.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0

0

0.5

3.20 1.0

0

0.5

4.10 1.0

0

0.5

1.0

Period I-2 3-10 I1 12-14 15 16 17 18 19-21 22 23 24 25 26 27-29 30 31 32 33 34 35-37 38 39-44 45-47 48

NA NA NA NA NA 1.0 0.5 1.5 1.0 0.5 0.0 0.5 0.0 0.5 0.0 0.5 0.0 0.5 0.0 0.5 0.0 0.5 0.0 0.5 0.0 1.0 0.5 1.5 1.0 1.5 1 . 0 1.5 1.0 1.5 1.0 0.5 0.0 0.5 0.0 0.5 0.0 NA NA

1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.5 1.5 1.5 1.5 1.5 1.0 1.0 1.0

Volume 16 Number 1 1994

1.0 1.0 1.0 1.0

0.0

0.0 0.0 0.0 0.0

k

'1.50 4.0~ 3.~ 3.0<3

2.0G

(17)

state vector - [I, P, a, ~, ~] - [u] decision vector disturbance vector -generated via the ARIMA time series model (a 's) and the observed deviations (~" and Es)

1.0 1.0 1.0 0.0 0.0 0.0

1.0

5.00

In this application of state augmentation, the consumer is assumed to have available to him a time series model of the spot-price sequence. Implicit in such models is the notion that the spot-price is dependent upon lagged price levels and forecast error values. The form of the model is

where 14I, = differenced spot-price series Zt = original spot-price series V = difference operator Using this model, we can generate a stochastic price model as shown in Figure 12. Consequently, the system state can be redefined as follows:

1.5 1.5 1.5 1.0 1.0 1.0

0.0 0.0 0.5 1.0 0.0 0.0 0.0

6.00 5.50[

Vl.5 State augmentation using time series models for spot-pr/ces

"7 I 1 = Vi'V+aV2+oZt

1.0 1.0 1.5 1.5 1.0 1.0 1.0

NA NA NA NA NA NA NA NA NA NA NA 0.5 NA NA NA NA NA 0.5 0.5 0.5 0.5 NA NA NA NA

ActuaL 3 0 minuteForecast Daitg Forecast

factor of 0.9 is shown in Table 4. By applying a winter spot-price sequence as shown in Figure 10 we can generate the response behaviour as given in Figure 11.

Wt

NA NA NA NA NA NA NA 0.5 0.5 1.0 1.5 0.5 0.5 0.5 NA NA 1.5 1.5 1.5 0.5 0.5 0.5 NA NA NA

1.0<3 0.5C 0.0( o ....

:~........ ,i ........ ~ . . . . . .

~"

io

Hours

"i2

+"

i+"-iS

++" i 8 " ~ o " + + ~ 2 " ~..+

in dog

Figure 10. Spot-pricing for winter day For practical implementation of a dynamic programming algorithm, typical discretization is as follows for the normalized storage consumer: • •

storage level (l), 0.0 to 1.0 in steps of 0.1 (11 values) forecast spot-prices (P), Pmax to P,,z, in steps of 0.1 p / k W h (typically 20-40 values)

43

Optimized reaction to spot-price tariffs." J. R. McDonald et al UPSTREAM OPERATING RATE

Note that as a feature of the discretization, the spot-price forecast and projected deviation may not lead to an exact price value. In that case the value of the spot-price would be taken as the discrete value in whose 'band' it lay. Greater accuracy is obviously achieved with smaller steps.

[]

STORAGE LEVEL 1.5

1.0

Dynamic programming algorithm Jk(lk, Pk-1, Pk- 2, Pk-*a, Pk-*9, Pk-50, Pk-2,0, PR- 2*I,

0,5 09 O0

,

Pk-2,2, Pk-2.8, P,-~89, &-~90, ak_., a~-2, a~-48, a k - 4 9 , a k - 5o, a k - 2 4 0 , a k - 2 4 1 , a k - 2 4 . 2 , a k - 2 8 8 ,

0.0

2= 00

00.00

a k - 289, a k - 2 9 0 ,

~k)

time of day (periods)

Figure 11. Response behaviour to winter spot-price sequence

[gk(lk, Pk, ak) -I- ~ ~,, q~+ tJk+ l[(fk(I k, Uk), =

min Ck e uk~Uk(~k)

spof-price (p/kWh)

fk(lk, Uk) generates the next storage level, h(P, a) generates the next spot-price via the moving

,,x

average time series model, the expectation is taken over the particular d.p.d. defined by ~k with j discrete deviation values.

,, J

nl

m

~ on

r

.............. Figure 12. Stochastic price model •

L

time

lagged price forecast errors (a), - v e : 0 : + v e in steps of 0.1 p/kW h (typically 21 values) expected price deviations (e), - v e : 0 : + v e in steps of 0.1 p/kW h discrete probability distributions of e"(~), a vector of prob. values for each e value decision variable (u), 0.0 to 1.5 in steps of 0.1 (16 values) upstream rate

• • •

As each state combination has to be considered when using the policy iteration approach to solving the dynamic programming algorithm, it can be seen that for a 48-period (day) planning horizon, there is potentially a significant computational burden. However the numerical approach offers a robust means of identifying an optimal operating strategy for the consumer. The system plant equation and constraint are given by Ik+ l = lk + Uk -- 1 I ~ u k + Ik ~< 2

(18)

The dynamic programming following structure:

algorithm

has

the

Terminal cost Over the N period finite planning horizon we have J~(IN, PJv- t, PN- 2, PN-4s, Ply-49, PN- 50, PN- 2,0,

P,v-24,, PN-242, P~-28a, P~'-289, PN-290, aN-l, (2N- 2, a N - 4 8 , ON-4. 9, a N - 50, a N - 240, a N - 241, aN- 242' aN-288'

aN-289,

aN-290,

~N)

min

gN(')

%eUs(IN) 44

(20) where

n~ ,i 1

t~ ~

)

h(P, a), e,+ ~, ~,+,]1~.~]

)

Pace errorsawayi~om forecast vaJuo

I

Li=l

(19)

In the following sub-sections, this dynamic programming algorithm will be solved by using the policy iteration approach. Additionally the consumer types, for which the optimal operating strategies are to be identified, will be defined as made up of composite instantaneous/storage modules. These composite models have been constructed to model the basic process components of three practical consumers for whom load data were available.

VII. C o m p o s i t e process models for practical consumers In order to demonstrate the effects of the application of the dynamic programming techniques on consumers other than the idealized types developed earlier, we shall now consider practical consumer process characteristics provided by a UK utility, the fundamental consumer 'building blocks' (i.e. instantaneous and storage models) can be used to fit composite process models to practical consumers. The three consumer types considered are: (1) Cement works This firm produces building industry staridard cement. The production process includes the initial processing of the principal components such as clay, lime and granite. Initial refining is carried out before the combination to produce cement powder. In addition to this, there is automatic packaging plant before the physical distribution stages. In terms of auxiliary plant and non-essential load the process has several conveyors and fans (forced and induced draft) as well as heating and lighting circuits. Through discussion with utility commercial engineering staff and by 'fitting' to the available load data for this consumer, a representative composite model for the process was built up. See Figure 13.

Electrical Power & Energy Systems

Optimized reaction to spot-price tariffs." J. R. McDonald et al

cloy and time mixing ptc~nt

cement refining plant

conveyors/fan: uncontrolled t o~d w

packaging stage Figure 13. Composite model of production process for cement

It is important to note that without direct access to detailed measurement and observation of the consumer's process, the composite model represents an intuitively correct model providing realistic response characteristics. This also holds for the following models. This firm is run on a three-shift basis. (2) International telephone exchange This is obviously a less dynamic type of load with the main source of demand stemming from a significant battery charging load. As this exchange is a major control point on the telephone network, there is a great deal of telephony and control equipment (mainly electronic) requiring an extensive DC supply system. With only minor auxiliary service requirements for heating and lighting, it can be seen that given a battery system with sufficient capacity, this particular load can be almost totally switched off during high spot-price periods. This fits well with an almost entirely instantaneous consumer model type. See Figure 14. The load is potentially constant over a day. (3) Engineerin9 works/Foundry This firm is mainly concerned with the production of cast-iron frame moulding and associated ironwork. There are two furnaces used to produce molten metal to then be poured into precast mouldings. As defined in earlier sections, the furnace plant is adequately modelled by the storage consumer model. These make up the main proportion of electricity demand with some minor non-essential loads. The firm varies its production programme over two or three shift days. See Figure 15. The actual capacities and expected energy consumption levels of the various individual components of these composite consumer models were arrived at using an

Volume 16 Number 1 1994

empirical and heuristic approach. This was the most appropriate method given that only limited information was available. VIII. O p t i m a l response of practical c o n s u m e r s t o s p o t - p r i c e sequences The three composite consumer models developed in the previous section can be readily incorporated into the cost functions of the dynamic programming algorithms used to identify optimal operating strategies. The spot-prices presented to these consumers were assumed to be adequately modelled by a time-series model. The planning horizon of these consumers was set as a 24 hours period and this was set into the framework of a finite horizon, optimization problem. Therefore the dynamic programming algorithm represented by equations (19) and (20) can be used to determine the optimal operating sequence of these consumers in response to the spot-prices presented to them. The following scenarios were analysed to determine the expected response of these consumers under varying conditions,

bctffery charger

fetephony equip, power

>~e S "~

heating & righting Figure 14. Model exchange

for

k,~

international

telephone

moutding assembty

itiary moutding

1031 heating & lighting Figure 15. Model of production of cast-iron frame moulding and associated ironwork

45

Optimized reaction to spot-price tariffs." J. R. McDonald et al (1) Load management January weekday This is a day during which the consumers would have been requested by the utility to curtail their demand at the evening peak time. This represents the existing 'load management' type contracts between industrial/commercial consumers and the power utility. As can be seen in Figure 16, the i'esponse as determined by the solution of the d.p. algorithm is markedly in line with what may be hoped for in so far as there is a dramatic decrease in demand at and around the peak period at 16.30-18.00 hours. During the Winter period there are obvious price peaks at this time which should discourage the participating consumers from presenting a high demand. (2) Non-load manaoement January weekday In this case no 'call for curtailment' will have been made to the consumers. However, as shown, the level of spot-prices still effected a slight pull-back of demand at the peak time. See Figure 17. (3) JanuarySunday As might be expected, there is little obvious load response because of relatively low industrial load level during the afternoon of Sunday. This is in close agreement with observed consumer behaviour. See Figure 18. (4) Summer weekday There is little or no observed effect on the demand at the peak periods. Similarly because of the relatively low level of the spot-prices there was no obvious response of the consumers. See Figure 19.

~

i\ ,ooi

/ \

~

CErIENT WORKS i~ITERNAIIOtI'~L TELEPHONEEXCHANGE.

/ / F ENGIHEERG It'WORKsF /OUNDR¥

2000! ,ooot ........

. . . . . . . . . . . . . . . .

. . . .

0 3 8 S 12 15 18 21 24 27 30 33 38 39 42 45 48 HALF/HOURLf PERIODS (STARTS OO,30HRS)

Figure 16. Profiles for a load-management January weekday 3500. CUSTOMERLOAO(kW) 3000. 2000.2500'

/.-.-CEMENT WORKS //--INTERNATIONAL IELEPHONEEXCHANGE, ~/--ENGINEERING WORKS/FOUNDRT

I000.

500. 0 0' '~' '~' '~'' i~ 'i~ 'i~ '~l' '~4 '~4 '~d '~ '~6 '~ '4~ '4~ '4e HALF/HOURLYPERIODS(STARTSOO,3OHRS)

Figure 17. Profiles January weekday

46

I

I

3OO0~

i

CEMENTWORKS

INTERH~TIONAL TELEFNONEEXCHANGE. ENGINEERINGWORKS/FOUNDRY

2500]

2oooJ 15001 Iooo~ !

/

s°°l _._____ O 3 6 9

12 15 tO 21 24 27 30 33 36 39 42 45 4G HALF/HOURLTPERIODS(STARrS OO,30HRS)

Figure 18. Profiles for January Sunday

(kW)

3500. CUSTOMERLOAD 3000.

2000.2500"~

/~CEMENT WORKS //--- INTERNATIONALTELEPHONEEXCHANGE. //C EN61NEERI NG WORKs/FOI'~NDRY

~ / /

// ~

15O0.

IO00] ~

I-0~LF/HOURLTPERIODS(STARTSOD,3OHRS)

3500.~ cusror~Ee LOAD(kW') I 3000" ~

3500~ CUSTOMERLOAD(kW)

for

a

non-load-management

Figure 19. Profiles for Summer weekday

IX. C o n c l u s i o n s With the privatization of the power supply industry in the UK, the scene is set for the introduction of spot-price tariffs to be seriously considered. This paper has attempted to show that this pricing method is consistent with the 'market force' features thought to play an essential part in the new industry structure. Emphasis has been placed on the fact that the most likely consumer participants would come from the industrial and commercial sectors as these could be considered as profit-maximizing and also in a position to make the investment in the equipment required to take part in a spot-pricing based tariff scenario. Similarly, the necessity for these consumers to be able to anticipate the price levels and spot-price behaviour has been shown to be extremely important. This paper has utilized several price models which would address this requirement giving accurate forecasts of the spot-prices. Generalized consumer process models were presented and used in the development of operating policy optimizing algorithms. This is a significant study given that the need for information consultants has been separately identified. It has been shown that the consumers can operate their process within production constraints in the light of rapidly changing prices and still minimize costs by avoiding high price periods. With some spot-price schemes already having been tried in the USA, it is timely to consider these issues in

Electrical Power & Energy Systems

Optimized reaction to spot-price tariffs: J. R. McDonald et al

the UK from both the supplier's and consumer's perspective.

X. A c k n o w l e d g m e n t s Jim McDonald wishes to extend his gratitude to the following of his industrial and academic colleagues for their contributions to the realization of this paper: Mr Roy Howard and Mrs Liz Cutting of Eastern Electricity, Mr Tom Berrie (Consultant) and Mr Bob Peddie (Consultant).

where T(J)(i) = optimal cost for the problem with state i,

stage cost ~ and cost-to-go function ctJ averaged over the transition probabilities T,(J)(i) = cost corresponding to a particular policy {/1,/~,/~. . . . } for the same problem. The proof of the application of contraction mapping to convergence upon an optimal solution can be found in References 8, 9 and 10. The state space combinations therefore lead us to defining the functions J and T~(J) as n-dimensional vectors:

?.,1

Xl. References 1 Bohn, R 'A theoretical analysis of customer response to rapidly changing electricity prices' MIT Working Paper No MIT-EL 81-001WP (January 1981) 2 McDonald, J R, Whiting PA and Lo, K L 'Spot-pricing: evaluation, simulation and modelling of dynamic tariff structures', Int. J. Electr. Power Energy Syst. Vol 16 No I (1994) 23-34 3 Allen, J C J and Peddle, R A 'The effect of dynamic electronic pricing on power system monitoring and control' 2nd Int. Conf. Power System Monitoring and Control, Durham, June 1986 (1986) 4 David, A K, Nutt, D J, Chang, C S and Lee, Y C 'The variation of electricity prices in response to supplydemand conditions and devices for consumer interaction' Int. J. Electr. Power Energy Svst. Vol 8 (1986) pp 101-114 5 Caramanis, M C, Bohn, R E and Schwep~, F C 'Optimal spot pricing: practice and theory' IEEE Trans. on PAS Vol PAS 101 No 9 (September 1982) pp 3234-3245 6 Manichaikul, Y and Schweppe, F C 'Physically based industrial electric load modelling' IEEE Trans. on PAS Vol PAS 98 No 4 (July 1979) pp 1439-1445 7 Bohn,R 'Industrial response to spot electricity prices: some empirical evidence' MIT Working Paper No MIT-EL-80016WP (February 1980) 8 Bertsekas, D P Dynamic programming: Determhfistic and stochastic models Prentice-Hall, Englewood Cliffs, NJ (1987) 9 Whittle, P Optimisation over time." Dynamic programmhtg and stochastic control Vol I Wiley, New York (1982) 10 Whittle, P Optimisati~.n over time: Dynamic programming and stochastic control Vol II Wiley, New York (1983)

Appendix I

LJ(n)/

(A.3)

L(j)(1)-

{T.(J)]

I I =

T~(J.)(2)

r.(J)(n)

In addition to these vectors, we can form the transition probability matrix based on the Markov model for the spot-price proposed earlier. The matrix has the following structure:

[p,]=

-Pill/a(1)'] P21[./t(2)]

... ".'"

pl,r~(l)l 1 P2,[./-t(2)]

_p,l[-/.t(n)l

""

p,n[/.t(n)l.J

The average cost vector gu is defined by:

[g,] =

-gl-], p(])-I1 gE2,.p(2)]

Thus we can reform Equation (A.2) into its matrix equivalent as follows: [Tu(J)] = [gul + ~ [ P , ] [ J ]

O(i, u) +

uEU(i) w

~x ~ pij(u)J(J')), j=l

i = 1,2,3 . . . . . n

(A.I)

and Tlt(J)(i)

= (t(i, ~(i)) + cx ~ p,i(~(i))J(j), j=l

i=1,2,3 ..... n

Volume 16 Number 1 1994

(A.2)

(A.6)

There exists a cost function [J,] which corresponds to a stationary policy which provides the unique solution to the following equation:

(A.?)

This leads us to an equation (I-ll - ct[P~,])[Ju] = [gfl

(A.8)

or equivalently, [Ju] = ([I] - ~t[Pul)-~[gu]

T(J)(i) = min

(A.5)

_g[n,p(n)]J

[J.] = [T.(J,)] = [g,] + ~[P,][J,]

A contraction mapping such as T(J)(x) can be thought of as a function which is defined on the state space S. T is a mapping which transforms a function J (in our case the cost-to-go function) defined on S into another function T(J) also on S as follows:

(A.4)

(A.9)

[I] denotes the n × n identity matrix. It is important to note that the invertibility of the matrix ([I] - ~t[Pu]) is assured. The equation to be solved, (A.9), in order to identify the consumer's optimal policy, can be dealt with by using a powerful numerical technique. As shown earlier, the full state, decision and disturbance spaces are discretized in our problem. This can lead to the matrices [J~,], [P~,] and [g,] becoming large.

47

Optimized reaction to spot-price tariffs." J. R. McDonald et al In itself, the optimality equation can be solved by straightforward matrix manipulation. However, the property of contraction mappings is used in order to identify the optimal solution, in this case, the minimizing policy. The policy iteration algorithm operates as follows. An initial stationary policy s ° = {/ao, 11o,/ao . . . . } is adopted, and the corresponding cost function j o = j o is calculated. Then an improved policy rt 1 = {/a t ,/at,/at. . . . . } is computed by the minimization in the dynamic programming equation (A.9) corresponding to Jp° and the process is repeated. The algorithm is based on the following proposition. Let 7z = {/a,/a,/a . . . . }

(A.10)

The policy iteration algorithm is structured as follows.

Step 1 - Initialization. Propose an initial policy. no = (/ao,/ao,/ao ....

Step 2 - Policy evaluation. Given ~k ~

f, k .k k 'l/Xl, /u'2, /a3 . . . .

= {/~,/~,/~ . . . . }

(A.11)

compute the corresponding cost function J,, from the linear system of equations, {[I-]

~[Pu,2}EJ.,]

-

= I-g.,]

Step 3 - Policy hnprovement. Obtain ,,,+,

=

....

i6 S

j=l

=min[~t(i,u)+~ ~ P,j(u'd.k(J' 3

g[i, /~(i)] + '~. p,i[~(i)]J~(j)

uEU(i)

j=l

= min g(i, u) + ct ~ pljru-]J.(j) 1 u~Uli)

j=l

= r(J.,)

lfJ.k = T(J.k) then stop, otherwise, (A.13)

This equation represents the contraction mappings corresponding to the cost incurred by the consumers in employing the policies n and ~. It follows therefore that

i~S

j= 1

or equivalently, (A.12)

or equivalently

Tf,(J.) = Z(J,)

a new policy,

}

gri,/a k+ '(i)3 + ct ~ pq[/ak+,(i)-]jvk(.j)

be stationary policies such that

J~(i) <~ J~(i),

the policy,

}

satisfying for all system states,

and

(A.14)

Furthermore, if n is not optimal, then strict inequality holds in this equation.

48

}

return to step 2 and

repeat the process.

Since the collection of all the possible admissible policies is finite (by the finiteness of the state, control and disturbance spaces) and an improved policy is generated at every iteration, it follows that the algorithm will find an optimal stationary policy in a finite number of iterations and thereby terminate. This is the main advantage of the policy iteration algorithm.

Electrical Power & Energy Systems