- Email: [email protected]
Formal integration: A program transformation technique
(',mlputer Lanslumles.Vol 6, p p 35 to 46, 1981
0096-0551 81 0100~5-125tl2 0tl 0 Copyrighl ~, 1981 Pcrg~mon Prc~s I Id
Printed in Great Brilain. All rights reserved
FORMAL
INTEGRATION:
TRANSFORMATION
A PROGRAM
TECHNIQUEt
MICHA SHAR1R Tel Aviv University, Department of Mathematical Sciences. Ramat Aviv, Tel Aviv, Israel
(Received 17 September 1980) Abstraet--A simple yet effective program transformation technique which turns complicated high-level expressions into incremental updating computations is introduced and discussed. This transformation, called Formal Integration, is based on a combination of general loop-fusion methods and formal differentiation. Its applicability is demonstrated in a variety of examples Program transformation
Formal differentiation
Formal integration
Loop fusion
1. I N T R O D U C T I O N
In this paper we study a simple program transformation technique called Formal Integration which aims at transforming high-level expressions into iterative computations which build up the value of the desired expression incrementally. In many cases "naive" computation of the original expression will be very inefficient, and when this is true the equivalent incremental computation obtained by transformation can improve the program efficiency considerably. The technique that is described is often used by programmers, but only in an informal and ad hoc manner, and we therefore consider its specific identification to be important for the understanding of the program development process, and eventually for the automatic generation of programs. We begin in Section 2 by describing a fundamental program transformation known as Formal Differentiation on which our integration method is based. Section 3 presents the formal integration technique, and Section 4 applies it to a series of examples. Concluding remarks appear in Section 5. Our terminology is largely standard, and uses conventional mathematical notation in combination with a few programming constructs. These include assignments, denoted by " : = " , and also iterations, written using a header of the form (*)
(for all X in A)
where A is any aggregate of objects (e.g. a set, tuple or string), and where iteration is in order if A is linearly ordered (e.g. a tuple or string). The iteration opened by the header (*) is terminated by the keyword "end". Conditional expressions using the "if-then-else" construct are also allowed.
2. F O R M A L
DIFFERENTIATION
To introduce our transformations, a short description is given of the more fundamental program transformation known as formal differentiation, on which they are based. This transformation, introduced by Earley [1], has been studied extensively by Paige and Schwartz (see Ref. [2], and especially Ref. [3]), and also by Fong and Ullman [4]. It generalizes Cocke's well known strength-reduction program optimization technique (cf. [5]) to higher-level objects. It can be described in the following general way: Suppose that we are given a class O of objects, and a family F of transformations between them. Suppose that each h in F takes one object A in O and transforms it into another hA in O, and that all objects A in O can be generated by these transformations i This work has been supported by ONR Grant N00014-75-C-0571.
36
MICHA SHARIR
from a common initial object Ao, i.e. for each A in O there must exist (at least one) sequence h~, h2... hn, of elements of F such that
(*)
A = hnhn-I ... hlAo.
We choose to view these transformations as "small" changes of the objects in O, and assume that there exists some computational model in which the objects in 0 are represented in such a way that any transformation of A in O into hA can be accomplished efficiently. (We will not try to make this notion of computational efficiency precise, but rather will use it informally.) As a typical example illustrating such a set-up, let O be the class of all subsets of some (finite) set E, and define F as follows: For each A subset E and each X in E, let X A denote AA {X} (note that in this paper " + " will denote set union, "*" will denote set intersection and A will denote set symmetric difference). That is, let X in E also denote a transformation which inserts X into any subset which does not contain it already, and deletes X from any subset which does contain it. Then, if we take the null subset as an initial object Ao, any set can be built up by a sequence (*) of such transformations, namely by successively inserting all its elements into the null set. Next let O1 and 02 be two such domains, and let F1, F2 be two corresponding families of transformations. Let K be a map from O1 into 02 whose value K(A) at some A in O1 is known. Suppose that A undergoes a small change, i.e. that we apply some h in F1 to it. If the corresponding change in K(A) is also small, i.e. if there exists g in F2 such that K(hA) = gK(A) then, following Paige and Schwartz, we will say that K is continuous at A (with respect to the change h), and (abusing standard terminology slightly) call g the formal derivative of K at A with respect to h. K will also be called continuous if g is a composition of a few elements of F2, in which case we can retain the intuitive idea that application of g is computationally efficient. As an example, we can return to the set-theoretic case considered earlier, and assume that O1 and 02 are power sets of two finite sets El, E2 respectively, and that F1, F2 are identified with El, E2 respectively in the manner explained above. The derivative of K(A) at A with respect to some X in El, which we will denote as DK(A, X), can then be identified with some (hopefully small) subset of E2, so that we can write K(XA) = K(A) A DK(A, X). For such continuous mappings K the following program transformation is available. Suppose that K(A) is repeatedly computed within some program loop, and that between successive computations of K(A), A undergoes only small changes (effected by transformations in F1). If K is continuous, then it is possible to eliminate the successive computations of K in this loop and instead do the following: (a) Compute an initial value of K upon entry to the loop. (b) Before each point at which A is changed by assignments of the form A:=hA insert the corresponding updating of K K:=gK where g is the formal derivative of K at A with respect to h. (Here K denotes a variable holding the current value of K(A).) This transformation, explored most extensively by Paige, is known as formal differentiation [2, 3, 6, 7]. As already noted, it generalizes the known technique of reduction in strength of algebraic operations 1-5] to higher level domains. Situations in which it becomes applicable occur quite frequently in programs written using very high level
Formal integration: a program transformation technique
37
dictions. In many such cases the transformed p r o g r a m becomes more efficient than the original one by at least an order of magnitude. A series of such examples can be found in [3]. 3. FORMAL INTEGRATION We now go on to describe a new application of formal differentiation to p r o g r a m construction. This involves the following transformation: As in the previous section, let K(A) be an expression in the variable A which is to be computed in a given program. Assume that we know how to construct A "incrementally" from an initial object Ao in O1, i.e. that we know a sequence of transformations hi, h2 . . . . . h n, in F1 such that
A = h,h,_l ...hxAo
(3.1)
and also that we can realize the construction (3.1) using some repetitive p r o g r a m construct such as recursion or iteration. (In this paper we concentrate on iteration, but will also show that our "esults can be easily extended to recursive programs.) Let us assume that our p r o g r a m contains an assignment of the form T : = K(A)
(3.2)
and that K is side-effect free. We can rewrite (3.1) as the following p r o g r a m segment, which builds up a copy A' of A incrementally, and which is inserted just before the point (3.2) at which K(A) is to be computed: A' := Ao; (for all i in [1 ... n]) A ' : = hiA'; end
(3.3)
We can then apply the simple loop-fusion transformation described in [6] to move the computation of K(A) into the loop (3.3). This gives the following equivalent but (temporarily) much less efficient program fragment: A' := Ao; T : = K(Ao); (for all i in [1 ... n])
A' : = hiA' ; T := K(A'); end
(3.4)
Now, if K is continuous at each A' with respect to the corresponding change hi, we can apply formal differentiation to the repeated computation of K in (3.4) to obtain A' := Ao; T := K(Ao); • (f°r all i in [ 1 . . . n ] ) T : = 9~T; A ' : = h~A'; end
(3.5)
where gl is the derivative of K at A' with respect to hi. We call the transformation which carries (3.2) into (3.5) evaluation of K by Formal Integration with respect to A. The name is derived from the analogy between algebraic differentiation and set-theoretic formal differentiation. To emphasize this analogy and also to give a more concrete example of our transformation, let us assume as in Section 2 that both A and K(A) have set values.
38
MICHASHARIR
In this case we can write (3.5) as follows (taking A0 to be the null set):
A ' : = 0; T : = K(0); (for all X in A) T := T ADK(A', X); A':=A'AIX}; (or rather A' := A' + IX] end
(3.6)
The analogy that we have in mind emerges more clearly if we write this in more compact form as A K(A) = K(0)A X in A DK(A', X )
(3.7)
where A' is the collection of all elements considered so far. This equation stands in formal analogy to the fundamental theorem of calculus, since it computes a function by a kind of definite integral of its derivative. R E M A R K : A similar application of formal differentiation to an iterative computation of a set-theoretic expression is noted by Paige [3] (see also [73), who uses a similar mechanism in the initialization phase of his formal differentiation procedure (although the object over which he iterates is usually a set). EXAMPLE: Let us apply formal integration to the following problem: Given a set of nodes N and a set E of directed edges between these nodes, find those nodes that have no predecessors. In formal terms, we want to compute N P = IX in N: for all Yin N : [ E X ]
not in E~;
Let us evaluate the right-hand side by formal integration with respect to E, to obtain a loop of the form (3.6). To this end, let K(E) denote this expression. First observe that K(0) = N. Also, K(E + [[a, b]}) = {X in N:for all Yin N : [ y x ] not in (E + l[a,b]l)l = K(E) * [X in N :for all Y in N : l- Y X] ~ [a, b] } = K ( E ) - IX in N:exists Yin N : [ Y X ] = [a,b]} = K(E) - [b}.
Thus Formal integration yields the following program (in which N P holds the value of K(E'), and where the auxiliary copy E' of E is not used and hence has been eliminated): NP:=N; (for all [a, b] in E) N P := N P - [b}; end This results in a linear program in the size of the graph, instead of a quadratic naive computation of NP. This has been a relatively simple example; more complicated examples will be exhibited in Section 4. REMARKS: ti) The iteration order in the loop (3.3) or (3.6) is still arbitrary. Our transformation is therefore not uniquely defined, since in order to fully, specify it one has also to specify the order of iteration (if more than one is possible). Selection of the "right" order can be crucial to the effectiveness of the transformation, as will be seen in the examples exhibited in Section 4. (2) The intermediate object A' in (3.3) is used there instead of A itself in order to preserve the present value of A, or to use this value to control the iteration [as in (3.6)]. When this is not essential, A itself may be rebuilt incrementally in (3.2), instead of A'. Moreover, it is often the case that A' becomes dead in (3.5) and can therefore be eliminated from the program.
Formal integration: a program transformation technique
39
(3) In some cases there is no need to create the l o o p (3.3), since there already exists a similar loop which builds-up A incrementally in a similar fashion. In such cases we can achieve the effect of formal integration by transforming the existing loop into one similar to (3.5). This reduces formal integration to the loop fusion transformation discussed in [6] (see also [3]). (4) Formal integration can be easily extended to handle composite expressions, or, equivalently, sequences of expressions each depending on the previous one. Consider, for example, a sequence of the form B := K(A); C := L(B);
(3.8)
This can be transformed into a single loop which builds A and simultaneously updates B and C. If the intermediate structure B is not subsequently used, it may even be eliminable, in which case the transformed code evaluates C as the composite function L- K of A by straight formal integration. (5) It is interesting to note that formal integration can be applied using recursion in place of iteration• For simplicity, assume that A is constructed as in (3.1), but that we implement this construction by calling a (linearly) recursive function/](n), defined so that if i = 0 then Ao else hift(i - 1).
~(i) =
Then, in order to compute K(A) by formal integration, we can extend this function so that it returns a pair of values [A, K(A)]. Denoting this extended function as A~, the recursive analog of (3.5) then becomes the evaluation of "~(n), where
AK(i) =
if i = 0 then [Ao, K(Ao)] else [hi(AK(i - 1)(1)), 9i(AK(i - 1)(2))].
(3.9)
This can be easily generalized to allow us to evaluate expressions that depend on trees or on any other recursively-defined object by formal integration. The remarks (1)-(4) apply, with obvious modifications, in the recursive case as well.
4. EXAMPLES Next we present a series of examples showing the application of formal integration in a variety of situations, proceeding now in a somewhat informal manner, since our major aim is to demonstrate the use of formal integration as a methodological programming tool designe d to be used by human programmers as well as by automatic systems. Note, however, that given a system capable of performing formal differentiation, such as that described in detail in [7], it would be relatively easy to extend it so that it can also perform formal integration. See also a concluding remark in Section 5.
I. Simple iterative expressions We begin with a very simple example, which illustrates the general observation that if the expression K(A) is simple enough to admit an "obvious" method of calculation, formal integration will simply recover this obvious technique. Suppose, to be specific, that A is a set, and let
(*)
K(A) = IX in AIP(X)I
where P(X) is some predicate. To transform this expression into a loop of the form (3.6), note that L(0) = 0, and that
DK(A,,X)={~X},
if P(X), otherwise.
40
MIdh, SHARIR
"
is independent of A', which is therefore eliminable from (3.6). We thus see that K(A) can be computed by T:=0; (for all X in A) T := T + (if P(X) then I X I else 0); end of course, this is the way in which K(A) is likely to be computed in any system recognizing the construct (*). II. A maximum occurrences problem Next let S be a string of characters, and suppose that we wish to find a character C which has a maximal number of occurrences in S. To write C as an expression in S, we need a notation which is lacking also in conventional mathematics, namely an operator which, when applied to some aggregate A and a (say real-valued) functionfover A, yields an element X in A for which f(X) is maximal. (The standard notation max
f (X)
XinA
yields the maximum value f(X), not X itself.) We will denote the required operator as max2, and define it so that when it is applied to a set f of pairs [X, f(X)] :X in A } it yields a pair whose second component is maximal. Having introduced this we can write a = [C,n] = max2[[X, # [iin[1 ... #S]:S(i) = Xl]:XinS}, where # gives the cardinality of an aggregate. We can evaluate a by formal integration with respect to S. S, being a string, can be generated by concatenating all its characters from left to right, so that (3.3) becomes S ' : = nullstring; (for all b in S) S' := S' with b; end
(add b to the right of S')
Let K(S) = max2 L(S) denote the expression defining a. Let us first determine how L(S) changes when b is concatenated to S. Obviously the only pair in L(S) which changes is that which begins with b, and the change consists of adding 1 to its second component. Regarding L(S) as a function, we can express this change as L(S with b ) = L(S)- lib, L(S)(b)]] + [[b.L(S)(b)+ 1]]. This implies that the corresponding change in K(S) is
K(S with b) = max2 {K(S), [b, L (S)(b) + 1]}. (Note that here we integrate a composite expression, as in (3.8).) Here, however, the intermediate data structure L cannot be eliminated, since it is used to update K(S). Thus we end up with the following loop which updates both structures as it rebuilds S (In what follows L' and K' are variables holding the values of L(S') and K(S') respectively; note also that S' has been eliminated from the loop, since it is not needed there; finally
Formal integration: a program transformation technique
41
note that L' is represented as a function, and that the functional assignment in the loop corresponds precisely to the change in L(S) given above): L ' : = 0; K ' : = fl; (denoting an undefined value) (for all b in S) L'(b) : = L'(b) + 1; K' := max2 {K', L'(b)]}; end It is interesting to note that when this problem was given to 100 students in an advanced undergraduate programming course, only a few (10) solved it by a program similar to the one derived above. Most of them computed L(S) first and then searched it to determine the maximum.
III. Longest upsequence This problem, posed by Dijkstra [8, 9] can be stated as follows: Given a sequence A~, A 2 . . . An of integers, find the length of the longest upsequence of A, i.e. the longest subsequence whose elements are in ascending order. More formally, we wish to compute I, where / = max[#Q:Q
c l l . . . n ' , ] f o r a l l i , . j i n Q : i < , j = ~ A i <~ Aj',.
A naive computation of 1 is exponential. To solve this problem in a more efficient way, we can evaluate 1 by formal integration with respect to the set S = [ 1 . . . n } . More specifically, we reconstruct S by iterating over its elements from left to right. (In this case selection of the "right" way of iterating through S has a crucial impact on the success of the transformation; the iteration order that we choose builds S up in a natural and obvious way, and also turns out to be "right".) Here (3.3) has the following form: S' := 0; (for all k in [ 1 . . . n ] ) S' := S' + {k}; end Let us write the expression defining l as
K(S) = max L(S) and observe that L(0) = {0}, and hence K(0) = 0. To proceed, we must differentiate L(S') with respect to the change in S' in the above loop. It is easily seen that
L(S' + {k]) = L(S') + [#Q:Qc S'+ {k}lkinQand for alli, j i n Q : i < j ~ A i < . A j}.) Hence the corresponding change in K is
K(S' + {k}) = m a x {K(S'), H(k)} where
H(k) = max [ # Q : Q ~ {1 ... k} Ik in Q and for all i, j in Q:i < j ~ Ai <~ A i}.
42
MICHA SHARIR
It thus remains to compute H(k) in a more efficient manner. To this end we expand H(k) as follows:
H ( k ) = 1 + m a x { # Q ' : Q ' c { 1 . . . k - 1}1 for all i, j in Q': i < j =:, Ai <<.Aj and for all i in Q':Ai <<.Ak} = 1 +max~#Q':Q'c ll...k1}l for all i, j in Q' :i < j ~ Ai <~ Aj and Am~xQ' ~< Akl = 1 + m a x l H ( j ) : j i n I1 . . . k - 1}fAj ~< Ak}. Having arrived at this equation, we can discard S' and L(S') altogether, and maintain only H, to obtain the following program: /:=0; H:=g; (for all k in [ 1 . . . n ] ) n(k) := 1 + max IH(j):jin I1 . . . k - 1}[Aj ~< Ak}; l := max l l, H(k)} ; end This is a reasonably efficient algorithm for computing the longest upsequence in A [having time complexity O(n2)], although it is not as efficient as Dijkstra's O[n'log(n)] algorithm, which considers an intermediate object different from and more subtle than the function H. IV. A simple recursive application: rightmost appearance in a list Next we shall show how formal integration can be applied in a recursive environment. Given a linear list L and an object A, consider the problem of finding the rightmost appearance of A in L(if one exists). Formally, we can write
R = m a x l j : L j = AI where it is understood that application of "max" to an empty set yields an undefined value (which we denote as f2). We assume that the basic list operations are "'hd L" (which gives the first element of L), "tl L" (which gives the tail of the list) and "conc(B, L)" (which produces a list whose first element is B, and whose tail in L). Then we can regenerate L using the following recursive code:
L(L) =
if L = empt3/then empty else conc(hd L, L(tl L))
where "empty" denotes the empty list. To apply formal integration in this setting, let K(L) denote the expression defining R. Note first that K(empty) = f2. It remains to differentiate K with respect to the change in L appearing in the above recursive equation. To this end, we put L ' : = conc(B, L), where B stands for hd L, and then express K(L') in terms of K(L). This gives K(L') = m a x l j : L j = AI = max((if B = A then I1~ else 0) + [ j : L ( j - 1 ) = AI). This last equation can be simplified by splitting it into two cases" If K(L) is undefined then only the first term in the above equation plays a role. On the other hand, if K(L) is
Formal integration: a program transformation technique
43
defined, then the second term contains numbers > 1 so that the first term can be discarded. All this gives
K(L') = if K(L) = ~ then (if B = A then 1 else ~) else 1 + K(L). Since this equation does not involve L explicitly, we can eliminate L from (3.9) to arrive at the following recursive computation of R: /~(L) = if L = empty then else if R(tl L) = f~ then if h d L = A then 1 else f2 else 1 + R(tl L). R E M A R K : When applied in a recursive environment, formal integration usually turns out to yield the same transformations which can be generated by the more general (but less structured) system of folding and unfolding transformations described by Burstall and Darlington [10]. Loosely put, we may view formal integration as some kind of "macro" which can ~oe defined in terms of the folding/unfolding primitives.
V. Garbage collection This example, which has been considered in Ref. [6], and more extensively in [12], simplifies when reformulated as an application of formal integration. The problem is to give a systematic derivation of the garbage collection and compaction algorithm of Dewar and McCann [11] from its high-level specifications. In this problem we deal with a heap, represented as a map H from addresses of block headers to tuples representing the block contents (which, for simplicity, are assumed to consist solely of pointers). Only blocks which can be reached from the first block, starting at 0, via a chain of pointers are to be regarded as active, and the problem requires us to compact these blocks together towards 0, and to adjust pointer values appropriately. In formal terms, we want to compute the new storage layout map H' defined by the following 3 equations: [U. # U] = min21[A, # A ] : A ~ domain H I0in A and
(for all X in A, Yin H(X): Yin AI. ll) The set U thereby defined is the collection of all active cells. (The operator rain2 is defined in a way similar to the definition of max2 above.) N = [ [ b , Z [ # H ( c ) : c in Ulc < b]]:b in U]
(2)
This defines the map N from active block headers to their new compacted addresses.
H' = l[Nlb), [N(c):c in H(b)]]:b in UI
(3)
Here H' is the new storage layout map, in which active blocks are "moved" to their new locations, and pointers within them are adjusted accordingly. While it is quite tempting to attempt evaluation of H' by direct formal integration with respect to H, this turns out to be infeasible for two reasons: (a) U is defined as the transitive closure of [01 under the map H; computation of U by incremental updating as H is grown incrementally is not efficient, so that it is better to compute U by standard transitive closure methods. (However such algorithms can in fact be obtained by combining formal differentiation with other program transformation techniques, for which see Ref. [6].I (b) The expression (3) which defines H' in terms of N, does not admit of an efficient formal derivative, mainly because of the manner in which N first appears within
44
MICHA SHARIR
that expression. Instead, to obtain an expression with a "good" derivative, we split the third expression into two subexpressions as follows: Q = lib, [N(c):c in H(b)]] :b in U} H' = {IN(b), Q(b)] : b in U } (Heuristically, Q represents the storage layout after pointers have been adjusted, but before blocks have been moved.) Taking the above considerations into account, it is still feasible to attempt evaluation of Q by formal integration with respect to U. We will show now that this can indeed be done quite effectively. Since Q is a composite expression of U, we perform this evaluation by iterating over U, building a copy U' and simultaneously updating both N and Q as U' is grown. Let us agree to iterate over U in increasing order of its elements. (3.3) will then have the following form: U ' : = 0; (for all b in U) U' := U' + [bl; end To apply formal integration, observe first that when U = 0, we have N=0 Q = {[b, [f2:c in H(b)]] :b in U} where, as before, f2 denotes an undefined value. (One should compare this initial value of Q with the similar initial value of H' to convince oneself of the much greater difficulty of differentiating H' than of differentiating Q,) Next we note that since we iterate over U in increasing order we have N(U' + [bl) = N(U') + l[b,Z[ #H(c):cin U']]} so if we define T = T(U') = Z [ # H ( c ) : c i n U'] then T is also an efficiently differentiable expression of U'. Indeed, T(0) = 0, and T(U' + [b]) = T(U') + #H(b). All this yields an efficient evaluation procedure for N by formal integration. Passing to Q, we have to express Q(N + [[b, T]}) in terms of Q(N). It is easily seen that Q changes only at pairs [c, [N(d):d in H(c)]] for which b is in H(c), and for each component in H(c) equal to b, the corresponding component in Q(N)(c) (which is presently undefined) should be set to T. In order to make this process time efficient, we want to use an auxiliary data structure R which maps each b in U to the collection of all pairs [c, i-I for which H(c)(i) = b. Written as a relation R = {[H(c)(i), [c, i ] ] : c i n U , / i n [ 1 . . . #H(c)]}. (The use of auxiliary data structures in formal differentiation is quite common. Paige I-3] calls expressions whose differentiation requires the use of auxiliary structures discontinuous, and describes in detail techniques for the introduction of such structures for differentiation of set theoretic expressions.) To sum up, the transformation produces the follow-
Formal integration: a program transformationtechnique
45
ing program fragment (in which U' and N are not used and hence eliminated): Q : = I[b, [ff2:c in H(b)]] :b in U I ; R := ~[H(b)(i), [b, i]] :b in U, i in [1 ... # H(b)]l ; T:=0; (for all b in U) (for all [c, i] in Rlbl) O(c)(i) := T; end; T := T + #H(b); end It should be noted that the program can be further transformed by evaluating both R and the initial expression for Q by formal integration with respect to U. Since U is built up incrementally in a preceding transitive closure step, there is no need to create a new loop of the form (3.3). Instead, assuming the initial value of U to be 0, we initialize both Q and R to 0. Then, whenever U is augmented by an element a U := U + {a}; we also update Q and R by their respective formal derivatives. After some massaging, the update code will have the form:
z:=[]; (for all i in [ 1 , . . #H(a)]) Z : = Z + [f2]; R := R + {[H(a)(i), [a, i]]}; end; Q(a) := Z; The last two algorithm fragments constitute a significant part of the Dewar-McCann algorithm [11]. It is instructive to note that several highly inventive programming decisions reflected in these fragments are derivable by judicious but systematic applications of formal integration. {As noted earlier, the idea of fusing two initialization loops together using formal differentiation, has initially been proposed by Paige [3].) 5. CONCLUSIONS In this paper we have described a simple but general program transformation called formal integration and demonstrated its applicability to a series of examples. Our main concern is methodological; that is, we have tried to view formal integration as a tool to be used systematically (but manually) in program development. Availability of this tool makes construction of certain algorithms a routine task, thereby relieving the programmer from the need to re-invent this transformation anew for each specific application. Moreover, once we have identified formal integration as a generally applicable transformation, it becomes possible to give concise descriptions of programs and algorithms, such as for instance: "This program computes expression Y by formal integration with respect to its parameter X." Such a statement will often define a program almost uniquely, in a form allowing easy communication and explanation of the program. We have not emphasized the rigorous formalization and mechanization of formal integration, since to do so would make it necessary to face more difficult technical problems. However, we expect that formal integration could already become part of a semi-automatic program development system (applicable to at least restricted classes of expressions; for example a modified version of the system presented in [7] should be capable of applying formal integration to certain set-theoretic expressions.) REFERENCES I. J. Earley, High-level iterators and a method for automatically designing data structure representation, J. Comput. Lang. 1, 321-342 (1976).
46
MICHA SHARIR
2~ R. Paige and J. T. Schwartz, Expression continuity and the Formal differentiation of algorithms, Proc. 4th POPL Co~!f 58-71 (1977). 3. R. Paige, Expression continuity and the Formal differentiation or algorithms, Courant Computer Science Report No. 15, Courant Institute (1979). 4~ A. C. Fong and J. D. Ullman, Inductive variables in very high level languages, Proc. 3rd POPL Conf. 104-112 (1976). 5, A. V. Aho and J. D. Ullman, Principles of Compiler Design. Addison-Wesley, New York (1977). 6. M. Sharir, Some observations concerning Formal differentiation of set theoretic expressions, Technical Report No. 16; Computer Science Department, Courant Institute 11979). To appear in TOPLAS. 7. R. Paige and S. Koenig, Finite differencing of computable Expressions, Technical Report LCSR-TR-8, Laboratory for Computer Science Research, Rutgers University (1980). 8. E. W. Dijkstra, Maximum length of a weakly ascending subsequence of a givefi sequence, International Summer School on Program Construction, Marktoberdorf, Germany (1978). 9. E. W. Dijkstra, Some beautiful arguments using mathematical induction, Acta Informat. 13, 1-8 (1980). 10. R. Burstall and J. Darlington, A transformation system for developing recursive programs, J. Ass. Comput. Mach. 24, 44-67 (1977). 11. R. B. K. Dewar and A. P. McCann, MACRO SPITBOL--a SNOBOL4 Compiler, Software Practice and Experience 7, 95-113 (1977). 12. R. B. K. Dewar, M. Sharir and E. Weixelbaum, Transformational derivation of a garbage collection algorithm, To be published. 13. A. C. Fong, Inductively computable constructs in very high level languages, Proc. 6th POPL Conf. 21-28 (1979). About the AUlbor--MIcHA SHARIR was born in 1950, received his Ph.D. in Mathematics at Tel-Aviv University in 1976. He has been a visiting assistant scientist at Courant Institute, New York University, until 1980, where he was working on the SETL project and he is now at Tel-Aviv University. His main research areas are analysis, optimization and transformation of very high level programs.
0096-0551 81 0100~5-125tl2 0tl 0 Copyrighl ~, 1981 Pcrg~mon Prc~s I Id
Printed in Great Brilain. All rights reserved
FORMAL
INTEGRATION:
TRANSFORMATION
A PROGRAM
TECHNIQUEt
MICHA SHAR1R Tel Aviv University, Department of Mathematical Sciences. Ramat Aviv, Tel Aviv, Israel
(Received 17 September 1980) Abstraet--A simple yet effective program transformation technique which turns complicated high-level expressions into incremental updating computations is introduced and discussed. This transformation, called Formal Integration, is based on a combination of general loop-fusion methods and formal differentiation. Its applicability is demonstrated in a variety of examples Program transformation
Formal differentiation
Formal integration
Loop fusion
1. I N T R O D U C T I O N
In this paper we study a simple program transformation technique called Formal Integration which aims at transforming high-level expressions into iterative computations which build up the value of the desired expression incrementally. In many cases "naive" computation of the original expression will be very inefficient, and when this is true the equivalent incremental computation obtained by transformation can improve the program efficiency considerably. The technique that is described is often used by programmers, but only in an informal and ad hoc manner, and we therefore consider its specific identification to be important for the understanding of the program development process, and eventually for the automatic generation of programs. We begin in Section 2 by describing a fundamental program transformation known as Formal Differentiation on which our integration method is based. Section 3 presents the formal integration technique, and Section 4 applies it to a series of examples. Concluding remarks appear in Section 5. Our terminology is largely standard, and uses conventional mathematical notation in combination with a few programming constructs. These include assignments, denoted by " : = " , and also iterations, written using a header of the form (*)
(for all X in A)
where A is any aggregate of objects (e.g. a set, tuple or string), and where iteration is in order if A is linearly ordered (e.g. a tuple or string). The iteration opened by the header (*) is terminated by the keyword "end". Conditional expressions using the "if-then-else" construct are also allowed.
2. F O R M A L
DIFFERENTIATION
To introduce our transformations, a short description is given of the more fundamental program transformation known as formal differentiation, on which they are based. This transformation, introduced by Earley [1], has been studied extensively by Paige and Schwartz (see Ref. [2], and especially Ref. [3]), and also by Fong and Ullman [4]. It generalizes Cocke's well known strength-reduction program optimization technique (cf. [5]) to higher-level objects. It can be described in the following general way: Suppose that we are given a class O of objects, and a family F of transformations between them. Suppose that each h in F takes one object A in O and transforms it into another hA in O, and that all objects A in O can be generated by these transformations i This work has been supported by ONR Grant N00014-75-C-0571.
36
MICHA SHARIR
from a common initial object Ao, i.e. for each A in O there must exist (at least one) sequence h~, h2... hn, of elements of F such that
(*)
A = hnhn-I ... hlAo.
We choose to view these transformations as "small" changes of the objects in O, and assume that there exists some computational model in which the objects in 0 are represented in such a way that any transformation of A in O into hA can be accomplished efficiently. (We will not try to make this notion of computational efficiency precise, but rather will use it informally.) As a typical example illustrating such a set-up, let O be the class of all subsets of some (finite) set E, and define F as follows: For each A subset E and each X in E, let X A denote AA {X} (note that in this paper " + " will denote set union, "*" will denote set intersection and A will denote set symmetric difference). That is, let X in E also denote a transformation which inserts X into any subset which does not contain it already, and deletes X from any subset which does contain it. Then, if we take the null subset as an initial object Ao, any set can be built up by a sequence (*) of such transformations, namely by successively inserting all its elements into the null set. Next let O1 and 02 be two such domains, and let F1, F2 be two corresponding families of transformations. Let K be a map from O1 into 02 whose value K(A) at some A in O1 is known. Suppose that A undergoes a small change, i.e. that we apply some h in F1 to it. If the corresponding change in K(A) is also small, i.e. if there exists g in F2 such that K(hA) = gK(A) then, following Paige and Schwartz, we will say that K is continuous at A (with respect to the change h), and (abusing standard terminology slightly) call g the formal derivative of K at A with respect to h. K will also be called continuous if g is a composition of a few elements of F2, in which case we can retain the intuitive idea that application of g is computationally efficient. As an example, we can return to the set-theoretic case considered earlier, and assume that O1 and 02 are power sets of two finite sets El, E2 respectively, and that F1, F2 are identified with El, E2 respectively in the manner explained above. The derivative of K(A) at A with respect to some X in El, which we will denote as DK(A, X), can then be identified with some (hopefully small) subset of E2, so that we can write K(XA) = K(A) A DK(A, X). For such continuous mappings K the following program transformation is available. Suppose that K(A) is repeatedly computed within some program loop, and that between successive computations of K(A), A undergoes only small changes (effected by transformations in F1). If K is continuous, then it is possible to eliminate the successive computations of K in this loop and instead do the following: (a) Compute an initial value of K upon entry to the loop. (b) Before each point at which A is changed by assignments of the form A:=hA insert the corresponding updating of K K:=gK where g is the formal derivative of K at A with respect to h. (Here K denotes a variable holding the current value of K(A).) This transformation, explored most extensively by Paige, is known as formal differentiation [2, 3, 6, 7]. As already noted, it generalizes the known technique of reduction in strength of algebraic operations 1-5] to higher level domains. Situations in which it becomes applicable occur quite frequently in programs written using very high level
Formal integration: a program transformation technique
37
dictions. In many such cases the transformed p r o g r a m becomes more efficient than the original one by at least an order of magnitude. A series of such examples can be found in [3]. 3. FORMAL INTEGRATION We now go on to describe a new application of formal differentiation to p r o g r a m construction. This involves the following transformation: As in the previous section, let K(A) be an expression in the variable A which is to be computed in a given program. Assume that we know how to construct A "incrementally" from an initial object Ao in O1, i.e. that we know a sequence of transformations hi, h2 . . . . . h n, in F1 such that
A = h,h,_l ...hxAo
(3.1)
and also that we can realize the construction (3.1) using some repetitive p r o g r a m construct such as recursion or iteration. (In this paper we concentrate on iteration, but will also show that our "esults can be easily extended to recursive programs.) Let us assume that our p r o g r a m contains an assignment of the form T : = K(A)
(3.2)
and that K is side-effect free. We can rewrite (3.1) as the following p r o g r a m segment, which builds up a copy A' of A incrementally, and which is inserted just before the point (3.2) at which K(A) is to be computed: A' := Ao; (for all i in [1 ... n]) A ' : = hiA'; end
(3.3)
We can then apply the simple loop-fusion transformation described in [6] to move the computation of K(A) into the loop (3.3). This gives the following equivalent but (temporarily) much less efficient program fragment: A' := Ao; T : = K(Ao); (for all i in [1 ... n])
A' : = hiA' ; T := K(A'); end
(3.4)
Now, if K is continuous at each A' with respect to the corresponding change hi, we can apply formal differentiation to the repeated computation of K in (3.4) to obtain A' := Ao; T := K(Ao); • (f°r all i in [ 1 . . . n ] ) T : = 9~T; A ' : = h~A'; end
(3.5)
where gl is the derivative of K at A' with respect to hi. We call the transformation which carries (3.2) into (3.5) evaluation of K by Formal Integration with respect to A. The name is derived from the analogy between algebraic differentiation and set-theoretic formal differentiation. To emphasize this analogy and also to give a more concrete example of our transformation, let us assume as in Section 2 that both A and K(A) have set values.
38
MICHASHARIR
In this case we can write (3.5) as follows (taking A0 to be the null set):
A ' : = 0; T : = K(0); (for all X in A) T := T ADK(A', X); A':=A'AIX}; (or rather A' := A' + IX] end
(3.6)
The analogy that we have in mind emerges more clearly if we write this in more compact form as A K(A) = K(0)A X in A DK(A', X )
(3.7)
where A' is the collection of all elements considered so far. This equation stands in formal analogy to the fundamental theorem of calculus, since it computes a function by a kind of definite integral of its derivative. R E M A R K : A similar application of formal differentiation to an iterative computation of a set-theoretic expression is noted by Paige [3] (see also [73), who uses a similar mechanism in the initialization phase of his formal differentiation procedure (although the object over which he iterates is usually a set). EXAMPLE: Let us apply formal integration to the following problem: Given a set of nodes N and a set E of directed edges between these nodes, find those nodes that have no predecessors. In formal terms, we want to compute N P = IX in N: for all Yin N : [ E X ]
not in E~;
Let us evaluate the right-hand side by formal integration with respect to E, to obtain a loop of the form (3.6). To this end, let K(E) denote this expression. First observe that K(0) = N. Also, K(E + [[a, b]}) = {X in N:for all Yin N : [ y x ] not in (E + l[a,b]l)l = K(E) * [X in N :for all Y in N : l- Y X] ~ [a, b] } = K ( E ) - IX in N:exists Yin N : [ Y X ] = [a,b]} = K(E) - [b}.
Thus Formal integration yields the following program (in which N P holds the value of K(E'), and where the auxiliary copy E' of E is not used and hence has been eliminated): NP:=N; (for all [a, b] in E) N P := N P - [b}; end This results in a linear program in the size of the graph, instead of a quadratic naive computation of NP. This has been a relatively simple example; more complicated examples will be exhibited in Section 4. REMARKS: ti) The iteration order in the loop (3.3) or (3.6) is still arbitrary. Our transformation is therefore not uniquely defined, since in order to fully, specify it one has also to specify the order of iteration (if more than one is possible). Selection of the "right" order can be crucial to the effectiveness of the transformation, as will be seen in the examples exhibited in Section 4. (2) The intermediate object A' in (3.3) is used there instead of A itself in order to preserve the present value of A, or to use this value to control the iteration [as in (3.6)]. When this is not essential, A itself may be rebuilt incrementally in (3.2), instead of A'. Moreover, it is often the case that A' becomes dead in (3.5) and can therefore be eliminated from the program.
Formal integration: a program transformation technique
39
(3) In some cases there is no need to create the l o o p (3.3), since there already exists a similar loop which builds-up A incrementally in a similar fashion. In such cases we can achieve the effect of formal integration by transforming the existing loop into one similar to (3.5). This reduces formal integration to the loop fusion transformation discussed in [6] (see also [3]). (4) Formal integration can be easily extended to handle composite expressions, or, equivalently, sequences of expressions each depending on the previous one. Consider, for example, a sequence of the form B := K(A); C := L(B);
(3.8)
This can be transformed into a single loop which builds A and simultaneously updates B and C. If the intermediate structure B is not subsequently used, it may even be eliminable, in which case the transformed code evaluates C as the composite function L- K of A by straight formal integration. (5) It is interesting to note that formal integration can be applied using recursion in place of iteration• For simplicity, assume that A is constructed as in (3.1), but that we implement this construction by calling a (linearly) recursive function/](n), defined so that if i = 0 then Ao else hift(i - 1).
~(i) =
Then, in order to compute K(A) by formal integration, we can extend this function so that it returns a pair of values [A, K(A)]. Denoting this extended function as A~, the recursive analog of (3.5) then becomes the evaluation of "~(n), where
AK(i) =
if i = 0 then [Ao, K(Ao)] else [hi(AK(i - 1)(1)), 9i(AK(i - 1)(2))].
(3.9)
This can be easily generalized to allow us to evaluate expressions that depend on trees or on any other recursively-defined object by formal integration. The remarks (1)-(4) apply, with obvious modifications, in the recursive case as well.
4. EXAMPLES Next we present a series of examples showing the application of formal integration in a variety of situations, proceeding now in a somewhat informal manner, since our major aim is to demonstrate the use of formal integration as a methodological programming tool designe d to be used by human programmers as well as by automatic systems. Note, however, that given a system capable of performing formal differentiation, such as that described in detail in [7], it would be relatively easy to extend it so that it can also perform formal integration. See also a concluding remark in Section 5.
I. Simple iterative expressions We begin with a very simple example, which illustrates the general observation that if the expression K(A) is simple enough to admit an "obvious" method of calculation, formal integration will simply recover this obvious technique. Suppose, to be specific, that A is a set, and let
(*)
K(A) = IX in AIP(X)I
where P(X) is some predicate. To transform this expression into a loop of the form (3.6), note that L(0) = 0, and that
DK(A,,X)={~X},
if P(X), otherwise.
40
MIdh, SHARIR
"
is independent of A', which is therefore eliminable from (3.6). We thus see that K(A) can be computed by T:=0; (for all X in A) T := T + (if P(X) then I X I else 0); end of course, this is the way in which K(A) is likely to be computed in any system recognizing the construct (*). II. A maximum occurrences problem Next let S be a string of characters, and suppose that we wish to find a character C which has a maximal number of occurrences in S. To write C as an expression in S, we need a notation which is lacking also in conventional mathematics, namely an operator which, when applied to some aggregate A and a (say real-valued) functionfover A, yields an element X in A for which f(X) is maximal. (The standard notation max
f (X)
XinA
yields the maximum value f(X), not X itself.) We will denote the required operator as max2, and define it so that when it is applied to a set f of pairs [X, f(X)] :X in A } it yields a pair whose second component is maximal. Having introduced this we can write a = [C,n] = max2[[X, # [iin[1 ... #S]:S(i) = Xl]:XinS}, where # gives the cardinality of an aggregate. We can evaluate a by formal integration with respect to S. S, being a string, can be generated by concatenating all its characters from left to right, so that (3.3) becomes S ' : = nullstring; (for all b in S) S' := S' with b; end
(add b to the right of S')
Let K(S) = max2 L(S) denote the expression defining a. Let us first determine how L(S) changes when b is concatenated to S. Obviously the only pair in L(S) which changes is that which begins with b, and the change consists of adding 1 to its second component. Regarding L(S) as a function, we can express this change as L(S with b ) = L(S)- lib, L(S)(b)]] + [[b.L(S)(b)+ 1]]. This implies that the corresponding change in K(S) is
K(S with b) = max2 {K(S), [b, L (S)(b) + 1]}. (Note that here we integrate a composite expression, as in (3.8).) Here, however, the intermediate data structure L cannot be eliminated, since it is used to update K(S). Thus we end up with the following loop which updates both structures as it rebuilds S (In what follows L' and K' are variables holding the values of L(S') and K(S') respectively; note also that S' has been eliminated from the loop, since it is not needed there; finally
Formal integration: a program transformation technique
41
note that L' is represented as a function, and that the functional assignment in the loop corresponds precisely to the change in L(S) given above): L ' : = 0; K ' : = fl; (denoting an undefined value) (for all b in S) L'(b) : = L'(b) + 1; K' := max2 {K', L'(b)]}; end It is interesting to note that when this problem was given to 100 students in an advanced undergraduate programming course, only a few (10) solved it by a program similar to the one derived above. Most of them computed L(S) first and then searched it to determine the maximum.
III. Longest upsequence This problem, posed by Dijkstra [8, 9] can be stated as follows: Given a sequence A~, A 2 . . . An of integers, find the length of the longest upsequence of A, i.e. the longest subsequence whose elements are in ascending order. More formally, we wish to compute I, where / = max[#Q:Q
c l l . . . n ' , ] f o r a l l i , . j i n Q : i < , j = ~ A i <~ Aj',.
A naive computation of 1 is exponential. To solve this problem in a more efficient way, we can evaluate 1 by formal integration with respect to the set S = [ 1 . . . n } . More specifically, we reconstruct S by iterating over its elements from left to right. (In this case selection of the "right" way of iterating through S has a crucial impact on the success of the transformation; the iteration order that we choose builds S up in a natural and obvious way, and also turns out to be "right".) Here (3.3) has the following form: S' := 0; (for all k in [ 1 . . . n ] ) S' := S' + {k}; end Let us write the expression defining l as
K(S) = max L(S) and observe that L(0) = {0}, and hence K(0) = 0. To proceed, we must differentiate L(S') with respect to the change in S' in the above loop. It is easily seen that
L(S' + {k]) = L(S') + [#Q:Qc S'+ {k}lkinQand for alli, j i n Q : i < j ~ A i < . A j}.) Hence the corresponding change in K is
K(S' + {k}) = m a x {K(S'), H(k)} where
H(k) = max [ # Q : Q ~ {1 ... k} Ik in Q and for all i, j in Q:i < j ~ Ai <~ A i}.
42
MICHA SHARIR
It thus remains to compute H(k) in a more efficient manner. To this end we expand H(k) as follows:
H ( k ) = 1 + m a x { # Q ' : Q ' c { 1 . . . k - 1}1 for all i, j in Q': i < j =:, Ai <<.Aj and for all i in Q':Ai <<.Ak} = 1 +max~#Q':Q'c ll...k1}l for all i, j in Q' :i < j ~ Ai <~ Aj and Am~xQ' ~< Akl = 1 + m a x l H ( j ) : j i n I1 . . . k - 1}fAj ~< Ak}. Having arrived at this equation, we can discard S' and L(S') altogether, and maintain only H, to obtain the following program: /:=0; H:=g; (for all k in [ 1 . . . n ] ) n(k) := 1 + max IH(j):jin I1 . . . k - 1}[Aj ~< Ak}; l := max l l, H(k)} ; end This is a reasonably efficient algorithm for computing the longest upsequence in A [having time complexity O(n2)], although it is not as efficient as Dijkstra's O[n'log(n)] algorithm, which considers an intermediate object different from and more subtle than the function H. IV. A simple recursive application: rightmost appearance in a list Next we shall show how formal integration can be applied in a recursive environment. Given a linear list L and an object A, consider the problem of finding the rightmost appearance of A in L(if one exists). Formally, we can write
R = m a x l j : L j = AI where it is understood that application of "max" to an empty set yields an undefined value (which we denote as f2). We assume that the basic list operations are "'hd L" (which gives the first element of L), "tl L" (which gives the tail of the list) and "conc(B, L)" (which produces a list whose first element is B, and whose tail in L). Then we can regenerate L using the following recursive code:
L(L) =
if L = empt3/then empty else conc(hd L, L(tl L))
where "empty" denotes the empty list. To apply formal integration in this setting, let K(L) denote the expression defining R. Note first that K(empty) = f2. It remains to differentiate K with respect to the change in L appearing in the above recursive equation. To this end, we put L ' : = conc(B, L), where B stands for hd L, and then express K(L') in terms of K(L). This gives K(L') = m a x l j : L j = AI = max((if B = A then I1~ else 0) + [ j : L ( j - 1 ) = AI). This last equation can be simplified by splitting it into two cases" If K(L) is undefined then only the first term in the above equation plays a role. On the other hand, if K(L) is
Formal integration: a program transformation technique
43
defined, then the second term contains numbers > 1 so that the first term can be discarded. All this gives
K(L') = if K(L) = ~ then (if B = A then 1 else ~) else 1 + K(L). Since this equation does not involve L explicitly, we can eliminate L from (3.9) to arrive at the following recursive computation of R: /~(L) = if L = empty then else if R(tl L) = f~ then if h d L = A then 1 else f2 else 1 + R(tl L). R E M A R K : When applied in a recursive environment, formal integration usually turns out to yield the same transformations which can be generated by the more general (but less structured) system of folding and unfolding transformations described by Burstall and Darlington [10]. Loosely put, we may view formal integration as some kind of "macro" which can ~oe defined in terms of the folding/unfolding primitives.
V. Garbage collection This example, which has been considered in Ref. [6], and more extensively in [12], simplifies when reformulated as an application of formal integration. The problem is to give a systematic derivation of the garbage collection and compaction algorithm of Dewar and McCann [11] from its high-level specifications. In this problem we deal with a heap, represented as a map H from addresses of block headers to tuples representing the block contents (which, for simplicity, are assumed to consist solely of pointers). Only blocks which can be reached from the first block, starting at 0, via a chain of pointers are to be regarded as active, and the problem requires us to compact these blocks together towards 0, and to adjust pointer values appropriately. In formal terms, we want to compute the new storage layout map H' defined by the following 3 equations: [U. # U] = min21[A, # A ] : A ~ domain H I0in A and
(for all X in A, Yin H(X): Yin AI. ll) The set U thereby defined is the collection of all active cells. (The operator rain2 is defined in a way similar to the definition of max2 above.) N = [ [ b , Z [ # H ( c ) : c in Ulc < b]]:b in U]
(2)
This defines the map N from active block headers to their new compacted addresses.
H' = l[Nlb), [N(c):c in H(b)]]:b in UI
(3)
Here H' is the new storage layout map, in which active blocks are "moved" to their new locations, and pointers within them are adjusted accordingly. While it is quite tempting to attempt evaluation of H' by direct formal integration with respect to H, this turns out to be infeasible for two reasons: (a) U is defined as the transitive closure of [01 under the map H; computation of U by incremental updating as H is grown incrementally is not efficient, so that it is better to compute U by standard transitive closure methods. (However such algorithms can in fact be obtained by combining formal differentiation with other program transformation techniques, for which see Ref. [6].I (b) The expression (3) which defines H' in terms of N, does not admit of an efficient formal derivative, mainly because of the manner in which N first appears within
44
MICHA SHARIR
that expression. Instead, to obtain an expression with a "good" derivative, we split the third expression into two subexpressions as follows: Q = lib, [N(c):c in H(b)]] :b in U} H' = {IN(b), Q(b)] : b in U } (Heuristically, Q represents the storage layout after pointers have been adjusted, but before blocks have been moved.) Taking the above considerations into account, it is still feasible to attempt evaluation of Q by formal integration with respect to U. We will show now that this can indeed be done quite effectively. Since Q is a composite expression of U, we perform this evaluation by iterating over U, building a copy U' and simultaneously updating both N and Q as U' is grown. Let us agree to iterate over U in increasing order of its elements. (3.3) will then have the following form: U ' : = 0; (for all b in U) U' := U' + [bl; end To apply formal integration, observe first that when U = 0, we have N=0 Q = {[b, [f2:c in H(b)]] :b in U} where, as before, f2 denotes an undefined value. (One should compare this initial value of Q with the similar initial value of H' to convince oneself of the much greater difficulty of differentiating H' than of differentiating Q,) Next we note that since we iterate over U in increasing order we have N(U' + [bl) = N(U') + l[b,Z[ #H(c):cin U']]} so if we define T = T(U') = Z [ # H ( c ) : c i n U'] then T is also an efficiently differentiable expression of U'. Indeed, T(0) = 0, and T(U' + [b]) = T(U') + #H(b). All this yields an efficient evaluation procedure for N by formal integration. Passing to Q, we have to express Q(N + [[b, T]}) in terms of Q(N). It is easily seen that Q changes only at pairs [c, [N(d):d in H(c)]] for which b is in H(c), and for each component in H(c) equal to b, the corresponding component in Q(N)(c) (which is presently undefined) should be set to T. In order to make this process time efficient, we want to use an auxiliary data structure R which maps each b in U to the collection of all pairs [c, i-I for which H(c)(i) = b. Written as a relation R = {[H(c)(i), [c, i ] ] : c i n U , / i n [ 1 . . . #H(c)]}. (The use of auxiliary data structures in formal differentiation is quite common. Paige I-3] calls expressions whose differentiation requires the use of auxiliary structures discontinuous, and describes in detail techniques for the introduction of such structures for differentiation of set theoretic expressions.) To sum up, the transformation produces the follow-
Formal integration: a program transformationtechnique
45
ing program fragment (in which U' and N are not used and hence eliminated): Q : = I[b, [ff2:c in H(b)]] :b in U I ; R := ~[H(b)(i), [b, i]] :b in U, i in [1 ... # H(b)]l ; T:=0; (for all b in U) (for all [c, i] in Rlbl) O(c)(i) := T; end; T := T + #H(b); end It should be noted that the program can be further transformed by evaluating both R and the initial expression for Q by formal integration with respect to U. Since U is built up incrementally in a preceding transitive closure step, there is no need to create a new loop of the form (3.3). Instead, assuming the initial value of U to be 0, we initialize both Q and R to 0. Then, whenever U is augmented by an element a U := U + {a}; we also update Q and R by their respective formal derivatives. After some massaging, the update code will have the form:
z:=[]; (for all i in [ 1 , . . #H(a)]) Z : = Z + [f2]; R := R + {[H(a)(i), [a, i]]}; end; Q(a) := Z; The last two algorithm fragments constitute a significant part of the Dewar-McCann algorithm [11]. It is instructive to note that several highly inventive programming decisions reflected in these fragments are derivable by judicious but systematic applications of formal integration. {As noted earlier, the idea of fusing two initialization loops together using formal differentiation, has initially been proposed by Paige [3].) 5. CONCLUSIONS In this paper we have described a simple but general program transformation called formal integration and demonstrated its applicability to a series of examples. Our main concern is methodological; that is, we have tried to view formal integration as a tool to be used systematically (but manually) in program development. Availability of this tool makes construction of certain algorithms a routine task, thereby relieving the programmer from the need to re-invent this transformation anew for each specific application. Moreover, once we have identified formal integration as a generally applicable transformation, it becomes possible to give concise descriptions of programs and algorithms, such as for instance: "This program computes expression Y by formal integration with respect to its parameter X." Such a statement will often define a program almost uniquely, in a form allowing easy communication and explanation of the program. We have not emphasized the rigorous formalization and mechanization of formal integration, since to do so would make it necessary to face more difficult technical problems. However, we expect that formal integration could already become part of a semi-automatic program development system (applicable to at least restricted classes of expressions; for example a modified version of the system presented in [7] should be capable of applying formal integration to certain set-theoretic expressions.) REFERENCES I. J. Earley, High-level iterators and a method for automatically designing data structure representation, J. Comput. Lang. 1, 321-342 (1976).
46
MICHA SHARIR
2~ R. Paige and J. T. Schwartz, Expression continuity and the Formal differentiation of algorithms, Proc. 4th POPL Co~!f 58-71 (1977). 3. R. Paige, Expression continuity and the Formal differentiation or algorithms, Courant Computer Science Report No. 15, Courant Institute (1979). 4~ A. C. Fong and J. D. Ullman, Inductive variables in very high level languages, Proc. 3rd POPL Conf. 104-112 (1976). 5, A. V. Aho and J. D. Ullman, Principles of Compiler Design. Addison-Wesley, New York (1977). 6. M. Sharir, Some observations concerning Formal differentiation of set theoretic expressions, Technical Report No. 16; Computer Science Department, Courant Institute 11979). To appear in TOPLAS. 7. R. Paige and S. Koenig, Finite differencing of computable Expressions, Technical Report LCSR-TR-8, Laboratory for Computer Science Research, Rutgers University (1980). 8. E. W. Dijkstra, Maximum length of a weakly ascending subsequence of a givefi sequence, International Summer School on Program Construction, Marktoberdorf, Germany (1978). 9. E. W. Dijkstra, Some beautiful arguments using mathematical induction, Acta Informat. 13, 1-8 (1980). 10. R. Burstall and J. Darlington, A transformation system for developing recursive programs, J. Ass. Comput. Mach. 24, 44-67 (1977). 11. R. B. K. Dewar and A. P. McCann, MACRO SPITBOL--a SNOBOL4 Compiler, Software Practice and Experience 7, 95-113 (1977). 12. R. B. K. Dewar, M. Sharir and E. Weixelbaum, Transformational derivation of a garbage collection algorithm, To be published. 13. A. C. Fong, Inductively computable constructs in very high level languages, Proc. 6th POPL Conf. 21-28 (1979). About the AUlbor--MIcHA SHARIR was born in 1950, received his Ph.D. in Mathematics at Tel-Aviv University in 1976. He has been a visiting assistant scientist at Courant Institute, New York University, until 1980, where he was working on the SETL project and he is now at Tel-Aviv University. His main research areas are analysis, optimization and transformation of very high level programs.