- Email: [email protected]
Animating matrix algorithms
Comput. & Graphics Vol. l l, No. 3, pp. 309-311,1987 Printed in Great Britain.
0097-8493/87 $3.00 + .00 O 1987 Pergamon Journals Ltd.
Graphics and Education
ANIMATING MATRIX A L G O ~ T H M S REINHARD FOSSMEIER Technische Univvrsi~t Mtinchen, Institut fllr Informatik, Postfach 202420, 8000 Miinchen, Federal Republic of Germany Abstract--Watching the changes effected by a numerical algorithm on a matrix during the run of this algorithm is helpful for the cognitive apprehension as well as for the intuitive insight into the function of the method in question. A program for educational use is described which offers various ways to reprcseat a matrix on a graphics screen; techniques of monitoring the run of existing programs without any changes in program code are discussed. The importance of a detailed understanding of numerical algorithms with respect to array processors and specially structured matrices is pointed out. WHY DISPLAY MATRICES GRAPHICALLY?
In numerical analysis, many of the basic algorithms deal with linear algebra problems, i.e., with matrices. Well-known examples are systems of linear equations and eigenvalue problems. Even though many German undergraduate students have a good understanding of the theoretical background (such as existence and uniqueness theorems), generally it is not easy to convey a full understanding of the sequences of operations necessary to solve such problems on a computer. Even if the student is able to code the variously nested loops that perform the task and is aware of their overall effect, he may often be wanting the full insight into the sequential process his program describes. Whereas, in general, it may be advantageous to construct an algorithm from small steps in a structured way without knowing in advance every detail of the resulting process, this is not true of many numerical algorithms. There are at least two reasons why it is highly desirable to understand their function as thoroughly as possible. Both reasons are closely related to efficiency, an issue of considerable importance with regard to numerical analysis. Firstly, modern "number crunching" mainframes may have not only one but an array of equal processors able to perform the same operation simultaneously on several sets of operands. Such a set, of course, must be selected so that each step operates only on results of preceding steps. A trivial example is the multiplication of a matrix A to a vector x, normally sequentialized as
f o r / : = 1 t o m do
begin y[i] := 0.0; for j : = 1 to n do
y[i] +:=A[i,j]. x[j] end where nearly every step depends on the result of the immediately preceding one. The following version
for i : = 1 to m do y[i] := 0.0;
I
for j : = 1 to n do
I fori:=ltomdo y[i] +:=A[i,j], x[j]; is less common; yet this form can be parallelized since the operations in each of the inner loops are totally independent of each other. This is an example where the awareness of the dynamic structure of the algorithm may be of crucial importance to efficiency. The second reason why the student/prngrammer should understand not only the effects but also the function of an algorithm is the fact that algorithms often exhibit a particular behavior when applied to particular operands, such as band shape or sparse matrices. For instance, Ganssian elimination of an equation system won't disturb the band shape of a matrix if pivot search is not performed but will double the band width otherwise, and it may totally destroy the sparseness. The latter usually has a disastrous effect on efficiency. At this point, the question arises whether a graphical representation might not only help the student in his comprehension of the dynamic structure of an algorithm but would also be more apt to convey an intuitive insight (which, of course, must be completed by a theoretical understanding) into its function. In general, this question has been answered positively (e.g., [ 1, 2]). A program for studying the possibilities of such an approach was created at the Technische Universit~t Miinchen [3]. In contrast to other existing packages (e.g., [4]), it does not display the manipulated data (often representable as more or less continuous functions in one or several dimensions), but the states of the involved matrices, which as a rule lack a straightforward topological structure. Therefore, the styles of representation which the program uses are derived from graph theory rather than from analysis. A major ob-
310
R. FOSSMEIER DO
0
DDO
O
[]
[] OO
0 []
[]
[] O
[]
[] []
O OOO
O
DO
D O [3
[]
[]
O 0 O []
D
[]
n []
[]
O O
[]
O O
[] []
O
Fig. 1. Block representation of matrices.
jective was to use existing implementations of algorithms with as few changes as possible. At present, the program uses two numerical algorithms and four styles of display, although new algorithms and styles can be added easily. STYLES OF REPRESENTATION The basic idea is that the state of the matrix treated should be continuously (i.e., with an update alter every change) displayed on the screen while the algorithm is running. The standard representation is a two-dimensional array whose elements, however, are not numbers but various shades of grey (colors might also be used, if ordered in an obvious way). This standard representation displays a lot of structural information and allows straightforward addressing of elements, lines, and columns; so it has been chosen for the editing of matrices. For a quicker job, or for use on simple output devices, a mere black-and-white representation is also available (see Fig. 1), which is fully sufficient to show, for example, sparseness or band shape. The two remaining styles of representation are the directed transition graph and the undirected graph (Fig. 2). The latter is intended for symmetric matrices and is especially useful if the underlying ordering of the index set is not linear but cyclic. THE MATRIX EDITOR
The matrix editor displays the matrix in block representation with elements in eight different intensities of grey/white. The mapping of values to these intensities may be chosen to be linear or logarithmic. The editor allows to set the values of elements, lines, col-
I
2
3
4
5
6
7
umns, diagonals and whole matrices to preselected or a random values. A probability for this process may also be set, allowing the creation of random sparse matrices. To generate symmetric matrices, the right upper half can be copied into the left lower one. The numerical value of a single element can be inquired. The results of all editing processes are displayed immediately; a graphic cursor shows the actual element, line, or column. Matrices may be named and stored as files and later be retrieved by their name. A matrix resulting from one algorithm can be used as input for another. ALGORITHMS STUDIED An algorithm must be given as a subroutine that works on a matrix either provided by the calling program (as an argument or global array) or otherwise accessible to it. All instructions that may alter the matrix must be suffixed by a call to a display subroutine which updates the screen. Further instructions may direct the cursor to indicate reading access to matrix elements, or may issue comments about certain phases of the algorithm. At crucial points the program may be stopped to give time to think. The first algorithm implemented was the elimination of a linear equation system by Gauss' method. It is relatively easy to survey, yet the visualization clearly helps to see the effects of permutations caused by pivot search. Moreover, thefill-in phenomenon, i.e., the loss of sparseness during elimination, is made visible, and with it the significance of the matrix profile rather than of its bandwidth. The effects of various orderings of the index set can be studied.
8
1 8
2
./7
\i%
+ 1
2
3
4
5
6
8
4 5
Fig. 2. The matrix of Fig. l(b) represented as transition graph and as undirected graph.
Animating matrix algorithms The graph representation gives a less good idea of fill-in but is very suitable to display line or column permutations due to pivot searching: rubber-band technique (not yet implemented) can present the permutation--really a discrete process--by a continuous movement, thus making things much more palpable. Band shape also comes out very well in this style, especially in the undirected graph where bands form a "ring" of edges. The second algorithm studied was Jacobi's method of solving the eigenvalue problem of a positive definite matrix. The matrix, created, for example, with the symmetry function of the editor, is displayed in a logarithmic scale since the algorithm aims at generating smaller and smaller nondiagonal elements. Here the graph representations are very useful, too, because nondiagonals clearly stand out in the graphs. POSSIBLE EXTENSIONS AS was mentioned above, it is easy to add further algorithms or display styles to the program. For example, color might be used to distinguish a larger scale of different numeric values, or several cursors in different colors might be introduced for elements accessed for writing, reading, and both. The present version of the program requires a call of the display subroutine to be inserted at every place in the algorithm that might effect a change on the matrix. Yet the program could be transformed into a "spy" program, which would monitor matrix changes without any interference with the procedure describing the algorithm. This way a procedure could be examined even if no source code is available, as long as it operates on a matrix provided by the calling program (which is the usual way). An implementation of such a spy program could work via shared memory o r - - o n an appropriate hardware--through interrupts, which could, for example, be generated as memory or page faults due to an invalid matrix address. The INTEL 80x87 numeric processors offer a convenient and adequate way of generating such interrupts for reading access by the use of NANs (invalid numbers, see [5]) as elements of a dummy matrix. This is very handy since this interrupt type normally is not used by the operating system. When activated by an interrupt, the spy program would update the screen, generate the correct matrix
311
address and perform the necessary storage access. Thus, reading as well as writing acc, ss to the matrix could be supervised, whereas the shared memory technique allows only to detect changes in the matrix values. The spy version, however, is not a substitute for the original version, because it necessarily lacks insight into the algorithm; it cannot issue comments or recognize patterns in operations, e.g., a permutation of lines or columns. It can merely give an x-ray picture of a method if the interior of a program is not accessible for lack of source code, or, if for some other reason one does not want to look into the program structure. IMPLEMENTATION The program has been written in the C programming language under the UNIX T M operating system. The hardware used is a microcomputer with INTEL 80286/ 87 processors and a black and white graphic display with 768 × 1008 pixels [6], both developed and constructed at the Insfitut ftir Informatik of the Technische Universi~t Miinchen. The graphic interface for the drawing of lines is the one of the Tektronix 4014 terminal; however, block graphics instructions are passed directly to the graphics processor in its internal syntax. With the exception of the device-dependent output syntax, the program should be quite portable among different UNIX versions; the adaptation from XENIX to UNIX System V was trivial. A portation to a personal computer is intended. REFERENCES 1. M. H. Brown and R. Sedgwick, "A System for Algorithm Animation." Computer Graphics 18(3), 177-186 (July 1984). 2. J. N~meth, "Komputila apogo de matematik-instruado en bazaj lernejoj," pp. 11!-115 in Matematiko, Instruado de Matematiko kaj Komputotekniko; Proceedings of the INTERKOMPUTO 83, ed. I. Koutny, Neumann Jfinos Sz~tmlt6g~ptudomfinyit~rsa~, Budapest (1982). 3. J. Miiller, Grafische Darstellung von Algorithmen der linearen Algebra, TechnischeUniversit~tMiinchen (1986). Diplomarbeit. 4. C. Moler, PC-MATLAB User's Guide Version 1.0, The MathWorks, Inc., Portola Valley (January 14, 1985). 5. J. F. Palmer and S. P. Morse, Die mathematischen Grundlagen der Numerik-Prozessoren 8087/80287, te-wi, Miinchen (1985). 6. A. Raith and Th. Schneider, Die Graphik-Sichtger/ite GSG-12 and GSG-15/GSG-17, Institute ftir Mathematik und Informatik, TechnischeUniversita'tMiinchen TUMI8608 0986).
0097-8493/87 $3.00 + .00 O 1987 Pergamon Journals Ltd.
Graphics and Education
ANIMATING MATRIX A L G O ~ T H M S REINHARD FOSSMEIER Technische Univvrsi~t Mtinchen, Institut fllr Informatik, Postfach 202420, 8000 Miinchen, Federal Republic of Germany Abstract--Watching the changes effected by a numerical algorithm on a matrix during the run of this algorithm is helpful for the cognitive apprehension as well as for the intuitive insight into the function of the method in question. A program for educational use is described which offers various ways to reprcseat a matrix on a graphics screen; techniques of monitoring the run of existing programs without any changes in program code are discussed. The importance of a detailed understanding of numerical algorithms with respect to array processors and specially structured matrices is pointed out. WHY DISPLAY MATRICES GRAPHICALLY?
In numerical analysis, many of the basic algorithms deal with linear algebra problems, i.e., with matrices. Well-known examples are systems of linear equations and eigenvalue problems. Even though many German undergraduate students have a good understanding of the theoretical background (such as existence and uniqueness theorems), generally it is not easy to convey a full understanding of the sequences of operations necessary to solve such problems on a computer. Even if the student is able to code the variously nested loops that perform the task and is aware of their overall effect, he may often be wanting the full insight into the sequential process his program describes. Whereas, in general, it may be advantageous to construct an algorithm from small steps in a structured way without knowing in advance every detail of the resulting process, this is not true of many numerical algorithms. There are at least two reasons why it is highly desirable to understand their function as thoroughly as possible. Both reasons are closely related to efficiency, an issue of considerable importance with regard to numerical analysis. Firstly, modern "number crunching" mainframes may have not only one but an array of equal processors able to perform the same operation simultaneously on several sets of operands. Such a set, of course, must be selected so that each step operates only on results of preceding steps. A trivial example is the multiplication of a matrix A to a vector x, normally sequentialized as
f o r / : = 1 t o m do
begin y[i] := 0.0; for j : = 1 to n do
y[i] +:=A[i,j]. x[j] end where nearly every step depends on the result of the immediately preceding one. The following version
for i : = 1 to m do y[i] := 0.0;
I
for j : = 1 to n do
I fori:=ltomdo y[i] +:=A[i,j], x[j]; is less common; yet this form can be parallelized since the operations in each of the inner loops are totally independent of each other. This is an example where the awareness of the dynamic structure of the algorithm may be of crucial importance to efficiency. The second reason why the student/prngrammer should understand not only the effects but also the function of an algorithm is the fact that algorithms often exhibit a particular behavior when applied to particular operands, such as band shape or sparse matrices. For instance, Ganssian elimination of an equation system won't disturb the band shape of a matrix if pivot search is not performed but will double the band width otherwise, and it may totally destroy the sparseness. The latter usually has a disastrous effect on efficiency. At this point, the question arises whether a graphical representation might not only help the student in his comprehension of the dynamic structure of an algorithm but would also be more apt to convey an intuitive insight (which, of course, must be completed by a theoretical understanding) into its function. In general, this question has been answered positively (e.g., [ 1, 2]). A program for studying the possibilities of such an approach was created at the Technische Universit~t Miinchen [3]. In contrast to other existing packages (e.g., [4]), it does not display the manipulated data (often representable as more or less continuous functions in one or several dimensions), but the states of the involved matrices, which as a rule lack a straightforward topological structure. Therefore, the styles of representation which the program uses are derived from graph theory rather than from analysis. A major ob-
310
R. FOSSMEIER DO
0
DDO
O
[]
[] OO
0 []
[]
[] O
[]
[] []
O OOO
O
DO
D O [3
[]
[]
O 0 O []
D
[]
n []
[]
O O
[]
O O
[] []
O
Fig. 1. Block representation of matrices.
jective was to use existing implementations of algorithms with as few changes as possible. At present, the program uses two numerical algorithms and four styles of display, although new algorithms and styles can be added easily. STYLES OF REPRESENTATION The basic idea is that the state of the matrix treated should be continuously (i.e., with an update alter every change) displayed on the screen while the algorithm is running. The standard representation is a two-dimensional array whose elements, however, are not numbers but various shades of grey (colors might also be used, if ordered in an obvious way). This standard representation displays a lot of structural information and allows straightforward addressing of elements, lines, and columns; so it has been chosen for the editing of matrices. For a quicker job, or for use on simple output devices, a mere black-and-white representation is also available (see Fig. 1), which is fully sufficient to show, for example, sparseness or band shape. The two remaining styles of representation are the directed transition graph and the undirected graph (Fig. 2). The latter is intended for symmetric matrices and is especially useful if the underlying ordering of the index set is not linear but cyclic. THE MATRIX EDITOR
The matrix editor displays the matrix in block representation with elements in eight different intensities of grey/white. The mapping of values to these intensities may be chosen to be linear or logarithmic. The editor allows to set the values of elements, lines, col-
I
2
3
4
5
6
7
umns, diagonals and whole matrices to preselected or a random values. A probability for this process may also be set, allowing the creation of random sparse matrices. To generate symmetric matrices, the right upper half can be copied into the left lower one. The numerical value of a single element can be inquired. The results of all editing processes are displayed immediately; a graphic cursor shows the actual element, line, or column. Matrices may be named and stored as files and later be retrieved by their name. A matrix resulting from one algorithm can be used as input for another. ALGORITHMS STUDIED An algorithm must be given as a subroutine that works on a matrix either provided by the calling program (as an argument or global array) or otherwise accessible to it. All instructions that may alter the matrix must be suffixed by a call to a display subroutine which updates the screen. Further instructions may direct the cursor to indicate reading access to matrix elements, or may issue comments about certain phases of the algorithm. At crucial points the program may be stopped to give time to think. The first algorithm implemented was the elimination of a linear equation system by Gauss' method. It is relatively easy to survey, yet the visualization clearly helps to see the effects of permutations caused by pivot search. Moreover, thefill-in phenomenon, i.e., the loss of sparseness during elimination, is made visible, and with it the significance of the matrix profile rather than of its bandwidth. The effects of various orderings of the index set can be studied.
8
1 8
2
./7
\i%
+ 1
2
3
4
5
6
8
4 5
Fig. 2. The matrix of Fig. l(b) represented as transition graph and as undirected graph.
Animating matrix algorithms The graph representation gives a less good idea of fill-in but is very suitable to display line or column permutations due to pivot searching: rubber-band technique (not yet implemented) can present the permutation--really a discrete process--by a continuous movement, thus making things much more palpable. Band shape also comes out very well in this style, especially in the undirected graph where bands form a "ring" of edges. The second algorithm studied was Jacobi's method of solving the eigenvalue problem of a positive definite matrix. The matrix, created, for example, with the symmetry function of the editor, is displayed in a logarithmic scale since the algorithm aims at generating smaller and smaller nondiagonal elements. Here the graph representations are very useful, too, because nondiagonals clearly stand out in the graphs. POSSIBLE EXTENSIONS AS was mentioned above, it is easy to add further algorithms or display styles to the program. For example, color might be used to distinguish a larger scale of different numeric values, or several cursors in different colors might be introduced for elements accessed for writing, reading, and both. The present version of the program requires a call of the display subroutine to be inserted at every place in the algorithm that might effect a change on the matrix. Yet the program could be transformed into a "spy" program, which would monitor matrix changes without any interference with the procedure describing the algorithm. This way a procedure could be examined even if no source code is available, as long as it operates on a matrix provided by the calling program (which is the usual way). An implementation of such a spy program could work via shared memory o r - - o n an appropriate hardware--through interrupts, which could, for example, be generated as memory or page faults due to an invalid matrix address. The INTEL 80x87 numeric processors offer a convenient and adequate way of generating such interrupts for reading access by the use of NANs (invalid numbers, see [5]) as elements of a dummy matrix. This is very handy since this interrupt type normally is not used by the operating system. When activated by an interrupt, the spy program would update the screen, generate the correct matrix
311
address and perform the necessary storage access. Thus, reading as well as writing acc, ss to the matrix could be supervised, whereas the shared memory technique allows only to detect changes in the matrix values. The spy version, however, is not a substitute for the original version, because it necessarily lacks insight into the algorithm; it cannot issue comments or recognize patterns in operations, e.g., a permutation of lines or columns. It can merely give an x-ray picture of a method if the interior of a program is not accessible for lack of source code, or, if for some other reason one does not want to look into the program structure. IMPLEMENTATION The program has been written in the C programming language under the UNIX T M operating system. The hardware used is a microcomputer with INTEL 80286/ 87 processors and a black and white graphic display with 768 × 1008 pixels [6], both developed and constructed at the Insfitut ftir Informatik of the Technische Universi~t Miinchen. The graphic interface for the drawing of lines is the one of the Tektronix 4014 terminal; however, block graphics instructions are passed directly to the graphics processor in its internal syntax. With the exception of the device-dependent output syntax, the program should be quite portable among different UNIX versions; the adaptation from XENIX to UNIX System V was trivial. A portation to a personal computer is intended. REFERENCES 1. M. H. Brown and R. Sedgwick, "A System for Algorithm Animation." Computer Graphics 18(3), 177-186 (July 1984). 2. J. N~meth, "Komputila apogo de matematik-instruado en bazaj lernejoj," pp. 11!-115 in Matematiko, Instruado de Matematiko kaj Komputotekniko; Proceedings of the INTERKOMPUTO 83, ed. I. Koutny, Neumann Jfinos Sz~tmlt6g~ptudomfinyit~rsa~, Budapest (1982). 3. J. Miiller, Grafische Darstellung von Algorithmen der linearen Algebra, TechnischeUniversit~tMiinchen (1986). Diplomarbeit. 4. C. Moler, PC-MATLAB User's Guide Version 1.0, The MathWorks, Inc., Portola Valley (January 14, 1985). 5. J. F. Palmer and S. P. Morse, Die mathematischen Grundlagen der Numerik-Prozessoren 8087/80287, te-wi, Miinchen (1985). 6. A. Raith and Th. Schneider, Die Graphik-Sichtger/ite GSG-12 and GSG-15/GSG-17, Institute ftir Mathematik und Informatik, TechnischeUniversita'tMiinchen TUMI8608 0986).