- Email: [email protected]
Iconic representation of visual data and models
Pattern Recognition Letters 12 (1991) 781-792 North-Holland
December 1991
Iconic representation of visual data and models F . F i e r e n s , J. V a n C l e y n e n b r e u g e l ,
P. Suetens* and A. Oosterlinck
ESAT-MI2, Department of Electrical Engineering, Katholieke Universiteit Leuven, Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium
Received 28 January 1991 Revised 19 July 1991
Abstract Fierens, F., J. Van Cleynenbreugel, P. Suetens and A. Oosterlinck, Iconic representation of visual data and models, Pattern Recognition Letters 12 (1991) 781-792. Knowledge based image analysis is a combination of digital signal processing and symbolic reasoning. In this paper, we will look at some problems connected to the symbolic reasoning approach to image interpretation and see how an iconic representation can help to solve some of them. We will show that many of the features and problems connected with both symbolic and iconic representation are complementary. J
1. Introduction Image analysis is the automatic interpretation of what is represented in one or more images in terms that are relevant to the application domain. The most natural terminology for the interpretation is not necessarily a verbal description. For example, the extraction of lines from satellite images and their interpretation as a road network is better described by the generation of a map than by a verbal description of the shape and spatial relations between the individual roads. The different levels of detail or abstraction encountered during an image analysis process pose different requirements to the data representation mechanisms. Processing of the data is very dependent on the representation scheme that is used. In
* P. Suetens is also a senior research associate of the National Fund for Scientific Research, Belgium . 016%8655/91/$03.50
this paper we will first take a closer look at the properties of the visual data that has to be processed. We will then examine the problems related with the classical expert system approach and discuss how the use of an iconic representation can help to circumvent some of these problems. Although Sloman (1975) in his articles objects to the term 'symbolic' when referring to verbal descriptions (which he calls 'propositional'), the term has become so familiar in the AI community that we will make use of it throughout the paper.
2. Properties of visual data During the analysis of an image, we can distinguish different interpretation levels. The classification given below is more or less subjective but serves to summarize the change of data properties and processing requirements during an interpretation process.
© 1991 - - Elsevier Science Publishers B.V. All rights reserved
781
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
2.1. Low level At the lowest level we have the raw image data consisting of pixels whose value represent intensity and sometimes color information. If we take as a typical example an image of 512 by 512 pixels, each pixel represented by 8 bits, then a quarter of a megabyte storage space is needed. Usually, images are preprocessed before segmentation algorithms are applied. This means that the lowest level in image analysis requires large amounts of memory to store the data. The processing algorithms at this level are usually very simple and require no special control strategies. The algorithms are often suited for parallel implementation. 2.2. Mid level We can consider the mid level to be the level that contains the results of a segmentation phase. The data at this level are classically regions or line segments obtained after the application of segmentation algorithms followed by either split-andmerge or erosion-dilation-thinning steps. Typical properties of the data at this level are: • There is still a large amount of data, typically a few hundred to a few thousand segments. • The number of different segment types that can be distinguished is very small. For line or edge segmentation we can distinguish for instance lines, junctions and unconnected points. Further distinctions (e.g., long lines, curved lines) already require some form of analysis. • The fact of having many data elements that can only be categorized according to a small number of types means that data structuring according to the type of data will not be very helpful for further processing. More important are the spatial structure and relationships between the segments. These offer an alternative way of structuring according to spatial proximity and topology. 2.3. High level The highest level contains the most domain or application dependent knowledge. This level contains the domain dependent description of the model or models to be instantiated. It is also the 782
December 1991
level at which the interpretation of the image contents becomes intelligible and interesting to the user because it is formulated in application dependent terms instead of in terms of spatial relations between rather abstract segments. Model description in vision is most natural if a hierarchical or multilevel representation can be used. Part-of relations as well as spatial relations play an important role. The different levels encountered during an image analysis process lead to a hierarchical representation pyramid as shown in Figure I. The lowest levels of this pyramid correspond to Mart's raw primal sketch (Marr (1982)). Each level in the pyramid is a conceptual entity but can itself form a hierarchical structure. At the lowest level images can be represented in multi resolution. The spatial network level can contain many layers, ranging from a network of the original segmentation to networks where conceptual grouping has only retained the most important perceptual entities. The higher the level in the pyramid, the greater the data reduction, but at the same time, the complexity of the data structuring and of the control strategies to process the data increases, as well as the application dependency. Very important to notice in this multi level representation scheme is the lack of continuity in the data representation. At the lower levels of the pyramid the data has a signal character whereas the higher level data has a symbolic nature (see Figure 1).
3. The use of symbolic representation and reasoning in image analysis As an example of symbolic representation and reasoning, we will consider a rule based expert system approach for the early interpretation of image data. The use of expert system techniques and symbolic reasoning to interpret an image is situated at the symbolic level of the representation pyramid. The input data of such systems consists of a symbolic description of the primitives extracted by a so-called segmentation phase. Because of the large amount of data and the lack of initial context many vision systems start with a data
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
driven or bottom up phase in order to combine the primitives into more meaningful entities.
3.1. A typical example We will take the segmentation and interpretation of blood vessels on a radiological image as a typical example of this kind of processing (Van Cleynenbreugel et al. (1988)). The segments extracted from the original image consist of rectangular structures representing parts of the blood vessels. These were obtained by combining the results of an edge detector and a ridge detector algorithm (Figure 2a) (Smets (1990)). • The model knowledge used at this level in a data driven system consists of more or less domain independent perceptual grouping heuristics. In the expert system approach that we consider, these heuristics are expressed in the form of rules. An example of such a perceptual grouping rule is shown in Figure 2b. The conditions in the if part of the rule are
December 1991
checked against the data present in the system by pattern matching. When data is found that satisfies all the conditions, the rule is marked for execution and possibly fired. The result of applying the example rule to the original segmentation data is shown in Figure 2c.
3.2. Problems approach
related
to
the
expert system
3.2.1. Data description In order to be able to reason with the segmentation results, the segmentation data has to be represented symbolically. This symbolic description entails a simplification of the data. Line segments are broken into linearized pieces so that they can be represented by their endpoints. Reasoning on these segments is based on comparing or matching the coordinates of their endpoints. As an example, suppose we want to establish proximity between segments. This is usually very important information for perceptual
A globalproperties (db records)
symbolic levels
t situationsN h
Sa
/
t
components+ X rt-ofand spatialrelations "~
t
lines+ edges + regions + spatialrelations
\
t iconic l e v e ~ / /
t input images
\
Figure 1. Representation pyramid.
783
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
grouping. Reasoning with the symbolic description of the segments allows to derive this information for the case represented in Figure 3a. The situations in Figure 3b will be overlooked. Visual data is most naturally represented at different levels of detail. The abstraction level of the
and and and and then
there is an edge <01> with orientation there is an edge (<02> <> ) with orientation <02> the distance between and <02> is small is almost equal to <02> is almost collinear to connect and <02>
(b)
December 1991
description however, must correspond to the abstraction level of the reasoning process. As an extreme example, the position and the intensity of every pixel in an image can be asserted as a symbolic fact (actually, the internal computer representation of an image can be considered as such). Although such a symbolic representation is complete and contains the same information as the original image, it is useless if the knowledge used for reasoning is not expressed in terms of pixel position and intensity. As a more realistic example, we can consider how to describe the results of the perceptual rule of Figure 2. The data to be described consists of long, curved blood vessel segments. On the one hand, we can easily describe general properties of each segment such as length, average diameter, average curvature and position of endpoints. This description however, is useless if we want to reason about proximity o f other blood vessels or about particular shape properties (see also the configurations of Figure 3b). On the other hand, we can represent a blood vessel segment by the chain of smaller rectangular segments out of which it consists. In this case, the reasoning knowledge (the rules) would have to be stated in terms of these chains of smaller segments and would lead to an extremely complicated and cumbersome knowledge description. Especially in the second case, there is the additional problem that the reasoning level needed to process the data descriptions is on the level o f internal representation and implementation and has little or no relation to the
¢,--
J (a) Figure 2. A rule based approach. 784
(b) Figure 3. Proximity between segments.
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
knowledge level of the human expert. Although the two data descriptions proposed in this example seem like two extreme cases, it is very difficult to find an intermediate symbolic description that does not pose the same problems.
3.2.2. Model description The automatic analysis of the image requires a model description. This description tells the analysis system what to expect in the image and how to interpret the image data. Symbolic description of these models poses approximately the same problems as the description of the image data. Again, especially the visual aspects of this description, like shape and spatial relations between object parts, pose the greatest problems. 3.2.3. Combinatorial behavior In the rule based approach as presented in the example, no structure is imposed upon the data. This data consists of object-attribute-value triplets that reside in an unstructured data pool, called the working memory. Some systems combine the rule based approach with object oriented structuring (ART, KEE, Nexpert Object), thus allowing some hierarchical structure to be imposed on the data. In both cases however, the spatial structure of the data, which is very important for vision, is lost. This means that data items cannot be reached by spatial access paths but only through pattern matching. This has serious effects on the efficiency of such a system. As an example, let us consider a simplified version of the perceptual grouping rule of Figure 2. if (the position of object.1 (the position of object.2 (close-to ?x ?y) then,.,
December 1991
This test however, will be applied to all possible combinations of objects, leading to a system whose run time increases as a combinatorial function of the number of objects present (Figure 4). In rules where more than two objects must be compared to each other, the situation is even worse. Although sophisticated schemes such as the RETE algorithm (Forgy (1982)) allow to transfer some of the work to a precompilation phase, the problem of combinatorial matching is only shifted, not resolved. The use of windows in the image to restrict the matching is another way to circumvent the problem. The S C H E M A system reported by Draper et al. (1989) uses such a windowing strategy. This kind of solution however can give rise to complicated knowledge description when results from neighboring windows have to be combined.
3.2.4. Discontinuity in the data representation We can consider the interpretation of an image as the construction of the representation pyramid of Figure 1. In systems that are to solve real world problems, this interpretation typically is not a straightforward bottom up or top down process, mainly because of errors in the low level segmentation phase. The initial segmentation however, can be used to establish an initial reasoning context. Within this context, which forms a partial inter-
run time
panern matching
exploration
is ?x) and is ?y) and
The le•hand side of this rule describes the conditions for proximity between two objects. The first condition contains no restrictions and will match with every object containing a position attribute. The same situation applies to the second condition. It is only in the third condition where the proximity check is made, that object pairs that do not passs the close-to test will be filtered out.
breakeven point
number of objects
Figure 4. Pattern matching versus exploration. 785
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
pretation, segmentation errors can be hypothesized. These hypotheses can then be verified by checking them with the image data. This leads to more reliable context information that allows a more complete and correct interpretation which in turn leads to more correct hypotheses. The higher levels of the interpretation pyramid interact more or less in the same way. The whole hypothesis verification process iterates through the different levels of the interpretation pyramid. This process, which requires complex control strategies, is further complicated by a discontinuity in the data representation. At the lowest levels o f the pyramid, the data has a signal-like nature whereas segmentation results and model specification at the higher levels are described symbolically. In order to communicate information between the image-like and the symbolic levels of the pyramid in a flexible way, special software interfaces must be provided. Most systems only provide a loose coupling between iconic and symbolic levels in the form of a call out facility to Fortran, Pascal or C routines from within the knowledge based tool. Such a loose coupling gives insufficient support for a flexible interaction between the different representation levels.
4. Iconic representation When we refer to the term iconic representation in an image analysis context, we mean that the information is represented by pixels in a digital image. Models of objects can be represented iconically by drawing them or by processing existing graphical representations. Original image data as well as segmentation results are examples of iconically represented data. 4.1. Established research The accurate description of the information present in a road-map shows a typical example of the need for iconic representations. It is fairly straightforward to represent the topology of the road network in a symbolic way by describing roads as connections between the different crossings. Global properties of individual roads such as length, position of the endpoints and average cur786
December 1991
vature can also be represented symbolically. Representing the exact shape of a road symbolically is less straightforward. Iconic description of such a road (i.e., making a drawing of it) offers an alternative. Iconic representations are a subset of the more general analog representation schemes that were studied by Sloman (1975). In his papers he made a clear distinction between analogical and propositional representations. Marr (1982) was one of the first vision researchers to actually use iconic representation. His idea of using a bitmap to limit the search in what he called a raw primal sketch, is an example of exploration versus purely symbolic reasoning (see also Section 4.2.3). Few vision systems actually make use of iconic representations. A system in which iconic representation does play an important role is W H I S P E R (Funt (1980)). This system simulates a blocks world where unstable configurations o f blocks are considered. Instead of reasoning symbolically on the situations presented to it, W H I S P E R uses diagrams that represent the blocks. In these diagrams, the subsequent situations of the block configurations are depicted. The effects of tumbling blocks are observed by means of a 'retina', i.e., a simulation of the human retina that has a number of interesting properties (focussing, similarity testing, etc.). W H I S P E R is interesting because it uses iconic representation and observes spatial configurations. The system however, only simulates a simplified physical situation. Image analysis is restricted to observation of the diagrams. No interpretation of real world image data takes place. A real image analysis system in which iconic and propositional representations are combined is NEUROLOGIST-1 (Shrihari and Xiang (1989)). The task o f NEUROLOGIST-I is to recognize physical entities and lesions in cross-section images of the human brain. The system however, does not reason directly on iconic representations of these cross-sections. Instead it reasons upon sets of vertices representing the convex hull of the physical entities. In this sense, the system reasons on a symbolic description rather than observing an iconic representation.
Volume 12, Number 12
""
PATTERN RECOGNITION LETTERS
L
December 1991
d
(a} Figure 5. Different environment shapes.
I-SEE, described in Fierens et al. (1990 and 1991) is a knowledge based development tool for image analysis that was used to implement the examples described below. Iconic representation and exploration are key concepts in the tool. Exploration happens by means of environment objects that have user defined or object dependent shapes (Figure 5). These environments are used in combination with an integrated image processing library. Environments can be overlayed on top of the original image and used for resegmentation purposes. The I-SEE tool supports the notion of label images in which the pixels are labels or pointers referring to symbolically described objects. This notion is similar to Marr's bitmaps or to occupancy arrays. Label images can be explored by means of environments. A label image, together with its symbolic description in the form of a spatial network can be extracted automatically from a binary image (e.g., the result of a segmentation phase). It must also be clear that an iconic representation cannot express all the knowledge about a model. Iconically represented information is usually restricted to shape, color or spatial organisation. Therefore, iconic representation is not an alternative to symbolic representation but must be seen as an additional representation paradigm. In the next section we will take a closer look at some properties of this iconic representation.
4.2. Properties 4.2.1. Description of objects and relations As already mentioned, the symbolic description
of visual information such as shape or spatial relations is very difficult. Attempts to describe this information verbally easily lead to descriptions that are either too general or too complex for spatial reasoning. If the system under development is based on symbolic reasoning, then the knowledge acquisition of the visual information is a tedious process. The iconic representation is in comparison a very natural one for representing shape or spatial properties. The knowledge acquisition consists of making a drawing of the model to be instantiated or to process a pictorial representation of a model into a line drawing. Both ways of obtaining an iconic model are relatively fast. Non-visual information can still be acquired and represented symbolically. More than one model can be needed to represent the relevant information. Examples are a set of two-dimensional projections of a three-dimensional object, seen from different angles or a set of alternative models if the exact appearance cannot be predicted. An example of the last case is a model of the cardiac blood vessels where the appearance can differ widely from one patient to another due to biological variability. Except for digitizing errors, the representation of the object shape is accurate and there is no need for simplification as was the case in the symbolic representation o f Section 3.2.1. Representation o f 3-D shape can be done by representing a number of distinguishing projections but if memory space and performance are not critical, 3-D arrays can be used to represent 3-D models. Again, this representation is accurate with respect to shape and spatial properties. 787
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
~(b) Figure 6. Establishing relations by exploration: (a) visibility, (b) inclusion.
Spatial relations between objects in an iconic representation are implicit in the image. Exploration in the iconic representation allows to establish relations that would otherwise have been difficult to extract from a symbolic database o f objects. We already gave an example o f the kind o f problems that can be encountered when trying to establish the proximity relation (Figure 3b). Proximity can be established directly from an iconic representation by means o f the environments o f Figure 5. An
environment can be put on top of an original or a labeled image and scanned. The shape of the environment implicitly defines the proximity relation. Circular environments (Figure 5a) define proximity as a uniform maximum distance from a specific point, environments with a triangular form (Figure 5b) define proximity with a directional aspect. Environments can also follow the shape of an image object (Figure 5c), an example o f proximity definition that would be very difficult in a purely symbolic way. Another example is the establishment o f the visibility relation between two objects. This relation can be important for hypothesizing segmentation errors. The situation where two collinear lines lie in close proximity but are not connected can indicate a segmentation error if no other segments lie in between them. The relation visibility between
ft------'7 Figure 7. Perceptual grouping in I-SEE. 788
December 1991
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
the endpoints of the two segments can check this last condition. Establishing the visibility relation in an iconic representation requires nothing more than checking the pixels in the image that lie on a straight line between the endpoints (Figure 6a). A last example of a relation which is complex to establish by symbolic reasoning is the relation inclusion. In the iconic representation this relation can be checked by drawing scanlines and checking the pixels until either the enclosing object or the image border is reached (Figure 6b).
4.2.2. Linear behavior Establishing the relation proximity in a symbolic manner gave rise to a combinatorial explosion. Using the iconic representations o f the I-SEE tool, this same relation can be established by creating environments at the endpoints or around the object under consideration, thus limiting the region of the image to be scanned. One or two region scans are needed for every object considered. The run time for establishing the proximity relation for an entire image will thus be a linear function o f the number o f objects present. The same reasoning can be applied to a number o f other spatial relations (visibility, inclusion . . . . ). For a small number of objects, combinatorial comparison o f endpoint coordinates can be more efficient than an environment scan. The position o f the break even point (Figure 4) where both processing schemes are equally efficient depends on the dimensions o f the environment and on the functionality that is required (do we need to detect the cases o f Figure 3b or not?). Any realistic application however, will be situated to the right o f this break even point. Therefore, linear behavior is an important feature that allows the scaling up from small test examples to real world applications.
4.2.3. Cottsistency The iconic representation, e.g. when used for representing segmentation results, can be considered as an iconic database. Image analysis systems that make use o f resegmentation can add or remove segments in this iconic database x~dthout consistency problems. Information in the database is present in an implicit form and can be extracted
December 1991
by exploration. If a visibility relation exists between two segments and resegmentation adds a new segment in between, then a new exploration will detect this change. Consistency problems only occur if relations between objects are extracted from the iconic database and represented symbolically. Figure 7 shows the intermediary steps o f the perceptual reasoning module o f I-SEE applied to the segmentation o f an X-ray image of a blood vessel tree. Instead o f continuously checking and updating information extracted from the segmentation image and represented in a symbolic form, the system keeps adding and removing segments in the iconic database. Processing relies mainly on exploration (observation) as an alternative to symbolic reasoning. At specific points in the processing, the information drawn in the iconic database is scanned and extracted into a symbolic network on which additional reasoning can take place.
4.2.4. Iconic inferencing In the previous example, iconic representation and exploration were used in combination with symbolic reasoning on the information present. It is also possible to use iconic representations to obtain end results without first extracting a symbolic description. The methodology o f "snakes' reported by Kass et al. (1988) is a form o f what we could call "iconic inference'. Snakes are digital curves or surfaces xvith a simulated physical behavior. Internal forces in the snake give it elastic and plastic properties, external forces try to deform the snake. These external forces can be applied by the user or by a knowledge based module, but can also be extracted f r o m the images themselves. The snake will be attracted or repulsed by these external forces and will deform in a (simulated) viscose medium until an equilibrium between internal and external forces has been reached. The deformation energy o f the snake at equilibrium gives information on how well the data fits the model. As an example o f the use o f this technique, consider Figure 8. The snake was tracked as a contour line on an iconically represented terrain elevation model. We have used it to detect roads on a S P O T satellite image o f a mountainous area. The (weak) heuristic that was implicitly used assumes that 789
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
roads in a mountainous area try to minimize altitude changes rather than distance. Therefore, they are likely to follow contour lines (lines of equal altitude). In order to use such a contour line snake for the detection of a road, a local environment was first created around it. Within this environment, the output of a line detector algorithm by Fishier and Wolf (1986) and a Sobel operator
were combined to yield a forcefield in which the snake was then embedded. The simulation module of the I-SEE tool, which iteratively solves the dynamic system of differential equations that represents the snake and its environment, deforms the snake until the equilibrium state is reached. The line detection filters used to generate the forcefield are such that the snake is attracted to
Figure 8. 'Iconic inferencing'. 790
December 1991
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
pixels with road-like properties. At the same time, the internal forces of the snake resist deformation, thus enforcing the original shape of the model. The end result, shown in Figure 8 is a compromise between the influence of the model and the influence of the data.
5. Discussion Finally, let us summarize the properties of both iconic and symbolic representation schemes. Iconic representations are easy to acquire and are a complete representation for shape, intensity and color information and for the representation of spatial relations. This completeness is usually with respect to a two-dimensional projection of the scene although in principle 3-D iconic representations are also possible. Iconically represented information is implicit and can be accessed through exploration. Together with the access through exploration, this implicitness ensures consistency when objects are added to or deleted from the representation. Exploration in an iconic representation is a process that has a linear run time behavior with respect to the number of objects in the system. This is an important feature for scaling up from a small testcase to a real world application. Finally, the representation is a natural one for visual information because it is close to the real world data that has to be represented. It is also a representation that is largely application independent. The processing of an image or a line drawing or the comparison between a segmentation and a model line drawing can to a large extent be performed without domain knowledge. Domain knowledge however, will enhance the reliability of the interpretation. Symbolic representations on the other hand allow data or knowledge to be represented at different levels of abstraction and generalization. Shape information and spatial relations between objects are difficult to represent and usually lead to oversimplification that is necessary for subsequent reasoning processes. Symbolic representations are well suited to represent other than visual information, which is not possible in an iconic
December 1991
representation. Some visual aspects o f the knowledge, like rotation, translation and scale are difficult to handle in a purely iconic manner. The flexibility of a symbolic reasoning system can be needed to handle these. Systems that use symbolic representation very often access their information through pattern matching. This gives rise to a combinatorial behavior which poses serious problems to any application. It is important to note that the properties of the iconic and the symbolic representation are more or less complementary. The iconic representation scheme is an ideal tool when combined with symbolic descriptions.
Acknowledgements This text presents research results of the Belgian National incentive-program for fundamental research in Artificial Intelligence, initiated by the Belgian State - Prime Minister's Office - Science Policy Programming. The scientific responsibility is assumed by its authors. The examples using satellite images are based on SPOT data under a SPOT-IMAGE(R) license acquired in collaboration with EUROSENSE T E C H N O L O G I E S N.V., Belgium.
References Draper, B.A., R.T. Collins, J. Brolio, A.R. Hanson and E.M. Riseman (1989). The Schema system. Int. J. Computer Vision 2, 209-250. Fierens, F., J. Van Cleynenbreugel, P. Suetens and A. Oosterlinck (1990). l-see: An AI-tool for image understanding. Engineering Applications of AI 3, 62-70. Fierens, F., J. Van Cleynenbreugel, P. Suetens and A. Oosterlinck (1991). A software environment for image database research. J. Visual Languages and Computing, to appear. Fishier, M.A. and H.C. Wolf (1986). A general approach to machine perception of linear structure in imaged data. Technical Note 276, SRI International. Forgy, C.L. (1982). Rete: A fast algorithm for the many pattern/many object pattern match problem. Artificial Intelligence 19, 17-37. Funt, B.V. (1980). Problem-solving with diagrammatic representations. Artificial Intelligence 13(3), 201-230. 791
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
Kass, M., A. Witkin and D. Terzopoulos (1988). Snakes: active contour models. Int. J. Computer Vision l, 321-331. M a r l D. 0982) Vision. Freeman, San Francisco, CA, 41-89. SIoman, A. (1975). Afterthoughts on analogical representations. Proc. Theoretical Issues in Natural Language Processing. Cambridge, MA, 164-168. Smets, C. 0990). A knowledge-based system for the automatic interpretation of blood vessels on angiograms. PhD thesis, Acta Biomedica Lovaniensia 20.
792
December 1991
Srihari, S.N. and Z. Xiang (1989). Spatial knowledge representation. Int. J. Pattern Recognition and Artificial Intelligence 3(1), 67-84. Van Cleynenbreugel, J., F. Fierens, P. Saetens and A. Oosterlinck (1988). Knowledge-based improvement of automatic image interpretation for restricted scenes: two case studies, hnage and Vision Computing 6(4), 238-246.
December 1991
Iconic representation of visual data and models F . F i e r e n s , J. V a n C l e y n e n b r e u g e l ,
P. Suetens* and A. Oosterlinck
ESAT-MI2, Department of Electrical Engineering, Katholieke Universiteit Leuven, Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium
Received 28 January 1991 Revised 19 July 1991
Abstract Fierens, F., J. Van Cleynenbreugel, P. Suetens and A. Oosterlinck, Iconic representation of visual data and models, Pattern Recognition Letters 12 (1991) 781-792. Knowledge based image analysis is a combination of digital signal processing and symbolic reasoning. In this paper, we will look at some problems connected to the symbolic reasoning approach to image interpretation and see how an iconic representation can help to solve some of them. We will show that many of the features and problems connected with both symbolic and iconic representation are complementary. J
1. Introduction Image analysis is the automatic interpretation of what is represented in one or more images in terms that are relevant to the application domain. The most natural terminology for the interpretation is not necessarily a verbal description. For example, the extraction of lines from satellite images and their interpretation as a road network is better described by the generation of a map than by a verbal description of the shape and spatial relations between the individual roads. The different levels of detail or abstraction encountered during an image analysis process pose different requirements to the data representation mechanisms. Processing of the data is very dependent on the representation scheme that is used. In
* P. Suetens is also a senior research associate of the National Fund for Scientific Research, Belgium . 016%8655/91/$03.50
this paper we will first take a closer look at the properties of the visual data that has to be processed. We will then examine the problems related with the classical expert system approach and discuss how the use of an iconic representation can help to circumvent some of these problems. Although Sloman (1975) in his articles objects to the term 'symbolic' when referring to verbal descriptions (which he calls 'propositional'), the term has become so familiar in the AI community that we will make use of it throughout the paper.
2. Properties of visual data During the analysis of an image, we can distinguish different interpretation levels. The classification given below is more or less subjective but serves to summarize the change of data properties and processing requirements during an interpretation process.
© 1991 - - Elsevier Science Publishers B.V. All rights reserved
781
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
2.1. Low level At the lowest level we have the raw image data consisting of pixels whose value represent intensity and sometimes color information. If we take as a typical example an image of 512 by 512 pixels, each pixel represented by 8 bits, then a quarter of a megabyte storage space is needed. Usually, images are preprocessed before segmentation algorithms are applied. This means that the lowest level in image analysis requires large amounts of memory to store the data. The processing algorithms at this level are usually very simple and require no special control strategies. The algorithms are often suited for parallel implementation. 2.2. Mid level We can consider the mid level to be the level that contains the results of a segmentation phase. The data at this level are classically regions or line segments obtained after the application of segmentation algorithms followed by either split-andmerge or erosion-dilation-thinning steps. Typical properties of the data at this level are: • There is still a large amount of data, typically a few hundred to a few thousand segments. • The number of different segment types that can be distinguished is very small. For line or edge segmentation we can distinguish for instance lines, junctions and unconnected points. Further distinctions (e.g., long lines, curved lines) already require some form of analysis. • The fact of having many data elements that can only be categorized according to a small number of types means that data structuring according to the type of data will not be very helpful for further processing. More important are the spatial structure and relationships between the segments. These offer an alternative way of structuring according to spatial proximity and topology. 2.3. High level The highest level contains the most domain or application dependent knowledge. This level contains the domain dependent description of the model or models to be instantiated. It is also the 782
December 1991
level at which the interpretation of the image contents becomes intelligible and interesting to the user because it is formulated in application dependent terms instead of in terms of spatial relations between rather abstract segments. Model description in vision is most natural if a hierarchical or multilevel representation can be used. Part-of relations as well as spatial relations play an important role. The different levels encountered during an image analysis process lead to a hierarchical representation pyramid as shown in Figure I. The lowest levels of this pyramid correspond to Mart's raw primal sketch (Marr (1982)). Each level in the pyramid is a conceptual entity but can itself form a hierarchical structure. At the lowest level images can be represented in multi resolution. The spatial network level can contain many layers, ranging from a network of the original segmentation to networks where conceptual grouping has only retained the most important perceptual entities. The higher the level in the pyramid, the greater the data reduction, but at the same time, the complexity of the data structuring and of the control strategies to process the data increases, as well as the application dependency. Very important to notice in this multi level representation scheme is the lack of continuity in the data representation. At the lower levels of the pyramid the data has a signal character whereas the higher level data has a symbolic nature (see Figure 1).
3. The use of symbolic representation and reasoning in image analysis As an example of symbolic representation and reasoning, we will consider a rule based expert system approach for the early interpretation of image data. The use of expert system techniques and symbolic reasoning to interpret an image is situated at the symbolic level of the representation pyramid. The input data of such systems consists of a symbolic description of the primitives extracted by a so-called segmentation phase. Because of the large amount of data and the lack of initial context many vision systems start with a data
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
driven or bottom up phase in order to combine the primitives into more meaningful entities.
3.1. A typical example We will take the segmentation and interpretation of blood vessels on a radiological image as a typical example of this kind of processing (Van Cleynenbreugel et al. (1988)). The segments extracted from the original image consist of rectangular structures representing parts of the blood vessels. These were obtained by combining the results of an edge detector and a ridge detector algorithm (Figure 2a) (Smets (1990)). • The model knowledge used at this level in a data driven system consists of more or less domain independent perceptual grouping heuristics. In the expert system approach that we consider, these heuristics are expressed in the form of rules. An example of such a perceptual grouping rule is shown in Figure 2b. The conditions in the if part of the rule are
December 1991
checked against the data present in the system by pattern matching. When data is found that satisfies all the conditions, the rule is marked for execution and possibly fired. The result of applying the example rule to the original segmentation data is shown in Figure 2c.
3.2. Problems approach
related
to
the
expert system
3.2.1. Data description In order to be able to reason with the segmentation results, the segmentation data has to be represented symbolically. This symbolic description entails a simplification of the data. Line segments are broken into linearized pieces so that they can be represented by their endpoints. Reasoning on these segments is based on comparing or matching the coordinates of their endpoints. As an example, suppose we want to establish proximity between segments. This is usually very important information for perceptual
A globalproperties (db records)
symbolic levels
t situationsN h
Sa
/
t
components+ X rt-ofand spatialrelations "~
t
lines+ edges + regions + spatialrelations
\
t iconic l e v e ~ / /
t input images
\
Figure 1. Representation pyramid.
783
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
grouping. Reasoning with the symbolic description of the segments allows to derive this information for the case represented in Figure 3a. The situations in Figure 3b will be overlooked. Visual data is most naturally represented at different levels of detail. The abstraction level of the
and and and and then
there is an edge <01> with orientation
(b)
December 1991
description however, must correspond to the abstraction level of the reasoning process. As an extreme example, the position and the intensity of every pixel in an image can be asserted as a symbolic fact (actually, the internal computer representation of an image can be considered as such). Although such a symbolic representation is complete and contains the same information as the original image, it is useless if the knowledge used for reasoning is not expressed in terms of pixel position and intensity. As a more realistic example, we can consider how to describe the results of the perceptual rule of Figure 2. The data to be described consists of long, curved blood vessel segments. On the one hand, we can easily describe general properties of each segment such as length, average diameter, average curvature and position of endpoints. This description however, is useless if we want to reason about proximity o f other blood vessels or about particular shape properties (see also the configurations of Figure 3b). On the other hand, we can represent a blood vessel segment by the chain of smaller rectangular segments out of which it consists. In this case, the reasoning knowledge (the rules) would have to be stated in terms of these chains of smaller segments and would lead to an extremely complicated and cumbersome knowledge description. Especially in the second case, there is the additional problem that the reasoning level needed to process the data descriptions is on the level o f internal representation and implementation and has little or no relation to the
¢,--
J (a) Figure 2. A rule based approach. 784
(b) Figure 3. Proximity between segments.
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
knowledge level of the human expert. Although the two data descriptions proposed in this example seem like two extreme cases, it is very difficult to find an intermediate symbolic description that does not pose the same problems.
3.2.2. Model description The automatic analysis of the image requires a model description. This description tells the analysis system what to expect in the image and how to interpret the image data. Symbolic description of these models poses approximately the same problems as the description of the image data. Again, especially the visual aspects of this description, like shape and spatial relations between object parts, pose the greatest problems. 3.2.3. Combinatorial behavior In the rule based approach as presented in the example, no structure is imposed upon the data. This data consists of object-attribute-value triplets that reside in an unstructured data pool, called the working memory. Some systems combine the rule based approach with object oriented structuring (ART, KEE, Nexpert Object), thus allowing some hierarchical structure to be imposed on the data. In both cases however, the spatial structure of the data, which is very important for vision, is lost. This means that data items cannot be reached by spatial access paths but only through pattern matching. This has serious effects on the efficiency of such a system. As an example, let us consider a simplified version of the perceptual grouping rule of Figure 2. if (the position of object.1 (the position of object.2 (close-to ?x ?y) then,.,
December 1991
This test however, will be applied to all possible combinations of objects, leading to a system whose run time increases as a combinatorial function of the number of objects present (Figure 4). In rules where more than two objects must be compared to each other, the situation is even worse. Although sophisticated schemes such as the RETE algorithm (Forgy (1982)) allow to transfer some of the work to a precompilation phase, the problem of combinatorial matching is only shifted, not resolved. The use of windows in the image to restrict the matching is another way to circumvent the problem. The S C H E M A system reported by Draper et al. (1989) uses such a windowing strategy. This kind of solution however can give rise to complicated knowledge description when results from neighboring windows have to be combined.
3.2.4. Discontinuity in the data representation We can consider the interpretation of an image as the construction of the representation pyramid of Figure 1. In systems that are to solve real world problems, this interpretation typically is not a straightforward bottom up or top down process, mainly because of errors in the low level segmentation phase. The initial segmentation however, can be used to establish an initial reasoning context. Within this context, which forms a partial inter-
run time
panern matching
exploration
is ?x) and is ?y) and
The le•hand side of this rule describes the conditions for proximity between two objects. The first condition contains no restrictions and will match with every object containing a position attribute. The same situation applies to the second condition. It is only in the third condition where the proximity check is made, that object pairs that do not passs the close-to test will be filtered out.
breakeven point
number of objects
Figure 4. Pattern matching versus exploration. 785
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
pretation, segmentation errors can be hypothesized. These hypotheses can then be verified by checking them with the image data. This leads to more reliable context information that allows a more complete and correct interpretation which in turn leads to more correct hypotheses. The higher levels of the interpretation pyramid interact more or less in the same way. The whole hypothesis verification process iterates through the different levels of the interpretation pyramid. This process, which requires complex control strategies, is further complicated by a discontinuity in the data representation. At the lowest levels o f the pyramid, the data has a signal-like nature whereas segmentation results and model specification at the higher levels are described symbolically. In order to communicate information between the image-like and the symbolic levels of the pyramid in a flexible way, special software interfaces must be provided. Most systems only provide a loose coupling between iconic and symbolic levels in the form of a call out facility to Fortran, Pascal or C routines from within the knowledge based tool. Such a loose coupling gives insufficient support for a flexible interaction between the different representation levels.
4. Iconic representation When we refer to the term iconic representation in an image analysis context, we mean that the information is represented by pixels in a digital image. Models of objects can be represented iconically by drawing them or by processing existing graphical representations. Original image data as well as segmentation results are examples of iconically represented data. 4.1. Established research The accurate description of the information present in a road-map shows a typical example of the need for iconic representations. It is fairly straightforward to represent the topology of the road network in a symbolic way by describing roads as connections between the different crossings. Global properties of individual roads such as length, position of the endpoints and average cur786
December 1991
vature can also be represented symbolically. Representing the exact shape of a road symbolically is less straightforward. Iconic description of such a road (i.e., making a drawing of it) offers an alternative. Iconic representations are a subset of the more general analog representation schemes that were studied by Sloman (1975). In his papers he made a clear distinction between analogical and propositional representations. Marr (1982) was one of the first vision researchers to actually use iconic representation. His idea of using a bitmap to limit the search in what he called a raw primal sketch, is an example of exploration versus purely symbolic reasoning (see also Section 4.2.3). Few vision systems actually make use of iconic representations. A system in which iconic representation does play an important role is W H I S P E R (Funt (1980)). This system simulates a blocks world where unstable configurations o f blocks are considered. Instead of reasoning symbolically on the situations presented to it, W H I S P E R uses diagrams that represent the blocks. In these diagrams, the subsequent situations of the block configurations are depicted. The effects of tumbling blocks are observed by means of a 'retina', i.e., a simulation of the human retina that has a number of interesting properties (focussing, similarity testing, etc.). W H I S P E R is interesting because it uses iconic representation and observes spatial configurations. The system however, only simulates a simplified physical situation. Image analysis is restricted to observation of the diagrams. No interpretation of real world image data takes place. A real image analysis system in which iconic and propositional representations are combined is NEUROLOGIST-1 (Shrihari and Xiang (1989)). The task o f NEUROLOGIST-I is to recognize physical entities and lesions in cross-section images of the human brain. The system however, does not reason directly on iconic representations of these cross-sections. Instead it reasons upon sets of vertices representing the convex hull of the physical entities. In this sense, the system reasons on a symbolic description rather than observing an iconic representation.
Volume 12, Number 12
""
PATTERN RECOGNITION LETTERS
L
December 1991
d
(a} Figure 5. Different environment shapes.
I-SEE, described in Fierens et al. (1990 and 1991) is a knowledge based development tool for image analysis that was used to implement the examples described below. Iconic representation and exploration are key concepts in the tool. Exploration happens by means of environment objects that have user defined or object dependent shapes (Figure 5). These environments are used in combination with an integrated image processing library. Environments can be overlayed on top of the original image and used for resegmentation purposes. The I-SEE tool supports the notion of label images in which the pixels are labels or pointers referring to symbolically described objects. This notion is similar to Marr's bitmaps or to occupancy arrays. Label images can be explored by means of environments. A label image, together with its symbolic description in the form of a spatial network can be extracted automatically from a binary image (e.g., the result of a segmentation phase). It must also be clear that an iconic representation cannot express all the knowledge about a model. Iconically represented information is usually restricted to shape, color or spatial organisation. Therefore, iconic representation is not an alternative to symbolic representation but must be seen as an additional representation paradigm. In the next section we will take a closer look at some properties of this iconic representation.
4.2. Properties 4.2.1. Description of objects and relations As already mentioned, the symbolic description
of visual information such as shape or spatial relations is very difficult. Attempts to describe this information verbally easily lead to descriptions that are either too general or too complex for spatial reasoning. If the system under development is based on symbolic reasoning, then the knowledge acquisition of the visual information is a tedious process. The iconic representation is in comparison a very natural one for representing shape or spatial properties. The knowledge acquisition consists of making a drawing of the model to be instantiated or to process a pictorial representation of a model into a line drawing. Both ways of obtaining an iconic model are relatively fast. Non-visual information can still be acquired and represented symbolically. More than one model can be needed to represent the relevant information. Examples are a set of two-dimensional projections of a three-dimensional object, seen from different angles or a set of alternative models if the exact appearance cannot be predicted. An example of the last case is a model of the cardiac blood vessels where the appearance can differ widely from one patient to another due to biological variability. Except for digitizing errors, the representation of the object shape is accurate and there is no need for simplification as was the case in the symbolic representation o f Section 3.2.1. Representation o f 3-D shape can be done by representing a number of distinguishing projections but if memory space and performance are not critical, 3-D arrays can be used to represent 3-D models. Again, this representation is accurate with respect to shape and spatial properties. 787
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
~(b) Figure 6. Establishing relations by exploration: (a) visibility, (b) inclusion.
Spatial relations between objects in an iconic representation are implicit in the image. Exploration in the iconic representation allows to establish relations that would otherwise have been difficult to extract from a symbolic database o f objects. We already gave an example o f the kind o f problems that can be encountered when trying to establish the proximity relation (Figure 3b). Proximity can be established directly from an iconic representation by means o f the environments o f Figure 5. An
environment can be put on top of an original or a labeled image and scanned. The shape of the environment implicitly defines the proximity relation. Circular environments (Figure 5a) define proximity as a uniform maximum distance from a specific point, environments with a triangular form (Figure 5b) define proximity with a directional aspect. Environments can also follow the shape of an image object (Figure 5c), an example o f proximity definition that would be very difficult in a purely symbolic way. Another example is the establishment o f the visibility relation between two objects. This relation can be important for hypothesizing segmentation errors. The situation where two collinear lines lie in close proximity but are not connected can indicate a segmentation error if no other segments lie in between them. The relation visibility between
ft------'7 Figure 7. Perceptual grouping in I-SEE. 788
December 1991
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
the endpoints of the two segments can check this last condition. Establishing the visibility relation in an iconic representation requires nothing more than checking the pixels in the image that lie on a straight line between the endpoints (Figure 6a). A last example of a relation which is complex to establish by symbolic reasoning is the relation inclusion. In the iconic representation this relation can be checked by drawing scanlines and checking the pixels until either the enclosing object or the image border is reached (Figure 6b).
4.2.2. Linear behavior Establishing the relation proximity in a symbolic manner gave rise to a combinatorial explosion. Using the iconic representations o f the I-SEE tool, this same relation can be established by creating environments at the endpoints or around the object under consideration, thus limiting the region of the image to be scanned. One or two region scans are needed for every object considered. The run time for establishing the proximity relation for an entire image will thus be a linear function o f the number o f objects present. The same reasoning can be applied to a number o f other spatial relations (visibility, inclusion . . . . ). For a small number of objects, combinatorial comparison o f endpoint coordinates can be more efficient than an environment scan. The position o f the break even point (Figure 4) where both processing schemes are equally efficient depends on the dimensions o f the environment and on the functionality that is required (do we need to detect the cases o f Figure 3b or not?). Any realistic application however, will be situated to the right o f this break even point. Therefore, linear behavior is an important feature that allows the scaling up from small test examples to real world applications.
4.2.3. Cottsistency The iconic representation, e.g. when used for representing segmentation results, can be considered as an iconic database. Image analysis systems that make use o f resegmentation can add or remove segments in this iconic database x~dthout consistency problems. Information in the database is present in an implicit form and can be extracted
December 1991
by exploration. If a visibility relation exists between two segments and resegmentation adds a new segment in between, then a new exploration will detect this change. Consistency problems only occur if relations between objects are extracted from the iconic database and represented symbolically. Figure 7 shows the intermediary steps o f the perceptual reasoning module o f I-SEE applied to the segmentation o f an X-ray image of a blood vessel tree. Instead o f continuously checking and updating information extracted from the segmentation image and represented in a symbolic form, the system keeps adding and removing segments in the iconic database. Processing relies mainly on exploration (observation) as an alternative to symbolic reasoning. At specific points in the processing, the information drawn in the iconic database is scanned and extracted into a symbolic network on which additional reasoning can take place.
4.2.4. Iconic inferencing In the previous example, iconic representation and exploration were used in combination with symbolic reasoning on the information present. It is also possible to use iconic representations to obtain end results without first extracting a symbolic description. The methodology o f "snakes' reported by Kass et al. (1988) is a form o f what we could call "iconic inference'. Snakes are digital curves or surfaces xvith a simulated physical behavior. Internal forces in the snake give it elastic and plastic properties, external forces try to deform the snake. These external forces can be applied by the user or by a knowledge based module, but can also be extracted f r o m the images themselves. The snake will be attracted or repulsed by these external forces and will deform in a (simulated) viscose medium until an equilibrium between internal and external forces has been reached. The deformation energy o f the snake at equilibrium gives information on how well the data fits the model. As an example o f the use o f this technique, consider Figure 8. The snake was tracked as a contour line on an iconically represented terrain elevation model. We have used it to detect roads on a S P O T satellite image o f a mountainous area. The (weak) heuristic that was implicitly used assumes that 789
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
roads in a mountainous area try to minimize altitude changes rather than distance. Therefore, they are likely to follow contour lines (lines of equal altitude). In order to use such a contour line snake for the detection of a road, a local environment was first created around it. Within this environment, the output of a line detector algorithm by Fishier and Wolf (1986) and a Sobel operator
were combined to yield a forcefield in which the snake was then embedded. The simulation module of the I-SEE tool, which iteratively solves the dynamic system of differential equations that represents the snake and its environment, deforms the snake until the equilibrium state is reached. The line detection filters used to generate the forcefield are such that the snake is attracted to
Figure 8. 'Iconic inferencing'. 790
December 1991
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
pixels with road-like properties. At the same time, the internal forces of the snake resist deformation, thus enforcing the original shape of the model. The end result, shown in Figure 8 is a compromise between the influence of the model and the influence of the data.
5. Discussion Finally, let us summarize the properties of both iconic and symbolic representation schemes. Iconic representations are easy to acquire and are a complete representation for shape, intensity and color information and for the representation of spatial relations. This completeness is usually with respect to a two-dimensional projection of the scene although in principle 3-D iconic representations are also possible. Iconically represented information is implicit and can be accessed through exploration. Together with the access through exploration, this implicitness ensures consistency when objects are added to or deleted from the representation. Exploration in an iconic representation is a process that has a linear run time behavior with respect to the number of objects in the system. This is an important feature for scaling up from a small testcase to a real world application. Finally, the representation is a natural one for visual information because it is close to the real world data that has to be represented. It is also a representation that is largely application independent. The processing of an image or a line drawing or the comparison between a segmentation and a model line drawing can to a large extent be performed without domain knowledge. Domain knowledge however, will enhance the reliability of the interpretation. Symbolic representations on the other hand allow data or knowledge to be represented at different levels of abstraction and generalization. Shape information and spatial relations between objects are difficult to represent and usually lead to oversimplification that is necessary for subsequent reasoning processes. Symbolic representations are well suited to represent other than visual information, which is not possible in an iconic
December 1991
representation. Some visual aspects o f the knowledge, like rotation, translation and scale are difficult to handle in a purely iconic manner. The flexibility of a symbolic reasoning system can be needed to handle these. Systems that use symbolic representation very often access their information through pattern matching. This gives rise to a combinatorial behavior which poses serious problems to any application. It is important to note that the properties of the iconic and the symbolic representation are more or less complementary. The iconic representation scheme is an ideal tool when combined with symbolic descriptions.
Acknowledgements This text presents research results of the Belgian National incentive-program for fundamental research in Artificial Intelligence, initiated by the Belgian State - Prime Minister's Office - Science Policy Programming. The scientific responsibility is assumed by its authors. The examples using satellite images are based on SPOT data under a SPOT-IMAGE(R) license acquired in collaboration with EUROSENSE T E C H N O L O G I E S N.V., Belgium.
References Draper, B.A., R.T. Collins, J. Brolio, A.R. Hanson and E.M. Riseman (1989). The Schema system. Int. J. Computer Vision 2, 209-250. Fierens, F., J. Van Cleynenbreugel, P. Suetens and A. Oosterlinck (1990). l-see: An AI-tool for image understanding. Engineering Applications of AI 3, 62-70. Fierens, F., J. Van Cleynenbreugel, P. Suetens and A. Oosterlinck (1991). A software environment for image database research. J. Visual Languages and Computing, to appear. Fishier, M.A. and H.C. Wolf (1986). A general approach to machine perception of linear structure in imaged data. Technical Note 276, SRI International. Forgy, C.L. (1982). Rete: A fast algorithm for the many pattern/many object pattern match problem. Artificial Intelligence 19, 17-37. Funt, B.V. (1980). Problem-solving with diagrammatic representations. Artificial Intelligence 13(3), 201-230. 791
Volume 12, Number 12
PATTERN RECOGNITION LETTERS
Kass, M., A. Witkin and D. Terzopoulos (1988). Snakes: active contour models. Int. J. Computer Vision l, 321-331. M a r l D. 0982) Vision. Freeman, San Francisco, CA, 41-89. SIoman, A. (1975). Afterthoughts on analogical representations. Proc. Theoretical Issues in Natural Language Processing. Cambridge, MA, 164-168. Smets, C. 0990). A knowledge-based system for the automatic interpretation of blood vessels on angiograms. PhD thesis, Acta Biomedica Lovaniensia 20.
792
December 1991
Srihari, S.N. and Z. Xiang (1989). Spatial knowledge representation. Int. J. Pattern Recognition and Artificial Intelligence 3(1), 67-84. Van Cleynenbreugel, J., F. Fierens, P. Saetens and A. Oosterlinck (1988). Knowledge-based improvement of automatic image interpretation for restricted scenes: two case studies, hnage and Vision Computing 6(4), 238-246.