Cooperation of fuzzy segmentation operators for correction aliasing phenomenon in 3D color Doppler imaging

Artificial Intelligence in Medicine 19 (2000) 121 – 154 www.elsevier.com/locate/artmed

Cooperation of fuzzy segmentation operators for correction aliasing phenomenon in 3D color Doppler imaging Ahmad Shahin * , Michel Me´nard , Michel Eboueya Uni6ersite de La Rochelle, A6enue Marillac, 17042 La Rochelle Cedex 1, France Received 10 September 1999; received in revised form 15 January 2000; accepted 25 February 2000

Abstract The study described in this paper concerns natural object modeling in the context of uncertain, imprecise and inconsistent representation. We propose a fuzzy system which offers a global modeling of object properties such as color, shape, velocity, etc. This modeling makes a transition from a low level reasoning (pixel level), which implies a local precise but uncertain representation, to a high level reasoning (region level), inducing a certain assignment. So, we use fuzzy structured partitions characterizing these properties. At this level. each property will have its own global modeling. Then, these different models are merged for decision making. Our approach was tested with several applications. In particular, we show here its performance in the area of blood flow analysis from 3D color Doppler images in order to quantify and study the development of this flow. We present methods that detect and correct aliasing phenomenon, i.e. inconsistent information. At first, the flow space is partitioned into fuzzy sectors where each sector is defined by a center, an angle and a direction. In parallel, the velocity information carried by the pixels is classified into fuzzy classes. Then, by combining these two partitions, we obtain the velocity distribution into sectors. Moreover, for each found path (from the first sector to the last one), we locate and correct inconsistent velocities by applying global rules. After extracting some meaningful sector features, the fuzzy modeling, applied to the aliasing correction, makes it possible to

 This work is part of a project that has been realized at the Laboratoire d’Informatique et d’Imagerie Industrielle of the University of La Rochelle in collaboration with the Consortium FISC: Fluide, Image, Signal Cardio-vasculaire (CHRU Poitiers, University of Poitiers and University of La Rochelle). * Corresponding author. Tel.: + 33-5-46458262; fax: + 33-5-46458242. E-mail addresses: [email protected] (A. Shahin), [email protected] (M. Me´nard), [email protected] (M. Eboueya)

0933-3657/00/$ - see front matter © 2000 Elsevier Science B.V. All rights reserved. PII: S 0 9 3 3 - 3 6 5 7 ( 0 0 ) 0 0 0 4 2 - 7

122

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

simplify and synthesize the blood flow direction. © 2000 Elsevier Science B.V. All rights reserved. Keywords: Global modeling of natural objects; Inconsistent information; Fuzzy sets; Fuzzy clustering; Fuzzy fusion; Doppler imaging

1. Introduction Non-invasive quantification of blood flow through the heart valves and in the big vessels is of major clinical importance in the assessment of cardiac output and valvular disease. It is widely used to detect anomalous blood flow in the vicinity of the diseased locations, such as high velocity jets and turbulent flow. The non-invasive techniques include velocity-encoded magnetic resonance imaging (MRI) [42,43] and Doppler ultrasound. Ultrasound methods, in particular pulsed Doppler and Color Doppler Flow Mapping (CFM1) (i.e. based on the transmission of pulsed ultrasound: bursts) [38,25,16], are capable of measuring blood velocities over a surface normal to the point of scanning. They considerably improve the quality of diagnosis, both in adult and pediatric cardiology [36,45]. Actually, they allow a quick and particularly sensitive analysis of the different disturbances of the intracardiac flows and therefore brings an informative complement, which is essential to the two-dimensional conventional echography (echography mode B or mode TM). CFM was qualified at its beginning as a cartography of real time 2D flow mapping. It is probable that in their hurry to explore the possibilities of such a tool, the first experimenters have partly neglected the technological constraints of this particular type of Doppler imagery. This was checked for the cartography, of the valvular regurgitation. The velocity quantification within the cardiac structures is carried out thanks to the techniques called continuous Doppler or pulsed Doppler. Historically. the continuous Doppler is the mode which has first been used in vascular pathology, and it was long believed that its use in cardiac pathology was impossible because of its spatial ambiguity (permanent emission and reception by separate transducer). This is why the pulsed Doppler with its space location qualities was developed bringing along the concepts of time gating (sample volume), ultrasonic pulse propagation, pulse repetition frequency, (PRF) and finally the concept of Nyquist limit. The guiding principle of this technique consists at detecting the phase difference between transmitted pulse trains and their echoes received at the level of the Doppler transducer [15,33]. The information provided by the two-dimensional Doppler is an item of information comprising the average velocity (obtained by the variation of Doppler frequency analyzed by the autocorrelator) and the variance of the blood flows on a preset anatomical sector. The latter denotes the studied flow organization from a hydrodynamic point of view. 1 This system determines the spatial distribution of the average velocity as a function of time across the entire scan plane. Therefore. CFM systems use multiple gates equally spaced along the ultrasound beam. Typically, the blood flow is coded, so that red color indicates forward flow (positive velocity) and blue backward flow (negative velocity).

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

123

As Fig. 1 shows, a color Doppler linage I, of an anatomical sector R, can be represented by using a polar frame of reference with as origin the site of the transmitter-sensor. The value I(r,f) for each point of the image then represents the average velocity of red blood corpuscles located in sampling volume VIIa,M (r,f) at a distance r of the sensor and in a direction f, compared to a vertical reference axis. The angle of browsing f also conditions the browsing rate by increasing the number of lines to be analyzed. In general, one uses the minimum angle imposed by the private clinic. These velocities are coded by a pallet of colors (cf. Fig. 2). In color Doppler, the sampling volume VIIa,M (r,f) defining the resolution of information is the smallest unit of volume in which the average velocity and the variance are calculated in a statistical way. It defines the image-point. This volume is explained only regarding two directions: “ according to the axial direction (on the axis of the ultrasonic pulse propagation): the dimension is a function of the duration of the temporal gate, IIa, corresponding to the duration of each ultrasonic pulse train;

Fig. 1. Color Doppler echographic image. For each section, the value I(r,f) for each point of the image then represents the average velocity of red blood corpuscles located in a sampling volume VIIa,M(r,f) at a distance r of the sensor and in one direction f, compared to a vertical reference. The u angle defines the exploration scan plane.

124

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

Fig. 2. Color range coding the velocities of flow in Doppler imagery. Velocities are represented in an image by values in {1, 2, …, 127). The velocity corresponding to the frequency of Nyquist is noted V 9 Nyquist. “

according to the lateral direction: the dimension is then calculated from the number M of laterally integrated ultrasound beams.

The obtained image is a rectangular matrix M× N (cf. Fig. 1) comprising the whole of information in form of a color coding. The M columns of the image are determined by a set of lateral ultrasound beams each one considering the average velocity and the variance of a number N of sampling volumes. Whereas pulsed Doppler provides some velocity information belonging to a precise point, color Doppler mapping estimates the spatial mean velocity of blood flow in a sample volume, across the transverse plane of a vessel or a valve. It is characterized by the concept of Doppler color sample volume (packet size): it is the elementary image point, (i.e. VIIa,M (r,f) P(k,l) where (k,l) are the coordinates of P) different from the pixel of the video screen, distributed along the echographic lines and defined by the axial and lateral resolutions of the system. The size of sample volumes conditions the sensitivity of Doppler color but to the detriment of its axial space resolution. A compromise is thus desirable. The axial resolution depends on the duration of the temporal gate corresponding to each pulse (burst). The lateral resolution depends on the number of lines used by angle of explored sector. In addition these two resolutions can be modified by the use of lateral and axial smoothing which integrates the adjacent packet-size values. The spatial as well as temporal precision remains an essential problem, but generally, it is significant to obtain most precise velocity information. In short, the information obtained by two-dimensional Doppler is composed of the average velocity, intensity and variance. The variation of Doppler frequency analyzed by the autocorrelator makes it possible to obtain information concerning

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

125

the average velocity, which is very near the maximum velocity, provided by pulsed Doppler in the event of laminar flow, but considerably different in the case of turbulent flow. The intensity is the factor which holds the concept of representativeness of a given frequency (thus velocity) in the frequency spectra. Finally the variance bears the concept of spectral spreading out around the average. A significant variance reveals a difficulty of precisely evaluating a velocity. It generally comes from an area of turbulence where the direction and the amplitude of the velocity vector are random. These three data items are represented on the color range available on most commercial echographs.

2. Problem These Doppler techniques assume only the incompressibility of the fluid. But there are several Doppler specifications estimating the spatial mean velocity of blood across the transverse plane of a vessel or valve [50,38,25]. The technique of the pulsed Doppler can present problems of aliasing for high velocities of the blood flow. The detection of the Doppler signal is submitted to the law of Nyquist: a spectral aliasing appears when the Doppler frequency is higher than the frequency of Nyquist fr/2 ( fr, repetition rate of the Doppler ultrasound beam). It results in a shift of the value of velocities, illustrated by Fig. 3. This phenomenon, known as aliasing (spectral recovery), is frequently met in cardiology. The fd, Doppler frequency band, being able to be detected without ambiguity must meet the following equation: fd B1/2fr. The images presenting an aliasing show an erroneous field of velocities which can lead to a false interpretation (cf. Fig. 4). It is then impossible to undertake quantitative measures from velocities. The field of velocities has therefore to be corrected before any calculation. As a consequence of these specifications, the information provided by CFM may often not be interpreted correctly due to the accuracy of the measurements:

Fig. 3. Effect of aliasing. When the flow velocities (continuous functions) exceed the limit of Nyquist, one obtains an erroneous value with a change of sign.

126

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

Fig. 4. Example of Doppler image with spectral aliasing. They show a mainly positive flow, the sensor and coded in red (positive velocity). The used angle is 20° for 30 lines of scanning. When the flow velocities exceed the limit of Nyquist, one obtains an erroneous area with a change of sign (negative velocity). The total number of samples by line is 128. No type of axial and lateral smoothing and lateral has been used. Adjustments of scanners were as follows: low filter velocity =0.19 m/s, ten bursts per line., sweep time total of 60 ms.

“

“

“

the images are corrupted by noise. Indeed the amplitudes of the measured signals are small. The characteristics of the noise generated by the Doppler imaging system are decorrelated from the characteristics of the Doppler information. The Doppler data are not accurate; an aliasing phenomenon, due to the limitations of Doppler systems in the maximum measurable velocity at a given depth, affects the measured data giving inconsistent velocities. This phenomenon is connected to the Nyquist theorem: the gating of the echoes at a fixed time after the burst is transmitted implies the sampling of the received signal; higher Doppler frequencies alias to lower frequencies, if they exceed the Nyquist-frequency (i.e. the velocity becomes ambiguous). So the Doppler data are ambiguous and inconsistent; the spatial information is also insufficient:  the resolution of the image is limited by the partial volume effect that comes from the sample volume size that is interrogated by burst; the velocity profiles at valves are generally non-uniform in each sample volume;  in three-dimensional imaging the number of acquisitions may be very limited due to long measurement time and it is difficult to maintain a constant quality of the scan-plane under all incidences.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

127

The measurement uncertainty is often represented by a probabilistic approach. But such a representation of measurements is not always adapted to the treatment of information, such as, for example, the managing distortions and deformations, the compensation of the lack of information in the images, and aggregation, in particular when symbolic knowledge is used. Various approaches have been used to handle uncertain information and imprecision in image processing. These approaches include expert systems, fuzzification, probabilistic reasoning and neural networks. In this paper, we will briefly review these approaches. In expert systems. pure knowledge-based and hybrid knowledge representation techniques have been used for automation of uncertain image interpretation [40,12]. In these papers an attempt is made to formally define objects, relations, and strategies on the basis of perceptual and conceptual levels of human knowledge processing using frame-based rules. Alternatively, model-based scene interpretation systems are used in [1,26] to resolve many of the inherent ambiguities and uncertainties present in real world image data. In [46,5] techniques for image interpretation based on knowledge-based systems are used for solving detection or classification problems. Recently, there were several methods concerning different models of intelligent systems for medical diagnosis which have been proposed: these methods include Bayesian approach and computer learning methods. In particular, these latter have been developed to extract knowledge from the database [11,18,44]. The general approach consists in two phases: first, a learning phase uses some sample cases, and second, the resulting system is used to classify new unsolved cases. The predicting learning method often consists in a k-nearest neighbors clustering with a defined measure of similarity. Neural networks [48] also seem to offer advantages for machine vision applications. In the approach taken in [7,21] a neural network is used to generate self-organizing models that can learn to eliminate noise and intrinsic errors introduced by the measurements. Unfortunately neural networks are useful in discrimination, but not in propagating related levels of uncertainty throughout the model. Since Zadeh introduced the theory of fuzzy, set in 1965 [51], the concept and method have been applied in a great number of scientific and technical disciplines. For example, fuzzy sets have been proposed for handling uncertain images in biomedical sciences when appropriate mathematical models could not be developed due to the complexity of the problem. It uses the tolerance for uncertainty and vagueness in fuzzy logic reasoning method to handle the complex decision problem in image processing. This theory has been used for labelling groups of pixels, i.e. regions or objects in images. In [3] fuzzy logic proposition are used to represent knowledge and fuzzy reasoning to model inference mechanisms. Several works have dealt with the description of uncertainty following regard of fuzzy set and probability. Fuzzy – Bayesian inference is one of hybrid fuzzy-probability models. The Bayesian theorem is an effective tool for reasoning in the case of uncertainty. An example of this reasoning model is given by [39] for diagnosis of pulmonary tuberculosis. Fuzzy neural networks provide a fuzzy logic system with the powerful computation and learning capabilities of neural networks [23,19,20]. One of the

128

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

main problems of neural fuzzy systems is to develop a systematic method of constructing its internal structure; that is, the fuzzy, term sets and the fuzzy rules in their hidden layers. In this paper, we consider the problem within the framework of fuzzy set theory. Since there is no single approach to handle uncertainty, fuzzy sets, learning methods and knowledge bases defined as sets of rules based on experiences, are complementary. So, we present a general procedure combining these approaches. The advantage is that the problems need not be modeled in a conventional mathematical way, and be more flexible and solved easily: “ fuzzy sets are universal approximators to handle complex, non linear, imprecise, and conflicting data. Fuzzy sets are also able to propagate uncertainty itself throughout the system. Moreover, fuzzy systems use a minimum number of rules; “ learning methods extract from images several prototypes (i.e. knowledge) that summarize them; “ sets of rules take into account additional or a priori information which is made up of prior knowledge about the imaging modality that is used. In color Doppler imaging, these priors express concepts of simple hemodynamic knowledge such as the geometry and the direction of the flow, the observed phenomenon varies slowly in time and space, regions are homogeneous. We show how the combination of uncertain data providing from a number of different features and symbolic knowledge can be used to resolve inconsistencies in uncertain data. These procedures integrate learning from examples using a fuzzy clustering algorithm translating measures of features (spatial, velocity ...) into fuzzy representations, classifications describing a set of features by fuzzy images, fusion of these fuzzy images, and aggregation of symbolic knowledge. The global objective of the study is the development of a procedure to quantify and study the development of blood flow within different heart cavities and the left ventricle in particular using the 3D CFM, taking into account the imprecision, ambiguity and inconsistency of the data. A key to the success of the approach is the attribute-oriented modeling and combining for generalization. Our work consists of three parts: Section 3 presents the tools used in data clustering and the cooperation of chosen fuzzy segmentation operators. In this section, we give the definition of the fuzzy images modeling both the features such as color, shape and spatial properties such as fuzzy sector, fuzzy mesh. This approach is applied in the area of blood flow analysis from Doppler images. Its particular interest, for a qualitative and descriptive study of abnormal and evolving blood flow, is shown in Section 4. The steps of this application can be described as follows: “ an acquisition time correction; CFM gives velocity information in an anatomical sector of space. We cannot neglect acquisition time. The first velocities are not computed at the same time as the last ones. In certain instances, it is preferable to remedy this time-lag by a temporal correction. Thus, it was possible to acquire consecutively delayed color images with a known incremental delay between

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

“

“

129

multiple sweeps of the computed acquisition time of an image column. The acquired columns are then structured. We obtain time-corrected flow-velocity. aliasing correction; when obtaining Doppler image velocities exceeds the Nyquist limit imposed by Shannon theorem, the resulting aliasing reverses flow direction. Thus it is impossible to analyse the slower velocities. We apply two fuzzy segmentations to the images: the first is based on velocity information. carried by the voxels. The second is spatial information. Then through combination, we correct inconsistent velocities and detect turbulences by searching for abnormal values. After extracting some meaningful region features. we can quantify and locate the flow along a cardiac cycle (systole, diastole); three dimensional reconstruction using fuzzy interpolation followed by a global spatiotemporal analysis of flow velocities. Ultrasound data are acquired image by image. Each image represents a conical section in the medial axis. The three dimensional marker is cylindrical. There is no information about flow velocities between images, so it is necessary to interpolate the image pixels to define voxels. We use a fuzzy interpolation to decrease loss of information. This flow volume will be spatio-temporally processed analysing the shape and intensity of flow velocities, and determining isovelocity regions.

3. Cooperation of fuzzy segmentation operators

“ “

“

The cooperation of fuzzy segmentation operators follows three steps: using a specific learning approach in order to define prototypes characterizing object properties from the database of examples; processing a classification stage describing image features or object shapes with related to photometry, and geometry. Then, color and shape properties of objects are translated into fuzzy representations which are defined by membership functions. For certain types of partition, the modeling is independent of geometric transformation, such as translation, rotation and deformations due to the complexity of natural objects. combining color and shape information represented by the fuzzy images previously defined.

This system finds its application in the natural object recognition, especially in the medical diagnostic, the color Doppler imaging analysis, the automatic fish sorting and the face recognition.

3.1. Clustering strategy by learning In image processing, the aim of clustering is to reduce the amount of data by, grouping similar pixels together. Partitional clustering attempts to subdivide the data set into a set of subsets or clusters which are pairwise disjoint, all non empty, and produce the original data set via union. Pixels belonging to the same cluster

130

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

share common properties (color, position) that distinguish them from the pixels belonging to other clusters. In this paper we consider the fuzzy clustering. This problem can be defined as follows: “ let Jl denotes the set of index (integer) {1, 2, ..., l}; “ let Z =(zk )k  J , be the family of data (i.e. features vector) where zk = (zk1, zk2, ..., n zkp )t is described by p features (i.e. zk R p); “ let V =(vi )i  J be a family of clusters. c The clustering criterion used to define good clusters is the objective function [2]: c

n

2 J mfcn (m,V) = % % m m ik d ik

(1)

i=1 k=1

where m  Mfcn is a fuzzy partition matrix: m\ 1 is a fuzzifier exponent, mik = mi (zk ) and dik = zk −6i G, G is a norm; dik is a measure of the distance from zk to the ith cluster prototype 6i  V = [61,…, 6c ]. V is a matrix of cluster centers. The fuzzy c-means algorithm (FcM) generates a fuzzy partition providing a measure of membership degree. mik of each pixel, zk, to a given cluster, vi. Its advantages are: “ FcM is less prone to local minima than crisp clustering algorithms since they make soft decisions in each iteration through the use of membership functions; “ in various fields, such as pattern recognition, data analysis and image processing, FcM is very robust and obtains good results in many clustering problems. The algorithm provides an iterative clustering of the search space and does not require any initial knowledge about the structure in the data set; “ the use of the fuzzy sets allows us to manage uncertainty on measures, lack of information, ... all characteristics which bring ambiguity notions; “ most fuzzy clustering algorithms are derived from the FcM algorithm, which minimizes the sum of squared distances from the prototypes weighted by the corresponding memberships [13,17,34,35,49]; “ these algorithms have been used very. successfully in many applications in which the final aim is to make a crisp decision such as pattern classification or image segmentation where it is necessary to make a defuzzification in order to assign each piece of data (gray levels) to only one cluster; the crisp data is divided into a specified number of subsets which need not be fuzzy but utilize fuzzy sets in developing the clusters; “ we may interpret memberships as probabilities or degrees of sharing; “ Krishnapuram and Keller [27] proposed a new clustering model named possibilistic c-means (PcM). where the constraint is relaxed. In this case, the value mik should be interpreted as the typicality of pixel zk relative to cluster vi. But PcM is very sensitive to good initializations and it sometimes generates coincident clusters. Moreover, values in m are very dependent on the choice of the additional parameters needed by PcM; “ in [31], we propose an extension of FcM algorithm allowing the distinction between ‘equally likely’ and ‘unknown’. We define partial ambiguity reject which introduces a discounting process between the classical FcM membership functions. In order to improve the performance of the algorithm in the presence of

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

131

noise, an amorphous noise cluster is defined. So, the membership functions are more immune to noise and the membership functions correspond more closely to the notion of compatibility. The approach described in this article can be generalized using this extended FcM algorithm where the membership degree to the ambiguity reject cluster of a pixel zk is made explicit. For example, this algorithm will permit the distinction ‘equally highly likely’ and ‘equally highly unlikely’. Using an appropriate acquisition mode, an image I can be divided into two images giving two complementary representations of the object into the image: 1. a color image or gray level image: IC, the image will be segmented into homogeneous regions by taking into account only the color property of each pixel of the object. 2. a binary image. IB, representing the shape of the object. Pixels will be clustered in classes by taking into account only their location in the object. The issue of the learning module is to build the prototypes (6i ). According to the type of problem, several strategies may be performed to build the prototypes (we use the terms ‘cluster centers’ and ‘prototypes’ interchangeably):

Pattern recognition approach: The problem of object recognition involves finding a match between a given image of an object and some image information stored in a database. However, there is great difficulty in matching visual data because there is no one-to-one correspondence between images and the object they describe. For example, natural objects (not manufactured) appear complex, in the sense that numerous and not explicit events can affect the representation of objects (i.e. natural deformation) possessing the same interpretation. There are many possible images corresponding to one object, and we would like to recognize the same object regardless of any changes in its images. Natural deformations make necessary the determination of an invariant reference without the particular structure of the object having be pointed out. For this context, we define centers of classes independent of the object natural deformations. In order to compare directly the images between themselves, each image is segmented with the same centers. In a second step, we extract features from segmented images. So the classification becomes a classical pattern recognition. Section 3.2.2 shows applications.

Diagnostic approach: In this context, the issue of the learning module is to build prototypes representative of a class or model. A class is defined by a specific event. For example, one class groups diastole images with a known pathology and a second class groups diastole images without pathology. The family of centers is obtained after application of FcM on one or (several) image representing each model in the database. A medical application is described in Section 3.2.2.

132

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

Analysis approach: The match between an image of an object and some image information stored in a database is not necessary in this approach. Thus, in the case of color Doppler imaging, FcM is applied on the set of exploration scan plans. Then, the centers are adapted to each analysis. The objective function is optimized according to each analyse. This medical application is described in Section 4. For each scenario, we want to form a set of prototypes that characterizes chromatic and spatial properties.

3.1.1. Prototypes obtained using chromatic properties Let c be an integer. It represents the number of classes (i.e. prototypes). Let X= U be the set of color values on which FcM algorithm is applied. U represents “ the system HSI (hue, saturation, intensity) for color images. In the fish sorting and face recognition applications, the prototypes are distributed at regular intervals (pattern recognition approach). The objective function is not optimized for each image. However, the obtained prototypes are independent of the object natural deformations; “ [0, 255] for a grey level image; “ the range of velocities for color Doppler imaging. In this case, FcM algorithm is applied on the set of exploration scan plans and converges to a local optimum (analyse approach). FcM is run using Euclidean norm. a weighting exponent m= 2 (found to produce the best results [31]) and an arbitrary initialization of the m matrix. The weighting exponent controls the extent of membership sharing between fuzzy clusters. 3.1.2. Prototypes obtained using spatial properties The reasoning applied here is made only on spatial coordinates of pixel. Let r be an integer. It represents the number of regions (i.e. prototypes). Prototypes are obtained: “ by application of FcM on the set X= V¦ Z 2, the universe of the coordinates (k,l) x of IB, where V represents the set of all pixels of IB (regarding to spatial position). This method orients the representation of objects in the sense of a structured partition which depends on the architecture of the associated components. This technique proves its performance to detect local deformations. In the application described in Section 3.2.2.1, the centers are obtained after application of FcM algorithm on one or several images representing each model in the database (diagnostic approach); “ by application of FcM on the set X= U of gray levels of Chanfrein distance images. However, in pattern recognition approach, as described in the previous subsection, the prototypes are distributed at regular intervals. So the objective function is not optimized for each image. However, prototypes are independent of the object natural deformations. We have detailed in [32] the interest of such reference on applications of complex object recognition; “ by defining the set of fuzzy sector images. The interest of such a technique is described in the medical application.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

133

3.2. Classification stage: fuzzy images and membership functions In this subsection, we formulate various fuzzy image models following the nature of prototypes estimated in the learning phase.

3.2.1. Fuzzy images computed using chromatic prototypes Let c be the number of classes. We show how we obtain a set of fuzzy images (ICi )i  J c, where each image characterizes a chromatic property. Let P(k,l) be an image pixel of value xX with X= U is the set of color values. Let 6Ci be the ith cluster center estimated previously. The segmentation of an image I is processed with an iteration mI C (x) = i

(1/ x −6Ci 2)1/(m − 1)

(2)

%cj = 1(1/ x −6Cj 2)1/(m − 1)

where mI C ={mI Ci (x), x  X, 15 i5 c} is the real matrix which defines a fuzzy c-partition of X, where mI Ci (x)  [0, 1] denotes the grade of membership of x in the ith fuzzy cluster. The fuzzy images (ICi )i  J c are defined as: (ICi )i  J c mI C (x(k,l)), i

i  Jc

Fig. 5 shows a fuzzy segmentation applied on a color Doppler image. In order to display the segmented images, each cluster is assigned an unique color. In addition, each pixel vector is (crisply) assigned to the cluster where it has its maximum membership. The pixel is then given the color corresponding to the cluster to which it was assigned.

Fig. 5. Fuzzy segmentation based on color property. In Doppler color imaging, color represents the blood velocity (a) Doppler color image (b) c =3 (‘positive velocities’ =red, ‘negative velocities’ = blue, ‘very low velocities’ = green). The family of centers has been obtained after application of FcM on several images. In order to display the segmented images, each cluster is assigned an unique color. Each pixel vector is (crisply) assigned to the cluster where it has its maximum membership. The pixel is then given the color corresponding to the cluster to which it was assigned.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

134

3.2.2. Fuzzy images computed using spatial prototypes Let r be the number of regions. The fuzzy images (IRi )i  Jr can be obtained from three different ways: 3.2.2.1. Case one: structured partition. Let X=V ¦ Z 2. the universe of the coordinates (k,l) x of IB on which FcM has been applied (cf. previous section). Let VRi be the center of the ith fuzzy cluster estimated previously. The segmentation of an image IB is processed with an iteration: mI R (x) = i

(1/d(x,i)2)1/(m − 1)

(3)

%rj = 1(1/d(x, j)2)1/(m − 1)

where d(x,i ) is the distance of the pixel P(x) to the center VRi. Let mI Ri = {mIRi (x), x X, 15i5 r}: the real matrix which defines a fuzzy r-partition of X, where mI Ri (x) [0, 1] denotes the grade of membership of P(x) in the ith fuzzy cluster. The set of all fuzzy images based on shape properties is therefore written as follows: (IRi )i  Jr mI R (x),

i Jr

i

Example: diagnostic approach Suppose that models constitute separate classes in the universe of shapes. At this level, the interpretation consists of searching the difference between the shapes to be analysed and each model. Local differences (i.e. deformations) could be measured by using the membership functions defined from the structured partition (for example: difference between membership functions of the model and shape to be diagnosed). The models correspond to the ‘normal’ patient or known pathologies. For example, this comparison could be applied on a diastole or systole image in order to analyse local deformations that led to the myocardial infarction. We show on Fig. 6a a structured partition obtained on a systole image. Fig. 6b shows the location of local deformations obtained by computing the difference between the membership functions of the shape to be analysed and a model of a healthy person. This analysis is applied on a diastole image. The model is not shown on the figure. Local deformations are very well detected [41].

3.2.2.2. Case two: a robust reference to natural distortions. In this subsection, we define a fuzzy mesh adapted to the object shape within an image IB. In that way. we use distance transformation which combines numerical features and geometrical shapes. Distance transformation [47] is a processing which transforms a binary image into a gray level image, where the gray level of each pixel represents the location of the pixel in the object (i.e. every pixel is given a gray, level value, gd, equal to the shortest distance from the pixel to the boundary). This fuzzy mesh is calculated as follows: 1. the distance image is determined on the object within the binary image IB.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

135

Fig. 6. Diastole and systole images with their structured partitions (r =30). (a) A systole image segmented using the centers computed on the model. (b) Figure shows the location of local deformations obtained by computing the difference between the membership functions of the shape to be analysed and a model of a healthy person.

2. The centers are located at regular intervals on gray level axis (pattern recognition approach). The segmentation of each image is processed with an iteration that satisfies: mI R (gd) = i

(1/ gd −6Ri 2)1/(m − 1)

(4)

%rj = 1(1/ gd −6Rj 2)1/(m − 1)

The measures of membership computed by, using the previous formulation define a fuzzy mesh of object. The real matrix mIR ={mI Ri, 05gd 5 L− 1, 15i 5r} defines a fuzzy r-partition of U, where mI Ri  [0,1] denotes the grade of membership of gd in the ith fuzzy cluster and L is the maximum distance level. Example: pattern recognition approach We show in [32] that this reference is robust to natural distortions. This method is applied in two problems: automatic fish sorting and face recognition, where objects are complex for being real-world ones. The reference, computed from the Chanfrein distance on the binary image, is given in Fig. 7. The fuzzy images characterize the fish shape and are independent of natural deformations. Fig. 7e shows the partition

136

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

of the object into five classes, representative of the shape. The cluster are regularly distributed: class 1, the closed to the boundary, merges into the edge. Class 5 merges into the object center. This allows us to determine a fuzzy mesh where interest areas are defined. These areas are independent of the natural object deformations and represent the depth of the shape. Descriptors of the shape (correlation matrix, surfaces corresponding to these areas), invariant to various changes in the image, are extracted. They can be stored in a database instead of the image itself. Fish sorting points out the robustness of our algorithm according to the natural deformation of the object [32]. The face recognition problem brings the same conclusion on the robustness according to the face variation (smile, grimace ...).

3.2.2.3. Case three: fuzzy sectors. Now each fuzzy image represents a fuzzy sector. A sector is defined by a center, an angle and a direction. Initially, the centers (6i )i = 1,2, ..., r are obtained by the technique of the structured partition. A significant number r of centers gives a better resolution at the expense of a more significant complexity (this implies local precise but no accurate information). For each center 6i, we seek the closest center cj. Let Vij denotes the line connecting 6i and 6j. Let the measurement of membership of an image-point P(k,l) z to a fuzzy sector Sij be [24]: Á 1 if à mSi, j (z) = Í(p2 − f ijz )/p2(1 − a) if à 0 if Ä

f ijz 5 ap/2 ap/2 B f ijz 5 p/2 f ijz \ p/2

(5)

Fig. 7. In order to display the segmented images, each cluster is assigned a unique color. In addition, each pixel vector is (crisply) assigned to the cluster where it has its maximum membership. The pixel is then given the color corresponding to the cluster to which it was assigned. (a) Fish example; (b) distance image of the fish; (c) fuzzy memberships permit the partitioning of points into four regions on the basis of measurements equal to the shortest distance from points to the boundary; (d) another fish with natural deformations and (e) fuzzy mesh with c= 5.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

137

where f ijz represents the angle between Viz and Vij, and a is a constant controlling the fuzzy limits of the sector (05aB 1). Viz denotes the line connecting 6i and z. The interest of such a technique is described in the medical application (cf. Section 4).

3.3. Fusion of fuzzy images: combining color and shape information Spatial relationships exhibited among regions in an image play an important role in the interpretation of a scene. While humans have an innate ability to recognize spatial relations, it has been difficult to produce algorithms to model these relationships. In high level image analyse are dealing with object features, such as shape, color or other characteristics, e.g. velocity. Such features in a scene are not only attributed by properties about the features themselves but also related to each other by cross-relations. In other words, an object is represented by features constrained by the properties and relations. So, the combination of information (i.e. the joint use of heterogeneous sources of information) making it possible to provide a more exploitable description of the image [4,30]. Bloch [4] presented an exhaustive study related to fusion in the field of the imagery. She gave a classification of the various approaches of fusion of information based on the digital techniques used in image processing. The objective of this section is to combine color (or velocity) and shape information on the object represented by fuzzy images previously defined. Thus our aim is to merge the (r+ c) images represented by, fuzzy sets where r(c) represents the number of areas (classes) resulting from modeling according to properties of shape (color). We propose to use measurements so as to evaluate the similarity between the fuzzy subsets, bringing to an interpretation of fusion: “ The correlation between two properties (color and shape) gives a measure of the relation between connected zones (FcM applied on shape properties) and homogeneous zones obtained with color properties. In order to combine two fuzzy models. the correlation matrix FC(I) is defined as follows: FC(I) = {FCmI , Ci

1 5i 5c,

15 j5r}

where mI Ci characterizes color properties, and mI Rj shape properties of the object. Pal defines in [37] the correlation between two membership functions: 4d 2(m1,m2) mFC12 =1 − (6) Z1 +Z2 where d is Kauffman’s distance between two fuzzy sets m1 and m2 [22], X1 = z {2m1(z) −1}2 and X2 =z {2m2(z)− 1}2. Fuzzy correlation between two membership functions is an efficient technique to represent image features and object shapes, in relation with photometry and geometry information. Example: pattern recognition approach We have tested this operator on real fish images in three color planes (HSI). We obtain a recognition rate of 91%. This improves the classical techniques which had obtained: 83% [29].

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

138

We have also applied the correlation operator to color images of faces issued from Essex University database. The results obtained were very satisfactory; it gives a relation between homogeneous structures (hair, faces, background) and their spatial situation within the image area without needing a prior knowledge about these structures (see [28] for methods with a prior knowledge). “ T-norm and T-conorm operators are applied to (r+ c) fuzzy images. This modeling allows us to represent an object with adjacency subclasses (classregions). So, we combine ICi with all spatial regions (IRi ). Thereby, the over-segmentation problems arise when using this technique. In fact, each class ICi is a homogeneous zone but not necessarily connected. In the other order. if two fuzzy sets have the same color property and if they are adjacent2, we have to consider them as one fuzzy set (homogenous class). So, we avoid the over-segment at ion by merging the fuzzy adjacency classes. Therefore the fusion is made as follows: 1. First, we compute the fuzzy intersection between (ICi )i  J c with all (IRj )j  J r. The result is the fuzzy set mv i given by j

mv i(z) =mI C (z) ‚mI R (z) j

i

(7)

j

2. Then we merge two sets mv i and mv i which have the same color property Ci j1 j2 and are adjacent (i.e. aj1j2 \g where g is an appropriate threshold). The result is the class mV i : j 1j 2

mV i

(z) =mv i (z)–mv i (z)

j 1j 2

j1

j2

(8)

where ‚ and – are, respectively a T-norm and a T-conorm [6]. These operators have been tested in the area of blood flow analysis from Doppler images. Particular interests, for qualitative and descriptive study of abnormal and evoluting blood flow, are shown in the next section. In short, the result of fusion thus appears as areas of spatio-chromatics and/or the matrixes of correlation and/or inclusion and/or resemblance, characterizing the chromatics and topological properties between pixels. 4. Medical application: color Doppler imaging In this section. we present a method founded on the theory of fuzzy sets for the analysis of blood flow velocities starting from the color Doppler echography images [41]. These images, obtained from a multiplanar transducer, define a series of scan planes representing a conical volume (cf. Section 1). The entire flow region of interest must be inside the scanning planes. We use a specific learning approach to define centers or prototypes characterizing velocity and spatial regions. This learning phase is applied on the set of exploration scan plans. The used adjacency coefficient aij, between region i and region j, is defined by Demko in [14]. This term represents a fuzzy relation of proximity between region i and region j. It is in the interval [0, 1] and the greater it is, the nearer are the two regions. 2

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

139

Also, we perform two segmentations on the Doppler image: one based on the velocity prototypes, and another one using fuzzy sectors based on the spatial prototypes (they define the vertex of the fuzzy sectors). The interest of these two segmentations is double. On the one hand, it enables us to correct the field of velocities, a phenomenon due to the physics of obtaining the echo-Doppler images, and on the other hand to analyze the form and intensity of the velocity flow. The results of our tests are carried out on images acquired on a hydraulic bench, which makes it possible to validate our method on known fields of velocity. The transducer is positioned in the longitudinal axis of the flow in order to obtain the best flow-ultrasound alignment.

4.1. Temporal correction A 2D Doppler color grabbing requires a time of about 100–200 ms, which is relatively long compared to the modifications of the blood flow rate. So the first velocities are not calculated at the same moment as the last [8]. The image cannot, thus be regarded as a true snapshot but rather as a juxtaposition of information depending on the position of the scan column browsing time. A precise quantitative study of the velocity profiles cannot be considered with such an image. A device makes it possible to build an image of the instantaneous velocities at one precise moment of the cardiac cycle. It requires the acquisition of a series of shifted images at multiple intervals (cf. Fig. 8). The columns of the images thus acquired are restructured to form an isotemporal image compared to the electrocardiogram of the patient. Fig. 9 shows the complete diagram of 2D Doppler image isotemporal acquisition [10]3.

4.2. Aliasing phenomenon correction Aliasing occurs when the sampling rate is lower than the double of the analyzed Doppler frequency. It is a limit in the determination of velocities slightly exceeding the Nyquist limit. It makes the analysis of the flow complex. The theorem of Shannon not being respected, there is no theoretical means to find the real sample velocity allocated. Even so, the knowledge of the phenomenon allows us to pose the three next hypotheses: “ the velocity is a bidimensional monotonous function in a defined flow; “ the limit of Nyquist is determined by the parameters of the scanner; “ when the variance of the distribution of velocities is important, there is incoherence of the average velocity. We refuse therefore to correct the region so defined. We only indicate the place of the turbulence on the image. The stenoses and the shunts are the causes. 3

Actually used at the cardiology service of CHRU Poitiers.

140

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

Fig. 8. Principle of the restitution of an isotemporal image at one moment Ti : T is the basic time of the synchronization given by the patient’s electrocardiogram (ECG), o is time necessary to the acquisition of a column and d presents an allowed time. Acquisitions are carried out during the low part of breathing; if the acquisition of the first column of n isotemporal image occurs at moment (Ti =T +d) then the following columns are obtained for a release of acquisition at the moment (Ti =T +d − k*o) with k = 1, 2,..., (M − 1) where M is the number of lateral beams.

A method of prediction-checking has been proposed by Coisne et al. [10], in order to perform spatial corrections by studying velocity values along each scan column. Three types of processing have been used: “ the first concerns the anatomical structure detection realized by the analysis of edges of the image; “ the second requires an algorithm of prediction applied on each column of the image, which allows us to determine values of velocities not calculated; “ finally, the third processing uses a classic interpolation algorithm correcting ill-determined values. The used algorithms necessitate furthermore the correct zone detection of turbulence. If, on a same line two turbulence areas are detected, the analysis of the intermediate values allows us to find a turbulent flow structure compatible with the physiology. An example of correction is presented on Fig. 10. The disadvantage of this technique is the generation of false corrections on edges of the flow, due to the utilization of a solely local prediction.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

141

4.2.1. Correction of the field of 6elocities by fuzzy modeling The method of reasoning follows three steps: “ using a specific learning approach to define centers or prototypes (i.e. membership functions) characterizing velocity and spatial regions. Presently two methods of partitioning are introduced:  Clustering carried out on 6elocity information. The learning approach is applied on the set of exploration scan planes denoted C= {I(u)}, u= 0, ...,

Fig. 9. System of Doppler image acquisition. On the CFM 750, the number of sampling volumes by line is 128. The pressure signal and the patient’s ECG signal are supervised by a monitor which transmits them to a PC. The PC orders the isotemporal scanning (acquisition). The scanner returns the scanned images to the PC.

Fig. 10. Correction of the field of velocities by the method of prediction-checking.

142

“

“

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

180. Clustering the data set C consists of dividing C into c subsets or clusters using FcM algorithm on the velocity information with c= 5 (tests, carried out on perfectly known velocity flows, showed that the best choice proved to be five clusters). For example, the centers of classes, found from a learning set of 35 images (35 exploration scan planes) are V ={0.973809, 0.440391, − 0.974812, − 0.444812, 0.010391}. Following the location of the center in the range of the velocities, the associated class will be characterized by the following type of velocities: significant positive velocity (F+), low positive velocity ( f + ), significant negative velocity (F− ), low negative velocity ( f− ) and very low positive or negative velocity (N). So, the centers of the classes are adapted to each flow. 2  Clustering with respect to fuzzy sectors. Let X= V¦ Z , the universe of the coordinates (k,l) x of I. We applied FcM algorithm on X. At the convergence, we obtain centers of clusters regularly distributed on the image I. These centers are used to define the fuzzy sectors (Si )i  Jr, and are adapted to each exploration scan plan. Processing a classification stage according to the found centers, in order to describe the set of features (velocity and spatial area) by fuzzy images. One iteration of FcM is used to generate fuzzy partitions from cluster centers found previously. The segmentation maps the image into an easy interpretable description. One can compute from this latter some features as areas, perimeters, compactness, orientation .... These features will allow us later to better analyse the scene in the image. The proposed system is capable of handling incomplete and other imprecise information both in the learning and processing phases. The basic idea of the method is to obtain some overlapping classes depending on the relative positions of the velocity classes as reflected by their training samples. Therefore, the feature space is automatically divided into a few subspaces using the training samples. This step is described more particularly in Sections 4.2.2 and 4.2.3. Correcting the inconsistent velocities. A knowledge-based system detects inconsistent velocities in relation to region properties. If pixel velocity is an inconsistent value, then the pixel value is changed. In this approach. the choice of the number of centers may be problematic because it represents the precision of the segmentation process: the higher the number of regions, the more local the flow analysis. The number of centers permits the adaptation of the level of analysis to the complexity of the flow (local, regional).

4.2.2. Fuzzy segmentation according to 6elocity prototypes The segmentation of the Doppler pixels into fuzzy classes is carried out by the application of FcM (with one iteration) on the velocity information by taking as classes the collection of velocities defined above. Fig. 11 shows the segmentation according to velocity prototypes.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

143

Fig. 11. Segmentation of the Doppler pixels carried out on velocity information with the following number of classes: (a) c =3 (F + = red, F − =blue, N =green) (b) c =5 (F + =red, f + =yellow, F − =purple, f =blue, N = green). The decomposition of velocities into five classes seems a good compromise between precision and complexity. In order to display the segmented images, each cluster is assigned an unique color. In addition, each pixel vector is (crisply) assigned to the cluster where it has its maximum membership. The pixel is then given the color corresponding to the cluster to which it was assigned.

4.2.3. Fuzzy segmentation according to fuzzy sectors The global objective is to develop a procedure to quantify and study the development of blood flow within different heart cavities and the left ventricle in particular. Properties between adjacency flow regions constitute an important part of features in the flow interpretation. Here, we consider the problem of describing such relationships from a preliminary segmentation in fuzzy regions. We have presented in Section 3 three methods that can be used to characterize spatial relationship of object regions in a digital image. The aim is to propose an efficient method for partition of image into fuzzy regions where the size of constructed regions is variable. The choice of the partition in fuzzy sectors is linked to the representation of the geometry of the flow and the function of velocities. By applying the suggested partition the analysis of flow is considered as possible. Let V = (6i )i  J r be the family of centers of clusters (arranged in ascending) found by applying FcM algorithm on the universe of the coordinates of I. Because of the angle of attack of the Doppler ultrasound beam, velocities in a horizontal direction cannot be given. So, it is necessary to take into account the privileged direction

144

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

(top“ bottom) (Fig. 12). We define the fuzzy sector S ti “ b characterizing this direction from the center 6i : suppose we have two points 6i and z. Let Viz denote the line connecting 6i and z, and let F iz be the angle between Viz and vertical line. The membership function for vertical sector S ti “ b (cf. Fig. 14b) is given by: Á 1 if f zz 5 ap/2 Ãp p i mS t “ b(z) = Í(2 − f z )/2(1 −a) if ap/2B f iz 5 p/2 (9) i à i 0 if f z \ p/2 Ä Moreover, for each center 6i, we define two other fuzzy sectors Sij and Sik (cf. Fig. 13). We seek the two closest centers 6j and 6k in the direction (top“ bottom) as follows:

Fig. 12. Let X= V¦ Z 2, the universe of the coordinates (k,l) x of I. We apply FcM algorithm on X. At the convergence, we obtain centers of clusters, 6i, regularly distributed on image I. These centers are useful in order to define the fuzzy sectors (Si ) and are adapted to each exploration scan plan. We present the case of 15 centers.

Fig. 13. Example of a circuit in an image. For the center 62, the two closest centers are 64 and 65 according to the direction top “bottom (V3 is rejected because of the angle of attack of the Doppler ultrasound beam).

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

145

Fig. 14. Shows fuzzy sectors with a = 0.25. The levels of gray characterize the values of membership. (a) Fuzzy sector, S8.9 of axis V8.9 in the general case; (b) fuzzy sector S t8“ b of vertical axis characterizing the “b top“ bottom direction of flow; (c) fuzzy sector S t8.9 of axis V8.9 by taking into account the orientations “b of the flow; (d) result of the fuzzy sector S t8.10 of axis V8.10. Let us note membership measurements in this sector are more significant than in (c).

6j  V:minl \ i

d(6i,6l ) uS t “ b(6l ) i

6k  V:minl \ i,l " j

d(6i,6l ) uS t “ b(6l ) i

The final membership degrees to which the spatial relation (top“ bottom) holds according to the fuzzy sectors of axis Vij and Vik is obtained as follows: b (z) =min(mSij (z), mS t “ b(z)) m tS“ ij i

b m tS“ (z) =min(mSik (z), mS t “ b(z)) ik i

We use Eq. (5) to define the membership of an image-point z to the fuzzy sectors Sik and Sij. The radius of these latter are d(6i,6j ) and d(6i,6k ), respectively. S tik“ b and S tij“ b are treated as fuzzy sets which capture ‘the spatial relation’ top “ bottom. The “b “b and S t8.10 are precised on Fig. 14. fuzzy sectors S t8.9 t“b Thus, the sectors S ij (to facilitate the writing, we adopt Si to indicate the sector S tij“ b or S tik“ b) enable us to define a circuit in the image. This one integrates several possible paths being able to be followed by the blood flow (cf. Fig. 15a).

4.2.4. Distribution of 6elocities of flow in the fuzzy sectors Our aim is to merge the (r + c) images represented by fuzzy sets where r(c) represents the number of sectors (classes) derived from modeling according to properties of direction (velocities).

146

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

Operators of T-norms between the fuzzy subsets are used allowing the distribution of velocities in the sectors to be defined. These operators apply to the (r+ c) images. The principle of this technique is to take a velocity class Ci, and to project it on all the sectors (Sj )j  Jr : the area v ij, intersection of the velocity class Ci with the Sj sector is obtained by: v ij =T(mCi,mSj )

(10)

where T corresponds to T-norm. We chose to use the min operator. The flow image is characterized by its principal direction and the proposed method used a graph environment to connect elementary regions. The direction of the principal flow is defined as follows: Let (6i )i  J r be the set of the vertex of the sectors. For the vertex 6i, the direction of the flow in the sector Si is defined by the path Vik where Vk = minl \ i d(6l,bi ), 6k V and bi is the geometrical barycenter of all the image-points of the sector Si weighted by v ij(z), the most representative of the sector Si : bi :maxj % zv ij(z) zI

This analysis enables us to find the direction of the principal flow (i.e. the characteristic path followed by the flow (cf. Fig. 15b)) (61, …6i, …6k ) from the first vertex to the last found. Fig. 16 shows the distribution of velocities on the path taken by the flow. It also presents the order of the positions of the velocity barycenters in the left–rightdirection.

Fig. 15. (a) Example of a complete circuit; (b) the characteristic path (61, …, 6i, …, 6k ) followed by the flow.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

147

Fig. 16. (a) Distribution of the centers of the velocity classes along the path followed by the flow. The sector S1 contains only null velocities. The sector S2 contains the five velocity classes. The barycenters of the velocities are located in S2 from left to right according to the following order (F+ , f +, N, f −, F −). The order of the spatial position of the five barycenters in the sector S5 is ( f− , F−, F+, f +, N). (b) location of fuzzy sector centers.

4.2.5. Detection of spectral aliasing The detection of spectral aliasing is obtained after the classification of velocities found on a path (61, …, 6i, …, 6k ). Velocities are perceived in two manners: either they are coherent, or they are inconsistent. Definition: inconsistent velocities. A 6elocity is considered inconsistent compared to the direction of a selected principal flow if and only if its barycenter has at least one of its neighbors whose 6elocity is a different sign. Fig. 17 shows the inconsistency of negative velocities being in the sectors S5, S7 and S9 compared to this direction of flow. Indeed. in the sector S5, we have the topological center of class F + which is a direct neighbor of the topological center of the class F − . That is physically impossible owing to the fact that velocities follow a continuous function.

4.2.6. Correction of the field of 6elocities The correction of these inconsistent areas is the final stage. I(k,l) z the color code of image I at pixel P(k,l). denotes the velocity of red blood corpuscules located in a sampling volume VIIa,M (r,f) at a distance r of the sensor and in one direction f, compared to a vertical reference. Suppose that z is an inconsistent velocity, z will be corrected in that z% by the following relation (cf. Fig. 18): z% =z +2× k ×VNyquist

148

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

where VNyquist is the velocity (i.e. color code) corresponding to the frequency of Nyquist. k  Z indicates the degree of spectral analysis. The value of k is deduced from the order of the velocities as shown below. The taking into account of the neighbors is carried out at the same time on the horizontal and vertical directions. We specify that when a spectral aliasing is detected, the principal direction of flow makes it possible to raise ambiguity on the choice of velocities to be corrected. A disadvantage of our method is that it can be rather complex to implement for the detection of spectral aliasing of a high order. We describe some nonexhaustive rules here corresponding to configurations of velocities. Other more complex configurations can be modelled in an equivalent way: 1. Highlighting of spectral aliasing of 1st order.

Fig. 17. The distribution of velocities on a path makes it possible to detect inconsistent velocities. For example, the sector S7 contains an inconsistent velocity (F −) because its barycenter has a neighboring barycenter (on its right) of different sign (F+ ).

Fig. 18. Correction of inconsistent velocities on a path.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

149

Fig. 19. (a) An image with spectral aliasing resulting from a sequence acquired on a hydraulic bench. The transducer is positioned in the longitudinal axis of the flow in order to obtain the best flow-ultrasound alignment. (b) Detection of inconsistent velocities using our fuzzy operators. (c) Corrected image by applying the rules defining in Section 4.2.6. (d) Result of the method of prediction-checking. We notice wrong corrections on the edge of the image. 

Simple sequence: if the order of velocities is horizontally (F+ , F− ) or (F− , F + ) (vertically

   

F+ F− or ) then F− is corrected and the aliasing is F− F+

of degree 1 (k= 1, i.e. z%= z+ 2× VNyquist);  Complex sequence: if the sequence of velocities is (F+ , F− , f− , f+ , F+ ) then F − , f − , f + and F+ are corrected with k= 1 (i.e. z%= z+ 2× VNyquist). 2. Highlighting of spectral aliasing of order 2. Example in the case of a complex sequence: if the order of velocity is (F+, F−, f− , f+ , F+ , F− ) then F− , f − , f + and F + are corrected with k=1 and then F− (the last one) is corrected with k =2 (aliasing of degree 2, i.e. z%= z + 4× VNyquist). The advantage of this method is the detection of inconsistent velocities in a global way. It allows, indeed, an interpretation of the flow as a whole, whereas the algorithm of prediction-checking proposed in [9] processes velocities locally (on the level pixel), which sometimes generates a wrong correction of velocities.

150

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

Fig. 19a shows an image with spectral aliasing, resulting from a sequence acquired on a hydraulic bench. The transducer is positioned in the longitudinal axis of the flow in order to obtain the best flow-ultrasound alignment. The jet is imaged by color Doppler flow mapping with a transducer facing the flow with the ultrasound beam path parallel to the jet axis. This flow presents an aliasing phenomenon due to the limitation of Doppler system in the maximum measurable velocity. Fig. 19b shows the detection of spectral aliasing using our fuzzy operators and Fig. 19c presents the corrected image. The interest of our algorithm is double: “ on the one hand, it analyses in a global manner the flow and allows thus incoherences that appear only in a global context to be detected: “ on the other hand, the fuzzy segmentation takes into account the local imprecisions on the velocity evaluation. So our algorithm has allowed a quantitative analysis. Evaluations of the corrected images were visual and done separately by two doctors. Fig. 19d presents the result obtained with a prediction-checking method defined in [9]. The disadvantage of this technique is the generation of false corrections on the edge of the flow, due to the utilization of a solely local prediction. We have attempted to verify our velocity vector correction technique by using a hydraulic bench capable of producing pulsatile velocity profiles. Poiseuille and pulsatile flow have been generated. The results of the corrections have been compared with the theoretical flow profiles.

4.3. Prospect: cartography of flow The fuzzy modeling applied above makes it possible to lead to a cartography of the image representing the distribution of the velocities (cf. Fig. 20). This results in analyzing the form and the intensity velocities of blood flow represented in one sequence of color Doppler images. The cartography characterizes each sector according to the properties of flow (zones of turbulence, laminar flow, significant velocities, etc.) from the attributes extracted on the sector (surfaces, barycenters, matrices of variances). It will then be possible to obtain images of simplified references characterizing particular pathologies making it possible to specify certain diagnoses. Table 1 shows the result of the distribution of the velocity F + in the fuzzy sectors of the polar section of an anatomic sector R. Let v ij be the intersection region of the velocity class C, with the Sj sector (cf. Section 4.2.4). The surface A(v Fj + ) of this region is defined as follows: A(v Fj + ) = % v Fj + (z) zI

It characterizes the presence or not of a particular class of the velocity. The variance makes it possible to detect the zones of turbulence that is essential for a robust estimation of the allased areas. A significant variance (for example in the sector S1 in Table 1 reveals a difficulty of precisely evaluating a velocity. The direction and the amplitude of the velocity vector are random in this section.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

151

5. Conclusion The contribution of fuzzy logic in the analysis of intracardiac blood flows facilitates the qualitative and descriptive study of abnormal flows as well as their temporal evolution on several examinations. The method that we propose makes it possible to synthesize the velocity information and their location, as well as the directions of principal flows in the form of a cartography. We are currently trying to define, from the 3D reconstruction of the blood flow (cf. Fig. 21), a 3D representation of the cartography in the form of a graph or of a synthetic 3D image. This cartography takes its true place when being associated to the anatomical image. We also think of extending this method to temporal information allowing a complete analysis of intracardiac blood flow throughout its cycle (systole, diastole). The prospect of this technique is in the development of medical criteria itself based on the image of cartography and associated with the development of velocity field analysis algorithms. This technique will also allow the follow-up of velocities on a set of unspecified sections as well as the determination of the areas of transverse iso-velocity. We think that this method will make it possible to carry out an assessment and an evolutionary follow-up of patients in determining a cartography of flows, of more or less high resolution (it depends on the number of classes chosen for the two fuzzy segmentations) of the examined scan plane. The choice of the angle of inclination of the scan plane is dictated by the principal direction of flow (central or offset direction).

Fig. 20. Fuzzy modeling makes it possible to synthesize the direction of flow (case of a laminar flow). (a) Result of the segmentation after correction (b) Simplification of the Doppler image: the synthesized image is generated by our algorithms.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

152

Table 1 The results of the distribution of the velocity F+ in the fuzzy sectors of the polar section of an anatomic sector R a Sector

Surface

X(

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17 —

0.000008 27.185017 39.032886 7.686763 45.270953 145.166679 238.789766 96.483518 192.725387 195.055439 312.310220 189.349940 93.817441 178.252349 278.876016 142.220128 168.187021 —

−54.895287 −44.137985 −40.643201 −30.050934 −27.250062 −21.644805 −18.358015 −19.358860 −16.808140 −8.391688 −5.312267 −2.546553 −4.512151 4.905526 6.732240 9.843898 9.792349 —

Y( 1.849232 −0.628210 −0.813650 −1.300715 2.327846 −1.293329 −0.720688 2.181646 0.807567 −1.308842 −0.884601 1.643792 3.504380 −2.734326 −0.397515 20892 −0.360289 —

Variance 0.971610 0.016250 0.011213 0.003149 0.006350 0.006359 0.005648 0.001968 0.002748 0.002057 0.002870 0.001516 0.000577 0.001394 0.001843 0.000778 0.000881 —

a The number of sectors is 23. X( and Y( represent the spatial coordinates of the barycenter of the velocity. These data allow the analysis of flow. The surface characterizes the presence or not of velocities. The variance makes it possible to detect the zones of turbulence.

Fig. 21. 3D reconstruction of the blood flow: two different sights of the isotemporal flow.

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

153

References [1] Abrarnczyk R, Pai ML. A nonmonotonic model based system for scene interpretation. Proc SPIE Int Soc Opt Eng 1990;1293:23–30. [2] Bezdek JC. Pattern Recognition with Fuzzy Objective Function Algorithms. NewYork: Plenum, 1981. [3] Binaghi E, Ventura AD, Rampini A, Schettini A. Fuzzy reasoning approach to similarity evaluation in image analysis. Int J Intell Syst 1993;8(7):749 – 69. [4] Bloch I, Fusion de donne´es, ensembles flous et morphologie mathe´matique en traitement d’images. Application a` l’imagerie me´dicale ce´re´brale et cardio-vasculaire multi-modalite´s. Habilitation a` diriger des recherches 007, E.N.S.T., Paris, France, April 1995. [5] Bombardier V, Jaulen MC, Bubel A, Bremont J. Cooperation of two fuzzy segmentation operators for digital substract angiograms analysis. Proc Fuzz IEEE 1997;2:1057 – 62. [6] Bouchon-Meunier B. La Logique Floue et ses Applications. Paris: Addison-Wesley, October 1995. [7] Carpenter GA, Grossberg S. A massively parallel architecture for a sel-organizing neural pattern recognition machine. Comp Vision Graphics Image Processing 1987;37:54 – 115. [8] Coisne D, Me´nard M. De´veloppement d’un mode`le de correction temporelle de l’imagerie doppler couleur par de´phasage de l’acquisition et reconstruction a posteriori. Socie´te´ Franc¸aise de Cardiologie, 23–30 May 1994. [9] Coisne D, Me´nard M, Allard D. A new global device for time correction of color doppler. 11th Symposium on Echocardiology, Rotterdam, Juin 1995. [10] Coisne D, Me´nard M, Allard D. Three dimensional reconstruction of color doppler imaging. An in vitro validation. In 11th Symposium on Echocardiology, Rotterdam, Juin 1995. [11] Cooper GF, Aliferis CY, Ambrosino R, et al. An evaluation of machine-learning methods for predicting pneumonia mortality. Artif Intell Med 1997;9:107 – 38. [12] Cullen PB, Hull JH, Sriliari SN. A constraint satisfaction approach to the resolution of uncertainty in image interpretation. Proc. Eighth Conf. Artificial Intelligence for Applications, 1992, pp 127 – 133. [13] Dave RN. Generalized fuzzy c-shells clustering and detection of circular and elliptical boundaries 1992;25:713–21. [14] Demko C, Zahzah EH. Image understanding using fuzzy isomorphism of fuzzy structures. In Proceedings of the International Joint Conference of the fourth IEEE International Conference on Fuzzy Systems and the Second International Fuzzy Engineering Symposium, Yokohama, Japan, March 1995, pp. 1665–1672. [15] Foley WD. Color Doppler Flow Imaging. Andover Medical Publishers, 1991. [16] Giarre´. Color Doppler: a novel method for the blood flow velocity vector reconstruction. PhD thesis, Ecole Polytechnique F6d6rale de Lausanne, 1996. [17] Gustafson DE, Kessel WC. Fuzzy clustering with a fuzzy covariance matrix. In Proc. IEEE CDC, San Diago, CA, January 10–12 1979, pp. 761 – 766. [18] Hadzicadic M, Hakenewerth A, Bohren B, et al. Concept formation vs logistic regression: predicting death in trauma patients., Artif. Intel. Med. Rotterdam, June 1996;8:493 – 504. [19] Havashi Y, Buckley JJ, Czogala E. Fuzzy neural network with fuzzy signals and weights. Int J Intell Syst 1993;8:527–37. [20] Fujioka H, Ishibuchi HR, Tanaka H. Neural networks that learn from fuzzy if – then rules. IEEE Trans Fuzzy Syst 1993;1:85–97. [21] Kammerer BR. Incorporating uncertainty in neural networks. Int J Patt Recog Artif Intell 1992;6(1):179–92. [22] Kaufmann A. Introduction a` la The´orie des Sous-Ensembles Flous. Paris: Masson, 1977. [23] Keller JM, Tahani H. Back propagation neural networks for fuzzy logic. Inf Sci 1992;62:205 – 21. [24] Keller JM, Wang X. Comparison of spatial relation definitions in computer vision. ISUMANAFIPS’95, College Park, MD, 1995, pp. 679 – 684. [25] Kim WY, Jens K, Poulsen K, Kim T, Staalsen N-H. A new doppler method for quantification of volumetric flow in vivo validation using color doppler. Am Coll Cardiol 1996;27:182 – 91. [26] Krishnapuram R, Keller J, Ma. Quantitative analysis of properties and spatial relations of fuzzy image regions. IEEE Trans. Fuzzy Syst. 1993. [27] Krishnapuram R. A possibilistic approach to clustering. IEEE Trans Fuzzy Syst 1993;1(2):98 – 110.

154

A. Shahin et al. / Artificial Intelligence in Medicine 19 (2000) 121–154

[28] Lee SY, Ham YK, Park R-H. Recognition of human faces using knowledge-based feature extraction and neuro-fuzzy algorithm. 1996; 29(11):1863 – 1876. [29] Loonis P. Contribution a` la minimisation de l’a priori en Reconnaissance des Formes. Conception d’un prototype de trieuse automatique de poissons par Vision Artificielle en milieu industriel. The`se de doctorat, Universite´ de La Rochelle, Laboratoire d’Informatique et d’Imagerie Industrielles. La Rochelle. France, January 1996. [30] Maitre H. Image fusion and decision in a context of multi-source images. In 9th Scandinavien Conf. on Image Analysis, Uppsala, Sweden, June 1995, pp. 139 – 153. [31] Menard M, Demko C, Loonis P. The fuzzy c + 2-means: solving the ambiguity reject in clustering. Pattern recognition (to be published). [32] Me´nard M, Loonis P, Shahin A. A priori minimization in pattern recognition: application to industrial fish sorting and face recognition by computer fusion. Fuzzy-IEEE’97, Barcelone, Espagne, July 1997. [33] Mitchell DG. Color doppler imaging: principles, limitations and artifacts. Radiology 1990;177:1 – 10. [34] Karayiannis NB. Generalized fuzzy k-means algorithms and their application in image compression. In SPIE Proceedings: Applications of Fuzzy Logic Technology II, vol. 2493, Orlando, FL, April 1995, pp. 206–217. [35] Karaylannis NB, Ravuri M. An integrated approach to fuzzy learning vector quantization and fuzzy c-means clustering. In: Dagli CH, et al., editors. ASME Press, vol. 4. New York, March 1997:247 – 52. [36] Ornoto R. Color Atlas of Real Time Tow Dimensional Echocardiography, 2nd edn. Philadelphia/ Tokyo: Lea & Febiger/Shindan to Chiryo Co Ltd, 1987:30. [37] Pal SK, Ghosh A. Image segmentation using fuzzy correlation. 1992;62:223 – 250. [38] Peronneau FA, Bournat JP, Bugon A, et al. Theoretical and pratical aspects of pulsed doppler flowmetry: real time application to the measure of instantaneous velocity profiles in vitro and in vivo. In: Cardiovascular Applications of Ultrasound. Amsterdam: Reneman, 1974:66 – 84. [39] Hoang Pluiong N, Ngoc Cuong T, Le Plui D. An approximate reasoning using fuzzy-bayesian inference for medical expert systems. In Proceedings of the Eighth International Fuzzy Systems Association World Congress, vol. 1, 1999, pp. 161 – 165. [40] Roberto V. Perceptual and conceptual representations in a geophysical image-understanding system. In Proc. 6th Int. Conf. Image Analysis and Processing. Progress in Image Analysis and Processing II, 1992, pp. 713–720. [41] Shahin A. Mode´lisation floue d’objets naturels en vision par ordinateur. The´se de doctorat, Universite´ de La Rochelle. Laboratoire d’Informatique et d’Imagerie Industrielle, La Rochelle, France, January 1998. [42] Sloth E, Houlind K, Oyre S, et al. Three-dimensional visualization of velocity profiles in the human main pulmonary artery using magnetic resonance phase velocity mapping. Am Heart J 1994;128:1130–8. [43] Sondergaard L, Thomsen C, et al. Mitral and aortic valvular flow: quantification with mr phase mapping. J Magn Reson Imaging 1992;2:295 – 302. [44] Soo V-W, Jan-Sing W, Wang S-P. Learning and discovery from a clinical database: an incremental concept formation approach. Artif Intell Med 1994;6:249 – 61. [45] Spain MG, et al. Quantitative assessment of mitral regurgitation by doppler color flow imaging: angiographic and hemodynamic correlations. J Am Coll Cardiol 1989;13:585 – 90. [46] Suli DY, Mersereau RM, Eisner RL, Pettigrew RI. Automatic boundary detection on cardiac magnetic resonance image sequences for four dimensional visualization of the left ventricle. IEEE Trans Pattern Anal Machine Intell 1990;1:149 – 54. [47] Thiel E. Les distances de chanfrein en analyse d’images: fondements et applications. The`se, Universite´ Joseph Fourier, Grenoblel, Grenoble, France, September 1994. [48] Vidal JL, Aguilar JM, Coronado JL, Zalama E. Multimodel real-world mapping and navigation system for autonomous mobile robots based on neural maps. Proc SPIE Int Soc Opt Eng 1992;1709(1):247–56. [49] Yang M-S. A survey of fuzzy clustering. Math. Comput. Modelling 1993;18(11):1 – 16. [50] Yoganathan AP, Cape EG, et al. Review of hydrodynamic principles for the cardiologist: applications to the study of blood flow and jets by imaging techniques. J Am Coll Cardiol 1988;12:1344 – 53. [51] Zadeh LA. Fuzzy sets. Info Control 1965;8:338 – 53.