- Email: [email protected]
A new class of similarity measures for robust image registration
COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 28, 1 7 6 - 1 8 4
(1984)
A New Class of Similarity Measures for Robust Image Registration A. VENOT, J. F. LEBRUCHEC,AND J. C. ROUCAYROL Formation de Recherche "Synth~se et Analyse d'images m~dicales,'" UER Cochin-Port Royal, Paris and INSERM U194, UER Piti~-Salpktrikre, Paris, France.
ReceivedJuly 1, 1983; revisedFebruary 21, 1984 The computer comparison of two images requires a registration step which is usually performed by optimizinga similarity measure with respect to the registration parameters. In this paper a new class of similaritymeasure is introduced, which is issued from the calculation of the number of sign changes in the scanned subtraction image. Using these similarity measures for the registration of dissimilar images leads to registration algorithms which are demonstrated to be far more robust than the methods currently in use (maximizationof the correlation coelticientor correlation/unction,minimizationof the sum of the absolutevaluesof the differences).Two medical applicationsof these image processing methods are presented: The registration of gamma ray images for change detectionpurpose and the alignmentof X ray digitized images (without and with iodine contrast) for improvingthe quality of the subtraction angiographic images. 9 1984 Academic Press, Inc.
I. INTRODUCTION The automatic comparison of two images for change detection is a standard problem in the disciplines of image processing and pattern recognition [1, 2, 3]. To form this comparison pixel by pixel, it is necessary to spatially register the images and thereby correct for relative translational shifts, magnification differences, rotational shifts, as well as geometrical and intensity distorsions [4, 5, 6, 7]. This registration step is usually performed by optimizing a similarity measure between the images according to the registration parameters. Among the most commonly described criteria, three particular ones are representative of those used in the algorithms of interest [5, 8]: First, the correlation coefficient (CC); second, the correlation function (CF); third, the sum of the absolute values of the differences (SAVD). These classical methods work correctly when the images to be compared are not too much dissimilar. When important changes exist between both images, a misregistration can happen. We propose in this paper a new class of similarity measures whose optimization with respect to the registration parameters permits robust registration procedures. The methodology we propose is derived from nonparametric statistical considerations (run tests) which are sometimes used in the field of mathematical modeling for the statistical test of a model [9, 10]. Two different cases are developed: first, the stochastic case when the noise level in the images is greater than the digitization precision and second, the deterministic case where the noise can be neglected in the images. In each case these new similarity measures are compared with classical ones (CC, CF, SAVD) using selected applications in the field of the medical imagery. 176 0734-189X/84 $3.00 Copyright ~ 1984 by Academic Press, Inc. All rights of reproduction in any form reserved.
A NEW CLASS OF SIMILARITY MEASURES
177
2. THE STOCHASTIC CASE
2.1. Construction of the Stochastic Similarity Measure Consider two images Fx(i, j) and F2(i, j) of the same object, which have been acquired in similar geometrical conditions (i, j = 1, 2 . . . . . n are the coordinates of the digitized image). Suppose that Fl(i , j) and F2(i, j) differ only because of the noise measurement which is assumed to be additive, zero mean, with a symmetric density function. The noise variance may vary from one point to another in the image. Let D(i, j) = Fl(i, j) - F2(i, j) be the subtraction image. The D(i, j) values exhibit random fluctuations; these fluctuations are zero mean and their density function is symmetric with a variance which is the sum of the variances of the corresponding pixels of Fx(i, j) and F2(i, j). Therefore the D(i, j) values are either positive or negative and there are many sign changes in the sequence of the D(i, j) values because there is a n equal probability of 0.5 for each sign + or - . Suppose that some part of image F2(i, j) is modified. The density function mean of every corresponding pixel of the subtraction image is no more null. There are fewer sign changes in the scanned subtraction image than when Fl(i, j) and F2(i, j) are similar. Therefore we propose a new similarity criterion between images Fl(i, j) and F2(i, j) defined as the number of sign changes in the sequence of the values of D(i, j) scanned line by line or column by column (e.g., in the sequence + - - - + there are 3 sign changes). This stochastic sign change criterion will be called the SSC criterion.
2.2. Statistical Properties of the SSC Criterion The density function of the number of sign changes in a sequence of + and - of equal probability has been calculated by statisticians [10].,In the case of a digitized image, there are a great number of signs + and - so that the asymptotic distribution of the number of signs + and - may be used. Therefore the SSC criterion distribution is normal with a mean and a variance, respectively, equal to N / 2 - 1 and NV/~-N [10]. This distribution permits one to derive a 95% confidence interval for the SSC criterion,
N/2 - 1 __ 1.961/N-/4.
(1)
Suppose two similar images which differ only because of the statistical fluctuations of the noise. The comparison of the SSC criterion value with the confidence interval (1) is a simple way to cheek that the noise characteristics are in agreement with the assumptions made for the construction of the SSC criterion. An example of application is given in Subsection 2.3. It must be emphasized that when the noise level decreases towards a null value, the SSC criterion value becomes null and this criterion cannot be considered further as a similarity criterion. A significant noise level is necessary for the validity of the SSC criterion.
2.3. Application to Gamma Ray Imaging The gamma ray images obtained with scintillation cameras are widely used in nuclear medicine. They are generally made of 105 to 3.105 events and digitized in 64 x 64 or 128 • 128 format. The statistical fluctuations are important because the number of counts in every pixel generally ranges between 30 and 250 counts so that
178
VENOT, LEBRUCHEC, AND ROUCAYROL
the variances range between 3r and ~ counts. For these values, the assumptions concerning the noise are valid and the SSC criterion can be used for the robust registration of similar and dissimilar images. There is no general case where it is possible to analytically demonstrate that the SSC criterion leads to robust registration procedures. Accordingly experimental results are presented first in the case of similar images, second for dramatically modified images. For every set of images, the
d
,..f
1.-
"~'Oo ~4 \".
I" ~9 9o, /'/i"
CF," :" /
. ,r-
%'9 9
.9
~
/
\ "-. /SAV0
\
9
iI Ii
\ -./
SSC\
.=,0
/'x ""
!\', I
r
"k
9
"
o~ .
i/% I/ dI 10
-8
-6
-4
-2
0
.
2
.
.
4
.
"\ ~B
5
8
to
pixel
FIG. 1. Bone scintigraphy images: (a) and (10) similar images only differing because of radioactive statistical fluctuations; (c) sign change image; (d) SSC, CC, CF, SAVD criterion values for translations from 0 to + 10 pixels of image Co) along the vertical axis.
179
A NEW CLASS OF SIMILARITY MEASURES
SSC, CC, CF, SAVD criterion values have been calculated for progressive translations of one of the images according to the vertical axis. The locations of the maxima (for SSC, CC, CF) and minima (for SAVD) of their values are determined and compared. Figs. la and lb show two bone scintigraphies of a patient, differing only because of statistical fluctuations (no displacement of the patient, same short acquisition time
,'2." ~" .....
d
"", "..~.~F 9
/
1I
~149149
ii/I ~.
/
,/SAVD..;
".
,.,,,
.2 .5
r,,,)
/
I
,/ ~,",~ "
,~%%%%
-10
-I]
-6
-4
-2
pixel
2
4
6
8
10
FIG. 2. Scintigraphic images of a liver phantom: (a) original image; (b) modified version of image (a); (c) sign change image; (d) SSC, CC, CF, SAVD criterion values for translations from 0 to + 10 pixels of image (b) along the vertical axis.
180
VENOT, LEBRUCHEC, AND ROUCAYROL
and acquisition parameters). The SSC criterion value corresponding to the figured window is 3002 for a total number of pixels of 6068. This number of sign changes is inside the confidence interval of 3033 __+78 derived from (1). Image lc is the sign change image (white corresponds to a sign change, black to none) and gives a pictorial illustration of the sign change random distribution in the subtraction image. Figure ld gives the values of the SCC, CC, CF, SAVD criterions for translations along the vertical axis varying from 0 to + 10 pixels. Their values have been normalized in the following way: for each criterion, one unit of ordinate axis corresponds to the difference between its maximum and minimum value in the - 10, +10 pixels interval. Thus, these normalized criteria can be compared. Every criterion is optimum for a null translation value which is the right value for every image in the following. Figures 2a and 2b show two scintigraphic images of a radioactive liver phantom. In the case of image 2b, lead materials have been set on the phantom to dramatically modify the image (there was no displacement of the phantom under the camera and the acquisition parameters remained constan0. Image 2c is the sign change image and demonstrates no sign change in the modified part of the image and random sign changes in the remaining similar parts. The SSC criterion value is 1993 for a window of 6068 pixels. In the same way as in Fig. 1, Fig. 2d gives the SSC, CC, CF, SAVD criterion values for translations along the vertical axis of image 2b. The SSC criterion is the only one to be maximum for a 0 translation value; CC is maximum for a translation of - 7 pixels; CF for - 6 pixels; and SAVD minimum for - 2 pixels. 3. THE DETERMINISTIC CASE
3.1. Construction of the Deterministic Similarity Measure Suppose now that the noise level in the two images Ft(i, j) and F2(i, j) is low compared with the precision of the digitization. Fl(i, j) and F2(i, j) are perfectly similar and the SSC criterion value is 0. Assume a periodic pattern is added on image F2(i , j) in the following way to obtain image F3(i, j),
F3(i,j)=F2(i,j)+ q
if i + j is even,
F3(i, j) = F2(i, j ) - q
if i + j is odd
(q is a small real or integer value). Now calculate the subtraction image D(i, j) = Fl(i, j ) - F3(i, j). The values of the D(i, j ) pixels are alternatively - q , + q, - q , + q . . . . . and the signs of D(i, j) pixels - , +, - , + . . . . . We define the deterministic sign change criterion (DSC criterion) as the number of sign changes in D(i, j). If there are N pixels in this image, the DSC criterion value is N-1.
3.2. Application to the Digitized X Ray Images Digital subtraction angiography systems are now widely used as additives to conventional angiographic suites in order to obtain angiographic images with small volumes of iodine contrast intravenously injected. In such images, the noise is low compared with the precision of the digitization, so that this physical device has properties which are in agreement with the assumptions made in the deterministic
A NEW CLASS OF SIMILARITY MEASURES
181
case [11]. Angiographic images are obtained by subtracting X ray images obtained sequentially without and with contrast medium. The digitization format is currently 512 • 512 with an 8-bit depth coding. One major problem of digital angiography concerns the movements of the patient which diminish the benefit of the subtraction procedure. If such a displacement occurs, the second image must be registered prior to subtraction. We propose to maximize the DSC criterion to perform this registration. As in the stochastic case, we will experimentally demonstrate that the DSC criterion leads to registration algorithms which are more robust than those provided by the optimization of the CC, CF, SAVD criterions. Figure 3 illustrates the case of similar images. Image 3a is a skull image obtained at the beginning of a cerebral angiography. Image 3b was obtained just after image 3a and is perfectly similar; a periodic pattern + 1, - 1 , + 1, - 1 . . . . , was added to this image for the calculation of this DSC criterion. Its value for progressive translations of image 3b along the vertical axis is given in Fig. 3c and compared to
% 9
9
\-" ;,.
-~, 9 "
~
t 9" o"
9"
-.
't,
OSC
,:
%
9 :
t,
:
t
"./ I
\
l
i~ 1
I f
l
,'"
".
"V.
1%-
%.
%.
I
%%,%
9"
". ".
"%,,%
-8
-6
-4
-2
"o "~,o% ".,o "x;
1
-10
~ ".
0
2
4
6
8
10
pixel
FIG. 3. SimilarX ray images: (a) X ray image of a skull; (b) same image with an added periodic pattern; (c) DSC, CC, CF, SAVD criterionvalues for translations from 0 to _ 10 pixels along the vertical axis (for the window drawn on image (a)).
182
VENOT, LEBRUCHEC, AND ROUCAYROL
the CC, CF, and SAVD values. The maximum DSC criterion value is 6067 for a selected window of 6068 pixels. DSC, CC, and SAVD criterions are optimum for a null translation and CF is maximum for a translation - 10 pixels. The robustness of the DSC method is illustrated by Fig. 4 which corresponds to the practical case of images collected during a carotid angiography without (image 4a) and with (4b) contrast medium intravenously injected. There was no movement
e
\
,.4\
\ ~176 ....."
9" "
0 "~
,.~"
:("/ " .'" / ~ "/ I \ .,'4 /
,..
"F~ I
/
\
',,/
/
/
j~t
/
~
I Ir
/sAvo
8 ~
OS~,~
"v
,
/
.
.A
I"
/
/ ~% ',
O. ~ -~o-a
"
-6
-4
-2
Ill
"~
,,
"" "--'tes 2
:4
"
~
:8
"-
lo
pixel FIG. 4. Dissimilar X ray images: (a) carotid angiography (without iodine contrast); (b) carotid angiography (with iodine contrast); (c) subtraction image for the window drawn on image (a); (d) sign chanse image in the selectedwindow; (e) DSC, CC, CF, SAVD criterionvalues for translations from0 to + 10 ppixels of image (b) along the vertical axis (in the window).
A NEW CLASS OF SIMILARITYMEASURES
183
of the patient and the subtraction permitted to get good quality angiographic images (image 4c). Image 4d is the sign change image and demonstrates no sign change in a large zone of the subtraction image (because of the contrast medium). Figure 4e permits the comparison of the SSC, CC, CF, and SAVD criterion values for progressive translations along the vertical axis. The DSC criterion is maximum (2227 sign changes in a 9900 pixel window) for a null translation. Same periodic pattern as for image 3b but contrast medium was added to image 4b for its calculations. CC and CF criteria are, respectively, maximum for - 1 and + 7 pixel translations and the SAVD criterion is minimum for a + 1 pixel translation. 4. DISCUSSION AND CONCLUSION A new class of similarity measures between images has been proposed. These criteria differ from the classical ones mainly because their values do not take into account the pixd values of the images. They depend in fact on the noise in the stochastic case or on the added pattern in the deterministic one. In the case of similar images (Figs. 1 and 3) the SSC and DSC criteria are demonstrated to be similarity measures as well as the CC and SAVD criteria. The CF criterion fails to be maximum for a null translation in the case of Fig. 3 images so that this criterion cannot be considered as a valid similarity measure for such images. The curves of Fig. 1 and Fig. 3 demonstrate a unimodal optimum for the SSC and DSC criteria as well as for the CC and SAVD criteria. This is convenient for the application of an optimization procedure to automatically calculate the registration parameters. The ability of the SSC and DSC criteria to provide robust registration procedures is demonstrated by the two examples of Fig. 2 and Fig. 4. When large disparities exist in the images, the maximum SSC and DSC criterion values are far lower than when the images are similar. Nevertheless the SSC and DSC criteria are the only ones that remain optimum for the true value (zero) of the translational shift; the CC and SAVD criteria are optimum for wrong values of the translation. Therefore we propose to use this new class of similarity measures for registration problems whenever important changes in the images can lead to misregistration with the classical criteria. This is typically what happens when digital angiography images are considered. For practical reasons, only translations were considered for the experimental demonstration of the robustness of this methodology but other geometric or gray scale transformations could have been convenient as well. Moreover this method was successfully used in nuclear medicine to automatically register very dissimilar scintigraphic images with respect to five parameters (two translational shifts, one rotation angle, two parameters of a linear transformation of the gray scale) [12, 13, 14, 15, 16]. In this context, the optimized SSC criterion value is useful to determine if the set of transformations which is proposed can lead to a correct registration. The optimized SSC criterion value must be in the range of the confidence interval (1) after registration of similar images. Furthermore, simple statistical tests can be implemented to determine if the adding of a new registration parameter brings a significant increase of the SSC criterion value. In both stochastic and deterministic cases, assuming that a correct registration of the images has been performed, the difference between the SSC or DSC criteria and their expected values (for similar images) gives a simple quantitation of the change dimensions which exist
184
VENOT, LEBRUCHEC, AND ROUCAYROL
in the images. This could be useful for the follow up of a pathology in the context of the medical imagery. A new look at the problem of image registration has been presented: The examples of applications are in the field of the medical imaging. Nevertheless there is no restriction to the application of these methods to the various other nonmedical imaging techniques. ACKNOWLEDGMENTS
The authors wish to express their gratitude to G. Frija who provided the X ray images. They thank C. Muse, J. L. Golmard, J. L. Steimer, and S. McWilliams for many useful comments and discussion. REFERENCES 1. L K. Aggarwal, L. S. Davis, and W. N. Martin, Correspondence processes in dynamic scene analysis, Proc. IEEE 69, 1981, 562-572. 2. R. L. LiUestrand, Techniques for change detection, IEEE Trans. Comput. C21, 1972, 654-659. 3. A. Rosenfeld, Image pattern recognition, Proc. IEEE 69, 1981, 596-605. 4. S. T. Barnard and W. B. Thompson, Disparity analysis of images, IEEE Trans. Pattern Anal. Mach. lntel. PAMI2, 1980, 333-340. 5. D. I. Barnea and H. F. Silverman, A class of algorithms for fast digital image registration, IEEE Trans. Comput. C21, 1972, 353-357. 6. H. Eghbali, K-S test for detecting changes from Landsat imagery data, IEEE Trans. Syst. Man Cybernet. SMCg, 1979, 17623. 7. W. K. Pratt, Correlation techniques of image registration, IEEE Trans. Aerosp. Electron. Syst. AESI0, 1974, 353-358. 8. M. Svedlow, C. D. McGillem, and P. E. Anuta, Image registration: Similarity measure and preprocessing method comparisons, IEEE Trans. Aerosp. Electron. Syst. AESI4, 1978, 141-149. 9. N. B. Draper and H. Smith, Applied Regression Analysis, Wiley, New York, 1966. 10. L D. Gibbons, Nonparametric Statistical Inference, McGraw Hill, New York, 1971. 11. G. Cohen, L: K. Wagner, and E. N. Rauschkolb, Evaluation of a digital subtraction angiography unit, Radiology (Easton, P) 144, 1981, 613-617. 12. A. Venot, J. F. Lebruchec, J. L. Golmard, and J. C, Roucayrol, An automated method for the normalization of scintigraphic images, J. NucL Med. 24, 1983, 529"531. 13. A. Venot, J. F. Lebruchec, J. L. Golmard, and J. C. Roucayrol, Digital methodsf0r change detection in scintigraphic images, J. Nuel. Med. 24, 1983, 67. 14. L. Pronzato, E. Walter, A. Venot, and J. F. Lebruchec, A general purpose global optimizer: Implementation and applications, Math. Comput. Simul., in press. 15. A. Venot, J. L. Golmard, J. F. Lebruehee, and J. C. Roucayrol, Digital methods for change detection in medical images, Proceedings, lnternat. Conf. Inform. Process. Med. Imaging, Nijhof, The Hague, 1984. 16. A. Venot, J. F. Lebruchec, E. Walter, L. Pronzato, V. Leclere, and I. C. Roucayrol, Automated methods for the registration of dissimilar images, Proceedings, First Image Syrups. CESTA, Image Process. Comput. Generated Images Appl., Biarritz, France, May 1984.
(1984)
A New Class of Similarity Measures for Robust Image Registration A. VENOT, J. F. LEBRUCHEC,AND J. C. ROUCAYROL Formation de Recherche "Synth~se et Analyse d'images m~dicales,'" UER Cochin-Port Royal, Paris and INSERM U194, UER Piti~-Salpktrikre, Paris, France.
ReceivedJuly 1, 1983; revisedFebruary 21, 1984 The computer comparison of two images requires a registration step which is usually performed by optimizinga similarity measure with respect to the registration parameters. In this paper a new class of similaritymeasure is introduced, which is issued from the calculation of the number of sign changes in the scanned subtraction image. Using these similarity measures for the registration of dissimilar images leads to registration algorithms which are demonstrated to be far more robust than the methods currently in use (maximizationof the correlation coelticientor correlation/unction,minimizationof the sum of the absolutevaluesof the differences).Two medical applicationsof these image processing methods are presented: The registration of gamma ray images for change detectionpurpose and the alignmentof X ray digitized images (without and with iodine contrast) for improvingthe quality of the subtraction angiographic images. 9 1984 Academic Press, Inc.
I. INTRODUCTION The automatic comparison of two images for change detection is a standard problem in the disciplines of image processing and pattern recognition [1, 2, 3]. To form this comparison pixel by pixel, it is necessary to spatially register the images and thereby correct for relative translational shifts, magnification differences, rotational shifts, as well as geometrical and intensity distorsions [4, 5, 6, 7]. This registration step is usually performed by optimizing a similarity measure between the images according to the registration parameters. Among the most commonly described criteria, three particular ones are representative of those used in the algorithms of interest [5, 8]: First, the correlation coefficient (CC); second, the correlation function (CF); third, the sum of the absolute values of the differences (SAVD). These classical methods work correctly when the images to be compared are not too much dissimilar. When important changes exist between both images, a misregistration can happen. We propose in this paper a new class of similarity measures whose optimization with respect to the registration parameters permits robust registration procedures. The methodology we propose is derived from nonparametric statistical considerations (run tests) which are sometimes used in the field of mathematical modeling for the statistical test of a model [9, 10]. Two different cases are developed: first, the stochastic case when the noise level in the images is greater than the digitization precision and second, the deterministic case where the noise can be neglected in the images. In each case these new similarity measures are compared with classical ones (CC, CF, SAVD) using selected applications in the field of the medical imagery. 176 0734-189X/84 $3.00 Copyright ~ 1984 by Academic Press, Inc. All rights of reproduction in any form reserved.
A NEW CLASS OF SIMILARITY MEASURES
177
2. THE STOCHASTIC CASE
2.1. Construction of the Stochastic Similarity Measure Consider two images Fx(i, j) and F2(i, j) of the same object, which have been acquired in similar geometrical conditions (i, j = 1, 2 . . . . . n are the coordinates of the digitized image). Suppose that Fl(i , j) and F2(i, j) differ only because of the noise measurement which is assumed to be additive, zero mean, with a symmetric density function. The noise variance may vary from one point to another in the image. Let D(i, j) = Fl(i, j) - F2(i, j) be the subtraction image. The D(i, j) values exhibit random fluctuations; these fluctuations are zero mean and their density function is symmetric with a variance which is the sum of the variances of the corresponding pixels of Fx(i, j) and F2(i, j). Therefore the D(i, j) values are either positive or negative and there are many sign changes in the sequence of the D(i, j) values because there is a n equal probability of 0.5 for each sign + or - . Suppose that some part of image F2(i, j) is modified. The density function mean of every corresponding pixel of the subtraction image is no more null. There are fewer sign changes in the scanned subtraction image than when Fl(i, j) and F2(i, j) are similar. Therefore we propose a new similarity criterion between images Fl(i, j) and F2(i, j) defined as the number of sign changes in the sequence of the values of D(i, j) scanned line by line or column by column (e.g., in the sequence + - - - + there are 3 sign changes). This stochastic sign change criterion will be called the SSC criterion.
2.2. Statistical Properties of the SSC Criterion The density function of the number of sign changes in a sequence of + and - of equal probability has been calculated by statisticians [10].,In the case of a digitized image, there are a great number of signs + and - so that the asymptotic distribution of the number of signs + and - may be used. Therefore the SSC criterion distribution is normal with a mean and a variance, respectively, equal to N / 2 - 1 and NV/~-N [10]. This distribution permits one to derive a 95% confidence interval for the SSC criterion,
N/2 - 1 __ 1.961/N-/4.
(1)
Suppose two similar images which differ only because of the statistical fluctuations of the noise. The comparison of the SSC criterion value with the confidence interval (1) is a simple way to cheek that the noise characteristics are in agreement with the assumptions made for the construction of the SSC criterion. An example of application is given in Subsection 2.3. It must be emphasized that when the noise level decreases towards a null value, the SSC criterion value becomes null and this criterion cannot be considered further as a similarity criterion. A significant noise level is necessary for the validity of the SSC criterion.
2.3. Application to Gamma Ray Imaging The gamma ray images obtained with scintillation cameras are widely used in nuclear medicine. They are generally made of 105 to 3.105 events and digitized in 64 x 64 or 128 • 128 format. The statistical fluctuations are important because the number of counts in every pixel generally ranges between 30 and 250 counts so that
178
VENOT, LEBRUCHEC, AND ROUCAYROL
the variances range between 3r and ~ counts. For these values, the assumptions concerning the noise are valid and the SSC criterion can be used for the robust registration of similar and dissimilar images. There is no general case where it is possible to analytically demonstrate that the SSC criterion leads to robust registration procedures. Accordingly experimental results are presented first in the case of similar images, second for dramatically modified images. For every set of images, the
d
,..f
1.-
"~'Oo ~4 \".
I" ~9 9o, /'/i"
CF," :" /
. ,r-
%'9 9
.9
~
/
\ "-. /SAV0
\
9
iI Ii
\ -./
SSC\
.=,0
/'x ""
!\', I
r
"k
9
"
o~ .
i/% I/ dI 10
-8
-6
-4
-2
0
.
2
.
.
4
.
"\ ~B
5
8
to
pixel
FIG. 1. Bone scintigraphy images: (a) and (10) similar images only differing because of radioactive statistical fluctuations; (c) sign change image; (d) SSC, CC, CF, SAVD criterion values for translations from 0 to + 10 pixels of image Co) along the vertical axis.
179
A NEW CLASS OF SIMILARITY MEASURES
SSC, CC, CF, SAVD criterion values have been calculated for progressive translations of one of the images according to the vertical axis. The locations of the maxima (for SSC, CC, CF) and minima (for SAVD) of their values are determined and compared. Figs. la and lb show two bone scintigraphies of a patient, differing only because of statistical fluctuations (no displacement of the patient, same short acquisition time
,'2." ~" .....
d
"", "..~.~F 9
/
1I
~149149
ii/I ~.
/
,/SAVD..;
".
,.,,,
.2 .5
r,,,)
/
I
,/ ~,",~ "
,~%%%%
-10
-I]
-6
-4
-2
pixel
2
4
6
8
10
FIG. 2. Scintigraphic images of a liver phantom: (a) original image; (b) modified version of image (a); (c) sign change image; (d) SSC, CC, CF, SAVD criterion values for translations from 0 to + 10 pixels of image (b) along the vertical axis.
180
VENOT, LEBRUCHEC, AND ROUCAYROL
and acquisition parameters). The SSC criterion value corresponding to the figured window is 3002 for a total number of pixels of 6068. This number of sign changes is inside the confidence interval of 3033 __+78 derived from (1). Image lc is the sign change image (white corresponds to a sign change, black to none) and gives a pictorial illustration of the sign change random distribution in the subtraction image. Figure ld gives the values of the SCC, CC, CF, SAVD criterions for translations along the vertical axis varying from 0 to + 10 pixels. Their values have been normalized in the following way: for each criterion, one unit of ordinate axis corresponds to the difference between its maximum and minimum value in the - 10, +10 pixels interval. Thus, these normalized criteria can be compared. Every criterion is optimum for a null translation value which is the right value for every image in the following. Figures 2a and 2b show two scintigraphic images of a radioactive liver phantom. In the case of image 2b, lead materials have been set on the phantom to dramatically modify the image (there was no displacement of the phantom under the camera and the acquisition parameters remained constan0. Image 2c is the sign change image and demonstrates no sign change in the modified part of the image and random sign changes in the remaining similar parts. The SSC criterion value is 1993 for a window of 6068 pixels. In the same way as in Fig. 1, Fig. 2d gives the SSC, CC, CF, SAVD criterion values for translations along the vertical axis of image 2b. The SSC criterion is the only one to be maximum for a 0 translation value; CC is maximum for a translation of - 7 pixels; CF for - 6 pixels; and SAVD minimum for - 2 pixels. 3. THE DETERMINISTIC CASE
3.1. Construction of the Deterministic Similarity Measure Suppose now that the noise level in the two images Ft(i, j) and F2(i, j) is low compared with the precision of the digitization. Fl(i, j) and F2(i, j) are perfectly similar and the SSC criterion value is 0. Assume a periodic pattern is added on image F2(i , j) in the following way to obtain image F3(i, j),
F3(i,j)=F2(i,j)+ q
if i + j is even,
F3(i, j) = F2(i, j ) - q
if i + j is odd
(q is a small real or integer value). Now calculate the subtraction image D(i, j) = Fl(i, j ) - F3(i, j). The values of the D(i, j ) pixels are alternatively - q , + q, - q , + q . . . . . and the signs of D(i, j) pixels - , +, - , + . . . . . We define the deterministic sign change criterion (DSC criterion) as the number of sign changes in D(i, j). If there are N pixels in this image, the DSC criterion value is N-1.
3.2. Application to the Digitized X Ray Images Digital subtraction angiography systems are now widely used as additives to conventional angiographic suites in order to obtain angiographic images with small volumes of iodine contrast intravenously injected. In such images, the noise is low compared with the precision of the digitization, so that this physical device has properties which are in agreement with the assumptions made in the deterministic
A NEW CLASS OF SIMILARITY MEASURES
181
case [11]. Angiographic images are obtained by subtracting X ray images obtained sequentially without and with contrast medium. The digitization format is currently 512 • 512 with an 8-bit depth coding. One major problem of digital angiography concerns the movements of the patient which diminish the benefit of the subtraction procedure. If such a displacement occurs, the second image must be registered prior to subtraction. We propose to maximize the DSC criterion to perform this registration. As in the stochastic case, we will experimentally demonstrate that the DSC criterion leads to registration algorithms which are more robust than those provided by the optimization of the CC, CF, SAVD criterions. Figure 3 illustrates the case of similar images. Image 3a is a skull image obtained at the beginning of a cerebral angiography. Image 3b was obtained just after image 3a and is perfectly similar; a periodic pattern + 1, - 1 , + 1, - 1 . . . . , was added to this image for the calculation of this DSC criterion. Its value for progressive translations of image 3b along the vertical axis is given in Fig. 3c and compared to
% 9
9
\-" ;,.
-~, 9 "
~
t 9" o"
9"
-.
't,
OSC
,:
%
9 :
t,
:
t
"./ I
\
l
i~ 1
I f
l
,'"
".
"V.
1%-
%.
%.
I
%%,%
9"
". ".
"%,,%
-8
-6
-4
-2
"o "~,o% ".,o "x;
1
-10
~ ".
0
2
4
6
8
10
pixel
FIG. 3. SimilarX ray images: (a) X ray image of a skull; (b) same image with an added periodic pattern; (c) DSC, CC, CF, SAVD criterionvalues for translations from 0 to _ 10 pixels along the vertical axis (for the window drawn on image (a)).
182
VENOT, LEBRUCHEC, AND ROUCAYROL
the CC, CF, and SAVD values. The maximum DSC criterion value is 6067 for a selected window of 6068 pixels. DSC, CC, and SAVD criterions are optimum for a null translation and CF is maximum for a translation - 10 pixels. The robustness of the DSC method is illustrated by Fig. 4 which corresponds to the practical case of images collected during a carotid angiography without (image 4a) and with (4b) contrast medium intravenously injected. There was no movement
e
\
,.4\
\ ~176 ....."
9" "
0 "~
,.~"
:("/ " .'" / ~ "/ I \ .,'4 /
,..
"F~ I
/
\
',,/
/
/
j~t
/
~
I Ir
/sAvo
8 ~
OS~,~
"v
,
/
.
.A
I"
/
/ ~% ',
O. ~ -~o-a
"
-6
-4
-2
Ill
"~
,,
"" "--'tes 2
:4
"
~
:8
"-
lo
pixel FIG. 4. Dissimilar X ray images: (a) carotid angiography (without iodine contrast); (b) carotid angiography (with iodine contrast); (c) subtraction image for the window drawn on image (a); (d) sign chanse image in the selectedwindow; (e) DSC, CC, CF, SAVD criterionvalues for translations from0 to + 10 ppixels of image (b) along the vertical axis (in the window).
A NEW CLASS OF SIMILARITYMEASURES
183
of the patient and the subtraction permitted to get good quality angiographic images (image 4c). Image 4d is the sign change image and demonstrates no sign change in a large zone of the subtraction image (because of the contrast medium). Figure 4e permits the comparison of the SSC, CC, CF, and SAVD criterion values for progressive translations along the vertical axis. The DSC criterion is maximum (2227 sign changes in a 9900 pixel window) for a null translation. Same periodic pattern as for image 3b but contrast medium was added to image 4b for its calculations. CC and CF criteria are, respectively, maximum for - 1 and + 7 pixel translations and the SAVD criterion is minimum for a + 1 pixel translation. 4. DISCUSSION AND CONCLUSION A new class of similarity measures between images has been proposed. These criteria differ from the classical ones mainly because their values do not take into account the pixd values of the images. They depend in fact on the noise in the stochastic case or on the added pattern in the deterministic one. In the case of similar images (Figs. 1 and 3) the SSC and DSC criteria are demonstrated to be similarity measures as well as the CC and SAVD criteria. The CF criterion fails to be maximum for a null translation in the case of Fig. 3 images so that this criterion cannot be considered as a valid similarity measure for such images. The curves of Fig. 1 and Fig. 3 demonstrate a unimodal optimum for the SSC and DSC criteria as well as for the CC and SAVD criteria. This is convenient for the application of an optimization procedure to automatically calculate the registration parameters. The ability of the SSC and DSC criteria to provide robust registration procedures is demonstrated by the two examples of Fig. 2 and Fig. 4. When large disparities exist in the images, the maximum SSC and DSC criterion values are far lower than when the images are similar. Nevertheless the SSC and DSC criteria are the only ones that remain optimum for the true value (zero) of the translational shift; the CC and SAVD criteria are optimum for wrong values of the translation. Therefore we propose to use this new class of similarity measures for registration problems whenever important changes in the images can lead to misregistration with the classical criteria. This is typically what happens when digital angiography images are considered. For practical reasons, only translations were considered for the experimental demonstration of the robustness of this methodology but other geometric or gray scale transformations could have been convenient as well. Moreover this method was successfully used in nuclear medicine to automatically register very dissimilar scintigraphic images with respect to five parameters (two translational shifts, one rotation angle, two parameters of a linear transformation of the gray scale) [12, 13, 14, 15, 16]. In this context, the optimized SSC criterion value is useful to determine if the set of transformations which is proposed can lead to a correct registration. The optimized SSC criterion value must be in the range of the confidence interval (1) after registration of similar images. Furthermore, simple statistical tests can be implemented to determine if the adding of a new registration parameter brings a significant increase of the SSC criterion value. In both stochastic and deterministic cases, assuming that a correct registration of the images has been performed, the difference between the SSC or DSC criteria and their expected values (for similar images) gives a simple quantitation of the change dimensions which exist
184
VENOT, LEBRUCHEC, AND ROUCAYROL
in the images. This could be useful for the follow up of a pathology in the context of the medical imagery. A new look at the problem of image registration has been presented: The examples of applications are in the field of the medical imaging. Nevertheless there is no restriction to the application of these methods to the various other nonmedical imaging techniques. ACKNOWLEDGMENTS
The authors wish to express their gratitude to G. Frija who provided the X ray images. They thank C. Muse, J. L. Golmard, J. L. Steimer, and S. McWilliams for many useful comments and discussion. REFERENCES 1. L K. Aggarwal, L. S. Davis, and W. N. Martin, Correspondence processes in dynamic scene analysis, Proc. IEEE 69, 1981, 562-572. 2. R. L. LiUestrand, Techniques for change detection, IEEE Trans. Comput. C21, 1972, 654-659. 3. A. Rosenfeld, Image pattern recognition, Proc. IEEE 69, 1981, 596-605. 4. S. T. Barnard and W. B. Thompson, Disparity analysis of images, IEEE Trans. Pattern Anal. Mach. lntel. PAMI2, 1980, 333-340. 5. D. I. Barnea and H. F. Silverman, A class of algorithms for fast digital image registration, IEEE Trans. Comput. C21, 1972, 353-357. 6. H. Eghbali, K-S test for detecting changes from Landsat imagery data, IEEE Trans. Syst. Man Cybernet. SMCg, 1979, 17623. 7. W. K. Pratt, Correlation techniques of image registration, IEEE Trans. Aerosp. Electron. Syst. AESI0, 1974, 353-358. 8. M. Svedlow, C. D. McGillem, and P. E. Anuta, Image registration: Similarity measure and preprocessing method comparisons, IEEE Trans. Aerosp. Electron. Syst. AESI4, 1978, 141-149. 9. N. B. Draper and H. Smith, Applied Regression Analysis, Wiley, New York, 1966. 10. L D. Gibbons, Nonparametric Statistical Inference, McGraw Hill, New York, 1971. 11. G. Cohen, L: K. Wagner, and E. N. Rauschkolb, Evaluation of a digital subtraction angiography unit, Radiology (Easton, P) 144, 1981, 613-617. 12. A. Venot, J. F. Lebruchec, J. L. Golmard, and J. C, Roucayrol, An automated method for the normalization of scintigraphic images, J. NucL Med. 24, 1983, 529"531. 13. A. Venot, J. F. Lebruchec, J. L. Golmard, and J. C. Roucayrol, Digital methodsf0r change detection in scintigraphic images, J. Nuel. Med. 24, 1983, 67. 14. L. Pronzato, E. Walter, A. Venot, and J. F. Lebruchec, A general purpose global optimizer: Implementation and applications, Math. Comput. Simul., in press. 15. A. Venot, J. L. Golmard, J. F. Lebruehee, and J. C. Roucayrol, Digital methods for change detection in medical images, Proceedings, lnternat. Conf. Inform. Process. Med. Imaging, Nijhof, The Hague, 1984. 16. A. Venot, J. F. Lebruchec, E. Walter, L. Pronzato, V. Leclere, and I. C. Roucayrol, Automated methods for the registration of dissimilar images, Proceedings, First Image Syrups. CESTA, Image Process. Comput. Generated Images Appl., Biarritz, France, May 1984.