- Email: [email protected]
Chapter 5 Automatic Correspondence Finding in Deformed Serial Sections
E.J. Karjalainen (Editor), Scienrific Computingand Auromation (Europe) 1990 0 1990 Elsevier Science Publishers B.V., Amsterdam
39
CHAPTER 5
Aiitornutic Correspondence Finding in Deformed Seriul Sections Y. J. Zhang Information Theory Group, Department of Electrical Engineering, Delft University of Technology, The Netherlands
Abstract Three dimensional objcct rcconstruction from serial sections is an important process in 3D iinagc analysis. One critical stcp involved in this process is the registration problcm. Wc proposc a gcncral approach for automatic rcgistration of deformed serial sections. The automatic profile rcgistration is realized in two consccutivc steps, that is thc onc to onc corrcspondcncc finding and followed by the onc by onc alignment. An automatic incthod callcd pattcrn matching is proposcd for the corrcspondcnce finding. This incthod is bascd on pattcrn rccognition principles and consists of dynamical patlcrn forming and gcomctric pattcrn testing. A practical implcmcntation of lhc pattcrn matching proccdurc in thc contcxt of 3D rcconstruction of mcgakaryocytc cells in bone marrow tissue of 2pin scctions is prcscntcd. Local gcomctric rclationship among profilcs of different objccts in Ihc samc scction and the statistic characteristics of section dcformation havc bccn cmploycd to ovcrcoinc the difficulty caused by scction distortion. Satisfactory rcsulls havc becn oblaincd with rathcr scriously dcformcd tissue sections. As thc matching patterns can bc dynamically conslructcd and thcy arc translation and rotation invariant, this method should also bc useful in a varicty of rcgistration applications.
1. Introduction In 3D iinagc analysis of biological structures, the rcconstruction of 3D objccts from scrial scctions is a popular proccss to study thc inncr and body structurcs of objccts. Onc csscntial process involvcd in thc 3D objcct rcconstruction is the alignmcnt of succcssivc profiles bclonging to samc objccts in consccutivc scctions [l]. A morc gcncral Lcrm uscd is iinagc rcgistration. Thc goal of rcgistration is to corrcct thc cvcntual position, oricntation, magnification and cvcn grcy lcvcl diffcrcnccs in thc iinagcs of scrial scctions [2].
40
Two classes of registration techniques can be distinguished: one class is the corrclation technique (see [2], [31), which consists of formatting a set of correlation mcasuremcnts between two image sections. Thcn the location of maximum correlation is dctcrmincd. Two imagcs can bc spatially registered according to this location via a section transformation. One important drawback of this technique is that it nccds enormous computation time for complicatcd image, even some techniques for reducing computation effort have bccn proposcd [4].Morcovcr, corrclation is also scnsitive to gcomctrical distortion [ 5 ] . Anothcr class is the landmark technique [6,7], that mcans in aligning sections with the hclp of landmarks, which can be artificial or anatomical ones. The usc of artificial landmark techniques is often limited, because the landmarks must be introduced before cutting the scctions and some distortion may also be introduced into the tissue during thc creation of thcse marks. On the othcr hand, the choice of anatomical landmarks is very problem-oriented and it is difficult to detect thcm automatically. Rccently, shapc points obtaincd from object contours have been proposed in 3D registration process [8]. Howcvcr, thc determination of shapc points requires the results of objcct scgmcntation. The automatic rcconstruction process will be even much complicated if a large population of objects should be treated and for cach object high resolution imagcs should be uscd to gct accurate mcasuremcnt results. In such a case, vcry expensive computation efforts would bc expcctcd. Wc have presented a two stcp reconstruction mcthod for rcsolving such a problcm [9]. Two typcs of reconstructions are splitcd: onc is symbolic rcconsuuction which involvcs idcntifying and linking thc scparatcd profilcs of an objcct cmbcddcd in a scrics of scclions; anothcr is pictorial rcconstruction which consists of grouping consccutivc profilcs bclonging to thc same objects, and rcconsuucting thcsc objcco in thc 3D space as thcy wcrc bcfore being cut. Corrcspondently, two lcvcls of rcgisuations arc also distinguishcd. Wc call them global lcvcl registration and local lcvcl registration, scparatcly. In thc formcr one, all pairs of corresponding objcct profilcs in two consccutivc scclions are to be idcntificd. That is, for cach profilc of an objcct in onc scction, to dccidc if it is continued in the adjaccnt section, and if it continues, finding thc corrcsponding onc in this section (this may be accomplished with somc lowcr rcsolution imagcs). Thcn, with Lhc hclp of this global result, we can geomctrically corrcct Lhc diffcrencc in oricnlation and position (also possiblc for scaling and/or grcy lcvcl) of these corresponding profiles with full rcsolution imagcs to gct prccise rcgistration rcsult in cach local lcvcl. As wc can scc, thc corrcspondcnce finding is a mandatory stcp in 3D objcct rcconslruction. This is bccausc thc rclalion bctwccn two profilcs of onc objcct in two scclions is first cstablishcd, thcn it bccomcs possible to pcrfcclly rcgistcr thcsc two profilcs. Abovc mcntioncd two classes of rcgistration tcchniyucs can bc uscd in thc sccond lcvcl of thc rcgisuation. Howcvcr, thcy arc not suitablc for the first lcvcl of rcgistration wlicn thc scctions arc individually dcformcd during section prcparation (this is oftcn thc casc cspccially with vcry thin scctions, and cach section is trcatcd scparatcly).
41
The basic assumption for most existing registration techniques is that the set of structures of an object can be considered as a rigid body and when the whole set changes its position and orientation in the space, all structures belong to this set always keep their original relative positions with regard each other. On basis of such an assumption, the registration can be achieved by a global transformation of the whole set. However, when the sections are deformed, the above assumption may be violated. Since the geometric relationship among different profiles in one section would be changed due to distortion, the relative positions of corresponding profiles in adjacent sections will also be altered. For such a situation, good one to one match cannot be obtained solely by a global transformation of the whole section. Because of the considerable disparity from section to section, perfectly enlisting a few pairs of corresponding object profiles could not ensure the correspondence between other pairs of object profiles. A direct one to one object registration becomes necessary in this case. In section 2, one automatic technique for the one to one correspondence finding in deformed serial section using local coherent information will be introduced. We call it pattern matching as it is based on pattern recognition principle and consists of dynamical pattern formatting and geometric pattern testing. Section 3 will give a real example of the use and practical implementation of pattern matching technique for the quantitative analysis of 3D megakaryocyte cells in bone marrow tissue. Finally, some considerations for improving the pcrformancc of pattern matching and using this technique to meet other applications are discussed in section 4.
2. Pattern matching
2 .I General description In this section we will present an automatic correspondence finding technique for the registration of profiles in adjacent sections. It is convenient to present every profile by a point located at the center of gravity when all profiles under consideration are rather dispersed in a large area, which is often the practical situation. With such a representation, one useful information which can be still exploited is the local relationship among vicinity profiles and such information would be less influenced by the section distortion. It is implied here that the deformation normally occurs in other places than the profiles themselves lying down. The block diagram of the whole procedure of pattern matching is shown in Figure 1. Each time, two conscculive scctions are taken into account for the correspondence finding. Thc section uscd as reference one is called matchcd section whereas the section to be matchcd is callcd matching section. When we use the point representation one point is callcd matchcd point or matching point according to the section it belongs to. For each matchcd point in matchcd section, the objectives are to determine if it has or has not a
42
Figure 1. Block diagram of pattern matching procedure.
hlatchcd scction
Rlatchcd object choicc
Rlatchcd pattern formation
objects choice
Matching pattern formation
I
idcn tification
corrcspondlng matching point in matching scction, and in the tormcr case, to idcntily 11s corrcspondcnt. Thc first Lisk is thc matchcd point selection in matchcd section, for which a corresponding point in matching scction will be lookcd. For cach sclcctcd matchcd point, a pattcrn is constructed by using thc information concerning this point and several surrounding points. Thc information rclatcd to onc pattcrn can be grouped into a pattern vcctor. Thc dimcnsion of the pattcrn vcctor dcpcnds on the quantity of information uscd to idcntily the pattcm. Thc pattern vector thus formcd has an onc to onc correspondcncc with this point. Next comcs the sclcction for potcntial matching points in the matching scction. The rcgion of scction, in which thc sclcction is taking placc, is callcd search rcgion. This rcgion can bc rcslrictcd to some parts of the section if a priori knowlcdge about the point distribution is availablc. Othcrwise, thc search region may be the wholc section. For each candidatc matching point, a corresponding pattern is also constructed similarly as for thc matchcd point. A numbcr of pattcrn similarity tests arc then performed, each time bctwccn thc sclected matchcd pattern and one of Ihe potential matching patterns. The decision about whcthcr any matching pattcrn matches thc given matchcd pattcrn is madc according lo thcsc tcsts. If a matching rcsult bctwccn the two pattcrns is obtaincd, thc rclation bctwccn two ccntcr points will bc idcntificd. This thrcc stcp proccdure is pcrformcd, in soinc cxtcnt, as a human bcing would do for such work [lo]. This incthod sharcs somc
43
common properties with template matching techniques (see, e.g., [ll]),however, some essential differences exist. Some detailed discussions will be given in following subsections.
2.2 Pattern construction The patterns are constructed dynamically as opposed to a previously fixed template in template matching techniques. For each matched point, m nearest surrounding points are chosen in the construction of the related matched pattern. This pattern will be composed of these m points together with the center point. The pattern is specified by the center point coordinates (XO, yo), the m distances measured from m surrounding points to the center point (dl, d2, ...,d,J and the m angles between m pairs of distance lines (81, 82, ..., 8,J A pattern vector can therefore be written as:
where 1 is the label of center point. Each element in pattern vector can be considered as a feature of the center point. The maximum of m distances is called the diameter of the pattern. For each potential matching point, a similar pattern needs to be constructed. All surrounding points falling into a circle (around the matching point) which has the same diameter as that of the matched pattern are taken into account. These pattern vectors can be written in a similar manncr:
The number n may be dirfcrent from m (and can be different from one pattern to another), because of the deformation of section and/or the end of continuation of objects. The above process for pattern formation is automatic. Moreover, this pattern formation method is rather flexible as here no specific size has been previously imposed to construct patterns. We also allow those constructed patterns to have a different number of points. The pattern thus constructed is called “absolute pattern”, because its center point has absolute coordinates. One example of absolute pattern is illustrated in Figure 2A. Each pattern thus formed is unique in the section. Every pattern vector belongs to a fixed point and presents specific properties of this point. To match corresponding points in two adjacent sections by means of their patterns, translation and rotation of patterns are necessary. The absolute pattern formed abovc is invariant to rotation around the center point because it is circularly defined, but it is not invariant to translation (see Fig. 2B). To overcome this inconvenience, we further construct, from each absolute pattern, a corresponding “relative pattern” by discarding the
44
A
B
Y
Y
0 - - -
-.
Yo
translation d
X
XO
Figure 2. Absolute pattern.
coordinates of center point from pattern vector. One such pattern is shown in Figure 3A. The relative pattern vector corresponding to (1) can be written as:
The relative pattern is not only invariant to rotation around the center point, but also invariant to translation (see Fig. 3B). The absolute pattern has a one to one correspondence with respect to the center point. The uniqueness of relative pattern is related to the number of surrounding points in Lhe pattern and also depends the distribution of objects in sections. This is because that one point set has its unique pattern description, but several sets do not necessarily have different pattern descriptions. Intuition tells us that when Lhe amount of points are expanded and the distribution of points is more random the uniqueness of the description becomes more sufficient. So the appropriate value for m should be determined by a leaming set.
A
B
Y / - - - - -
Diameter \
\
I
''. I
I
/ /Angle
' a . -
I
\
0
Figure 3. Relative pattern.
1
Distance X
I
Y
45
2.3 Pattern similarity test and decision making The pattcrn similarity can be trcatcd only in certain specific positions and oricntations as opposite to systematic calculations of every possible translation and rotation value of the tcmplatc relative to the refcrcnce image. The relative patterns proposed above are used in similarity tcst. As patterns would contain various numbers of points, the dimensions of thcir pattcrn vectors will bc different and term-by-term based matching (e.g., [12] p. 428) can not be pcrfomcd. A gcomctrical technique has been used to assist the similarity tcst. Two tcsling pattcrns (one from each section) can be unlike in position and orientation. Wc first translatc the matching pattern to put its center overlap with the center of the matchcd pattcrn. Since relative patterns are under consideration, this proccss just implies to put thcse pattcrns in the center of the coordinate system. Then, we rotate the matching pattcrn until two distance lines (one from each pattern) coincide. This produces a combincd configuration of two pattcrns. Continuing rotate the matching pattern, we will obtain all combincd test configurations. At each tcst configuration, the similarity of the two pattcrns will be investigated. Here diffcrcnt criteria of similarity arc available. We have uscd the absolute difference measure bctwccn two testing patterns as its cumulative nature allows it to be incorporatcd into fast matching algorithms [13]. For cvcry point of the matched pattern, its nearest neighbor (in the Euclidcan scnsc) in thc matching section is first looked for. If two center points arc corrcspondcnts, there will be one test configuration where such distances are rather small and this shows the match position. This is predictable because the matching points aftcr translation and rotation will fall into the vicinity of relatcd matched points. If two pattcrns have a different number of points, it would imply that therc are some points which do not have their correspondents in adjacent section (this is also possible even if the two pattcrns have same numbers of points). If one point has not its correspondent in adjaccnt scction, its ncarcst ncighbor will be a point which has no relation to it. In this case, the mcasurcd dislance would be rather big. Taking such a distance into consideration will make the tcst to give erroneous results. To solve this problem, one thrcshold is uscd to eliminate thcse ncarcst ncighbors from funhcr calculation if thcir minimum distances with points of matchcd pattcm cxcecd a given Icvcl. Thc distancc of ncarcst neighbor givcs an index of thc similarity. This mcasurc can be uscd directly to calculatc thc goodncss of the match. Howcvcr, a mcrit function which is bascd on the wcightcd distance mcasure for mcasuring thc quality of match can also bc dcfincd. From onc sidc, it pcrmits to incorporatc somc apriori knowlcdgc about point sct and give accuratc judgcmcnt of match; from othcr sidc, it can hclp thc choice of thc threshold value mcnlioncd abovc. This function should be invcrscly proportional to thc distancc bctwccn a pair of comparcd profilcs. The highcr the function value, the better is the match.
46 ~~
Relative patterns formation
I Y Relative pattern rotation
1
Distance measurement
I
1
I
5 I
D z Threshold
1-
Calculation of merit function
I
Calculation of RMS of merit values
I Searching the maximum of RMS I 1
Matching Point pair
j
Nomatching
Figure 4. Flowchart of pattern similarity test.
Thc pattcrn tcsls arc pcrformcd for all potcntial matching patterns. The final dccision about the goodncss of match is made aftcr all those pattern similarity tests. The root of mcan square (RMS) of all merit function valucs can be uscd for thc final dccision. Using RMS has the advantage of being more accuratc than the normal mcan value, cspccially whcn thc numbcr of mcasurcmcnls is small [14]. Thc potcntial matching pattcrn with minimum RMS of merit function valucs is considcrcd as thc corrcsponding pattcrn of Lhc matchcd pattcrn. Thc flowchart of pattcrn similarity test and dccision making is shown in Figurc 4. Oncc a match has bccn found for two profilcs, thcy can be rcspcctivcly labcled.
47
3. Real application
3.1 Material and method Hcrc we prcscnt a rcal expcrimcnt using above proposed technique. In the quantitative analysis or mcgakaryocytcs in bone marrow tissue, serial sectioning technique has been used whcrc the tissue was cut into a number of consecutive 2-pm thick scctions to reveal the inncr swucture undcr light microscope [9]. The 3D reconstruction of megakaryocyte cells was first rcstrictcd by the serious deformation of sections caused by a relatively complex scction prcparation procedure (some deformation examples can be found in [15]). One point rcprcscntation example of mcgakaryocyte cells in tissue is given in Figure 5, whcrc for two consecutive sections, not only the numbers of points (profiles) GROUP LABEL : 33 SECTION LABEL : 107 NUMBER OF CELLS : 112
. . ..
-
..
.
...:*:.... ... . -,.-: . . . . . . . . .. . ... -* *
I
I
-
-*.
..
*
. :1
..... - . I
...-
.
GROUP LABEL : 33 SECTION LABEL : 108 NUMBER OF CELLS : 137
0
.
.. . ..
. *
..
... -..'.-.:...
. ... * . . . . .. : *. %
*
Figure 5. Point rcprescntation examplcs of deformed section.
1 .
I
48
are different, but also the arrangements of points are rather dissimilar. Secondly, the expected big computation effort is also serious. The average surface of one section is about 6x107 pm2. A sampling rate of 5 pixel/pm has been employed for the accuracy of measurement, thus with full resolution, the image area will be 1.5~109pixels (corresponding to about six thousand 512*512 images) per section. The reconstruction approach indicated in the first section has been uscd to reduce the storage and computationalrequirements. Above pattern matching technique has been employed to find the corresponding profiles in those deformed serial sections. The process is carried out as described in last section. In the beginning, the first section in a group is chosen as the matched section, for each profile in this section, we look for its correspondent in the second section, with the pattern matching method. Then the next pair of sections is taken into account, this process continues to the last section of group.
3.2 Merit function For this application, a merit function indicating the quality of match is derived based on the statistical characteristics of mismatch error in the section and calculated according to thc probability theory. As described and tested in [15], the dissimilarity mainly caused by the section distortion in both axes of the rectangular Cartesian system are normally distributed. If X and Y are the random variables denoting the mismatch along x and y axes, respectively, their probability density functions can be written as follows (see, e.g., [16]):
P( Y) =
1 Exp (2 7 r y 2 cry
[*) 2.”y
where a, and orare the standard deviation of X and Y, respectively. As mentioned above the Euclidean distance has been used in similarity tests. This distance is a function of X and Y and is also a random variable. If we denote it by R, then:
The probability density function of R can be written as [17]:
49
whcrc p(x, y) is the joint probability density function of X and Y, and &) is the impulse function. , = ojl, It is reasonable to assume that X, Y are indcpcndcnt random variables and cr = a bccause the distortion is random, which has no dominant orientation. Under this condition, wc havc:
Taking (8) into (7), we finally gct:
Thc distribution of R is a Raylcigh distribution which attains its maximum at r = cr. Normalizing equation (9) yields :
Thc mcrit function is thcn dcfincd as:
I M(r)=
if O l r l o
I
if
r>o
Thc curvc of merit function is shown in Figure 6, where x = r / cr. This function is a singlc valued function of r. Thc good match rcccivcs morc rcward than a bad onc. For x 2 1, Figurc 6. Curve of a mcrit function.
0
0.5
1
1.s
2
2.6
3
3.5
50
thc function value is proportional to the probability of R and thus takes into account the statistical charactcristics of mismatch caused by the distortion of section. From Figure 6, one will notice that as the curve recedes from x = 1 ( r = o), it dcsccnds fastcr and faster until it reaches x = 43 ( r = 0*43). Bcyond that point, the dcsccnt is cvcr slower. It can further bc shown that this point is an inflcxion point. The valuc ( r = d 3 ) is choscn as the threshold as mcntioned in last scction. Only the distimccs bctwccn a point of matched pattcrn and its nearest neighbor which do not exceed this thrcshold will be taken into account for decision making.
3.3 Results In onc of our cxpcriments for thc 3D rcconstruction of mcgakaryocyte cclls, totally 1926 mcgakaryocyte profilcs sclccted from 15 consecutive sections have been ucatcd. Thc pattern matching proccdurc has been applied to those 14 scction pairs (the two sections shown in Figure 5 is one pair of them) to rcgistcr those profilcs. Thc automatically cstablishcd rclations bctwcen adjaccnt profilcs have bccn comparcd with the opcrator's obscrvations undcr microscope for the vcrification purpose. Thcsc rcsults are listed in Table 1. Thc fist column givcs the labels of scctions (matchcd/matching scction). The sccond column shows thc numbers of cclls in rcspcctcd matched and matching scction, rcspcctivcly. Thc sum numbcr givcn bclow is the numbcr of cclls lo bc matched. Thc rcsults arc rathcr satisfactory since 95.63% profilcs arc corrcctly assigncd by the automatic pattcrn matching proccdurc (rangc of 90.30% to 99.27%). Only 4.37% (rangc or 0.73% to 9.70%) of asscssmcnts arc incorrcct. In an othcr trial, whcn only global transformation has bccn used, the corrcspondcncc finding error ratc was attaincd 33.21% to 41.97% for difkrcnt scctions. The accuracy of our tcchniquc is quitc satisfactory. Bascd on thc rcsult of global rcgistration, 51 complctc mcgakaryocytc cclls in thosc sections havc bccn symbolically rcconstructcd. More dctails can be found in [ 151.
4. Discussion Thc pcrformancc of thc abovc mcntioncd corrcspondcncc finding tcchniquc can bc improvcd in scvcral ways in practical applications. In casc of most of objccts extending into scvcral consccutivc scctions, the profilc distributions in thosc scctions should havc somc rclations cvcn if thc scctions arc deformed. This likcncss may oftcn bc pcrccivcd whcn thc diffcrcncc in oricntation among sections has bccn corrcctcd. This implies, likc in thc two-stagc tcrnplalc matching tcchniqucs [18], a two stiigc pattcrn matching proccdurc can also bc cmploycd to takc advantagc of this likcncss. At thc first stagc, a rough transformation (translation and/or rotation of the wholc scction) applying thc lcast squarcs criterion (such as proposcd in [6]), but can be detcrmincd automatically as mentioncd abovc) can bc cmployed aftcr the rcgistration of scvcral pairs of corrcsponding
51
TABLE 1 Pattcm matching results. ~~
Scclion pair
# of cells
Correct
1021103 1031104 1041105 105/106 1061107 107/108 1081109 109/110 1101111 1111112 1121113 1131114 1141115 115/116
1331134 1341129 1291137 1371137 1371112 1121137 1371139 1391117 1171136 1361116 1161131 1311108 1081120 1201140
124 121 126 130 130 104 136 135 112 134 114 121 102 119
1786
1708
Sum
Avcrage
%
93.23 90.30 97.67 94.90 94.90 92.86 99.27 97.12 95.73 98.53 98.28 92.37 94.44 99.17 95.63
Error
9 13 3
7 7
8 1
4 5 2 2 10 6
1
78
%
6.77 9.70 2.23 5.10 5.10 7.14 0.73 2.88 4.27 1.47 1.72 7.63 5.56 0.83 4.37
profiles in adjaccnt sections. This transformation allows to reduce the search region bccausc it would cstablish a preliminary correspondcnce between two sections. Only those matching points falling into the search region will be considered as potential matching points. In many cases, this region will be quite small compared with the whole scction area. As a result, only a small number of patterns are to be constructed and to be tcstcd. In this way, thc calculations will be greatly reduccd. Another improvemcnt may be madc is to rcducc thc computation effort for tcsting no-match patterns. Since the cumulativc diffcrcnce mcasurcment has bccn employed, it is possible that some judgcmcnt can bc madc bcforc all thc matching patterns arc complctcly tcstcd. For example, whcn the diffcrcncc bctwcen two tcsting pattcrs is already big enough that continuing calculation would be worthlcss, thc Lcst for current patterns can bc stoppcd. In our expcricncc, the nurnbcr of potcntial matching points can bc much largcr than thc. numbcr of malchcd points, thc computation rcduction is also considcrable. Slatistically, the largcr the mismatch bctwccn two patterns, thc shortcr is the computation. Thc pattcrn matching tcchniquc has certain generality because the patterns for matching arc dynamically constructed, the pattern can be any size so it is suitablc for the cases whcrc thcrc is grcat variation of the patterns to be matchcd. Thc principal idca behind this tcchniquc is morc gcncral, i.c., to extract characteristic descriptions for the rcgislration and classification of pattcrns. From this point of vicw, some straightforward modifications to adapt this tcchniquc to dilfcrcnt situations can be easily madc. For cxamplc, wc can introduce the tcxturc information of image into thc pattcrn vector whcn such an information is availablc to match tcxturc imagcs. Also thc basis of match can bc not limited to
52
spatial patterns, it can be any group of atlributes associated with the pcrccption of patterns. Finally, we want to point out that the symbolic reconstruction based on correspondcnce finding is a very useful process in 3D image analysis. On one hand, following 3D pictorial rcconstruction bccomes possiblc based on the relationship among different parts of same objects. On thc other hand, certain analysis tasks, such as 3D objcct numbcr counting and 3D object volume measurement, become possible cvcn before 3D pictorial rcconstruction. In many 3D image analysis applications, this will give cnough information and savc a lot of processing efforts. We can say that the early proposed two lcvcl rcgistration approach is not only a short-cutting procedure for registration of serial sections, but also a useful conuibution to the whole 3D image analysis task.
Acknowledgements We would like to express our heartfelt thanks to Professor G. Cantrainc and the mcnibcrs of h e clcclronics group, as well as Dr. J. M. Paulus and the mernbcrs of thc hcmatology group, of LiCge University, Belgium. The support of the Netherlands’ Project Tcam for Computer Application (SPIN), Thrce-dimensional Image Analysis Projcct is also gratefully apprcciatcd.
References 1.
Jolmson EM, Capowski JJ. Principles of Reconstruction and Thrce-Dimensional Display of Scrial Sections using a Computer. In: RR Mize ed. The Microcomputer in Cell and Neuro-
biology Research. Elscvicr, 1985. Pratt WK. Digital I m g e Processing. John Wiley & Sons, 1978. Gentile AN, Harth A.The Alignmcnt of Serial Sections by Spatial Filtering. Comput Biomcd Res 1978; 11: 537-551. 4. Barncl DI, Silverman HF. A Class of Algorithms for Fast Digital h a g c Registration. IEEE Trans Computers 1972; C-21(2): 179-186. 5. Rosenfeld A. Image Pattcrn Rccognition. Proceeding o f l E E E 1981; 69(5): 596-605. 6. Chawla SD, Glass L, Frciwald S, Proctor JW.An Intcractive Coniputcr Graphic Systcm Tor 3-D Stcrcoscopic Rcconstruction from Scrial Sections: Analysis of Mctastatic Growth. Cornput Biol Mcd 1982; 12(3): 223-232. 7. Kropf N, Sobcl I, Lcvinthal C. Scrial Section Reconstruction Using CAKTOS in Thc Microcomputer. In: RR Mizc cd. Cell and Neurobiology Research. Elscvicr, 1985. 8. Mcrickcl M. 3D Rcconstruction: The registration Problcm. Compul Vision Graphics Image Process 1988; 42(2): 206-219. 9. Zhang YJ. A 3-D Irnagc Analysis Systcm: Quantitation of Mcgakaryocytc. Submitted to Cytomctry 1990. 10. Tou JT, Gonzalcz RC. Paltern Recognition Principles. Reading, MA: Addison-Wcsley, 1974. 11. Koscnfcld A, Kak AC. Digilal Picture Processing. Acadcmic Prcss, 1976.
2. 3.
53
12. Gonzalcz RC, Wintz PA. Digital Image Processing (second edition). Reading, MA: Addison-Wcslcy, 1987. 13. Aggarwal JK, Davis LS. Martin WN. Correspondence Processes in Dynamic Sccne Analysis. Proceeding of IEEE 1981: 69(5): 562-572. 14. Wang LX et al. IIandbook of Mathematics. Peking, 1979. 15. Zhang YJ.Development of a 3 - 0 computer image analysis system, application to megakaryocyte quantitation. Ph.D. dissertation, Libge University, Belgium, 1989. 16. Zar JH. Biostatistical analysis. Prentice-Hall Inc, 1984. 17. Pugachev VS. Theory of Random Functions. Reading, MA: Addison-Wesley, 1965. 18. Vandcrbrug GJ, Kosenfcld A. Two-Stage Template Matching. IEEE Trans Computers 1977; C-26(4): 384-393.
39
CHAPTER 5
Aiitornutic Correspondence Finding in Deformed Seriul Sections Y. J. Zhang Information Theory Group, Department of Electrical Engineering, Delft University of Technology, The Netherlands
Abstract Three dimensional objcct rcconstruction from serial sections is an important process in 3D iinagc analysis. One critical stcp involved in this process is the registration problcm. Wc proposc a gcncral approach for automatic rcgistration of deformed serial sections. The automatic profile rcgistration is realized in two consccutivc steps, that is thc onc to onc corrcspondcncc finding and followed by the onc by onc alignment. An automatic incthod callcd pattcrn matching is proposcd for the corrcspondcnce finding. This incthod is bascd on pattcrn rccognition principles and consists of dynamical patlcrn forming and gcomctric pattcrn testing. A practical implcmcntation of lhc pattcrn matching proccdurc in thc contcxt of 3D rcconstruction of mcgakaryocytc cells in bone marrow tissue of 2pin scctions is prcscntcd. Local gcomctric rclationship among profilcs of different objccts in Ihc samc scction and the statistic characteristics of section dcformation havc bccn cmploycd to ovcrcoinc the difficulty caused by scction distortion. Satisfactory rcsulls havc becn oblaincd with rathcr scriously dcformcd tissue sections. As thc matching patterns can bc dynamically conslructcd and thcy arc translation and rotation invariant, this method should also bc useful in a varicty of rcgistration applications.
1. Introduction In 3D iinagc analysis of biological structures, the rcconstruction of 3D objccts from scrial scctions is a popular proccss to study thc inncr and body structurcs of objccts. Onc csscntial process involvcd in thc 3D objcct rcconstruction is the alignmcnt of succcssivc profiles bclonging to samc objccts in consccutivc scctions [l]. A morc gcncral Lcrm uscd is iinagc rcgistration. Thc goal of rcgistration is to corrcct thc cvcntual position, oricntation, magnification and cvcn grcy lcvcl diffcrcnccs in thc iinagcs of scrial scctions [2].
40
Two classes of registration techniques can be distinguished: one class is the corrclation technique (see [2], [31), which consists of formatting a set of correlation mcasuremcnts between two image sections. Thcn the location of maximum correlation is dctcrmincd. Two imagcs can bc spatially registered according to this location via a section transformation. One important drawback of this technique is that it nccds enormous computation time for complicatcd image, even some techniques for reducing computation effort have bccn proposcd [4].Morcovcr, corrclation is also scnsitive to gcomctrical distortion [ 5 ] . Anothcr class is the landmark technique [6,7], that mcans in aligning sections with the hclp of landmarks, which can be artificial or anatomical ones. The usc of artificial landmark techniques is often limited, because the landmarks must be introduced before cutting the scctions and some distortion may also be introduced into the tissue during thc creation of thcse marks. On the othcr hand, the choice of anatomical landmarks is very problem-oriented and it is difficult to detect thcm automatically. Rccently, shapc points obtaincd from object contours have been proposed in 3D registration process [8]. Howcvcr, thc determination of shapc points requires the results of objcct scgmcntation. The automatic rcconstruction process will be even much complicated if a large population of objects should be treated and for cach object high resolution imagcs should be uscd to gct accurate mcasuremcnt results. In such a case, vcry expensive computation efforts would bc expcctcd. Wc have presented a two stcp reconstruction mcthod for rcsolving such a problcm [9]. Two typcs of reconstructions are splitcd: onc is symbolic rcconsuuction which involvcs idcntifying and linking thc scparatcd profilcs of an objcct cmbcddcd in a scrics of scclions; anothcr is pictorial rcconstruction which consists of grouping consccutivc profilcs bclonging to thc same objects, and rcconsuucting thcsc objcco in thc 3D space as thcy wcrc bcfore being cut. Corrcspondently, two lcvcls of rcgisuations arc also distinguishcd. Wc call them global lcvcl registration and local lcvcl registration, scparatcly. In thc formcr one, all pairs of corresponding objcct profilcs in two consccutivc scclions are to be idcntificd. That is, for cach profilc of an objcct in onc scction, to dccidc if it is continued in the adjaccnt section, and if it continues, finding thc corrcsponding onc in this section (this may be accomplished with somc lowcr rcsolution imagcs). Thcn, with Lhc hclp of this global result, we can geomctrically corrcct Lhc diffcrencc in oricnlation and position (also possiblc for scaling and/or grcy lcvcl) of these corresponding profiles with full rcsolution imagcs to gct prccise rcgistration rcsult in cach local lcvcl. As wc can scc, thc corrcspondcnce finding is a mandatory stcp in 3D objcct rcconslruction. This is bccausc thc rclalion bctwccn two profilcs of onc objcct in two scclions is first cstablishcd, thcn it bccomcs possible to pcrfcclly rcgistcr thcsc two profilcs. Abovc mcntioncd two classes of rcgistration tcchniyucs can bc uscd in thc sccond lcvcl of thc rcgisuation. Howcvcr, thcy arc not suitablc for the first lcvcl of rcgistration wlicn thc scctions arc individually dcformcd during section prcparation (this is oftcn thc casc cspccially with vcry thin scctions, and cach section is trcatcd scparatcly).
41
The basic assumption for most existing registration techniques is that the set of structures of an object can be considered as a rigid body and when the whole set changes its position and orientation in the space, all structures belong to this set always keep their original relative positions with regard each other. On basis of such an assumption, the registration can be achieved by a global transformation of the whole set. However, when the sections are deformed, the above assumption may be violated. Since the geometric relationship among different profiles in one section would be changed due to distortion, the relative positions of corresponding profiles in adjacent sections will also be altered. For such a situation, good one to one match cannot be obtained solely by a global transformation of the whole section. Because of the considerable disparity from section to section, perfectly enlisting a few pairs of corresponding object profiles could not ensure the correspondence between other pairs of object profiles. A direct one to one object registration becomes necessary in this case. In section 2, one automatic technique for the one to one correspondence finding in deformed serial section using local coherent information will be introduced. We call it pattern matching as it is based on pattern recognition principle and consists of dynamical pattern formatting and geometric pattern testing. Section 3 will give a real example of the use and practical implementation of pattern matching technique for the quantitative analysis of 3D megakaryocyte cells in bone marrow tissue. Finally, some considerations for improving the pcrformancc of pattern matching and using this technique to meet other applications are discussed in section 4.
2. Pattern matching
2 .I General description In this section we will present an automatic correspondence finding technique for the registration of profiles in adjacent sections. It is convenient to present every profile by a point located at the center of gravity when all profiles under consideration are rather dispersed in a large area, which is often the practical situation. With such a representation, one useful information which can be still exploited is the local relationship among vicinity profiles and such information would be less influenced by the section distortion. It is implied here that the deformation normally occurs in other places than the profiles themselves lying down. The block diagram of the whole procedure of pattern matching is shown in Figure 1. Each time, two conscculive scctions are taken into account for the correspondence finding. Thc section uscd as reference one is called matchcd section whereas the section to be matchcd is callcd matching section. When we use the point representation one point is callcd matchcd point or matching point according to the section it belongs to. For each matchcd point in matchcd section, the objectives are to determine if it has or has not a
42
Figure 1. Block diagram of pattern matching procedure.
hlatchcd scction
Rlatchcd object choicc
Rlatchcd pattern formation
objects choice
Matching pattern formation
I
idcn tification
corrcspondlng matching point in matching scction, and in the tormcr case, to idcntily 11s corrcspondcnt. Thc first Lisk is thc matchcd point selection in matchcd section, for which a corresponding point in matching scction will be lookcd. For cach sclcctcd matchcd point, a pattcrn is constructed by using thc information concerning this point and several surrounding points. Thc information rclatcd to onc pattcrn can be grouped into a pattern vcctor. Thc dimcnsion of the pattcrn vcctor dcpcnds on the quantity of information uscd to idcntily the pattcm. Thc pattern vector thus formcd has an onc to onc correspondcncc with this point. Next comcs the sclcction for potcntial matching points in the matching scction. The rcgion of scction, in which thc sclcction is taking placc, is callcd search rcgion. This rcgion can bc rcslrictcd to some parts of the section if a priori knowlcdge about the point distribution is availablc. Othcrwise, thc search region may be the wholc section. For each candidatc matching point, a corresponding pattern is also constructed similarly as for thc matchcd point. A numbcr of pattcrn similarity tests arc then performed, each time bctwccn thc sclected matchcd pattern and one of Ihe potential matching patterns. The decision about whcthcr any matching pattcrn matches thc given matchcd pattcrn is madc according lo thcsc tcsts. If a matching rcsult bctwccn the two pattcrns is obtaincd, thc rclation bctwccn two ccntcr points will bc idcntificd. This thrcc stcp proccdure is pcrformcd, in soinc cxtcnt, as a human bcing would do for such work [lo]. This incthod sharcs somc
43
common properties with template matching techniques (see, e.g., [ll]),however, some essential differences exist. Some detailed discussions will be given in following subsections.
2.2 Pattern construction The patterns are constructed dynamically as opposed to a previously fixed template in template matching techniques. For each matched point, m nearest surrounding points are chosen in the construction of the related matched pattern. This pattern will be composed of these m points together with the center point. The pattern is specified by the center point coordinates (XO, yo), the m distances measured from m surrounding points to the center point (dl, d2, ...,d,J and the m angles between m pairs of distance lines (81, 82, ..., 8,J A pattern vector can therefore be written as:
where 1 is the label of center point. Each element in pattern vector can be considered as a feature of the center point. The maximum of m distances is called the diameter of the pattern. For each potential matching point, a similar pattern needs to be constructed. All surrounding points falling into a circle (around the matching point) which has the same diameter as that of the matched pattern are taken into account. These pattern vectors can be written in a similar manncr:
The number n may be dirfcrent from m (and can be different from one pattern to another), because of the deformation of section and/or the end of continuation of objects. The above process for pattern formation is automatic. Moreover, this pattern formation method is rather flexible as here no specific size has been previously imposed to construct patterns. We also allow those constructed patterns to have a different number of points. The pattern thus constructed is called “absolute pattern”, because its center point has absolute coordinates. One example of absolute pattern is illustrated in Figure 2A. Each pattern thus formed is unique in the section. Every pattern vector belongs to a fixed point and presents specific properties of this point. To match corresponding points in two adjacent sections by means of their patterns, translation and rotation of patterns are necessary. The absolute pattern formed abovc is invariant to rotation around the center point because it is circularly defined, but it is not invariant to translation (see Fig. 2B). To overcome this inconvenience, we further construct, from each absolute pattern, a corresponding “relative pattern” by discarding the
44
A
B
Y
Y
0 - - -
-.
Yo
translation d
X
XO
Figure 2. Absolute pattern.
coordinates of center point from pattern vector. One such pattern is shown in Figure 3A. The relative pattern vector corresponding to (1) can be written as:
The relative pattern is not only invariant to rotation around the center point, but also invariant to translation (see Fig. 3B). The absolute pattern has a one to one correspondence with respect to the center point. The uniqueness of relative pattern is related to the number of surrounding points in Lhe pattern and also depends the distribution of objects in sections. This is because that one point set has its unique pattern description, but several sets do not necessarily have different pattern descriptions. Intuition tells us that when Lhe amount of points are expanded and the distribution of points is more random the uniqueness of the description becomes more sufficient. So the appropriate value for m should be determined by a leaming set.
A
B
Y / - - - - -
Diameter \
\
I
''. I
I
/ /Angle
' a . -
I
\
0
Figure 3. Relative pattern.
1
Distance X
I
Y
45
2.3 Pattern similarity test and decision making The pattcrn similarity can be trcatcd only in certain specific positions and oricntations as opposite to systematic calculations of every possible translation and rotation value of the tcmplatc relative to the refcrcnce image. The relative patterns proposed above are used in similarity tcst. As patterns would contain various numbers of points, the dimensions of thcir pattcrn vectors will bc different and term-by-term based matching (e.g., [12] p. 428) can not be pcrfomcd. A gcomctrical technique has been used to assist the similarity tcst. Two tcsling pattcrns (one from each section) can be unlike in position and orientation. Wc first translatc the matching pattern to put its center overlap with the center of the matchcd pattcrn. Since relative patterns are under consideration, this proccss just implies to put thcse pattcrns in the center of the coordinate system. Then, we rotate the matching pattcrn until two distance lines (one from each pattern) coincide. This produces a combincd configuration of two pattcrns. Continuing rotate the matching pattern, we will obtain all combincd test configurations. At each tcst configuration, the similarity of the two pattcrns will be investigated. Here diffcrcnt criteria of similarity arc available. We have uscd the absolute difference measure bctwccn two testing patterns as its cumulative nature allows it to be incorporatcd into fast matching algorithms [13]. For cvcry point of the matched pattern, its nearest neighbor (in the Euclidcan scnsc) in thc matching section is first looked for. If two center points arc corrcspondcnts, there will be one test configuration where such distances are rather small and this shows the match position. This is predictable because the matching points aftcr translation and rotation will fall into the vicinity of relatcd matched points. If two pattcrns have a different number of points, it would imply that therc are some points which do not have their correspondents in adjacent section (this is also possible even if the two pattcrns have same numbers of points). If one point has not its correspondent in adjaccnt scction, its ncarcst ncighbor will be a point which has no relation to it. In this case, the mcasurcd dislance would be rather big. Taking such a distance into consideration will make the tcst to give erroneous results. To solve this problem, one thrcshold is uscd to eliminate thcse ncarcst ncighbors from funhcr calculation if thcir minimum distances with points of matchcd pattcm cxcecd a given Icvcl. Thc distancc of ncarcst neighbor givcs an index of thc similarity. This mcasurc can be uscd directly to calculatc thc goodncss of the match. Howcvcr, a mcrit function which is bascd on the wcightcd distance mcasure for mcasuring thc quality of match can also bc dcfincd. From onc sidc, it pcrmits to incorporatc somc apriori knowlcdgc about point sct and give accuratc judgcmcnt of match; from othcr sidc, it can hclp thc choice of thc threshold value mcnlioncd abovc. This function should be invcrscly proportional to thc distancc bctwccn a pair of comparcd profilcs. The highcr the function value, the better is the match.
46 ~~
Relative patterns formation
I Y Relative pattern rotation
1
Distance measurement
I
1
I
5 I
D z Threshold
1-
Calculation of merit function
I
Calculation of RMS of merit values
I Searching the maximum of RMS I 1
Matching Point pair
j
Nomatching
Figure 4. Flowchart of pattern similarity test.
Thc pattcrn tcsls arc pcrformcd for all potcntial matching patterns. The final dccision about the goodncss of match is made aftcr all those pattern similarity tests. The root of mcan square (RMS) of all merit function valucs can be uscd for thc final dccision. Using RMS has the advantage of being more accuratc than the normal mcan value, cspccially whcn thc numbcr of mcasurcmcnls is small [14]. Thc potcntial matching pattcrn with minimum RMS of merit function valucs is considcrcd as thc corrcsponding pattcrn of Lhc matchcd pattcrn. Thc flowchart of pattcrn similarity test and dccision making is shown in Figurc 4. Oncc a match has bccn found for two profilcs, thcy can be rcspcctivcly labcled.
47
3. Real application
3.1 Material and method Hcrc we prcscnt a rcal expcrimcnt using above proposed technique. In the quantitative analysis or mcgakaryocytcs in bone marrow tissue, serial sectioning technique has been used whcrc the tissue was cut into a number of consecutive 2-pm thick scctions to reveal the inncr swucture undcr light microscope [9]. The 3D reconstruction of megakaryocyte cells was first rcstrictcd by the serious deformation of sections caused by a relatively complex scction prcparation procedure (some deformation examples can be found in [15]). One point rcprcscntation example of mcgakaryocyte cells in tissue is given in Figure 5, whcrc for two consecutive sections, not only the numbers of points (profiles) GROUP LABEL : 33 SECTION LABEL : 107 NUMBER OF CELLS : 112
. . ..
-
..
.
...:*:.... ... . -,.-: . . . . . . . . .. . ... -* *
I
I
-
-*.
..
*
. :1
..... - . I
...-
.
GROUP LABEL : 33 SECTION LABEL : 108 NUMBER OF CELLS : 137
0
.
.. . ..
. *
..
... -..'.-.:...
. ... * . . . . .. : *. %
*
Figure 5. Point rcprescntation examplcs of deformed section.
1 .
I
48
are different, but also the arrangements of points are rather dissimilar. Secondly, the expected big computation effort is also serious. The average surface of one section is about 6x107 pm2. A sampling rate of 5 pixel/pm has been employed for the accuracy of measurement, thus with full resolution, the image area will be 1.5~109pixels (corresponding to about six thousand 512*512 images) per section. The reconstruction approach indicated in the first section has been uscd to reduce the storage and computationalrequirements. Above pattern matching technique has been employed to find the corresponding profiles in those deformed serial sections. The process is carried out as described in last section. In the beginning, the first section in a group is chosen as the matched section, for each profile in this section, we look for its correspondent in the second section, with the pattern matching method. Then the next pair of sections is taken into account, this process continues to the last section of group.
3.2 Merit function For this application, a merit function indicating the quality of match is derived based on the statistical characteristics of mismatch error in the section and calculated according to thc probability theory. As described and tested in [15], the dissimilarity mainly caused by the section distortion in both axes of the rectangular Cartesian system are normally distributed. If X and Y are the random variables denoting the mismatch along x and y axes, respectively, their probability density functions can be written as follows (see, e.g., [16]):
P( Y) =
1 Exp (2 7 r y 2 cry
[*) 2.”y
where a, and orare the standard deviation of X and Y, respectively. As mentioned above the Euclidean distance has been used in similarity tests. This distance is a function of X and Y and is also a random variable. If we denote it by R, then:
The probability density function of R can be written as [17]:
49
whcrc p(x, y) is the joint probability density function of X and Y, and &) is the impulse function. , = ojl, It is reasonable to assume that X, Y are indcpcndcnt random variables and cr = a bccause the distortion is random, which has no dominant orientation. Under this condition, wc havc:
Taking (8) into (7), we finally gct:
Thc distribution of R is a Raylcigh distribution which attains its maximum at r = cr. Normalizing equation (9) yields :
Thc mcrit function is thcn dcfincd as:
I M(r)=
if O l r l o
I
if
r>o
Thc curvc of merit function is shown in Figure 6, where x = r / cr. This function is a singlc valued function of r. Thc good match rcccivcs morc rcward than a bad onc. For x 2 1, Figurc 6. Curve of a mcrit function.
0
0.5
1
1.s
2
2.6
3
3.5
50
thc function value is proportional to the probability of R and thus takes into account the statistical charactcristics of mismatch caused by the distortion of section. From Figure 6, one will notice that as the curve recedes from x = 1 ( r = o), it dcsccnds fastcr and faster until it reaches x = 43 ( r = 0*43). Bcyond that point, the dcsccnt is cvcr slower. It can further bc shown that this point is an inflcxion point. The valuc ( r = d 3 ) is choscn as the threshold as mcntioned in last scction. Only the distimccs bctwccn a point of matched pattcrn and its nearest neighbor which do not exceed this thrcshold will be taken into account for decision making.
3.3 Results In onc of our cxpcriments for thc 3D rcconstruction of mcgakaryocyte cclls, totally 1926 mcgakaryocyte profilcs sclccted from 15 consecutive sections have been ucatcd. Thc pattern matching proccdurc has been applied to those 14 scction pairs (the two sections shown in Figure 5 is one pair of them) to rcgistcr those profilcs. Thc automatically cstablishcd rclations bctwcen adjaccnt profilcs have bccn comparcd with the opcrator's obscrvations undcr microscope for the vcrification purpose. Thcsc rcsults are listed in Table 1. Thc fist column givcs the labels of scctions (matchcd/matching scction). The sccond column shows thc numbers of cclls in rcspcctcd matched and matching scction, rcspcctivcly. Thc sum numbcr givcn bclow is the numbcr of cclls lo bc matched. Thc rcsults arc rathcr satisfactory since 95.63% profilcs arc corrcctly assigncd by the automatic pattcrn matching proccdurc (rangc of 90.30% to 99.27%). Only 4.37% (rangc or 0.73% to 9.70%) of asscssmcnts arc incorrcct. In an othcr trial, whcn only global transformation has bccn used, the corrcspondcncc finding error ratc was attaincd 33.21% to 41.97% for difkrcnt scctions. The accuracy of our tcchniquc is quitc satisfactory. Bascd on thc rcsult of global rcgistration, 51 complctc mcgakaryocytc cclls in thosc sections havc bccn symbolically rcconstructcd. More dctails can be found in [ 151.
4. Discussion Thc pcrformancc of thc abovc mcntioncd corrcspondcncc finding tcchniquc can bc improvcd in scvcral ways in practical applications. In casc of most of objccts extending into scvcral consccutivc scctions, the profilc distributions in thosc scctions should havc somc rclations cvcn if thc scctions arc deformed. This likcncss may oftcn bc pcrccivcd whcn thc diffcrcncc in oricntation among sections has bccn corrcctcd. This implies, likc in thc two-stagc tcrnplalc matching tcchniqucs [18], a two stiigc pattcrn matching proccdurc can also bc cmploycd to takc advantagc of this likcncss. At thc first stagc, a rough transformation (translation and/or rotation of the wholc scction) applying thc lcast squarcs criterion (such as proposcd in [6]), but can be detcrmincd automatically as mentioncd abovc) can bc cmployed aftcr the rcgistration of scvcral pairs of corrcsponding
51
TABLE 1 Pattcm matching results. ~~
Scclion pair
# of cells
Correct
1021103 1031104 1041105 105/106 1061107 107/108 1081109 109/110 1101111 1111112 1121113 1131114 1141115 115/116
1331134 1341129 1291137 1371137 1371112 1121137 1371139 1391117 1171136 1361116 1161131 1311108 1081120 1201140
124 121 126 130 130 104 136 135 112 134 114 121 102 119
1786
1708
Sum
Avcrage
%
93.23 90.30 97.67 94.90 94.90 92.86 99.27 97.12 95.73 98.53 98.28 92.37 94.44 99.17 95.63
Error
9 13 3
7 7
8 1
4 5 2 2 10 6
1
78
%
6.77 9.70 2.23 5.10 5.10 7.14 0.73 2.88 4.27 1.47 1.72 7.63 5.56 0.83 4.37
profiles in adjaccnt sections. This transformation allows to reduce the search region bccausc it would cstablish a preliminary correspondcnce between two sections. Only those matching points falling into the search region will be considered as potential matching points. In many cases, this region will be quite small compared with the whole scction area. As a result, only a small number of patterns are to be constructed and to be tcstcd. In this way, thc calculations will be greatly reduccd. Another improvemcnt may be madc is to rcducc thc computation effort for tcsting no-match patterns. Since the cumulativc diffcrcnce mcasurcment has bccn employed, it is possible that some judgcmcnt can bc madc bcforc all thc matching patterns arc complctcly tcstcd. For example, whcn the diffcrcncc bctwcen two tcsting pattcrs is already big enough that continuing calculation would be worthlcss, thc Lcst for current patterns can bc stoppcd. In our expcricncc, the nurnbcr of potcntial matching points can bc much largcr than thc. numbcr of malchcd points, thc computation rcduction is also considcrable. Slatistically, the largcr the mismatch bctwccn two patterns, thc shortcr is the computation. Thc pattcrn matching tcchniquc has certain generality because the patterns for matching arc dynamically constructed, the pattern can be any size so it is suitablc for the cases whcrc thcrc is grcat variation of the patterns to be matchcd. Thc principal idca behind this tcchniquc is morc gcncral, i.c., to extract characteristic descriptions for the rcgislration and classification of pattcrns. From this point of vicw, some straightforward modifications to adapt this tcchniquc to dilfcrcnt situations can be easily madc. For cxamplc, wc can introduce the tcxturc information of image into thc pattcrn vector whcn such an information is availablc to match tcxturc imagcs. Also thc basis of match can bc not limited to
52
spatial patterns, it can be any group of atlributes associated with the pcrccption of patterns. Finally, we want to point out that the symbolic reconstruction based on correspondcnce finding is a very useful process in 3D image analysis. On one hand, following 3D pictorial rcconstruction bccomes possiblc based on the relationship among different parts of same objects. On thc other hand, certain analysis tasks, such as 3D objcct numbcr counting and 3D object volume measurement, become possible cvcn before 3D pictorial rcconstruction. In many 3D image analysis applications, this will give cnough information and savc a lot of processing efforts. We can say that the early proposed two lcvcl rcgistration approach is not only a short-cutting procedure for registration of serial sections, but also a useful conuibution to the whole 3D image analysis task.
Acknowledgements We would like to express our heartfelt thanks to Professor G. Cantrainc and the mcnibcrs of h e clcclronics group, as well as Dr. J. M. Paulus and the mernbcrs of thc hcmatology group, of LiCge University, Belgium. The support of the Netherlands’ Project Tcam for Computer Application (SPIN), Thrce-dimensional Image Analysis Projcct is also gratefully apprcciatcd.
References 1.
Jolmson EM, Capowski JJ. Principles of Reconstruction and Thrce-Dimensional Display of Scrial Sections using a Computer. In: RR Mize ed. The Microcomputer in Cell and Neuro-
biology Research. Elscvicr, 1985. Pratt WK. Digital I m g e Processing. John Wiley & Sons, 1978. Gentile AN, Harth A.The Alignmcnt of Serial Sections by Spatial Filtering. Comput Biomcd Res 1978; 11: 537-551. 4. Barncl DI, Silverman HF. A Class of Algorithms for Fast Digital h a g c Registration. IEEE Trans Computers 1972; C-21(2): 179-186. 5. Rosenfeld A. Image Pattcrn Rccognition. Proceeding o f l E E E 1981; 69(5): 596-605. 6. Chawla SD, Glass L, Frciwald S, Proctor JW.An Intcractive Coniputcr Graphic Systcm Tor 3-D Stcrcoscopic Rcconstruction from Scrial Sections: Analysis of Mctastatic Growth. Cornput Biol Mcd 1982; 12(3): 223-232. 7. Kropf N, Sobcl I, Lcvinthal C. Scrial Section Reconstruction Using CAKTOS in Thc Microcomputer. In: RR Mizc cd. Cell and Neurobiology Research. Elscvicr, 1985. 8. Mcrickcl M. 3D Rcconstruction: The registration Problcm. Compul Vision Graphics Image Process 1988; 42(2): 206-219. 9. Zhang YJ. A 3-D Irnagc Analysis Systcm: Quantitation of Mcgakaryocytc. Submitted to Cytomctry 1990. 10. Tou JT, Gonzalcz RC. Paltern Recognition Principles. Reading, MA: Addison-Wcsley, 1974. 11. Koscnfcld A, Kak AC. Digilal Picture Processing. Acadcmic Prcss, 1976.
2. 3.
53
12. Gonzalcz RC, Wintz PA. Digital Image Processing (second edition). Reading, MA: Addison-Wcslcy, 1987. 13. Aggarwal JK, Davis LS. Martin WN. Correspondence Processes in Dynamic Sccne Analysis. Proceeding of IEEE 1981: 69(5): 562-572. 14. Wang LX et al. IIandbook of Mathematics. Peking, 1979. 15. Zhang YJ.Development of a 3 - 0 computer image analysis system, application to megakaryocyte quantitation. Ph.D. dissertation, Libge University, Belgium, 1989. 16. Zar JH. Biostatistical analysis. Prentice-Hall Inc, 1984. 17. Pugachev VS. Theory of Random Functions. Reading, MA: Addison-Wesley, 1965. 18. Vandcrbrug GJ, Kosenfcld A. Two-Stage Template Matching. IEEE Trans Computers 1977; C-26(4): 384-393.