- Email: [email protected]
A focus checking technique for image analysis systems
0031 3203/81/070231 07 $02.00/0 Pergamon Press Ltd. C) 1981 Pattern Recognition Society
Pattern Reco@nition Vol. 14, Nos. I 6, pp. 231 237, 1981. Pnnted in Great Britain.
A FOCUS CHECKING TECHNIQUE FOR IMAGE ANALYSIS SYSTEMS JAMES H. TUCKER and M A R G A R E T S T A R K Medical Research Council, Clinical and Population Cytogenetics Unit, Western General Hospital, Crewe Road, Edinburgh, EH4 2XU (Received 9 January 1980; in revisedform 1 May 1980; receivedfor publication 22 December 1980)
Abstract - The paper describes a method of checking the accuracy of focus of images scanned with a raster scanner, by means of direct analysis of the data from a single scan. The parameter used is a measure of the high frequency response of the system to sharp edges in the image, and is obtained by comparing entries of tables of density differences between adjacent and near-adjacent pixels along scanlines. The technique has been incorporated into an autofocussing module of an automated cytology prescreening system, and has been found to give adequate warning of focus errors when scanning a wide range of normal and abnormal cytology specimens. Image
Cervical
Cytology
Prescreening
Focus
INTRODUCTION One of the primary requirements of a practical image analysis system is that the image must be focussed correctly before analysis commences. In many applications, particularly those such as automated cervical smear prescreening which involve the analysis of microscopic field images, this requirement can only be fulfilled by using complex and expensive optical, mechanical and electronic systems, or by the use of time-consuming software procedures. This paper describes a focus checking procedure which has been developed for use in an autofocussing module on the CERVISCAN image analysis system. CERVISCAN is an experimental simulation system for the development of techniques for the automated prescreening of cervical cytology specimens. Full details of the system have been given elsewhere31) Briefly, it comprises a computer controlled microscope, a Vidicon scanner, and a PDP9 mini computer. The computer is programmed to automatically scan cervical scrape specimens as a series of 220 x 220/lm fields, and to use densitometric and morphological image analysis tests to locate any abnormal cells. The pixel is 1 /~m, densitometric resolution 16 grey levels each of 0.04 o.d. units, and the microscope uses a Leitz x 16 (N.A. = 0.45) objective giving a depth of field of + 2 ,um. In previous CERVISCAN experiments, focus was maintained manually by watching a television monitor screen connected directly to the Vidicon scanner, and stopping the scan for manual readjustment whenever the image was seen to become defocussed. This method is obviously unsuitable for use in a full-speed prescreening system (in which very high field analysis rates must be maintained for economic reasons), and is
Autofocus
Highpass
in any case prone to error, and extremely tiring for the operator during long runs. For these reasons experiments were started to develop an autofocussing module for the system. Several different autofocussing systems have been described in the literature t2- 51 but these fall into two groups. The first group includes systems in which two auxiliary scanners are used at slightly different focal planes on either side of the main image scanner plane, and a focus correction signal is obtained from the difference between the two scanner signals. The second group includes those systems in which each image is scanned several times at different focal planes, and some feature (e.g. total integrated optical density) is measured which maximises when the image is correctly focussed. Both of these techniques were however considered unsuitable for direct implementation in CERVISCAN. The first was unacceptable beffause of the difficulty and cost of carrying out the necessary hardware modifications. The second method, although simple to implement, would substantially reduce the overall scan rate of the CERVISCAN to an unacceptable level because the system scan rate is at present limited largely by the scanner, and this method would entail several scans of each field. For this reason attention was turned to the development of some method of checking the focus of each image during analysis, so that a full focus search could be actuated only when necessary to correct focus errors. Individual field images from cervical scrapes vary widely both between specimens (because of different cell types and cell concentration) and within specimens (because of uneven cell distribution) and so a focus checking parameter must be insensitive to image content, as well as sensitive to focus errors. However, almost all specimens are made up predominantly of
231
232
JAMESH. TUCKERand MARGARETSTARK
individual cells containing darkly-stained nuclei with sharp, well defined boundaries. Since one of the major effects of focus errors is to cause loss of a sharpness across sharp edges in the image, the relative rate of change in scanner signals across such boundaries can be used to monitor the accuracy of focus. This is the principle of the FOCWATCH focus checking parameter described below.
FOCWATCH PARAMETER FOR FOCUS CHECKING
The FOCWATCH parameter is based on the highfrequency response of a raster scan image analysis system to any sharp edges present in the image. For simplicity and ease of computation, measurements are only made in the direction of scan-lines. FOCWATCH is computed from the differences between density values of adjacent and near adjacent pixel planes. Each pixei is examined, and two difference values are computed (Fig. 1). The first, D2, represents the difference over two pixels, and the second gives the difference over four pixels (D4). As each pixel is processed, two 16-entry histograms, p and q are incremented if the following conditions are fulfilled:
CURRENT LAST BUT LAST 1 NEXT ONE i i
--
m
.i.
./.
a
n > K1
p(n) = p(n) + 1
If n > K 1 and
D2/D4 > K2
q(n) = q(n) + 1
(K~ and K 2 a r e constants). After all pixels in the image have been analysed, the total number of useful D1 intervals is tested against the limit K3. If insufficient were seen, the scanner is stepped onto the next field. However, if sufficient were seen then FOCWATCH (F) is calculated from the histograms as follows: F = n(a)~
\p(x)J
where a is the set ofvalues between K ~and 16 for which p(x) > K 4 n(a) is the number of entries in set a
K 3 and K4 are constants Block diagrams of the FOCWATCH interval analysis and statistic testing routines are shown in Fig. 2. In the interval analysis routines, a circular stack is used to retain the optical densities of the four most recent pixels; the D2 and D4 values are computed from the stack entries. Any pixels for which D4 contains both positive and negative differences are ignored in order to prevent anomalous entries. At the end of each image, the number of'useful' D4 entries (i.e. those with D4 > KI) is calculated, and if sufficient were seen the value of F is computed and tested. In order to remove any specimen dependence, the test for individual fields is compared against a running average of all values so far obtained in the current specimen. In fields giving a value of F lower than a certain proportion (Ks) of this running average, a focus error flag is set.
RESULTS
J
(a)
SI (b)
If
,-,
D2 ~
Let D4 = n
(c)
Fig. 1. (a) Derivation of D2 and D4 optical density differences. D 2 = density difference between 'current' and 'last' pixels. D4 = density difference between 'next' and 'last-but one' pixels. (b) D2 and D4 intervals from a sharply-focussed edge. (c) D z and D4 intervals from a poorly-focussed edge.
Values for the various limits used in the FOCWATCH interval analysis and statistic testing are shown in Table 1. These were determined by examining many matrices of D2 and D4 values for different fields and by experiments with the overall system. Table 2 shows typical matrices obtained from when the same field is scanned firstly in focus, and then slightly out of focus. Two major changes are observed when the field becomes defocussed. Firstly, there is a general shift towards the origin because of the reduction in high D4 values. Secondly, the proportion of 'outlier' points in the high-D4, high D2 region (region A in Table 2) is reduced. It is this latter difference that the limits for K 1 and K 2 in equation 3 were selected to accentuate. The relative proportion of such outlier points generally increases with the value of D4.
A focus checking technique for image analysis systems
Figure 3 shows histograms o f p and q when the two fields shown are scanned in and out of focus. Although the two fields give very different histograms because of the different image content, they both show the same general behaviour when the images become defocussed. Figure 4 shows the field-by-field variation in the F O C W A T C H parameter obtained during multiplefield scans of the same two specimens at different focal planes. The in-focus values for F O C W A T C H are significantly higher in the specimen containing mostly small dark nuclei, than in the specimen containing large pale nuclei. Nevertheless, the values for in-focus fields are generally quite consistent within individual specimens, and change markedly when the images are defocussed by a significant amount. The lower limit for the number of D,, intervals required to produce an acceptable F O C W A T C H calculation (K3) represents a compromise between the reduction in statistical fluctuations in the relatively small q values and the need to carry out focus checks as frequently as possible, particularly in specimens containing few cells.
L
New Pixel
[ I
[
I I Ge, O2,~K2.O~ t
233
I
DISCUSSION
Add to q(n)
The F O C W A T C H technique differs from the earlier autofocussing parameters mainly because it is derived from two measurements obtained simultaneously from a single scan, rather than from one parameter measured in two or more scans. However it requires an image which contains a number of sharp edges across scan lines, a condition which is fulfilled by the great majority of cervical cytology specimens. The two major effects of image defocussing on the D2 vs D4 matrix (Fig. 3) are easily related to changes in appearance of the image. The shift of population towards the origin is created by the breakdown of many of the high-D4 intervals into two or more intervals of lower value. This is most apparent in specimens in which a high proportion of the objects are small and dark. The reduction in the number of high° D2 entries in the high D4 rows is caused by the lower risetime across sharp edges. This is more noticeable in high-D4 rows because it is only in these that the percentage digitisation resolution is sufficient to resolve such differences.
1
a
MAGE
ENO OF
Y
CALCULATEq (x/ for x=K,--~.16P(x)
]
where p'(x)>K4
I
Get F=Mean value
Table 1. Experimentally-determined values for limits used in FOCWATCH system
J Set Focuseuor Flag J I
L
Update"lc with F
b
I i
I
Fig. 2. Block diagrams of FOCWATCH programs. (a) Interval analysis routine. (b) Statistic testing routine.
Limit
Value
Kl K3
4 0.6 512
K4
50
K5
0.75
K2
Lower limit of D4 for p Lower limit of D2/D, for q Minimum number of total p entries required Minimum p(x) for which p(x)/q(x) is included in F F limit (of mean F value) below which a focus error signal is generated
234
JAMES H. TUCKER and MARGARET STARK
Table 2. Matrices ofD 2 vs D4 values for in-focus and out-of-focus scans of a cervical cell field D4 0 I
0
I I
2 I
3 I
4 i
5 I
6 1
7 I
8 1
9
I0
II I
12 I
15 I
14 I
15
p
q
I
I
[
I -259 277
u2
536
2 -
271
39
3
270
69
6
310
4 -
200
98
41
5 -
179 105
60
6 -
115 130 I01
40
69
345 5
344
[23
367 23 ,o
3 9 4 IO
7-
7 4 121 I10
8 -
25
89
97 102
57 I 5
9 -
5
29
51
84
68
6
28
67
60
31 113
2
8
18
56
30 115
8
5
2
2
11
5
2
tO
--
I]
--
2
~
410 36 377 7
2 I
5
259 22 2 0 5 13 26
107
/
12--
2
8 I
II
i
I
45-
i
[
14
5
3
I
I
2
I
I
I
I[
-
5-
I
15 D2 0
I
2
3
4
5
6
7
8
I
9
I0
II
I2
13
14
15
P 718
0 718 46053~
991 710
2
685
72
5
5 9 0 187
I0
787
4
361 5 0 6
47
714
5
155 315
1251
598
5
29 171 155 15 ]
552
2
I
165
I
45
-
6
-
I
-
5
i
m
6 D4
7 8
9 I0 II I2 15
4
44 5
8 3 55 22
15
I
4 I
I,
/ L_
14 15
The FOCWATCH parameter is based on the proportion of high D2/D,, entries at each D4 row, rather than the simpler and more obvious overall average D2/D4 ratio, because this was found to be a much more sensitive and image-independent measure of focus accuracy. As the image is defocussed, the average D2/D,, ratio is dominated by intervals in the increasing proportion of scan-lines which traverse oblique edges near the periphery of nuclei, thus giving rise to changes in the distribution in D4 intervals which usually mask any significant changes in the average. Conversely the count of entries with relatively high Dz/D,, ratios is dominated largely by scan-lines at or near the diameter of a nucleus where the most rapid density changes occur.
Although the basic FOCWATCH parameter has been shown to give a good indication of focus accuracy
in cervical cytology specimens, it suffers from two drawbacks. Firstly, it is not completely independent of image content: its value is substantially higher for focussed images of fields containing a high proportion of small, dark nuclei than for others containing mostly large, pale nuclei. In practice this difficulty has been overcome satisfactorily by means of the 'running average' within-specimen normalisation; in general successive fields from a specimen contain a very similar mixture of cell types, and so a satisfactory average value is quickly attained. The second drawback is that the number of outlier D2/D,, points on which the FOCWATCH parameter is based represents an extremely small statistical proportion of the total intervals counted, so that relatively large numbers of nuclei must be scanned to obtain an acceptable confidence level in the figure obtained. In scanty
A focus checking technique for image analysis systems ~
235
Q
~''.'6
"p
•
Q
~4, e e O ~
•
se
eLe t
g R i
*
i II td
~
11
% ~
*
L~
QJ
•
~
•
~ b
IMII
~,..,t • j
s
•
• e*
•
• i
o'
41" ~LPjL~
i•,..,
I¢ el.
..x •
• •
LB ALP
~
,j. ,,r
•
.,
•
,
,
,~ .
,
4
'
*..z_-%-
.~ e" .. •
o
l _
#M
el'
1.
bl c_
[_
600"
\
400-
p.
IN FOCI.~
750-
IN FOCUS
in
500
po
-30
i n -30
-20
200.
-20
250- 10
10 0
1
2
3
J5
6
7
I
8 9 10 11 12 13 14 15 16 n
0
0 !
2
3
5
6
7
S" 9
I0 11 12 13 14 1.5 16 n
1000-
FOCUS-40
FOCUS-40
600-
750-
400-
p.
qn
L_
2oo-
-30
500.
p.
30 o
-20 250-10
0
I
2 3
5
6
7
8
9 I0 II 12 13 14 15 16
0
n
1 2
oL
3 ,I 5
6 7
8 9
-20 - I0
0
lO 11 12 13 14 15 16 n
Fig. 3. Histograms ofp and q values for two typical cervical cytology image,s. (a) Micrographs of fields ( x 400) from specimens L0396 and L0397. (b) Histograms when images are correctly focussed. (c) Histograms when images are defocussed by 4 #m.
specimens this may require several fields to be scanned before a value is obtained, during which the focal accuracy can change significantly. The F O C W A T C H system has now been incorporated into a complete autofocussing unit in a CERVISCAN trial system. The autofoeussing unit is made up from four sub-units: (1) A focus predictor routine. This automatically adjusts the z axis position between each field step according to a linear ramp. The initial end-points are set manually before starting to scan a specimen, and
are then corrected automatically at the beginning and end of each subsequent swathe. (2) The F O C W A T C H focus checking routine. This checks the focal accuracy of each field before analysis commences, and issues an error signal if the FOCWATCH value is too low. (3) The focus correction routine. A focus error signal causes the analysis of the field to be abandoned, and the focus correction routine then automatically rescans the field at a series of different z axis positions to find the position of maximum integrated optical
236
JAMES H . TUCKER a n d MARGARET STARK
Z(O.lu Steps) • ..,..,,.,,,
+40
........
+20
-20 ..........
40
:J -6000
-6000
-4000
-3000
-2000
1000
"B,
SLIDE L0396
+2000 +3000 *40O0 X FIELD CENTRE POSITION (IJ)
+1000
15 FI
-6000
-5000
-4000
-3000
-2000
-I000
II
qO00
*2000
+3000
,4'ooo
X FIELO CENTREPOSITION(}1) SLIDE L0397
Fig. 4. Field-by-field variation for FOCWATCH over 1 swathe (50 fields) of the two specimens L0396 and L0397. density. (3) Interlocks are provided to prevent cycling on fields which contain insufficient image content. {4) The focus monitor routine. This prints a digital 'map' of the focus position for each scanned field, and at which autofocus scans were requested. This trial system has now been in use for a period of several months, carrying out 20,000-cell (or I cm x I cm} scans on routine cervical cytology specimens. At the time of writing, 59 specimens have been scanned in this way, and in only three cases (5 %} has the result been rejected because of obvious failure of the autofocus system. The main cause of failure in these specimens was a very uneven distribution of cells, with large areas of the slide containing too few cells for adequate focus checking or focus correction. Although the FOCWATCH system has so far been implemented entirely in software, the analysis time could be considerably reduced by the use of relatively simple hardware designed to carry out the interval analysis and table construction directly from the digitised scanner output signal. The subsequent parameter calculation could then be carried out from the p and q values with very little time overhead. CONCLUSION
A method of checking the accuracy of focus of scanned images directly from the output data from a
raster scanner has been developed and tested. The method is relatively quick and inexpensive, and gives adequate warning of focussing errors when scanning the majority of cervical cytology specimens. The general concept may well be suited to a number of other image analysis applications.
SUMMARY
The paper describes a method of checking the accuracy of focus of images scanned with a raster scanner, by means of direct analysis of the data from a single scan. The algorithm was designed for use in a computer image analysis system for the automated analysis of cervical cell images, an application which is characterised by the wide differences in the content of successive images. The algorithm operates by measuring the rate of change of scanner signal across sharp edges in the image. The rate of change is measured by comparing the density difference between adjacent and nearadjacent pixels in the scan-line data. As each image point is analysed in turn, two histograms are constructed containing the number of intervals fulfilling certain criteria, and these are used to compute a "FOCWATCH' value for the data, provided sufficient
A focus checking technique for image analysis systems intervals have been processed. The resulting value is compared with the running average for the current specimen, and a refocussing signal is issued whenever the value is less than a predetermined proportion of the average. Results are presented showing the effect of applying this procedure to typical cervical smear fields. Wide differences in the F O C W A T C H values are obtained when the images are defocussed, although the in-focus values for different fields are not necessarily similar. The method has been incorporated in a complete autofocus module in the experimental system, and has given excellent results.
REFERENCES 1. J. H. Tucker, Cerviscan - - an image analysis system for experiments in automatic cervical smear screening, Comp. Biomed. Res. 9, 93-107 (1976). 2. J. E. Green, A practical application of computer pattern recognition research. The Abbott ADCS00 Differential Classifier, J. Histochem. Cytochem. 27, 160-173 (1979). 3. M. A. Kujoory, B. H. Mayall and M. L. Mendelsohn, Focus-assist device for a flying spot microscope, IEEE Trans. bio-med. Electron. 20, 126-132 (1973). 4. The LARC Differential White Blood Cell Analyzer. Coming Glass Corp., Medfield, Mass., U.S.A. 5. D.C. Mason and D. K. Green, Automatic focussing of a computer-controlled microscope, IEEE Trans. bio-med. Electron. 22, 312-317 (1975).
About the Author - - JIM TUCKERwas born in Geneva, Switzerland in 1939. In 1956 he joined Associated
Electrical Industries Ltd., as a technician apprentice, and subsequently obtained a B.Sc.(Tech.) degree in electrical engineering from City University, London, (1963) and a Ph.D. degree from Cambridge University (1968). His major research interests at that time were the application of tunnel diodes to high-speed scratchpad memories for computers, and the system design of hierarchical memory systems. In 1971 he joined the Medical Research Council Clinical & Population Cytogenetics Unit, Edinburgh and has since worked on the application of computer pattern recognition and image processing techniques to the automated screening of cervical smears. MARGARETSTARKwas born in Falkirk, Scotland. She is currently pursuing a Higher National Certificate course in Computer studies at Napier College, Edinburgh. She has been employed by the Medical Research Council since 1972, where she has worked on the automated analysis of cervical smears, and on chromosome analysis using flow system techniques.
About the Author
237
Pattern Reco@nition Vol. 14, Nos. I 6, pp. 231 237, 1981. Pnnted in Great Britain.
A FOCUS CHECKING TECHNIQUE FOR IMAGE ANALYSIS SYSTEMS JAMES H. TUCKER and M A R G A R E T S T A R K Medical Research Council, Clinical and Population Cytogenetics Unit, Western General Hospital, Crewe Road, Edinburgh, EH4 2XU (Received 9 January 1980; in revisedform 1 May 1980; receivedfor publication 22 December 1980)
Abstract - The paper describes a method of checking the accuracy of focus of images scanned with a raster scanner, by means of direct analysis of the data from a single scan. The parameter used is a measure of the high frequency response of the system to sharp edges in the image, and is obtained by comparing entries of tables of density differences between adjacent and near-adjacent pixels along scanlines. The technique has been incorporated into an autofocussing module of an automated cytology prescreening system, and has been found to give adequate warning of focus errors when scanning a wide range of normal and abnormal cytology specimens. Image
Cervical
Cytology
Prescreening
Focus
INTRODUCTION One of the primary requirements of a practical image analysis system is that the image must be focussed correctly before analysis commences. In many applications, particularly those such as automated cervical smear prescreening which involve the analysis of microscopic field images, this requirement can only be fulfilled by using complex and expensive optical, mechanical and electronic systems, or by the use of time-consuming software procedures. This paper describes a focus checking procedure which has been developed for use in an autofocussing module on the CERVISCAN image analysis system. CERVISCAN is an experimental simulation system for the development of techniques for the automated prescreening of cervical cytology specimens. Full details of the system have been given elsewhere31) Briefly, it comprises a computer controlled microscope, a Vidicon scanner, and a PDP9 mini computer. The computer is programmed to automatically scan cervical scrape specimens as a series of 220 x 220/lm fields, and to use densitometric and morphological image analysis tests to locate any abnormal cells. The pixel is 1 /~m, densitometric resolution 16 grey levels each of 0.04 o.d. units, and the microscope uses a Leitz x 16 (N.A. = 0.45) objective giving a depth of field of + 2 ,um. In previous CERVISCAN experiments, focus was maintained manually by watching a television monitor screen connected directly to the Vidicon scanner, and stopping the scan for manual readjustment whenever the image was seen to become defocussed. This method is obviously unsuitable for use in a full-speed prescreening system (in which very high field analysis rates must be maintained for economic reasons), and is
Autofocus
Highpass
in any case prone to error, and extremely tiring for the operator during long runs. For these reasons experiments were started to develop an autofocussing module for the system. Several different autofocussing systems have been described in the literature t2- 51 but these fall into two groups. The first group includes systems in which two auxiliary scanners are used at slightly different focal planes on either side of the main image scanner plane, and a focus correction signal is obtained from the difference between the two scanner signals. The second group includes those systems in which each image is scanned several times at different focal planes, and some feature (e.g. total integrated optical density) is measured which maximises when the image is correctly focussed. Both of these techniques were however considered unsuitable for direct implementation in CERVISCAN. The first was unacceptable beffause of the difficulty and cost of carrying out the necessary hardware modifications. The second method, although simple to implement, would substantially reduce the overall scan rate of the CERVISCAN to an unacceptable level because the system scan rate is at present limited largely by the scanner, and this method would entail several scans of each field. For this reason attention was turned to the development of some method of checking the focus of each image during analysis, so that a full focus search could be actuated only when necessary to correct focus errors. Individual field images from cervical scrapes vary widely both between specimens (because of different cell types and cell concentration) and within specimens (because of uneven cell distribution) and so a focus checking parameter must be insensitive to image content, as well as sensitive to focus errors. However, almost all specimens are made up predominantly of
231
232
JAMESH. TUCKERand MARGARETSTARK
individual cells containing darkly-stained nuclei with sharp, well defined boundaries. Since one of the major effects of focus errors is to cause loss of a sharpness across sharp edges in the image, the relative rate of change in scanner signals across such boundaries can be used to monitor the accuracy of focus. This is the principle of the FOCWATCH focus checking parameter described below.
FOCWATCH PARAMETER FOR FOCUS CHECKING
The FOCWATCH parameter is based on the highfrequency response of a raster scan image analysis system to any sharp edges present in the image. For simplicity and ease of computation, measurements are only made in the direction of scan-lines. FOCWATCH is computed from the differences between density values of adjacent and near adjacent pixel planes. Each pixei is examined, and two difference values are computed (Fig. 1). The first, D2, represents the difference over two pixels, and the second gives the difference over four pixels (D4). As each pixel is processed, two 16-entry histograms, p and q are incremented if the following conditions are fulfilled:
CURRENT LAST BUT LAST 1 NEXT ONE i i
--
m
.i.
./.
a
n > K1
p(n) = p(n) + 1
If n > K 1 and
D2/D4 > K2
q(n) = q(n) + 1
(K~ and K 2 a r e constants). After all pixels in the image have been analysed, the total number of useful D1 intervals is tested against the limit K3. If insufficient were seen, the scanner is stepped onto the next field. However, if sufficient were seen then FOCWATCH (F) is calculated from the histograms as follows: F = n(a)~
\p(x)J
where a is the set ofvalues between K ~and 16 for which p(x) > K 4 n(a) is the number of entries in set a
K 3 and K4 are constants Block diagrams of the FOCWATCH interval analysis and statistic testing routines are shown in Fig. 2. In the interval analysis routines, a circular stack is used to retain the optical densities of the four most recent pixels; the D2 and D4 values are computed from the stack entries. Any pixels for which D4 contains both positive and negative differences are ignored in order to prevent anomalous entries. At the end of each image, the number of'useful' D4 entries (i.e. those with D4 > KI) is calculated, and if sufficient were seen the value of F is computed and tested. In order to remove any specimen dependence, the test for individual fields is compared against a running average of all values so far obtained in the current specimen. In fields giving a value of F lower than a certain proportion (Ks) of this running average, a focus error flag is set.
RESULTS
J
(a)
SI (b)
If
,-,
D2 ~
Let D4 = n
(c)
Fig. 1. (a) Derivation of D2 and D4 optical density differences. D 2 = density difference between 'current' and 'last' pixels. D4 = density difference between 'next' and 'last-but one' pixels. (b) D2 and D4 intervals from a sharply-focussed edge. (c) D z and D4 intervals from a poorly-focussed edge.
Values for the various limits used in the FOCWATCH interval analysis and statistic testing are shown in Table 1. These were determined by examining many matrices of D2 and D4 values for different fields and by experiments with the overall system. Table 2 shows typical matrices obtained from when the same field is scanned firstly in focus, and then slightly out of focus. Two major changes are observed when the field becomes defocussed. Firstly, there is a general shift towards the origin because of the reduction in high D4 values. Secondly, the proportion of 'outlier' points in the high-D4, high D2 region (region A in Table 2) is reduced. It is this latter difference that the limits for K 1 and K 2 in equation 3 were selected to accentuate. The relative proportion of such outlier points generally increases with the value of D4.
A focus checking technique for image analysis systems
Figure 3 shows histograms o f p and q when the two fields shown are scanned in and out of focus. Although the two fields give very different histograms because of the different image content, they both show the same general behaviour when the images become defocussed. Figure 4 shows the field-by-field variation in the F O C W A T C H parameter obtained during multiplefield scans of the same two specimens at different focal planes. The in-focus values for F O C W A T C H are significantly higher in the specimen containing mostly small dark nuclei, than in the specimen containing large pale nuclei. Nevertheless, the values for in-focus fields are generally quite consistent within individual specimens, and change markedly when the images are defocussed by a significant amount. The lower limit for the number of D,, intervals required to produce an acceptable F O C W A T C H calculation (K3) represents a compromise between the reduction in statistical fluctuations in the relatively small q values and the need to carry out focus checks as frequently as possible, particularly in specimens containing few cells.
L
New Pixel
[ I
[
I I Ge, O2,~K2.O~ t
233
I
DISCUSSION
Add to q(n)
The F O C W A T C H technique differs from the earlier autofocussing parameters mainly because it is derived from two measurements obtained simultaneously from a single scan, rather than from one parameter measured in two or more scans. However it requires an image which contains a number of sharp edges across scan lines, a condition which is fulfilled by the great majority of cervical cytology specimens. The two major effects of image defocussing on the D2 vs D4 matrix (Fig. 3) are easily related to changes in appearance of the image. The shift of population towards the origin is created by the breakdown of many of the high-D4 intervals into two or more intervals of lower value. This is most apparent in specimens in which a high proportion of the objects are small and dark. The reduction in the number of high° D2 entries in the high D4 rows is caused by the lower risetime across sharp edges. This is more noticeable in high-D4 rows because it is only in these that the percentage digitisation resolution is sufficient to resolve such differences.
1
a
MAGE
ENO OF
Y
CALCULATEq (x/ for x=K,--~.16P(x)
]
where p'(x)>K4
I
Get F=Mean value
Table 1. Experimentally-determined values for limits used in FOCWATCH system
J Set Focuseuor Flag J I
L
Update"lc with F
b
I i
I
Fig. 2. Block diagrams of FOCWATCH programs. (a) Interval analysis routine. (b) Statistic testing routine.
Limit
Value
Kl K3
4 0.6 512
K4
50
K5
0.75
K2
Lower limit of D4 for p Lower limit of D2/D, for q Minimum number of total p entries required Minimum p(x) for which p(x)/q(x) is included in F F limit (of mean F value) below which a focus error signal is generated
234
JAMES H. TUCKER and MARGARET STARK
Table 2. Matrices ofD 2 vs D4 values for in-focus and out-of-focus scans of a cervical cell field D4 0 I
0
I I
2 I
3 I
4 i
5 I
6 1
7 I
8 1
9
I0
II I
12 I
15 I
14 I
15
p
q
I
I
[
I -259 277
u2
536
2 -
271
39
3
270
69
6
310
4 -
200
98
41
5 -
179 105
60
6 -
115 130 I01
40
69
345 5
344
[23
367 23 ,o
3 9 4 IO
7-
7 4 121 I10
8 -
25
89
97 102
57 I 5
9 -
5
29
51
84
68
6
28
67
60
31 113
2
8
18
56
30 115
8
5
2
2
11
5
2
tO
--
I]
--
2
~
410 36 377 7
2 I
5
259 22 2 0 5 13 26
107
/
12--
2
8 I
II
i
I
45-
i
[
14
5
3
I
I
2
I
I
I
I[
-
5-
I
15 D2 0
I
2
3
4
5
6
7
8
I
9
I0
II
I2
13
14
15
P 718
0 718 46053~
991 710
2
685
72
5
5 9 0 187
I0
787
4
361 5 0 6
47
714
5
155 315
1251
598
5
29 171 155 15 ]
552
2
I
165
I
45
-
6
-
I
-
5
i
m
6 D4
7 8
9 I0 II I2 15
4
44 5
8 3 55 22
15
I
4 I
I,
/ L_
14 15
The FOCWATCH parameter is based on the proportion of high D2/D,, entries at each D4 row, rather than the simpler and more obvious overall average D2/D4 ratio, because this was found to be a much more sensitive and image-independent measure of focus accuracy. As the image is defocussed, the average D2/D,, ratio is dominated by intervals in the increasing proportion of scan-lines which traverse oblique edges near the periphery of nuclei, thus giving rise to changes in the distribution in D4 intervals which usually mask any significant changes in the average. Conversely the count of entries with relatively high Dz/D,, ratios is dominated largely by scan-lines at or near the diameter of a nucleus where the most rapid density changes occur.
Although the basic FOCWATCH parameter has been shown to give a good indication of focus accuracy
in cervical cytology specimens, it suffers from two drawbacks. Firstly, it is not completely independent of image content: its value is substantially higher for focussed images of fields containing a high proportion of small, dark nuclei than for others containing mostly large, pale nuclei. In practice this difficulty has been overcome satisfactorily by means of the 'running average' within-specimen normalisation; in general successive fields from a specimen contain a very similar mixture of cell types, and so a satisfactory average value is quickly attained. The second drawback is that the number of outlier D2/D,, points on which the FOCWATCH parameter is based represents an extremely small statistical proportion of the total intervals counted, so that relatively large numbers of nuclei must be scanned to obtain an acceptable confidence level in the figure obtained. In scanty
A focus checking technique for image analysis systems ~
235
Q
~''.'6
"p
•
Q
~4, e e O ~
•
se
eLe t
g R i
*
i II td
~
11
% ~
*
L~
QJ
•
~
•
~ b
IMII
~,..,t • j
s
•
• e*
•
• i
o'
41" ~LPjL~
i•,..,
I¢ el.
..x •
• •
LB ALP
~
,j. ,,r
•
.,
•
,
,
,~ .
,
4
'
*..z_-%-
.~ e" .. •
o
l _
#M
el'
1.
bl c_
[_
600"
\
400-
p.
IN FOCI.~
750-
IN FOCUS
in
500
po
-30
i n -30
-20
200.
-20
250- 10
10 0
1
2
3
J5
6
7
I
8 9 10 11 12 13 14 15 16 n
0
0 !
2
3
5
6
7
S" 9
I0 11 12 13 14 1.5 16 n
1000-
FOCUS-40
FOCUS-40
600-
750-
400-
p.
qn
L_
2oo-
-30
500.
p.
30 o
-20 250-10
0
I
2 3
5
6
7
8
9 I0 II 12 13 14 15 16
0
n
1 2
oL
3 ,I 5
6 7
8 9
-20 - I0
0
lO 11 12 13 14 15 16 n
Fig. 3. Histograms ofp and q values for two typical cervical cytology image,s. (a) Micrographs of fields ( x 400) from specimens L0396 and L0397. (b) Histograms when images are correctly focussed. (c) Histograms when images are defocussed by 4 #m.
specimens this may require several fields to be scanned before a value is obtained, during which the focal accuracy can change significantly. The F O C W A T C H system has now been incorporated into a complete autofocussing unit in a CERVISCAN trial system. The autofoeussing unit is made up from four sub-units: (1) A focus predictor routine. This automatically adjusts the z axis position between each field step according to a linear ramp. The initial end-points are set manually before starting to scan a specimen, and
are then corrected automatically at the beginning and end of each subsequent swathe. (2) The F O C W A T C H focus checking routine. This checks the focal accuracy of each field before analysis commences, and issues an error signal if the FOCWATCH value is too low. (3) The focus correction routine. A focus error signal causes the analysis of the field to be abandoned, and the focus correction routine then automatically rescans the field at a series of different z axis positions to find the position of maximum integrated optical
236
JAMES H . TUCKER a n d MARGARET STARK
Z(O.lu Steps) • ..,..,,.,,,
+40
........
+20
-20 ..........
40
:J -6000
-6000
-4000
-3000
-2000
1000
"B,
SLIDE L0396
+2000 +3000 *40O0 X FIELD CENTRE POSITION (IJ)
+1000
15 FI
-6000
-5000
-4000
-3000
-2000
-I000
II
qO00
*2000
+3000
,4'ooo
X FIELO CENTREPOSITION(}1) SLIDE L0397
Fig. 4. Field-by-field variation for FOCWATCH over 1 swathe (50 fields) of the two specimens L0396 and L0397. density. (3) Interlocks are provided to prevent cycling on fields which contain insufficient image content. {4) The focus monitor routine. This prints a digital 'map' of the focus position for each scanned field, and at which autofocus scans were requested. This trial system has now been in use for a period of several months, carrying out 20,000-cell (or I cm x I cm} scans on routine cervical cytology specimens. At the time of writing, 59 specimens have been scanned in this way, and in only three cases (5 %} has the result been rejected because of obvious failure of the autofocus system. The main cause of failure in these specimens was a very uneven distribution of cells, with large areas of the slide containing too few cells for adequate focus checking or focus correction. Although the FOCWATCH system has so far been implemented entirely in software, the analysis time could be considerably reduced by the use of relatively simple hardware designed to carry out the interval analysis and table construction directly from the digitised scanner output signal. The subsequent parameter calculation could then be carried out from the p and q values with very little time overhead. CONCLUSION
A method of checking the accuracy of focus of scanned images directly from the output data from a
raster scanner has been developed and tested. The method is relatively quick and inexpensive, and gives adequate warning of focussing errors when scanning the majority of cervical cytology specimens. The general concept may well be suited to a number of other image analysis applications.
SUMMARY
The paper describes a method of checking the accuracy of focus of images scanned with a raster scanner, by means of direct analysis of the data from a single scan. The algorithm was designed for use in a computer image analysis system for the automated analysis of cervical cell images, an application which is characterised by the wide differences in the content of successive images. The algorithm operates by measuring the rate of change of scanner signal across sharp edges in the image. The rate of change is measured by comparing the density difference between adjacent and nearadjacent pixels in the scan-line data. As each image point is analysed in turn, two histograms are constructed containing the number of intervals fulfilling certain criteria, and these are used to compute a "FOCWATCH' value for the data, provided sufficient
A focus checking technique for image analysis systems intervals have been processed. The resulting value is compared with the running average for the current specimen, and a refocussing signal is issued whenever the value is less than a predetermined proportion of the average. Results are presented showing the effect of applying this procedure to typical cervical smear fields. Wide differences in the F O C W A T C H values are obtained when the images are defocussed, although the in-focus values for different fields are not necessarily similar. The method has been incorporated in a complete autofocus module in the experimental system, and has given excellent results.
REFERENCES 1. J. H. Tucker, Cerviscan - - an image analysis system for experiments in automatic cervical smear screening, Comp. Biomed. Res. 9, 93-107 (1976). 2. J. E. Green, A practical application of computer pattern recognition research. The Abbott ADCS00 Differential Classifier, J. Histochem. Cytochem. 27, 160-173 (1979). 3. M. A. Kujoory, B. H. Mayall and M. L. Mendelsohn, Focus-assist device for a flying spot microscope, IEEE Trans. bio-med. Electron. 20, 126-132 (1973). 4. The LARC Differential White Blood Cell Analyzer. Coming Glass Corp., Medfield, Mass., U.S.A. 5. D.C. Mason and D. K. Green, Automatic focussing of a computer-controlled microscope, IEEE Trans. bio-med. Electron. 22, 312-317 (1975).
About the Author - - JIM TUCKERwas born in Geneva, Switzerland in 1939. In 1956 he joined Associated
Electrical Industries Ltd., as a technician apprentice, and subsequently obtained a B.Sc.(Tech.) degree in electrical engineering from City University, London, (1963) and a Ph.D. degree from Cambridge University (1968). His major research interests at that time were the application of tunnel diodes to high-speed scratchpad memories for computers, and the system design of hierarchical memory systems. In 1971 he joined the Medical Research Council Clinical & Population Cytogenetics Unit, Edinburgh and has since worked on the application of computer pattern recognition and image processing techniques to the automated screening of cervical smears. MARGARETSTARKwas born in Falkirk, Scotland. She is currently pursuing a Higher National Certificate course in Computer studies at Napier College, Edinburgh. She has been employed by the Medical Research Council since 1972, where she has worked on the automated analysis of cervical smears, and on chromosome analysis using flow system techniques.
About the Author
237