Shading correction and calibration in bacterial fluorescence measurement by image processing system

computer methods and programs in biomedicine

ELSEVIER

Computer Methods and Programs in Biomedicine 44 (1994) 61-67

Shading correction and calibration in bacterial fluorescence measurement by image processing system M.H.F. Wilkinson Laboratory for Medical Microbiology, University of Groningen, Oostersingel 59, 9713 EZ Groningen, The Netherlands Received 26 October 1993; revision received 17 January 1994; accepted 21 March 1994

Abstract

An image processing system with applications in bacterial (immuno-)fluorescence measurement has been developed. To reach quantitative results, correction for non-uniformities in system sensitivity, both as a function of time (calibration for drifts) and as a function of image coordinates (shading correction), is essential. Both problems can be handled simultaneously by acquiring images of a uniformly fluorescent, solid standard as a reference image. To measure bacterial fluorescence, the average fluorescence intensity of isolated areas of interest (the bacteria) is computed, and corrected using the reference images. Two shading correction methods are theoretically and experimentally compared: direct averaging in the corrected image, and (weighted and unweighted) averaging using the raw image and a separate shading image to determine the weights and correct for shading during the averaging. The latter method proved computationally 3.5-6.5 times faster on average and reduced propagation of truncation errors during computation, resulting in 40% less noise, for 8-bits/pixel images.

Keywords: Bacterial fluorescence; Quantitative microscopy; Image processing

1. Introduction

In recent years, there has been a rapidly growing interest in quantitative microscopy. Within this trend, an important focus is on fluorescence microscopy [1]. The ability to measure ion concentrations [2], membrane potentials [31 or antibody titres [4], to name but a few possibilities, is a powerful tool in the research of many biological processes at the microscopic level. In our laboratory, a microbiological digital image processing system has been * Corresponding author.

developed, with applications in both morphology and quantitative indirect immuno-fluorescence [5,6]. Designed as a low cost image processing system, it is based on personal computers equipped with image processor boards. Titres of serum antibodies against faecal flora or cultured bacteria can be determined with a high degree of accuracy using this method [4,7]. The system measures fluorescence by determining the average surface brightness of specified areas of interest (AOIs) thought to coincide with bacteria. To date, our method focuses solely on bacteria, but the method could equally apply to eukaryote cells, organelles or any other features in

0169-2607/94/$07.00 © 1994 Elsevier Science Ireland Ltd. All rights reserved SSDI 0169-2607(94)01480-4

M. H.F. Wilkinson / Comput. Methods Programs Biomed. 44 (1994) 61-67

62

the image. To acquire calibrated fluorescence data, a number of technical issues had to be resolved, some of which are dealt with in this article. Two different methods of computation of AOI average fluorescence level will be compared, regarding both noise propagation and computational efficiency. The first involves averaging the grey levels of all pixels in the area of interest in an image which has been corrected for shading and camera dark current. The second uses an image which has only been corrected for dark current, and computes both weighted and unweighted averages, using a shading reference image to determine the weights and to correct for shading during the averaging process. It will be shown that in AOI fluorescence level measurement, methods which use a separate shading image lead to a reduced propagation of truncation errors, and are more efficient computationally. The conclusions drawn from this comparison should be valid for a range of image processing systems based on personal computers equipped with 8-bits/pixei image processor boards.

in which (S - D) is the mean value of the shading mask, corrected for the dark background:

2. Computational methods and theory

An important source of noise is the digitizer (or quantizer). This noise is inherent to any quantizing process. The quantizer cannot reproduce any signal exactly; instead, signals within a small range are assigned to one of a series of'bins'. The output of the digitizer corresponds to the number of the bin the signal was put in. The more bins the digitizer has, the smaller each bin can be, and the smaller the quantization error becomes. Typically, video digitizers have 256 or 4096 bins, although larger and smaller values can be found. Quantization errors of a different sort can be found in the image processing within the computer. As most forms of image memory can only store a limited number of values per pixei (256 or 4096 are common), the results of computations are usually stored with this limited accuracy. Truncation of fractional parts of results can degrade the accuracy. On a system with 256 allowed pixel values, an error of 0.5% full scale can easily be made, which increases rapidly for very faint areas of the image. One obvious way of reducing these errors is increasing the number of bits per pixel. For each added bit, the round-off error is halved, so 12 or 16 bits

2.1. Shading correction All image sensors suffer from two fundamental problems: the sensitivity to light will vary as a function of image coordinates, and the response to absolute darkness will not be zero, and also vary as a function of image position. The former type of imperfection is usually called shading, the latter dark current. Both imperfections will vary with time, but if the system is at all stal~le, these variations will be slow. If the system is linear, these imperfections can be removed from the signal (or image) very simply, by measuring the sensitivity and dark current of each pixel and correcting other images using these data. Normally, the shading is measured by acquiring an image (S) of a uniformly fluorescent or luminescent target. The dark current is measured by acquiring an image (D) with the camera shut of from light. A fluorescent image F can then be corrected simply: C-

F-D S-D

- -

( S - D)

(1)

X-I

E /~_n\=

x=O

Y-|

~_~ s[x,y]- d[x,y] y-o

XY

(2)

in which X and Y are the width and height of the image measured in pixels. If we use a previously corrected shading mask, Eq. 1 becomes: F!

c-

s'

(s')

O)

in which S' is simply S - D, and F ' is F - D. Note that the factor (S') = (S - / 9 ) in the equations can be replaced by the intrinsic fluorescence surface brightness of the reference image, yielding a calibrated image, or brightness of an AOI.

2.2. Noise &troduced & the digit&er and image processor

M.H.F. Wilkinson/ Comput. Methods Programs Biomed. 44 (1994) 61-67

per pixel (instead of 8) would decrease this error by a factor of 16 or 256, but only at the expense of an increase in the required amount of memory. Many image processing boards do not support an arbitrarily large number of bits per pixel, so it would seem worthwhile to look to alternative possibilities to reduce this round off error.

2.3. Shading correction of areas of interest of N pixels In many cases we are only interested in the brightness of certain areas of interest. In our image prgcessing system these AOIs are bacteria, the fluorescence of which must be measured. Two methods have been tested. The first method involves simply taking an average of the pixels within the AOI, in the corrected image C. This average is simply: N-I

fAoI -- i= 0 N

-- (C)AoI

(4)

where we introduce the notation: N-I

Xi i=0 (X} AOI --

N

which is the average of X over the area of interest. An alternative approach is computing a weighted average, using S" (x,y) or S' 2(x,y) as weights, since it has been demonstrated that the values of C con-tain less noise if S' is high [8]. These weighted averages become: N-I

N-I S[ 2 C:

fAOI = (S')

i=0 N-1

] ~ S( 2 i-0

---- (St)

(S'F')AOI ( S ' 2)AOI

N-1

i=0 N-I

]~

JAOl = (S') s~

(5b)

In Eq. 5afAoi is just the average of pixel values in the AOI in image F ' over the average of the corresponding pixels in image S'. This would imply that if we only need the mean fluorescence of a number of AOIs, and do not need a fully corrected image (for storage or display) we can reduce the number of computations dramatically. To correct an entire image, of horizontal dimension X and vertical dimension Y, X Y divisions and multiplications are needed, and round off errors are introduced in every pixel before they are averaged. Unweighted averaging directly from the corrected image C costs N additions, one multiplication and one division. Using F and S', the computational cost of Eq. 4 is N additions, N + 1 divisions (and divide by zero checks!). The first form of weighted averaging using Eq. 5a costs 2N additions, one multiplication and one division. Finally, Eq. 5b needs 2N multiplications, 2N additions and one division. Since addition is much faster than multiplication or division, the gain in computational speed of Eq. 5a is obvious. Besides, Eqs. 5a and 5b can be implemented completely in integer mathemtaics so round off errors do not propagate at all. Alternatively Eq. 4 could be implemented in such a way that the explicit form of Ci is used (i.e. Eq. 3):

s: fAol = (S') -

63

i=0 N

F_' S'

(6)

i-0

---- ( S ' )

(F')AO~I (S ')AOI

(5a)

If this equation is implemented in (double precision) floating point mathematics, the truncation errors should be minimal (yet not zero). One should,

64

M.H.F. Wilkinson/ Comput. Methods Programs Biomed. 44 (1994) 61-67

however, bear in mind that floating point mathematics tends to be rather slower than integer mathematics on personal computers. The computational efficiency arguments assume that the computational burden lies on the central processing unit of the computer. If specialized hardware to perform these operations is present (specifically, image ratioing), it is probably more efficient to use such hardware, and to relieve the main processor of much of the computational burden. However, truncation error issues do still remain, and unless an adequate number of bits per pixel is used, they could add significantly to the total noise.

3. System and methods

3.1. Hardware The hardware used in our image processing systems consists of IBM-AT compatible computers (based on either 80286, 80386 or 80486) with at least 2 megabytes (MB) of memory and hard disks of 65-120 MB. These computers are equipped with an MVP-AT frame-grabber and image processing board (MATROX, Dorval, Canada) which performs much of the actual image processing. Image ratioing is not available in hardware and the digitizer is limited to 8 bits of resolution (256 levels). The camera in use is a Peltier-cooled, industrial CCD camera (Loral Fairchild CCD 5000/1, Loral Fairchild, Sunnyvale, CA). The Peltier element cools the CCD chip to 20°C below the ambient temperature, to reduce thermal noise from the CCD. A special purpose IBM-PC expansion board has been developed by the author [6], which allows software controlled increases in the exposure time from the usual 33 ms (at 30 frames/s) to more than 30 s, in increments of 33 ms. The expansion board also allows control over the automatic gain control (AGC) of the CCD camera, and over shutters mounted in front of the ultraviolet and normal light sources of the microscope used in our system. To switch the camera AGC on and off, the cameras have been modified slightly. Normally, the AGC is controlled by a jumper inside the camera control unit. In our modification the jumper's contacts have been connected to a plug mounted in the control unit's front. This plug is connected to a relay switch which is controlled by the computer. When the con-

tacts are closed (default in our system) the AGC is switched on, when open the AGC is switched off. The camera was mounted on a Leitz Orthoplan using a 5 0 x , NA = 1.00 PL Fluotar objective (Leitz, Wetzlar, Germany), with an HBO I00 high pressure mercury vapour lamp (OSRAM, Germany). The magnification on the screen of the image processors was 0.117 #m/pixel, using a 2.5 x photo-eyepiece (eyepiece projection). This resolution puts the system at, or just above the Nyquistfrequency, if the cut-off frequency of the optics is as good as that measured by Hiraoka et al. [9]. We can therefore expect to lose little spatial information. A uranyl glass fluorescence reference (Schott, Mainz, Germany), was used as calibration and shading image source. Uranyl glass is a frequent choice when calibrating the fluorescence of fluorescein iso-thiocyanate (FITC) stained slides, as its excitation and emission characteristics match those of FITC closely [10].

3.2. Software The programmes of the GRID system (Groningen Reduction of Image Data) were all developed in our laboratory, using mainly Microsoft Pascal 4.0, C 5.1 and, occasionally, assembler (Microsoft MASM). All programmes run under Microsoft MS-DOS, versions 2.0 and upwards. All procedures providing direct control over the MVP-AT boards were from the IMAGER-AT library, supplied by MATROX. For portability reasons, these procedures are never called directly, but are always addressed via an interface library, written by the authors. This layered approach does sacrifice some speed, but it has paid off well when the earlier software was ported from a different hardware platform, since only the interface library needed to be rewritten, not entire programmes. A number of procedures were written for the acquisition of reference images and correction of images. These procedures were linked to an extensible interpreter, called MEDEYES [11]. This allowed us to test the procedures under a variety of conditions. Once the procedures were tested, a short, special purpose program was written to perform all measurements and measure the execution time of each method.

M.H.F. Wilkinson/Comput. Methods Programs Biomed. 44 (1994) 61-67

3.3. Acquisition of reference and target images Raw dark current images were acquired by shutting off the camera from all light, and taking an exposure of 8 video frames (0.267 s). Dark current reference images were created from these raw images by multiple image averaging: 4 raw images were added and divided by 4. Raw shading images were acquired by focusing on the uranyl glass, taking an exposure of at most 8 frames (otherwise the digitizer saturated), and subtracting a raw dark background, taken at the same exposure time, from this image. Shading reference images were made by multiple image averaging, using 32 raw images. This reduces random noise in the resulting image. To measure the effect of different methods of correction, it is best to have images of known characteristics. The most obvious choice is the uniform fluorescence image itself. Images of the uranyl glass slide, acquired at an exposure time of 8 video frames (0.267 s), were used as a test target for shading correction. 3.4. Noise estimation Noise propagation in fluorescence measurement of AOIs was done by selecting a set of arbitrary AOIs, and computing the average brightness within these AOIs in the uranyl target images, using all methods outlined in the background section. As the intrinsic fluorescence of the uranyl glass is uniform, all differences in the measurements of average brightness can be considered noise. Therefore, root mean square (rms) noise was estimated as the standard deviation of the distribution of average AOI fluorescence values. 4. Results

The average fluorescence of 100 square areas of interest in a uranyl target image were measured for AOI surface areas of 4, 9, 16, 25, 36, 49, 64, 81,100, 121, and 144 pixeis. Unweighted averages computed from C (using Eq. 4), and from F ' and S' (using Eq. 6), and weighted averages, computed from F' and S' (using Eqs. 5a and 5b), were measured. The results are shown in Fig. 1. For AOIs of less than 50 pixels there is a marked decrease in rms error. For AOIs of more than 50 pixels, the rms errors are

65

0.80

0.60 Computed from C \

0.40 Computed from F' and S' 0.20

0.00 0

50

100

150

AOI Size

Fig. 1. RMS errors in AO! pixel averages (in grey levels) as a function of AOI size. The solid curve uses AO! averages computed from the shading corrected image C directly, the three dashed curves use AOI averages computed from the dark current corrected image F' and S ' . The curves for the two weighted and the unweighted averages overlap completely.

almost constant for all methods: 0.340 ± 0.003 for methods using F a n d S' and 0.474 ± 0.004 for the method using C. The ratio of these values is 1.394 ± 0.017. In all cases the methods using F' and S' proved superior to the method using C. The results for Eqs. 5a, 5b and 6 were identical down to less than 0.3%. Fig. 2a and b shows the total computing time for all methods, on both a 16-MHz 80286-, and a 33MHz 80486-based machine. The total computing time was measured for each of the groups of 100 AOIs, as 100 is about the average number of AOIs per image in our normal experiments. For the method using the corrected image C, the total computing time includes the time needed to correct the entire image for shading. On the 80286-based machine this was 14.55 s, and on the 80486-based machine 5.33 s. If this time is not included, the method using only C is twice as fast as the others. The most likely explanation is that the performance is limited by the I/O to the image processing board. The method using C only has to read the relevant pixels in one image, the others need to read from two (F' and S'). This would also explain the relatively slight boost in performance when an 80486 instead of an 80286 processor is used: only 40%.

M.H.F. Wilkinson / Comput. Methods Programs Biomed. 44 (1994) 61-67

66 20 ¸

unweighted: computed from C 15

aj

E

10

unweighted: computed

ic-

.---"

..-""

from F' and S' 5

~'Y~wNgthed:

computed from F' and S'

0 50

0

100

150

8 ¸ Computed from C 6 ¸

E I-

f

4 ~ .~ ~ - >

~

~

~

~S~

-~- -~- -~. -_~ ~

-~- ~

2 .~JJ"

Computed from F' and S'

,/ f 0

.

.

0

.

.

,

.

.

.

.

50

r

100

.

.

.

.

150

AOI Size

Fig. 2. Computing time (in seconds) for average fluorescence of 100 AOls as a function of AOI size (in pixels). (a) The results for a 16-MHz 80286-based computer. (b) The results for a 33MHz 80486-based computer. The solid curve uses unweighted averaging from C. The dashed curves use weighted and unweighted averages computed from F ' and S ' .

5. Discussion

Of the four methods tested in AOI fluorescence measurement, all methods using a separate shading image out-perform full image correction in noise performance by almost 40%. This difference can only be due to accumulation of round-off errors, as unweighted averaging using a separate shading image performs as well as the weighted methods. In fact, since no round off error is added by an integer mathemtaics implementation of Eqs. 5a and 5b at all, and the floating point implementation of Eq. 6 should introduce very little error, any differ-

ence between the results of these three methods can be attributed purely to the weights used. The fact that only slight differences between weighted and unweighted averages were found is probably due to the fact that the shading images were very smooth. Over a single AOI the weights did not differ by more than 2-3%. It should be noted that increasing the number of bits per pixel by 4 or 8, to 12 or 16 bits, for image C should yield a similar reduction of noise, as the truncation errors would be reduced by a factor of 16 or 256. This improvement would require twice as much image memory as the current implementation, and would not be supported by all our image processing and display devices. The speed advantage on our hardware of the methods using a separate shading image is evident from Fig. 2a and b. On the 80286-based machine, the performance of the unweighted method using F' and S' is slower than the weighted methods, due to the limited floating point mathematics performance of the machine. Using S(x,y) as weights is the fastest method, but the difference with the S' 2(x,y) method is hardly significant. The average size of the AOIs in our research is about 35 pixels. On average there are about 100 such AOIs in each field of view. This would mean that the method using S(x,y) as weights is about 6.5 times as fast as full image correction on the 80286, and 3.5 times as fast on the 80486, on average. This is quite a sizeable gain. It is also evident from these results that the gain in efficiency is highly dependent on the hardware used. The gains found here should be indicative of the gains to be expected for the large category of personal computer-based systems without hardware support for image ratioing. On systems with a fast histogram processor (as the MVP-AT used here), it could be advantageous to compute the sums in Eq. 5a from the histograms of the AOIs, but only if the AOIs are so large that the 256 additions and multiplications (for 8 bits per pixel) outweigh the N additions needed otherwise. This is not the case in our application. The main disadvantage of the use of a separate shading image is the need to store this image until the images are processed. In practice, the fluorescence images are processed the night after acquisition, so no great storage penalty is incurred. Some bookkeeping is

M.H.F. Wilkinson/Comput. Methods Programs Biomed. 44 (1994) 61-67

needed to ensure the correct shading image is used during analysis. At least two images must be stored in the image processor memory simultaneously. If this is not possible, the algorithm would probably become prohibitively slow, as swapping to disk, or main memory is very costly. As our processor boards can hold up to 4 images of 512 by 512 pixels, this is not a problem in our system. The final disadvantage is the need to read two images from disk instead of one. This adds some 3-4 s to the processing time on the slowest system. Our experience indicates that (in a properly climate controlled room), shading images need not be recorded more than once a day. This means that a shading image need only be read at the start of each analysis batch. A single batch contain some 60 images (in our case), so only 0.07 s is added per image. As has been stated before, the uranyl reference glass can also be used for calibration of the signal, and conversion into physical units [10]. Currently we have defined the intrinsic surface fluorescence brightness of the uranyl glass to be 10. This means that the fluorescence values given by our system are expressed in rather arbitrary 'uranyl units'. Calibration to molecular equivalent of soluble fluorescein (MESF) per square micrmeter is a next step, which could be done utilizing FITC-coated latex beads used in the calibration of fluorescence activated cell sorters (FACS) [12]. This future step in our calibration could well yield valuable estimates of the actual number of antibody molecules bound to the surface of bacteria. However, calibration using beads would not replace the current calibration scheme, because the FITC-coated beads do not cover the entire field of view, and can therefore not calibrate each pixel. The beads would be used to cross-calibrate, converting uranyl units into MESF//~m 2.

Acknowledgements This research was supported by the Dutch Foundation for Micro-Morphological Systems Development and the Institut fiir Microbiologie, Herborn-Dill, Germany.

67

References [1] H.J. Tanke, Does light microscopy have a future? J. Microsc. 155 (1989) 405-418. [2] R.Y. Tsien, Fluorescent indicators of ion concentrations, in Methods in Cell Biology, Vol. 30, Fluorescence Microscopy of Living Cells in Culture, Part B, eds. L.S. Taylor and Y.L. Wang, pp. 127-156 (Academic Press, San Diego, 1989). [3] Gross, D. and Loew, M.L., Fluorescent indicators of membrane potential: microspectrofluorimetry and imaging, in Methods in Cell Biology, Vol. 30, Fluorescence Microscopy of Living Cells in Culture, Part B, eds. L.S. Taylor and Y.L. Wang, pp. 193-219 (Academic Press, San Diego, 1989). [4] H.Z. Apperloo-Renkema, M.H.F. Wilkinson, D.G. Oenema, and D. Van der Waaij, Objective quantitation of serum antibody titres against enterobacteriaceae using indirect immunofluorescence, read by videocamera and image processing system, Med. Microbiol. lmmunol. 180 (1991) 93-100. [5] B.C. Meijer, G.J. Kootstra, and M.H.F. Wilkinson, A theoretical and practical investigation into the characterisation of bacterial species by image analysis, Binary Comput. Microbiol. 2 (1990) 21-31. [6] M.H.F. Wilkinson, G.J. Jansen, and D. Van der Waaij, Very low level fluorescence detection and imaging using a long exposure charge coupled device system, in Microscopic Imaging, Fluorescence Detection and Microinjection in Biotechnology, eds. P. Bach, P. Poole and J. Mottley, pp. 221-230 (Pergamom Press, London, 1993). [7] G.J. Jansen, M.H.F. Wilkinson, B. Deddens, and D. Van der Waaij, Statistical evaluation of an improved quantitative immunofluorescence method of measuring serum antibody levels directed against the intestinal flora, J. Microbiol. Methods 17 (1993) 137-144. [8] Z. Jericevic, B. Wiese, J. Bryan, and L.C. Smith, Validation of an imaging system: evaluate and validate a microscope imaging system for quantitative studies, in Methods in Cell Biology, Vol. 30, Fluorescence Microscopy of Living Cells in Culture, Part B, eds. L.S. Taylor and Y.L. Wang, pp. 48-83 (Academic Press, San Diego, 1989). [9] Y. Hiraoka, J.W. Sedat, and D.A. Agard, The use of a charge-coupled device for quantitative optical microscopy of biological structures, Science 238 (1987) 36-41. [10] C.E.D. Taylor and G.V. Heimer, Quantitative immunofluorescence studies, Ann. NY Acad. Sci. 254 (1975) 151-156. [11] B.C. Meijer, Medeyes: an extensible language for image analysis, Binary Comput. M icrobiol. 3 (1991)57-60. [12] R.F. Vogt, G.D. Cross, D.L. Philips, L.O. Henderson, and W.H. Hannon, Interlaboratory study of cellular fluorescence intensity measurement with fluorescein-labeled microbead standards, Cytometry 12 (1991 ) 525-536.