- Email: [email protected]
The statistical analysis of plant part appearance — a review
Computers and Electronics in Agriculture 31 (2001) 169– 190 www.elsevier.com/locate/compag
The statistical analysis of plant part appearance — a review Graham W. Horgan * Biomathematics and Statistics Scotland, Rowett Research Institute, Aberdeen AB21 9SB, Scotland, UK Received 8 June 2000; received in revised form 31 October 2000; accepted 27 November 2000
Abstract Visual appearance is widely used in assessing the variety or cultivar of crop samples, and in biological taxonomy in general. With computer storage capabilities becoming more easily accessible, image databases of plant specimens can be readily constructed. To make full use of this powerful facility, an important task is to compare different images, or to summarise them. This review addresses the question of how this may be done, given that images are stored as typically 105 –107 highly structured measurements. Image analysis is a wide subject area, and this review covers those parts of it which are relevant to this task. The task is subdivided into four (2 ×2) categories according to whether we are interested in outline shape only or in colour details, and whether the global position of object details is important, or only their local distribution. We refer to these approaches as local shape, global shape, colour distribution and colour detail (eigenimage analysis). As far as possible, we avoid a priori ideas regarding what features of objects are important, and seek methods which capture the full variability of their appearance. The techniques are illustrated by reference to work on carrots. © 2001 Elsevier Science B.V. All rights reserved. Keywords: Image analysis; Eigenimage; Eigenshape; Discriminant analysis; Image matching; Crop cultivars; Morphology
1. Introduction Visual appearance is widely used in assessing the variety or cultivar of crop samples, and in biological taxonomy in general. For some characteristics, objective * Corresponding author. Fax: + 44-1224-716608. E-mail address: [email protected] (G.W. Horgan). 0168-1699/01/$ - see front matter © 2001 Elsevier Science B.V. All rights reserved. PII: S0168-1699(00)00190-3
170
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
accurate measurement is readily made. Size or distance between well defined features is the best example of this. For others, an objective definition is possible, but objective measurement presents difficulties. For example, colour can in theory be measured by suitable spectrophotometric equipment, but such tools can be expensive and time consuming to use, and so colour is often assessed subjectively. Likewise, the proportion of a leaf or other plant part affected by fungi, or some other problem, is well defined, but usually estimated subjectively. It is well known that there can be significant differences between different assessors, and even a single assessor is influenced by a carry-over effect from earlier specimens. Finally, some important characteristics, such as the degree to which a leaf is serrated, are usually not even objectively defined. With computer storage capabilities becoming more easily accessible, image databases of plant specimens can be readily contructed. To make full use of this powerful facility, an important task is to compare different images or to summarise them. A particularly useful ability would be to find the best match between a new image specimen and those in the database. As images are just numerical data, albeit very highly structured, it is in theory possible for all these tasks to be done automatically and objectively. We are concerned in this paper with the statistical methods available for the study of object appearance. We wish to quantify numerically distinguishing characteristics between different groups of plants. A broad range of techniques is called for. Our aim is to review these approaches, and illustrate their role in crop variety recognition. Although we focus on this horticultural application, the ideas are of relevance to the wider research topic of image retrieval. This is the study of managing and searching databases of images using image content rather than subjective, text-based indexing. For a flavour of this work, see Choubey and Raghavan (1997), Mehtre et al. (1997), Huang and Huang (1998). Much of their work is aimed at image databases which are very wide in scope, such as a library or publishing house might hold, and where the image criteria sought can be very general or abstract (for example, find an image representing ‘strength’). We will be concerned with the more specialised applications of image comparison, where the class of object is known (for example, the set of images to be searched are known to be varieties of carrots) and the differences between varieties are subtle (for example, how tapered the carrots are near the tip). In many applications, information about object appearance will come from two-dimensional images produced under ordinary visible light. Other image formation strategies, using different radiation sources, or fluoroscopy could be used. Types of imaging could be envisaged which would give three-dimensional information. This is much more demanding computationally, and is not considered here.
1.1. Definitions An image will be taken to be a two-dimensional array of pixel intensities fij, i = 1…m; j= 1…n. It is now usual to work with colour images, in which case a red, green and blue intensity is specified for each pixel. Fig. 1 shows nine images of
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
171
Brussels sprouts plants. Sometimes we will work with outline silhouettes only, in which case fij will take values of 0 and 1 only. Sometimes it may be convenient to handle such outlines by listing positions on the outline, pixels within the object having neighbouring pixels without. Approaches to object characterisation will be influenced largely by which aspects are considered informative and which are a nuisance. Position and orientation are usually unimportant. The photographer could have positioned the plant differently in the image or stood in a different position and the object appearance would be unchanged. A reflection (mirror image) of a plant would not usually be considered different from the original. Size may or may not be important. It can only be used if we know the scale of each image. Many plant parts are physically flexible. The leaves on the Brussels sprouts plants can be moved with respect to each other. The crown of a tree continually changes shape on a windy day. The leaves of plants can be moved, but are not elastic. For plant parts such as these, features are needed which are invariant under this flexibility. The object features discussed will be of four types, depending on whether (i) we are dealing with outline shapes only or with the variation in brightness or colour within the outline, and (ii) whether we are treating the object as rigid (so that global features can be considered), or flexible. Fig. 2 shows the terminology we will use for
Fig. 1. Images of three varieties of Brussels sprouts plants. Each column is a separate variety. From left to right the varieties are Target (T), Heracles (H) and Golfer (G).
172
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Fig. 2. Classification of approaches to object characterisation.
these situations. Although flexibility or rigidity are considered as attributes of the plant, the use of outline shape and colour detail will be considered as complementary; they can both provide information. An important aim in our object characterisation is that where possible we wish to avoid a priori specification of what features should be examined. Although botanists already list many features for measurement on plants for taxonomic purposes, we concentrate here on general appearance. The next four sections of the paper will describe the four approaches in Fig. 2. Section 2 looks at local shape and Section 3 at global shape. Section 4 considers colour distributions. Section 5 shows how a principal component approach, termed eigenimage analysis, can be used to condense the colour details within a shape outline. Section 6 concludes with a brief discussion. The examples used for illustration are mostly derived from work on carrot cultivar discrimination, more fully described by Horgan et al. (2000).
2. Local shape Local shape refers to those shape properties of an object which do not depend on where they are in the image. Fig. 3 shows some sub-leaflets of carrot leaves, photographed in a laboratory and processed (by seeking pixels with green intensity above a threshold) to give outline shapes. Leaves such as these are very flexible. The large scale details of the shape of each leaf reflect only the way in which the photographer arranged them. The only details whose shapes are meaningful and invariant are the shapes of the smallest features, the individual pinnae (tips) at the end of each leaf, and the clefts between them. The shape of the pinnae should be assessed without accounting for where they are in the image, and the resulting distribution aggregated. There are a number of ways we can obtain some summary of local shape. We can do this by identifying individual small features, by morphological measurements, or by texture.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
173
Fig. 3. Subleaflets of the carrot varieties (a) Amsterdam Forcing and (b) Chantenay Red Cored 2, and (c) plots of percentage area of leaflets relative to original image after operations of erosion, dilation, opening and closing.
174
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Fig. 4. (a) Fitting a disc (black) inside the tip of a leaf pinna (light grey); (b) Morphological erosion (dark grey) is the set of centres of discs which can fit inside the pinna; (c) Morphological opening (dark grey) is the union of such discs. For the purposes of illustration, the left-hand side of the pinna is cut off.
2.1. Identifying small features We might consider that the best way to measure the properties of pinnae, is to locate a sample of them, and then calculate some descriptors. This is not difficult. If we define H rij to be the proportion of pixels within a distance r of pixel position (i, j ) which are leaf (rather than background), then for a suitable choice of r, pinnae may be found as local minima of H rij. We can then obtain the density of pinnae, per unit area of image or of leaf. The clefts between the pinnae can be obtained by a similar operation on the background. The sharpness of a pinna could be easily evaluated by looking at the slopes of tangents to the leaf in its vicinity, or simply by the value of H rij at the minimum.
2.2. Morphology This is a powerful set of tools based on set operations with groups of pixels. Descriptions may be found in Serra (1982), Glasbey and Horgan (1995). The most basic morphological operations are termed erosions and openings. Using discs (for simplicity-other structures could be used) we see where a disc of radius r can fit within the leaf. The locus of the centres of such discs is termed the erosion of the leaf by a disc of radius r. The union of such discs is termed the opening. If we perform the same operations on the background, we get what are termed the dilation and closing, respectively, of the leaf. The use of such test discs (termed structuring elements) is illustrated in Fig. 4. The discs are of necessity lattice approximations. The use of morphological operations in this way is closely related to the distance transform (Borgefors, 1986) which assigns to each pixel its distance from the nearest boundary. By performing these morphological operations with a range of disc sizes, and measuring how much leaf area results, we get a useful characterisation of local shape. This is done in Fig. 3(c) for leaves from two carrot varieties. The plot shows
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
175
the leaf areas after erosion, opening, closing and dilation. The patterns are similar for both sets of leaves, with the clearest difference being in the dilation curve. The morphological operations used here do not fully describe a leaf’s appearance, in that they could not be used to reconstruct it or to simulate it. Such a description would need to involve a fuller model of the leaf’s structure, and would probably be based on branching processes to model the leaf skeleton and a thickness function to model local size. Note also that although the leaves resemble the structures created by iterative function fractals (Barnsley, 1993), these cannot usefully be used to model them, as the details of the structure are highly sensitive to minute changes in the functions’ parameters.
2.3. Texture Texture in images is a property which is widely understood, although it eludes precise definition. One version is that it is the distribution and spatial arrangement of short range variation in intensity. If we regard the alternating of leaf and background, or of spots on a leaf, as a form of texture, then there is a wealth of image analysis tools for describing this. A review is given by Reed and Dubuf (1993). The most widely used approach to texture, and one of the earliest to be proposed is based on co-occurrence matrices (Haralick et al., 1973). These contain counts of the proportion Pij of times specified pairs of intensities i and j occur at a specified separation. Summaries of these matrices are then calculated. Three which are often suggested are, angular second moment ij P 2ij contrast ij (i −j )2Pij entropy ij Pij log Pij They will all tend to respond to local roughness of texture, and the differences between them are more important in greyscale images. If the image is binary, as in an outline image, it may be simpler to just note the proportions P10 and P01 for a range of separations, and to calculate Pij only in the neighbourhood of edges. An advantage of a texture based approach is the wide availability of software for assessing texture. It is likely to be the best method of detecting differences in features such as patterns of spots or striations that often occur on the leaves or other parts of plants. Perhaps the main disadvantage is that it can be difficult to predict which texture measures will be useful in capturing patterns of interest. Other related approaches to local shape description could also be used. Boundary length density relative to leaf area, which measures the roughness of the boundary and the compactness of the leaves is easy to calculate. These and other measures of shape are discussed in Glasbey and Horgan (1995). The use of fractal descriptions of leaf boundaries, such as the fractal dimension, has been examined (Campbell, 1996; Borkowski, 1999; Mancuso, 1999). These enjoy the advantage of being well understood mathematically, and in some cases potentially offering insight into leaf growth processes.
176
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
3. Global shape Global shape is concerned with rigid objects. Shape is defined as the relative positions of the parts of an object outline — those properties of the outline which are unaffected by position and orientation. Sometimes, one may also wish to discard differences in overall size, although this will often not be appropriate for plant parts, where size does matter. An object and its reflection ( mirror-image) may also be treated as indistinguishable. Fig. 5 shows six carrots of the same variety, sliced longitudinally and photographed in a laboratory. Although the reproduction here is monochrome, the original images were in colour. Freshly cut carrots are rigid, and shape details of the whole carrot are relevant. The relative positions of particular parts (the crown and tip for example) are informative. Root shape is an important characteristic of carrots, and its quantification is not new. Bleasdale and Thompson (1963) fitted power law curves to carrot diameter variation from crown to tip. The statistical analysis of this type of shape is most often based on landmarks. These are the positions of a finite number of well-defined features of the carrot, not necessarily on the outline. We describe this approach first, and then look at what progress may be made without landmarks.
3.1. Shape analysis with landmarks 3.1.1. Defining landmarks Landmarks are well defined positions on an object, whose relative positions, when taken together, capture all of the relevant shape information about the object. For example, on a human face, one could use features such as the edges of the mouth, the pupil of the eye etc. It is assumed that we can assign these features horizontal and vertical co-ordinate positions, and then proceed to analyse these
Fig. 5. Six carrots of variety Rusty.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
177
Fig. 6. Landmark positions on a carrot outline. O denotes the centroid. The other positions are explained in the text.
much simplified data. For three dimensional objects, we may use three co-ordinates. Landmark positions may be located manually, by a suitable automatic method (as below). In the example we use here, however, there are few natural biologically definable landmark positions on a carrot. We may use geometrically defined positions instead. Images are first processed to provide silhouettes by thresholding the grey levels to separate the carrot from background, and removing any small (i.e. non-carrot) connected sets of pixels that remained in the background. Positions can then be selected on each outline. For the carrots this was done as shown in Fig. 6. The landmarks were found automatically as follows. 1. The left most pixel position (A). 2. The most ‘north-eastern’ (B) and ‘south-eastern’ (C) pixel positions. Their midpoint (D) was taken as the right (crown) end of the carrot. 3. For seven equally spaced horizontal positions between A and D, the top and bottom Y co-ordinate of the carrot outline was obtained. Since the horizontal positions of the landmarks between A and D are linearly determined from those of A and D, they may be discarded. We have thus reduced the outline shape to 20 variables, the co-ordinates of A, B and C, and the top and bottom of the carrot at seven intermediate points.
3.1.2. Statistical analysis The statistical analysis of shape using landmarks has been well developed. A good introduction to the subject is given by Dryden and Mardia (1998). There are two established approaches. The first is associated with Bookstein (1978, 1991, 1997). Two landmarks are considered fixed, and the positions of the rest are considered relative to them. By mapping the fixed landmark positions to a horizontal line segment, the relative positions of the other can be plotted and described using ordinary arithmetic. This approach has the advantage of being readily understood and handled, and amenable to standard multivariate statistical methods. It has the disadvantage that the choice of fixed landmarks is arbitrary, and results may be dependent on this. These Bookstein shape co-ordinates are best suited to situations where shape variations are small relative to the scale of the
178
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
whole object. The Kendall co-ordinate system (see Kendall, 1989) is similar, but less widely known. The other approach to landmark-based shape analysis is based on Procrustes rotations, where the first stage is to align the shapes, by translation, scaling and rotation, to match each other as closely as possible. A full description is presented by Goodall (1991a). For a biological application see Glasbey et al. (1995). The resulting statistical analysis is more demanding, but can claim advantages of full objectivity and optimal use of the data. Either Bookstein or Procrustes co-ordinates could be used for the set of carrot landmarks. We have used an approach which can be considered as a variation on the Procrustes co-ordinates, which was simpler to use and could readily handle the processing of data from new carrots. For ease of handling in later processing stages, the carrots pixels (not just the outline) were expressed relative to the position of their centroid (the mean i and j co-ordinates). The orientation of the carrots, which only varied slightly, was obtained as that of a line from the tip to the centre of the crown (see Fig. 6). The landmark positions were recorded relative to the centroid, and rotated to correspond to a horizontal orientation for the whole carrot. No further Procrustes matching was done. In particular, there was no size scaling, since carrot size was considered relevant, not something to be discarded.
3.1.3. Principal component analysis Principal component analysis (Krzanowski, 1993) is a common starting point for examining and describing variation in shape. These are linear combinations of the original shape co-ordinates which express as much as possible of the total variation among the co-ordinates, while being orthogonal (uncorrelated) to each other. Examination of the effect of altering each component will indicate the main types of shape variation present in the original landmarks. The principal components of the 20 outline shape variables were obtained from their covariance matrix. These have been termed ‘eigenshapes’ in the biological literature (Ray, 1992), although Goodall (1991b) uses the term in a different context. The effect of varying individual components is shown for the first six in Fig. 7. The variance accounted for and a suggested interpretation is shown in Table 1. Components four and five change sign on reflection of the carrot about the line AD, an irrelevant change. Only the absolute value is informative, and so this was used for matching and discrimination purposes. This is an inelegant solution. It would be more desirable to account for the reflection symmetry in the formulation of the principal component analysis, or to process the images and reflect some carrots to ensure that, for example, they all ‘curved the same way’. The question arises as to how many principal component scores are useful to record, a question common to all applications of this technique, and not confined to its use in shape. Some guidance is offered by Krzanowski (1993).
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
179
3.1.4. Other approaches Principal component analysis is often used as a first step in examining shape data, as it is otherwise difficult to get a good impression of what variation is present. If desired, it is then possible, in the context of either Bookstein or Goodall co-ordinates, to develop versions of many other statistical tools, summaries, correlation and regression, analysis of variance and discriminant analysis. For more details, see Dryden and Mardia (1998). A weakness of the analyses described here is that they take no account of shape variations which are biologically important but subtle at the scale of the whole object. For example, the exact shape of the shoulders at the crown of the carrot is believed to vary significantly among varieties. It should be possible to account for this by having more, and more carefully placed, landmarks in such important parts of the object. The extra effort needed would result more from the trouble of
Fig. 7. Principal components of carrot shape variation. Numbers relate to the components given in Table 1.
180
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Table 1 Principal components of outline shape Component
Variance (%)
Interpretation
1 2 3 4 5 6
72.0 10.8 8.2 3.4 2.0 0.9
Short and thick 6s. long and thin Tapering (cylinder 6s. cone) Thickness Bending Asymmetry Tapering at tip
obtaining the positions of the extra landmarks than from the computational resources needed to handle them.
3.2. Shape analysis without landmarks Treating shape through landmarks has the appealing property that all the shape details in the landmarks is accessible to statistical analyses. If our landmarks are complete enough, then we are working with all the information in the shape. However, there are many situations in which landmarks are not a practical approach. It may be impractical to define them for some objects (such as the outline of a potato) which do not have well defined features, or it may be too time consuming to identify them manually when automatic methods are unavailable or unreliable. We describe in this section three other approaches which may be used in these situations.
3.2.1. Shape summaries These are features such as area/perimeter ratios, magnitudes and ratios of moments, (summaries such as the mean squared distance from the centre of gravity), maximum distances within outlines, the size of circumscribing or inscribing discs or polygons etc. They all have the property that they depend on the shape, and they are independent of position and orientation. They may, thus, be used to discriminate between shapes. Several may be used together, and may then be termed a ‘feature vector’. The advantages of using such summaries are that they are easy to understand and are usually easy and quick to calculate. The choice of features is usually based on what is expected to have discriminating power, and what may readily be calculated. A disadvantage is that we may miss useful shape attributes that are not reflected in the features used. Details and examples of these ideas can be found in Glasbey and Horgan (1995). For some biological applications, see Draper and Keefe (1989), Van de Vooren et al. (1991). 3.2.2. Fourier series Any periodic function may be represented as a sum of cosine and sine functions with same period, and multiples of it. The amplitudes of each of these functions are
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
181
easily determined, and convey information about the function. When a function does not contain rapid changes, the earliest terms in this sum, termed the Fourier series, will be a good approximation to the function. If we trace around the outline of a shape, we get two periodic functions, the horizontal and vertical co-ordinates of positions on the outline. These may be represented as a Fourier series. This is illustrated for a carrot outline in Fig. 8. The set of function amplitudes (coefficients) in the Fourier series form a shape summary, and may be used in a similar way to the shape summaries described above. Fourier series enjoy the advantage that if sufficient terms are taken, then all important features of the shape outline will be captured. A disadvantage is that a large number of terms may be required. For more information on this approach to shape representation, see Granlund (1972), Crimmins (1982). For a variation, see Zahn and Roskies (1972). For biological examples, see Rohlf and Archie (1984), Mou and Stoermer (1992).
Fig. 8. Carrot outline. Approximation by Fourier series is shown for (b) 2, (c) 5 and (d) 12 terms.
182
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
3.2.3. Whole shape methods It is also possible to handle shapes without landmarks or points of any sort, in a non-parametric way. There is a substantial cost in computational effort, but the approach has some appealing features, and can be considered simpler to deal with. We proceed as follows. Images are converted to binary values, and an object is a connected set of black pixels. Two objects are considered to have the same shape if one can be translated and rotated (and if we wish, reflected and/or scaled) so that it coincides with the other. Since we are dealing with pixel positions on a lattice, we will use the best digital approximations to rotation. If two objects have a different shape, then we may ask how much they differ. Various answers to this can be envisaged. The most natural one is that it depends on how much they overlap. We could measure the difference as the area of the non-overlapping regions of the objects when this has been minimised by translation and rotation. Another possibility is Baddeley’s delta distance (Baddeley and Molchanov, 1998), which is based on the distance transform (the distance of each point from the boundary) of the objects. Once differences between shapes have been quantified, standard statistical methods can be constructed. Between and within group variation can be compared. An average can be defined as that shape whose mean distance from the shapes in question is minimised. Principal co-ordinate analysis (Krzanowski, 1993), permits study of the structure of variation among carrots, and this can be developed to study association with other non-shape variables. Principal co-ordinate analysis is similar to principal component analysis, but is derived from a matrix of differences between pairs of observations (whereas principal component analysis is derived from the covariance or correlation matrix of a set of variables). It produces a set of uncorrelated scores for each observation (shape), with the difference between scores being the best approximation to the original shape difference matrix. Fig. 9 shows a plot of the first two principal co-ordinates from 40 carrots. Two outliers on the first component, which are unusually small carrots, and one on the second component, a highly tapered carrot, are apparent. This analysis is described in more detail by Horgan (2000). Related work, which avoids full Procrustes matching, is presented by Wright et al. (1997), while a mathematical theory is pursued by Stoyan and Molchanov (1997), Baddeley and Molchanov (1998). 4. Colour distribution The previous two sections dealt with the appearance of the outline shape of a plant part. We now consider the variation in colour within a part. In keeping with the distinction between local and global properties, we first look at the colour distribution of an object, not considering where within the object that colour is observed. It should be apparent that the differences in appearance between the Brussels Sprouts images in Fig. 1, which can be seen even in the monochrome reproduction, are more (although not completely) a matter of colour frequency than their positional distribution. This is a consequence of the flexibility of the leaves.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
183
Fig. 9. (a) Initial placing of 40 carrots; (b) The best alignment; (c) Principal co-ordinate plot based on overlap area differences.
All colours we perceive are determined by the responses they produce in three cell types in the retina with well known spectral responses. Due to this, most technological handling of colour (photography, TV signals, digital images) imitates these three components with the familiar red, green, blue (RGB) system. Any generalisation to a more full-spectrum approach (for which instrumentation is available)
184
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
could be implemented by increasing the number of components from three. This is done in the science of chemometrics (Krzanowski et al., 1995). Each pixel in a colour image will have an associated red, green and blue intensity (R,G,B). Since the image is digital, colour components will take integer values, most commonly in the range 0…255 (256= 28 intensities) or 0…4095 (4096= 212) If we ignore the pixel position, then a full description of the colour of the object requires only that we specify the three-dimensional distribution f(r,g,b), the proportion of pixels which have red intensity r, green intensity g and blue intensity b. For comparing images, it is often more convenient to work with the cumulative distribution F(r,g,b), the proportion of pixels with red intensity less than r, green less than g etc. Given two images, with colour distributions F (1)(r,g,b) and F (2)(r,g,b), we may quantify the difference between them as rgb (F (1)(r, g, b)− F (2)(r,g,b))2 or max F (1)(r, g, b) −F (2)(r, g, b) . A difference measure can, thus, be created for any pair of images. Fig. 10 shows a principal co-ordinate analysis based on differences among 48 images similar to (and including) those in Fig. 1, six from each of eight varieties. Good, but not perfect, separation of varieties can be seen.
Fig. 10. Scatterplot of first two principal co-ordinates based on the image histogram difference. Each of eight varieties is labelled with a different letter. The three varieties in Fig. 1 are T, H and G.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
185
4.1.1. Colour summaries Comparing histograms will suffice for quantifying and seeking similarities between objects. We may also wish to summarise the distribution. It is less clear how this should be done. A natural choice would be to determine the mean vector and covariance matrix. This will not be sufficient in many cases, as colour distributions will often be multimodal. Adding more moments or trying moment generating or other basis functions are unlikely to be satisfactory, as they are sensitive to high order components. We do not offer a general solution, but suggest that summaries should be proposed specifically for each application. In the case of the Brussels sprouts example, it was found (Horgan et al., 1995) that good discrimination between varieties could be obtained using the mean of R the mean of G the mean of B the proportion of pixels for which G\ 200 the proportion of pixels for which G\ 225. Here the choice of summaries was influenced by the fact that variety discrimination was assisted by summaries reflecting the proportions of very bright pixels. For an example of colour histograms in the context of wider image databases, see Colombo et al. (2000).
5. Colour detail — eigenimage analysis The global shape analysis described earlier considers variation in the outline shape, but ignores colour variation within the outline. This can also vary significantly between varieties. In this section we consider how to summarise its variation. In many situations, variation within the outline can be studied by adaptation of the tools already described. We look for features within the outline, attempt to isolate them, and then look at their shape or colour summaries. This might be useful for the core in the carrot images, or for the eyes in images of human faces. The advantage of this approach is that we are in full control of the features we use, and how we use them. The disadvantages are that we need to be able to isolate features, and we must know in advance which are likely to be useful. A more general approach is to seek a principal component analysis, where we now regard the colour information at each pixel as a separate variable. This application of principal component analysis to images in this way first appears to have been used for images of human faces (Kirby and Sirovich, 1990; Craw and Cameron, 1992; Lanitis et al., 1995, 1997), where the results have sometimes been termed eigenfaces.
186
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Fig. 11. Warping carrots in Fig. 5 to a common shape.
5.1.1. Warping to a common shape If the objects are placed, suitably centred, within an image and the principal components obtained, they would be influenced by outline shape variation as well as colour variation within the object. The outline shape has already been dealt with separately, and we would like to look at variation in colour within the outline only. To do this, we first distort the images so that all carrots have the same outline shape. There are many ways to warp one object or image to another. A discussion of recent ideas is given by Glasbey and Mardia (1998). In the case of the carrots, a simple algorithm was used. First, a simple linear magnification or reduction was applied to transform the line AD in each carrot to a length of 350 pixels. The mean thickness, over all carrots, at each of the 350 horizontal positions, was then obtained, and these 1-pixel-wide vertical slices were then translated and magnified or reduced by linear interpolation so as to be centred on the line AD and of mean thickness for that slice. The results of this transform, on the carrots in Fig. 5, is shown in Fig. 11. 5.1.2. Obtaining eigenimage loadings The eigenimages are simply the principal components of variation in the images, regarding the intensity at each pixel position as a separate variable. Although it may easily be done using all three colour components, it was more efficient to confine use to the green component. The blue component was too uniformly dark and the red component too uniformly bright within the carrot outline to add any additional information. There are nearly 27 000 pixel positions, so that a direct calculation of their covariance matrix (C) is not feasible. However, the number (N) of images is much less. The components may be obtained from the eigenvectors of a N× N matrix T, where
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
187
Table 2 Principal components of greylevels Component
Variance (%)
Interpretation
1 2 3 4 5 6
30.7 9.7 8.0 3.2 2.3 1.9
Core/cortex contrast Core thickness Tapering of core Core/cortex boundary brightness Crown vs. tip contrast within core Asymmetrical positioning of core
Tij =% Vi Vj, and Vi is the i th image with its mean intensity subtracted from each pixel, and the sum extends over all the pixels in the image. This calculation is tractable, although calculating each element of T, of which there are N(N+1)/2 for N carrots, requires a few seconds of computing time. The variation accounted for by the first six components, and a tentative interpretation is shown in Table 2. Fig. 12 shows images of the eigenimage loadings. For ease of interpretation, the loadings were divided into five intensity classes; brighter areas are positive and darker areas negative. The k th score for any individual carrot image Ui can be obtained by multiplying the eigenimage by the warped carrot image.
Fig. 12. Carrot eigenimage loadings. Numbers refer to components in Table 2.
188
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Sik =% EkUi, summation being over pixel positions. These scores taken together are a summary of the variation within the object outline, and may be used for matching and discrimination.
6. Discussion Digital images typically have 105 –107 pixels, and so are observations of very high dimension. These dimensions have much structure. In the biological sciences at least, there is also considerable variation between objects. It is this that makes image analysis a demanding discipline, and one with many unsolved problems. In this paper, we have shown how multivariate techniques of data reduction can be used to condense the huge amount of image data to a tractable set of features. The approaches described can be used in many situations where we have to compare a number of images of the same type. They have the appeal that they are able to extract features of relevance, and are able to do so without needing to use any prior knowledge of those features which vary among the objects of study. There are many aspects to this work where developments can be pursued. These are of two types. The first is that the techniques for object characterisation we have presented can be further refined. For the carrot application we have used for illustration, this should be most fruitful in the study of global shape. The landmarks we have used may be failing to capture some features of interest. This could be improved by defining a richer set of landmarks. For example, in the case of carrots, more accurate identification of the position and shape of the crown should help. Some of the difficulties in shape description may be overcome by studying shape without landmarks, and we intend to investigate this further. The second aspect of the research which might be developed is appropriate accounting for the various sources of variation in the appearance of plants. We have not so far attempted to deal with the structure of variation in crop measurements that is known to occur between years and between the sites where the crops are grown, or the extent to which these interact with each other and with specific crop varieties. This does not necessarily require new methods of image manipulation. A more complex modelling of multivariate variation, beyond the simple one-level approach of principal component analysis, should prove fruitful. Knowledge of the main sources of such variation can be useful to crop specialists and to industry in producing more uniform plant material.
Acknowledgements This work was supported by the Scottish Agricultural Science Agency and the Scottish Executive Rural Affairs Department.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
189
References Baddeley, A., Molchanov, I., 1998. Averaging of random sets based on their distance functions. J. Math. Imaging Vision 8, 79–92. Barnsley, M.F., 1993. Fractal Image Compression Wellesley Peters. Bleasdale, J.K.A., Thompson, R., 1963. An objective method of recording and comparing the shapes of carrot roots. J. Hort. Sci. 38, 232–241. Bookstein, F.L., 1978. The Measurement of Biological Shape and Shape Change. Springer-Verlag, Berlin. Bookstein, F.L., 1991. Morphometric Tools for Landmark Data: Geometry and Biology. Cambridge University Press, Cambridge. Bookstein, F.L., 1997. Shape and the information in medical images: a decade of the morphometric synthesis. Comput. Vis. Image Understanding 66, 97 – 118. Borgefors, G., 1986. Distance transforms in digital images. Comput. Vis. Graphics Image Processing 34, 344–371. Borkowski, W., 1999. Fractal dimension based features are useful descriptors of leaf complexity and shape. Can. J. Forest Res.-Revue Canadienne de Recherche Forestiere 29, 1301– 1310. Campbell, R.D., 1996. Describing the shapes of fern leaves: a fractal geometrical approach. Acta Biotheoretica 44, 119–142. Choubey, S.K., Raghavan, V.V., 1997. Generic and fully automatic content-based image retrieval using color. Pattern Recognition Lett. 18, 11 – 13. Colombo, C., Del Bimbo, A., Genovesi, I., 2000. Interactive image retrieval by color distribution content. Integrated Comput. Aided Eng. 7, 79 – 88. Craw, I., Cameron, P., 1992. Face Recognition by computer. In: Proc. Br. Machine Vis. Conf. Springer-Verlag, Berlin, pp. 489–507. Crimmins, T.R., 1982. A complete set of Fourier descriptors for two-dimensional shapes. IEEE Transactions on Systems Man Cybernetics 12, 848– 855. Draper, S.R., Keefe, P.D., 1989. Machine vision for the characterisation and identification of cultivars. Plant Varieties Seeds 2, 53–62. Dryden, I.L., Mardia, K.V., 1998. Statistical Shape Analysis. Wiley, Chichester. Glasbey, C.A., Horgan, G.W., 1995. Image Analysis for the Biological Sciences. Wiley, Chichester. Glasbey, C.A., Mardia, K.V., 1998. A review of image warping methods. J. Appl. Stat. 25, 155– 171. Glasbey, C.A., Horgan, G.W., Gibson, G.J., Hitchcock, D., 1995. Fish shape analysis using landmarks. Biometrical J. 37, 481–495. Goodall, C., 1991a. Procrustes methods in the statistical analysis of shape (with discussion). J. Royal Stat. Soc. Series B 53, 285–339. Goodall, C., 1991b. Eigenshape analysis of a cut-grow mapping for triangles, and its application to phyllotaxis in plants. SIAM J. Appl. Math. 51, 775– 798. Granlund, G.H., 1972. Fourier preprocessing for hand print character recognition. IEEE Transactions Comput. 21, 195–201. Haralick, R.M., Shanmugam, K. Dinstein, I., 1973. IEEE Transactions on Syst. Man Cybernetics, SMC-3, 610–621. Horgan, G., Talbot, M., Davey, J., 1995. Plant variety colour assessment using a still video camera. J. Plant Varieties Seeds 8, 161–170. Horgan, G., Talbot, M., Davey, J., 2000. Use of statistical image analysis to discriminate carrot cultivars. Comput. Electron. Agric. 31, 191– 199. Horgan, G.W., 2000. Principal component analysis of random particles. J. Math. Imaging Vis. 12, 169–175. Huang, C.L., Huang, D.H., 1998. A content-based image retrieval system. Image Vis. Comput. 16, 149–163. Kendall, D.G., 1989. A survey of the statistical theory of shape. Stat. Sci. 4, 87 – 120. Kirby, M., Sirovich, L., 1990. Application of the Karhunen-Loeve procedure for the characterisation of human faces. IEEE Pattern Anal. Machine Intelligence 12, 103– 108.
190
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Krzanowski, W.J., 1993. Principles of Multvariate Analysis: A User’s Perspective. Oxford University Press, Oxford. Krzanowski, W.J., Jonathan, P., McCarthy, W.V., Thomas, M.R., 1995. Discriminant-analysis with singular covariance matrices-methods and applications to spectroscopic data. Appl. Stat. 44, 101–115. Lanitis, A., Taylor, C.J., Cootes, T.F., 1995. Automatic face identification system using flexible appearance models. Image Vision Comput. 13, 393– 401. Lanitis, A., Taylor, C.J., Cootes, T.F., 1997. Automatic interpretation and coding of face images using flexible models. IEEE Transactions Pattern Anal. Machine Intelligence 19, 743– 756. Mancuso, S., 1999. Fractal geometry-based image analysis of grapevine leaves using the box counting algorithm. Vitis 38, 97–100. Mehtre, B.M., Kankanhalli, M.S., Lee, W.F., 1997. Shape measures for content based image retrieval: A comparison. Information Processing Management 33, 319– 337. Mou, D., Stoermer, E.F., 1992. Separating Tabellaria (Bacillariophyceae) shape groups based on Fourier descriptors. J. Phycol. 28, 386–395. Ray, T.S., 1992. Landmark eigenshape analysis-homologous contours-leaf shape in syngonium (Araceae). Am. J. Botany 79, 69–76. Reed, T.R., Dubuf, J.M.H., 1993. A review of recent texture segmentation and feature-extraction techniques. CVGIP-Image Understanding 57, 359– 372. Rohlf, E.J., Archie, J.W., 1984. A comparison of Fourier methods for the description of wing shape in mosquitoes (Diptera: Culicidae). Syst. Zool. 33, 302– 317. Serra, J., 1982. Image Analysis and Mathematical Morphology. Academic Press, London. Stoyan, D., Molchanov, I.S., 1997. Set-valued means of random particles. J. Math. Imaging Vis. 7, 111–121. Van de Vooren, J.G., Polder, G., van der Heijden, G.W.A.M., 1991. Application of image analysis for variety testing of mushroom. Euphytica 57, 245– 250. Wright, D., Stander, J., Nicolaides, K., 1997. Nonparametric density estimation and discrimination from shapes. Appl. Stat. 46, 365–382. Zahn, T.C., Roskies, R.Z., 1972. Fourier descriptors for plane closed curves. IEEE Transactions Comput. C21, 269–281.
.
The statistical analysis of plant part appearance — a review Graham W. Horgan * Biomathematics and Statistics Scotland, Rowett Research Institute, Aberdeen AB21 9SB, Scotland, UK Received 8 June 2000; received in revised form 31 October 2000; accepted 27 November 2000
Abstract Visual appearance is widely used in assessing the variety or cultivar of crop samples, and in biological taxonomy in general. With computer storage capabilities becoming more easily accessible, image databases of plant specimens can be readily constructed. To make full use of this powerful facility, an important task is to compare different images, or to summarise them. This review addresses the question of how this may be done, given that images are stored as typically 105 –107 highly structured measurements. Image analysis is a wide subject area, and this review covers those parts of it which are relevant to this task. The task is subdivided into four (2 ×2) categories according to whether we are interested in outline shape only or in colour details, and whether the global position of object details is important, or only their local distribution. We refer to these approaches as local shape, global shape, colour distribution and colour detail (eigenimage analysis). As far as possible, we avoid a priori ideas regarding what features of objects are important, and seek methods which capture the full variability of their appearance. The techniques are illustrated by reference to work on carrots. © 2001 Elsevier Science B.V. All rights reserved. Keywords: Image analysis; Eigenimage; Eigenshape; Discriminant analysis; Image matching; Crop cultivars; Morphology
1. Introduction Visual appearance is widely used in assessing the variety or cultivar of crop samples, and in biological taxonomy in general. For some characteristics, objective * Corresponding author. Fax: + 44-1224-716608. E-mail address: [email protected] (G.W. Horgan). 0168-1699/01/$ - see front matter © 2001 Elsevier Science B.V. All rights reserved. PII: S0168-1699(00)00190-3
170
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
accurate measurement is readily made. Size or distance between well defined features is the best example of this. For others, an objective definition is possible, but objective measurement presents difficulties. For example, colour can in theory be measured by suitable spectrophotometric equipment, but such tools can be expensive and time consuming to use, and so colour is often assessed subjectively. Likewise, the proportion of a leaf or other plant part affected by fungi, or some other problem, is well defined, but usually estimated subjectively. It is well known that there can be significant differences between different assessors, and even a single assessor is influenced by a carry-over effect from earlier specimens. Finally, some important characteristics, such as the degree to which a leaf is serrated, are usually not even objectively defined. With computer storage capabilities becoming more easily accessible, image databases of plant specimens can be readily contructed. To make full use of this powerful facility, an important task is to compare different images or to summarise them. A particularly useful ability would be to find the best match between a new image specimen and those in the database. As images are just numerical data, albeit very highly structured, it is in theory possible for all these tasks to be done automatically and objectively. We are concerned in this paper with the statistical methods available for the study of object appearance. We wish to quantify numerically distinguishing characteristics between different groups of plants. A broad range of techniques is called for. Our aim is to review these approaches, and illustrate their role in crop variety recognition. Although we focus on this horticultural application, the ideas are of relevance to the wider research topic of image retrieval. This is the study of managing and searching databases of images using image content rather than subjective, text-based indexing. For a flavour of this work, see Choubey and Raghavan (1997), Mehtre et al. (1997), Huang and Huang (1998). Much of their work is aimed at image databases which are very wide in scope, such as a library or publishing house might hold, and where the image criteria sought can be very general or abstract (for example, find an image representing ‘strength’). We will be concerned with the more specialised applications of image comparison, where the class of object is known (for example, the set of images to be searched are known to be varieties of carrots) and the differences between varieties are subtle (for example, how tapered the carrots are near the tip). In many applications, information about object appearance will come from two-dimensional images produced under ordinary visible light. Other image formation strategies, using different radiation sources, or fluoroscopy could be used. Types of imaging could be envisaged which would give three-dimensional information. This is much more demanding computationally, and is not considered here.
1.1. Definitions An image will be taken to be a two-dimensional array of pixel intensities fij, i = 1…m; j= 1…n. It is now usual to work with colour images, in which case a red, green and blue intensity is specified for each pixel. Fig. 1 shows nine images of
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
171
Brussels sprouts plants. Sometimes we will work with outline silhouettes only, in which case fij will take values of 0 and 1 only. Sometimes it may be convenient to handle such outlines by listing positions on the outline, pixels within the object having neighbouring pixels without. Approaches to object characterisation will be influenced largely by which aspects are considered informative and which are a nuisance. Position and orientation are usually unimportant. The photographer could have positioned the plant differently in the image or stood in a different position and the object appearance would be unchanged. A reflection (mirror image) of a plant would not usually be considered different from the original. Size may or may not be important. It can only be used if we know the scale of each image. Many plant parts are physically flexible. The leaves on the Brussels sprouts plants can be moved with respect to each other. The crown of a tree continually changes shape on a windy day. The leaves of plants can be moved, but are not elastic. For plant parts such as these, features are needed which are invariant under this flexibility. The object features discussed will be of four types, depending on whether (i) we are dealing with outline shapes only or with the variation in brightness or colour within the outline, and (ii) whether we are treating the object as rigid (so that global features can be considered), or flexible. Fig. 2 shows the terminology we will use for
Fig. 1. Images of three varieties of Brussels sprouts plants. Each column is a separate variety. From left to right the varieties are Target (T), Heracles (H) and Golfer (G).
172
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Fig. 2. Classification of approaches to object characterisation.
these situations. Although flexibility or rigidity are considered as attributes of the plant, the use of outline shape and colour detail will be considered as complementary; they can both provide information. An important aim in our object characterisation is that where possible we wish to avoid a priori specification of what features should be examined. Although botanists already list many features for measurement on plants for taxonomic purposes, we concentrate here on general appearance. The next four sections of the paper will describe the four approaches in Fig. 2. Section 2 looks at local shape and Section 3 at global shape. Section 4 considers colour distributions. Section 5 shows how a principal component approach, termed eigenimage analysis, can be used to condense the colour details within a shape outline. Section 6 concludes with a brief discussion. The examples used for illustration are mostly derived from work on carrot cultivar discrimination, more fully described by Horgan et al. (2000).
2. Local shape Local shape refers to those shape properties of an object which do not depend on where they are in the image. Fig. 3 shows some sub-leaflets of carrot leaves, photographed in a laboratory and processed (by seeking pixels with green intensity above a threshold) to give outline shapes. Leaves such as these are very flexible. The large scale details of the shape of each leaf reflect only the way in which the photographer arranged them. The only details whose shapes are meaningful and invariant are the shapes of the smallest features, the individual pinnae (tips) at the end of each leaf, and the clefts between them. The shape of the pinnae should be assessed without accounting for where they are in the image, and the resulting distribution aggregated. There are a number of ways we can obtain some summary of local shape. We can do this by identifying individual small features, by morphological measurements, or by texture.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
173
Fig. 3. Subleaflets of the carrot varieties (a) Amsterdam Forcing and (b) Chantenay Red Cored 2, and (c) plots of percentage area of leaflets relative to original image after operations of erosion, dilation, opening and closing.
174
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Fig. 4. (a) Fitting a disc (black) inside the tip of a leaf pinna (light grey); (b) Morphological erosion (dark grey) is the set of centres of discs which can fit inside the pinna; (c) Morphological opening (dark grey) is the union of such discs. For the purposes of illustration, the left-hand side of the pinna is cut off.
2.1. Identifying small features We might consider that the best way to measure the properties of pinnae, is to locate a sample of them, and then calculate some descriptors. This is not difficult. If we define H rij to be the proportion of pixels within a distance r of pixel position (i, j ) which are leaf (rather than background), then for a suitable choice of r, pinnae may be found as local minima of H rij. We can then obtain the density of pinnae, per unit area of image or of leaf. The clefts between the pinnae can be obtained by a similar operation on the background. The sharpness of a pinna could be easily evaluated by looking at the slopes of tangents to the leaf in its vicinity, or simply by the value of H rij at the minimum.
2.2. Morphology This is a powerful set of tools based on set operations with groups of pixels. Descriptions may be found in Serra (1982), Glasbey and Horgan (1995). The most basic morphological operations are termed erosions and openings. Using discs (for simplicity-other structures could be used) we see where a disc of radius r can fit within the leaf. The locus of the centres of such discs is termed the erosion of the leaf by a disc of radius r. The union of such discs is termed the opening. If we perform the same operations on the background, we get what are termed the dilation and closing, respectively, of the leaf. The use of such test discs (termed structuring elements) is illustrated in Fig. 4. The discs are of necessity lattice approximations. The use of morphological operations in this way is closely related to the distance transform (Borgefors, 1986) which assigns to each pixel its distance from the nearest boundary. By performing these morphological operations with a range of disc sizes, and measuring how much leaf area results, we get a useful characterisation of local shape. This is done in Fig. 3(c) for leaves from two carrot varieties. The plot shows
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
175
the leaf areas after erosion, opening, closing and dilation. The patterns are similar for both sets of leaves, with the clearest difference being in the dilation curve. The morphological operations used here do not fully describe a leaf’s appearance, in that they could not be used to reconstruct it or to simulate it. Such a description would need to involve a fuller model of the leaf’s structure, and would probably be based on branching processes to model the leaf skeleton and a thickness function to model local size. Note also that although the leaves resemble the structures created by iterative function fractals (Barnsley, 1993), these cannot usefully be used to model them, as the details of the structure are highly sensitive to minute changes in the functions’ parameters.
2.3. Texture Texture in images is a property which is widely understood, although it eludes precise definition. One version is that it is the distribution and spatial arrangement of short range variation in intensity. If we regard the alternating of leaf and background, or of spots on a leaf, as a form of texture, then there is a wealth of image analysis tools for describing this. A review is given by Reed and Dubuf (1993). The most widely used approach to texture, and one of the earliest to be proposed is based on co-occurrence matrices (Haralick et al., 1973). These contain counts of the proportion Pij of times specified pairs of intensities i and j occur at a specified separation. Summaries of these matrices are then calculated. Three which are often suggested are, angular second moment ij P 2ij contrast ij (i −j )2Pij entropy ij Pij log Pij They will all tend to respond to local roughness of texture, and the differences between them are more important in greyscale images. If the image is binary, as in an outline image, it may be simpler to just note the proportions P10 and P01 for a range of separations, and to calculate Pij only in the neighbourhood of edges. An advantage of a texture based approach is the wide availability of software for assessing texture. It is likely to be the best method of detecting differences in features such as patterns of spots or striations that often occur on the leaves or other parts of plants. Perhaps the main disadvantage is that it can be difficult to predict which texture measures will be useful in capturing patterns of interest. Other related approaches to local shape description could also be used. Boundary length density relative to leaf area, which measures the roughness of the boundary and the compactness of the leaves is easy to calculate. These and other measures of shape are discussed in Glasbey and Horgan (1995). The use of fractal descriptions of leaf boundaries, such as the fractal dimension, has been examined (Campbell, 1996; Borkowski, 1999; Mancuso, 1999). These enjoy the advantage of being well understood mathematically, and in some cases potentially offering insight into leaf growth processes.
176
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
3. Global shape Global shape is concerned with rigid objects. Shape is defined as the relative positions of the parts of an object outline — those properties of the outline which are unaffected by position and orientation. Sometimes, one may also wish to discard differences in overall size, although this will often not be appropriate for plant parts, where size does matter. An object and its reflection ( mirror-image) may also be treated as indistinguishable. Fig. 5 shows six carrots of the same variety, sliced longitudinally and photographed in a laboratory. Although the reproduction here is monochrome, the original images were in colour. Freshly cut carrots are rigid, and shape details of the whole carrot are relevant. The relative positions of particular parts (the crown and tip for example) are informative. Root shape is an important characteristic of carrots, and its quantification is not new. Bleasdale and Thompson (1963) fitted power law curves to carrot diameter variation from crown to tip. The statistical analysis of this type of shape is most often based on landmarks. These are the positions of a finite number of well-defined features of the carrot, not necessarily on the outline. We describe this approach first, and then look at what progress may be made without landmarks.
3.1. Shape analysis with landmarks 3.1.1. Defining landmarks Landmarks are well defined positions on an object, whose relative positions, when taken together, capture all of the relevant shape information about the object. For example, on a human face, one could use features such as the edges of the mouth, the pupil of the eye etc. It is assumed that we can assign these features horizontal and vertical co-ordinate positions, and then proceed to analyse these
Fig. 5. Six carrots of variety Rusty.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
177
Fig. 6. Landmark positions on a carrot outline. O denotes the centroid. The other positions are explained in the text.
much simplified data. For three dimensional objects, we may use three co-ordinates. Landmark positions may be located manually, by a suitable automatic method (as below). In the example we use here, however, there are few natural biologically definable landmark positions on a carrot. We may use geometrically defined positions instead. Images are first processed to provide silhouettes by thresholding the grey levels to separate the carrot from background, and removing any small (i.e. non-carrot) connected sets of pixels that remained in the background. Positions can then be selected on each outline. For the carrots this was done as shown in Fig. 6. The landmarks were found automatically as follows. 1. The left most pixel position (A). 2. The most ‘north-eastern’ (B) and ‘south-eastern’ (C) pixel positions. Their midpoint (D) was taken as the right (crown) end of the carrot. 3. For seven equally spaced horizontal positions between A and D, the top and bottom Y co-ordinate of the carrot outline was obtained. Since the horizontal positions of the landmarks between A and D are linearly determined from those of A and D, they may be discarded. We have thus reduced the outline shape to 20 variables, the co-ordinates of A, B and C, and the top and bottom of the carrot at seven intermediate points.
3.1.2. Statistical analysis The statistical analysis of shape using landmarks has been well developed. A good introduction to the subject is given by Dryden and Mardia (1998). There are two established approaches. The first is associated with Bookstein (1978, 1991, 1997). Two landmarks are considered fixed, and the positions of the rest are considered relative to them. By mapping the fixed landmark positions to a horizontal line segment, the relative positions of the other can be plotted and described using ordinary arithmetic. This approach has the advantage of being readily understood and handled, and amenable to standard multivariate statistical methods. It has the disadvantage that the choice of fixed landmarks is arbitrary, and results may be dependent on this. These Bookstein shape co-ordinates are best suited to situations where shape variations are small relative to the scale of the
178
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
whole object. The Kendall co-ordinate system (see Kendall, 1989) is similar, but less widely known. The other approach to landmark-based shape analysis is based on Procrustes rotations, where the first stage is to align the shapes, by translation, scaling and rotation, to match each other as closely as possible. A full description is presented by Goodall (1991a). For a biological application see Glasbey et al. (1995). The resulting statistical analysis is more demanding, but can claim advantages of full objectivity and optimal use of the data. Either Bookstein or Procrustes co-ordinates could be used for the set of carrot landmarks. We have used an approach which can be considered as a variation on the Procrustes co-ordinates, which was simpler to use and could readily handle the processing of data from new carrots. For ease of handling in later processing stages, the carrots pixels (not just the outline) were expressed relative to the position of their centroid (the mean i and j co-ordinates). The orientation of the carrots, which only varied slightly, was obtained as that of a line from the tip to the centre of the crown (see Fig. 6). The landmark positions were recorded relative to the centroid, and rotated to correspond to a horizontal orientation for the whole carrot. No further Procrustes matching was done. In particular, there was no size scaling, since carrot size was considered relevant, not something to be discarded.
3.1.3. Principal component analysis Principal component analysis (Krzanowski, 1993) is a common starting point for examining and describing variation in shape. These are linear combinations of the original shape co-ordinates which express as much as possible of the total variation among the co-ordinates, while being orthogonal (uncorrelated) to each other. Examination of the effect of altering each component will indicate the main types of shape variation present in the original landmarks. The principal components of the 20 outline shape variables were obtained from their covariance matrix. These have been termed ‘eigenshapes’ in the biological literature (Ray, 1992), although Goodall (1991b) uses the term in a different context. The effect of varying individual components is shown for the first six in Fig. 7. The variance accounted for and a suggested interpretation is shown in Table 1. Components four and five change sign on reflection of the carrot about the line AD, an irrelevant change. Only the absolute value is informative, and so this was used for matching and discrimination purposes. This is an inelegant solution. It would be more desirable to account for the reflection symmetry in the formulation of the principal component analysis, or to process the images and reflect some carrots to ensure that, for example, they all ‘curved the same way’. The question arises as to how many principal component scores are useful to record, a question common to all applications of this technique, and not confined to its use in shape. Some guidance is offered by Krzanowski (1993).
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
179
3.1.4. Other approaches Principal component analysis is often used as a first step in examining shape data, as it is otherwise difficult to get a good impression of what variation is present. If desired, it is then possible, in the context of either Bookstein or Goodall co-ordinates, to develop versions of many other statistical tools, summaries, correlation and regression, analysis of variance and discriminant analysis. For more details, see Dryden and Mardia (1998). A weakness of the analyses described here is that they take no account of shape variations which are biologically important but subtle at the scale of the whole object. For example, the exact shape of the shoulders at the crown of the carrot is believed to vary significantly among varieties. It should be possible to account for this by having more, and more carefully placed, landmarks in such important parts of the object. The extra effort needed would result more from the trouble of
Fig. 7. Principal components of carrot shape variation. Numbers relate to the components given in Table 1.
180
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Table 1 Principal components of outline shape Component
Variance (%)
Interpretation
1 2 3 4 5 6
72.0 10.8 8.2 3.4 2.0 0.9
Short and thick 6s. long and thin Tapering (cylinder 6s. cone) Thickness Bending Asymmetry Tapering at tip
obtaining the positions of the extra landmarks than from the computational resources needed to handle them.
3.2. Shape analysis without landmarks Treating shape through landmarks has the appealing property that all the shape details in the landmarks is accessible to statistical analyses. If our landmarks are complete enough, then we are working with all the information in the shape. However, there are many situations in which landmarks are not a practical approach. It may be impractical to define them for some objects (such as the outline of a potato) which do not have well defined features, or it may be too time consuming to identify them manually when automatic methods are unavailable or unreliable. We describe in this section three other approaches which may be used in these situations.
3.2.1. Shape summaries These are features such as area/perimeter ratios, magnitudes and ratios of moments, (summaries such as the mean squared distance from the centre of gravity), maximum distances within outlines, the size of circumscribing or inscribing discs or polygons etc. They all have the property that they depend on the shape, and they are independent of position and orientation. They may, thus, be used to discriminate between shapes. Several may be used together, and may then be termed a ‘feature vector’. The advantages of using such summaries are that they are easy to understand and are usually easy and quick to calculate. The choice of features is usually based on what is expected to have discriminating power, and what may readily be calculated. A disadvantage is that we may miss useful shape attributes that are not reflected in the features used. Details and examples of these ideas can be found in Glasbey and Horgan (1995). For some biological applications, see Draper and Keefe (1989), Van de Vooren et al. (1991). 3.2.2. Fourier series Any periodic function may be represented as a sum of cosine and sine functions with same period, and multiples of it. The amplitudes of each of these functions are
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
181
easily determined, and convey information about the function. When a function does not contain rapid changes, the earliest terms in this sum, termed the Fourier series, will be a good approximation to the function. If we trace around the outline of a shape, we get two periodic functions, the horizontal and vertical co-ordinates of positions on the outline. These may be represented as a Fourier series. This is illustrated for a carrot outline in Fig. 8. The set of function amplitudes (coefficients) in the Fourier series form a shape summary, and may be used in a similar way to the shape summaries described above. Fourier series enjoy the advantage that if sufficient terms are taken, then all important features of the shape outline will be captured. A disadvantage is that a large number of terms may be required. For more information on this approach to shape representation, see Granlund (1972), Crimmins (1982). For a variation, see Zahn and Roskies (1972). For biological examples, see Rohlf and Archie (1984), Mou and Stoermer (1992).
Fig. 8. Carrot outline. Approximation by Fourier series is shown for (b) 2, (c) 5 and (d) 12 terms.
182
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
3.2.3. Whole shape methods It is also possible to handle shapes without landmarks or points of any sort, in a non-parametric way. There is a substantial cost in computational effort, but the approach has some appealing features, and can be considered simpler to deal with. We proceed as follows. Images are converted to binary values, and an object is a connected set of black pixels. Two objects are considered to have the same shape if one can be translated and rotated (and if we wish, reflected and/or scaled) so that it coincides with the other. Since we are dealing with pixel positions on a lattice, we will use the best digital approximations to rotation. If two objects have a different shape, then we may ask how much they differ. Various answers to this can be envisaged. The most natural one is that it depends on how much they overlap. We could measure the difference as the area of the non-overlapping regions of the objects when this has been minimised by translation and rotation. Another possibility is Baddeley’s delta distance (Baddeley and Molchanov, 1998), which is based on the distance transform (the distance of each point from the boundary) of the objects. Once differences between shapes have been quantified, standard statistical methods can be constructed. Between and within group variation can be compared. An average can be defined as that shape whose mean distance from the shapes in question is minimised. Principal co-ordinate analysis (Krzanowski, 1993), permits study of the structure of variation among carrots, and this can be developed to study association with other non-shape variables. Principal co-ordinate analysis is similar to principal component analysis, but is derived from a matrix of differences between pairs of observations (whereas principal component analysis is derived from the covariance or correlation matrix of a set of variables). It produces a set of uncorrelated scores for each observation (shape), with the difference between scores being the best approximation to the original shape difference matrix. Fig. 9 shows a plot of the first two principal co-ordinates from 40 carrots. Two outliers on the first component, which are unusually small carrots, and one on the second component, a highly tapered carrot, are apparent. This analysis is described in more detail by Horgan (2000). Related work, which avoids full Procrustes matching, is presented by Wright et al. (1997), while a mathematical theory is pursued by Stoyan and Molchanov (1997), Baddeley and Molchanov (1998). 4. Colour distribution The previous two sections dealt with the appearance of the outline shape of a plant part. We now consider the variation in colour within a part. In keeping with the distinction between local and global properties, we first look at the colour distribution of an object, not considering where within the object that colour is observed. It should be apparent that the differences in appearance between the Brussels Sprouts images in Fig. 1, which can be seen even in the monochrome reproduction, are more (although not completely) a matter of colour frequency than their positional distribution. This is a consequence of the flexibility of the leaves.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
183
Fig. 9. (a) Initial placing of 40 carrots; (b) The best alignment; (c) Principal co-ordinate plot based on overlap area differences.
All colours we perceive are determined by the responses they produce in three cell types in the retina with well known spectral responses. Due to this, most technological handling of colour (photography, TV signals, digital images) imitates these three components with the familiar red, green, blue (RGB) system. Any generalisation to a more full-spectrum approach (for which instrumentation is available)
184
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
could be implemented by increasing the number of components from three. This is done in the science of chemometrics (Krzanowski et al., 1995). Each pixel in a colour image will have an associated red, green and blue intensity (R,G,B). Since the image is digital, colour components will take integer values, most commonly in the range 0…255 (256= 28 intensities) or 0…4095 (4096= 212) If we ignore the pixel position, then a full description of the colour of the object requires only that we specify the three-dimensional distribution f(r,g,b), the proportion of pixels which have red intensity r, green intensity g and blue intensity b. For comparing images, it is often more convenient to work with the cumulative distribution F(r,g,b), the proportion of pixels with red intensity less than r, green less than g etc. Given two images, with colour distributions F (1)(r,g,b) and F (2)(r,g,b), we may quantify the difference between them as rgb (F (1)(r, g, b)− F (2)(r,g,b))2 or max F (1)(r, g, b) −F (2)(r, g, b) . A difference measure can, thus, be created for any pair of images. Fig. 10 shows a principal co-ordinate analysis based on differences among 48 images similar to (and including) those in Fig. 1, six from each of eight varieties. Good, but not perfect, separation of varieties can be seen.
Fig. 10. Scatterplot of first two principal co-ordinates based on the image histogram difference. Each of eight varieties is labelled with a different letter. The three varieties in Fig. 1 are T, H and G.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
185
4.1.1. Colour summaries Comparing histograms will suffice for quantifying and seeking similarities between objects. We may also wish to summarise the distribution. It is less clear how this should be done. A natural choice would be to determine the mean vector and covariance matrix. This will not be sufficient in many cases, as colour distributions will often be multimodal. Adding more moments or trying moment generating or other basis functions are unlikely to be satisfactory, as they are sensitive to high order components. We do not offer a general solution, but suggest that summaries should be proposed specifically for each application. In the case of the Brussels sprouts example, it was found (Horgan et al., 1995) that good discrimination between varieties could be obtained using the mean of R the mean of G the mean of B the proportion of pixels for which G\ 200 the proportion of pixels for which G\ 225. Here the choice of summaries was influenced by the fact that variety discrimination was assisted by summaries reflecting the proportions of very bright pixels. For an example of colour histograms in the context of wider image databases, see Colombo et al. (2000).
5. Colour detail — eigenimage analysis The global shape analysis described earlier considers variation in the outline shape, but ignores colour variation within the outline. This can also vary significantly between varieties. In this section we consider how to summarise its variation. In many situations, variation within the outline can be studied by adaptation of the tools already described. We look for features within the outline, attempt to isolate them, and then look at their shape or colour summaries. This might be useful for the core in the carrot images, or for the eyes in images of human faces. The advantage of this approach is that we are in full control of the features we use, and how we use them. The disadvantages are that we need to be able to isolate features, and we must know in advance which are likely to be useful. A more general approach is to seek a principal component analysis, where we now regard the colour information at each pixel as a separate variable. This application of principal component analysis to images in this way first appears to have been used for images of human faces (Kirby and Sirovich, 1990; Craw and Cameron, 1992; Lanitis et al., 1995, 1997), where the results have sometimes been termed eigenfaces.
186
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Fig. 11. Warping carrots in Fig. 5 to a common shape.
5.1.1. Warping to a common shape If the objects are placed, suitably centred, within an image and the principal components obtained, they would be influenced by outline shape variation as well as colour variation within the object. The outline shape has already been dealt with separately, and we would like to look at variation in colour within the outline only. To do this, we first distort the images so that all carrots have the same outline shape. There are many ways to warp one object or image to another. A discussion of recent ideas is given by Glasbey and Mardia (1998). In the case of the carrots, a simple algorithm was used. First, a simple linear magnification or reduction was applied to transform the line AD in each carrot to a length of 350 pixels. The mean thickness, over all carrots, at each of the 350 horizontal positions, was then obtained, and these 1-pixel-wide vertical slices were then translated and magnified or reduced by linear interpolation so as to be centred on the line AD and of mean thickness for that slice. The results of this transform, on the carrots in Fig. 5, is shown in Fig. 11. 5.1.2. Obtaining eigenimage loadings The eigenimages are simply the principal components of variation in the images, regarding the intensity at each pixel position as a separate variable. Although it may easily be done using all three colour components, it was more efficient to confine use to the green component. The blue component was too uniformly dark and the red component too uniformly bright within the carrot outline to add any additional information. There are nearly 27 000 pixel positions, so that a direct calculation of their covariance matrix (C) is not feasible. However, the number (N) of images is much less. The components may be obtained from the eigenvectors of a N× N matrix T, where
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
187
Table 2 Principal components of greylevels Component
Variance (%)
Interpretation
1 2 3 4 5 6
30.7 9.7 8.0 3.2 2.3 1.9
Core/cortex contrast Core thickness Tapering of core Core/cortex boundary brightness Crown vs. tip contrast within core Asymmetrical positioning of core
Tij =% Vi Vj, and Vi is the i th image with its mean intensity subtracted from each pixel, and the sum extends over all the pixels in the image. This calculation is tractable, although calculating each element of T, of which there are N(N+1)/2 for N carrots, requires a few seconds of computing time. The variation accounted for by the first six components, and a tentative interpretation is shown in Table 2. Fig. 12 shows images of the eigenimage loadings. For ease of interpretation, the loadings were divided into five intensity classes; brighter areas are positive and darker areas negative. The k th score for any individual carrot image Ui can be obtained by multiplying the eigenimage by the warped carrot image.
Fig. 12. Carrot eigenimage loadings. Numbers refer to components in Table 2.
188
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Sik =% EkUi, summation being over pixel positions. These scores taken together are a summary of the variation within the object outline, and may be used for matching and discrimination.
6. Discussion Digital images typically have 105 –107 pixels, and so are observations of very high dimension. These dimensions have much structure. In the biological sciences at least, there is also considerable variation between objects. It is this that makes image analysis a demanding discipline, and one with many unsolved problems. In this paper, we have shown how multivariate techniques of data reduction can be used to condense the huge amount of image data to a tractable set of features. The approaches described can be used in many situations where we have to compare a number of images of the same type. They have the appeal that they are able to extract features of relevance, and are able to do so without needing to use any prior knowledge of those features which vary among the objects of study. There are many aspects to this work where developments can be pursued. These are of two types. The first is that the techniques for object characterisation we have presented can be further refined. For the carrot application we have used for illustration, this should be most fruitful in the study of global shape. The landmarks we have used may be failing to capture some features of interest. This could be improved by defining a richer set of landmarks. For example, in the case of carrots, more accurate identification of the position and shape of the crown should help. Some of the difficulties in shape description may be overcome by studying shape without landmarks, and we intend to investigate this further. The second aspect of the research which might be developed is appropriate accounting for the various sources of variation in the appearance of plants. We have not so far attempted to deal with the structure of variation in crop measurements that is known to occur between years and between the sites where the crops are grown, or the extent to which these interact with each other and with specific crop varieties. This does not necessarily require new methods of image manipulation. A more complex modelling of multivariate variation, beyond the simple one-level approach of principal component analysis, should prove fruitful. Knowledge of the main sources of such variation can be useful to crop specialists and to industry in producing more uniform plant material.
Acknowledgements This work was supported by the Scottish Agricultural Science Agency and the Scottish Executive Rural Affairs Department.
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
189
References Baddeley, A., Molchanov, I., 1998. Averaging of random sets based on their distance functions. J. Math. Imaging Vision 8, 79–92. Barnsley, M.F., 1993. Fractal Image Compression Wellesley Peters. Bleasdale, J.K.A., Thompson, R., 1963. An objective method of recording and comparing the shapes of carrot roots. J. Hort. Sci. 38, 232–241. Bookstein, F.L., 1978. The Measurement of Biological Shape and Shape Change. Springer-Verlag, Berlin. Bookstein, F.L., 1991. Morphometric Tools for Landmark Data: Geometry and Biology. Cambridge University Press, Cambridge. Bookstein, F.L., 1997. Shape and the information in medical images: a decade of the morphometric synthesis. Comput. Vis. Image Understanding 66, 97 – 118. Borgefors, G., 1986. Distance transforms in digital images. Comput. Vis. Graphics Image Processing 34, 344–371. Borkowski, W., 1999. Fractal dimension based features are useful descriptors of leaf complexity and shape. Can. J. Forest Res.-Revue Canadienne de Recherche Forestiere 29, 1301– 1310. Campbell, R.D., 1996. Describing the shapes of fern leaves: a fractal geometrical approach. Acta Biotheoretica 44, 119–142. Choubey, S.K., Raghavan, V.V., 1997. Generic and fully automatic content-based image retrieval using color. Pattern Recognition Lett. 18, 11 – 13. Colombo, C., Del Bimbo, A., Genovesi, I., 2000. Interactive image retrieval by color distribution content. Integrated Comput. Aided Eng. 7, 79 – 88. Craw, I., Cameron, P., 1992. Face Recognition by computer. In: Proc. Br. Machine Vis. Conf. Springer-Verlag, Berlin, pp. 489–507. Crimmins, T.R., 1982. A complete set of Fourier descriptors for two-dimensional shapes. IEEE Transactions on Systems Man Cybernetics 12, 848– 855. Draper, S.R., Keefe, P.D., 1989. Machine vision for the characterisation and identification of cultivars. Plant Varieties Seeds 2, 53–62. Dryden, I.L., Mardia, K.V., 1998. Statistical Shape Analysis. Wiley, Chichester. Glasbey, C.A., Horgan, G.W., 1995. Image Analysis for the Biological Sciences. Wiley, Chichester. Glasbey, C.A., Mardia, K.V., 1998. A review of image warping methods. J. Appl. Stat. 25, 155– 171. Glasbey, C.A., Horgan, G.W., Gibson, G.J., Hitchcock, D., 1995. Fish shape analysis using landmarks. Biometrical J. 37, 481–495. Goodall, C., 1991a. Procrustes methods in the statistical analysis of shape (with discussion). J. Royal Stat. Soc. Series B 53, 285–339. Goodall, C., 1991b. Eigenshape analysis of a cut-grow mapping for triangles, and its application to phyllotaxis in plants. SIAM J. Appl. Math. 51, 775– 798. Granlund, G.H., 1972. Fourier preprocessing for hand print character recognition. IEEE Transactions Comput. 21, 195–201. Haralick, R.M., Shanmugam, K. Dinstein, I., 1973. IEEE Transactions on Syst. Man Cybernetics, SMC-3, 610–621. Horgan, G., Talbot, M., Davey, J., 1995. Plant variety colour assessment using a still video camera. J. Plant Varieties Seeds 8, 161–170. Horgan, G., Talbot, M., Davey, J., 2000. Use of statistical image analysis to discriminate carrot cultivars. Comput. Electron. Agric. 31, 191– 199. Horgan, G.W., 2000. Principal component analysis of random particles. J. Math. Imaging Vis. 12, 169–175. Huang, C.L., Huang, D.H., 1998. A content-based image retrieval system. Image Vis. Comput. 16, 149–163. Kendall, D.G., 1989. A survey of the statistical theory of shape. Stat. Sci. 4, 87 – 120. Kirby, M., Sirovich, L., 1990. Application of the Karhunen-Loeve procedure for the characterisation of human faces. IEEE Pattern Anal. Machine Intelligence 12, 103– 108.
190
G.W. Horgan / Computers and Electronics in Agriculture 31 (2001) 169–190
Krzanowski, W.J., 1993. Principles of Multvariate Analysis: A User’s Perspective. Oxford University Press, Oxford. Krzanowski, W.J., Jonathan, P., McCarthy, W.V., Thomas, M.R., 1995. Discriminant-analysis with singular covariance matrices-methods and applications to spectroscopic data. Appl. Stat. 44, 101–115. Lanitis, A., Taylor, C.J., Cootes, T.F., 1995. Automatic face identification system using flexible appearance models. Image Vision Comput. 13, 393– 401. Lanitis, A., Taylor, C.J., Cootes, T.F., 1997. Automatic interpretation and coding of face images using flexible models. IEEE Transactions Pattern Anal. Machine Intelligence 19, 743– 756. Mancuso, S., 1999. Fractal geometry-based image analysis of grapevine leaves using the box counting algorithm. Vitis 38, 97–100. Mehtre, B.M., Kankanhalli, M.S., Lee, W.F., 1997. Shape measures for content based image retrieval: A comparison. Information Processing Management 33, 319– 337. Mou, D., Stoermer, E.F., 1992. Separating Tabellaria (Bacillariophyceae) shape groups based on Fourier descriptors. J. Phycol. 28, 386–395. Ray, T.S., 1992. Landmark eigenshape analysis-homologous contours-leaf shape in syngonium (Araceae). Am. J. Botany 79, 69–76. Reed, T.R., Dubuf, J.M.H., 1993. A review of recent texture segmentation and feature-extraction techniques. CVGIP-Image Understanding 57, 359– 372. Rohlf, E.J., Archie, J.W., 1984. A comparison of Fourier methods for the description of wing shape in mosquitoes (Diptera: Culicidae). Syst. Zool. 33, 302– 317. Serra, J., 1982. Image Analysis and Mathematical Morphology. Academic Press, London. Stoyan, D., Molchanov, I.S., 1997. Set-valued means of random particles. J. Math. Imaging Vis. 7, 111–121. Van de Vooren, J.G., Polder, G., van der Heijden, G.W.A.M., 1991. Application of image analysis for variety testing of mushroom. Euphytica 57, 245– 250. Wright, D., Stander, J., Nicolaides, K., 1997. Nonparametric density estimation and discrimination from shapes. Appl. Stat. 46, 365–382. Zahn, T.C., Roskies, R.Z., 1972. Fourier descriptors for plane closed curves. IEEE Transactions Comput. C21, 269–281.
.