- Email: [email protected]
Enhancing colour differences in images of diseased mushrooms
Computers and Electronics in Agriculture 26 (2000) 187 – 198 www.elsevier.com/locate/compag
Enhancing colour differences in images of diseased mushrooms Tu¨nde Vı´zha´nyo´ *, Jo´zsef Felfo¨ldi Department of Physics and Control, Uni6ersity of Horticulture and Food Industry, Somlo´i u´t 14 -16, Budapest H-1118, Hungary
Abstract Discoloration of mushrooms (senescence, damage, and bacterial infection) is an undesirable phenomenon in mushroom houses and on the market. Simple cluster analysis was not sufficient to discriminate the browning caused by disease from the natural browning of the mushroom. The transformation of RGB values to a* and b* colour components and the elimination of intensity gave a definitely better separation for the diseases. The best rate of separation was achieved by means of a vectorial normalisation, which was developed on the basis of the statistical analysis of the distribution of the colour points. © 2000 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Image analysis; Image processing; Colour enhancement; Browning; Mushroom
1. Introduction The increasing mushroom production world-wide induces higher expectations for mushroom quality. Both consumers and retailers become more critical though they are ready to pay for the higher quality. High quality mushrooms are generally defined as fresh, white, blemish-free, clean, uniform closed cups or buttons (Berendse, 1984). The colour of the cap is the most important consideration of fresh mushrooms, since it is the first characteristic consumers notice. The cultivated mushroom (Agaricus bisporus) is highly susceptible to blemishes caused by a range of bacterial and fungal pathogens, and discoloration induced by bruising, storage and physiological disorders. * Corresponding author. 0168-1699/00/$ - see front matter © 2000 Elsevier Science Ireland Ltd. All rights reserved. PII: S 0 1 6 8 - 1 6 9 9 ( 0 0 ) 0 0 0 7 1 - 5
188
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
The most important disease on mushroom farms is brown blotch. By the first description of the disease (Tolaas, 1915) it induces a discoloration of all or part of the cap with the colour varying from pale yellow to a rich chocolate brown. Pseudomonas tolaasii, which causes brown blotch, is a very common bacterium and is likely to be present on most mushroom farms. It is generally agreed by growers and scientists that the disease occurs when the relative humidity is high in the mushroom house and the surface of the mushroom remains wet (Gandy, 1967; Nair, 1974). The causative organism of another prevalent disease (ginger blotch) on the farms is Pseudomonas gingerii. The discoloration of ginger blotch is readily distinguishable from the chocolate-brown brown blotch. The discoloration is initially paled yellowish-red-brown, which finally becomes a reddish, ginger colour (Wong et al., 1982). None of the above mentioned diseases is desirable on the mushroom farms. Nowadays, when more and more farms have mushroom harvester machines (Reed et al., 1995) combined with an appropriate camera and software to select the mushrooms by size for picking image analysis could be used for other tasks as well. Computer vision has been applied to objective measurement of developmental stage of mushrooms (van Loon, 1996). It was found that cap opening of mushrooms correlated the best with the stage of development except for tightly closed mushrooms. The relative gill size proved to be the best alternative to determine the developmental stage. Gill colour, determined as the mode of its grey value histogram, also showed a good correlation with the developmental stage. Heinemann et al. (1994) developed an automated mushroom inspection and grading system based on a combination of a machine vision and an expert system. The goal of their research was to combine the individual grading criteria algorithms into an inspection algorithm that will determine the overall quality of the mushroom. Monochrome imaging was used for the grading analysis using intensity to determine colour of mushrooms. The inspection system was trained based on one inspector’s evaluations, and comparisons were made between the trained vision system versus the inspectors as well as between the inspectors. It was found that on the average, the percentage of misclassifications by the trained vision system when compared to the inspectors’ evaluation was no greater than the difference in evaluation between the inspectors. Most image analysis based mushroom quality assessment methods, as it can be seen above, use greyscale image for grading, though the fine differences between the mechanical damage and the diseases can be found in the colour information. By the spectral analyses on the colour of different mushroom diseases Vı´zha´nyo´ and Tillett (1998) concluded that the colour of the developed, senescent mushroom differs from any browning of mushroom diseases. There is a definite need for research dealing with the combination of machine vision and artificial intelligence techniques for mushroom inspection. By means of suitable software and learning material an alarm system developed for warning in case of disease outbreaks could enable the grower to stop the spread of the disease in time. Rapid diagnosis of the cause of particular blemishes is important in selecting the appropriate control measures; however, this can be difficult without a
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
189
trained pathologist, and even then, identification may take several days. Computer analysis of camera images of mushroom caps may offer several advantages over visual assessment. It may be possible to discriminate types of blemish mathematically from the spectral characteristics. Information about blemishes with known causes can be stored and compared with new images. The equipment could be operated by a non-specialist, and give an immediate, objective result. The objective was to find the best method by which the diseased spots can be separated from the healthy but possibly mechanically damaged or matured cap surface and to try to discriminate the two tested diseases from each other. Intensity normalisation and image transformation techniques were applied in order to enhance colour differences in true-colour images of diseased mushrooms.
2. Materials and methods
2.1. Mushroom culti6ars Champignon mushrooms (A. bisporus) from strain A12 were used for the investigations. The tests were conducted on mushrooms with blemishes on the cap surface, which were experimentally induced caused by known mushroom pathogens (P. tolaasii, P. gingerii ) in the mushroom houses of Horticultural Research International Wellesbourne, UK. The mushroom samples were picked in a well-developed state of the certain disease, when the spots both in size and colour showed the typical features of the diseases. Typical examples of the diseased mushroom groups are represented in Fig. 1(a, b) respectively.
2.2. Image acquisition and pre-processing Images were captured with a SONY DXC-151 AP Colour Video Camera in a chamber where the lighting conditions were controlled to give diffuse lighting. The size of the chamber was 840×840× 600 mm. The samples were placed on trays covered with a non-reflecting black material as the background of the field of view. The camera was mounted above the top of the chamber with 450 mm distance from the lens to the mushroom samples. The lens
Fig. 1. (a) Brown blotch; (b) ginger blotch.
190
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
type was HOYA 49 mm +1 Computar TV Zoom (1:1.6/12.5-75) lens. The images were stored at a partial resolution of 768 × 479 pixels. The captured images were resized and converted to 320 × 200 pixels Windows bitmap files for the later analysis and processing.
3. Image transformation methods and results
3.1. Cluster analysis The program developed for the cluster analysis of true-colour bitmap images is for classification of the image pixels described by their RGB values into groups of different colour properties. The base of the classification is the Euclidean distance in the RGB colour space. The result of the analysis is a true colour image for each cluster consisting of all the pixels belonging to the given cluster. The cluster analysis provides the average RGB values and the pixel number of the clusters as well. As the result of the analysis a clustered part of the mushroom shown in Fig. 1(b) consisting of the diseased spots is shown in Fig. 2. The classification was suitable for the disease though the cluster also contained those mushroom parts which had similar intensity and colour as the spots. These parts were the pixels along the edge of the mushroom caps. The analysis of the other samples of the tested diseases gave more or less the same results. Therefore, further steps were needed to enhance the existing colour differences between diseased and damaged areas in the mushroom image.
Fig. 2. Misclassification of the cluster analysis.
3.2. Intensity elimination Because the intensity of the pixels of the mushroom surface varies in a wide range while the colour distance of the diseased and healthy pixels with similar intensity is relatively small. The above-described misclassification occurred with cluster analysis. Therefore the problem can be solved by image transformation increasing the distance between the clusters of different colours meantime decreasing the effect of
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
191
the similarity in intensity. The most simple way of the intensity elimination is to choose a colour representation (such as the CIE 1976 L*a*b* [CIELAB] colour system) in which the intensity information is separated from the colour information. The RGB values of the images were converted into L*, a* and b* colour components using the algorithms and coefficients according to Truppel and Herold (1996). In order to determine the difference in colour of the different origin discoloration represented by the a*b* components the intensity component (L*) was neglected and a pseudocolour image was calculated to visualise the results of the transformation. The RGB components of the new image were calculated as follows to fit the values into the 0 255 range: R = 15 · a* +128 G =30 · b* +128 B=0 The separation was generally more effective than with the cluster analysis. The darker edges were not involved in the highlighted clusters, and the result was acceptable for some cases and diseases but still did not provide satisfactory separation. The normalisation method was used to decrease the intensity effect instead of absolute elimination of the intensity information.
3.3. Intensity normalisation For the appropriate analysis the whole colour range, the natural colour changes of the mushrooms and the trend of the difference of the diseases to the healthy mushroom have to be investigated. Fig. 3 shows the colour points of a healthy, white mushroom in the RGB colour space. It can be seen that the colour points of the healthy mushroom are situated along a line (principal axis, PA). According to the further analysis, the same principal axis characterises the natural discoloration of mushrooms.
Fig. 3. The principal axis and the colour points of a single healthy mushroom in the RGB colour space.
192
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
Fig. 4. Principal axis and colour points (the ginger blotch diseased points are highlighted).
Plotting the colour points of a diseased mushroom (Fig. 4) it can be seen that the pixels of the diseased spots (brown blotch) form a well distinguishable, separated cluster which runs parallel with the principal axis. The variance of the data is practically produced by the distribution along the principal axis. In the other directions, which are perpendicular to the line of the principal axis, the extent of the cloud is small. This could explain the problem with cluster analysis as well: the distances in the cluster are smaller between the different coloured pixels than between the different intensities. Therefore, such image transformation had to be applied which was invariant to the position of the pixels along the principal axis but enhanced the measure of the difference of the colour points from the principal axis. The simple algebraic normalisation such as I = R+G+B,
r =R/I · 255,
g =G/I · 255,
r+g+b = 255
did not provide satisfactory result. Cluster analysis performed on those normalised images generally classified the diseased spots correctly but problems occurred similar to the ones in the case of L*a*b* normalisation.
3.4. Vectorial normalisation According to the known trend of the colour variation determined by the principal axis equation a more effective vectorial normalisation method could be introduced. The method is represented in Fig. 5. A (R, G, and B): white endpoint of the principal axis in the colour space; B (R, G, and B): brown endpoint of the principal axis in the colour space;
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
193
P (R, G, and B): colour point of each pixel of the mushroom image in the colour space; a, b and p1 are the vectors pointing from the origo to A, B or P respectively; The p1 determined by its R, G, B components is the vector to be transformed. The steps of the transformation are: v0 =b −a:
vector determining the PA
(1)
v1 =v0/ v0 :
is the unit vector of PA
(2)
p%1 =p1 −a:
the vector pointing from A to P
(3)
p2 =v1 ×(p%1 ×v1)
(4)
The p2 is the transformed colour vector, which is perpendicular to PA and pointing from PA to P. The transformation of the colour components of each pixel belonging to the mushroom image is based on the determination of the equation of the principal axis and the determination of the average ‘white point’ (A) and ‘brown point’ (B) calculated analysing several healthy or naturally discolored mushroom images. The p1 vector was transformed to give the distance and the direction of the P colour point to the principal axis in the RGB colour space (Eqs. (1)–(4).). Therefore, in p2 the intensity is eliminated. The distribution and the range of the components of the normalised vector (p2) are represented in Fig. 6(a, b). It can be seen in the histograms of the RGB components of p2 that in a ginger blotch diseased mushroom the transformed vectors related to the healthy mushroom areas give a peak around zero in each colour components, while the discolouration formed a separate peak in the red and the blue colour component. In the direction of green there was no such effect of the discolouration. From the histograms of the brown blotch disease it could be concluded that the development of the disease had two effects. One is the formation of spots, which are obviously visible, and peculiar to the disease, the other one is that the ground colour of the remaining mushroom cap changes to a slight beige colour.
Fig. 5. Base vectors used for the vectorial normalisation.
194
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
Fig. 6. (a) Histogram of the RGB components of p2 vectors in a ginger blotch diseased mushroom. (b) Histogram of the RGB components of the p2 vectors in a brown blotch diseased mushroom.
To find out more about the discoloured areas and the situation of their pixels related to the PA a projection of the colour points to a plain perpendicular to the principal axis was plotted (Fig. 7a, b.). It was possible to specify a threshold distance by the statistical analysis of the distribution of the healthy colour points around the centre. All the colour points situated in-between the threshold distance in any directions in the colour space were considered as healthy mushroom points (Fig. 7a).
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
195
Fig. 7. (a) A projection perpendicular to the principal axis of the healthy mushroom colour points. (b) A projection of the colour points of a ginger blotch diseased mushroom.
196
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
The colour points out of this range moved from the centre representing the healthy mushroom points into two opposite directions (Fig. 7b). A bigger amount moves to the less blue area while the others form a cluster in the opposite direction. It was supposed that the different directions gave different mushroom colour and area. To determine which coloured parts of the mushroom belonged to which move the original images were re-coloured according to the two directions (Fig. 8). It could be concluded that the points moving to the direction of smaller blue component consisted of the diseased parts of the mushroom whilst the others belonged to the shady, edge parts of the mushroom cap. Considering this fact, the diseased areas of any mushrooms in the images could be found. This method was applied to the detection of diseased spots of test mushroom images. An expert panel collected three different groups of twenty mushroom images: group of mushrooms infected with P. tolaasii (brown blotch disease), group of mushrooms infected with P. gingerii (ginger blotch), and the group of healthy mushrooms. The last group contained some hand picked therefore slightly bruised samples. Using the method for these groups all of the diseased mushrooms were identified as ‘diseased’, which means irrespective of the type of disease, the blemishes were detected. In none of the images of the healthy mushrooms was any pixel detected as ‘diseased’. As the graphic output of the method, the re-coloured images with the spots detected and painted as ‘diseased’ were evaluated. The results were in a very good accordance with the assessment of the expert panel. The RGB colour components of the pixels, which were detected as diseased, were stored in files. Applying the method for samples of the brown blotch or ginger blotch diseases a representative sample of the colour components for the tested diseases as a test material was collected for a discriminant analysis. The SPSS Systat 7.0 for Windows (SPSS Inc., Chicago, 1997) statistical program package provided the Fisher’s linear discriminant functions to classify brown blotch from ginger blotch disease. The total correct classification ratio was 85% on the test material. By use of a re-colouring program the healthy mushroom area was coloured white, and two different colours were used to re-colour the brown blotch and the ginger blotch diseases by the obtained Fisher’s discriminant functions. Because each sample group contained exclusively a single disease (namely, it did not happen that on a mushroom cap both the diseases
Fig. 8. Re-coloured sample; painted area was recognised as ‘diseased’.
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
197
were present) the misclassification could be evaluated on the basis of the ratio of the pixel numbers of each class represented by the three colours in the re-coloured images. It could be concluded that 81% of the diseased area on the test material was correctly classified. 4. Conclusions Methods were tested to recognise and identify bacterial disease caused discoloration in mushroom images recorded by machine vision system. Vectorial normalisation method was developed to decrease the effect of the natural discoloration of the mushroom surface and increase the differences in the image caused by disease. Conclusions were as follows: (1) Both the colour points of the healthy and the naturally senescent mushrooms are situated along a line in the RGB colour space (principal axis). (2) If the mushroom is diseased, the colour points of the disease are off the trend of the healthy mushroom points. (3) Method was developed to enhance the difference of the diseased colour points from the principal axis and to suppress the difference along the principal axis. (4) A threshold distance from the principal axis was found to separate the healthy colour points from the diseased colour points. (5) The developed method was effective in detection of discoloration caused by disease and strictly discriminated it from other discoloration of different origin. (6) The method identified all of the diseased spots as ‘diseased’ and none of the healthy, senescent mushroom parts were detected as ‘diseased’. (7) The results in identification of the two tested diseases were acceptable. To test the algorithm the method developed on the characteristics of the brown blotch and the ginger blotch disease will be applied on samples of other mushroom diseases. Further aim could be to involve the shape and the texture information into the disease identification as well, and to investigate the effectiveness of the method on different developmental stages of the diseases.
Acknowledgements The authors wish to thank Ralph Noble and others at Horticultural Research International, Wellesbourne, UK for providing the diseased mushrooms to perform this work.
References Berendse, H., 1984. Attitudes to mushrooms revealed in bureau survey. Supplement to the Fruit Trades Journal, November 9.
198
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
Gandy, D.G., 1967. The epidemiology of bacterial blotch of the cultivated mushroom. Report GlassHouse Crops Research Institute, UK 1966, 150 – 154. Heinemann, P.H., Huges, R., Morrow, C.T., Sommer III, H.J., Beelman, R.B., Wuest, P.J., 1994. Grading of mushrooms using a machine vision system. Trans. ASAE 37(5): 1671 – 1677. van Loon, P.C.C., 1996. Het bepalen van het ontwikkelingsstadium bij de champignon met computer beeldanalyse. Champignoncultuur 40 (9), 347 – 353. Nair, N.G., 1974. Methods of control for bacteriological blotch disease of the cultivated mushroom with special reference to biological control. Mushroom J. 16, 140 – 144. Reed, J.N., Crook, S., He, W., 1995. Harvesting mushrooms by robot. In: Elliott (Ed.), Science and Cultivation of Edible Fungi, Balkema, Rotterdam, pp. 385 – 391. Tolaas, A.G., 1915. A bacterial disease of cultivated mushrooms. Phytopathol. 5, 51 – 54. Truppel, I., Herold, B., 1996. Bedeuntung von Licht und Farbe fu¨r die Qualita¨tsbeurteilung von Fru¨chten. Bornimer Agrartechnische Berichte. Heft 11, 61 – 73. Vı´zha´nyo´, T., Tillett, R.D., 1998. Analysis of mushroom spectral characteristics. Proc. AgEng’98 International Conference on Agricultural Engineering, Oslo, 24 – 27 August 1998. Paper No. 98F08019981–7. Wong, W.C., Fletcher, J.T., Unsworth, B.A., Preece, T.F., 1982. A note on ginger blotch, a new bacterial disease of the cultivated mushroom, Agaricus bisporus. J. Appl. Bacteriol. 52, 43 – 48.
.
Enhancing colour differences in images of diseased mushrooms Tu¨nde Vı´zha´nyo´ *, Jo´zsef Felfo¨ldi Department of Physics and Control, Uni6ersity of Horticulture and Food Industry, Somlo´i u´t 14 -16, Budapest H-1118, Hungary
Abstract Discoloration of mushrooms (senescence, damage, and bacterial infection) is an undesirable phenomenon in mushroom houses and on the market. Simple cluster analysis was not sufficient to discriminate the browning caused by disease from the natural browning of the mushroom. The transformation of RGB values to a* and b* colour components and the elimination of intensity gave a definitely better separation for the diseases. The best rate of separation was achieved by means of a vectorial normalisation, which was developed on the basis of the statistical analysis of the distribution of the colour points. © 2000 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Image analysis; Image processing; Colour enhancement; Browning; Mushroom
1. Introduction The increasing mushroom production world-wide induces higher expectations for mushroom quality. Both consumers and retailers become more critical though they are ready to pay for the higher quality. High quality mushrooms are generally defined as fresh, white, blemish-free, clean, uniform closed cups or buttons (Berendse, 1984). The colour of the cap is the most important consideration of fresh mushrooms, since it is the first characteristic consumers notice. The cultivated mushroom (Agaricus bisporus) is highly susceptible to blemishes caused by a range of bacterial and fungal pathogens, and discoloration induced by bruising, storage and physiological disorders. * Corresponding author. 0168-1699/00/$ - see front matter © 2000 Elsevier Science Ireland Ltd. All rights reserved. PII: S 0 1 6 8 - 1 6 9 9 ( 0 0 ) 0 0 0 7 1 - 5
188
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
The most important disease on mushroom farms is brown blotch. By the first description of the disease (Tolaas, 1915) it induces a discoloration of all or part of the cap with the colour varying from pale yellow to a rich chocolate brown. Pseudomonas tolaasii, which causes brown blotch, is a very common bacterium and is likely to be present on most mushroom farms. It is generally agreed by growers and scientists that the disease occurs when the relative humidity is high in the mushroom house and the surface of the mushroom remains wet (Gandy, 1967; Nair, 1974). The causative organism of another prevalent disease (ginger blotch) on the farms is Pseudomonas gingerii. The discoloration of ginger blotch is readily distinguishable from the chocolate-brown brown blotch. The discoloration is initially paled yellowish-red-brown, which finally becomes a reddish, ginger colour (Wong et al., 1982). None of the above mentioned diseases is desirable on the mushroom farms. Nowadays, when more and more farms have mushroom harvester machines (Reed et al., 1995) combined with an appropriate camera and software to select the mushrooms by size for picking image analysis could be used for other tasks as well. Computer vision has been applied to objective measurement of developmental stage of mushrooms (van Loon, 1996). It was found that cap opening of mushrooms correlated the best with the stage of development except for tightly closed mushrooms. The relative gill size proved to be the best alternative to determine the developmental stage. Gill colour, determined as the mode of its grey value histogram, also showed a good correlation with the developmental stage. Heinemann et al. (1994) developed an automated mushroom inspection and grading system based on a combination of a machine vision and an expert system. The goal of their research was to combine the individual grading criteria algorithms into an inspection algorithm that will determine the overall quality of the mushroom. Monochrome imaging was used for the grading analysis using intensity to determine colour of mushrooms. The inspection system was trained based on one inspector’s evaluations, and comparisons were made between the trained vision system versus the inspectors as well as between the inspectors. It was found that on the average, the percentage of misclassifications by the trained vision system when compared to the inspectors’ evaluation was no greater than the difference in evaluation between the inspectors. Most image analysis based mushroom quality assessment methods, as it can be seen above, use greyscale image for grading, though the fine differences between the mechanical damage and the diseases can be found in the colour information. By the spectral analyses on the colour of different mushroom diseases Vı´zha´nyo´ and Tillett (1998) concluded that the colour of the developed, senescent mushroom differs from any browning of mushroom diseases. There is a definite need for research dealing with the combination of machine vision and artificial intelligence techniques for mushroom inspection. By means of suitable software and learning material an alarm system developed for warning in case of disease outbreaks could enable the grower to stop the spread of the disease in time. Rapid diagnosis of the cause of particular blemishes is important in selecting the appropriate control measures; however, this can be difficult without a
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
189
trained pathologist, and even then, identification may take several days. Computer analysis of camera images of mushroom caps may offer several advantages over visual assessment. It may be possible to discriminate types of blemish mathematically from the spectral characteristics. Information about blemishes with known causes can be stored and compared with new images. The equipment could be operated by a non-specialist, and give an immediate, objective result. The objective was to find the best method by which the diseased spots can be separated from the healthy but possibly mechanically damaged or matured cap surface and to try to discriminate the two tested diseases from each other. Intensity normalisation and image transformation techniques were applied in order to enhance colour differences in true-colour images of diseased mushrooms.
2. Materials and methods
2.1. Mushroom culti6ars Champignon mushrooms (A. bisporus) from strain A12 were used for the investigations. The tests were conducted on mushrooms with blemishes on the cap surface, which were experimentally induced caused by known mushroom pathogens (P. tolaasii, P. gingerii ) in the mushroom houses of Horticultural Research International Wellesbourne, UK. The mushroom samples were picked in a well-developed state of the certain disease, when the spots both in size and colour showed the typical features of the diseases. Typical examples of the diseased mushroom groups are represented in Fig. 1(a, b) respectively.
2.2. Image acquisition and pre-processing Images were captured with a SONY DXC-151 AP Colour Video Camera in a chamber where the lighting conditions were controlled to give diffuse lighting. The size of the chamber was 840×840× 600 mm. The samples were placed on trays covered with a non-reflecting black material as the background of the field of view. The camera was mounted above the top of the chamber with 450 mm distance from the lens to the mushroom samples. The lens
Fig. 1. (a) Brown blotch; (b) ginger blotch.
190
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
type was HOYA 49 mm +1 Computar TV Zoom (1:1.6/12.5-75) lens. The images were stored at a partial resolution of 768 × 479 pixels. The captured images were resized and converted to 320 × 200 pixels Windows bitmap files for the later analysis and processing.
3. Image transformation methods and results
3.1. Cluster analysis The program developed for the cluster analysis of true-colour bitmap images is for classification of the image pixels described by their RGB values into groups of different colour properties. The base of the classification is the Euclidean distance in the RGB colour space. The result of the analysis is a true colour image for each cluster consisting of all the pixels belonging to the given cluster. The cluster analysis provides the average RGB values and the pixel number of the clusters as well. As the result of the analysis a clustered part of the mushroom shown in Fig. 1(b) consisting of the diseased spots is shown in Fig. 2. The classification was suitable for the disease though the cluster also contained those mushroom parts which had similar intensity and colour as the spots. These parts were the pixels along the edge of the mushroom caps. The analysis of the other samples of the tested diseases gave more or less the same results. Therefore, further steps were needed to enhance the existing colour differences between diseased and damaged areas in the mushroom image.
Fig. 2. Misclassification of the cluster analysis.
3.2. Intensity elimination Because the intensity of the pixels of the mushroom surface varies in a wide range while the colour distance of the diseased and healthy pixels with similar intensity is relatively small. The above-described misclassification occurred with cluster analysis. Therefore the problem can be solved by image transformation increasing the distance between the clusters of different colours meantime decreasing the effect of
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
191
the similarity in intensity. The most simple way of the intensity elimination is to choose a colour representation (such as the CIE 1976 L*a*b* [CIELAB] colour system) in which the intensity information is separated from the colour information. The RGB values of the images were converted into L*, a* and b* colour components using the algorithms and coefficients according to Truppel and Herold (1996). In order to determine the difference in colour of the different origin discoloration represented by the a*b* components the intensity component (L*) was neglected and a pseudocolour image was calculated to visualise the results of the transformation. The RGB components of the new image were calculated as follows to fit the values into the 0 255 range: R = 15 · a* +128 G =30 · b* +128 B=0 The separation was generally more effective than with the cluster analysis. The darker edges were not involved in the highlighted clusters, and the result was acceptable for some cases and diseases but still did not provide satisfactory separation. The normalisation method was used to decrease the intensity effect instead of absolute elimination of the intensity information.
3.3. Intensity normalisation For the appropriate analysis the whole colour range, the natural colour changes of the mushrooms and the trend of the difference of the diseases to the healthy mushroom have to be investigated. Fig. 3 shows the colour points of a healthy, white mushroom in the RGB colour space. It can be seen that the colour points of the healthy mushroom are situated along a line (principal axis, PA). According to the further analysis, the same principal axis characterises the natural discoloration of mushrooms.
Fig. 3. The principal axis and the colour points of a single healthy mushroom in the RGB colour space.
192
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
Fig. 4. Principal axis and colour points (the ginger blotch diseased points are highlighted).
Plotting the colour points of a diseased mushroom (Fig. 4) it can be seen that the pixels of the diseased spots (brown blotch) form a well distinguishable, separated cluster which runs parallel with the principal axis. The variance of the data is practically produced by the distribution along the principal axis. In the other directions, which are perpendicular to the line of the principal axis, the extent of the cloud is small. This could explain the problem with cluster analysis as well: the distances in the cluster are smaller between the different coloured pixels than between the different intensities. Therefore, such image transformation had to be applied which was invariant to the position of the pixels along the principal axis but enhanced the measure of the difference of the colour points from the principal axis. The simple algebraic normalisation such as I = R+G+B,
r =R/I · 255,
g =G/I · 255,
r+g+b = 255
did not provide satisfactory result. Cluster analysis performed on those normalised images generally classified the diseased spots correctly but problems occurred similar to the ones in the case of L*a*b* normalisation.
3.4. Vectorial normalisation According to the known trend of the colour variation determined by the principal axis equation a more effective vectorial normalisation method could be introduced. The method is represented in Fig. 5. A (R, G, and B): white endpoint of the principal axis in the colour space; B (R, G, and B): brown endpoint of the principal axis in the colour space;
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
193
P (R, G, and B): colour point of each pixel of the mushroom image in the colour space; a, b and p1 are the vectors pointing from the origo to A, B or P respectively; The p1 determined by its R, G, B components is the vector to be transformed. The steps of the transformation are: v0 =b −a:
vector determining the PA
(1)
v1 =v0/ v0 :
is the unit vector of PA
(2)
p%1 =p1 −a:
the vector pointing from A to P
(3)
p2 =v1 ×(p%1 ×v1)
(4)
The p2 is the transformed colour vector, which is perpendicular to PA and pointing from PA to P. The transformation of the colour components of each pixel belonging to the mushroom image is based on the determination of the equation of the principal axis and the determination of the average ‘white point’ (A) and ‘brown point’ (B) calculated analysing several healthy or naturally discolored mushroom images. The p1 vector was transformed to give the distance and the direction of the P colour point to the principal axis in the RGB colour space (Eqs. (1)–(4).). Therefore, in p2 the intensity is eliminated. The distribution and the range of the components of the normalised vector (p2) are represented in Fig. 6(a, b). It can be seen in the histograms of the RGB components of p2 that in a ginger blotch diseased mushroom the transformed vectors related to the healthy mushroom areas give a peak around zero in each colour components, while the discolouration formed a separate peak in the red and the blue colour component. In the direction of green there was no such effect of the discolouration. From the histograms of the brown blotch disease it could be concluded that the development of the disease had two effects. One is the formation of spots, which are obviously visible, and peculiar to the disease, the other one is that the ground colour of the remaining mushroom cap changes to a slight beige colour.
Fig. 5. Base vectors used for the vectorial normalisation.
194
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
Fig. 6. (a) Histogram of the RGB components of p2 vectors in a ginger blotch diseased mushroom. (b) Histogram of the RGB components of the p2 vectors in a brown blotch diseased mushroom.
To find out more about the discoloured areas and the situation of their pixels related to the PA a projection of the colour points to a plain perpendicular to the principal axis was plotted (Fig. 7a, b.). It was possible to specify a threshold distance by the statistical analysis of the distribution of the healthy colour points around the centre. All the colour points situated in-between the threshold distance in any directions in the colour space were considered as healthy mushroom points (Fig. 7a).
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
195
Fig. 7. (a) A projection perpendicular to the principal axis of the healthy mushroom colour points. (b) A projection of the colour points of a ginger blotch diseased mushroom.
196
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
The colour points out of this range moved from the centre representing the healthy mushroom points into two opposite directions (Fig. 7b). A bigger amount moves to the less blue area while the others form a cluster in the opposite direction. It was supposed that the different directions gave different mushroom colour and area. To determine which coloured parts of the mushroom belonged to which move the original images were re-coloured according to the two directions (Fig. 8). It could be concluded that the points moving to the direction of smaller blue component consisted of the diseased parts of the mushroom whilst the others belonged to the shady, edge parts of the mushroom cap. Considering this fact, the diseased areas of any mushrooms in the images could be found. This method was applied to the detection of diseased spots of test mushroom images. An expert panel collected three different groups of twenty mushroom images: group of mushrooms infected with P. tolaasii (brown blotch disease), group of mushrooms infected with P. gingerii (ginger blotch), and the group of healthy mushrooms. The last group contained some hand picked therefore slightly bruised samples. Using the method for these groups all of the diseased mushrooms were identified as ‘diseased’, which means irrespective of the type of disease, the blemishes were detected. In none of the images of the healthy mushrooms was any pixel detected as ‘diseased’. As the graphic output of the method, the re-coloured images with the spots detected and painted as ‘diseased’ were evaluated. The results were in a very good accordance with the assessment of the expert panel. The RGB colour components of the pixels, which were detected as diseased, were stored in files. Applying the method for samples of the brown blotch or ginger blotch diseases a representative sample of the colour components for the tested diseases as a test material was collected for a discriminant analysis. The SPSS Systat 7.0 for Windows (SPSS Inc., Chicago, 1997) statistical program package provided the Fisher’s linear discriminant functions to classify brown blotch from ginger blotch disease. The total correct classification ratio was 85% on the test material. By use of a re-colouring program the healthy mushroom area was coloured white, and two different colours were used to re-colour the brown blotch and the ginger blotch diseases by the obtained Fisher’s discriminant functions. Because each sample group contained exclusively a single disease (namely, it did not happen that on a mushroom cap both the diseases
Fig. 8. Re-coloured sample; painted area was recognised as ‘diseased’.
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
197
were present) the misclassification could be evaluated on the basis of the ratio of the pixel numbers of each class represented by the three colours in the re-coloured images. It could be concluded that 81% of the diseased area on the test material was correctly classified. 4. Conclusions Methods were tested to recognise and identify bacterial disease caused discoloration in mushroom images recorded by machine vision system. Vectorial normalisation method was developed to decrease the effect of the natural discoloration of the mushroom surface and increase the differences in the image caused by disease. Conclusions were as follows: (1) Both the colour points of the healthy and the naturally senescent mushrooms are situated along a line in the RGB colour space (principal axis). (2) If the mushroom is diseased, the colour points of the disease are off the trend of the healthy mushroom points. (3) Method was developed to enhance the difference of the diseased colour points from the principal axis and to suppress the difference along the principal axis. (4) A threshold distance from the principal axis was found to separate the healthy colour points from the diseased colour points. (5) The developed method was effective in detection of discoloration caused by disease and strictly discriminated it from other discoloration of different origin. (6) The method identified all of the diseased spots as ‘diseased’ and none of the healthy, senescent mushroom parts were detected as ‘diseased’. (7) The results in identification of the two tested diseases were acceptable. To test the algorithm the method developed on the characteristics of the brown blotch and the ginger blotch disease will be applied on samples of other mushroom diseases. Further aim could be to involve the shape and the texture information into the disease identification as well, and to investigate the effectiveness of the method on different developmental stages of the diseases.
Acknowledgements The authors wish to thank Ralph Noble and others at Horticultural Research International, Wellesbourne, UK for providing the diseased mushrooms to perform this work.
References Berendse, H., 1984. Attitudes to mushrooms revealed in bureau survey. Supplement to the Fruit Trades Journal, November 9.
198
T. Vı´zha´nyo´, J. Felfo¨ldi / Computers and Electronics in Agriculture 26 (2000) 187–198
Gandy, D.G., 1967. The epidemiology of bacterial blotch of the cultivated mushroom. Report GlassHouse Crops Research Institute, UK 1966, 150 – 154. Heinemann, P.H., Huges, R., Morrow, C.T., Sommer III, H.J., Beelman, R.B., Wuest, P.J., 1994. Grading of mushrooms using a machine vision system. Trans. ASAE 37(5): 1671 – 1677. van Loon, P.C.C., 1996. Het bepalen van het ontwikkelingsstadium bij de champignon met computer beeldanalyse. Champignoncultuur 40 (9), 347 – 353. Nair, N.G., 1974. Methods of control for bacteriological blotch disease of the cultivated mushroom with special reference to biological control. Mushroom J. 16, 140 – 144. Reed, J.N., Crook, S., He, W., 1995. Harvesting mushrooms by robot. In: Elliott (Ed.), Science and Cultivation of Edible Fungi, Balkema, Rotterdam, pp. 385 – 391. Tolaas, A.G., 1915. A bacterial disease of cultivated mushrooms. Phytopathol. 5, 51 – 54. Truppel, I., Herold, B., 1996. Bedeuntung von Licht und Farbe fu¨r die Qualita¨tsbeurteilung von Fru¨chten. Bornimer Agrartechnische Berichte. Heft 11, 61 – 73. Vı´zha´nyo´, T., Tillett, R.D., 1998. Analysis of mushroom spectral characteristics. Proc. AgEng’98 International Conference on Agricultural Engineering, Oslo, 24 – 27 August 1998. Paper No. 98F08019981–7. Wong, W.C., Fletcher, J.T., Unsworth, B.A., Preece, T.F., 1982. A note on ginger blotch, a new bacterial disease of the cultivated mushroom, Agaricus bisporus. J. Appl. Bacteriol. 52, 43 – 48.
.