- Email: [email protected]
Using the mobile phone as Munsell soil-colour sensor: An experiment under controlled illumination conditions
Computers and Electronics in Agriculture 99 (2013) 200–208
Contents lists available at ScienceDirect
Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag
Using the mobile phone as Munsell soil-colour sensor: An experiment under controlled illumination conditions Luis Gómez-Robledo a,⇑,1, Nuria López-Ruiz b,1, Manuel Melgosa a, Alberto J. Palma b, Luis Fermín Capitán-Vallvey c, Manuel Sánchez-Marañón d a
Departamento de Óptica, Facultad de Ciencias, Universidad de Granada, 18071 Granada, Spain ECsens, Departamento de Electrónica y Tecnología de Computadores, ETSIIT, Universidad de Granada, 18071 Granada, Spain Grupo de Investigación Espectrometría en Fase Sólida, Departamento de Química Analítica, Facultad de Ciencia, Universidad de Granada, 18071 Granada, Spain d Departamento de Edafología y Química Agrícola, Facultad de Ciencias, Universidad de Granada, 18071 Granada, Spain b c
a r t i c l e
i n f o
Article history: Received 30 January 2013 Received in revised form 24 July 2013 Accepted 2 October 2013
Keywords: Android Colorimetry Digital camera Munsell soil-colour charts
a b s t r a c t Soil colour has been determined in most cases by using Munsell soil-colour charts, sometimes with spectrometers, and occasionally with digital cameras. The objective here is to assess whether a mobile phone, which has all the requirements to capture and process digital images, might also be able to provide an objective evaluation of soil colour under controlled illumination. For this, we have developed an Android application that takes a picture of a soil sample, allowing the user to select the region of interest and then, after a RGB image-processing and a polynomial process transform between colour spaces, the Munsell (HVC) and CIE (XYZ) coordinates appear on the screen of mobile phone. In this way, a commercial HTC smartphone estimated the colour of 60 crumbled soil samples between 2.9YR and 2.3Y with a mean error of 3.75 ± 1.81 CIELAB units, taking as a reference the colour measurements performed with a spectroradiometer. The Munsell hue had the worst estimates (mean error of 2.72 ± 1.61 Munsell units) because of its geometric mismatch with the RGB colour space and for being defined to illuminant C, different of the D65 source under which the phone camera took the pictures. Because the measuring errors were lower than those described in the literature for the visual determination of soil colour, and the application also worked successfully in a different smartphone than the one used in its development, we think that current experimental results encourage the expectations of using smartphones in the field as soil-colour sensors. Ó 2013 Elsevier B.V. All rights reserved.
1. Introduction Specifying colour by the Munsell system is usual for artists, designers, scientists, engineers, and government regulators (ASTM, 2008). For example, in the natural sciences, it has been used to identify and record the colours of specimens such as human skin, flowers, foliage, minerals, and soils. Specifically in soil science, Munsell colour has far-reaching implications for the examination, description, and classification of soils (Soil Survey Staff, 1999; IUSS Working Group WRB, 2006), as well as in studying soil genesis and evaluation, being a valuable indicator of soil structure and components (Sánchez-Marañón et al., 2004). Although the specialized work in soil colour has been based on instrumental measurements (Torrent and Barrón, 1993; Viscarra Rossel and Webster, 2011), the prevalent practice in soil science is visual determination. This seeks ⇑ Corresponding author. Address: Departamento de Óptica, Universidad de Granada, Cuesta del Hospicio s/n, 18071 Granada, Spain. Tel.: +34 958241903. E-mail address: [email protected] (L. Gómez-Robledo). 1 Both authors contributed equally to this work. 0168-1699/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.compag.2013.10.002
the closest match between the soil sample and one of the standard chips contained in the Munsell soil-colour charts, i.e. artificially coloured papers mounted on constant hue charts, showing value (lightness) and chroma (colour intensity) variations in the vertical and horizontal directions, respectively. Thus, the Munsell designation of that chip (hue H, value V, and chroma C) is assigned to the soil sample under study. Several accuracy problems, however, have been reported previously in relation to identifying the colour of soil specimens using Munsell charts (Sánchez-Marañón et al., 1995, 2005, 2011), all of them related to the three main factors affecting the psychophysical character of colour: illumination conditions, sample characteristics, and the observer’s sensitivities (Berns, 2000). Currently, the increasing demand of soil data for applications such as precision agriculture and dynamic models for monitoring environmental change has spurred the development of proximal soil sensors (Viscarra Rossel et al., 2011). The rationale is to collect larger amounts of data using simpler, cheaper, faster, and less laborious techniques than conventional soil analyses, even at the cost of less accuracy. The point is that many more measurements will
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
counteract their lower accuracy. Since soil colour is related to soil components, and, by extension, properties or conditions depending on them (Soil Survey Staff, 1993), proximal soil-colour sensing may provide an integral way of comparing soils, including evolution, degradation, pedoclimate, and fertility analyses. Digital cameras have already been proposed as soil-colour sensors (Aydemir et al., 2004; Viscarra Rossel et al., 2008; O’Donnell et al., 2011) due to the possibility of gaining reliable colour information from RGB digital images. Currently, the only colour measure by pixels is based on RGB signals, and several authors (Meyer et al., 2004; Viscarra Rossel et al., 2006; León et al., 2006; Rodriguez-Pulido et al., 2012) have reported computational solutions that allow digital images to be transformed into standard colour spaces from each pixel of the digital RGB image. The requirements for soil-colour measurement are available or can be implemented in the current typical mobile phone. This includes a high-resolution digital camera, with a low-power highperformance processor at running frequencies of up to 1 GHz, sophisticated operating systems offering multi-tasking, Java support, and options for installing and running externally developed applications. Initially, mobile phones with built-in cameras were used as imaging devices to collect and transmit digital data to an off-site laboratory or external computer, which processed the information and returned the analysis results to the phone. Thus, mobile phones have been used, for example, for biological and forensic applications (Cadle et al., 2010), telemedicine (Martinez et al., 2008), and for bad-smell monitoring of the living environment (Nakamoto et al., 2009). However, only recently, has a mobile phone been used for on-site data processing to determine specific analyte concentrations from single-use chemical reactive membranes considering hue changes from blue to magenta (García et al., 2011). The potential of the mobile phone suggests that it may answer the increasing demand of objective soil-colour data. As this electronic device becomes available for everyone, in contrast to the more complex and costly colorimeters and spectrometers, its use as a proximal sensor in the field becomes more feasible. However, never before has a mobile phone been colorimetrically tested to discriminate among the colour gamut encompassed by the Munsell soil charts. Thus, against a dichotomous choice of colours (García et al., 2011), the mobile should now distinguish between different reddish, brownish, and yellowish hues, from dark to light and of variable intensity. On the other hand, the few works that have measured soil colour with digital cameras (e.g. Viscarra Rossel et al., 2008) needed external software for calculations, when ad hoc solutions performing the immediate computational conversion on the same platform that produced the RGB images would be desirable. Moreover, as outlined in the literatures (Hong et al., 2001; Westland and Ripamonti, 2004), the RGB colour space is device-dependent, and we cannot be sure of the effectiveness of standard transformation equations (Wyszecki and Stiles, 2000) to establish colorimetric coordinates from the RGB information. Finally, we also suspect the possible influence of natural daylight conditions (Sánchez-Marañón et al., 2011), which deserves a separate study. Accordingly, before going to the field, laboratory experience is necessary. In the present work, our overall goal was to investigate in the laboratory the potential of a mobile phone (smartphone) to capture soil-colour images and process them with a colorimetric rationale, returning the Munsell notations corresponding to the digital RGB captured images. To do so, our specific objectives were: (i) to develop a custom image-processing application for mobile phone; (ii) to build a model and estimate its parameters for establishing Munsell notations from RGB measurements; (iii) to implement the image processing and the conversion model RGB ? Munsell, making them work together as software in the mobile phone;
201
and (iv) to assess the accuracy achieved by the mobile phone with respect to that of commercial spectrometers. 2. Materials and methods 2.1. Developing custom image processing for a mobile phone Although there is a wide range of operating systems for developing applications in mobile phones such as Android, Symbian, BlackBerry or Windows Mobile, the former (Android 2.2, Google, EEUU) was selected here for several reasons. Firstly, Android is well established in most communication devices, including mobile phones, netbooks, tablets, and even in electrical appliances such as microwaves and washing machines (Puder and Antebi, 2013). Secondly, Android uses an open code and a free license, allowing a wide community of developers to expand and improve its functionality. Thirdly, Java is the language used by Android, enabling the use of libraries and other applications previously developed for this language. Our software was designed to read colour information from digital images in JPEG format captured with the built-in camera of a smartphone and to carry out processes of colorimetric analysis. The smartphone HTC Desire HD (HTC Corporation, Taiwan), 4.8400 2.6800 0.4600 in size and 164 g in weight, has the necessary computing power to run the program, also including a 4.300 touchsensitive screen with a resolution of 800 480 pixels and an integrated camera up to 8 megapixels with CMOS autofocus sensor. For the device setting, we fixed the ISO parameter (sensitivity of the image sensor to light) to 100, according to the daylight illumination conditions (6500 K), the white balance in the daylight option for high luminance (1580 lx), and the flash in the off position. The smartphone saved the picture in JPEG format with a size of 1952 3264 pixels, a weight of around 1 MB, a resolution of 72 dots per inch, and using 8-bits per channel (24 bits in total for the red, green, and blue channels). Once our application is installed in the mobile phone the user interface on the screen may serve to show all the functionalities (Fig. 1). At program startup, a main menu shows four options (Fig. 1a). To normalize any colour to a white, in accordance with colorimetry (CIE, 2004), a calibration option was included to determine the RGB values from the photo of a reference white (Konica Minolta PTFE, Fig. 1b and c). A square frame over the picture could be moved, expanded, or contracted, in order to select the region of interest to process. The colour information of each pixel inside the cropped area was directly read by the phone, our program giving the statistical mode for R, G, and B. For an analysis of a heterogeneous colour image, the mode has proved more suitable than the mean value (García et al., 2011), removing the undesirable effect of noisy pixels. The same RGB information could be extracted from each picture of colour samples (Fig. 1d and e), which, after being normalized with the reference-white coordinates (Fig. 1c), can be shown and saved in the phone (Fig. 1f). This normalization was made for each pixel following Eq. (1), where n (8 in our case) represented the number of bits per pixel for each colour channel (Salmeron et al., 2012).
RGBnormalized ¼ 2n RGBacquired =RGBwhite
ð1Þ
2.2. Transformation equations from RGB to XYZ and HVC As shown in Fig. 1f, we wanted the mobile-phone application to provide not only RGBnormalized (henceforth RGB) values, but also the colour designation in the CIE (XYZ) and Munsell (HVC) systems. While the Munsell parameters are widely used in soil science, the tristimulus values XYZ are the basis of instrumental colorime-
202
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
Fig. 1. Screen of the HTC smartphone running the Android app for measuring soil colour: (a) main menu; (b) calibration with a reference white; (c) RGB coordinates of the reference white; (d) selection of a region of interest for one chip of the Munsel soil colour charts using a square frame; (e) a picture with a soil sample; and (f) colour coordinates of a soil sample.
try. The two possible solutions to find XYZ and HVC from the original RGB were either standard transformation equations as those collected by Viscarra Rossel et al. (2006) or empirical equations established from our colour measurements. The standard equations were processed by using the ColoSol software (Viscarra Rossel, 2006) while, for developing empirical equations, we used the colour measurements of 238 chips from a recent edition of Munsell soil-colour charts (Munsell Color Com-
pany, 2000) which were completely new. The measurements of each chip were undertaken with the HTC smartphone running our image-processing application for determining its RGB variables, as well as with a spectroradiometer Konica Minolta CS2000 (Tokyo, Japan) for registering the reflected spectral-power distribution between 380 and 780 nm at 2 nm steps. For calculating the XYZ tristimulus values, we assumed the CIE 1964 Standard Observer (CIE, 2004). In a dark room, the chips of each Munsell
Fig. 2. Measuring process: (a) geometry, and (b) a photograph under the experimental conditions with a Munsell chart and the reference white.
203
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
chart together with the reference white were placed inside a GretagMacbeth Spectralight III lighting booth (X-Rite, Switzerland) equipped with a D65 simulator (Fig. 2), in agreement with the standard ASTM (2008) for the Munsell colour determination. We fixed the position of smartphone and spectroradiometer and moved the Munsell chart to focus the image on the centre of each chip. Three replicates of each measure were taken. Using the measured variables RGB and XYZ, in addition to the HVC variables taken from the notation of each chip in the Munsell charts, we computed polynomial transformations between colour spaces (Johnson, 1996) by the pseudo inverse method (Penrose, 1955), in order to build statistical prediction models of XYZ and HVC from RGB. In the matrix relation i = Td, where i (from independent device colour) is XYZ or HVC (3 1 matrix), d (from dependent device colour) is an RGB n 1 matrix, and we calculated the T matrix of dimension 3 n (n = number of polynomial coefficients) which provided the nearest relationship between vectors i and d. These transformations were made by using Matlab2009b (Mathworks, EEUU) and following the steps suggested by Westland and Ripamonti (2004). The partition of Munsell hue for the calculations was 10 for 10R, 12.5 for 2.5YR, and so forth up to 22.5 for 2.5Y, and 25 for 5Y. In addition to statistical coefficients, the accuracy of the models was examined by the differences between measured and predicted values using the CIELAB colour-difference formula DEab (CIE, 2004) and the Munsell colour-difference formula DEM (Godlove, 1951), depending on whether RGB was transformed to the CIE XYZ or Munsell colour systems, respectively. 2.3. Validation of the mobile-phone application To assess the accuracy of mobile-phone application for colour sensing in different samples from those used to develop it, we employed new soil-colour objects and an increasingly stringent validation plan. First, we selected colour samples from the Natural Colour System (NCS, Sweden) Atlas covering the complete colour gamut encompassed in the Munsell soil-colour charts. Although the characteristics of these colour samples were similar to those of the chips of Munsell soil-colour charts, i.e. pieces of artificially coloured plain paper, they were made with different pigments and the resulting colours did not match those of the Munsell chips but rather matched intermediate colours. The selection process among available 1950 NCS colours was initially visual, followed
by measuring each of the visually selected ones (100 samples) with a Konica Minolta 2600d spectrophotometer (Tokyo, Japan) to quantify the Munsell colour. This instrument has an illuminating/ viewing geometry d/8 and two Xenon lamps as the light source, making measurements of the light reflected by the specimen surface with the specular component excluded between 360 and 740 nm at 10 nm intervals. Among the colour indices in the output of display, this instrument provided the Munsell colour parameters HVC. The final selection consisted of 40 NCS samples which Munsell colour parameters ranged between 0.1YR and 5.5Y in hue, between 2.8/ and 8.2/ in value, and between /0.5 and /8 in chroma, all these colours being approximately homogenously distributed over the colour space corresponding to the seven Munsell soil-colour charts. Subsequently, we tested the application in soil samples. Like the NCS samples, the colours of our soil samples were within the space of Munsell soil-colour charts, but now we managed textured and heterogeneous colour samples, which implied a more challenging test for the mobile phone. To cover the widest possible soil-colour range of our pedogenic environment, we inspected the soil samples gathered in our laboratories. Finally, 45 samples were chosen, all from Mediterranean soils with different degrees of development such as Entisols, Inceptisols, Vertisols, and Alfisols. These samples had also been used previously in colorimetric studies to evaluate the effects of illumination, sample state, and observer on the soil-colour determination by Munsell charts (Sánchez-Marañón et al., 1995, 2005, 2011). As usual in laboratory colour studies (Torrent and Barrón, 1993), we prepared air-dried samples of fine earth (soil particles < 2 mm) for the 45 study cases plus 15 samples taken from the same group but grinding and homogenizing them in an agate mortar until we obtained a powder with particle sizes of less than 50 lm. After the powder was placed in circular plastic containers (15 mm in diameter and 5 mm thick) with the upper surface open and levelled, their spectrophotometric colour was measured using the Konica Minolta 2600d spectrophotometer, the results ranging from 2.9YR to 2.3Y in hue, 3.8 to 7.0 in value, and 1.9 to 5.6 in chroma. All colour samples for the validation process were also measured with the HTC smartphone and spectroradiometer under the same experimental conditions described in Fig. 2. In addition, the colour of soil samples was also tested with another mobile phone, a Samsung Galaxy S2 (Samsung, South Korea) smartphone.
Table 1 Polynomial models for transforming RGB to XYZ and HVC. The r, DEab , and DEM are, respectively, Pearson’s correlation coefficient, mean CIELAB and mean Godlove colour difference between measured and predicted values in the 238 chips of Munsell soil-colour charts. Terms of the polynomial
r for XYZ models
r for HVC models
DEab
DEM
[1,R,G,B] [1,R,G,B,RGB] [1,R,G,B,RG,RB,GB] [1,R,G,B,RG,RB,GB,RGB] [1,R,G,B,RG,RB,GB,R2,G2,B2] [1,R,G,B,RG,RB,GB,R2,G2,B2,RGB] [1,R,G,B,RG,RB,GB,R2,G2,B2,RGB,R3,G3,B3] [1,R,G,B,RG,RB,GB,R2,G2,B2,RGB,R2G,G2B,B2R,R2B,G2R,B2G,R3, G3,B3,R2GB,RG2B,RGB2] [1,G,B,RG,R2,G2,RGB,B3]
0.9811 0.9943 0.9965 0.9965 0.9973 0.9973 0.9974 0.9980 0.9972
0.9090 0.9101 0.9246 0.9259 0.9547 0.9562 0.9568 0.9695 0.9407
9.65 6.03 3.44 3.39 2.14 2.07 1.85 1.75 2.03
2.03 1.53 1.12 1.12 1.03 1.02 1.02 0.97 1.08
Table 2 2 3 Coefficients of the polynomial statistical models /ðRGBÞ ¼ a þ bG þ cB þ dRG þ eR2 þ fG þ gRGB þ hB built to compute XYZ and HVC from RGB. a
b
c
d
e
X Y Z
1.3502 0.7246 2.1291
0.0759 0.1146 0.1028
0.0082 0.0172 0.0879
1.05E04 3.20E04 4.80E04
0.000381 3.06E04 1.04E04
f 0.000124 0.000604 0.000174
g
H V C
10.5649 2.3252 1.8311
0.4622 0.0303 0.0157
0.2192 0.0069 0.0351
2.54E03 5.36E05 3.80E04
1.92E04 4.33E05 2.89E04
6.09E04 1.61E05 0.000163
h 2.79E06 3.08E06 2.41 E06
7.98E06 2.40E07 3.00E07
1.31E06 1.54E06 2.13E06 4.64E06 2.96E08 2.25E07
204
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
100
35 Polynomial Colosol
H - Computed
X- Computed
80
Polynomial Colosol
30
60 40 20
25 20 15 10 5
0
0 0
20
40
60
80
0
100
5
10
15
20
25
30
35
H - Munsell soil charts
X- Spectroradiometer 100 Polynomial Colosol
Polynomial Colosol
8
V - Computed
Y - Computed
80 60 40 20 0
6
4
2
0 0
20
40
60
80
100
0
2
Y- Spectroradiometer 100
6
8
14 Polynomial Colosol
Polynomial Colosol
12
C - Computed
80
Z - Computed
4
V- Munsell soil charts
60 40 20
10 8 6 4 2
0
0 0
20
40
60
80
100
Z- Spectroradiometer
0
2
4
6
8
10
12
14
C - Munsell soil charts
Fig. 3. Colour coordinates (XYZ and HVC) measured vs. computed from the RGB signals of HTC smartphone using empirical polynomial transformations and ColoSol software (n = 238 chips of the Munsell soil-colour charts).
250
RGB mobile
200 150 100
R G B
50 0 0
50
100
150
200
250
RGB from XYZ Fig. 4. RGB coordinates calculated with ColoSol from XYZ-spectroradiometric measurements vs. RGB coordinates measured with the HTC smartphone.
This phone has a dual core processor and runs the version Android 4.1.2. The built-in camera has also 8 megapixels and it was used setting equals parameters that described before for the HTC smartphone except the resolution, which in this case was of 3264 2448 pixels, increasing the weight of each picture to 2 MB. In this way, the same Android application was installed and used by a device other than the one used in the design, in order to compute again the soil colours under laboratory conditions. All colour measurements were replicated three times to assess their reproducibility. 3. Results and discussion 3.1. Transformation equations from RGB to XYZ and HVC Once the image processing worked as expected in the design (Fig. 1), the following major hurdle was to transform the RGB signals of the mobile phone to CIE XYZ and Munsell HVC colour
205
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
Table 3 Some statistics of the relationships between colour coordinates measured with the spectroradiometer (XYZ) or annotated in the Munsell soil-colour charts (HVC) and computed from the RGB values registered by the HTC smartphone, using the empirical polynomial transformation and ColoSol conversion program (n = 238 chips of the Munsell soil-colour charts, r = Pearson’s correlation coefficient, GFC = goodness of fit coefficient, RMSE = root mean square error, STRESS = standardized residual sum of squares). The mean and standard deviation of CIELAB (DEab) and Munsell (DEM) colour differences are also given. Polynomial
ColoSol
X r GFC RMSE STRESS Mean DEab SD DEab
Y
0.9971 0.9992 1.4 4.0
r GFC RMSE STRESS Mean DH/DV/DC SD DH/DV/DC Mean DEM SD DEM
Z
0.9972 0.9992 1.3 4.1 2.0 1.1
0.9973 0.9990 1.2 4.5
V
C
H
0.8313 0.9882 2.75 15.31 2.1 1.7
0.9977 0.9998 0.12 2.07 0.1 0.1 1.1 0.5
0.9708 0.9919 0.52 12.7 0.4 0.3
0.8795 0.9776 5.4 21.0 4.7 2.6
80
%
60
40
20
Soil samples Munsell NCS
0 1
2
3
0.9942 0.9964 2.9 8.5
H
100
0
X
4
5
6
7
8
9
10
ΔE* ab
coordinates. In addition to the simplest solution, i.e. standard colour-space transformations available in the colour literature (e.g. Viscarra Rossel et al., 2006), we also tested statistical transformation models developed from the data measured in the laboratory using the Munsell soil-colour charts. Table 1 shows some of these statistical models, from a model of order one with four terms to a polynomial complex dependence of order three and 23 terms, although we found that cubic models with eight coefficients (last row in Table 1) optimised the computational work to be made by the smartphone providing a high correlation and relatively low colour differences between measured and predicted values. Table 2 lists the coefficients of the two cubic polynomial transformation equations forming matrices of dimension 3 8. For the calculations, we used the mean of three replicated measures on each sample, although its dispersion (standard deviation) was negligible both for RBG (<0.01) and XYZ (<0.03) because of the stability of the light source and homogeneity of the measured surface. The polynomial transformations proved slightly better than the standard transformations using the ColoSol software (Fig. 3). This latter underestimated the colour coordinates, except Munsell C, which was overestimated, while Z and H were the worst estimated parameters. This may be because the standard transformation
Y
Z
0.9945 0.9948 4,3 10.2 14.0 4.5
0.9904 0.9892 5.5 14.6
V
C 0.9965 0.9964 0.6 8.5 0.5 0.3 3.2 1.1
0.9764 0.9924 2.0 12.3 1.8 0.9
matrices were developed for different conditions (Wyszecki and Stiles, 2000) from those used here, the RGB values of a same colour might change because of their device dependence (Hong et al., 2001; Westland and Ripamonti, 2004) or the influence of light intensity (Viscarra Rossel et al., 2008), and the standard linear transformations could not exactly match that of real cameras. Certainly, the RGB measured from the mobile phone and the RGB linearly transformed with ColoSol from XYZ of the Munsell chips showed a similar relation (Fig. 4) to the typical curve for image devices, indicating a non-linear response of the sensor (Westland and Ripamonti, 2004). Whatever the reason, as in other studies dealing with the transformation between colour spaces (Johnson, 1996), the mean colour differences between measured and estimated values were lower using the polynomial transformations (Table 2), as concluded from results shown in Table 3. The prediction from the mobile phone of tristimulus XYZ proved more accurate than did the Munsell parameters HVC with either transformation, according to the statistical coefficients listed in Table 3. These coefficients are the Pearson’s linear correlation coefficient, the goodness of fit (GFC) coefficient (Romero et al., 1997), the root mean square (RMSE) error, and the standardized residual sum of squares (STRESS) index (García et al., 2007), which values for perfect fit are 1, 1, 0 and 0, respectively. The estimates of H were the worst. This indicates that using a spectroradiometer as a reference made it possible to find better transformation equations, presumably because the mobile-phone camera and spectroradiometer worked under the same illumination conditions (D65), including light quality and intensity, whereas Munsell colour space was defined with illuminant C. On the other hand, the RGB and XYZ colour spaces have similar geometries, whereas the cylindrical structure of Munsell system, where H is the angular coordinate, could hamper the transformation functions. It is known that when the Munsell radial coordinate C is low the uncertainty in the angular coordinate H is higher, because H becomes indeterminate at null chroma C. In our case, for example, of the 35 chips (16 in the chart 5Y and 12 in the chart 10R) estimated with DH > 4 units, 72% had C 6 2, 20% were extremely light and chromatic (7/8, 8/8), and 8% had C = 3. Accordingly, our H model performed worst for extreme colours located in the bottom left corner and upper right corner of some Munsell charts. With the polynomial transformation, the average of the colour differences between measured and predicted values in the 238 chips of Munsell charts was 2.0 ± 1.1 CIELAB units. It is noteworthy that more than 90% of the samples were measured by the mobile
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
The reliability of the mobile phone to detect colour was initially checked in the NCS samples covering the colour gamut of the Munsell soil-colour charts. Due partly to the even surface and homogenous colour of the samples, as well as the stability of illumination, the replicated measurements in each sample had negligible dispersion (standard deviation <0.06 in RGB with the mobile phone and <0.98 in XYZ with the spectroradiometer and <0.21 in HVC with the spectrophotometer). The differences in each Munsell colour coordinate (absolute values) between the determinations made with the app in the HTC mobile phone and the ones provided by our spectrophotometer are shown in Fig. 6 and the averages are listed in Table 4. The hue differences (DH) ranged between 4 and 7, with 70% of the samples within the [2, 2] interval. A closest agreement was found in Munsell value and chroma, with a range of DV and DC differences [2, 2] and [0, 2] units, respectively, most of the colour samples being within the interval [0, 1]. Despite the low mean value in DH (Table 4), its standard deviation was on the same order, indicating that some colour samples had a major error. According to the CIELAB differences found between the mobile phone estimates and spectroradiometric measurements (CIE tristimulus values XYZ), the NCS samples with the worst results were, like the above-mentioned major error in Munsell chips, samples yellower than 2.5Y (H > 22.5) and with low chroma or very light and chromatic (samples NCS-25, NCS-22, NCS-9, and NCS-1 in Table 5). Although the mean difference was 6.45 CIELAB units, it reached up to 9.4 CIELAB units in some extreme samples (Table 5), which produced the high standard deviation value of 2.05 CIELAB units listed in Table 4. Around 50% of the NCS samples were measured using the HTC mobile phone with an error of less than 6 CIELAB units (Fig. 5). The measuring errors of the mobile phone dramatically decreased in soil samples with respect to those in NCS samples when the spectroradiometer was considered as the reference (Fig. 5), and consequently their mean DEab was lower in Table 4, but increased somewhat when the spectrophotometer was considered (Fig. 6), primarily DH. On the one hand, this different behaviour could be attributed both to the illumination conditions, identical only for the phone camera and spectroradiometer, and the problems once again with the Munsell space geometry. On the other hand, the best DEab results in fine-earth soil samples, despite their uneven and textured surface with heterogeneous soil-pigment mixtures, might be explained by the fact that, although covering a wide gamut from 2.9YR to 2.3Y, these Mediterranean soil samples did not have extreme colours in hue, lightness, or chromaticity, which must have improved the performance of the mobile phone. Probably the relatively worse DEab results for the ground and homogenized soil samples than for fine-earth soil samples (Table 4) was also due to a somewhat more limiting colour gamut in the 15 soils from which fine earth as well as ground samples were prepared. For these 15 soils (on the average, 1.0Y and 3.0 in chroma), however, there was no statistically significant difference (P < 0.05) between the mobile-phone errors in fine earth (mean DEab ¼ 4:46) and ground (mean DEab ¼ 5:04) samples. The results indicate that
100 NCS Soils
80
60
%
3.2. Performance of the mobile-phone application in the validation samples
although the ground samples exhibited a more homogeneous aspect, it did not lessen the measuring errors of mobile phone, probably because of the way in which the colour of all pixels in a picture were managed in order to compute the soil colour as a whole. The mobile phone calculated the mode of the pixels present in an area of interest and with the use of this statistical parameter, anomalous pixels in a picture of fine earth did not influence the final colour coordinates. Table 5, which lists the best and worst study cases with their colour coordinates measured with the spectrophotometer (HVC), spectroradiometer (XYZ), and mobile phone (RGB), confirmed the good results for the soil samples: SOIL-va, SOIL-ck, SOIL-sn, SOIL-
40
20
0 0
1
2
3
4
5
6
7
8
9
10
ΔH 100 NCS
80
Soils
60
%
phone with accuracy of 4 CIELAB units or better (Fig. 5). When it is taken into account that the theoretical colour visual threshold was around 1.0 CIELAB units, that the practical threshold for most colorimetric applications is around 3–4 units, and that only thresholds greater than 5–6 units should be considered erroneous (Melgosa et al., 1992; Huang et al., 2012), the current results suggest that the implementation of polynomial transformations from RGB to XYZ or HVC in the mobile-phone application is the best option to provide accurate colour information.
40
20
0 0
1
2
3
4
5
6
7
8
9
10
9
10
ΔV 100 NCS Soils
80
60
%
206
40
20
0 0
1
2
3
4
5
6
7
8
ΔC Fig. 6. Histograms for NCS (n = 40) and soil (n = 60) samples with the differences (absolute value) in Munsell hue (DH), value (DV), and chroma (DC) between data measured with the spectrophotometer and predicted with the HTC smartphone.
207
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
ll, and SOIL-sh even with errors close to the theoretical colour threshold (1–2 CIELAB units). There was, however, an apparent lack of consistency in the magnitudes of CIELAB DEab and Munsell (DH, DV, DC) differences between measured and predicted values. Thus, for example, SOIL-sh had greater CIELAB difference (1.9 units) than SOIL-va (1.3 units), just the opposite behaviour that Munsell differences (0.3, 0.2, 0.1 against 0.6, 0.1, 0.2, for DH, DV, DC respectively). This fact, common in Table 5, has already been statistically-discussed with the results of Table 4 and Figs. 5 and 6, indicating that there was no a good correlation between spectrophotometric and spectroradiometric measurements. In this way, higher values in DEab did not necessarily correspond to higher errors in HVC and vice versa. Considering the entire dataset of soil samples (n = 60), we found that the discrepancies between the HTC smartphone and spectrophotometer ranged from 6 to 6 for Munsell hue, so that 60% of samples were determined with an error of less than 3.0 units (Fig. 6). For Munsell value and chroma, most soil samples had an error of less than 1.0 Munsell unit. The total colour difference between HTC smartphone and spectroradiometer was on the average 3.7 ± 1.8 CIELAB units, and more than 80% of soil samples
were measured by the mobile phone with an error lower than 5 CIELAB units (Fig. 5). It also bears mentioning that this error by the mobile phone in sensing colour is lower than those previously reported when the same soil samples were visually determined using Munsell soil-colour charts. Specifically, a group of 10 soil scientists judging soil colour under controlled illumination had a mean error of 4.4 CIELAB units in ground soil samples and 10.2 in aggregated soil samples with respect to instrumental measurement, and their inter-observer variability, i.e. disagreement among the ten Munsell notations on a given sample, was 5.1 CIELAB units (Sánchez-Marañón et al., 1995). Differences of up 5.5 CIELAB units were also measured by Sánchez-Marañón et al. (2005) in Munsell soil-colour charts from different editions, manufacturers, and degree of use, so inducing the same error in the soil colour determination. In addition, the error made by the smartphone estimating Munsell hue (59.0% of soils with DH > 2 units, Fig. 6) was lower than the variation in Munsell hue of soils observed under different natural daylight conditions (78.6% of soils with DH > 2 units, Sánchez-Marañón et al., 2011). Finally, once the mobile-phone application developed with the HTC smartphone was installed in another device (Samsung smart-
Table 4 Differences in Munsell hue, value, and chroma (DH, DV, and DC in absolute value) and CIELAB colour differences DEab between data measured with a spectrophotometer (HVC) and a spectroradiometer (XYZ) and estimated with two mobile phones (HTC and Samsung). Group of samples
Colour parameter
Mean value
Standard deviation
NCS samples HTC smartphone (n = 40)
DH DV DC DEab
1.69 0.66 0.36 6.45
1.69 0.22 0.34 2.05
Ground soil samples HTC smartphone (n = 15)
DH DV DC DEab
2.05 0.57 0.32 5.04
1.23 0.31 0.20 2.47
Fine-earth soil samples HTC smartphone (n = 45)
DH DV DC DEab
2.72 0.57 0.81 3.31
1.70 0.36 0.60 1.59
Soil samples (fine-earth plus ground samples) Samsung smartphone (n = 60)
DH DV DC DEab
3.38 0.63 0.25 5.46
1.55 0.26 0.17 2.46
Table 5 The five NCS- and SOIL-samples with the lowest (italics) and highest CIELAB colour differences DEab between the colour measured with a spectroradiometer (XYZ, mean of 3 replicates with SD < 1.38) and predicted from the RGB values (mean of 3 replicates with SD < 0.11) by the HTC smartphone. The Munsell HVC measured with a spectrophotometer (mean of 3 replicates with SD < 0.24) and the differences (DH, DV, DC) with the predicted values by the mobile phone are also listed. Sample
H
V
C
X
Y
Z
R
G
B
DH
DV
DC
DEab
NCS-38 NCS-28 NCS-21 NCS-30 NCS-35 SOIL-va SOIL-ck SOIL-sn SOIL-ll SOIL-sh
16.9 12.9 10.4 13.7 15.1 21.7 21.4 19.9 19.9 19.5
7.7 7.5 3.7 6.5 6.3 4.4 6.0 4.9 4.0 3.8
0.5 1.1 6.6 7.0 8.0 1.9 2.7 3.6 2.6 2.6
55.3 57.2 14.8 48.3 44.7 12.7 30.7 21.9 20.6 11.9
57.6 58.9 11.3 43.1 39.8 13.0 31.6 21.2 20.9 11.2
63.5 62.2 5.5 22.7 16.1 9.5 21.9 12.7 13.3 7.3
222.0 231.0 165.0 255.0 255.0 105.6 166.4 149.4 140.8 105.6
202.0 202.0 41.0 145.0 136.0 75.2 135.6 98.0 98.0 59.1
206.0 198.0 41.0 107.0 74.0 56.3 94.2 56.3 65.5 39.9
4.6 1.9 1.1 2.3 3.6 0.6 3.5 2.3 2.1 0.3
0.4 0.7 0.4 0.9 0.9 0.1 0.0 0.2 1.0 0.2
0.3 0.3 0.1 0.6 1.3 0.2 0.1 0.0 0.3 0.1
2.3 2.6 2.7 3.9 4.2 1.3 1.7 1.8 1.8 1.9
NCS-4 NCS-25 NCS-22 NCS-9 NCS-1 SOIL-vg SOIL-dd SOIL-ps SOIL-mo SOIL-bc
18.2 23.1 23.3 25.5 22.5 22.6 19.8 15.9 22.2 20.5
6.2 3.3 7.6 4.3 7.1 5.1 5.1 4.0 5.4 6.1
5.0 2.7 2.1 2.5 6.3 2.3 3.7 5.0 2.1 3.9
35.2 6.7 41.3 14.7 40.0 23.4 26.9 20.2 23.5 32.8
34.2 6.9 43.6 15.7 40.9 24.3 27.0 18.3 24.6 33.2
18.2 3.8 34.4 10.2 16.1 19.2 15.1 6.8 17.1 19.6
231.0 99.0 222.0 140.0 239.0 157.9 181.0 167.7 178.3 221.3
156.0 65.0 193.0 112.0 179.0 124.8 110.4 65.4 129.1 162.4
99.0 41.0 156.0 74.0 82.0 85.0 74.0 33.0 109.7 103.4
0.3 1.6 1.3 0.4 0.1 1.8 1.6 0.5 4.2 1.2
1.0 0.8 0.2 1.0 0.6 0.6 0.6 0.7 0.5 1.1
0.6 0.3 0.2 0.0 0.0 0.4 0.8 1.3 0.6 0.8
9.2 9.4 9.4 9.4 9.4 7.6 7.8 7.9 9.8 11.1
208
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
phone), we measured the soil samples again under the same experimental conditions (Fig. 2). The results with the Samsung smartphone were only slightly worse than using the HTC (Table 4) but still acceptable. This indicates similarities in the performance of both mobile-phone cameras to capture the RGB signals and facilities in the operating system to interchange the software. 4. Conclusions Our results indicate that the technical resources of the current mobile phones can be exploited to use these electronic devices, which are prevalent worldwide and accessible to everyone, as soil-colour sensors. The RGB signals captured by the camera and converted in colour coordinates by a software application running inside the same smartphone, can thus achieve objective, easy, rapid, and cheap colour measurements. Under controlled illumination conditions, the measurements seem to be also more accurate than visual soil-colour determination using Munsell charts. Essential requirements of the mobile-phone application were a moveable and visible frame on the smartphone screen to select the region of interest in the picture, a calibration or normalization tool for referencing colour to a standard white, and transformation equations to convert RGB values in Munsell or CIE colour coordinates. In converting the image signals to colour, a polynomic-process transform fitted to the colour gamut of the Munsell soil-colour charts worked better than standard equations from the colour literature, and also proved more efficient in reference to XYZ coordinates (very similar in geometry to RBG) measured with a spectroradiometer under the same experimental conditions as the phone camera. Accordingly, although our initial objective was to reproduce the Munsell notation of soil samples, the results suggest that CIE coordinates are more accurately predicted for this application. Programming in Android allowed the application to be successfully interchanged between different devices. However, to achieve an application working similarly in different models of mobile phones constitutes a challenge, because it depends mainly on the quality of mobile-phone camera. The use of the mobile phone under non-controlled (variable) illumination conditions is another step forward for our future research. Acknowledgements This work has been partially funded by Ministry of Economy and Competitivity (Spain) under FIS2010-19839 Research Project, and Junta de Andalucía (Spain) under Project PE10-TIC5997, with European Regional Development Fund (ERDF) support. References ASTM, 2008. Standard Practice for Specifying Color by the Munsell System. ASTM International D 1535-08, PA, USA. Aydemir, S., Keskin, S., Drees, L.R., 2004. Quantification of soil features using digital image processing (DIP) techniques. Geoderma 119, 1–8. Berns, R., 2000. Billmeyer and Saltzman’s Principles of Color Technology. Wiley Interscience. Cadle, B.A., Rasmus, K.C., Varela, J.A., Leverich, L.S., O’Neill, C.E., Bachtell, R.K., Cooper, D.C., 2010. Cellular phone-based image acquisition and quantitative ratiometric method for detecting cocaine and benzoylecgonine for biological and forensic applications. Subst. Abuse 4, 21–33. CIE, 2004. Publication 15:2004. In: Bureau, C.C. (Ed.), Colorimetry, 3rd ed., Vienna. García, P.A., Huertas, r., Melgosa, M., Cui, G., 2007. Measurement of the relationship between perceived and computed color difference. J. Opt. Soc. Am. A 24, 1823– 1829.
García, A., Erenas, M.M., Marinetto, E.D., Abad, C.A., de Orbe-Paya, I., Palma, A.J., Capitán-Vallvey, L.F., 2011. Mobile phone platform as portable chemical analyzer. Sensor. Actuat. B – Chem. 156, 350–359. Godlove, I.H., 1951. Improved color-difference formula, with applications to the perceptibility and acceptability of fadings. J. Opt. Soc. Am. 41, 760–772. Hong, G., Luo, M.R., Rhodes, P.A., 2001. A study of digital camera colorimetric characterization based on polynomial modeling. Color Res. Appl. 26, 76–84. Huang, M., Liu, H., Cui, G., Luo, M.R., Melgosa, M., 2012. Evaluation of threshold color differences using printed samples. J. Opt. Soc. Am. A 29, 883–891. IUSS Working Group WRB, 2006. World reference base for soil resources 2006. World Soil Resources Reports 103. Johnson, T., 1996. Methods for characterizing colour scanners and digital cameras. Displays 16, 183–191. León, K., Mery, D., Pedreschi, F., León, J., 2006. Color measurement in L⁄a⁄b⁄ units from RGB digital images. Food Res. Int. 39, 1084–1091. Martinez, A.W., Phillips, S.T., Carrilho, E., Thomas 3rd., S.W., Sindi, H., Whitesides, G.M., 2008. Simple telemedicine for developing regions: camera phones and paper-based microfluidic devices for real-time, off-site diagnosis. Anal. Chem. 80, 3699–3707. Melgosa, M., Hita, E., Romero, J., del Barco, L.J., 1992. Some classical color differences calculated with new formulas. J. Opt. Soc. Am. A 9, 1247–1254. Meyer, G.E., Camargo Neto, J., Jones, D.D., Hindman, T.W., 2004. Intensified fuzzy clusters for classifying plant, soil, and residue regions of interest from color images. Comput. Electron. Agric. 42, 161–180. Munsell Color Company, 2000. Munsell Soil Color Charts. Munsell Color Co., Baltimore, MD. Nakamoto, T., Ikeda, T., Hirano, H., Arimoto, T., 2009. Humidity compensation by neural network for bad-smell sensing system using gas detector tube and builtin camera. 2009 IEEE Sensors, pp. 281–286. NCS – Natural Colour SystemÒÓ, 2013.
Contents lists available at ScienceDirect
Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag
Using the mobile phone as Munsell soil-colour sensor: An experiment under controlled illumination conditions Luis Gómez-Robledo a,⇑,1, Nuria López-Ruiz b,1, Manuel Melgosa a, Alberto J. Palma b, Luis Fermín Capitán-Vallvey c, Manuel Sánchez-Marañón d a
Departamento de Óptica, Facultad de Ciencias, Universidad de Granada, 18071 Granada, Spain ECsens, Departamento de Electrónica y Tecnología de Computadores, ETSIIT, Universidad de Granada, 18071 Granada, Spain Grupo de Investigación Espectrometría en Fase Sólida, Departamento de Química Analítica, Facultad de Ciencia, Universidad de Granada, 18071 Granada, Spain d Departamento de Edafología y Química Agrícola, Facultad de Ciencias, Universidad de Granada, 18071 Granada, Spain b c
a r t i c l e
i n f o
Article history: Received 30 January 2013 Received in revised form 24 July 2013 Accepted 2 October 2013
Keywords: Android Colorimetry Digital camera Munsell soil-colour charts
a b s t r a c t Soil colour has been determined in most cases by using Munsell soil-colour charts, sometimes with spectrometers, and occasionally with digital cameras. The objective here is to assess whether a mobile phone, which has all the requirements to capture and process digital images, might also be able to provide an objective evaluation of soil colour under controlled illumination. For this, we have developed an Android application that takes a picture of a soil sample, allowing the user to select the region of interest and then, after a RGB image-processing and a polynomial process transform between colour spaces, the Munsell (HVC) and CIE (XYZ) coordinates appear on the screen of mobile phone. In this way, a commercial HTC smartphone estimated the colour of 60 crumbled soil samples between 2.9YR and 2.3Y with a mean error of 3.75 ± 1.81 CIELAB units, taking as a reference the colour measurements performed with a spectroradiometer. The Munsell hue had the worst estimates (mean error of 2.72 ± 1.61 Munsell units) because of its geometric mismatch with the RGB colour space and for being defined to illuminant C, different of the D65 source under which the phone camera took the pictures. Because the measuring errors were lower than those described in the literature for the visual determination of soil colour, and the application also worked successfully in a different smartphone than the one used in its development, we think that current experimental results encourage the expectations of using smartphones in the field as soil-colour sensors. Ó 2013 Elsevier B.V. All rights reserved.
1. Introduction Specifying colour by the Munsell system is usual for artists, designers, scientists, engineers, and government regulators (ASTM, 2008). For example, in the natural sciences, it has been used to identify and record the colours of specimens such as human skin, flowers, foliage, minerals, and soils. Specifically in soil science, Munsell colour has far-reaching implications for the examination, description, and classification of soils (Soil Survey Staff, 1999; IUSS Working Group WRB, 2006), as well as in studying soil genesis and evaluation, being a valuable indicator of soil structure and components (Sánchez-Marañón et al., 2004). Although the specialized work in soil colour has been based on instrumental measurements (Torrent and Barrón, 1993; Viscarra Rossel and Webster, 2011), the prevalent practice in soil science is visual determination. This seeks ⇑ Corresponding author. Address: Departamento de Óptica, Universidad de Granada, Cuesta del Hospicio s/n, 18071 Granada, Spain. Tel.: +34 958241903. E-mail address: [email protected] (L. Gómez-Robledo). 1 Both authors contributed equally to this work. 0168-1699/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.compag.2013.10.002
the closest match between the soil sample and one of the standard chips contained in the Munsell soil-colour charts, i.e. artificially coloured papers mounted on constant hue charts, showing value (lightness) and chroma (colour intensity) variations in the vertical and horizontal directions, respectively. Thus, the Munsell designation of that chip (hue H, value V, and chroma C) is assigned to the soil sample under study. Several accuracy problems, however, have been reported previously in relation to identifying the colour of soil specimens using Munsell charts (Sánchez-Marañón et al., 1995, 2005, 2011), all of them related to the three main factors affecting the psychophysical character of colour: illumination conditions, sample characteristics, and the observer’s sensitivities (Berns, 2000). Currently, the increasing demand of soil data for applications such as precision agriculture and dynamic models for monitoring environmental change has spurred the development of proximal soil sensors (Viscarra Rossel et al., 2011). The rationale is to collect larger amounts of data using simpler, cheaper, faster, and less laborious techniques than conventional soil analyses, even at the cost of less accuracy. The point is that many more measurements will
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
counteract their lower accuracy. Since soil colour is related to soil components, and, by extension, properties or conditions depending on them (Soil Survey Staff, 1993), proximal soil-colour sensing may provide an integral way of comparing soils, including evolution, degradation, pedoclimate, and fertility analyses. Digital cameras have already been proposed as soil-colour sensors (Aydemir et al., 2004; Viscarra Rossel et al., 2008; O’Donnell et al., 2011) due to the possibility of gaining reliable colour information from RGB digital images. Currently, the only colour measure by pixels is based on RGB signals, and several authors (Meyer et al., 2004; Viscarra Rossel et al., 2006; León et al., 2006; Rodriguez-Pulido et al., 2012) have reported computational solutions that allow digital images to be transformed into standard colour spaces from each pixel of the digital RGB image. The requirements for soil-colour measurement are available or can be implemented in the current typical mobile phone. This includes a high-resolution digital camera, with a low-power highperformance processor at running frequencies of up to 1 GHz, sophisticated operating systems offering multi-tasking, Java support, and options for installing and running externally developed applications. Initially, mobile phones with built-in cameras were used as imaging devices to collect and transmit digital data to an off-site laboratory or external computer, which processed the information and returned the analysis results to the phone. Thus, mobile phones have been used, for example, for biological and forensic applications (Cadle et al., 2010), telemedicine (Martinez et al., 2008), and for bad-smell monitoring of the living environment (Nakamoto et al., 2009). However, only recently, has a mobile phone been used for on-site data processing to determine specific analyte concentrations from single-use chemical reactive membranes considering hue changes from blue to magenta (García et al., 2011). The potential of the mobile phone suggests that it may answer the increasing demand of objective soil-colour data. As this electronic device becomes available for everyone, in contrast to the more complex and costly colorimeters and spectrometers, its use as a proximal sensor in the field becomes more feasible. However, never before has a mobile phone been colorimetrically tested to discriminate among the colour gamut encompassed by the Munsell soil charts. Thus, against a dichotomous choice of colours (García et al., 2011), the mobile should now distinguish between different reddish, brownish, and yellowish hues, from dark to light and of variable intensity. On the other hand, the few works that have measured soil colour with digital cameras (e.g. Viscarra Rossel et al., 2008) needed external software for calculations, when ad hoc solutions performing the immediate computational conversion on the same platform that produced the RGB images would be desirable. Moreover, as outlined in the literatures (Hong et al., 2001; Westland and Ripamonti, 2004), the RGB colour space is device-dependent, and we cannot be sure of the effectiveness of standard transformation equations (Wyszecki and Stiles, 2000) to establish colorimetric coordinates from the RGB information. Finally, we also suspect the possible influence of natural daylight conditions (Sánchez-Marañón et al., 2011), which deserves a separate study. Accordingly, before going to the field, laboratory experience is necessary. In the present work, our overall goal was to investigate in the laboratory the potential of a mobile phone (smartphone) to capture soil-colour images and process them with a colorimetric rationale, returning the Munsell notations corresponding to the digital RGB captured images. To do so, our specific objectives were: (i) to develop a custom image-processing application for mobile phone; (ii) to build a model and estimate its parameters for establishing Munsell notations from RGB measurements; (iii) to implement the image processing and the conversion model RGB ? Munsell, making them work together as software in the mobile phone;
201
and (iv) to assess the accuracy achieved by the mobile phone with respect to that of commercial spectrometers. 2. Materials and methods 2.1. Developing custom image processing for a mobile phone Although there is a wide range of operating systems for developing applications in mobile phones such as Android, Symbian, BlackBerry or Windows Mobile, the former (Android 2.2, Google, EEUU) was selected here for several reasons. Firstly, Android is well established in most communication devices, including mobile phones, netbooks, tablets, and even in electrical appliances such as microwaves and washing machines (Puder and Antebi, 2013). Secondly, Android uses an open code and a free license, allowing a wide community of developers to expand and improve its functionality. Thirdly, Java is the language used by Android, enabling the use of libraries and other applications previously developed for this language. Our software was designed to read colour information from digital images in JPEG format captured with the built-in camera of a smartphone and to carry out processes of colorimetric analysis. The smartphone HTC Desire HD (HTC Corporation, Taiwan), 4.8400 2.6800 0.4600 in size and 164 g in weight, has the necessary computing power to run the program, also including a 4.300 touchsensitive screen with a resolution of 800 480 pixels and an integrated camera up to 8 megapixels with CMOS autofocus sensor. For the device setting, we fixed the ISO parameter (sensitivity of the image sensor to light) to 100, according to the daylight illumination conditions (6500 K), the white balance in the daylight option for high luminance (1580 lx), and the flash in the off position. The smartphone saved the picture in JPEG format with a size of 1952 3264 pixels, a weight of around 1 MB, a resolution of 72 dots per inch, and using 8-bits per channel (24 bits in total for the red, green, and blue channels). Once our application is installed in the mobile phone the user interface on the screen may serve to show all the functionalities (Fig. 1). At program startup, a main menu shows four options (Fig. 1a). To normalize any colour to a white, in accordance with colorimetry (CIE, 2004), a calibration option was included to determine the RGB values from the photo of a reference white (Konica Minolta PTFE, Fig. 1b and c). A square frame over the picture could be moved, expanded, or contracted, in order to select the region of interest to process. The colour information of each pixel inside the cropped area was directly read by the phone, our program giving the statistical mode for R, G, and B. For an analysis of a heterogeneous colour image, the mode has proved more suitable than the mean value (García et al., 2011), removing the undesirable effect of noisy pixels. The same RGB information could be extracted from each picture of colour samples (Fig. 1d and e), which, after being normalized with the reference-white coordinates (Fig. 1c), can be shown and saved in the phone (Fig. 1f). This normalization was made for each pixel following Eq. (1), where n (8 in our case) represented the number of bits per pixel for each colour channel (Salmeron et al., 2012).
RGBnormalized ¼ 2n RGBacquired =RGBwhite
ð1Þ
2.2. Transformation equations from RGB to XYZ and HVC As shown in Fig. 1f, we wanted the mobile-phone application to provide not only RGBnormalized (henceforth RGB) values, but also the colour designation in the CIE (XYZ) and Munsell (HVC) systems. While the Munsell parameters are widely used in soil science, the tristimulus values XYZ are the basis of instrumental colorime-
202
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
Fig. 1. Screen of the HTC smartphone running the Android app for measuring soil colour: (a) main menu; (b) calibration with a reference white; (c) RGB coordinates of the reference white; (d) selection of a region of interest for one chip of the Munsel soil colour charts using a square frame; (e) a picture with a soil sample; and (f) colour coordinates of a soil sample.
try. The two possible solutions to find XYZ and HVC from the original RGB were either standard transformation equations as those collected by Viscarra Rossel et al. (2006) or empirical equations established from our colour measurements. The standard equations were processed by using the ColoSol software (Viscarra Rossel, 2006) while, for developing empirical equations, we used the colour measurements of 238 chips from a recent edition of Munsell soil-colour charts (Munsell Color Com-
pany, 2000) which were completely new. The measurements of each chip were undertaken with the HTC smartphone running our image-processing application for determining its RGB variables, as well as with a spectroradiometer Konica Minolta CS2000 (Tokyo, Japan) for registering the reflected spectral-power distribution between 380 and 780 nm at 2 nm steps. For calculating the XYZ tristimulus values, we assumed the CIE 1964 Standard Observer (CIE, 2004). In a dark room, the chips of each Munsell
Fig. 2. Measuring process: (a) geometry, and (b) a photograph under the experimental conditions with a Munsell chart and the reference white.
203
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
chart together with the reference white were placed inside a GretagMacbeth Spectralight III lighting booth (X-Rite, Switzerland) equipped with a D65 simulator (Fig. 2), in agreement with the standard ASTM (2008) for the Munsell colour determination. We fixed the position of smartphone and spectroradiometer and moved the Munsell chart to focus the image on the centre of each chip. Three replicates of each measure were taken. Using the measured variables RGB and XYZ, in addition to the HVC variables taken from the notation of each chip in the Munsell charts, we computed polynomial transformations between colour spaces (Johnson, 1996) by the pseudo inverse method (Penrose, 1955), in order to build statistical prediction models of XYZ and HVC from RGB. In the matrix relation i = Td, where i (from independent device colour) is XYZ or HVC (3 1 matrix), d (from dependent device colour) is an RGB n 1 matrix, and we calculated the T matrix of dimension 3 n (n = number of polynomial coefficients) which provided the nearest relationship between vectors i and d. These transformations were made by using Matlab2009b (Mathworks, EEUU) and following the steps suggested by Westland and Ripamonti (2004). The partition of Munsell hue for the calculations was 10 for 10R, 12.5 for 2.5YR, and so forth up to 22.5 for 2.5Y, and 25 for 5Y. In addition to statistical coefficients, the accuracy of the models was examined by the differences between measured and predicted values using the CIELAB colour-difference formula DEab (CIE, 2004) and the Munsell colour-difference formula DEM (Godlove, 1951), depending on whether RGB was transformed to the CIE XYZ or Munsell colour systems, respectively. 2.3. Validation of the mobile-phone application To assess the accuracy of mobile-phone application for colour sensing in different samples from those used to develop it, we employed new soil-colour objects and an increasingly stringent validation plan. First, we selected colour samples from the Natural Colour System (NCS, Sweden) Atlas covering the complete colour gamut encompassed in the Munsell soil-colour charts. Although the characteristics of these colour samples were similar to those of the chips of Munsell soil-colour charts, i.e. pieces of artificially coloured plain paper, they were made with different pigments and the resulting colours did not match those of the Munsell chips but rather matched intermediate colours. The selection process among available 1950 NCS colours was initially visual, followed
by measuring each of the visually selected ones (100 samples) with a Konica Minolta 2600d spectrophotometer (Tokyo, Japan) to quantify the Munsell colour. This instrument has an illuminating/ viewing geometry d/8 and two Xenon lamps as the light source, making measurements of the light reflected by the specimen surface with the specular component excluded between 360 and 740 nm at 10 nm intervals. Among the colour indices in the output of display, this instrument provided the Munsell colour parameters HVC. The final selection consisted of 40 NCS samples which Munsell colour parameters ranged between 0.1YR and 5.5Y in hue, between 2.8/ and 8.2/ in value, and between /0.5 and /8 in chroma, all these colours being approximately homogenously distributed over the colour space corresponding to the seven Munsell soil-colour charts. Subsequently, we tested the application in soil samples. Like the NCS samples, the colours of our soil samples were within the space of Munsell soil-colour charts, but now we managed textured and heterogeneous colour samples, which implied a more challenging test for the mobile phone. To cover the widest possible soil-colour range of our pedogenic environment, we inspected the soil samples gathered in our laboratories. Finally, 45 samples were chosen, all from Mediterranean soils with different degrees of development such as Entisols, Inceptisols, Vertisols, and Alfisols. These samples had also been used previously in colorimetric studies to evaluate the effects of illumination, sample state, and observer on the soil-colour determination by Munsell charts (Sánchez-Marañón et al., 1995, 2005, 2011). As usual in laboratory colour studies (Torrent and Barrón, 1993), we prepared air-dried samples of fine earth (soil particles < 2 mm) for the 45 study cases plus 15 samples taken from the same group but grinding and homogenizing them in an agate mortar until we obtained a powder with particle sizes of less than 50 lm. After the powder was placed in circular plastic containers (15 mm in diameter and 5 mm thick) with the upper surface open and levelled, their spectrophotometric colour was measured using the Konica Minolta 2600d spectrophotometer, the results ranging from 2.9YR to 2.3Y in hue, 3.8 to 7.0 in value, and 1.9 to 5.6 in chroma. All colour samples for the validation process were also measured with the HTC smartphone and spectroradiometer under the same experimental conditions described in Fig. 2. In addition, the colour of soil samples was also tested with another mobile phone, a Samsung Galaxy S2 (Samsung, South Korea) smartphone.
Table 1 Polynomial models for transforming RGB to XYZ and HVC. The r, DEab , and DEM are, respectively, Pearson’s correlation coefficient, mean CIELAB and mean Godlove colour difference between measured and predicted values in the 238 chips of Munsell soil-colour charts. Terms of the polynomial
r for XYZ models
r for HVC models
DEab
DEM
[1,R,G,B] [1,R,G,B,RGB] [1,R,G,B,RG,RB,GB] [1,R,G,B,RG,RB,GB,RGB] [1,R,G,B,RG,RB,GB,R2,G2,B2] [1,R,G,B,RG,RB,GB,R2,G2,B2,RGB] [1,R,G,B,RG,RB,GB,R2,G2,B2,RGB,R3,G3,B3] [1,R,G,B,RG,RB,GB,R2,G2,B2,RGB,R2G,G2B,B2R,R2B,G2R,B2G,R3, G3,B3,R2GB,RG2B,RGB2] [1,G,B,RG,R2,G2,RGB,B3]
0.9811 0.9943 0.9965 0.9965 0.9973 0.9973 0.9974 0.9980 0.9972
0.9090 0.9101 0.9246 0.9259 0.9547 0.9562 0.9568 0.9695 0.9407
9.65 6.03 3.44 3.39 2.14 2.07 1.85 1.75 2.03
2.03 1.53 1.12 1.12 1.03 1.02 1.02 0.97 1.08
Table 2 2 3 Coefficients of the polynomial statistical models /ðRGBÞ ¼ a þ bG þ cB þ dRG þ eR2 þ fG þ gRGB þ hB built to compute XYZ and HVC from RGB. a
b
c
d
e
X Y Z
1.3502 0.7246 2.1291
0.0759 0.1146 0.1028
0.0082 0.0172 0.0879
1.05E04 3.20E04 4.80E04
0.000381 3.06E04 1.04E04
f 0.000124 0.000604 0.000174
g
H V C
10.5649 2.3252 1.8311
0.4622 0.0303 0.0157
0.2192 0.0069 0.0351
2.54E03 5.36E05 3.80E04
1.92E04 4.33E05 2.89E04
6.09E04 1.61E05 0.000163
h 2.79E06 3.08E06 2.41 E06
7.98E06 2.40E07 3.00E07
1.31E06 1.54E06 2.13E06 4.64E06 2.96E08 2.25E07
204
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
100
35 Polynomial Colosol
H - Computed
X- Computed
80
Polynomial Colosol
30
60 40 20
25 20 15 10 5
0
0 0
20
40
60
80
0
100
5
10
15
20
25
30
35
H - Munsell soil charts
X- Spectroradiometer 100 Polynomial Colosol
Polynomial Colosol
8
V - Computed
Y - Computed
80 60 40 20 0
6
4
2
0 0
20
40
60
80
100
0
2
Y- Spectroradiometer 100
6
8
14 Polynomial Colosol
Polynomial Colosol
12
C - Computed
80
Z - Computed
4
V- Munsell soil charts
60 40 20
10 8 6 4 2
0
0 0
20
40
60
80
100
Z- Spectroradiometer
0
2
4
6
8
10
12
14
C - Munsell soil charts
Fig. 3. Colour coordinates (XYZ and HVC) measured vs. computed from the RGB signals of HTC smartphone using empirical polynomial transformations and ColoSol software (n = 238 chips of the Munsell soil-colour charts).
250
RGB mobile
200 150 100
R G B
50 0 0
50
100
150
200
250
RGB from XYZ Fig. 4. RGB coordinates calculated with ColoSol from XYZ-spectroradiometric measurements vs. RGB coordinates measured with the HTC smartphone.
This phone has a dual core processor and runs the version Android 4.1.2. The built-in camera has also 8 megapixels and it was used setting equals parameters that described before for the HTC smartphone except the resolution, which in this case was of 3264 2448 pixels, increasing the weight of each picture to 2 MB. In this way, the same Android application was installed and used by a device other than the one used in the design, in order to compute again the soil colours under laboratory conditions. All colour measurements were replicated three times to assess their reproducibility. 3. Results and discussion 3.1. Transformation equations from RGB to XYZ and HVC Once the image processing worked as expected in the design (Fig. 1), the following major hurdle was to transform the RGB signals of the mobile phone to CIE XYZ and Munsell HVC colour
205
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
Table 3 Some statistics of the relationships between colour coordinates measured with the spectroradiometer (XYZ) or annotated in the Munsell soil-colour charts (HVC) and computed from the RGB values registered by the HTC smartphone, using the empirical polynomial transformation and ColoSol conversion program (n = 238 chips of the Munsell soil-colour charts, r = Pearson’s correlation coefficient, GFC = goodness of fit coefficient, RMSE = root mean square error, STRESS = standardized residual sum of squares). The mean and standard deviation of CIELAB (DEab) and Munsell (DEM) colour differences are also given. Polynomial
ColoSol
X r GFC RMSE STRESS Mean DEab SD DEab
Y
0.9971 0.9992 1.4 4.0
r GFC RMSE STRESS Mean DH/DV/DC SD DH/DV/DC Mean DEM SD DEM
Z
0.9972 0.9992 1.3 4.1 2.0 1.1
0.9973 0.9990 1.2 4.5
V
C
H
0.8313 0.9882 2.75 15.31 2.1 1.7
0.9977 0.9998 0.12 2.07 0.1 0.1 1.1 0.5
0.9708 0.9919 0.52 12.7 0.4 0.3
0.8795 0.9776 5.4 21.0 4.7 2.6
80
%
60
40
20
Soil samples Munsell NCS
0 1
2
3
0.9942 0.9964 2.9 8.5
H
100
0
X
4
5
6
7
8
9
10
ΔE* ab
coordinates. In addition to the simplest solution, i.e. standard colour-space transformations available in the colour literature (e.g. Viscarra Rossel et al., 2006), we also tested statistical transformation models developed from the data measured in the laboratory using the Munsell soil-colour charts. Table 1 shows some of these statistical models, from a model of order one with four terms to a polynomial complex dependence of order three and 23 terms, although we found that cubic models with eight coefficients (last row in Table 1) optimised the computational work to be made by the smartphone providing a high correlation and relatively low colour differences between measured and predicted values. Table 2 lists the coefficients of the two cubic polynomial transformation equations forming matrices of dimension 3 8. For the calculations, we used the mean of three replicated measures on each sample, although its dispersion (standard deviation) was negligible both for RBG (<0.01) and XYZ (<0.03) because of the stability of the light source and homogeneity of the measured surface. The polynomial transformations proved slightly better than the standard transformations using the ColoSol software (Fig. 3). This latter underestimated the colour coordinates, except Munsell C, which was overestimated, while Z and H were the worst estimated parameters. This may be because the standard transformation
Y
Z
0.9945 0.9948 4,3 10.2 14.0 4.5
0.9904 0.9892 5.5 14.6
V
C 0.9965 0.9964 0.6 8.5 0.5 0.3 3.2 1.1
0.9764 0.9924 2.0 12.3 1.8 0.9
matrices were developed for different conditions (Wyszecki and Stiles, 2000) from those used here, the RGB values of a same colour might change because of their device dependence (Hong et al., 2001; Westland and Ripamonti, 2004) or the influence of light intensity (Viscarra Rossel et al., 2008), and the standard linear transformations could not exactly match that of real cameras. Certainly, the RGB measured from the mobile phone and the RGB linearly transformed with ColoSol from XYZ of the Munsell chips showed a similar relation (Fig. 4) to the typical curve for image devices, indicating a non-linear response of the sensor (Westland and Ripamonti, 2004). Whatever the reason, as in other studies dealing with the transformation between colour spaces (Johnson, 1996), the mean colour differences between measured and estimated values were lower using the polynomial transformations (Table 2), as concluded from results shown in Table 3. The prediction from the mobile phone of tristimulus XYZ proved more accurate than did the Munsell parameters HVC with either transformation, according to the statistical coefficients listed in Table 3. These coefficients are the Pearson’s linear correlation coefficient, the goodness of fit (GFC) coefficient (Romero et al., 1997), the root mean square (RMSE) error, and the standardized residual sum of squares (STRESS) index (García et al., 2007), which values for perfect fit are 1, 1, 0 and 0, respectively. The estimates of H were the worst. This indicates that using a spectroradiometer as a reference made it possible to find better transformation equations, presumably because the mobile-phone camera and spectroradiometer worked under the same illumination conditions (D65), including light quality and intensity, whereas Munsell colour space was defined with illuminant C. On the other hand, the RGB and XYZ colour spaces have similar geometries, whereas the cylindrical structure of Munsell system, where H is the angular coordinate, could hamper the transformation functions. It is known that when the Munsell radial coordinate C is low the uncertainty in the angular coordinate H is higher, because H becomes indeterminate at null chroma C. In our case, for example, of the 35 chips (16 in the chart 5Y and 12 in the chart 10R) estimated with DH > 4 units, 72% had C 6 2, 20% were extremely light and chromatic (7/8, 8/8), and 8% had C = 3. Accordingly, our H model performed worst for extreme colours located in the bottom left corner and upper right corner of some Munsell charts. With the polynomial transformation, the average of the colour differences between measured and predicted values in the 238 chips of Munsell charts was 2.0 ± 1.1 CIELAB units. It is noteworthy that more than 90% of the samples were measured by the mobile
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
The reliability of the mobile phone to detect colour was initially checked in the NCS samples covering the colour gamut of the Munsell soil-colour charts. Due partly to the even surface and homogenous colour of the samples, as well as the stability of illumination, the replicated measurements in each sample had negligible dispersion (standard deviation <0.06 in RGB with the mobile phone and <0.98 in XYZ with the spectroradiometer and <0.21 in HVC with the spectrophotometer). The differences in each Munsell colour coordinate (absolute values) between the determinations made with the app in the HTC mobile phone and the ones provided by our spectrophotometer are shown in Fig. 6 and the averages are listed in Table 4. The hue differences (DH) ranged between 4 and 7, with 70% of the samples within the [2, 2] interval. A closest agreement was found in Munsell value and chroma, with a range of DV and DC differences [2, 2] and [0, 2] units, respectively, most of the colour samples being within the interval [0, 1]. Despite the low mean value in DH (Table 4), its standard deviation was on the same order, indicating that some colour samples had a major error. According to the CIELAB differences found between the mobile phone estimates and spectroradiometric measurements (CIE tristimulus values XYZ), the NCS samples with the worst results were, like the above-mentioned major error in Munsell chips, samples yellower than 2.5Y (H > 22.5) and with low chroma or very light and chromatic (samples NCS-25, NCS-22, NCS-9, and NCS-1 in Table 5). Although the mean difference was 6.45 CIELAB units, it reached up to 9.4 CIELAB units in some extreme samples (Table 5), which produced the high standard deviation value of 2.05 CIELAB units listed in Table 4. Around 50% of the NCS samples were measured using the HTC mobile phone with an error of less than 6 CIELAB units (Fig. 5). The measuring errors of the mobile phone dramatically decreased in soil samples with respect to those in NCS samples when the spectroradiometer was considered as the reference (Fig. 5), and consequently their mean DEab was lower in Table 4, but increased somewhat when the spectrophotometer was considered (Fig. 6), primarily DH. On the one hand, this different behaviour could be attributed both to the illumination conditions, identical only for the phone camera and spectroradiometer, and the problems once again with the Munsell space geometry. On the other hand, the best DEab results in fine-earth soil samples, despite their uneven and textured surface with heterogeneous soil-pigment mixtures, might be explained by the fact that, although covering a wide gamut from 2.9YR to 2.3Y, these Mediterranean soil samples did not have extreme colours in hue, lightness, or chromaticity, which must have improved the performance of the mobile phone. Probably the relatively worse DEab results for the ground and homogenized soil samples than for fine-earth soil samples (Table 4) was also due to a somewhat more limiting colour gamut in the 15 soils from which fine earth as well as ground samples were prepared. For these 15 soils (on the average, 1.0Y and 3.0 in chroma), however, there was no statistically significant difference (P < 0.05) between the mobile-phone errors in fine earth (mean DEab ¼ 4:46) and ground (mean DEab ¼ 5:04) samples. The results indicate that
100 NCS Soils
80
60
%
3.2. Performance of the mobile-phone application in the validation samples
although the ground samples exhibited a more homogeneous aspect, it did not lessen the measuring errors of mobile phone, probably because of the way in which the colour of all pixels in a picture were managed in order to compute the soil colour as a whole. The mobile phone calculated the mode of the pixels present in an area of interest and with the use of this statistical parameter, anomalous pixels in a picture of fine earth did not influence the final colour coordinates. Table 5, which lists the best and worst study cases with their colour coordinates measured with the spectrophotometer (HVC), spectroradiometer (XYZ), and mobile phone (RGB), confirmed the good results for the soil samples: SOIL-va, SOIL-ck, SOIL-sn, SOIL-
40
20
0 0
1
2
3
4
5
6
7
8
9
10
ΔH 100 NCS
80
Soils
60
%
phone with accuracy of 4 CIELAB units or better (Fig. 5). When it is taken into account that the theoretical colour visual threshold was around 1.0 CIELAB units, that the practical threshold for most colorimetric applications is around 3–4 units, and that only thresholds greater than 5–6 units should be considered erroneous (Melgosa et al., 1992; Huang et al., 2012), the current results suggest that the implementation of polynomial transformations from RGB to XYZ or HVC in the mobile-phone application is the best option to provide accurate colour information.
40
20
0 0
1
2
3
4
5
6
7
8
9
10
9
10
ΔV 100 NCS Soils
80
60
%
206
40
20
0 0
1
2
3
4
5
6
7
8
ΔC Fig. 6. Histograms for NCS (n = 40) and soil (n = 60) samples with the differences (absolute value) in Munsell hue (DH), value (DV), and chroma (DC) between data measured with the spectrophotometer and predicted with the HTC smartphone.
207
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
ll, and SOIL-sh even with errors close to the theoretical colour threshold (1–2 CIELAB units). There was, however, an apparent lack of consistency in the magnitudes of CIELAB DEab and Munsell (DH, DV, DC) differences between measured and predicted values. Thus, for example, SOIL-sh had greater CIELAB difference (1.9 units) than SOIL-va (1.3 units), just the opposite behaviour that Munsell differences (0.3, 0.2, 0.1 against 0.6, 0.1, 0.2, for DH, DV, DC respectively). This fact, common in Table 5, has already been statistically-discussed with the results of Table 4 and Figs. 5 and 6, indicating that there was no a good correlation between spectrophotometric and spectroradiometric measurements. In this way, higher values in DEab did not necessarily correspond to higher errors in HVC and vice versa. Considering the entire dataset of soil samples (n = 60), we found that the discrepancies between the HTC smartphone and spectrophotometer ranged from 6 to 6 for Munsell hue, so that 60% of samples were determined with an error of less than 3.0 units (Fig. 6). For Munsell value and chroma, most soil samples had an error of less than 1.0 Munsell unit. The total colour difference between HTC smartphone and spectroradiometer was on the average 3.7 ± 1.8 CIELAB units, and more than 80% of soil samples
were measured by the mobile phone with an error lower than 5 CIELAB units (Fig. 5). It also bears mentioning that this error by the mobile phone in sensing colour is lower than those previously reported when the same soil samples were visually determined using Munsell soil-colour charts. Specifically, a group of 10 soil scientists judging soil colour under controlled illumination had a mean error of 4.4 CIELAB units in ground soil samples and 10.2 in aggregated soil samples with respect to instrumental measurement, and their inter-observer variability, i.e. disagreement among the ten Munsell notations on a given sample, was 5.1 CIELAB units (Sánchez-Marañón et al., 1995). Differences of up 5.5 CIELAB units were also measured by Sánchez-Marañón et al. (2005) in Munsell soil-colour charts from different editions, manufacturers, and degree of use, so inducing the same error in the soil colour determination. In addition, the error made by the smartphone estimating Munsell hue (59.0% of soils with DH > 2 units, Fig. 6) was lower than the variation in Munsell hue of soils observed under different natural daylight conditions (78.6% of soils with DH > 2 units, Sánchez-Marañón et al., 2011). Finally, once the mobile-phone application developed with the HTC smartphone was installed in another device (Samsung smart-
Table 4 Differences in Munsell hue, value, and chroma (DH, DV, and DC in absolute value) and CIELAB colour differences DEab between data measured with a spectrophotometer (HVC) and a spectroradiometer (XYZ) and estimated with two mobile phones (HTC and Samsung). Group of samples
Colour parameter
Mean value
Standard deviation
NCS samples HTC smartphone (n = 40)
DH DV DC DEab
1.69 0.66 0.36 6.45
1.69 0.22 0.34 2.05
Ground soil samples HTC smartphone (n = 15)
DH DV DC DEab
2.05 0.57 0.32 5.04
1.23 0.31 0.20 2.47
Fine-earth soil samples HTC smartphone (n = 45)
DH DV DC DEab
2.72 0.57 0.81 3.31
1.70 0.36 0.60 1.59
Soil samples (fine-earth plus ground samples) Samsung smartphone (n = 60)
DH DV DC DEab
3.38 0.63 0.25 5.46
1.55 0.26 0.17 2.46
Table 5 The five NCS- and SOIL-samples with the lowest (italics) and highest CIELAB colour differences DEab between the colour measured with a spectroradiometer (XYZ, mean of 3 replicates with SD < 1.38) and predicted from the RGB values (mean of 3 replicates with SD < 0.11) by the HTC smartphone. The Munsell HVC measured with a spectrophotometer (mean of 3 replicates with SD < 0.24) and the differences (DH, DV, DC) with the predicted values by the mobile phone are also listed. Sample
H
V
C
X
Y
Z
R
G
B
DH
DV
DC
DEab
NCS-38 NCS-28 NCS-21 NCS-30 NCS-35 SOIL-va SOIL-ck SOIL-sn SOIL-ll SOIL-sh
16.9 12.9 10.4 13.7 15.1 21.7 21.4 19.9 19.9 19.5
7.7 7.5 3.7 6.5 6.3 4.4 6.0 4.9 4.0 3.8
0.5 1.1 6.6 7.0 8.0 1.9 2.7 3.6 2.6 2.6
55.3 57.2 14.8 48.3 44.7 12.7 30.7 21.9 20.6 11.9
57.6 58.9 11.3 43.1 39.8 13.0 31.6 21.2 20.9 11.2
63.5 62.2 5.5 22.7 16.1 9.5 21.9 12.7 13.3 7.3
222.0 231.0 165.0 255.0 255.0 105.6 166.4 149.4 140.8 105.6
202.0 202.0 41.0 145.0 136.0 75.2 135.6 98.0 98.0 59.1
206.0 198.0 41.0 107.0 74.0 56.3 94.2 56.3 65.5 39.9
4.6 1.9 1.1 2.3 3.6 0.6 3.5 2.3 2.1 0.3
0.4 0.7 0.4 0.9 0.9 0.1 0.0 0.2 1.0 0.2
0.3 0.3 0.1 0.6 1.3 0.2 0.1 0.0 0.3 0.1
2.3 2.6 2.7 3.9 4.2 1.3 1.7 1.8 1.8 1.9
NCS-4 NCS-25 NCS-22 NCS-9 NCS-1 SOIL-vg SOIL-dd SOIL-ps SOIL-mo SOIL-bc
18.2 23.1 23.3 25.5 22.5 22.6 19.8 15.9 22.2 20.5
6.2 3.3 7.6 4.3 7.1 5.1 5.1 4.0 5.4 6.1
5.0 2.7 2.1 2.5 6.3 2.3 3.7 5.0 2.1 3.9
35.2 6.7 41.3 14.7 40.0 23.4 26.9 20.2 23.5 32.8
34.2 6.9 43.6 15.7 40.9 24.3 27.0 18.3 24.6 33.2
18.2 3.8 34.4 10.2 16.1 19.2 15.1 6.8 17.1 19.6
231.0 99.0 222.0 140.0 239.0 157.9 181.0 167.7 178.3 221.3
156.0 65.0 193.0 112.0 179.0 124.8 110.4 65.4 129.1 162.4
99.0 41.0 156.0 74.0 82.0 85.0 74.0 33.0 109.7 103.4
0.3 1.6 1.3 0.4 0.1 1.8 1.6 0.5 4.2 1.2
1.0 0.8 0.2 1.0 0.6 0.6 0.6 0.7 0.5 1.1
0.6 0.3 0.2 0.0 0.0 0.4 0.8 1.3 0.6 0.8
9.2 9.4 9.4 9.4 9.4 7.6 7.8 7.9 9.8 11.1
208
L. Gómez-Robledo et al. / Computers and Electronics in Agriculture 99 (2013) 200–208
phone), we measured the soil samples again under the same experimental conditions (Fig. 2). The results with the Samsung smartphone were only slightly worse than using the HTC (Table 4) but still acceptable. This indicates similarities in the performance of both mobile-phone cameras to capture the RGB signals and facilities in the operating system to interchange the software. 4. Conclusions Our results indicate that the technical resources of the current mobile phones can be exploited to use these electronic devices, which are prevalent worldwide and accessible to everyone, as soil-colour sensors. The RGB signals captured by the camera and converted in colour coordinates by a software application running inside the same smartphone, can thus achieve objective, easy, rapid, and cheap colour measurements. Under controlled illumination conditions, the measurements seem to be also more accurate than visual soil-colour determination using Munsell charts. Essential requirements of the mobile-phone application were a moveable and visible frame on the smartphone screen to select the region of interest in the picture, a calibration or normalization tool for referencing colour to a standard white, and transformation equations to convert RGB values in Munsell or CIE colour coordinates. In converting the image signals to colour, a polynomic-process transform fitted to the colour gamut of the Munsell soil-colour charts worked better than standard equations from the colour literature, and also proved more efficient in reference to XYZ coordinates (very similar in geometry to RBG) measured with a spectroradiometer under the same experimental conditions as the phone camera. Accordingly, although our initial objective was to reproduce the Munsell notation of soil samples, the results suggest that CIE coordinates are more accurately predicted for this application. Programming in Android allowed the application to be successfully interchanged between different devices. However, to achieve an application working similarly in different models of mobile phones constitutes a challenge, because it depends mainly on the quality of mobile-phone camera. The use of the mobile phone under non-controlled (variable) illumination conditions is another step forward for our future research. Acknowledgements This work has been partially funded by Ministry of Economy and Competitivity (Spain) under FIS2010-19839 Research Project, and Junta de Andalucía (Spain) under Project PE10-TIC5997, with European Regional Development Fund (ERDF) support. References ASTM, 2008. Standard Practice for Specifying Color by the Munsell System. ASTM International D 1535-08, PA, USA. Aydemir, S., Keskin, S., Drees, L.R., 2004. Quantification of soil features using digital image processing (DIP) techniques. Geoderma 119, 1–8. Berns, R., 2000. Billmeyer and Saltzman’s Principles of Color Technology. Wiley Interscience. Cadle, B.A., Rasmus, K.C., Varela, J.A., Leverich, L.S., O’Neill, C.E., Bachtell, R.K., Cooper, D.C., 2010. Cellular phone-based image acquisition and quantitative ratiometric method for detecting cocaine and benzoylecgonine for biological and forensic applications. Subst. Abuse 4, 21–33. CIE, 2004. Publication 15:2004. In: Bureau, C.C. (Ed.), Colorimetry, 3rd ed., Vienna. García, P.A., Huertas, r., Melgosa, M., Cui, G., 2007. Measurement of the relationship between perceived and computed color difference. J. Opt. Soc. Am. A 24, 1823– 1829.
García, A., Erenas, M.M., Marinetto, E.D., Abad, C.A., de Orbe-Paya, I., Palma, A.J., Capitán-Vallvey, L.F., 2011. Mobile phone platform as portable chemical analyzer. Sensor. Actuat. B – Chem. 156, 350–359. Godlove, I.H., 1951. Improved color-difference formula, with applications to the perceptibility and acceptability of fadings. J. Opt. Soc. Am. 41, 760–772. Hong, G., Luo, M.R., Rhodes, P.A., 2001. A study of digital camera colorimetric characterization based on polynomial modeling. Color Res. Appl. 26, 76–84. Huang, M., Liu, H., Cui, G., Luo, M.R., Melgosa, M., 2012. Evaluation of threshold color differences using printed samples. J. Opt. Soc. Am. A 29, 883–891. IUSS Working Group WRB, 2006. World reference base for soil resources 2006. World Soil Resources Reports 103. Johnson, T., 1996. Methods for characterizing colour scanners and digital cameras. Displays 16, 183–191. León, K., Mery, D., Pedreschi, F., León, J., 2006. Color measurement in L⁄a⁄b⁄ units from RGB digital images. Food Res. Int. 39, 1084–1091. Martinez, A.W., Phillips, S.T., Carrilho, E., Thomas 3rd., S.W., Sindi, H., Whitesides, G.M., 2008. Simple telemedicine for developing regions: camera phones and paper-based microfluidic devices for real-time, off-site diagnosis. Anal. Chem. 80, 3699–3707. Melgosa, M., Hita, E., Romero, J., del Barco, L.J., 1992. Some classical color differences calculated with new formulas. J. Opt. Soc. Am. A 9, 1247–1254. Meyer, G.E., Camargo Neto, J., Jones, D.D., Hindman, T.W., 2004. Intensified fuzzy clusters for classifying plant, soil, and residue regions of interest from color images. Comput. Electron. Agric. 42, 161–180. Munsell Color Company, 2000. Munsell Soil Color Charts. Munsell Color Co., Baltimore, MD. Nakamoto, T., Ikeda, T., Hirano, H., Arimoto, T., 2009. Humidity compensation by neural network for bad-smell sensing system using gas detector tube and builtin camera. 2009 IEEE Sensors, pp. 281–286. NCS – Natural Colour SystemÒÓ, 2013.