- Email: [email protected]
Leaf Scanner: A portable and low-cost multispectral corn leaf scanning device for precise phenotyping
Computers and Electronics in Agriculture 167 (2019) 105069
Contents lists available at ScienceDirect
Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag
Leaf Scanner: A portable and low-cost multispectral corn leaf scanning device for precise phenotyping
T
Libo Zhanga, Liangju Wanga, Jialei Wangb, Zhihang Songa, Tanzeel U. Rehmana, ⁎ Thirawat Bureetesa, Dongdong Maa, Ziling Chena, Samantha Neenoa, Jian Jina, a b
Department of Agricultural and Biological Engineering, Purdue University, West Lafayette, IN 47907, United States School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, United States
A R T I C LE I N FO
A B S T R A C T
Keywords: Plant health Nitrogen content Leaf chlorophyll content Leaf Scanner device Leaf area
During recent years, portable plant phenotyping instruments have become increasingly important to monitor conditions of plant health non-destructively in the greenhouse and field environments. These devices, such as the Soil Plant Analysis Development (SPAD) meter, leverage indices that represent leaf chlorophyll content due to its strong correlation with leaf nitrogen (N) content. However, instruments such as SPAD meters are expensive and measure only individual, restricted leaf regions per data capture. In this publication, we developed a Leaf Scanner device to analyze the chlorophyll content distributions of whole leaves with greater efficiency and precision. The validating samples for this device were top-collared corn leaves grown under high and low N treatments in a greenhouse. For each region of a corn leaf, this device rapidly flashed visible and near infrared (NIR) LEDs to obtain the visible and NIR transmittance images of leaves. These images were summarized pixelwisely into the Green NDVI index. A sufficient high framerate permitted continuous collection of regional index images. Using image registration techniques, these regional images were stitched together into a ‘leaf panorama’. The total pixel amount of each leaf panorama, thus, served as a substitute for leaf area. A Minolta SPAD-502Plus meter collected ground-truth measurements along the length of each leaf sample. The results showed that there was a strong correlation between the average Leaf Scanner’s measurements and averaged SPAD values (R2 = 0.92), and the Leaf Scanner was able to clearly detect differences between high and low fertilized samples in terms of chlorophyll content and leaf area. Furthermore, this Leaf Scanner device enabled us to study the chlorophyll content distributions on the plants under different treatments.
1. Introduction Nitrogen (N) is the major limiting nutrient in most soils and is, thus, critical for plant growth (Daughtry et al., 2000). Indeed, the N deficiency suppresses corn plant growth rate. For instance, the leaf area of a low N treatment plant is smaller than a high N plant (Zhao et al., 2003). The leaf chlorophyll content is a common biochemical parameter to indicate the conditions of plant health because of its close link with the N content (Yoder and Pettigrew-Crosby, 1995). The normalized difference vegetation index (NDVI) allows us to represent the chlorophyll contents or the greenness of plants (Jackson et al., 1983; Ma et al., 2019a; Ma et al., 2019b). It employs the chlorophyll’s high absorption in the red band and high reflectivity in the near-infrared (NIR) band (Zhang et al., 2019). The chlorophyll content and leaf area can be nondestructively recorded using plant phenotyping technologies, whereas,
the traditional techniques need to cut plant tissues and conduct analysis with specific laboratory equipment (Shepherd et al., 1996; Wilhelm et al., 2000; Fystro, 2002; Mokhtarpour et al., 2010; Tester and Langridge, 2010; Fiorani and Schurr, 2013; Cavallo et al., 2017; Xiong et al., 2017). However, the current plant phenotyping devices such as SPAD meters measure only individual, restricted leaf region per data capture and cannot represent nutrient distribution across the leaf samples. Thus, an opportunity exists to devise a portable device that can monitor health conditions of crop plants comprehensively and nondestructively. The limiting factor to the most existing portable crop-monitoring devices is their ability to gather as much data per leaf as possible because of the inter-leaf nutrient distributions. For instance, the SPAD meter (Minolta Co., Ltd., Osaka, Japan) is one of the most widely utilized diagnostic tools to non-invasively measure leaf chlorophyll
⁎ Corresponding author at: Department of Agricultural and Biological Engineering, Purdue University, 225 South University Street, West Lafayette, IN 47907, United States. E-mail address: [email protected] (J. Jin).
https://doi.org/10.1016/j.compag.2019.105069 Received 10 April 2019; Received in revised form 18 October 2019; Accepted 21 October 2019 0168-1699/ © 2019 Published by Elsevier B.V.
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
the battery output and the anodes of LEDs to improve the stability of LEDs when they were working. And two commonly used 2N2222 NPN transistors were installed amid the cathodes of LEDs and the GPIOs of the Raspberry Pi to achieve the high-speed on and off of LEDs. In the Leaf Scanner device, the SMD white LED was approximately $ 0.20 each, and the SMD NIR LED was approximately $0.65 each. The Raspberry Pi Noir camera module V2 was $24.58, and the Raspberry Pi 3 model B motherboard was $34.49. The TalentCell rechargeable lithium ion battery was $24.99, and the DROK DC voltage regulator was $15.35. Including the expenses of almost all the components, the gross material cost of the Leaf Scanner was $130. The cost is much lower than the current instruments such as the Minolta SPAD-502Plus meter (approximately $2500). The cost can be lower if we optimize the design and configuration of the device in the future (Carbone et al. 2019). For instance, the Raspberry Pi Noir V1 camera with a lower price also meets our needs, and a more compact microcontroller saves both space and money.
contents by collecting transmittance intensity of visible and NIR light (Uddling et al., 2007). Its price is approximately $2500, and its measuring area is 2 mm × 3 mm. Ciganda et al. (2008) indicated that its readings from different spots on a same leaf could have up to 500% difference. The successful SPAD meter measurements are impacted by measurement positions on the leaf (Ata-Ul-Karim et al., 2014; Yang et al., 2014). These fluctuations can be explained by the distributions of chlorophyll across the leaves, which are not uniform and can be extremely differential in localized patches due to disease (Schepers et al., 1992; Chapman and Barreto, 1997; Ward et al., 2007; Lin et al., 2010). Besides the SPAD meter, the MultispeQ (Osei-Bonsu et al., 2016) and the LI-COR photosynthesis systems (LI-COR Biosciences, Lincoln, NE, USA) are also devices that have been used to detect chlorophyll levels. The MultispeQ uses different peak emission wavelengths of LEDs to achieve different measurements such as leaf chlorophyll content. Its measuring area is about 1 cm2 on the leaf surface. The LI-COR integrates LED light source to provide controllable environment surrounding a certain leaf area (2 cm2 for 6400-40 leaf chamber and 6 cm2 for 6400-02B leaf chamber). However, none of these devices collects the image of a whole leaf. They all measure only a small local spot on the leaf, and the measurement result is very vulnerable to the change of the measurement position on the leaf (Cao et al., 2016). Besides, one such spot does not well represent the whole leaf’s chlorophyll level which normally contains big variance. As a result, these devices above fall short of measuring chlorophyll distribution variances due to their limited measuring areas (Weber et al., 2012; Hooper et al., 2002). To more precisely and efficiently analyze the conditions of plant health and the distributions of chlorophyll contents of the whole leaves, we developed a portable Leaf Scanner. This device captured the transmittance radiation of visible and NIR LEDs through the leaves. It was compared against the SPAD meter in its ability to detect N contents of plants under different treatments, and tested as a method to compute leaf area of living plants.
2.2. Workflow In this publication, the controlling programs were written in Python. Before scanning, the Pi camera parameters such as camera sensor mode, resolution, framerate, ISO, exposure mode, shutter speed, exposure compensation, auto white balance mode and brightness needed to be initialized (Pereira et al., 2014). The visible LEDs were turned on first and one RGB image was captured. The NIR LEDs were then turned on and another NIR image was captured. To rapidly flash the visible and NIR LEDs and capture corresponding images, the video port of the Pi camera was activated, and the image was captured into a stream whose values were stored in an array. The arrays were finally saved as JPEG images to the local SD card. The images were then copied to the PC for further processing. The Leaf Scanner was capable of capturing images with a framerate of 12–15 fps, which meant that we were able to obtain at least 6 pairs of visible and NIR images per second. The entire workflow of the Leaf Scanner is described in Fig. 2.
2. Design of Leaf Scanner 2.1. Hardware configuration
3. Materials and methods The Leaf Scanner was designed in a way that a leaf could be pulled through a self-contained imaging chamber. The case of the device was designed using the Solidworks 2017 software (Dassault Systèmes SOLIDWORKS Corp., MA, USA) and printed using a MakerBot Replicator + 3D printer (MakerBot Industries, LLC, NY, USA) with black PLA filament materials. As illustrated in Fig. 1, the Leaf Scanner was mainly comprised of a lighting box, a Raspberry Pi camera, and a Raspberry pi microcontroller. The lighting box consisted of 24 SMD white LEDs (peaks at 450 and 600 nm, Cree Inc., NC, USA) and 24 SMD NIR LEDs (peak at 880 nm, Kingbright Co., CA, USA). The area of the lighting box was 12 cm (maximum leaf width) × 9 cm (scanning direction). The two categories of LEDs were uniformly distributed in the lighting box, and underneath them, a 1/8 in. thick Teflon panel (McMaster-Carr Supply Company, IL, USA) served as a light diffuser (Tomer et al., 2017; Noviyanto and Abdulla, 2019). The Raspberry Pi Noir camera module V2 (Raspberry Pi foundation, UK) was used to capture visible and NIR images. It was built on a high quality 8 megapixel Sony IMX219 image sensor, and there was no infrared filter on the lens. A piece of glass was installed above the camera lens to protect the imager and support the leaves when they were pulled through the sensor. The Raspberry Pi 3 model B motherboard (Raspberry Pi foundation, UK) was used as the microcontroller to rapidly flash the visible and NIR LEDs and control the Pi camera to capture images (Edwards, 2013). Both the LEDs and Raspberry Pi were powered by a TalentCell rechargeable lithium ion battery (12 V 3000mAh, TalentCell Technology Co., Ltd., Guangdong, China). A DROK DC voltage regulator (converting 3–34 V to 4–35 V, DROK, Guangdong, China) was installed amid
3.1. Plant samples The corn plants (hybrid B73 × Mo17 genotype) were grown in the Lilly greenhouses and plant growth facility at Purdue University (40°25′19.6″N, 86°55′7.8″W). The plants were grown in the mix soil of Profile® Greens Grade™ and Sungro Metro-Mix® 510, and divided into two N treatment plots: 200 ppm versus 25 ppm. All of the plants were watered as needed and fertilized weekly. The greenhouse temperature was maintained between 23.9 and 28.9 °C in the daytime and between 21.1 and 25.6 °C at night. The complementary light was provided with 600 W high pressure sodium bulbs set to a 14-h photoperiod. 3.2. Data acquisition Once the plants were at V6 or V7 stage (BBCH scale: 33 or 34), 19 of high N plants and 19 of low N plants were randomly selected as the experimental samples. For each plant, the top-collared leaf was scanned from the collar to the tip using this Leaf Scanner device. The entire scanning length was approximately 50–60 cm, and it was completed within approximately 50 s. The position difference between two next visible and NIR images was approximately 1.6 mm. In this publication, to simplify the leaf stitching, we assumed that there was no significant leaf position difference between each pair of visible and NIR images. Also, the Minolta SPAD-502Plus meter (Minolta Co., Ltd., Osaka, Japan) was utilized to measure SPAD values of 10 equidistant spots on the same side of the leaf midrib from the collar to the tip. The captured images were finally duplicated from the Raspberry Pi’s local SD card to 2
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Fig. 1. The configuration and leaf scanning prototype of the Leaf Scanner device. (a) Schematic diagram (1. light diffuser 2. glass 3. camera chamber 4. battery 5. scan button 6. pre-tightening spring 7. device holder 8. lighting chamber). (b) LED board. (c) Corn leaf scanning in the greenhouse.
Fig. 2. The entire workflow of the Leaf Scanner device. 3
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
PC, and the recorded SPAD data was written into an excel file.
based on the coordinates of the best pair of matched points (Fig. 3(a)). Generally, after the RANSAC examination, the homographic matrix would be calculated, and the geometric transformation would be estimated in terms of the matched points. However, it was not always robust due to the monotonous features on the leaf surface. Therefore, in this publication, we selected the best matching point pair, vertically cut the images and stitched them based on the coordinates of the best pair of matched points. The final Green NDVI mosaic results are demonstrated in Fig. 3(b) and (c). The Green NDVI panoramas enable us to further study the chlorophyll content distribution across the leaf surface and monitor the leaf area changes of plants under different treatments.
3.3. Leaf panorama stitching The Raspberry Pi Noir V2 camera required either high red light intensity or high exposure time to return visible images with qualified red bands. This could not be abided since it would sacrifice framerate, which needed to be guaranteed for continuous data collection. Therefore, in this publication, we computed the Green NDVI conforming the Eq. (1) instead of NDVI (Gitelson et al., 1996). As Daughtry et al. (2000) indicated, there was a strong correlation between normalized NIR/Green and NIR/Red (R2 > 0.95). And Patrick et al. (2017) suggested that the correlation between Green NDVI and manual assessment of wilt disease in peanuts (R2 = 0.81) was higher than that between NDVI and manual disease assessment (R2 = 0.71). In this publication, the Green NDVI image was derived from a region’s two visible and NIR images. The visible image was captured when the visible LEDs were on. And the NIR image was captured when the NIR LEDs were on. Both the visible and NIR images were RGB images because the RGB sensor without the NIR filter was used in the Leaf Scanner device.
Green NDVI =
NIR − Green NIR + Green
3.4. Data analysis Firstly, for each plant, we averaged the Green NDVI panorama and the SPAD measurements of the top-collared leaf, and conducted the linear regression analysis to examine the correlation between the averaged Green NDVI and SPAD values. Secondly, we compared the Green NDVI distribution with the SPAD measurements distribution along the leaf length direction. Thirdly, from the Green NDVI panorama, we extracted the leaf area and applied the paired t-test to compare the differences of leaf areas between high and low N treatments plants.
(1)
where NIR denotes the red band from the NIR image, and Green denotes the green band from the visible image. The Green NDVI images of leaf regions were stitched to a ‘leaf panorama’ using an algorithm written in the Matlab R2016a software (The MathWorks Inc., MA, USA). Firstly, between two next Green NDVI images, the corner features (Rockett, 2003) using minimum eigenvalue algorithm were detected, and interest points were extracted. Secondly, the matching points were determined based on the pairwise distances between detected features (Lowe, 2004). Thirdly, the random sample consensus (RANSAC) method was utilized to remove the outliers of the matched points (Tarsha-Kurdi et al., 2007). Fourthly, the inliers of the matched points were sorted based on their pairwise distances to pick out the best pair of matched points. Also, the matched points were restrained by their relative positions on the two next Green NDVI images. The leaf was moving from right to left, and this matched point on the latter image was slightly forward than that on the former image. Finally, the two Green NDVI images were vertically cut and stitched
4. Results 4.1. Averaged Green NDVI and SPAD values As illustrated in Fig. 4, both the averaged SPAD and the Green NDVI values from the high N treatment plants demonstrated higher values than those from the low N treatment plants. This difference was proved to be statistically significant through the paired t-test. The p-values of both the averaged SPAD values and the averaged Green NDVI values between different N treatments were less than 0.05 (5.36e−10 and 5.14e−08, respectively). As illustrated in Fig. 5, a linear regression model was fitted between the averaged Green NDVI and SPAD values. The result showed that the R-squared value was 0.92. Thus, there was a strong correlation between the averaged Green NDVI and SPAD measurements. This Leaf Scanner device was capable of estimating the chlorophyll contents of plants.
Fig. 3. Leaf Green NDVI panorama stitching. (a) Best pair of matched points and cutting lines. (b) Green NDVI panorama of the high N leaf. (c) Green NDVI panorama of the low N leaf. 4
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Fig. 4. Box plot of averaged SPAD and Green NDVI values under different N treatments. The horizontal lines inside the boxes are median lines. There are no units for both Green NDVI and SPAD values.
Fig. 5. The linear regression fit between the averaged Green NDVI and SPAD values. Fig. 7. Box plot of leaf areas in pixels of plants under different N treatments.
4.2. Distribution of Green NDVI and SPAD values
chlorophyll content first increased from the leaf collar to the tip, and then slightly dropped down near the leaf tip region. The continuous nature of the Green NDVI distribution permitted it to be fitted with second degree polynomials for both treatment groups. The R2 of the high and low N fit curves were 0.97 and 0.99, respectively. The fit curves of high N and low N had the same shape (both quadratic coef−b ficients equaled -2E-07). The axis of symmetry ( x = 2a ) of the high N
It is crucial to explore the chlorophyll content distribution across the leaf surface because of the non-uniform nutrient concentrations on the whole leaf. In this publication, we summarized the SPAD and Green NDVI value distributions from the leaf collar to the tip, which enabled us to seek out underlying trends. After averaging the distributions for plants in each treatment, we obtained two SPAD distribution curves for high and low N leaves, respectively (Fig. 6(a)). For each Green NDVI panorama, we averaged each pixel column (a sliver perpendicular to the vein of the leaf) to obtain the Green NDVI distribution along the leaf length direction. Through binning different leaf lengths to the same span, we also obtained two averaged Green NDVI distribution curves for high and low N leaves, respectively (Fig. 6(b)). Currently, all pixels in the column including midrib pixels are included. Removing midrib pixels can be done as another option for the future version. Both the SPAD value and Green NDVI distributions denoted that the
curve was approximately in the middle of the leaf ( 1250 ), whereas, the 1600 axis of symmetry of the low N curve was approximately at the three 1000 fourths position from the leaf collar to the tip ( ). The result was 1600 based on 38 corn plants, and it requires more tests to verify this conclusion. Therefore, this Leaf Scanner device enabled us to further study the chlorophyll content distributions on the plants under different treatments. It was difficult to achieve this goal using current portable instruments such as the SPAD meter due to their limited measure areas.
Fig. 6. The distribution of SPAD and Green NDVI values from the leaf collar to the tip. (a) 10 SPAD measurements on each leaf. (b) Green NDVI distribution curves are fitted with the second degree polynomials. 5
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Fig. 8. The leaf curling problem statement with a Green NDVI panorama.
Fig. 9. The Green NDVI panorama of a leaf cut off from a corn plant.
outliers. The curling situation might be worse when the plants are under water stress treatments. Therefore, to obtain the panoramas of complete leaves in more stressed conditions, the next version of the Leaf Scanner device will include improvements to mechanisms configuring the leaf into a spread position when scanning.
4.3. Comparison of leaf area The total leaf pixel amount was extracted from the Green NDVI panorama and regarded as the relative leaf area. To compare the difference of leaf areas of plants under different N treatments, we applied the paired t-test. As illustrated in Fig. 7, the p-value (0.0265) of total leaf pixels between different N treatments was less than 0.05. Therefore, there existed a significant difference between the leaf areas of high and low N treatments plants. In this publication, we were not able to collect ground truth measurements for the leaf areas. We will utilize the Li-Cor Leaf Area Meter (LI-COR Biosciences, Lincoln, NE, USA) or the CI-202 Laser Area Meter (CID Bio-Science, Inc., WA, USA) to collate the measurements of this Leaf Scanner device in the following tests (Cutini et al., 1998).
5.3.2. Stitching improvement It was difficult to avoid hand vibration and uneven speed when scanning the living plants, which might impact the leaf stitching quality. As exhibited in Fig. 9, a leaf was cut off and pulled through the imaging chamber with a uniform speed, and the leaf regions were stitched using the same algorithm. The stitching seams on this leaf panorama were almost invisible, and the stitching quality was improved. Thus, optimizing the stitching algorithm to fit the working environment is required in the development of next version device.
5. Discussion 6. Conclusions 5.1. Device advantages The existing portable instruments targeting at estimating the conditions of plant health non-destructively, generally, measure only one small leaf area at one measurement. They are not efficient to cover the entire leaf even the entire plant and estimate the chlorophyll content distribution. In this publication, we initiated the leaf scanning and stitching approaches with the low-cost components (gross cost was $130), and designed a Leaf Scanner device. This device utilized the index of Green NDVI to represent chlorophyll contents. This device manifested a strong correlation (R2 = 0.92) with the averaged measurements from the Minolta SPAD-502Plus meter. Furthermore, this Leaf Scanner device enabled us to fit polynomial regressions and study the chlorophyll content distributions on the plants under different treatments. It was difficult to achieve this goal using current portable instruments such as the SPAD meter due to their limited measure areas. In addition, this device exhibited that plants under high and low N treatments had a significant difference on the leaf areas (p-value < 0.05). For the next version of this Leaf Scanner device, we will raise image capturing framerate and increase the scanning speed by optimizing the configuration of LEDs.
In this publication, a portable and low-cost Leaf Scanner device was designed to measure the health conditions of plants in the greenhouse or field environments. The device captures the transmittance of visible and NIR light through the corn leaf and works in the same way as the SPAD meter to indicate the chlorophyll content of plants. The advantages of this Leaf Scanner device are: (1) manifesting the chlorophyll distribution across the leaf surface instead of a few small spots on the leaf, (2) comparing the leaf areas of plants under different treatments since it is able to collect the whole leaf information. 5.2. Device limitations However, there are some limitations about this low-cost device. The images captured by this device are 8-bit images. Compared with the widely used 12-bit or 16-bit sensors, this device has a limited ability to handle complicated ambient lighting conditions. In addition, the measurement result is largely impacted by the plant stage and the senescence process of the plant. The relationships between NDVI, chlorophyll and N content of plants vary with different crop genotypes, species and conditions, such as diseased or senescent plants. Thus, it is necessary to conduct more tests using this Leaf Scanner device and collect corresponding ground truth measurements to calibrate it.
Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
5.3. Opportunities for future work
Acknowledgements
5.3.1. Leaf curling Leaf curling sometimes led to leaf folding during scanning, occurring especially often at the leaf collars. The Green NDVI values were eccentrically high at these doubled areas, as marked by a red color ellipse in Fig. 8. In this publication, these high values were removed as
The authors are grateful to Shelby Gruss in the Department of Agronomy at Purdue University for her great work in cultivating the corn plants. And we also thank Ron Steiner, the coordinator in the Lilly 6
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Greenhouses and Plant Growth Facility at Purdue University, for his help with well controlling the greenhouse environment.
conveyor system for indoor phenotyping. Comput. Electron. Agric. 166, 104979. https://doi.org/10.1016/j.compag.2019.104979. Mokhtarpour, H., Christopher, B.S., Saleh, G., Selamat, A.B., Asadi, M.E., Kamkar, B., 2010. Non-destructive estimation of maize leaf area, fresh weight, and dry weight using leaf length and leaf width. Commun. Biometry Crop Sci. 5 (1), 19–26. Noviyanto, A., Abdulla, W.H., 2019. Segmentation and calibration of hyperspectral imaging for honey analysis. Comput. Electron. Agric. 159, 129–139. https://doi.org/ 10.1016/j.compag.2019.02.006. Osei-Bonsu, I., Hoh, D., TerAvest, D., Kuhlgert, S., Austic, G., Weebadde, P., Kramer, D.M., 2016. MultispeQ Beta: a tool for large-scale plant phenotyping connected to the open PhotosynQ network. R. Soc. Open Sci. 3 (10), 160592. https://doi.org/10. 1098/rsos.160592. Patrick, A., Pelham, S., Culbreath, A., Holbrook, C.C., Godoy, I.J.D., Li, C., 2017. High throughput phenotyping of tomato spot wilt disease in peanuts using unmanned aerial systems and multispectral imaging. IEEE Instrum. Meas. Mag. 20 (3), 4–12. https://doi.org/10.1109/MIM.2017.7951684. Pereira, V., Fernandes, V.A., Sequeira, J., 2014. Low cost object sorting robotic arm using Raspberry Pi. In: 2014 IEEE Global Humanitarian Technology Conference-South Asia Satellite. GHTC-SAS 2014, pp. 1–6. Rockett, P.I., 2003. Performance assessment of feature detection algorithms: a methodology and case study on corner detectors. IEEE Trans. Image Process. 12 (12), 1668–1676. https://doi.org/10.1109/TIP.2003.818041. Schepers, J.S., Francis, D.D., Vigil, M., Below, F.E., 1992. Comparison of corn leaf nitrogen concentration and chlorophyll meter readings. Commun. Soil Sci. Plant Anal. 23 (17–20), 2173–2187. https://doi.org/10.1080/00103629209368733. Shepherd, M.A., Stockdale, E.A., Powlson, D.S., Jarvis, S.C., 1996. The influence of organic nitrogen mineralization on the management of agricultural systems in the UK. Soil Use Manage. 12 (2), 76–85. https://doi.org/10.1111/j.1475-2743.1996. tb00963.x. Tarsha-Kurdi, F., Landes, T., Grussenmeyer, P., 2007. Hough-transform and extended RANSAC algorithms for automatic detection of 3D building roof planes from Lidar data. In: ISPRS Workshop on Laser Scanning 2007 and SilviLaser 2007, XXXVI, pp. 407–412. Tester, M., Langridge, P., 2010. Breeding technologies to increase crop production in a changing world. Science 327 (5967), 818–822. https://doi.org/10.1126/science. 1183700. Tomer, N., McGlone, A., Künnemeyer, R., 2017. Validated simulations of diffuse optical transmission measurements on produce. Comput. Electron. Agric. 134, 94–101. https://doi.org/10.1016/j.compag.2017.01.006. Uddling, J., Gelang-Alfredsson, J., Piikki, K., Pleijel, H., 2007. Evaluating the relationship between leaf chlorophyll concentration and SPAD-502 chlorophyll meter readings. Photosynth. Res. 91 (1), 37–46. https://doi.org/10.1007/s11120-006-9077-5. Ward, J.M.J., Stromberg, E.L., Nowell, D.C., Nutter, F.W., 2007. Gray leaf spot: A disease of global importance in maize production. Plant Dis. 83 (10), 884–895. https://doi. org/10.1094/pdis.1999.83.10.884. Weber, V.S., Araus, J.L., Cairns, J.E., Sanchez, C., Melchinger, A.E., Orsini, E., 2012. Prediction of grain yield using reflectance spectra of canopy and leaves in maize plants grown under different water regimes. Field Crops Res. 128, 82–90. https://doi. org/10.1016/j.fcr.2011.12.016. Wilhelm, W.W., Ruwe, K., Schlemmer, M.R., 2000. Comparison of three leaf area index meters in a corn canopy. Crop Sci. 40, 1179–1183. https://doi.org/10.2135/ cropsci2000.4041179x. Xiong, X., Yu, L., Yang, W., Liu, M., Jiang, N., Wu, D., Liu, Q., 2017. A high-throughput stereo-imaging system for quantifying rape leaf traits during the seedling stage. Plant Methods 13 (1), 1–17. https://doi.org/10.1186/s13007-017-0157-7. Yang, H., Yang, J., Lv, Y., He, J., 2014. SPAD values and nitrogen nutrition index for the evaluation of rice nitrogen status. Plant Prod. Sci. 17 (1), 81–92. https://doi.org/10. 1626/pps.17.81. Yoder, B.J., Pettigrew-Crosby, R.E., 1995. Predicting nitrogen and chlorophyll content and concentrations from reflectance spectra (400–2500 nm) at leaf and canopy scales. Remote Sens. Environ. 53 (3), 199–211. https://doi.org/10.1016/00344257(95)00135-N. Zhang, L., Maki, H., Ma, D., Sánchez-Gallego, J.A., Mickelbart, M.V., Wang, L., Rehman, T.U., Jin, J., 2019. Optimized angles of the swing hyperspectral imaging system for single corn plant. Comput. Electron. Agric. 156, 349–359. https://doi.org/10.1016/j. compag.2018.11.030. Zhao, D., Reddy, K.R., Kakani, V.G., Read, J.J., Carter, G.A., 2003. Corn (Zea mays L.) growth, leaf pigment concentration, photosynthesis and leaf hyperspectral reflectance properties as affected by nitrogen supply. Plant Soil 257 (1), 205–217. https://doi.org/10.1023/A: 1026233732507.
Appendix A. Supplementary material Supplementary data to this article can be found online at https:// doi.org/10.1016/j.compag.2019.105069. References Ata-Ul-Karim, S.T., Zhu, Y., Yao, X., Cao, W., 2014. Determination of critical nitrogen dilution curve based on leaf area index in rice. Field Crops Res. 167, 76–85. https:// doi.org/10.1016/j.fcr.2014.07.010. Cao, W., Tian, Y., Cao, Q., Yuan, Z., Liu, X., Ata-Ul-Karim, S.T., Zhu, Y., 2016. Optimal Leaf Positions for SPAD Meter Measurement in Rice. Front. Plant Sci. 7, 1–10. https://doi.org/10.3389/fpls.2016.00719. Carbone, M.S., Seyednasrollah, B., Rademacher, T.T., Basler, D., Le Moine, J.M., Beals, S., Beasley, J., Greene, A., Kelroy, J., Richardson, A.D., 2019. Flux Puppy – an opensource software application and portable system design for low-cost manual measurements of CO2 and H2O fluxes. Agric. For. Meteorol. 274, 1–6. https://doi.org/10. 1016/j.agrformet.2019.04.012. Cavallo, D.P., Cefola, M., Pace, B., Logrieco, A.F., Attolico, G., 2017. Contactless and nondestructive chlorophyll content prediction by random forest regression: A case study on fresh-cut rocket leaves. Comput. Electron. Agric. 140, 303–310. https://doi.org/ 10.1016/j.compag.2017.06.012. Chapman, S.C., Barreto, H.J., 1997. Using a chlorophyll meter to estimate specific leaf nitrogen of tropical maize during vegetative growth. Agron. J. 89 (4), 557–562. https://doi.org/10.2134/agronj1997.00021962008900040004x. Ciganda, V., Gitelson, A., Schepers, J., 2008. Vertical profile and temporal variation of chlorophyll in maize canopy: Quantitative “crop vigor” indicator by means of reflectance-based techniques. Agron. J. 100 (5), 1409–1417. https://doi.org/10.2134/ agronj2007.0322. Cutini, A., Matteucci, G., Mugnozza, G.S., 1998. Estimation of leaf area index with the LiCor LAI 2000 in deciduous forests. For. Ecol. Manage. 105, 55–65. https://doi.org/ 10.1016/S0378-1127(97)00269-7. Daughtry, C.S.T., Walthall, C.L., Kim, M.S., Colstoun, E.B.D., McMurtrey, J.E., 2000. Estimating corn leaf chlorophyll concentration from leaf and canopy reflectance. Remote Sens. Environ. 74 (2), 229–239. https://doi.org/10.1016/S0034-4257(00) 00113-9. Edwards, C., 2013. Not-so-humble Raspberry Pi gets big ideas. Eng. Technol. 8 (3), 30–33. https://doi.org/10.1049/et.2013.0301. Fiorani, F., Schurr, U., 2013. Future Scenarios for Plant Phenotyping. Annu. Rev. Plant Biol. 64 (1), 267–291. https://doi.org/10.1146/annurev-arplant-050312-120137. Fystro, G., 2002. The prediction of C and N content and their potential mineralization in heterogeneous soil samples using Vis-NIR spectroscopy and comparative methods. Plant Soil 246 (2), 139–149. https://doi.org/10.1023/A:1020612319014. Gitelson, A.A., Kaufman, Y.J., Merzlyak, M.N., 1996. Use of a green channel in remote sensing of global vegetation from EOS-MODIS. Remote Sens. Environ. 58 (3), 289–298. https://doi.org/10.1016/S0034-4257(96)00072-7. Hooper, D.U., Cardon, Z.G., Chapin, F.S., Durant, M., 2002. Corrected calculations for soil and ecosystem measurements of CO2 flux using the LI-COR 6200 portable photosynthesis system. Oecologia 132 (1), 1–11. https://doi.org/10.1007/s00442-0020870-3. Jackson, R.D., Slater, P.N., Pinter, P.J., 1983. Discrimination of growth and water stress in wheat by various vegetation indices through clear and turbid atmospheres. Remote Sens. Environ. 13 (3), 187–208. https://doi.org/10.1016/0034-4257(83)90039-1. Lin, F., Qiu, L., Deng, J., Shi, Y., Chen, L., Wang, K., 2010. Investigation of SPAD meterbased indices for estimating rice nitrogen status. Comput. Electron. Agric. 71 (SUPPL. 1), 60–65. https://doi.org/10.1016/j.compag.2009.09.006. Lowe, D.G., 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60 (2), 91–110. https://doi.org/10.1023/B:VISI.0000029664. 99615.94. Ma, D., Carpenter, N., Maki, H., Rehman, T.U., Tuinstra, M.R., Jin, J., 2019a. Greenhouse environment modeling and simulation for microclimate control. Comput. Electron. Agric. 162, 134–142. https://doi.org/10.1016/j.compag.2019.04.013. Ma, D., Carpenter, N., Amatya, S., Maki, H., Wang, L., Zhang, L., Neeno, S., Tuinstra, M.R., Jin, J., 2019b. Removal of greenhouse microclimate heterogeneity with
7
Contents lists available at ScienceDirect
Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag
Leaf Scanner: A portable and low-cost multispectral corn leaf scanning device for precise phenotyping
T
Libo Zhanga, Liangju Wanga, Jialei Wangb, Zhihang Songa, Tanzeel U. Rehmana, ⁎ Thirawat Bureetesa, Dongdong Maa, Ziling Chena, Samantha Neenoa, Jian Jina, a b
Department of Agricultural and Biological Engineering, Purdue University, West Lafayette, IN 47907, United States School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, United States
A R T I C LE I N FO
A B S T R A C T
Keywords: Plant health Nitrogen content Leaf chlorophyll content Leaf Scanner device Leaf area
During recent years, portable plant phenotyping instruments have become increasingly important to monitor conditions of plant health non-destructively in the greenhouse and field environments. These devices, such as the Soil Plant Analysis Development (SPAD) meter, leverage indices that represent leaf chlorophyll content due to its strong correlation with leaf nitrogen (N) content. However, instruments such as SPAD meters are expensive and measure only individual, restricted leaf regions per data capture. In this publication, we developed a Leaf Scanner device to analyze the chlorophyll content distributions of whole leaves with greater efficiency and precision. The validating samples for this device were top-collared corn leaves grown under high and low N treatments in a greenhouse. For each region of a corn leaf, this device rapidly flashed visible and near infrared (NIR) LEDs to obtain the visible and NIR transmittance images of leaves. These images were summarized pixelwisely into the Green NDVI index. A sufficient high framerate permitted continuous collection of regional index images. Using image registration techniques, these regional images were stitched together into a ‘leaf panorama’. The total pixel amount of each leaf panorama, thus, served as a substitute for leaf area. A Minolta SPAD-502Plus meter collected ground-truth measurements along the length of each leaf sample. The results showed that there was a strong correlation between the average Leaf Scanner’s measurements and averaged SPAD values (R2 = 0.92), and the Leaf Scanner was able to clearly detect differences between high and low fertilized samples in terms of chlorophyll content and leaf area. Furthermore, this Leaf Scanner device enabled us to study the chlorophyll content distributions on the plants under different treatments.
1. Introduction Nitrogen (N) is the major limiting nutrient in most soils and is, thus, critical for plant growth (Daughtry et al., 2000). Indeed, the N deficiency suppresses corn plant growth rate. For instance, the leaf area of a low N treatment plant is smaller than a high N plant (Zhao et al., 2003). The leaf chlorophyll content is a common biochemical parameter to indicate the conditions of plant health because of its close link with the N content (Yoder and Pettigrew-Crosby, 1995). The normalized difference vegetation index (NDVI) allows us to represent the chlorophyll contents or the greenness of plants (Jackson et al., 1983; Ma et al., 2019a; Ma et al., 2019b). It employs the chlorophyll’s high absorption in the red band and high reflectivity in the near-infrared (NIR) band (Zhang et al., 2019). The chlorophyll content and leaf area can be nondestructively recorded using plant phenotyping technologies, whereas,
the traditional techniques need to cut plant tissues and conduct analysis with specific laboratory equipment (Shepherd et al., 1996; Wilhelm et al., 2000; Fystro, 2002; Mokhtarpour et al., 2010; Tester and Langridge, 2010; Fiorani and Schurr, 2013; Cavallo et al., 2017; Xiong et al., 2017). However, the current plant phenotyping devices such as SPAD meters measure only individual, restricted leaf region per data capture and cannot represent nutrient distribution across the leaf samples. Thus, an opportunity exists to devise a portable device that can monitor health conditions of crop plants comprehensively and nondestructively. The limiting factor to the most existing portable crop-monitoring devices is their ability to gather as much data per leaf as possible because of the inter-leaf nutrient distributions. For instance, the SPAD meter (Minolta Co., Ltd., Osaka, Japan) is one of the most widely utilized diagnostic tools to non-invasively measure leaf chlorophyll
⁎ Corresponding author at: Department of Agricultural and Biological Engineering, Purdue University, 225 South University Street, West Lafayette, IN 47907, United States. E-mail address: [email protected] (J. Jin).
https://doi.org/10.1016/j.compag.2019.105069 Received 10 April 2019; Received in revised form 18 October 2019; Accepted 21 October 2019 0168-1699/ © 2019 Published by Elsevier B.V.
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
the battery output and the anodes of LEDs to improve the stability of LEDs when they were working. And two commonly used 2N2222 NPN transistors were installed amid the cathodes of LEDs and the GPIOs of the Raspberry Pi to achieve the high-speed on and off of LEDs. In the Leaf Scanner device, the SMD white LED was approximately $ 0.20 each, and the SMD NIR LED was approximately $0.65 each. The Raspberry Pi Noir camera module V2 was $24.58, and the Raspberry Pi 3 model B motherboard was $34.49. The TalentCell rechargeable lithium ion battery was $24.99, and the DROK DC voltage regulator was $15.35. Including the expenses of almost all the components, the gross material cost of the Leaf Scanner was $130. The cost is much lower than the current instruments such as the Minolta SPAD-502Plus meter (approximately $2500). The cost can be lower if we optimize the design and configuration of the device in the future (Carbone et al. 2019). For instance, the Raspberry Pi Noir V1 camera with a lower price also meets our needs, and a more compact microcontroller saves both space and money.
contents by collecting transmittance intensity of visible and NIR light (Uddling et al., 2007). Its price is approximately $2500, and its measuring area is 2 mm × 3 mm. Ciganda et al. (2008) indicated that its readings from different spots on a same leaf could have up to 500% difference. The successful SPAD meter measurements are impacted by measurement positions on the leaf (Ata-Ul-Karim et al., 2014; Yang et al., 2014). These fluctuations can be explained by the distributions of chlorophyll across the leaves, which are not uniform and can be extremely differential in localized patches due to disease (Schepers et al., 1992; Chapman and Barreto, 1997; Ward et al., 2007; Lin et al., 2010). Besides the SPAD meter, the MultispeQ (Osei-Bonsu et al., 2016) and the LI-COR photosynthesis systems (LI-COR Biosciences, Lincoln, NE, USA) are also devices that have been used to detect chlorophyll levels. The MultispeQ uses different peak emission wavelengths of LEDs to achieve different measurements such as leaf chlorophyll content. Its measuring area is about 1 cm2 on the leaf surface. The LI-COR integrates LED light source to provide controllable environment surrounding a certain leaf area (2 cm2 for 6400-40 leaf chamber and 6 cm2 for 6400-02B leaf chamber). However, none of these devices collects the image of a whole leaf. They all measure only a small local spot on the leaf, and the measurement result is very vulnerable to the change of the measurement position on the leaf (Cao et al., 2016). Besides, one such spot does not well represent the whole leaf’s chlorophyll level which normally contains big variance. As a result, these devices above fall short of measuring chlorophyll distribution variances due to their limited measuring areas (Weber et al., 2012; Hooper et al., 2002). To more precisely and efficiently analyze the conditions of plant health and the distributions of chlorophyll contents of the whole leaves, we developed a portable Leaf Scanner. This device captured the transmittance radiation of visible and NIR LEDs through the leaves. It was compared against the SPAD meter in its ability to detect N contents of plants under different treatments, and tested as a method to compute leaf area of living plants.
2.2. Workflow In this publication, the controlling programs were written in Python. Before scanning, the Pi camera parameters such as camera sensor mode, resolution, framerate, ISO, exposure mode, shutter speed, exposure compensation, auto white balance mode and brightness needed to be initialized (Pereira et al., 2014). The visible LEDs were turned on first and one RGB image was captured. The NIR LEDs were then turned on and another NIR image was captured. To rapidly flash the visible and NIR LEDs and capture corresponding images, the video port of the Pi camera was activated, and the image was captured into a stream whose values were stored in an array. The arrays were finally saved as JPEG images to the local SD card. The images were then copied to the PC for further processing. The Leaf Scanner was capable of capturing images with a framerate of 12–15 fps, which meant that we were able to obtain at least 6 pairs of visible and NIR images per second. The entire workflow of the Leaf Scanner is described in Fig. 2.
2. Design of Leaf Scanner 2.1. Hardware configuration
3. Materials and methods The Leaf Scanner was designed in a way that a leaf could be pulled through a self-contained imaging chamber. The case of the device was designed using the Solidworks 2017 software (Dassault Systèmes SOLIDWORKS Corp., MA, USA) and printed using a MakerBot Replicator + 3D printer (MakerBot Industries, LLC, NY, USA) with black PLA filament materials. As illustrated in Fig. 1, the Leaf Scanner was mainly comprised of a lighting box, a Raspberry Pi camera, and a Raspberry pi microcontroller. The lighting box consisted of 24 SMD white LEDs (peaks at 450 and 600 nm, Cree Inc., NC, USA) and 24 SMD NIR LEDs (peak at 880 nm, Kingbright Co., CA, USA). The area of the lighting box was 12 cm (maximum leaf width) × 9 cm (scanning direction). The two categories of LEDs were uniformly distributed in the lighting box, and underneath them, a 1/8 in. thick Teflon panel (McMaster-Carr Supply Company, IL, USA) served as a light diffuser (Tomer et al., 2017; Noviyanto and Abdulla, 2019). The Raspberry Pi Noir camera module V2 (Raspberry Pi foundation, UK) was used to capture visible and NIR images. It was built on a high quality 8 megapixel Sony IMX219 image sensor, and there was no infrared filter on the lens. A piece of glass was installed above the camera lens to protect the imager and support the leaves when they were pulled through the sensor. The Raspberry Pi 3 model B motherboard (Raspberry Pi foundation, UK) was used as the microcontroller to rapidly flash the visible and NIR LEDs and control the Pi camera to capture images (Edwards, 2013). Both the LEDs and Raspberry Pi were powered by a TalentCell rechargeable lithium ion battery (12 V 3000mAh, TalentCell Technology Co., Ltd., Guangdong, China). A DROK DC voltage regulator (converting 3–34 V to 4–35 V, DROK, Guangdong, China) was installed amid
3.1. Plant samples The corn plants (hybrid B73 × Mo17 genotype) were grown in the Lilly greenhouses and plant growth facility at Purdue University (40°25′19.6″N, 86°55′7.8″W). The plants were grown in the mix soil of Profile® Greens Grade™ and Sungro Metro-Mix® 510, and divided into two N treatment plots: 200 ppm versus 25 ppm. All of the plants were watered as needed and fertilized weekly. The greenhouse temperature was maintained between 23.9 and 28.9 °C in the daytime and between 21.1 and 25.6 °C at night. The complementary light was provided with 600 W high pressure sodium bulbs set to a 14-h photoperiod. 3.2. Data acquisition Once the plants were at V6 or V7 stage (BBCH scale: 33 or 34), 19 of high N plants and 19 of low N plants were randomly selected as the experimental samples. For each plant, the top-collared leaf was scanned from the collar to the tip using this Leaf Scanner device. The entire scanning length was approximately 50–60 cm, and it was completed within approximately 50 s. The position difference between two next visible and NIR images was approximately 1.6 mm. In this publication, to simplify the leaf stitching, we assumed that there was no significant leaf position difference between each pair of visible and NIR images. Also, the Minolta SPAD-502Plus meter (Minolta Co., Ltd., Osaka, Japan) was utilized to measure SPAD values of 10 equidistant spots on the same side of the leaf midrib from the collar to the tip. The captured images were finally duplicated from the Raspberry Pi’s local SD card to 2
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Fig. 1. The configuration and leaf scanning prototype of the Leaf Scanner device. (a) Schematic diagram (1. light diffuser 2. glass 3. camera chamber 4. battery 5. scan button 6. pre-tightening spring 7. device holder 8. lighting chamber). (b) LED board. (c) Corn leaf scanning in the greenhouse.
Fig. 2. The entire workflow of the Leaf Scanner device. 3
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
PC, and the recorded SPAD data was written into an excel file.
based on the coordinates of the best pair of matched points (Fig. 3(a)). Generally, after the RANSAC examination, the homographic matrix would be calculated, and the geometric transformation would be estimated in terms of the matched points. However, it was not always robust due to the monotonous features on the leaf surface. Therefore, in this publication, we selected the best matching point pair, vertically cut the images and stitched them based on the coordinates of the best pair of matched points. The final Green NDVI mosaic results are demonstrated in Fig. 3(b) and (c). The Green NDVI panoramas enable us to further study the chlorophyll content distribution across the leaf surface and monitor the leaf area changes of plants under different treatments.
3.3. Leaf panorama stitching The Raspberry Pi Noir V2 camera required either high red light intensity or high exposure time to return visible images with qualified red bands. This could not be abided since it would sacrifice framerate, which needed to be guaranteed for continuous data collection. Therefore, in this publication, we computed the Green NDVI conforming the Eq. (1) instead of NDVI (Gitelson et al., 1996). As Daughtry et al. (2000) indicated, there was a strong correlation between normalized NIR/Green and NIR/Red (R2 > 0.95). And Patrick et al. (2017) suggested that the correlation between Green NDVI and manual assessment of wilt disease in peanuts (R2 = 0.81) was higher than that between NDVI and manual disease assessment (R2 = 0.71). In this publication, the Green NDVI image was derived from a region’s two visible and NIR images. The visible image was captured when the visible LEDs were on. And the NIR image was captured when the NIR LEDs were on. Both the visible and NIR images were RGB images because the RGB sensor without the NIR filter was used in the Leaf Scanner device.
Green NDVI =
NIR − Green NIR + Green
3.4. Data analysis Firstly, for each plant, we averaged the Green NDVI panorama and the SPAD measurements of the top-collared leaf, and conducted the linear regression analysis to examine the correlation between the averaged Green NDVI and SPAD values. Secondly, we compared the Green NDVI distribution with the SPAD measurements distribution along the leaf length direction. Thirdly, from the Green NDVI panorama, we extracted the leaf area and applied the paired t-test to compare the differences of leaf areas between high and low N treatments plants.
(1)
where NIR denotes the red band from the NIR image, and Green denotes the green band from the visible image. The Green NDVI images of leaf regions were stitched to a ‘leaf panorama’ using an algorithm written in the Matlab R2016a software (The MathWorks Inc., MA, USA). Firstly, between two next Green NDVI images, the corner features (Rockett, 2003) using minimum eigenvalue algorithm were detected, and interest points were extracted. Secondly, the matching points were determined based on the pairwise distances between detected features (Lowe, 2004). Thirdly, the random sample consensus (RANSAC) method was utilized to remove the outliers of the matched points (Tarsha-Kurdi et al., 2007). Fourthly, the inliers of the matched points were sorted based on their pairwise distances to pick out the best pair of matched points. Also, the matched points were restrained by their relative positions on the two next Green NDVI images. The leaf was moving from right to left, and this matched point on the latter image was slightly forward than that on the former image. Finally, the two Green NDVI images were vertically cut and stitched
4. Results 4.1. Averaged Green NDVI and SPAD values As illustrated in Fig. 4, both the averaged SPAD and the Green NDVI values from the high N treatment plants demonstrated higher values than those from the low N treatment plants. This difference was proved to be statistically significant through the paired t-test. The p-values of both the averaged SPAD values and the averaged Green NDVI values between different N treatments were less than 0.05 (5.36e−10 and 5.14e−08, respectively). As illustrated in Fig. 5, a linear regression model was fitted between the averaged Green NDVI and SPAD values. The result showed that the R-squared value was 0.92. Thus, there was a strong correlation between the averaged Green NDVI and SPAD measurements. This Leaf Scanner device was capable of estimating the chlorophyll contents of plants.
Fig. 3. Leaf Green NDVI panorama stitching. (a) Best pair of matched points and cutting lines. (b) Green NDVI panorama of the high N leaf. (c) Green NDVI panorama of the low N leaf. 4
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Fig. 4. Box plot of averaged SPAD and Green NDVI values under different N treatments. The horizontal lines inside the boxes are median lines. There are no units for both Green NDVI and SPAD values.
Fig. 5. The linear regression fit between the averaged Green NDVI and SPAD values. Fig. 7. Box plot of leaf areas in pixels of plants under different N treatments.
4.2. Distribution of Green NDVI and SPAD values
chlorophyll content first increased from the leaf collar to the tip, and then slightly dropped down near the leaf tip region. The continuous nature of the Green NDVI distribution permitted it to be fitted with second degree polynomials for both treatment groups. The R2 of the high and low N fit curves were 0.97 and 0.99, respectively. The fit curves of high N and low N had the same shape (both quadratic coef−b ficients equaled -2E-07). The axis of symmetry ( x = 2a ) of the high N
It is crucial to explore the chlorophyll content distribution across the leaf surface because of the non-uniform nutrient concentrations on the whole leaf. In this publication, we summarized the SPAD and Green NDVI value distributions from the leaf collar to the tip, which enabled us to seek out underlying trends. After averaging the distributions for plants in each treatment, we obtained two SPAD distribution curves for high and low N leaves, respectively (Fig. 6(a)). For each Green NDVI panorama, we averaged each pixel column (a sliver perpendicular to the vein of the leaf) to obtain the Green NDVI distribution along the leaf length direction. Through binning different leaf lengths to the same span, we also obtained two averaged Green NDVI distribution curves for high and low N leaves, respectively (Fig. 6(b)). Currently, all pixels in the column including midrib pixels are included. Removing midrib pixels can be done as another option for the future version. Both the SPAD value and Green NDVI distributions denoted that the
curve was approximately in the middle of the leaf ( 1250 ), whereas, the 1600 axis of symmetry of the low N curve was approximately at the three 1000 fourths position from the leaf collar to the tip ( ). The result was 1600 based on 38 corn plants, and it requires more tests to verify this conclusion. Therefore, this Leaf Scanner device enabled us to further study the chlorophyll content distributions on the plants under different treatments. It was difficult to achieve this goal using current portable instruments such as the SPAD meter due to their limited measure areas.
Fig. 6. The distribution of SPAD and Green NDVI values from the leaf collar to the tip. (a) 10 SPAD measurements on each leaf. (b) Green NDVI distribution curves are fitted with the second degree polynomials. 5
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Fig. 8. The leaf curling problem statement with a Green NDVI panorama.
Fig. 9. The Green NDVI panorama of a leaf cut off from a corn plant.
outliers. The curling situation might be worse when the plants are under water stress treatments. Therefore, to obtain the panoramas of complete leaves in more stressed conditions, the next version of the Leaf Scanner device will include improvements to mechanisms configuring the leaf into a spread position when scanning.
4.3. Comparison of leaf area The total leaf pixel amount was extracted from the Green NDVI panorama and regarded as the relative leaf area. To compare the difference of leaf areas of plants under different N treatments, we applied the paired t-test. As illustrated in Fig. 7, the p-value (0.0265) of total leaf pixels between different N treatments was less than 0.05. Therefore, there existed a significant difference between the leaf areas of high and low N treatments plants. In this publication, we were not able to collect ground truth measurements for the leaf areas. We will utilize the Li-Cor Leaf Area Meter (LI-COR Biosciences, Lincoln, NE, USA) or the CI-202 Laser Area Meter (CID Bio-Science, Inc., WA, USA) to collate the measurements of this Leaf Scanner device in the following tests (Cutini et al., 1998).
5.3.2. Stitching improvement It was difficult to avoid hand vibration and uneven speed when scanning the living plants, which might impact the leaf stitching quality. As exhibited in Fig. 9, a leaf was cut off and pulled through the imaging chamber with a uniform speed, and the leaf regions were stitched using the same algorithm. The stitching seams on this leaf panorama were almost invisible, and the stitching quality was improved. Thus, optimizing the stitching algorithm to fit the working environment is required in the development of next version device.
5. Discussion 6. Conclusions 5.1. Device advantages The existing portable instruments targeting at estimating the conditions of plant health non-destructively, generally, measure only one small leaf area at one measurement. They are not efficient to cover the entire leaf even the entire plant and estimate the chlorophyll content distribution. In this publication, we initiated the leaf scanning and stitching approaches with the low-cost components (gross cost was $130), and designed a Leaf Scanner device. This device utilized the index of Green NDVI to represent chlorophyll contents. This device manifested a strong correlation (R2 = 0.92) with the averaged measurements from the Minolta SPAD-502Plus meter. Furthermore, this Leaf Scanner device enabled us to fit polynomial regressions and study the chlorophyll content distributions on the plants under different treatments. It was difficult to achieve this goal using current portable instruments such as the SPAD meter due to their limited measure areas. In addition, this device exhibited that plants under high and low N treatments had a significant difference on the leaf areas (p-value < 0.05). For the next version of this Leaf Scanner device, we will raise image capturing framerate and increase the scanning speed by optimizing the configuration of LEDs.
In this publication, a portable and low-cost Leaf Scanner device was designed to measure the health conditions of plants in the greenhouse or field environments. The device captures the transmittance of visible and NIR light through the corn leaf and works in the same way as the SPAD meter to indicate the chlorophyll content of plants. The advantages of this Leaf Scanner device are: (1) manifesting the chlorophyll distribution across the leaf surface instead of a few small spots on the leaf, (2) comparing the leaf areas of plants under different treatments since it is able to collect the whole leaf information. 5.2. Device limitations However, there are some limitations about this low-cost device. The images captured by this device are 8-bit images. Compared with the widely used 12-bit or 16-bit sensors, this device has a limited ability to handle complicated ambient lighting conditions. In addition, the measurement result is largely impacted by the plant stage and the senescence process of the plant. The relationships between NDVI, chlorophyll and N content of plants vary with different crop genotypes, species and conditions, such as diseased or senescent plants. Thus, it is necessary to conduct more tests using this Leaf Scanner device and collect corresponding ground truth measurements to calibrate it.
Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
5.3. Opportunities for future work
Acknowledgements
5.3.1. Leaf curling Leaf curling sometimes led to leaf folding during scanning, occurring especially often at the leaf collars. The Green NDVI values were eccentrically high at these doubled areas, as marked by a red color ellipse in Fig. 8. In this publication, these high values were removed as
The authors are grateful to Shelby Gruss in the Department of Agronomy at Purdue University for her great work in cultivating the corn plants. And we also thank Ron Steiner, the coordinator in the Lilly 6
Computers and Electronics in Agriculture 167 (2019) 105069
L. Zhang, et al.
Greenhouses and Plant Growth Facility at Purdue University, for his help with well controlling the greenhouse environment.
conveyor system for indoor phenotyping. Comput. Electron. Agric. 166, 104979. https://doi.org/10.1016/j.compag.2019.104979. Mokhtarpour, H., Christopher, B.S., Saleh, G., Selamat, A.B., Asadi, M.E., Kamkar, B., 2010. Non-destructive estimation of maize leaf area, fresh weight, and dry weight using leaf length and leaf width. Commun. Biometry Crop Sci. 5 (1), 19–26. Noviyanto, A., Abdulla, W.H., 2019. Segmentation and calibration of hyperspectral imaging for honey analysis. Comput. Electron. Agric. 159, 129–139. https://doi.org/ 10.1016/j.compag.2019.02.006. Osei-Bonsu, I., Hoh, D., TerAvest, D., Kuhlgert, S., Austic, G., Weebadde, P., Kramer, D.M., 2016. MultispeQ Beta: a tool for large-scale plant phenotyping connected to the open PhotosynQ network. R. Soc. Open Sci. 3 (10), 160592. https://doi.org/10. 1098/rsos.160592. Patrick, A., Pelham, S., Culbreath, A., Holbrook, C.C., Godoy, I.J.D., Li, C., 2017. High throughput phenotyping of tomato spot wilt disease in peanuts using unmanned aerial systems and multispectral imaging. IEEE Instrum. Meas. Mag. 20 (3), 4–12. https://doi.org/10.1109/MIM.2017.7951684. Pereira, V., Fernandes, V.A., Sequeira, J., 2014. Low cost object sorting robotic arm using Raspberry Pi. In: 2014 IEEE Global Humanitarian Technology Conference-South Asia Satellite. GHTC-SAS 2014, pp. 1–6. Rockett, P.I., 2003. Performance assessment of feature detection algorithms: a methodology and case study on corner detectors. IEEE Trans. Image Process. 12 (12), 1668–1676. https://doi.org/10.1109/TIP.2003.818041. Schepers, J.S., Francis, D.D., Vigil, M., Below, F.E., 1992. Comparison of corn leaf nitrogen concentration and chlorophyll meter readings. Commun. Soil Sci. Plant Anal. 23 (17–20), 2173–2187. https://doi.org/10.1080/00103629209368733. Shepherd, M.A., Stockdale, E.A., Powlson, D.S., Jarvis, S.C., 1996. The influence of organic nitrogen mineralization on the management of agricultural systems in the UK. Soil Use Manage. 12 (2), 76–85. https://doi.org/10.1111/j.1475-2743.1996. tb00963.x. Tarsha-Kurdi, F., Landes, T., Grussenmeyer, P., 2007. Hough-transform and extended RANSAC algorithms for automatic detection of 3D building roof planes from Lidar data. In: ISPRS Workshop on Laser Scanning 2007 and SilviLaser 2007, XXXVI, pp. 407–412. Tester, M., Langridge, P., 2010. Breeding technologies to increase crop production in a changing world. Science 327 (5967), 818–822. https://doi.org/10.1126/science. 1183700. Tomer, N., McGlone, A., Künnemeyer, R., 2017. Validated simulations of diffuse optical transmission measurements on produce. Comput. Electron. Agric. 134, 94–101. https://doi.org/10.1016/j.compag.2017.01.006. Uddling, J., Gelang-Alfredsson, J., Piikki, K., Pleijel, H., 2007. Evaluating the relationship between leaf chlorophyll concentration and SPAD-502 chlorophyll meter readings. Photosynth. Res. 91 (1), 37–46. https://doi.org/10.1007/s11120-006-9077-5. Ward, J.M.J., Stromberg, E.L., Nowell, D.C., Nutter, F.W., 2007. Gray leaf spot: A disease of global importance in maize production. Plant Dis. 83 (10), 884–895. https://doi. org/10.1094/pdis.1999.83.10.884. Weber, V.S., Araus, J.L., Cairns, J.E., Sanchez, C., Melchinger, A.E., Orsini, E., 2012. Prediction of grain yield using reflectance spectra of canopy and leaves in maize plants grown under different water regimes. Field Crops Res. 128, 82–90. https://doi. org/10.1016/j.fcr.2011.12.016. Wilhelm, W.W., Ruwe, K., Schlemmer, M.R., 2000. Comparison of three leaf area index meters in a corn canopy. Crop Sci. 40, 1179–1183. https://doi.org/10.2135/ cropsci2000.4041179x. Xiong, X., Yu, L., Yang, W., Liu, M., Jiang, N., Wu, D., Liu, Q., 2017. A high-throughput stereo-imaging system for quantifying rape leaf traits during the seedling stage. Plant Methods 13 (1), 1–17. https://doi.org/10.1186/s13007-017-0157-7. Yang, H., Yang, J., Lv, Y., He, J., 2014. SPAD values and nitrogen nutrition index for the evaluation of rice nitrogen status. Plant Prod. Sci. 17 (1), 81–92. https://doi.org/10. 1626/pps.17.81. Yoder, B.J., Pettigrew-Crosby, R.E., 1995. Predicting nitrogen and chlorophyll content and concentrations from reflectance spectra (400–2500 nm) at leaf and canopy scales. Remote Sens. Environ. 53 (3), 199–211. https://doi.org/10.1016/00344257(95)00135-N. Zhang, L., Maki, H., Ma, D., Sánchez-Gallego, J.A., Mickelbart, M.V., Wang, L., Rehman, T.U., Jin, J., 2019. Optimized angles of the swing hyperspectral imaging system for single corn plant. Comput. Electron. Agric. 156, 349–359. https://doi.org/10.1016/j. compag.2018.11.030. Zhao, D., Reddy, K.R., Kakani, V.G., Read, J.J., Carter, G.A., 2003. Corn (Zea mays L.) growth, leaf pigment concentration, photosynthesis and leaf hyperspectral reflectance properties as affected by nitrogen supply. Plant Soil 257 (1), 205–217. https://doi.org/10.1023/A: 1026233732507.
Appendix A. Supplementary material Supplementary data to this article can be found online at https:// doi.org/10.1016/j.compag.2019.105069. References Ata-Ul-Karim, S.T., Zhu, Y., Yao, X., Cao, W., 2014. Determination of critical nitrogen dilution curve based on leaf area index in rice. Field Crops Res. 167, 76–85. https:// doi.org/10.1016/j.fcr.2014.07.010. Cao, W., Tian, Y., Cao, Q., Yuan, Z., Liu, X., Ata-Ul-Karim, S.T., Zhu, Y., 2016. Optimal Leaf Positions for SPAD Meter Measurement in Rice. Front. Plant Sci. 7, 1–10. https://doi.org/10.3389/fpls.2016.00719. Carbone, M.S., Seyednasrollah, B., Rademacher, T.T., Basler, D., Le Moine, J.M., Beals, S., Beasley, J., Greene, A., Kelroy, J., Richardson, A.D., 2019. Flux Puppy – an opensource software application and portable system design for low-cost manual measurements of CO2 and H2O fluxes. Agric. For. Meteorol. 274, 1–6. https://doi.org/10. 1016/j.agrformet.2019.04.012. Cavallo, D.P., Cefola, M., Pace, B., Logrieco, A.F., Attolico, G., 2017. Contactless and nondestructive chlorophyll content prediction by random forest regression: A case study on fresh-cut rocket leaves. Comput. Electron. Agric. 140, 303–310. https://doi.org/ 10.1016/j.compag.2017.06.012. Chapman, S.C., Barreto, H.J., 1997. Using a chlorophyll meter to estimate specific leaf nitrogen of tropical maize during vegetative growth. Agron. J. 89 (4), 557–562. https://doi.org/10.2134/agronj1997.00021962008900040004x. Ciganda, V., Gitelson, A., Schepers, J., 2008. Vertical profile and temporal variation of chlorophyll in maize canopy: Quantitative “crop vigor” indicator by means of reflectance-based techniques. Agron. J. 100 (5), 1409–1417. https://doi.org/10.2134/ agronj2007.0322. Cutini, A., Matteucci, G., Mugnozza, G.S., 1998. Estimation of leaf area index with the LiCor LAI 2000 in deciduous forests. For. Ecol. Manage. 105, 55–65. https://doi.org/ 10.1016/S0378-1127(97)00269-7. Daughtry, C.S.T., Walthall, C.L., Kim, M.S., Colstoun, E.B.D., McMurtrey, J.E., 2000. Estimating corn leaf chlorophyll concentration from leaf and canopy reflectance. Remote Sens. Environ. 74 (2), 229–239. https://doi.org/10.1016/S0034-4257(00) 00113-9. Edwards, C., 2013. Not-so-humble Raspberry Pi gets big ideas. Eng. Technol. 8 (3), 30–33. https://doi.org/10.1049/et.2013.0301. Fiorani, F., Schurr, U., 2013. Future Scenarios for Plant Phenotyping. Annu. Rev. Plant Biol. 64 (1), 267–291. https://doi.org/10.1146/annurev-arplant-050312-120137. Fystro, G., 2002. The prediction of C and N content and their potential mineralization in heterogeneous soil samples using Vis-NIR spectroscopy and comparative methods. Plant Soil 246 (2), 139–149. https://doi.org/10.1023/A:1020612319014. Gitelson, A.A., Kaufman, Y.J., Merzlyak, M.N., 1996. Use of a green channel in remote sensing of global vegetation from EOS-MODIS. Remote Sens. Environ. 58 (3), 289–298. https://doi.org/10.1016/S0034-4257(96)00072-7. Hooper, D.U., Cardon, Z.G., Chapin, F.S., Durant, M., 2002. Corrected calculations for soil and ecosystem measurements of CO2 flux using the LI-COR 6200 portable photosynthesis system. Oecologia 132 (1), 1–11. https://doi.org/10.1007/s00442-0020870-3. Jackson, R.D., Slater, P.N., Pinter, P.J., 1983. Discrimination of growth and water stress in wheat by various vegetation indices through clear and turbid atmospheres. Remote Sens. Environ. 13 (3), 187–208. https://doi.org/10.1016/0034-4257(83)90039-1. Lin, F., Qiu, L., Deng, J., Shi, Y., Chen, L., Wang, K., 2010. Investigation of SPAD meterbased indices for estimating rice nitrogen status. Comput. Electron. Agric. 71 (SUPPL. 1), 60–65. https://doi.org/10.1016/j.compag.2009.09.006. Lowe, D.G., 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60 (2), 91–110. https://doi.org/10.1023/B:VISI.0000029664. 99615.94. Ma, D., Carpenter, N., Maki, H., Rehman, T.U., Tuinstra, M.R., Jin, J., 2019a. Greenhouse environment modeling and simulation for microclimate control. Comput. Electron. Agric. 162, 134–142. https://doi.org/10.1016/j.compag.2019.04.013. Ma, D., Carpenter, N., Amatya, S., Maki, H., Wang, L., Zhang, L., Neeno, S., Tuinstra, M.R., Jin, J., 2019b. Removal of greenhouse microclimate heterogeneity with
7