Crop Status Sensing Based on Machine Vision for Precision Farming

Crop Status Sensing Based on Machine Vision for Precision Farming

CROP STATUS SENSING BASED ON MACHINE VISION FOR PRECISION FARMING Noburo Noguchi*, John F. Reid**, Kazunobu Ishii***, and Hideo Terao**** *Contact a...

465KB Sizes 0 Downloads 37 Views

CROP STATUS SENSING BASED ON MACHINE VISION FOR PRECISION FARMING

Noburo Noguchi*, John F. Reid**, Kazunobu Ishii***, and Hideo Terao****

*Contact author: Associate Professor, Bio-Production Engineering, Graduate School of Agriculture, Hokkaido University, Sapporo 060-8589, JAPAN **Manager, Technical Center, Deere and Companies, Moline, IL 61265, USA ***Assistant Professor, Bio-Production Engineering, Graduate School ofAgriculture, Hokkaido University, Sapporo 060-8589, JAPAN *** *Professor, Bio-Production Engineering, Graduate School ofAgriculture, Hokkaido University, Sapporo 060-8589, JAPAN

Abstract: Sensors are an essential part of intelligent agricultural machinery. Machine vision, in particular, can supply information about current crop status, including maturity and weed infestations. The information gathered through machine vision and other sensors such as GPS can be used to create field management schedules for chemical application, cultivation and harvest. The purpose of the study is to develop an intelligent machine vision system for an agricultural mobile robot A mUlti-spectral imaging system was developed to remotely obtain crop status on a field. The developed multi-spectral imaging system (MSIS) consists of an imaging sensor, an ilumination sensor, a differential GPS, and a portable computer. The imaging sensor was a custom-developed 3-CCD camera, which contains t~lfee separate optical paths and CCD image plane. Special optical filters were installed over the sensors providing three video channels of Green (G), Red (R), and near infrared (NfR) . The field experiment of MSIS was conducted using cornfield. The? for estimation of the crop height for estimation of the SPAD value showed 0.92 by using both showed 0.73. And, the reflectance and leaf area infonnation. As a result, the developed vision system enables a robot to recognize the crop status and efficiently conduct field operation through the timely information. The outputs include crop stress maps with nitrogen deficiency indexes on the field which is available on building a database for precise field management.

r

Keywords; vision intelligence, remote sensing, GIS, machine vision, multi-spectral imaging system

is not the solution. A new mode of thought. a new agricultural technology is required for the future. Intelligent robotic tractors are one potential solution (Noguchi, et al., 1997, 1998). Sensors are an essential part of intelligent agricultural machinery. Machine VISIOn, in particular, can supply information about current crop status, including maturity and weed infestations. The infonnation gathered through machine vision and other sensors such as GPS can be used to create field management schedules for chemical application, cultivation and harvest. The purpose of Ule study is to develop an intelligent machine vision system for an agricultural mobiles robot as shown in Fig. 1 (Noguchi, et aI., 1999). The vision system developed is able to simutaneously detect crop

1. INTRODUCTION

Agriculture in developed countries after the Industrial Revolution has tended to favor increases in energy input through the use of larger tractors and increased chemical and fertilizer application. AltllOUgh this agricultural technology has negative societal and environmental implications, it has supported food for rapidly increasing human population. In western countries, "sustainable agriculture", was developed to reduce the environmental impact of production agriculture. A.i the same time. the global agricultural workforce continues to shrink; each worker is responsible for greater areas of land. Simply continuing the current trend toward larger and heavier equipment

34

U.S.A.). The imaging sensor was a customdeveloped 3-CCD camera, which contains three separate optical paths and CCD image plane. Special optical filters were installed over the sensors providing three video channels of Green (G), Red (R), and near infrared (NIR) shown in Fig. 2. These three channels have center wavelengths of 55Oom, 65Onm, and 800nm and bandwidth of approximately 1000m for each channel. In addition, the aperture of lens and CCD gain in each channel can be control1ed through RS232C to ensure a wide dynamic range of the sensor. Images were captured by an image sensor with a resolution of 640H X 480V at 8 bit/pixe!. A flame grabber (FlashBus MV, Integral Technologies, U.S.A.) was used to acquire images through a high speed PCI interface. On the other hands, an ambient illumination sensor was employed for compensating light power change. The ambient illumination sensor had three filters of G, R, and NIR, which are almost same bandwidth with those of MS IS. The MS IS system was attached on the front of the mobile robot. Because the mobile robot has an RTK-GPS and a Fiber optic gyroscope for navigation, the infonnation from the MSIS can be applied for the GIS mapping as well as for real-time fertilizer or chemical application. In test platform, the another machine vision (Kodak Megaplus 1.6i) was also installed on the system to investigate the hardware perfonnance of the MSIS. Two machine visions on the robot were set at- same field-ofview.

rows and gather field infonnation. In particular, nitrogen stress of the crop is important factor for crop growth and yield. Abmad, et al. (1999) and Iida, et af (2000) con finned that greelUless index could express nitrogen stress status of crops. In fact, a SPAD meter which was developed and commercialized by Minolta Co Ltd. detects a kind of Greenness Index (SPAD value) based on a ratio of transmittance of two LEDs (65Onm and 94Oom). A SPAD meter has been widely used as a sensor for crop nitrogen status. (Schepers, et al. 1992 and Kim, et al. 2000) But, since the SPAD meter can measure the nitrogen stress with pinching a leave by a detector that is contact-base method, it isn't useful for the sensor attached on the robot. The sensor for detecting the crop status must adopt non-contact base method. Besides, in respect of crop status parameters, the height is also important as the crop status index as well. Therefore, the intelligent vision system based on multi-spectral imaging system (MS IS) was developed for detecting nitrogen stress and crop height. The vision system enables a robot to recognize the crop status and efficiently conduct field operation through the timely infonnation. The outputs include crop stress maps with nitrogen deficiency indexes on the field.

2. MULTI-SPECRAL IMAGING SYSTEM (MSIS) 2.1 MSIS hardware The developed MSIS consists of an imaging sensor, an illumination sensor (SKRI850A, 4channel, Skye Instruments Ltd., U.K.), an RTKGPS (MS750, Trimble, U.S.A.), and a portable computer (Field Pac, Do1ch Computer Systems,

2.2 Image Processing One of the important issues of the image processing is to segment out various unnecessary

RISP _____ eco Sensor

Vision Intelligence for Precision Agriculture

~1lC1t.-·

l. .

V"'rl~.I • ..(Iri.

Il1IpWttm-1

VlaIon I.... . ..01 !:lOP iIlId_ .. • ReoognIdoiI • GtowIII 0I1:n!P&

.~ .~

. "...~

• YWd MI&Miion

. 'F rilZllr IIppilc::8ikiO

INFRARI!D

GREEN

Fig.

1 Concept of VlSlon agricultural mobile robot.

intelligence

for

35

Fig. 2

Construction of MS IS.

regions in the image. For instance, soil and weed regions have to be segmented from the image before decision-making of the crop status. And, even after segmenting the image, regions of shadow and directly reflected from the crop leaves have to be considered on image processing. In the paper, the NIR image was utilized for segmenting the vegetation and soil because these regions can be easily segmented using an NIR image. Nitrogen stress of crop is usually detected by reflectance of green band from leaves because the chlorophyll is well correlated with nitrogen contained in leaves. On the other hand, the chlorophyll absorbs red band light. As described above, a SPAD meter uses red LED transmittance to detect the nitrogen stress of crop. Besides, the crop height can also tell the crop status, which is available for precise field management. Therefore, the goal of the crop parameters detected by the MSIS in the paper was to remotely estimate crop height, chlorophyll content and SPAD value. Because the MSIS can control the CCD gains as well as the aperture, the MS IS response relating to light reflectance can be calculated as follows,

i - responce

3. FIELD EXPERIMENT 3.1 Hardware performance based on outdoor environment To confirm validity of the MSIS concept, the precise experiment of MSIS was conducted in a field. Most important is whether the MSIS can precisely measure the chlorophyll like a SPAD meter. In this research, corn was chosen as a test crop and MSIS measured leaf samples with various SPAD values in outdoor environment. Chlorophyll obtained by chemical analyses was employed as reference data for the MS IS. Fig. 5 shows the experiment procedure for evaluating MSIS. A field-of-interest (FOl) completely corresponded with a leaf area for chemical analysis. (approximately 10 cm x 3 cm). In the experiment, MSIS and MegaPlus simultaneously obtained an image of same field-of-view (FOV), and the images were acquired under two different light conditions. Fig. 6 shows the accuracy of the MSIS estimating chlorophyll content in a leave. The value of (G-responseINIR-response) was adopted to estimate the chlorophyll content. High correlation between MSIS response and chlorophyll-a content, which was R2 "of 0.8246, was obtained in the experiment. From this result, it was confirmed that the MSIS could remotely detect the chlorophyll content. Moreover, high correlation between the SPAD value and chlorophyll-a content was already proved by many other researches. It was concluded that the MSIS could predict SPAD value as well as chlorophyll-a content.

re!. =__

1-

refNlR

re! I

XI = ----'-

EIA/I Hence, X is gray level. E is aperture, AI is output from the illumination sensor. i is G, R, NIR channel. As you can see, the AI compensates outdoor illumination change. MSIS response can be individually acquired for three channels; Red, Green and NIR. Basically, an estimator of the chlorophyll content and SPAD value using MSIS response was developed. The leaf area index was defined as ratio of leaf area to whole image area as shown in Fig. 4. The estimator for crop height was developed based on a defined leaf area index. The prior knowledge that was a leaf area had high correlation with leaf area in growing period, was utilized to develop the estimator.

3.2 GIS mapping by field experiment In above session, it was clear that the MSIS concept was valid to detect the" crop nitrogen status. The field experiment for the MSIS was conducted at a cornfield in Hokkaido University,

nlroi

NIR inage histogral'll

leaf area index (L,u) Leaf area LAI= Whole

Computer

11lumination Sensor

)(100

arM. c.pturo

&N O boordl

Fig. 4 Segmentation of vegetation and definition of leaf area index.

Fig. 3 Experiment platform.

36

geographic posItIOn was continuously updated from the GPS and associated with each image. Corn stresses detected by MSIS were clearly corresponded with SPAD map. Fig. 11 shows the estimation accuracy of the yield calculated from MSIS. The results were investigated about relationship between LAI and actual yield. Jf was 0.94. And, R.M.S. error was 0.47 kg. It was concluded that MSIS can provide the information of yield before the harvest.

100

,....., E

..3

+' boO

95

_

R2

=0.7275

90

..t:

'Qi

85

C.

80

:--

..t:

....0

0

~ • •

75 70

150

170

190

210

230

250

270

Leaf area index [%]

4. CONCLUSIONS

Visi on Intelligence for an agricultural mobile robot was presented. Since agriCUltural systems are dramatically affected by changing information from time-valiant environment. The robot must have a sensory device for collecting useful information from the field. Real-time MSIS response information can provide the crop health and growth to the robot and farmers. In next step, the decision-making by the robot itself regarding fertilizer treatment is required to real-time nitrogen application. The tendency of future technology will not be limited to robot area. The MSIS can contribute to Precision Farming and identification of crop growth model as complex system .

Fig. 9 Accuracy of estimating crop height by MSIS on the field .

Actual measured SPAD

..... ~

. . · ... 1; ,

",U . • • I

. ·,....-.. 1. ,

tt.a:1'! , • • · ....ct!

4 1.

-....

....a .... l"f

4.1"' I .L6Ii!:

LOoIiI·"" .... ,"

ACKNOWLEDGEMENT

· 1.41

i' , a •

. ..-

This research was supported by Japan Ministry of Agriculture, Forestry and Fisheries, and the equipment was provided by Case Corporation, V.S .. All of the mentioned supports and assistance are gratefully acknowledged.

t. ._.....'" .... ...... . . . . 4. 1.

...

Estimated SPAD by MSIS

Fig. 10 GIS map of estimated crop SPAD in the field. REFERENCE

15

5

Ahmad, I.S., Reid, J.F., Noguchi, N. and Hansen, A.C. : Nitrogen sensing for precision agriculture using chlorophyll maps, 1999, ASAE Paper 993035.

R2:O.94 R.M.S.=O.47(kg)

L-__ 10

~

12

____L -_ _ 14

~

_ _ _ _L -_ _

16

18

~

Iida, T. , Noguchi , N., Ishii, K., Terao, H.: Nitrogen Stress Sensing. System using Machine Vision for Precision Farming (part I), 2000, Journal of the Japanese Society of Agricultural Machinery, 62(2), 87-93.

_ __ _

20

LA! [%]

Kim, Y., Reid, J.F. , Zhang, Q., Hansen, A.C. : Infotoronic Decision-making for a Field Crop Sensing System in Precision Agriculture, 2000, Proceedings of IFAC BIO-Robotics Il, 289-294.

Fig.ll Corn yield estimation by LAI calculated from MSIS.

38

Noguchi, N., Ishii, K., and Terao, H.; Development of an Agricultural Mobile Robot using a GeomagneticDirection Sensor and Image Sensors, 1997, Journal of Agricultural Engineering Research, 67(1), l-

IS.

Noguchi, N., J.F. Reid, Q. Zhang, L. Tian and A.C. Hansen: Vision Intelligence for Mobile Agro-Robotic System, 1999, Journal of Robotics and Mechatronics, Vol.ll No.3, 193-199. Sheppers, J.S., Francis, 0.0., Vigil, M., Below, F.E.: Comparison of corn leaf nitrogen concentration and chlorophyll meter reading, 1992, Communications in Soil Science and Plant Analysis, 23(17-20), 2174-2187.

Noguchi, N., Reid, J.F.,Zhang, Q., and Tian, L.F.; Vision Intelligence for Precision Fanning using Fuzzy Logic Optimized Genetic Algorithm and Artificial Neural Network, 1998, ASAE Paper No.983034.

39