- Email: [email protected]
Factors affecting human hand grasp type in tomato fruit-picking: A statistical investigation for ergonomic development of harvesting robot
Computers and Electronics in Agriculture 157 (2019) 90–97
Contents lists available at ScienceDirect
Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag
Original papers
Factors affecting human hand grasp type in tomato fruit-picking: A statistical investigation for ergonomic development of harvesting robot
T
⁎
Zhiguo Lia,c, , Fengli Miaoa,b, Zhibo Yangb, Pengpeng Chaia,b, Shanju Yanga a
College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China School of Mechanical and Power Engineering, Henan Polytechnic University, Jiaozuo 454003, China c Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, China b
A R T I C LE I N FO
A B S T R A C T
Keywords: Tomato picking Grasp type Human body characteristics Fruit sizes Harvesting robot
The aim of this study is to investigate the factors that affect the human hand grasp type in tomato fruit picking. The correlation between the parameters of the growing environment, fruit size, human body characteristics and choice of grasp type was investigated using a multinomial logistic regression analysis. In tomato fruit-picking, Power palm-Thumb abduction was the most frequently used grasp type, followed by Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch. The plant posture and position of fruits in a tomato plant had no significant effect on the choice of the grasp type during tomato picking (p > 0.05). On the other hand, the major diameter of the fruit, right-hand length, forearm length, middle-finger length, and stature (participant height) had significant contributions on the choice of the grasp type (p < 0.05). With other independent variables unchanged, the probability that Power palm-Thumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Pinch when the major diameter of fruits were 1 mm longer; when both the stature and forearm length of participants were 1 cm longer, the corresponding probability that Power palmThumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Power palmThumb adduction, respectively; when both the hand length and middle-finger length of participants were 1 mm longer, the corresponding probability that Power palm-Thumb adduction and Pinch, respectively, were chosen for picking was higher than that of Power palm-Thumb abduction. This study provided an in-depth scientific guidance for dimensional synthesis of fruit-harvesting robots as well as in making grasp planning algorithms for intelligent picking of multi-finger end-effectors from the viewpoint of ergonomics.
1. Introduction Fresh tomato fruits are essential components of numerous human diets; consequently, worldwide, there is a great demand for their production (Li et al., 2017b; Li and Thomas, 2014). With the rapid increase in world population, the cultivation areas of tomatoes in greenhouses have been steadily increased in recent years (Li et al., 2017a; Li and Thomas, 2016). Hence, several researchers began to investigate how to harvest fresh tomato fruits using agricultural robots in greenhouses; subsequently, several tomato-harvesting robots were developed (Bloch et al., 2018; Feng et al., 2014; Liu et al., 2018; Wang et al., 2017; Yang et al., 2019). However, to date, performing a rapid, precise, and nondamage-causing harvesting of greenhouse-grown tomatoes remains a problem for these robots (Bac et al., 2017; Shamshiri et al., 2018; Silwal et al., 2017; Zhang et al., 2018). Generally, humans can precisely and efficiently pick different sizes of tomato fruits cultivated in an
⁎
unstructured greenhouse environment without causing damage to the produce through the coordination of brain, eyes, and hands. From the viewpoint of bionics, the working performance of agricultural harvesting robots can be improved by creating a bionic design that is based on the human body structure and hand-grasping techniques (Clement et al., 2011; Graichen et al., 2015). However, such would require an indepth understanding of human picking behaviours. Grasping is a highly complex movement that requires the coordination of several hand joints and muscles guided by the brain and vision (Hodson, 2018; Stern, 2015). Hence, grasping is crucial to the robot’s harvesting performance (e.g., efficient, precise, non-damaging) during picking (Hochberg et al., 2012; Li et al., 2013). Understanding how humans grasp tomato fruits, the awareness of influencing factors and limitations associated with each grasp, and knowing the type of grasp typically used in tomatopicking have a significant scientific value in ergonomic design as well as in creating effective grasping planning strategies for harvesting robot
Corresponding author at: College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China. E-mail address: [email protected] (Z. Li).
https://doi.org/10.1016/j.compag.2018.12.047 Received 18 October 2018; Received in revised form 16 December 2018; Accepted 23 December 2018 0168-1699/ © 2018 Elsevier B.V. All rights reserved.
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
fingers. Previous research on grasping can be classified into two aspects. One of these is related to the human grasp taxonomy. One of the earliest studies illustrated that the grasp of humans can be classified as power grasp and precision grasp (Napier, 1956). In considering the thumb motion, Cutkosky (1989) constructed a taxonomy of 16 grasp types used in manufacturing, such as large diameter wrap, adducted thumb wrap, and thumb-four finger pinch. Moreover, Feix et al. (2009) summarised the human grasp type into 33 categories based on the opposition type of fingers, palm and thumb, virtual finger assignments, and the position of the thumb. These were then simplified into 17 categories without considering the object shape and size. Bullock et al. (2013) added the non-prehensile grasp to the 33 grasp types listed by Feix et al. (2009) and found that the most commonly used grasp type in households and machine shops was the medium wrap. Vergara et al. (2014) proposed that the human grasp type in daily living activities can be divided into nine categories based on the type of interaction between the hand and object. Furthermore, they found that the most commonly used grasps are the pinch, non-prehensile grasp, cylindrical grasp, lateral pinch, and lumbrical grasp. Other factors that affect the human grasp type include object properties and human body characteristics. Lee and Jung (2014) proposed that object properties (such as shape, size, and direction) had a significant effect on the choice of grasp type. The pinching and grasping postures are commonly used to pick up angled and non-angled objects, respectively (Kyota and Saito, 2012). An object’s shape can delimit the types of hand postures that potentially come into contact with a surface, as well as restrict the orientation and potential contact location of the hand (Lee and Jung, 2014); the larger the object, the more fingers are required to stably grasp an object. Hence, Feix et al. (2014) proposed that the power grasp was generally applied to large and heavy objects, whereas the pinch was always applied to small and light objects. Wong and Whishaw (2004) proposed that gender and human body characteristics, such as finger size, had significant effects in choosing how many fingers are employed in the pinch; females generally use more fingers to grasp an object than males do, thereby resulting in different grasp type choices. Park et al. (2014) established a strong linear relationship between the ratio of object size to hand length and the six types of precision grasps. Pouydebat et al. (2011) proposed that children used the ‘thumb–finger pad’ less frequently and the ‘power grasp’ more often than adults do; adults exhibited a clear tendency to use the ‘thumb–finger pad’ to grasp large objects, whereas children used the ‘power grasp’ as often as the ‘thumb–finger pad’ grasp. In summary, significant progress has been achieved in comprehending the human grasp behaviour on objects that are encountered in daily living activities and manufacturing industry. However, information is limited regarding the correlations between object characteristics (such as inclination angle), human body characteristics (such as stature, shoulder tip height, hand length, and middle finger length), and the grasp type choice. Furthermore, previous studies have not been related to the development of tomato-picking robots yet; hence, inconsiderable information is available regarding the relationships between the growing environment parameters (such as fruit position and posture, and the geometric characteristics of fruits), human body characteristics (such as stature, forearm length, and hand size), and grasp type choice for tomato fruit-picking. Consequently, from the perspective of ergonomics, it is difficult to acquire in-depth scientific guidance for dimensional synthesis of fruit-harvesting robots as well as in making grasp planning algorithms for intelligent picking of end-effectors. Accordingly, the objective of this study is to investigate the factors that affect the choice of grasp type in tomato fruit-picking by employing a multinomial logistic regression analysis.
2. Materials and methods 2.1. Hand-picking tomato fruits The experiments were conducted in December 2017 at the Jiaozuo Manfeng Vegetable Planting Base. In one of the Helios greenhouses where Hg118 tomato fruits are grown, 12 columns of tomato plants were randomly chosen for this study. Thereafter, they were sorted into six groups. In each group, 40 tomato fruits of different sizes, postures, and positions were selected and numbered in order. Subsequently, six professional growers with different statures (heights) were recruited as participants. All selected participants were right-handed and had no history of illness or injury on their extremities. Each participant, as usual, picked 40 tomato fruits grouped by number naturally. In total, 240 tomato fruits (40 fruits per participant × 6 participants) in a greenhouse (temperature: 25 ± 2 °C; relative humidity: 70–75%) were hand-picked within 72 h during the test. To pick a tomato fruit, each participant was asked to stand and face the fruit. Thereafter, the participant grasps the fruit with the right hand, followed by the left hand to apply a plucking force to separate the fruit from the stem. These postharvest stem-loose tomato fruits can avoid puncture damage from stems when they are transported (Desmet et al., 2003). 2.2. Measurements of characteristics of the growing environment, fruit, and human body Before the tomato hand-picking tests, the body characteristics of the six participants were measured according to anthropometric standards (Claire et al., 2012; Frisancho, 2008). The stature (Hh), shoulder tip height (Hs), and the lengths of the upper arm (Lu) and forearm (Ll) were measured using a folding ruler to an accuracy of 1 mm (Fig. 1a). The upper arm is located between the shoulder and elbow joints, and the forearm is located between the elbow and wrist joints. Considering that the participants picked tomato fruits using right hand, the right hand of each participant was kept as a posture for taking a photo with a digital camera (Canon IXUS 95IS) from its top (Fig. 1b), the captured image was transmitted into the computer and then processed by the Digimizer Version 4.2.6.0 for extracting three dimensions: width Wh, hand length Lh, and middle finger length Lm with the assistance of a calibration line. The inclination angle, α, between the stem–blossom axis and the line perpendicular to the ground (Fig. 1c) was used to describe the posture of a tomato fruit in the three-dimensional greenhouse space; the height (Ht) of the fruit relative to the ground (Fig. 1d) was used to describe the position of the tomato fruit in the plant. The fruit inclination angles were measured using an electronic digital goniometer to an accuracy of 0.05°, and the fruit heights relative to the ground were measured using a folding ruler. The foregoing fruit inclination angle and height relative to the ground were included among the growing environment parameters of tomato fruits utilised in the regression analysis, which was later conducted. An electronic digital calliper was also used to measure three geometric parameters of fruits, such as height, H, and diameters D1 and D2 (Fig. 1c). The major diameter, D1, and minor diameter, D2, represented the lengths of the long and short axes, respectively, on the equatorial cross-section of the fruit. 2.3. Types of human hand grasps of tomato fruits The palm side hand surface was divided into 15 regions and numbered (Fig. 1b) accordingly. According to the grasp taxonomy proposed by Feix et al. (2016), the grasp types were defined as four types: Power palm-Thumb abduction, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch (Fig. 2a, b, c, d, respectively). In Power palm-Thumb abduction type, the fingers, palm (the 15th region), and thumb wrapped simultaneously around the tomato fruit, and the thumb generally opposed the other fingers. In Power palm-Thumb adduction type, the fingers and palm (the 15th region) wrapped around the tomato fruit, and the 91
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
Fig. 1. Characteristics of the growing environment, fruit, and human body. (a) human body characteristics: Hh – stature, Hs – shoulder tip height, Lu – upper arm length, Ll – forearm length; (b) hand size: Lh – hand length, Wh – hand width, Lm – middle finger length; (c) fruit posture and coordinate system: α – fruit inclination angle, H – fruit height, D1 – major diameter of fruit, D2 – minor diameter of fruit; (d) height Ht of fruit relative to the ground.
thumb was adducted on the side of the fingers. In Power pad-Thumb abduction type, the fingers and thumb wrapped around the tomato fruit, and there were at least two contact regions with each finger; the thumb opposed the other fingers and the tomato fruit did not directly come into contact with the palm (the 15th region). In Pinch type, the fingers and thumb squeezed the opposite sides of the tomato using fingertips
(e.g., the 1st, 4th, 7th, 10th, 13th region) without coming into contact with the palm. After each fruit was hand-picked off, the sequence numbers of the contact regions involved in grasping the tomato as well as the thumb posture were recorded and then used to identify its corresponding grasp type by a designated skilled observer who was trained before the test.
Fig. 2. Four grasping types. (a) Power palm-Thumb abduction type; (b) Power palm-Thumb adduction type; (c) Power pad-Thumb abduction type; (d) Pinch type. 92
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
2.4. Statistical analysis
P (Y = i|X ) ⎤ Gi (X ) = ln ⎡ = βi0 + ⎢ P (Y = 0|X ) ⎦ ⎥ ⎣
All statistical analyses were performed using the SPSS software, version 24.0 (IBM Corporation, New York, USA); the significance level was set to 0.05.
P (Y = m|X) =
2.4.1. Multinomial logistic regression The human hand grasp types of tomato fruits had four categories, which were nominal multiple-outcome response variables. A multinomial logistic regression analysis was deemed as an appropriate method to investigate the relationship between a dependent variable with more than two categories and a set of independent variables (including categorical variables and/or continuous variables). Therefore, this method was adapted to investigate the factors that affected the human hand grasp types in tomato-picking. During modelling, the grasp type was set as the dependent variable, Y, with four outcome categories, namely, Power palm-Thumb abduction, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, coded as 0, 1, 2, and 3, respectively. The parameters of the growing environment (inclination angle, α, and height, Ht, of the fruit relative to the ground), the fruit characteristics (height, H, major diameter, D1, and minor diameter, D2), and the human body characteristics (stature or height, Hh, shoulder tip height, Hs, upper arm length, Lu, forearm length, Ll, right hand length, Lh, right hand middle finger length, Lm, and right hand width, Wh) were considered as a possible set of independent variables. Using the multinomial logistic regression module in the SPSS software, Power palm-Thumb abduction was selected as the reference category with which the other three grasp types would be compared. All independent variables were selected as covariates. Subsequently, the multinomial logistic regression procedure automatically selected (through a series of distinct steps) the best independent variable predictors to be included in the model. The forward entry method through the likelihood ratio tests at a significance level of α = 0.05 was used for this selection. Lastly, a four-category logistic regression model of the main effects was fitted through the multinomial logistic regression analysis. The basic four-outcome category model with j independent variables, denoted by vector X, can be described by Eq. (1). There were three logit functions—GB(X), GC(X), and GD(X), which corresponded to grasp types: Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, respectively; Y = 0 was used as the referent category corresponding to grasp type: Power palm-Thumb abduction. The conditional probability P(Y = m|X) of each category in the four-outcome category model are expressed by Eq. (2).
t
∑ βij xj j=1
(1)
eGm (X ) 3
∑k = 0 eGk (X )
(2)
where Gi(X) is the ith logit function corresponding to grasp type i, namely grasp types Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch coded as 1, 2, and 3, respectively; P(Y = m|X) is the conditional probability of the mth grasp category, namely grasp types Power palm-Thumb abduction, Power palm-Thumb adduction, Power padThumb abduction and Pinch, coded by 0, 1, 2, and 3, respectively; X is a term that indicates all independent covariates; Vector β00 = 0 and G0(X) = 0.
2.4.2. Diagnostics of the fitted model Variable selection: By the forward entry method, a likelihood ratio χ2 statistic was used to select some independent variable predictors that contributed significantly to the model. At each step, an independent variable whose addition caused the largest statistically significant change in the −2Log likelihood was added to the model. Model fitting information: After the model was built, a likelihood ratio χ2 statistic was used to test the overall significance of the multinomial logistic regression model. Nagelkerke’s R2 statistic in the pseudo R2 statistics was used to summarise the proportion of variance in the dependent variable associated with the predictor (independent) variables. In the multinomial logistic regression model, a larger R2 value (with a maximum of 1) indicates that the considerable number of variations is explained by the model (Hosmer and Lemeshow, 2000). The classification table was employed to determine how well the model identified the choice of human hand grasp type during picking. The partial regression coefficients with significant negative values decreased the probability of the corresponding grasp type with respect to reference grasp type: Power palm-Thumb abduction. On the other hand, the partial regression coefficients with significant positive values increased the probability of the corresponding response grasp type (Hosmer and Lemeshow, 2000). The Wald test was used to verify the significance of individual parameters in the multinomial logistic regression model (Jia and Du, 2010). The odds ratio reported the change in the probability of the response grasp type versus the reference category (grasp type: Power palm-Thumb abduction) for a unit change in one independent covariate.
Fig. 3. Development and diagnostics of multinomial logistic regression model. (a) stepwise multinomial logistic regression, m-major diameter of tomato fruit, h-hand length, l-forearm length, f-middle-finger length, s-stature; (b) proportion of each grasp type accounting for the total number of grasps in the experiment, Type APower palm-Thumb abduction, Type B-Power palm-Thumb adduction, Type C-Power pad-Thumb abduction, Type D-Pinch; (c) prediction results of the final multinomial logistic regression model, Type A-Power palm-Thumb abduction, Type B-Power palm-Thumb adduction, Type C-Power pad-Thumb abduction, Type DPinch. 93
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
independent variables; parameter estimates are summarised in Table 1. The likelihood ratio χ2 test led to the conclusion that, overall, the final model was significant (p < 0.05). The frequencies of grasp types Power palm-Thumb abduction, Power palm-Thumb adduction, Power padThumb abduction and Pinch used in the picking experiments were 102, 58, 44, and 36, respectively, which accounted for 43, 24, 18, and 15%, respectively, of the total number of grasps (Fig. 3b). This showed that Power palm-Thumb abduction was most frequently used in tomato fruitpicking, followed by grasp types Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch. The likelihood ratio tests confirmed that the major diameter (D1) of the tomato fruit, the picker’s right-hand length (Lh), forearm length (Ll), middle-finger length, (Lm) and stature (Hh) had significant contributions on the choice of grasp type during tomato fruit-picking (p < 0.05). On the other hand, the inclination angle (α), height of the fruit relative to the ground (Ht), height (H) and minor diameter (D2) of the fruit, shoulder tip height (Hs) of the participant, upper arm length (Lu), and right-hand width (Wh) had no significant effect on the grasp type choice during tomato fruit-picking (p > 0.05). Nagelkerke’s R2 statistic was 0.6, which indicated that 60% of dependent variable data can be predicted by this multinomial logistic regression model. Fig. 3c shows the total number of observations of each grasp type used in the experiment and the number of correct predictions of each grasp type in model 5. Of these, 72 of the 102 observations of Power palm-Thumb abduction were predicted correctly; 41 of the 58 observations of Power palm-Thumb adduction were predicted correctly; 12 of the 44 observations of Power pad-Thumb abduction were predicted correctly; 21 of the 36 observations of Pinch were predicted correctly. Overall, 60.8% of the grasp types used in the experiment was predicted correctly by the final multinomial logistic regression model.
Table 1 Summary of the multinomial logistic regression model for separate grasp type. Explanatory variable
β
Wald
Sig.
Exp (β)
95% Confidence Interval for Exp (β) Lower Bound
Upper Bound
G1(X) = ln (P(Power palm-Thumb adduction |X)/P(Power palm-Thumb abduction |X)) Intercept −8.052 2.588 0.110 — — — Major diameter −0.013 0.548 0.459 0.987 0.952 1.022 Hand length 0.320 10.545 0.001 1.377 1.135 1.670 Middle-finger −0.089 0.141 0.707 0.914 0.573 1.458 length Stature −0.108 0.415 0.520 0.898 0.646 1.247 Forearm length −1.045 14.975 0.000 0.352 0.207 0.597 G2(X) = ln (P(Power pad-Thumb abduction |X)/P(Power palm-Thumb abduction |X)) Intercept 33.310 12.448 0.000 — — — Major diameter −0.048 4.694 0.030 0.953 0.912 0.995 Hand length 0.061 0.793 0.373 1.063 0.930 1.214 Middle-finger 0.262 2.133 0.144 1.300 0.914 1.848 length Stature −0.320 6.935 0.008 0.726 0.573 0.922 Forearm length −0.369 0.863 0.353 0.691 0.317 1.506 G3(X) = ln (P(Pinch |X)/P(Power palm-Thumb abduction |X)) Intercept 18.853 4.078 0.043 — — Major diameter −0.161 30.590 0.000 0.851 0.804 Hand length −0.301 4.195 0.041 0.740 0.555 Middle-finger 0.662 3.884 0.049 1.938 1.004 length Stature 0.007 0.003 0.958 1.007 0.778 Forearm length −0.239 0.200 0.654 0.788 0.277
— 0.901 0.987 3.774 1.304 2.240
Note: All variables entered in the model are significant (α = 0.05); β: Parameter estimates, Exp(β) – Odds ratio estimates; Reference category is Power palmThumb abduction, coded as 0; Gi(X): ith logit function corresponding to grasp type i, namely, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, coded as 1, 2, and 3, respectively; P(Y = m|X): conditional probability of the mth grasp category, namely Power palm-Thumb abduction, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, coded as 0, 1, 2 and 3, respectively; X: term indicating all independent covariates.
3.2. Effect of growing environment parameters In the experiment, the frequency distribution of the inclination angle, α, of selected tomato fruits relative to a line perpendicular to the ground is presented in Fig. 4a. Considering that the tomato fruits were randomly selected before the experiment, it was evident that the inclination angles of most tomato fruits in the greenhouse were naturally within the range of 13–90°. Single fruits in a plant always had small inclination angles, whereas fruits in a cluster always had large inclination angles. This is because the limited spaces and short stems forced all the tomato fruits in a cluster to a radiative distribution at different inclination angles. When more than four fruits were in a cluster, at least one of the tomato fruits would have an inclination angle larger than 90°. Fig. 4b lists the frequency distribution of height, Ht, relative to the ground of the selected tomato fruits, based on a previous
3. Results and discussion 3.1. General multinomial logistic regression model In the process of stepwise multinomial logistic regression, several models were evaluated; their −2Log likelihood value is depicted in Fig. 3a. Each label in the horizontal axis shows the independent variables that are inputted to the model. Model 5 was the final multinomial logistic regression model, which only included five important
Fig. 4. Frequency distribution of inclination angle and relative height of fruits. (a) frequency distribution of inclination angle, α, of fruits relative to a line perpendicular to the ground; (b) frequency distribution of height, Ht, of fruits relative to the ground. 94
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
abduction and Pinch with respect to Power palm-Thumb abduction (p < 0.05). For the logit function, G2(X), the coefficient of the major diameter was negative; this illustrated that with other independent variables unchanged, the participants preferred to use Power palmThumb abduction rather than Power pad-Thumb abduction when a larger major diameter of tomato fruits had to be picked. When the major diameter of tomato fruits was 1 mm longer, the ratio of the probabilities of choosing Power pad-Thumb abduction to that of Power palm-Thumb abduction decreased by 4.7%. Similarly, for the logit function, G3(X), the coefficient of the major diameter was negative. This illustrated that the participants preferred to use Power palm-Thumb abduction rather than Pinch when a larger major diameter of tomato fruits had to be picked. When the major diameter of tomato fruits was 1 mm longer, the ratio of the probabilities of choosing Pinch to that of Power palm-Thumb abduction decreased by 14.9%. Power palm-Thumb abduction was most commonly used grasp type in picking large tomato fruits with an average major diameter (D1 ± standard error) of 79.8 ± 12.9 mm; Power pad-Thumb abduction and Pinch were most commonly used grasp type to pick small tomato fruits with an average major diameter of 71.7 ± 10.4 mm and 61.2 ± 11.1 mm, respectively. Similarly, Lee and Jung (2014), proposed that the five-finger power grasp was always used to grasp objects 8–12 cm in size, and the three-finger pinch was always used to grasp objects 1–4 cm in size. Feix et al. (2014) analysed the relationship between object size and grasp type using violin plots, and proposed that power grasps were generally applied to handle large and heavy objects, whereas precision grasps were always employed to handle small objects. This is because as the size of tomato fruits increased, their masses synchronously increased (Li et al., 2011). Accordingly, the participants will attempt to use the palm and more fingers to power grasp the fruit (Feix et al., 2014); the greater contact area and higher friction force between hand skin and fruit improved the grasping stability (Vatavu and Zaiti, 2013; Wong and Whishaw, 2004). Moreover, the large contact area reduces the chance of discomfort or pain that may result from high pressure or pinch (Seo and Armstrong, 2008). When the fruit is small, the participants preferred to grasp it with the distal phalanges of their fingers than with the palm; this allowed the picker to have a more sensitive tactile perception compared to that of the power grasp (Pouydebat et al., 2011).
statistical result of the growing distribution of tomato fruits in the experimental greenhouse. In the greenhouse, the height of tomato fruits relative to ground was in the range of 11–160 cm. Typically, there were only a few fruits located at the upper positions of tomato plants. Multinomial logistic regression analysis showed that the two growing environment parameters (inclination angle of fruits and their height relative to the ground) had no significant effect on the probability of the grasp type choice in tomato fruit-picking (p > 0.05). The inclination angle of fruits and their height relative to the ground reflected the posture and position of tomato fruits in a three-dimensional space. Hence, the regression results further illustrated that the posture and position of fruits in a tomato plant had no significant effect on the choice of grasping type. During manual harvesting, the participants stood at a good position in front of the fruit, and employed their most comfortable grasp type to pick the tomato fruit. Consequently, it was found that fruit postures had no significant effect on the choice of grasp type. Furthermore, the heights of most tomato fruits relative to the ground (Fig. 4b) were always lower than the shoulder height of participants (Fig. 6b). Therefore, they can adjust their arm positions or standing postures to match the fruit height and change their wrist postures for easy fruit-picking. Clearly, the fruit positions did not affect the choice of grasp type. This explanation was confirmed by Schot et al. (2010), who proposed that the azimuth and distance of a spherical object from a subject had a strong association with the grasp orientation, the elbow angle between the upper arm and forearm, and the shoulder elevation angle between the upper arm and vertical. The results were similar to the findings of Lee and Jung (2014), who illustrated that the association between grasp type and object (e.g., cylinders and square pillars) orientation was not significant; however, object orientation had a stronger influence on the wrist posture. 3.3. Effect of fruit geometric parameters The height, H, major diameter, D1, and minor diameter, D2, of the experimental tomato samples were in ranges of 32–88 mm, 42–120 mm, and 39–110 mm, respectively (Fig. 5). The geometric mean diameter, GMD, which was always used for characterising the size of irregular fruits, widely varied from 39 to 103 mm. This illustrated that the sampling was well-balanced and representative of the entire tomato population. Multinomial logistic regression analysis showed that the major diameter, D1, had a significant effect on the probability of grasp type choice (p < 0.05); however, the residual fruit parameters had no significant effect on this choice (p > 0.05). In the multinomial logistic regression model (Table 1), the major diameter, D1, had no significant effect on the probability of choosing Power palm-Thumb adduction with respect to Power palm-Thumb abduction (p > 0.05). On the other hand, the major diameter, D1, had a significant effect on the probability of the choice of Power pad-Thumb
3.4. Effect of human body parameters The stature, Hh, shoulder tip height, Hs, upper arm length, Lu, forearm length, Ll, hand length, Lh, hand width, Wh, and middle finger length, Lm, of the participants in the experiment ranged 156–178 cm, 130–149 cm, 22–32 cm, 22–27 cm, 172–197 mm, 70–94 mm, and 70–81 mm, respectively (Fig. 6). This illustrated the range of each independent variable in the multinomial logistic regression model (Table 1). The analysis showed that the stature, Hh, forearm length, Ll, hand length, Lh, and middle finger length, Lm, had a significant effect on the probability of the grasp type choice (p < 0.05); however, the residual human body parameters had no significant effect on this choice (p > 0.05). In the multinomial logistic regression model (Table 1), for the first logit function, G1(X), the hand length, Lh, and forearm length, Ll, had significant effects on the probability of choosing Power palm-Thumb adduction relative to Power palm-Thumb abduction (p < 0.05), whereas the residual body parameters had no significant effect on this probability (p > 0.05). The coefficient of the hand length was positive; this illustrated that the longer the hand, the more the participants preferred to use Power palm-Thumb adduction rather than Power palm-Thumb abduction to pick tomato fruits. When the hand length is 1 mm longer, the ratio of probabilities of choosing Power palm-Thumb adduction to Power palm-Thumb abduction for picking was increased by 37.7%. However, the effect of forearm length was the exact opposite of this case. Participants with a longer forearm length preferred to use Power palm-Thumb
Fig. 5. Geometric characteristics of tomato fruits. H-Height of tomato fruit, D1Major diameter of tomato fruit, D2-Minor diameter of tomato fruit, GMDGeometric mean dimeter, defined by (H × D1 × D2)1/3. 95
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
Fig. 6. Geometric characteristics of participants. (a) Geometric characteristics of human bodies: Hh – stature, Hs – shoulder tip height, Lu – upper arm length, Ll – forearm length; (b) Geometric characteristics of right hands: Lh – hand length, Wh – hand width, Lm – middle finger length.
easier to stably pinch the tomato fruit with no assistance from the palm. There is no information available in literature about the correlation between the human body parameters and grasp type choice during picking; hence, no comparison is possible.
abduction rather than Power palm-Thumb adduction. The ratio of probabilities of choosing Power palm-Thumb adduction to Power palm-Thumb abduction for picking decreased by 64.8% when the forearm length is 1 cm longer. For the second logit function, G2(X), the stature, Hh, had a significant effect on the probability of the choice of Power pad-Thumb abduction relative to Power palm-Thumb abduction (p < 0.05), whereas residual body parameters had no significant effect on this probability (p > 0.05). When the coefficient of stature was negative, it illustrated that with other independent variables unchanged, more participants preferred to use Power palm-Thumb abduction rather than Power padThumb abduction to pick tomato fruits. When the stature of participants is 1 cm longer, the ratio of the probabilities of choosing Power padThumb abduction to Power palm-Thumb abduction for picking decreased by 27.4%. For the third logit function, G3(X), the hand length, Lh, and middle finger length, Lm, had significant effects on the probability of choosing Pinch relative to Power palm-Thumb abduction (p < 0.05), whereas the residual body parameters had no significant effect on this probability (p > 0.05). When the coefficient of hand length was negative, it illustrated that participants with longer hands preferred to use Power palmThumb abduction rather than Pinch to pick tomato fruits. When the hand 1 mm longer, the ratio of probabilities of choosing Pinch to Power palmThumb abduction for picking decreased by 26%. However, the effect of the middle-finger length was the exact opposite of this case. Participants with longer middle fingers preferred to use Pinch rather than Power palm-Thumb abduction in tomato fruit-picking. The ratio of the probabilities of choosing Pinch to Power palm-Thumb abduction for picking increased by 93.8% when the middle finger is 1 mm longer. In the experiment, tall participants (stature average value ± standard error: 168.8 ± 6.7 cm) with long forearms (average value ± standard error: 25.4 ± 1.7 cm) preferred to use Power palmThumb abduction to pick tomato fruits. This was attributed to the visual estimation level of the location and size of the visible part of the fruit, which determined the grasp type choice. Most tomato fruits were inclined and partially occluded by leaves or other fruits. During picking, the different statures of participants had a differentiated estimation of the stable grasp points on the fruit surface based on the partial visible region when they looked at the tomato fruit. The higher the stature value (taller participants), the more the participants were able to obtain visual information on the fruit. Furthermore, a long forearm increased the working space of the hand. Hence, it was easier for participants to handle a tomato fruit by Power palm-Thumb abduction; otherwise, it was difficult to handle the fruit based on the information provided only by a small part of the visible region of the fruit surface. The participants with short hands (average value ± standard error: 178.1 ± 7.6 mm) and long middle fingers (average value ± standard error: 75.9 ± 2.9 mm) preferred to use Pinch for picking because a long middle finger made it
4. Conclusions In this study, the correlation between the parameters of the growing environment, fruit size, human body characteristics and human hand grasp type choice in tomato fruit-picking was investigated using multinomial logistic regression analysis. Power palm-Thumb abduction was most frequently used grasp type to pick tomato fruits, followed by Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch. The posture and position of fruits in the tomato plant had no significant effect on the grasp type choice (p > 0.05). The major diameter of the fruit, right-hand length, forearm length, middle-finger length, and stature had significant contributions on the grasp type choice for tomatopicking (p < 0.05). With other independent variables unchanged, the probability that Power palm-Thumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Pinch when the major diameter of fruits were 1 mm longer; when both the stature and forearm length of participants were 1 cm longer, the corresponding probability that Power palm-Thumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Power palmThumb adduction, respectively; when both the hand length and middlefinger length of participants were 1 mm longer, the corresponding probability that Power palm-Thumb adduction and Pinch, respectively, were chosen for picking was higher than that of Power palm-Thumb abduction. This study provided an in-depth scientific guidance for dimensional synthesis of fruit-harvesting robots as well as in making grasp planning algorithms for intelligent picking of multi-finger endeffectors from the viewpoint of ergonomics.
Acknowledgements This work was supported by a European Marie Curie International Incoming Fellowship (326847 and 912847), a European Marie Curie International Incoming Fellowship (Z111021801) and Shaanxi Project of Science and Technology Activities for Returning from Overseas (2018030).
Appendix A. Supplementary material Supplementary data to this article can be found online at https:// doi.org/10.1016/j.compag.2018.12.047. 96
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
References
Li, Z., Li, P., Liu, J., 2011. Physical and mechanical properties of tomato fruits as related to robot’s harvesting. J. Food Eng. 103, 170–178. Li, Z., Li, P., Yang, H., Wang, Y., 2013. Stability tests of two-finger tomato grasping for harvesting robots. Biosyst. Eng. 116, 163–170. Li, Z., Miao, F., Andrews, J., 2017b. Mechanical models of compression and impact on fresh fruits. Compr. Rev. Food Sci. Food Saf. 16, 1296–1312. Li, Z., Thomas, C., 2014. Quantitative evaluation of mechanical damage to fresh fruits. Trends Food Sci. Technol. 35, 138–150. Li, Z., Thomas, C., 2016. Multiscale biomechanics of tomato fruits: a review. Crit. Rev. Food Sci. Nutr. 56, 1222–1230. Liu, J., Li, Z., Li, P., 2018. Rapid and Non-Destructive Manipulation Study of Tomato Harvesting Robot. Science Press, Beijing. Napier, J.R., 1956. The prehensile movements of human hand. J. Bone Joint Surg. 38, 902–913. Park, J., Seo, N.J., Son, J., Kim, W., Cheong, J., 2014. Postural variation of hand precision grips by object size. J. Mech. Sci. Technol. 28, 1641–1651. Pouydebat, E., Reghem, E., Borel, A., Gorce, P., 2011. Diversity of grip in adults and young humans and chimpanzees (pan troglodytes). Behav. Brain Res. 218, 21–28. Schot, W.D., Brenner, E., Smeets, J.B.J., 2010. Posture of the arm when grasping spheres to place them elsewhere. Exp. Brain Res. 204, 163–171. Seo, N.J., Armstrong, T.J., 2008. Investigation of grip force, normal force, contact area, hand size, and handle size for cylindrical handles. Hum. Factors 50, 734–744. Shamshiri, R.R., Weltzien, C., Hameed, I.A., Yule, I.J., Grift, T.E., Balasundram, S.K., Pitonakova, L., Ahmad, D., Chowdhary, G., 2018. Research and development in agricultural robotics: a perspective of digital farming. Int. J. Agric. Biol. Eng. 11, 1–14. Silwal, A., Davidson, J.R., Karkee, M., Mo, C., Zhang, Q., Lewis, K., 2017. Design, integration, and field evaluation of a robotic apple harvester. J. Field Rob. 34, 1140–1159. Stern, E., 2015. Embodied cognition: a grasp on human thinking. Nature 524, 158–159. Vatavu, R.D., Zaiti, I.A., 2013. Automatic recognition of object size and shape via userdependent measurements of the grasping hand. Int. J. Hum Comput Stud. 71, 590–607. Vergara, M., Sancho-Bru, J.L., Gracia-Ibanez, V., Perez-Gonzalez, A., 2014. An introductory study of common grasps used by adults during performance of activities of daily living. J. Hand Ther. 27, 225–234. Wang, L., Zhao, B., Fan, J., Hu, X., Wei, S., Li, Y., Zhou, Q., Wei, C., 2017. Development of a tomato harvesting robot used in greenhouse. Int. J. Agric. Biol. Eng. 10, 140–149. Wong, Y.J., Whishaw, I.Q., 2004. Precision grasps of children and young and old adults: individual differences in digit contact strategy, purchase pattern, and digit posture. Behav. Brain Res. 154, 113–123. Yang, S., Fu, B., Hou, Y., Chen, S., Li, Z., Wang, S., 2019. Transient cooling and operational performance of the cryogenic part in reverse brayton air refrigerator. Energy 167, 921–938. Zhang, B., Zhou, J., Meng, Y., Zhang, N., Gu, B., Yan, Z., Idris, S.I., 2018. Comparative study of mechanical damage caused by a two-finger tomato gripper with different robotic grasping patterns for harvesting robots. Biosyst. Eng. 171, 245–257.
Bac, C.W., Hemming, J., Tuijl, B.A.J., Barth, R., Wais, E., Henten, E.J., 2017. Performance evaluation of a harvesting robot for sweet pepper. J. Field Rob. 34, 1123–1139. Bloch, V., Degani, A., Bechar, A., 2018. A methodology of orchard architecture design for an optimal harvesting robot. Biosyst. Eng. 166, 126–137. Bullock, I.M., Zheng, J.Z., Rosa, S.D.L., Guertler, C., Dollar, A.M., 2013. Grasp frequency and usage in daily household and machine shop tasks. IEEE Trans. Haptic 6, 296–308. Claire, C.G., Cynthia, L.B., Bruce, B., Joseph, L.P., Patricia, B., Stephen, P.P., 2012. Anthropometric Survey of U.S. Army Personnel: Methods and Summary Statistics. U.S. Army Natick Soldier Research, Development and Engineering Center Natick, Massachusetts. Clement, R.G.E., Bugler, K.E., Oliver, C.W., 2011. Bionic prosthetic hands: a review of present technology and future aspirations. The Surgeon 9, 336–340. Cutkosky, M.R., 1989. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Trans. Robot. Automat. 5, 269–279. Desmet, M., Lammertyn, J., Scheerlinck, N., Verlinden, B.E., Nicolai, B.M., 2003. Determination of puncture injury susceptibility of tomatoes. Postharvest Biol. Technol. 27, 293–303. Feix, T., Bullock, I.M., Dollar, A.M., 2014. Analysis of human grasping behavior: object characteristics and grasp type. IEEE Trans. Haptic 7, 311–323. Feix, T., Pawlik, R., Schmiedmayer, H., Romero, J., Kragic, D., 2009. A comprehensive grasp taxonomy. In: RSS workshop on Understanding the Human Hand for Advancing Robotic Manipulation, Seattle, USA. Feix, T., Romero, J., Schmiedmayer, H., Dollar, A.M., Kragic, D., 2016. The grasp taxonomy of human grasp types. IEEE Trans. Hum.-Mach. Syst. 46, 66–77. Feng, Q., Cheng, W., Zhou, J., Wang, X., 2014. Design of structured-light vision system for tomato harvesting robot. Int. J. Agric. Biol. Eng. 7, 19–26. Frisancho, A.R., 2008. Anthropometric Standards, second ed. University of Michigan Press, Michigan, USA. Graichen, K., Hentzelt, S., Hildebrandt, A., Karcher, N., Gaibert, N., Knubben, E., 2015. Control design for a bionic kangaroo. Control Eng. Pract. 42, 106–117. Hochberg, L.R., Bacher, D., Jarosiewicz, B., Masse, N.Y., Simeral, J.D., Vogel, J., Haddadin, S., Liu, J., Cash, S.S., van der Smagt, P., Donoghue, J.P., 2012. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375. Hodson, R., 2018. A gripping problem. Nature 557, s23–s25. Hosmer, D., Lemeshow, S., 2000. Applied Logistic Regression, second ed. WileyInterscience Publication, New York. Jia, L., Du, Q., 2010. SPSS Statistical Analysis Standard Tutorial. Post & Telecom Press, Beijing. Kyota, F., Saito, S., 2012. Fast grasp synthesis for various shaped objects. Comput. Graphics Forum 31, 765–774. Lee, K.S., Jung, M.C., 2014. Common patterns of voluntary grasp types according to object shape, size, and direction. Int. J. Ind. Ergon. 44, 761–768. Li, Z., Andrews, J., Wang, Y., 2017a. Mathematical modelling of mechanical damage to tomato fruits. Postharvest Biol. Technol. 126, 50–56.
97
Contents lists available at ScienceDirect
Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag
Original papers
Factors affecting human hand grasp type in tomato fruit-picking: A statistical investigation for ergonomic development of harvesting robot
T
⁎
Zhiguo Lia,c, , Fengli Miaoa,b, Zhibo Yangb, Pengpeng Chaia,b, Shanju Yanga a
College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China School of Mechanical and Power Engineering, Henan Polytechnic University, Jiaozuo 454003, China c Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, China b
A R T I C LE I N FO
A B S T R A C T
Keywords: Tomato picking Grasp type Human body characteristics Fruit sizes Harvesting robot
The aim of this study is to investigate the factors that affect the human hand grasp type in tomato fruit picking. The correlation between the parameters of the growing environment, fruit size, human body characteristics and choice of grasp type was investigated using a multinomial logistic regression analysis. In tomato fruit-picking, Power palm-Thumb abduction was the most frequently used grasp type, followed by Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch. The plant posture and position of fruits in a tomato plant had no significant effect on the choice of the grasp type during tomato picking (p > 0.05). On the other hand, the major diameter of the fruit, right-hand length, forearm length, middle-finger length, and stature (participant height) had significant contributions on the choice of the grasp type (p < 0.05). With other independent variables unchanged, the probability that Power palm-Thumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Pinch when the major diameter of fruits were 1 mm longer; when both the stature and forearm length of participants were 1 cm longer, the corresponding probability that Power palmThumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Power palmThumb adduction, respectively; when both the hand length and middle-finger length of participants were 1 mm longer, the corresponding probability that Power palm-Thumb adduction and Pinch, respectively, were chosen for picking was higher than that of Power palm-Thumb abduction. This study provided an in-depth scientific guidance for dimensional synthesis of fruit-harvesting robots as well as in making grasp planning algorithms for intelligent picking of multi-finger end-effectors from the viewpoint of ergonomics.
1. Introduction Fresh tomato fruits are essential components of numerous human diets; consequently, worldwide, there is a great demand for their production (Li et al., 2017b; Li and Thomas, 2014). With the rapid increase in world population, the cultivation areas of tomatoes in greenhouses have been steadily increased in recent years (Li et al., 2017a; Li and Thomas, 2016). Hence, several researchers began to investigate how to harvest fresh tomato fruits using agricultural robots in greenhouses; subsequently, several tomato-harvesting robots were developed (Bloch et al., 2018; Feng et al., 2014; Liu et al., 2018; Wang et al., 2017; Yang et al., 2019). However, to date, performing a rapid, precise, and nondamage-causing harvesting of greenhouse-grown tomatoes remains a problem for these robots (Bac et al., 2017; Shamshiri et al., 2018; Silwal et al., 2017; Zhang et al., 2018). Generally, humans can precisely and efficiently pick different sizes of tomato fruits cultivated in an
⁎
unstructured greenhouse environment without causing damage to the produce through the coordination of brain, eyes, and hands. From the viewpoint of bionics, the working performance of agricultural harvesting robots can be improved by creating a bionic design that is based on the human body structure and hand-grasping techniques (Clement et al., 2011; Graichen et al., 2015). However, such would require an indepth understanding of human picking behaviours. Grasping is a highly complex movement that requires the coordination of several hand joints and muscles guided by the brain and vision (Hodson, 2018; Stern, 2015). Hence, grasping is crucial to the robot’s harvesting performance (e.g., efficient, precise, non-damaging) during picking (Hochberg et al., 2012; Li et al., 2013). Understanding how humans grasp tomato fruits, the awareness of influencing factors and limitations associated with each grasp, and knowing the type of grasp typically used in tomatopicking have a significant scientific value in ergonomic design as well as in creating effective grasping planning strategies for harvesting robot
Corresponding author at: College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China. E-mail address: [email protected] (Z. Li).
https://doi.org/10.1016/j.compag.2018.12.047 Received 18 October 2018; Received in revised form 16 December 2018; Accepted 23 December 2018 0168-1699/ © 2018 Elsevier B.V. All rights reserved.
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
fingers. Previous research on grasping can be classified into two aspects. One of these is related to the human grasp taxonomy. One of the earliest studies illustrated that the grasp of humans can be classified as power grasp and precision grasp (Napier, 1956). In considering the thumb motion, Cutkosky (1989) constructed a taxonomy of 16 grasp types used in manufacturing, such as large diameter wrap, adducted thumb wrap, and thumb-four finger pinch. Moreover, Feix et al. (2009) summarised the human grasp type into 33 categories based on the opposition type of fingers, palm and thumb, virtual finger assignments, and the position of the thumb. These were then simplified into 17 categories without considering the object shape and size. Bullock et al. (2013) added the non-prehensile grasp to the 33 grasp types listed by Feix et al. (2009) and found that the most commonly used grasp type in households and machine shops was the medium wrap. Vergara et al. (2014) proposed that the human grasp type in daily living activities can be divided into nine categories based on the type of interaction between the hand and object. Furthermore, they found that the most commonly used grasps are the pinch, non-prehensile grasp, cylindrical grasp, lateral pinch, and lumbrical grasp. Other factors that affect the human grasp type include object properties and human body characteristics. Lee and Jung (2014) proposed that object properties (such as shape, size, and direction) had a significant effect on the choice of grasp type. The pinching and grasping postures are commonly used to pick up angled and non-angled objects, respectively (Kyota and Saito, 2012). An object’s shape can delimit the types of hand postures that potentially come into contact with a surface, as well as restrict the orientation and potential contact location of the hand (Lee and Jung, 2014); the larger the object, the more fingers are required to stably grasp an object. Hence, Feix et al. (2014) proposed that the power grasp was generally applied to large and heavy objects, whereas the pinch was always applied to small and light objects. Wong and Whishaw (2004) proposed that gender and human body characteristics, such as finger size, had significant effects in choosing how many fingers are employed in the pinch; females generally use more fingers to grasp an object than males do, thereby resulting in different grasp type choices. Park et al. (2014) established a strong linear relationship between the ratio of object size to hand length and the six types of precision grasps. Pouydebat et al. (2011) proposed that children used the ‘thumb–finger pad’ less frequently and the ‘power grasp’ more often than adults do; adults exhibited a clear tendency to use the ‘thumb–finger pad’ to grasp large objects, whereas children used the ‘power grasp’ as often as the ‘thumb–finger pad’ grasp. In summary, significant progress has been achieved in comprehending the human grasp behaviour on objects that are encountered in daily living activities and manufacturing industry. However, information is limited regarding the correlations between object characteristics (such as inclination angle), human body characteristics (such as stature, shoulder tip height, hand length, and middle finger length), and the grasp type choice. Furthermore, previous studies have not been related to the development of tomato-picking robots yet; hence, inconsiderable information is available regarding the relationships between the growing environment parameters (such as fruit position and posture, and the geometric characteristics of fruits), human body characteristics (such as stature, forearm length, and hand size), and grasp type choice for tomato fruit-picking. Consequently, from the perspective of ergonomics, it is difficult to acquire in-depth scientific guidance for dimensional synthesis of fruit-harvesting robots as well as in making grasp planning algorithms for intelligent picking of end-effectors. Accordingly, the objective of this study is to investigate the factors that affect the choice of grasp type in tomato fruit-picking by employing a multinomial logistic regression analysis.
2. Materials and methods 2.1. Hand-picking tomato fruits The experiments were conducted in December 2017 at the Jiaozuo Manfeng Vegetable Planting Base. In one of the Helios greenhouses where Hg118 tomato fruits are grown, 12 columns of tomato plants were randomly chosen for this study. Thereafter, they were sorted into six groups. In each group, 40 tomato fruits of different sizes, postures, and positions were selected and numbered in order. Subsequently, six professional growers with different statures (heights) were recruited as participants. All selected participants were right-handed and had no history of illness or injury on their extremities. Each participant, as usual, picked 40 tomato fruits grouped by number naturally. In total, 240 tomato fruits (40 fruits per participant × 6 participants) in a greenhouse (temperature: 25 ± 2 °C; relative humidity: 70–75%) were hand-picked within 72 h during the test. To pick a tomato fruit, each participant was asked to stand and face the fruit. Thereafter, the participant grasps the fruit with the right hand, followed by the left hand to apply a plucking force to separate the fruit from the stem. These postharvest stem-loose tomato fruits can avoid puncture damage from stems when they are transported (Desmet et al., 2003). 2.2. Measurements of characteristics of the growing environment, fruit, and human body Before the tomato hand-picking tests, the body characteristics of the six participants were measured according to anthropometric standards (Claire et al., 2012; Frisancho, 2008). The stature (Hh), shoulder tip height (Hs), and the lengths of the upper arm (Lu) and forearm (Ll) were measured using a folding ruler to an accuracy of 1 mm (Fig. 1a). The upper arm is located between the shoulder and elbow joints, and the forearm is located between the elbow and wrist joints. Considering that the participants picked tomato fruits using right hand, the right hand of each participant was kept as a posture for taking a photo with a digital camera (Canon IXUS 95IS) from its top (Fig. 1b), the captured image was transmitted into the computer and then processed by the Digimizer Version 4.2.6.0 for extracting three dimensions: width Wh, hand length Lh, and middle finger length Lm with the assistance of a calibration line. The inclination angle, α, between the stem–blossom axis and the line perpendicular to the ground (Fig. 1c) was used to describe the posture of a tomato fruit in the three-dimensional greenhouse space; the height (Ht) of the fruit relative to the ground (Fig. 1d) was used to describe the position of the tomato fruit in the plant. The fruit inclination angles were measured using an electronic digital goniometer to an accuracy of 0.05°, and the fruit heights relative to the ground were measured using a folding ruler. The foregoing fruit inclination angle and height relative to the ground were included among the growing environment parameters of tomato fruits utilised in the regression analysis, which was later conducted. An electronic digital calliper was also used to measure three geometric parameters of fruits, such as height, H, and diameters D1 and D2 (Fig. 1c). The major diameter, D1, and minor diameter, D2, represented the lengths of the long and short axes, respectively, on the equatorial cross-section of the fruit. 2.3. Types of human hand grasps of tomato fruits The palm side hand surface was divided into 15 regions and numbered (Fig. 1b) accordingly. According to the grasp taxonomy proposed by Feix et al. (2016), the grasp types were defined as four types: Power palm-Thumb abduction, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch (Fig. 2a, b, c, d, respectively). In Power palm-Thumb abduction type, the fingers, palm (the 15th region), and thumb wrapped simultaneously around the tomato fruit, and the thumb generally opposed the other fingers. In Power palm-Thumb adduction type, the fingers and palm (the 15th region) wrapped around the tomato fruit, and the 91
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
Fig. 1. Characteristics of the growing environment, fruit, and human body. (a) human body characteristics: Hh – stature, Hs – shoulder tip height, Lu – upper arm length, Ll – forearm length; (b) hand size: Lh – hand length, Wh – hand width, Lm – middle finger length; (c) fruit posture and coordinate system: α – fruit inclination angle, H – fruit height, D1 – major diameter of fruit, D2 – minor diameter of fruit; (d) height Ht of fruit relative to the ground.
thumb was adducted on the side of the fingers. In Power pad-Thumb abduction type, the fingers and thumb wrapped around the tomato fruit, and there were at least two contact regions with each finger; the thumb opposed the other fingers and the tomato fruit did not directly come into contact with the palm (the 15th region). In Pinch type, the fingers and thumb squeezed the opposite sides of the tomato using fingertips
(e.g., the 1st, 4th, 7th, 10th, 13th region) without coming into contact with the palm. After each fruit was hand-picked off, the sequence numbers of the contact regions involved in grasping the tomato as well as the thumb posture were recorded and then used to identify its corresponding grasp type by a designated skilled observer who was trained before the test.
Fig. 2. Four grasping types. (a) Power palm-Thumb abduction type; (b) Power palm-Thumb adduction type; (c) Power pad-Thumb abduction type; (d) Pinch type. 92
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
2.4. Statistical analysis
P (Y = i|X ) ⎤ Gi (X ) = ln ⎡ = βi0 + ⎢ P (Y = 0|X ) ⎦ ⎥ ⎣
All statistical analyses were performed using the SPSS software, version 24.0 (IBM Corporation, New York, USA); the significance level was set to 0.05.
P (Y = m|X) =
2.4.1. Multinomial logistic regression The human hand grasp types of tomato fruits had four categories, which were nominal multiple-outcome response variables. A multinomial logistic regression analysis was deemed as an appropriate method to investigate the relationship between a dependent variable with more than two categories and a set of independent variables (including categorical variables and/or continuous variables). Therefore, this method was adapted to investigate the factors that affected the human hand grasp types in tomato-picking. During modelling, the grasp type was set as the dependent variable, Y, with four outcome categories, namely, Power palm-Thumb abduction, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, coded as 0, 1, 2, and 3, respectively. The parameters of the growing environment (inclination angle, α, and height, Ht, of the fruit relative to the ground), the fruit characteristics (height, H, major diameter, D1, and minor diameter, D2), and the human body characteristics (stature or height, Hh, shoulder tip height, Hs, upper arm length, Lu, forearm length, Ll, right hand length, Lh, right hand middle finger length, Lm, and right hand width, Wh) were considered as a possible set of independent variables. Using the multinomial logistic regression module in the SPSS software, Power palm-Thumb abduction was selected as the reference category with which the other three grasp types would be compared. All independent variables were selected as covariates. Subsequently, the multinomial logistic regression procedure automatically selected (through a series of distinct steps) the best independent variable predictors to be included in the model. The forward entry method through the likelihood ratio tests at a significance level of α = 0.05 was used for this selection. Lastly, a four-category logistic regression model of the main effects was fitted through the multinomial logistic regression analysis. The basic four-outcome category model with j independent variables, denoted by vector X, can be described by Eq. (1). There were three logit functions—GB(X), GC(X), and GD(X), which corresponded to grasp types: Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, respectively; Y = 0 was used as the referent category corresponding to grasp type: Power palm-Thumb abduction. The conditional probability P(Y = m|X) of each category in the four-outcome category model are expressed by Eq. (2).
t
∑ βij xj j=1
(1)
eGm (X ) 3
∑k = 0 eGk (X )
(2)
where Gi(X) is the ith logit function corresponding to grasp type i, namely grasp types Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch coded as 1, 2, and 3, respectively; P(Y = m|X) is the conditional probability of the mth grasp category, namely grasp types Power palm-Thumb abduction, Power palm-Thumb adduction, Power padThumb abduction and Pinch, coded by 0, 1, 2, and 3, respectively; X is a term that indicates all independent covariates; Vector β00 = 0 and G0(X) = 0.
2.4.2. Diagnostics of the fitted model Variable selection: By the forward entry method, a likelihood ratio χ2 statistic was used to select some independent variable predictors that contributed significantly to the model. At each step, an independent variable whose addition caused the largest statistically significant change in the −2Log likelihood was added to the model. Model fitting information: After the model was built, a likelihood ratio χ2 statistic was used to test the overall significance of the multinomial logistic regression model. Nagelkerke’s R2 statistic in the pseudo R2 statistics was used to summarise the proportion of variance in the dependent variable associated with the predictor (independent) variables. In the multinomial logistic regression model, a larger R2 value (with a maximum of 1) indicates that the considerable number of variations is explained by the model (Hosmer and Lemeshow, 2000). The classification table was employed to determine how well the model identified the choice of human hand grasp type during picking. The partial regression coefficients with significant negative values decreased the probability of the corresponding grasp type with respect to reference grasp type: Power palm-Thumb abduction. On the other hand, the partial regression coefficients with significant positive values increased the probability of the corresponding response grasp type (Hosmer and Lemeshow, 2000). The Wald test was used to verify the significance of individual parameters in the multinomial logistic regression model (Jia and Du, 2010). The odds ratio reported the change in the probability of the response grasp type versus the reference category (grasp type: Power palm-Thumb abduction) for a unit change in one independent covariate.
Fig. 3. Development and diagnostics of multinomial logistic regression model. (a) stepwise multinomial logistic regression, m-major diameter of tomato fruit, h-hand length, l-forearm length, f-middle-finger length, s-stature; (b) proportion of each grasp type accounting for the total number of grasps in the experiment, Type APower palm-Thumb abduction, Type B-Power palm-Thumb adduction, Type C-Power pad-Thumb abduction, Type D-Pinch; (c) prediction results of the final multinomial logistic regression model, Type A-Power palm-Thumb abduction, Type B-Power palm-Thumb adduction, Type C-Power pad-Thumb abduction, Type DPinch. 93
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
independent variables; parameter estimates are summarised in Table 1. The likelihood ratio χ2 test led to the conclusion that, overall, the final model was significant (p < 0.05). The frequencies of grasp types Power palm-Thumb abduction, Power palm-Thumb adduction, Power padThumb abduction and Pinch used in the picking experiments were 102, 58, 44, and 36, respectively, which accounted for 43, 24, 18, and 15%, respectively, of the total number of grasps (Fig. 3b). This showed that Power palm-Thumb abduction was most frequently used in tomato fruitpicking, followed by grasp types Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch. The likelihood ratio tests confirmed that the major diameter (D1) of the tomato fruit, the picker’s right-hand length (Lh), forearm length (Ll), middle-finger length, (Lm) and stature (Hh) had significant contributions on the choice of grasp type during tomato fruit-picking (p < 0.05). On the other hand, the inclination angle (α), height of the fruit relative to the ground (Ht), height (H) and minor diameter (D2) of the fruit, shoulder tip height (Hs) of the participant, upper arm length (Lu), and right-hand width (Wh) had no significant effect on the grasp type choice during tomato fruit-picking (p > 0.05). Nagelkerke’s R2 statistic was 0.6, which indicated that 60% of dependent variable data can be predicted by this multinomial logistic regression model. Fig. 3c shows the total number of observations of each grasp type used in the experiment and the number of correct predictions of each grasp type in model 5. Of these, 72 of the 102 observations of Power palm-Thumb abduction were predicted correctly; 41 of the 58 observations of Power palm-Thumb adduction were predicted correctly; 12 of the 44 observations of Power pad-Thumb abduction were predicted correctly; 21 of the 36 observations of Pinch were predicted correctly. Overall, 60.8% of the grasp types used in the experiment was predicted correctly by the final multinomial logistic regression model.
Table 1 Summary of the multinomial logistic regression model for separate grasp type. Explanatory variable
β
Wald
Sig.
Exp (β)
95% Confidence Interval for Exp (β) Lower Bound
Upper Bound
G1(X) = ln (P(Power palm-Thumb adduction |X)/P(Power palm-Thumb abduction |X)) Intercept −8.052 2.588 0.110 — — — Major diameter −0.013 0.548 0.459 0.987 0.952 1.022 Hand length 0.320 10.545 0.001 1.377 1.135 1.670 Middle-finger −0.089 0.141 0.707 0.914 0.573 1.458 length Stature −0.108 0.415 0.520 0.898 0.646 1.247 Forearm length −1.045 14.975 0.000 0.352 0.207 0.597 G2(X) = ln (P(Power pad-Thumb abduction |X)/P(Power palm-Thumb abduction |X)) Intercept 33.310 12.448 0.000 — — — Major diameter −0.048 4.694 0.030 0.953 0.912 0.995 Hand length 0.061 0.793 0.373 1.063 0.930 1.214 Middle-finger 0.262 2.133 0.144 1.300 0.914 1.848 length Stature −0.320 6.935 0.008 0.726 0.573 0.922 Forearm length −0.369 0.863 0.353 0.691 0.317 1.506 G3(X) = ln (P(Pinch |X)/P(Power palm-Thumb abduction |X)) Intercept 18.853 4.078 0.043 — — Major diameter −0.161 30.590 0.000 0.851 0.804 Hand length −0.301 4.195 0.041 0.740 0.555 Middle-finger 0.662 3.884 0.049 1.938 1.004 length Stature 0.007 0.003 0.958 1.007 0.778 Forearm length −0.239 0.200 0.654 0.788 0.277
— 0.901 0.987 3.774 1.304 2.240
Note: All variables entered in the model are significant (α = 0.05); β: Parameter estimates, Exp(β) – Odds ratio estimates; Reference category is Power palmThumb abduction, coded as 0; Gi(X): ith logit function corresponding to grasp type i, namely, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, coded as 1, 2, and 3, respectively; P(Y = m|X): conditional probability of the mth grasp category, namely Power palm-Thumb abduction, Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch, coded as 0, 1, 2 and 3, respectively; X: term indicating all independent covariates.
3.2. Effect of growing environment parameters In the experiment, the frequency distribution of the inclination angle, α, of selected tomato fruits relative to a line perpendicular to the ground is presented in Fig. 4a. Considering that the tomato fruits were randomly selected before the experiment, it was evident that the inclination angles of most tomato fruits in the greenhouse were naturally within the range of 13–90°. Single fruits in a plant always had small inclination angles, whereas fruits in a cluster always had large inclination angles. This is because the limited spaces and short stems forced all the tomato fruits in a cluster to a radiative distribution at different inclination angles. When more than four fruits were in a cluster, at least one of the tomato fruits would have an inclination angle larger than 90°. Fig. 4b lists the frequency distribution of height, Ht, relative to the ground of the selected tomato fruits, based on a previous
3. Results and discussion 3.1. General multinomial logistic regression model In the process of stepwise multinomial logistic regression, several models were evaluated; their −2Log likelihood value is depicted in Fig. 3a. Each label in the horizontal axis shows the independent variables that are inputted to the model. Model 5 was the final multinomial logistic regression model, which only included five important
Fig. 4. Frequency distribution of inclination angle and relative height of fruits. (a) frequency distribution of inclination angle, α, of fruits relative to a line perpendicular to the ground; (b) frequency distribution of height, Ht, of fruits relative to the ground. 94
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
abduction and Pinch with respect to Power palm-Thumb abduction (p < 0.05). For the logit function, G2(X), the coefficient of the major diameter was negative; this illustrated that with other independent variables unchanged, the participants preferred to use Power palmThumb abduction rather than Power pad-Thumb abduction when a larger major diameter of tomato fruits had to be picked. When the major diameter of tomato fruits was 1 mm longer, the ratio of the probabilities of choosing Power pad-Thumb abduction to that of Power palm-Thumb abduction decreased by 4.7%. Similarly, for the logit function, G3(X), the coefficient of the major diameter was negative. This illustrated that the participants preferred to use Power palm-Thumb abduction rather than Pinch when a larger major diameter of tomato fruits had to be picked. When the major diameter of tomato fruits was 1 mm longer, the ratio of the probabilities of choosing Pinch to that of Power palm-Thumb abduction decreased by 14.9%. Power palm-Thumb abduction was most commonly used grasp type in picking large tomato fruits with an average major diameter (D1 ± standard error) of 79.8 ± 12.9 mm; Power pad-Thumb abduction and Pinch were most commonly used grasp type to pick small tomato fruits with an average major diameter of 71.7 ± 10.4 mm and 61.2 ± 11.1 mm, respectively. Similarly, Lee and Jung (2014), proposed that the five-finger power grasp was always used to grasp objects 8–12 cm in size, and the three-finger pinch was always used to grasp objects 1–4 cm in size. Feix et al. (2014) analysed the relationship between object size and grasp type using violin plots, and proposed that power grasps were generally applied to handle large and heavy objects, whereas precision grasps were always employed to handle small objects. This is because as the size of tomato fruits increased, their masses synchronously increased (Li et al., 2011). Accordingly, the participants will attempt to use the palm and more fingers to power grasp the fruit (Feix et al., 2014); the greater contact area and higher friction force between hand skin and fruit improved the grasping stability (Vatavu and Zaiti, 2013; Wong and Whishaw, 2004). Moreover, the large contact area reduces the chance of discomfort or pain that may result from high pressure or pinch (Seo and Armstrong, 2008). When the fruit is small, the participants preferred to grasp it with the distal phalanges of their fingers than with the palm; this allowed the picker to have a more sensitive tactile perception compared to that of the power grasp (Pouydebat et al., 2011).
statistical result of the growing distribution of tomato fruits in the experimental greenhouse. In the greenhouse, the height of tomato fruits relative to ground was in the range of 11–160 cm. Typically, there were only a few fruits located at the upper positions of tomato plants. Multinomial logistic regression analysis showed that the two growing environment parameters (inclination angle of fruits and their height relative to the ground) had no significant effect on the probability of the grasp type choice in tomato fruit-picking (p > 0.05). The inclination angle of fruits and their height relative to the ground reflected the posture and position of tomato fruits in a three-dimensional space. Hence, the regression results further illustrated that the posture and position of fruits in a tomato plant had no significant effect on the choice of grasping type. During manual harvesting, the participants stood at a good position in front of the fruit, and employed their most comfortable grasp type to pick the tomato fruit. Consequently, it was found that fruit postures had no significant effect on the choice of grasp type. Furthermore, the heights of most tomato fruits relative to the ground (Fig. 4b) were always lower than the shoulder height of participants (Fig. 6b). Therefore, they can adjust their arm positions or standing postures to match the fruit height and change their wrist postures for easy fruit-picking. Clearly, the fruit positions did not affect the choice of grasp type. This explanation was confirmed by Schot et al. (2010), who proposed that the azimuth and distance of a spherical object from a subject had a strong association with the grasp orientation, the elbow angle between the upper arm and forearm, and the shoulder elevation angle between the upper arm and vertical. The results were similar to the findings of Lee and Jung (2014), who illustrated that the association between grasp type and object (e.g., cylinders and square pillars) orientation was not significant; however, object orientation had a stronger influence on the wrist posture. 3.3. Effect of fruit geometric parameters The height, H, major diameter, D1, and minor diameter, D2, of the experimental tomato samples were in ranges of 32–88 mm, 42–120 mm, and 39–110 mm, respectively (Fig. 5). The geometric mean diameter, GMD, which was always used for characterising the size of irregular fruits, widely varied from 39 to 103 mm. This illustrated that the sampling was well-balanced and representative of the entire tomato population. Multinomial logistic regression analysis showed that the major diameter, D1, had a significant effect on the probability of grasp type choice (p < 0.05); however, the residual fruit parameters had no significant effect on this choice (p > 0.05). In the multinomial logistic regression model (Table 1), the major diameter, D1, had no significant effect on the probability of choosing Power palm-Thumb adduction with respect to Power palm-Thumb abduction (p > 0.05). On the other hand, the major diameter, D1, had a significant effect on the probability of the choice of Power pad-Thumb
3.4. Effect of human body parameters The stature, Hh, shoulder tip height, Hs, upper arm length, Lu, forearm length, Ll, hand length, Lh, hand width, Wh, and middle finger length, Lm, of the participants in the experiment ranged 156–178 cm, 130–149 cm, 22–32 cm, 22–27 cm, 172–197 mm, 70–94 mm, and 70–81 mm, respectively (Fig. 6). This illustrated the range of each independent variable in the multinomial logistic regression model (Table 1). The analysis showed that the stature, Hh, forearm length, Ll, hand length, Lh, and middle finger length, Lm, had a significant effect on the probability of the grasp type choice (p < 0.05); however, the residual human body parameters had no significant effect on this choice (p > 0.05). In the multinomial logistic regression model (Table 1), for the first logit function, G1(X), the hand length, Lh, and forearm length, Ll, had significant effects on the probability of choosing Power palm-Thumb adduction relative to Power palm-Thumb abduction (p < 0.05), whereas the residual body parameters had no significant effect on this probability (p > 0.05). The coefficient of the hand length was positive; this illustrated that the longer the hand, the more the participants preferred to use Power palm-Thumb adduction rather than Power palm-Thumb abduction to pick tomato fruits. When the hand length is 1 mm longer, the ratio of probabilities of choosing Power palm-Thumb adduction to Power palm-Thumb abduction for picking was increased by 37.7%. However, the effect of forearm length was the exact opposite of this case. Participants with a longer forearm length preferred to use Power palm-Thumb
Fig. 5. Geometric characteristics of tomato fruits. H-Height of tomato fruit, D1Major diameter of tomato fruit, D2-Minor diameter of tomato fruit, GMDGeometric mean dimeter, defined by (H × D1 × D2)1/3. 95
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
Fig. 6. Geometric characteristics of participants. (a) Geometric characteristics of human bodies: Hh – stature, Hs – shoulder tip height, Lu – upper arm length, Ll – forearm length; (b) Geometric characteristics of right hands: Lh – hand length, Wh – hand width, Lm – middle finger length.
easier to stably pinch the tomato fruit with no assistance from the palm. There is no information available in literature about the correlation between the human body parameters and grasp type choice during picking; hence, no comparison is possible.
abduction rather than Power palm-Thumb adduction. The ratio of probabilities of choosing Power palm-Thumb adduction to Power palm-Thumb abduction for picking decreased by 64.8% when the forearm length is 1 cm longer. For the second logit function, G2(X), the stature, Hh, had a significant effect on the probability of the choice of Power pad-Thumb abduction relative to Power palm-Thumb abduction (p < 0.05), whereas residual body parameters had no significant effect on this probability (p > 0.05). When the coefficient of stature was negative, it illustrated that with other independent variables unchanged, more participants preferred to use Power palm-Thumb abduction rather than Power padThumb abduction to pick tomato fruits. When the stature of participants is 1 cm longer, the ratio of the probabilities of choosing Power padThumb abduction to Power palm-Thumb abduction for picking decreased by 27.4%. For the third logit function, G3(X), the hand length, Lh, and middle finger length, Lm, had significant effects on the probability of choosing Pinch relative to Power palm-Thumb abduction (p < 0.05), whereas the residual body parameters had no significant effect on this probability (p > 0.05). When the coefficient of hand length was negative, it illustrated that participants with longer hands preferred to use Power palmThumb abduction rather than Pinch to pick tomato fruits. When the hand 1 mm longer, the ratio of probabilities of choosing Pinch to Power palmThumb abduction for picking decreased by 26%. However, the effect of the middle-finger length was the exact opposite of this case. Participants with longer middle fingers preferred to use Pinch rather than Power palm-Thumb abduction in tomato fruit-picking. The ratio of the probabilities of choosing Pinch to Power palm-Thumb abduction for picking increased by 93.8% when the middle finger is 1 mm longer. In the experiment, tall participants (stature average value ± standard error: 168.8 ± 6.7 cm) with long forearms (average value ± standard error: 25.4 ± 1.7 cm) preferred to use Power palmThumb abduction to pick tomato fruits. This was attributed to the visual estimation level of the location and size of the visible part of the fruit, which determined the grasp type choice. Most tomato fruits were inclined and partially occluded by leaves or other fruits. During picking, the different statures of participants had a differentiated estimation of the stable grasp points on the fruit surface based on the partial visible region when they looked at the tomato fruit. The higher the stature value (taller participants), the more the participants were able to obtain visual information on the fruit. Furthermore, a long forearm increased the working space of the hand. Hence, it was easier for participants to handle a tomato fruit by Power palm-Thumb abduction; otherwise, it was difficult to handle the fruit based on the information provided only by a small part of the visible region of the fruit surface. The participants with short hands (average value ± standard error: 178.1 ± 7.6 mm) and long middle fingers (average value ± standard error: 75.9 ± 2.9 mm) preferred to use Pinch for picking because a long middle finger made it
4. Conclusions In this study, the correlation between the parameters of the growing environment, fruit size, human body characteristics and human hand grasp type choice in tomato fruit-picking was investigated using multinomial logistic regression analysis. Power palm-Thumb abduction was most frequently used grasp type to pick tomato fruits, followed by Power palm-Thumb adduction, Power pad-Thumb abduction and Pinch. The posture and position of fruits in the tomato plant had no significant effect on the grasp type choice (p > 0.05). The major diameter of the fruit, right-hand length, forearm length, middle-finger length, and stature had significant contributions on the grasp type choice for tomatopicking (p < 0.05). With other independent variables unchanged, the probability that Power palm-Thumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Pinch when the major diameter of fruits were 1 mm longer; when both the stature and forearm length of participants were 1 cm longer, the corresponding probability that Power palm-Thumb abduction was chosen for picking was higher than those of Power pad-Thumb abduction and Power palmThumb adduction, respectively; when both the hand length and middlefinger length of participants were 1 mm longer, the corresponding probability that Power palm-Thumb adduction and Pinch, respectively, were chosen for picking was higher than that of Power palm-Thumb abduction. This study provided an in-depth scientific guidance for dimensional synthesis of fruit-harvesting robots as well as in making grasp planning algorithms for intelligent picking of multi-finger endeffectors from the viewpoint of ergonomics.
Acknowledgements This work was supported by a European Marie Curie International Incoming Fellowship (326847 and 912847), a European Marie Curie International Incoming Fellowship (Z111021801) and Shaanxi Project of Science and Technology Activities for Returning from Overseas (2018030).
Appendix A. Supplementary material Supplementary data to this article can be found online at https:// doi.org/10.1016/j.compag.2018.12.047. 96
Computers and Electronics in Agriculture 157 (2019) 90–97
Z. Li et al.
References
Li, Z., Li, P., Liu, J., 2011. Physical and mechanical properties of tomato fruits as related to robot’s harvesting. J. Food Eng. 103, 170–178. Li, Z., Li, P., Yang, H., Wang, Y., 2013. Stability tests of two-finger tomato grasping for harvesting robots. Biosyst. Eng. 116, 163–170. Li, Z., Miao, F., Andrews, J., 2017b. Mechanical models of compression and impact on fresh fruits. Compr. Rev. Food Sci. Food Saf. 16, 1296–1312. Li, Z., Thomas, C., 2014. Quantitative evaluation of mechanical damage to fresh fruits. Trends Food Sci. Technol. 35, 138–150. Li, Z., Thomas, C., 2016. Multiscale biomechanics of tomato fruits: a review. Crit. Rev. Food Sci. Nutr. 56, 1222–1230. Liu, J., Li, Z., Li, P., 2018. Rapid and Non-Destructive Manipulation Study of Tomato Harvesting Robot. Science Press, Beijing. Napier, J.R., 1956. The prehensile movements of human hand. J. Bone Joint Surg. 38, 902–913. Park, J., Seo, N.J., Son, J., Kim, W., Cheong, J., 2014. Postural variation of hand precision grips by object size. J. Mech. Sci. Technol. 28, 1641–1651. Pouydebat, E., Reghem, E., Borel, A., Gorce, P., 2011. Diversity of grip in adults and young humans and chimpanzees (pan troglodytes). Behav. Brain Res. 218, 21–28. Schot, W.D., Brenner, E., Smeets, J.B.J., 2010. Posture of the arm when grasping spheres to place them elsewhere. Exp. Brain Res. 204, 163–171. Seo, N.J., Armstrong, T.J., 2008. Investigation of grip force, normal force, contact area, hand size, and handle size for cylindrical handles. Hum. Factors 50, 734–744. Shamshiri, R.R., Weltzien, C., Hameed, I.A., Yule, I.J., Grift, T.E., Balasundram, S.K., Pitonakova, L., Ahmad, D., Chowdhary, G., 2018. Research and development in agricultural robotics: a perspective of digital farming. Int. J. Agric. Biol. Eng. 11, 1–14. Silwal, A., Davidson, J.R., Karkee, M., Mo, C., Zhang, Q., Lewis, K., 2017. Design, integration, and field evaluation of a robotic apple harvester. J. Field Rob. 34, 1140–1159. Stern, E., 2015. Embodied cognition: a grasp on human thinking. Nature 524, 158–159. Vatavu, R.D., Zaiti, I.A., 2013. Automatic recognition of object size and shape via userdependent measurements of the grasping hand. Int. J. Hum Comput Stud. 71, 590–607. Vergara, M., Sancho-Bru, J.L., Gracia-Ibanez, V., Perez-Gonzalez, A., 2014. An introductory study of common grasps used by adults during performance of activities of daily living. J. Hand Ther. 27, 225–234. Wang, L., Zhao, B., Fan, J., Hu, X., Wei, S., Li, Y., Zhou, Q., Wei, C., 2017. Development of a tomato harvesting robot used in greenhouse. Int. J. Agric. Biol. Eng. 10, 140–149. Wong, Y.J., Whishaw, I.Q., 2004. Precision grasps of children and young and old adults: individual differences in digit contact strategy, purchase pattern, and digit posture. Behav. Brain Res. 154, 113–123. Yang, S., Fu, B., Hou, Y., Chen, S., Li, Z., Wang, S., 2019. Transient cooling and operational performance of the cryogenic part in reverse brayton air refrigerator. Energy 167, 921–938. Zhang, B., Zhou, J., Meng, Y., Zhang, N., Gu, B., Yan, Z., Idris, S.I., 2018. Comparative study of mechanical damage caused by a two-finger tomato gripper with different robotic grasping patterns for harvesting robots. Biosyst. Eng. 171, 245–257.
Bac, C.W., Hemming, J., Tuijl, B.A.J., Barth, R., Wais, E., Henten, E.J., 2017. Performance evaluation of a harvesting robot for sweet pepper. J. Field Rob. 34, 1123–1139. Bloch, V., Degani, A., Bechar, A., 2018. A methodology of orchard architecture design for an optimal harvesting robot. Biosyst. Eng. 166, 126–137. Bullock, I.M., Zheng, J.Z., Rosa, S.D.L., Guertler, C., Dollar, A.M., 2013. Grasp frequency and usage in daily household and machine shop tasks. IEEE Trans. Haptic 6, 296–308. Claire, C.G., Cynthia, L.B., Bruce, B., Joseph, L.P., Patricia, B., Stephen, P.P., 2012. Anthropometric Survey of U.S. Army Personnel: Methods and Summary Statistics. U.S. Army Natick Soldier Research, Development and Engineering Center Natick, Massachusetts. Clement, R.G.E., Bugler, K.E., Oliver, C.W., 2011. Bionic prosthetic hands: a review of present technology and future aspirations. The Surgeon 9, 336–340. Cutkosky, M.R., 1989. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Trans. Robot. Automat. 5, 269–279. Desmet, M., Lammertyn, J., Scheerlinck, N., Verlinden, B.E., Nicolai, B.M., 2003. Determination of puncture injury susceptibility of tomatoes. Postharvest Biol. Technol. 27, 293–303. Feix, T., Bullock, I.M., Dollar, A.M., 2014. Analysis of human grasping behavior: object characteristics and grasp type. IEEE Trans. Haptic 7, 311–323. Feix, T., Pawlik, R., Schmiedmayer, H., Romero, J., Kragic, D., 2009. A comprehensive grasp taxonomy. In: RSS workshop on Understanding the Human Hand for Advancing Robotic Manipulation, Seattle, USA. Feix, T., Romero, J., Schmiedmayer, H., Dollar, A.M., Kragic, D., 2016. The grasp taxonomy of human grasp types. IEEE Trans. Hum.-Mach. Syst. 46, 66–77. Feng, Q., Cheng, W., Zhou, J., Wang, X., 2014. Design of structured-light vision system for tomato harvesting robot. Int. J. Agric. Biol. Eng. 7, 19–26. Frisancho, A.R., 2008. Anthropometric Standards, second ed. University of Michigan Press, Michigan, USA. Graichen, K., Hentzelt, S., Hildebrandt, A., Karcher, N., Gaibert, N., Knubben, E., 2015. Control design for a bionic kangaroo. Control Eng. Pract. 42, 106–117. Hochberg, L.R., Bacher, D., Jarosiewicz, B., Masse, N.Y., Simeral, J.D., Vogel, J., Haddadin, S., Liu, J., Cash, S.S., van der Smagt, P., Donoghue, J.P., 2012. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375. Hodson, R., 2018. A gripping problem. Nature 557, s23–s25. Hosmer, D., Lemeshow, S., 2000. Applied Logistic Regression, second ed. WileyInterscience Publication, New York. Jia, L., Du, Q., 2010. SPSS Statistical Analysis Standard Tutorial. Post & Telecom Press, Beijing. Kyota, F., Saito, S., 2012. Fast grasp synthesis for various shaped objects. Comput. Graphics Forum 31, 765–774. Lee, K.S., Jung, M.C., 2014. Common patterns of voluntary grasp types according to object shape, size, and direction. Int. J. Ind. Ergon. 44, 761–768. Li, Z., Andrews, J., Wang, Y., 2017a. Mathematical modelling of mechanical damage to tomato fruits. Postharvest Biol. Technol. 126, 50–56.
97