Integration of perception capabilities in gripper design using graspability maps

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Available online at www.sciencedirect.com

ScienceDirect journal homepage: www.elsevier.com/locate/issn/15375110

Research Paper Special Issue: Robotic Agriculture

Integration of perception capabilities in gripper design using graspability maps Danny Eizicovits a,*, Bart van Tuijl b, Sigal Berman a, Yael Edan a a b

Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva, Israel Wageningen UR Greenhouse Horticulture, Wageningen, The Netherlands

article info

Agricultural environments impose high demands on robotic grippers since the objects to be

Article history:

grasped (e.g., fruit) suffer from inherent uncertainties in size, shape, weight, and texture, are

Published online xxx

typically highly sensitive to excessive force, and tend to be partly or fully occluded. This paper presents a methodology for evaluating the influence of perception capabilities on grasping and on gripper design using graspability maps. Graspability maps are spatial representations

Keywords:

of grasp quality grades from wrist poses (position and orientation) about an object and are

Gripper design

generated using simulation. A new module was developed to enable the insertion of object

Grasping

pose errors for testing the effects of perception inaccuracies on grasping. The methodology

Sensing

was implemented for comparing two grippers (Fin-Ray and Lip-type) for harvesting two

Sweet-pepper harvesting

sweet-pepper cultivars. A 3D model of each gripper was constructed and suitable grasp

Agricultural robots

quality measures were developed and validated in a physical environment. Task and gripperspecific grasp quality measures were developed for each implementation. Sensitivity analyses included varying pepper dimensions and perception inaccuracies. These were followed by analyses of the influence of gripper design parameters on grasp capabilities. Results indicate that the Lip-type gripper is less sensitive to inaccuracies in object orientation, while both grippers are similarly sensitive to inaccuracies in object position. Specific perception system demands and design recommendations are given for each gripper, and cultivar. The results illustrate the importance of integrating perception analysis in the gripper design phase and the utility of the graspability simulation tool for design analysis. © 2015 IAgrE. Published by Elsevier Ltd. All rights reserved.

1.

Introduction

Grippers are a critical component in robotic systems required to perform object manipulation. They must be appropriate for

the robotic arm, the object to be grasped, the environment in which the system operates, and the task at hand. While standard mechanical structures are typically used for robotic manipulators, grippers are typically re-designed for each implementation. Thus, gripper design is an important part of

* Corresponding author. Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, P.O.B. 653, Beer-Sheva 84105, Israel. Tel.: þ972 8 647 9785. E-mail address: [email protected] (D. Eizicovits). http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016 1537-5110/© 2015 IAgrE. Published by Elsevier Ltd. All rights reserved. Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

2

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Nomenclature TPR FPR TP FP FN TN NL IL FCA SD CVR QLIP QBFS

True-Positive Rate False-Positives Rate True-Positive False-Positive False-Negative True-Negative Netherland's Israel force closure angle stability distance cylinder voxel rate quality measure used for the Lip-type gripper quality measure used for the Fin-Ray gripper

robotic system design. A well designed gripper can contribute to successful system operation, increase overall system reliability, simplify requirements from other system components, and decrease system implementation costs (Brown & Brost, 1997; Raghav, Kumar, & Senger, 2012). Previous research on gripper design suggested six main guidelines (Zaki, Soliman, Mahgoub, & El-Shafei, 2010): minimize gripper weight, grasp object securely, multiple object gripping with a single gripper, fully encompass the object within the gripper, do not deform the object, and minimize size taking into account proper object-gripper interaction. In many cases there are interactions between these different guidelines. Current gripper design methodologies try to optimize gripper kinematics (Ciocarlie, Hicks, & Stanford, 2013; Li, Liu, Li, & Li, 2008) and dynamics (Blanes, Mellado, Ortiz, & Valera, 2011; Cao, Gu, Li, & Liu, 2013) taking into account object constraints (e.g., size, shape, weight, slipperiness, fragility, and accessibility) and task constraints (force and motion) (Blanes et al., 2011; Zaki et al., 2010). Design approaches include empirical methods (Chelpanov & Kolpashnikow, 1986; Giannaccini, Dogramadzi, & Pipe, 2011; Sam & Nefti, 2011; Spiers, Baillie, Pipe, & Persad, 2012), and objective functions optimizing geometric parameters, e.g., capability index, or grasping index (Berman & Nof, 2011; Brown & Brost, 1997; Cutkosky, 1989; Gorce & Fontaine, 1996; Saravanan, Ramabalan, Ebenezer, & Dharmaraja, 2009; Streusand & Turner, 2011; Walsh, 1984). The gripper design process is also influenced by the manipulation abilities of the robotic-gripper system which are also highly dependent on the capabilities of the sensory system. The task of the sensory system is to provide the geometric description of the objects within the environment (Fantoni, Gabelloni, & Tilli, 2012). Although there is a vast body of literature that links grasping to perception (Coelho, Piater, & Grupen, 2001; Detry et al., 2011; Moreno, Hornstein, & SantosVictor, 2011), their interactions have been addressed in the context of grasp planning and control and not at the design phase. However, gripper design highly influences both motion planning and control (Boubekri & Chakraborty, 2002; Ceccarelli, Figliolini, Ottaviano, Mata, & Criado, 2000), thus it is important to address the perception-action links during design. For example, pepper harvesting requires cutting the pepper's stem at the peduncle without harming the fruit or the

plant. Hence, the accuracy with which the fruit pose (position and orientation) and peduncle can be detected determines the operation possibilities of the gripper and imposes constraints on gripper design. One of the major challenges in the development of a selective harvesting robot is how to grasp, detach, and manipulate the fruit without damaging it or the plant (Edan, 1999). Gripper design has been addressed in many agricultural robotics research and development projects, e.g. harvesting of tomatoes (Ceccarelli et al., 2000; Li, Li, & Liu, 2011; Li, Li, Yang, & Wang, 2013; Ling et al., 2004; Monta, Kondo, & Ting, 1998), melons (Edan, Haghighi, Stroshine, & CardenasWeber, 1992; Wolf, Bar-Or, Edan, & Peiper, 1990), apples , Boedrij, Beckers, & Claesen, 2008; De-An, (Baeten, Donne Jidong, Wei, Ying, & Yu, 2011), cucumbers (van Henten et al., 2002), asparagus (Irie, Taguchi, Horie, & Ishimatsu, 2009), grapes (Monta, Kondo, & Shibano, 1995), cabbages (Murakami, Otsuka, Inoue, & Sugimoto, 1995), cherries (Tanigaki, Fujiura, Akase, & Imagawa, 2008), strawberries (Hayashi et al., 2010), radishes (Foglia & Reina, 2006), peppers (Hemming, Bac, et al., 2014; Hemming, Ruizendaal, Hofstee, & van Henten, 2014; Hemming, van Tuijl, Gauchel, & Wais, 2014; Kitamura & Oka, 2005; mushrooms, Reed, Miles, Butler, Baldwin, & Noble, 2001), and in a similar context on grippers for food products (Blanes et al., 2011; Chua, Ilschner, & Caldwell, 2003; Pettersson, Davis, Gray, Dodd, & Ohlsson, 2010), Fruit detection algorithms for robotic applications have been extensively studied (Kapach, Barnea, Mairon, Edan, & Ben-Shahar, 2012) and specifically recently in a parallel study for sweet pepper detection (Hemming, Ruizendaal, et al., 2014), however linking perception into the gripper design process has not been investigated. While perception is a critical element in most environments, in agricultural environments its importance is amplified due to inherent complexities (e.g., occlusions, and dynamic lighting conditions) and uncertainties (e.g., highly variable fruit size, shape, and location) caused by the natural biological variability. The current research suggests a methodology for evaluating the effects of perception capabilities during the mechanical design of the gripper, i.e., prior to full system integration. The method is based on a simulation tool developed to analyse the influence of perception errors on grasp quality based on models of the designed gripper and object to be grasped. Analysing the sensitivity of the gripper design to perception errors can be derived by analysing the effects of errors in object pose estimation on grasp quality using graspability maps. Graspability maps provide an efficient representation of grasp quality grades for grasp poses about the object and have been used for comparison between different grippers (Eizicovits & Berman, 2014a, 2014b; Roa et al., 2014; Roa, Hertkorn, Zacharias, Borst, & Hirzinger, 2011). The current paper details the building blocks of the proposed methodology and simulation tool, and demonstrates its use with a case study of developing a gripper for sweet-pepper selective harvesting. We demonstrate an evaluation of the influence of perception errors for comparison between two different gripper designs and their parameters. The presented methodology complements the mechanical gripper design methodology developed in a parallel research (van Tuijl, 2015;

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

van Tuijl & Wais, 2014). The two gripper designs were selected based on this methodology and a requirements analysis defined in Hemming, Bac, & van Tuijl, 2011. Both grippers were integrated into an operational robotic system which was field tested (Hemming, Bac, et al., 2014). The field tests focused on evaluation of the integrated system and not specifically the grippers. The remainder of the paper is organized as follows: Section 2 presents the developed methodology and a case study implementation for pepper selective harvesting. Within the context of the case study, Section 3 presents the results of a validation experiment, comparison of two gripper types, and gripper parameters analysis. Discussion on the results is given within each experiment in Section 3. Conclusions are presented in Section 4.

2. Method e linking perception into gripper design The proposed design methodology includes three stages: Mechanical design, Simulation adaptation, and Perception sensitivity analysis (Fig. 1). The Mechanical design stage (van Tuijl, 2015) starts with problem definition which includes a description of the task characteristics, the environment, and additional system constraints, e.g., limitations caused by the manipulator or perception apparatus. The feasible state-of-the-art solutions are established based on this description. For example, pepper selective harvesting is currently done manually due to the high value of the crop and the high cutting precision required. Based on the problem definition, state-of-the-art designs, and the system requirements (defined in Hemming et al., 2011) several conceptual gripper designs are defined and physical prototypes of these designs are built and field tested (van Tuijl, 2015). The outcome of the Mechanical design stage is used as an input for the intermediate Simulation adaptation stage. This stage is initiated by construction of 3D models of the conceptual gripper prototypes and objects to be grasped (Section 2.2). Task and performance quality measures are defined and adapted for each gripper based on its mechanical design and operation method (Section 2.3.2). For example, grasp quality of a two jaw gripper is often defined based on geometrical relations between the finger aperture and the object, while for vacuum grippers grasp quality depends on the contact surface and the applied air pressure. Graspability maps are generated for each object and gripper tuple using simulation based on the 3D models of the gripper and object and the required graspability map resolution (Eizicovits & Berman, 2014a, 2014b). The soundness of the models, measures, and adequacy of the generated graspability maps are validated based on an experiment with physical entities (Section 2.3.3) in a standalone manual operation mode of the gripper. When experimenting with a physical system it is typically more feasible to measure grasp success rather than grasp quality. Thus, to simplify comparison between the modelled graspability maps and physical measurements a threshold can be used to transform the grasp quality grade to a binary successes measure. The graspability map generation method is validated by examining the correspondence between the

3

graspability map and the physical experiment for comparable grasp poses, e.g., by computing the true positive rate (TPR) and false positive rate (FPR) (Section 3.1). If the TPR and FPR are below and above a respectively predefined threshold then the graspability simulation software should be adapted and rechecked, otherwise the simulation can be considered as valid and the graspability maps can be used for evaluating the influence of perception errors on the gripper design. The influence of perception errors on the designed gripper is evaluated using the graspability maps in the Perception sensitivity analysis stage. The influence of perception errors on the designed gripper is tested by inserting changes emulating perception errors in object pose and evaluating the difference between the grasp success rate for an object with such pose errors and an object placed at the correct pose (Section 2.4). The process is done as follows: first a graspability map is generated for the gripper and object with the object at the correct pose. This is defined as the “accurate” condition. Then the object is linearly transformed, i.e., translated or rotated according to the induced pose error, and then the grasps from the “accurate” condition graspability map are executed on the transformed object. Grasps which remain successful are saved in a new graspability map. This process is executed for all combinations of the considered gripper and object dimensions and for all gripper designs. Finally, if one of the examined designs is found suitable, e.g., robust to perception errors within the perception error range expected in the system, the output can be used for updating the developed physical prototypes. If this is not the case the process must be reinitiated at the gripper definition stage. In the current research, updating the physical grippers in the case study was out of scope, but the developed methodology supports this inclusion.

2.1. Case study: linking perception into gripper design for pepper selective harvesting A robotic system for selective harvesting of sweet-peppers in greenhouses was developed for the V-cropping system of sweet-peppers in the Netherlands (Hemming, Bac, et al., 2014; Hemming, van Tuijl, et al., 2014; van Tuijl, Wais, & Edan, 2013). The system included two basic modules situated on two adjacent carriers: a manipulator module with nine degrees of freedom (DOF) and a sensor module with several cameras and an illumination system. The current section details the implementation of the pepper harvesting gripper design method for this system.

2.2.

Mechanical design

Two different grippers were selected and tested based on the mechanical design scheme (van Tuijl, 2015; Hemming, van Tuijl, et al., 2014; van Tuijl et al., 2013) and the robotic system requirements (Hemming et al., 2011): the Fin-Ray endeffector (FESTO AG & Co. KG, Germany, Gauchel & Saller, 2012) the Lip-Type end-effector (Wageningen UR Greenhouse Horticulture, The Netherlands, van Tuijl & Wais, 2014). Both grippers were equipped with a cutting mechanism as the fruit must be cut precisely at its peduncle. Both grippers were additionally equipped with a small camera mounted on top of

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

4

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Fig. 1 e Methodology for analysing the effects of perception on gripper design, flow chart. Th% is a predefined threshold, typically 95%.

the gripper to guide the gripper towards the fruit. Preliminary experiments were manually conducted with both grippers in Hemming, van Tuijl, et al., 2014, however this did not include analysis of perception constraints on the process of the gripper design. The Fin-Ray gripper (Fig. 2 (a)) has four fingers and is based on utilizing the “Fin-Ray effect”, for adjusting the fingers shape to the curvature of the fruit (Gauchel & Saller, 2012). Scissors were mounted on top of the gripper for detaching the pepper from the plant after a secure grasp is achieved. After cutting, the grasp is maintained to enable the manipulator to transport the fruit to a storage bin. For this design the grasp pose must facilitate the exact positioning of the scissors with respect to the peduncle. The Lip-Type gripper (Fig. 2 (b), van Tuijl & Wais, 2014) has two moving lips which can surround the fruit and a suction cup to stabilize the fruit for facilitating a successful detachment. After the fruit is stabilized, the two lips are closed, cutting the peduncle as they meet. If the peduncle obstructed either one of the two lips, the other lip continues to move until a full lip closure is achieved. As a result, the two lips meet at the peduncle regardless of its orientation with respect to the stabilization pose. This feature reduces the likelihood of cutting into the fruit or damaging the stem (Hemming, van Tuijl, et al., 2014). However, the grasp success of the Lip-Type gripper

depends on the ability of the suction cup to stabilize the fruit. This was not modelled in the simulation of the current work. After cutting, the suction cup is disconnected and the fruit freefalls into a chute that transports the fruit to a storage bin.

2.3.

Simulation adaptation

Graspability maps were constructed for both grippers using the developed simulation tool. The simulation was validated by comparing its outcome to results obtained using the physical gripper prototypes and physical peppers. The simulation tool was fully developed in MATLAB™ (Version R2010B, Mathworks, Natick, Mass, USA) and includes several interrelated modules: a module to simulate the 3D object and gripper data, a module for an efficient search of high quality grasps based on the developed grasp quality measures (Section 2.3.2), a module for evaluating collision avoidance between the gripper, the knife y scissors and the fruit and a module for analysing the effects of perception inaccuracies on the gripper capabilities. The first three developed modules are based on previous research (Eizicovits & Berman, 2014a) with important differences noted in Section 2.3. The last module (evaluation of perception inaccuracies on gripper design) was newly developed. Its components and its implementation are elaborated in Section 2.4.

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

5

Fig. 2 e (a) Fin-Ray gripper 3D model e Left: MATALB™ and Right: MESHLAB. (b) Lip-type gripper 3D model e Left: MATALB™ and Right: MESHLAB.

2.3.1.

Simulation setup

The grippers were modelled in MATLAB™ based on the STL files of the gripper design using MESHLAB open source (Cignoni et al., 2008; http://meshlab.sourceforge.net) (Fig. 2, left). Both grippers were represented as point clouds for facilitating manipulation in the 3D space. Additionally, a 3D model of a sweet-pepper was generated using Autodesk 123D™ (Chandler & Fryer, 2013; http://www.123dapp.com) software. The pepper dimensions were then manipulated in MATLAB™ based on the dimensions of the average pepper size in cultivar systems in the Netherlands (NL) and in Israel (IL) (Fig. 3).

For the Fin-Ray gripper the grasp quality, QFS, was determined based on a weighted average between force closure and stability, (Eq. (1)). Force closure (Nguyen, 1986) was quantified by the force closure angle (FCA), which represents the normalized difference between the friction cone contact angles (qi ; qj ), and the angle (w) between the normal (ni) and the vector Pij, Fig. 4. Stability was quantified by the stability distance (SD), which is the distance between the centre of mass and the vector between the contact positions normalized by the largest feature of the object (Fig. 4, Eizicovits & Berman, 2014a).
QFS ¼

2.3.2.

Task and performance measures

Performance quality of a pepper selective harvesting task is quantified by both grasp and cutting quality based on measures developed and elaborated in (Eizicovits & Berman, 2014a, 2014b) and adapted as stated below. Since harvesting success is the main objective of the developed system and since the mechanisms and operation method of the two grippers are very different, a binary task success measure was used rather than a quality measure for their comparison. For both grippers, cutting success was determined based on the simulated trajectory of the scissors or lips. If the scissors or lips crossed the peduncle without harming the fruit (i.e., if they did not collide with the fruit) then the cut was determined as successful (“1”).

0:5$FCA þ 0:5$SDL FCA > 0; SDL > 0 0 else

(1)

Grasp success, QBFS, was determined as “successful” when QFS was above 0.7 or “unsuccessful” otherwise. This threshold value was determined based on a physical experiment conducted with a two-jaw gripper with soft fingers (Eizicovits & Berman, 2014b). For the Lip-type gripper grasp success, QLip, was defined based on the surface covered by the suction cup. The grasp was considered successful if 80% of the suction cup surface touched the fruit's surface:

QLip ¼

8 > > > <

Pp 1

> > > :0

i¼1

P

pi

> 0:8

(2)

else

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

6

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Fig. 3 e Illustration of the 3D pepper model generated in Autodesk 123D (right) and transformed to MATLAB™ (left).

Fig. 4 e Grasp quality measures used for the simulation: force closure angle (FCA) and stability distance (SD).

It must be noted that the ability of the suction cup to stabilize the fruit is difficult to model due to the uneven surface and obstructions such as dust, leaves and branches which can interfere with the suction surface. Gripper capabilities were analysed based on the generated graspability map using two performance measures: the total number of successful grasps and grasp cylinder voxel rate (CVR), (Eq. (3)) which is an adaptation of the grasp voxel rate (Eizicovits & Berman, 2014a) to the cylindrical objects. CVR ¼

PVn

i¼1 Vi

Vn

cVi ; Vn 2Cylinder

(3)

Where Vn is the normalized number of voxels within a cylindrical shape about the pepper, and Vi is 1 if a grasp with quality > 0 was found in voxel i and 0 if not.

2.3.3.

Graspability map validation

The validation experiments were conducted using a gripper rig, the two gripper prototypes, and six Israeli cultivar sweetpeppers (Fig. 5). The rig facilitates positioning and orientating

the gripper in all six DOF (translation and rotation). The position and angles tested were in 0.5 mm and 1 intervals respectively and were defined by using linear and angular rulers. Levelling feet and water level gauges were used to verify that the rig was accurately levelled with respect to the horizon. A camera tripod fastened to the floor was used to fixate the sweet-peppers during the experiments. Each sweetpepper was mounted on a camera tripod, in front of the rig (Fig. 5B). A water level gauge in two directions in the horizontal plane on top of the sweet-pepper was used to validate that the pepper was levelled with respect to the horizon. To fixate the pepper to the tripod a stiff strip of aluminium was pierced through the sweet-pepper and bolted onto a handle of the tripod. Nine different grasping positions were tested for each pepper. For the first position the rig was positioned such that the tool centre point (TCP) of the gripper was located at the centre of the pepper when the gripper was oriented along the normal to the vertical directions of the pepper. The other eight positions were in different positions on a 10 mm square about the first pose, i.e., at intervals of 10 mm along the plane parallel to the pepper (the coronal plane) (Fig. 6). Different grasp orientations were tested for the Fin-Ray gripper in the following manner: once a successful grasp was found (physical environment) the gripper was rotated in the roll or pitch direction until it reached its maximal value, i.e., when increasing the orientation would have caused damage to the fruit. In this study, all damages were classified as cuts in fruits or plant stems. The maximal orientation value was saved as the threshold between successful and unsuccessful grasps. In the simulation environment, for each of the roll or pitch orientations, two orientations were tested: one above the threshold and one below the threshold. Thus, for poses in which the grasp was successful four more poses were tested (two for roll and two for the pitch). Different orientations were not tested for the Lip-type gripper since once a successful grasp was found the grasp remained successful even for orientations which were larger than 30 due to the lip operation mechanism.

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

7

Fig. 5 e (A) End-effector rig and general setup. (B) Sweet-pepper mounted on the camera tripod.

All six tested peppers had different dimensions, but they were all close to the average Israeli pepper size. They were all picked a day before the experiment and all were commonly shaped peppers without unusual deformations (Table 1). 3D point clouds of all six peppers were generated in MATLAB™ based on the original pepper model (Fig. 3). H is the height top to bottom, D1 is the width at 0.2  H distance from the top, D2 is the width at 0.2  H distance from the bottom, PV is the vertical length of the peduncle and PH is the horizontal height of the peduncle. Task success in the physical environment was determined by the operator. The operator determined the task as “successful” when the grasp was stable (successful grasping), no

damage was caused to the fruit by the gripper or the knives, and when the scissors or lips were capable of cutting the fruit (successful cutting). Actual cutting was not conducted to enable repetitions on the same fruit from different poses. True positive (TP) represented grasps which were successful in both the simulation and the physical experiment and true negative (TN) represents grasps which were unsuccessful in both experiments. False positive (FP) and false negative (FN) represent discrepancies between the simulation and the physical tests, i.e., grasps which were successful (unsuccessful) in the simulation but were unsuccessful (successful) in the physical experiment respectively. Validation was assessed using the true positive rate (TPR), where

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

8

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Fig. 6 e Left: Illustration of the grasping points (grey circles) on the pepper. Right: The measured dimensions for each pepper. H is the height from top to bottom, D1 is the width at a 0.2 £ H distance from the top, D2 is the width at a 0.2 £ H distance from the bottom, PV is the vertical length of the peduncle and PH is the horizontal height of the peduncle. Pi represents grasping position number i.

TPR ¼ FP/(FP þ TN) and the false positive rate (FPR), where FPR¼FP/(FP þ TN).

2.4. 2.4.1.

Perception sensitivity analyses

2.4.3.

Setup

One hundred peppers were collected from both Israeli (red Bell peppers) and Dutch (Capsicum Annuum) greenhouses (50 from each). The dimensions of each pepper were measured (L, D1, D2, PV, and PH) and the maximal, average and 80% size of all the peppers were calculated for each cultivar type (Israeli and Dutch). Based on these values, six representative peppers were defined, three for each cultivar type (Table 2). A 3D point cloud model was created for each pepper and a graspability map was generated using MATLAB™ for each pepper and each gripper configuration. These graspability maps represent the baseline “accurate” condition, i.e., when there are no perception errors.

2.4.2.

Gripper comparison

The graspability maps and pepper models were used for comparing the two gripper models in light of perception errors (as described in Fig. 1). The perception errors were induced by rotating or translating the pepper model and testing successful grasps defined by the “accurate” graspability map as

Table 1 e Pepper dimensions (cm) in the validation experiment.

L D2 D1 PV PH

described in Section 2. The pepper orientations were from 0 to 20 with a step size of 2 and position errors ranged from 0 to 2 cm with a step size of 0.1 cm, in all axes.

1

2

3

4

5

6

98 73 88 33 27

91 97 101 27 19

97 79 93 31 27

95 72 84 41 0

94 80 85 32 30

101 85 100 20 40

Gripper parameters

The environment was not modelled and thus the effects of the environment were not taken into account. Enlarging the lip radius may complicate collision avoidance with respect to the trellises and nearby peppers. Thus, in general it is desirable to have a gripper with as small as possible footprint. Analyses of the gripper parameters focused on testing one design parameter per gripper. The selected design parameter was the one which was most influential during the perception sensitivity analysis (Section 3.2). For the Fin-Ray gripper the distance between the scissors and the gripper's fingers was examined (Fig. 2a), as in several cases during the validation experiments, it seemed that increasing this distance might improve the robustness of the gripper to orientation errors. For the Lip-type gripper the radius of the lip volume was examined (Fig. 2b). This radius should be as small as possible in order to avoid collisions with the environment. However, a small radius might damage the gripper's capability to encompass peppers with large dimensions. For each gripper five different parameter values were tested and compared. For the Fin-Ray gripper, the scissors distance was examined with the following dimensions: 2, 3, 3.5, 4 and 5 (all cm units). For the Lip-type gripper, the dimension of the lip radius was examined with the following values: 5.6, 6.2, 7.0, 8.1, and 8.5 (all in cm units). The dimensions of the physical prototypes were 3 cm for the scissors distance and 6.2 cm for the lip radius. For each parameter value a complete perception sensitivity analysis was conducted (as described in Fig. 1), i.e. orientations from 0 to 20 (step size 2 ) and position errors from 0 to 2 cm (step size of 0.1 cm). The analysis was done for the 80% sized NL and IL peppers (Table 2).

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

9

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Table 2 e Pepper dimensions (mm) in the Netherland's (NL) and Israel (IL).

(D1þD2)/2 L PV PH

NL_AVG

NL_80%

NL_MAX

IL_AVG

IL_80%

IL_MAX

75 94 15 30

84 104 16 31

101 116 17 38

90 99 14 15

96 106 17 29

110 122 20 38

NL e Nederland's pepper type; IL e Israeli pepper type; AVG, 80%, MAX e pepper size.

3.

Results and discussion

3.1.

Validation

The results of the validation analysis are summarized in Table 3. The True positive rate for both the Fin-Ray (100%) and the Lip-type grippers (95%) was very high, while the False positive rate for both the Fin-Ray (0%) and the Lip-type grippers (5%) was very low. The small differences between the simulation and the physical environment were due to minor differences between the physical and modelled peppers. The pepper model assumed complete symmetry with respect to the pepper's vertical axis yet some of the physical peppers had minor symmetry violations which caused the differences in the

Table 3 e Results for True and False positive rates for the Fin-Ray and Lip-type grippers.

True positive True negative False positive False negative True positive rate False positive rate

Fin-Ray gripper

Lip-type gripper

20 81 0 1 100% 0%

34 18 1 1 95% 5%

results (96% success vs. 100%). For example, one of the physical peppers had a slight dimension change of about 3 mm at one side therefore a grasp pose at the pepper side (position 5) was successful in the simulation but not in the physical environment.

3.2.

Gripper comparison

An illustration of two graspability maps in the “accurate” condition is shown in Fig. 7. It can be seen that the Lip-type gripper has in general more cells with successful grasps than the Fin-Ray gripper. It can also be seen that most of the grasps for the Lip-type gripper are distributed at the centre and to the bottom of the fruit, while for the Fin-Ray gripper most of the grasps are distributed at the centre and to the top of the fruit. Results for the perception sensitivity analysis, indicate that both grippers had a large total number of successful grasps (>500 grasps), when there are no perception inaccuracies (the “accurate” condition), in all cases except for the IL maximal size (Fig. 8). The IL maximal size was out of range for the Liptype gripper and thus there were no successful grasps for this pepper and gripper tuple. For all other peppers the Lip-type gripper had a larger total number of successful grasps (~1500e4000) than the Fin-Ray gripper (~500e1500), Fig. 8. Analysis of orientation perception inaccuracies, indicates that the Lip-type gripper had a large total number of successful

Fig. 7 e Graspability maps. Left: the Fin-Ray gripper. Right: the Lip-type gripper. The blue dots represent cells with poses that lead to successful grasping and cutting. Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

10

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Fig. 8 e Total number of found grasps with respect to perception errors. Left: position errors in cm. Right: Orientation errors in degrees. A. NL pepper type. B. IL pepper type.

grasps (~1000 and above) when orientation inaccuracies were applied and the number of possible successful grasps did not decline to zero even when the orientation error was raised up to 20 . This is not the case for the Fin-Ray gripper, where its total number of grasps declined towards zero when the orientation inaccuracies reached 10e15 depending on the pepper size (Fig. 8). Analysis of the position inaccuracies shows that both grippers were highly affected by the position perception inaccuracies, as for both grippers the total number of grasps declined to zero at about 1 cm of position error (Fig. 8). Results of the CVR measure indicate that orientation errors of 10e15 in the Fin-Ray gripper caused a very large reduction in CVR, while the Lip-type gripper had a high CVR score even for large orientation errors (about 20 ) and this score is almost not affected by orientation errors (Fig. 9). Position errors affected both grippers in a similar way and at about 1 cm of Position inaccuracy, the CVR converged to zero for both grippers (Fig. 9). It was also noticeable that CVR of

the Lip-type gripper was higher for all the peppers and dimensions, except for the maximal IL pepper for which no grasps were found as it was outside the range the gripper was designed for.

3.3.

Gripper parameter analysis

The total number of grasps in the “accurate” condition for all the configurations was above 500 grasps except for the small dimension of the Lip-type gripper with the IL pepper type. This was due to the fact that the pepper's size exceeded the dimensions of the gripper (). The total number of grasps for the Fin-Ray gripper was similar for the 3 cm and 3.5 cm configurations and both were superior to the rest of the configurations for both IL and NL peppers. For the Lip-type gripper the lip radius of 8.1 cm and 8.5 cm had the largest number of grasps and the lip radius of 7 cm had a large number of grasps (above 2000) for the NL pepper type.

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

11

Fig. 9 e CVR in percent with respect to perception errors. Left: position errors in cm. Right: orientation errors in degrees. A. NL pepper type. B. IL pepper type.

Based on the CVR measure the configuration of 3 cm scissors distance for the Fin-Ray gripper is superior to all other configurations for the NL pepper type (Fig. 10A). However, for the IL pepper type the 3.5 cm scissors distance was slightly better than the 3 cm configuration, especially when the perception orientation error was large (Fig. 10B). For the Lip-type gripper results indicate that increasing the lip radius increases the number of found grasps (Table 4). However, this does not improve CVR beyond a certain value which depends on the size of the harvested fruit (Fig. 11). For example the largest total number of grasps is obtained for lip radius of 8.5 cm. However, the CVR for lip radius of 7 cm is higher (especially with orientation perception errors, Fig. 11A) and thus, since smaller radii are preferable, the best configuration for the NL pepper type is a lip radius of 7 cm. For the IL pepper type the best configuration is a lip radius of 8.1 cm (Fig. 11B).

Based on the conducted simulations it is clear that the size differences between different pepper types affects gripper design. It is also clear that several gripper dimension configurations lead to improved performance. For the Fin-Ray gripper, the best configurations are scissors distances of 3 cm (NL peppers) and 3.5 cm (IL peppers). For the Lip-type gripper the best configurations are the lip radius of 7 cm (NL peppers) and 8.1 cm (IL peppers). When comparing the best configurations of both grippers, the Lip-type gripper has a higher CVR in the “accurate” condition (~40% VS ~24%). The comparison also shows that when perception inaccuracies were inserted the differences between the grippers were increased. For the Fin-Ray gripper an inaccuracy of 20 or 1.2 cm reduces the CVR to approximately zero (Fig. 10). This is not the case for the Lip-type gripper, where successful grasps are found for orientation inaccuracies of 20 and for position inaccuracies of 1.5 cm (Fig. 11). Thus, the Lip-type gripper is more suitable for the task of

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

12

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Fig. 10 e CVR in percentages for different scissors distances for the Fin-Ray gripper. Left: position errors in cm. Right: orientation errors in degrees. A. NL pepper type. B. IL pepper type.

Table 4 e Total number of grasps and CVR measures in the “accurate” condition. Fin-Ray gripper

Total number of grasps

CVR[%]

Lip-type gripper

Scissors distance [cm]

NL peppers

IL peppers

Lip radius [cm]

NL peppers

IL peppers

2 3 3.5 4 5 2 3 3.5 4 5

1025 1241 1208 1173 892 20.05 23.5 23.5 23.5 19.6

1400 1438 1371 1259 1031 21.7 22.4 22.6 21.9 20.8

5.6 6.2 7.0 8.1 8.5 5.6 6.2 7.0 8.1 8.5

594 1258 2321 3621 3623 19.1 27.4 39.6 38.4 37.3

0 328 1078 2840 2999 0 14.4 25.1 33.9 33.9

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

13

Fig. 11 e CVR in percentages for different lip radiuses of the Lip-type gripper. Left: position errors in cm. Right: orientation errors in degrees. A. NL pepper type. B. IL pepper type.

selective pepper harvesting than the Fin-Ray gripper for all pepper types.

3.4.

Field experiments

Final field experiments, for the complete harvesting system, were conducted in a in a parallel study (Hemming, Bac, et al., 2014; Hemming, van Tuijl, et al., 2014), in a Dutch sweetpepper greenhouse using the two grippers (Fin-Ray and Liptype) studied in the current work. The tested system included the robotic and sensors apparatuses and the control system (fruit detection, fruit localization, and motion planning). The results of the field experiments were qualitative and thus comparisons of the simulation results and the field experiments were limited. However, despite this limitation, there was high correspondence between them, and the conclusions obtained in the simulation process in this paper were similar to the field experiments. The successful operation of the Fin-

Ray gripper required more precise positioning and orientation than the Lip-type gripper. Success in cutting the peppers and stem damage were in line with simulation results. The Fin-Ray gripper had a lower cutting success than the Lip-type gripper, and in addition the Fin-Ray gripper caused stem damage while the Lip-type gripper did not. One of the reasons for the damage to the stem caused by the Fin-Ray gripper was that the calibration inaccuracies of the perception system ranged from 0.5 cm to 3.5 cm. This corresponds to the simulation results which indicated that position inaccuracy of 1 cm and above would result with many unsuccessful grasps. Unlike the simulation results, in the field experiments the grasp success of the Fin-Ray gripper was higher than for the Lip-type gripper. This was due to the fact that the air pressure used in the field experiment for the suction cup was low and thus the pepper was not stabilized properly prior to the cutting operation. This caused many of the executed grasps to fail and in some cases the peduncles were only partially cut. As aforementioned, the

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

14

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Fig. 12 e Field robotic system.

simulation did not model the effects of the air pressure on suction cup operation. Furthermore, additional environmental obstructions (e.g., uneven surface, dust, leaves) could influence the suction. Adding this to the simulation can assist in determining the required pressure for future operation (Fig. 12).

4.

Conclusions

The main innovation of the current work is in suggesting a methodology for analysing the influence of perception errors on gripper design using graspability maps. The presented methodology and simulation tool can be used to effectively reduce the number of developed physical gripper prototypes and the number of physical experiments. In addition the methodology facilitates establishing insights regarding the mechanical gripper design and the requirements from the perception apparatus early on in the system development process and most importantly prior to full system integration. Use of the method can lead to reduced development cost and time, and can considerably improve overall system performance. The developed method is suitable for additional application domains in which the robot is required to identify and grasp an object. It is especially required in the agricultural domain is which both perception and grasping are difficult due to inherent uncertainties in both the environment and the object to be grasped. The results in the simulation showed that the design of the Lip-type gripper is more suitable for the task of pepper selective harvesting, since it was less sensitive to inaccuracies in object orientation. For the Lip-type gripper successful grasps were found for orientation errors of 20 , while for the Fin-Ray gripper errors of about 10 in object orientation dramatically reduced the number of successful grasp to almost zero. Both grippers were similarly sensitive to inaccuracies in object position, and were able to endure only errors of 1 cm or less.

Although adjustments are needed to apply the proposed methodology for different fruits and grippers, similar quality measures may be used in multiple cases where harvesting requirements or gripper design bear similarities to the ones studied in the current work. For example, the QLIP quality measure used for the Lip-type gripper in the presented pepper harvesting case study is also suitable for a similar gripper used for melon harvesting (Edan, Rogozin, Flash, & Miles, 2000), or the QBFS quality measure used for the Fin-Ray gripper is also suitable for a similar gripper used for apple harvesting (De-An et al., 2011). When the harvesting method is very different, e.g., for the cucumber harvesting system (van Henten et al., 2002), a quality measure that accounts for the specific system is required (e.g., the thermal cutting system). The proposed method extends and augments classical mechanical design. It facilitates analysis of the interdependencies between the mechanical gripper design and the perception apparatus. The simulation can be extended to integrate additional components such as environment models, grasp dynamics, or cutting forces. The increased analysis capabilities offered by the additional modelling should be weighed against the complexity of the modelled phenomena and its expected influence on system performance. Despite intensive development of harvesting robots in the last three decades most of the developed systems have not been commercialized (Bac, Henten, Hemming, & Edan, 2014). The results of our study signify that integration of perception capabilities in the gripper design stage is critical for facilitating the development of suitable gripper for commercial selective harvesting robots. Such an integrated design will most probably benefit additional robotic systems for precision agriculture, which are required to interact with the environment and handle produce. The Fin-Ray gripper fingers have a future potential to grasp delicate fruits and vegetables and whereby the grasp needs to be maintained to transport the product to a storage bin like tomatoes or eggplants. The Liptype gripper is suited for smooth surfaced, round and or

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

oblong fruits which are sturdy enough to be dropped over a short distance after they have been cut from the plant such as sweet peppers and oranges.

Acknowledgements This research was funded by the European Commission in the 7th Framework Programme (CROPS GA no. 246252) and partially supported by the Helmsley Charitable Trust through the Agricultural, Biological and Cognitive Robotics Centre and by the Rabbi W. Gunther Plaut Chair in Manufacturing Engineering, both at Ben-Gurion University of the Negev.

references

Bac, C. W., Henten, E. J., Hemming, J., & Edan, Y. (2014). Harvesting robots for high-value crops: state-of-the-art review and challenges ahead. Journal of Field Robotics, 31(6), 888e911. , K., Boedrij, S., Beckers, W., & Claesen, E. (2008). Baeten, J., Donne Autonomous fruit picking machine: a robotic apple harvester. In Field and service robotics (pp. 531e539). Berlin Heidelberg: Springer. Berman, S., & Nof, S. (2011). Collaborative control theory for robotic systems with reconfigurable end effectors. In International Conference on Production Research (ICPR). Stuttgart: Germany. Blanes, C., Mellado, M., Ortiz, C., & Valera, A. (2011). Review. Technologies for robot grippers in pick and place operations for fresh fruits and vegetables. Spanish Journal of Agricultural Research, 9(4), 1130e1141. Boubekri, N., & Chakraborty, P. (2002). Robotic grasping: gripper designs, control methods and grasp configurations e a review of research. Journal of Integrated Manufacturing Systems, 13(7), 520e531. Brown, R. G., & Brost, R. C. (1997). A 3-d modular gripper design tool. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 2332e2339). Cao, W. B., Gu, W. J., Li, W. M., & Liu, J. D. (2013). Design of the automatic control system of gripper-type picking up seedling mechanism. In IEEE International Computer Conference on Wavelet Active Media Technology and Information Processing (pp. 279e282). Ceccarelli, M., Figliolini, G., Ottaviano, E., Mata, A. S., & Criado, E. J. (2000). Designing a robotic gripper for harvesting horticulture products. Robotica, 18(01), 105e111. Chandler, J., & Fryer, J. (2013). Autodesk 123D catch: how accurate is it. Geomatics World, 2(21), 28e30. Chelpanov, I. B., & Kolpashnikow, S. N. (1986). Problems with the mechanics of industrial robot grippers. In T. Pham, & W. B. Heginbotham (Eds.), Robot grippers (pp. 35e44). Berlin: Springer-Verlag. Chua, P. Y., Ilschner, T., & Caldwell, D. G. (2003). Robotic manipulation of food products e a review. Industrial Robot: An International Journal, 30(4), 345e354. Cignoni, P., Callieri, M., Corsini, M., Dellepiane, M., Ganovelli, F., & Ranzuglia, G. (2008, July). MeshLab: an open-source mesh processing tool. In Eurographics Italian Chapter Conference (Vol. 2008, pp. 129e136). Ciocarlie, M., Hicks, F. M., & Stanford, S. (2013). Kinetic and dimensional optimization for a tendon-driven gripper. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 2751e2758). Coelho, J., Piater, J., & Grupen, R. (2001). Developing haptic and visual perceptual categories for reaching and grasping with a humanoid robot. Robotics and Autonomous Systems, 37(2), 195e218.

15

Cutkosky, M. R. (1989). On grasp choice, grasp models and the design of hands for manufacturing tasks. IEEE Transactions on Robotics and Automation, 5(3), 269e279. De-An, Z., Jidong, L., Wei, J., Ying, Z., & Yu, C. (2011). Design and control of an apple harvesting robot. Biosystems Engineering, 110(2), 112e122. Detry, R., Kraft, D., Kroemer, O., Bodenhagen, L., Peters, J., Kru¨ger, N., et al. (2011). Learning grasp affordance densities. Journal of Behavioral Robotics, 2, 1e17. Edan, Y. (1999). Food and agricultural robots. In S. Y. Nof (Ed.), The Handbook of Industrial Robotics (2nd ed., pp. 1143e1155). John Wiley and Sons. Edan, Y., Haghighi, K., Stroshine, R., & Cardenas-Weber, M. (1992). Robot gripper analysis: finite element modelling and optimization. Applied Engineering in Agriculture, 8(4), 563e570. Edan, Y., Rogozin, D., Flash, T., & Miles, G. E. (2000). Robotic melon harvesting. IEEE Transactions on Robotics and Automation, 16(6), 831e835. Eizicovits, D., & Berman, S. (2014a). Efficient sensory-grounded grasp pose quality mapping for gripper design and online grasp planning. Robotics and Autonomous Systems, 62(8), 1208e1219. Eizicovits, D., & Berman, S. (2014b). Grasp quality measures based on point cloud input. In International Conference of Agricultural Engineering (AgEng). Zurich: Switzerland. Fantoni, G., Gabelloni, D., & Tilli, J. (2012). How to design new grippers by analogy. Report: Department of Mechanical, Nuclear and Production Engineering, University of Pisa. Foglia, M. M., & Reina, G. (2006). Agricultural robot for radicchio harvesting. Journal of Field Robotics, 23(6e7), 363e377. Gauchel, W., & Saller, S. (2012). Adaptive gripper jaws for high-value crops harvesting. Germany: Proceedings 8th International Fluid Power Conference, Dresden. March 26-28. Giannaccini, M. E., Dogramadzi, S., & Pipe, T. (2011). Solutions for a variable compliance gripper design. In Proceedings of the 12th Annual Conference on Towards autonomous robotic systems (TAROS'11) (pp. 410e411). Berlin, Heidelberg: Springer. Gorce, P., & Fontaine, J. G. (1996). Design methodology approach for flexible grippers. Journal of Intelligent and Robotic Systems, 15, 307e328. Hayashi, S., Shigematsu, K., Yamamoto, S., Kobayashi, K., Kohno, Y., Kamata, J., et al. (2010). Evaluation of a strawberryharvesting robot in a field test. Biosystems Engineering, 105(2), 160e171. Hemming, J., Bac, W. C., & van Tuijl, B. A. J. (2011). CROPS project deliverable 5.1: Report with design objectives and requirements for sweet-pepper harvesting. Wageningen, The Netherlands: Wageningen UR Greenhouse Horticulture. Hemming, J., Bac, W. C., van Tuijl, B. A. J., Barth, R., Bontsema, J., & Pekkeriet, E. (2014). A robot for harvesting sweet-pepper in greenhouses. In International Conference on Agricultural Engineering (AgEng), Zurich, 6e10 July. Hemming, J., Ruizendaal, J., Hofstee, J. W., & van Henten, E. J. (2014). Fruit detectability analysis for different camera positions in sweet-pepper. Sensors, 14(4), 6032e6044. Hemming, J., van Tuijl, B. A. J., Gauchel, W., & Wais, E. (2014). Field test of different end-effectors for robotic harvesting of sweetpepper. Personal communications. In International Horticultural Congress (IHC2014), Brisbane, Australia. Wageningen, The Netherlands: Wageningen UR Greenhouse Horticulture. van Henten, E. J., Hemming, J., van Tuijl, B. A. J., Kornet, J. G., Bontsema, J., Meuleman, J., et al. (2002). An autonomous robot for harvesting cucumbers in greenhouses. Journal of Autonomous Robots, 13(3), 241e258. Irie, N., Taguchi, N., Horie, T., & Ishimatsu, T. (2009). Asparagus harvesting robot coordinated with 3-d vision sensor. In IEEE International Conference on Industrial Technology (ICIT) (pp. 1e6).

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016

16

b i o s y s t e m s e n g i n e e r i n g x x x ( 2 0 1 6 ) 1 e1 6

Kapach, K., Barnea, E., Mairon, R., Edan, Y., & Ben-Shahar, O. (2012). Computer vision for fruit harvesting robots e state of the art and challenges ahead. International Journal of Computer Vision Robot, 3(1/2), 4e34. Kitamura, S., & Oka, K. (2005). Recognition and cutting system of sweet pepper for picking robot in greenhouse horticulture. In IEEE International Conference on Mechatronics and Automation (Vol. 4, pp. 1807e1812). Li, Z., Li, P., & Liu, J. (2011). Physical and mechanical properties of tomato fruits as related to robot's harvesting. Journal of food engineering, 103(2), 170e178. Li, Z., Liu, J., Li, P., & Li, W. (2008). Analysis of workspace and kinematics for a tomato harvesting robot. In IEEE International Conference on Intelligent Computation Technology and Automation (Vol. 1, pp. 823e827). Li, Z., Li, P., Yang, H., & Wang, Y. (2013). Stability tests of twofinger tomato grasping for harvesting robots. Biosystems Engineering, 116(2), 163e170. Ling, P., Ehsani, R., Ting, K. C., Chi, Y. T., Ramalingam, N., Klingman, M. H., et al. (2004). Sensing and end-effector for a robotic tomato harvester. In American Society of Agricultural Engineers (ASAE) Annual Meeting. Paper No. 043088. Monta, M., Kondo, N., & Shibano, Y. (1995). Agricultural robot in grape production system. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 2504e2509). Monta, M., Kondo, N., & Ting, K. C. (1998). End-effectors for tomato harvesting robot. Journal of Artificial Intelligence for Biology and Agriculture, 12, 11e25. Moreno, P., Hornstein, J., & Santos-Victor, J. (2011). Learning to grasp from point clouds. Technical report. Vislab-TR001/2011, Department of Electrical and Computer Engineering, Instituto cnico & Instituto de Sistemas e Robotica, Portugal. Superior Te Murakami, N., Otsuka, K., Inoue, K., & Sugimoto, M. (1995). Robotic cabbage harvester. In Proceedings of the International Symposium on Automation and Robotics in Bio Production and Processing (Vol. 2, pp. 25e32). Nguyen, V. (1986). Constructing force-closure grasps. In IEEE Int. Conf. Robot. Autom (Vol. 3, pp. 1368e1373). Pettersson, A., Davis, S., Gray, J. O., Dodd, T. J., & Ohlsson, T. (2010). Design of a magnetorheological robot gripper for handling of delicate food products with varying shapes. Journal of Food Engineering, 98(3), 332e338. Raghav, V., Kumar, J., & Senger, S. S. (2012). Design and optimization of robotic gripper: a Review. In Proceedings of the National Conference on Trends and Advances in Mechanical Engineering. Reed, J. N., Miles, S. J., Butler, J., Baldwin, M., & Noble, R. (2001). AE e Automation and Emerging Technologies: automatic

mushroom harvester development. Journal of Agricultural Engineering Research, 78(1), 15e23. Roa, M. A., Chen, Z., Staal, I. C., Muirhead, J. N., Maier, A., Pleintinger, B., et al. (2014). Towards a functional evaluation of manipulation performance in dexterous robotic hand design. In IEEE International Conference on Robotics and Automation (ICRA) (pp. 6800e6807). Roa, M. A., Hertkorn, K., Zacharias, F., Borst, C., & Hirzinger, G. (2011). Graspability map: a tool for evaluating grasp capabilities. In IEEE International Conference on Intelligent Robots and Systems (IROS), San Francisco, USA (pp. 1768e1774). Sam, R., & Nefti, S. (2011). A novel, flexible and multifunctional handling device based on Bernoulli principle. In International Conference on System Engineering and Technology (pp. 166e171). Saravanan, R., Ramabalan, S., Ebenezer, N., & Dharmaraja, C. (2009). Evolutionary multi criteria design optimization of robot grippers. Applied Soft Computing, 9(1), 159e172. Spiers, A., Baillie, S., Pipe, T., & Persad, R. (2012). Experimentally driven design of a palpating gripper with minimally invasive surgery considerations. In IEEE Haptics Symposium (HAPTICS) (pp. 261e266). Streusand, D. B., & Turner, C. J. (2011). A design methodology based process for robotic gripper design. In American Society of Mechanical Engineers (ASME) International Design Engineering Technical Conferences and Computers (pp. 345e358). Tanigaki, K., Fujiura, T., Akase, A., & Imagawa, J. (2008). Cherryharvesting robot. Computers and Electronics in Agriculture, 63(1), 65e72. van Tuijl, B. A. J. (2015). The design and field test of an end-effector for a sweet-pepper harvesting robot. Internal report, Wageningen UR. Wageningen, The Netherlands: Greenhouse technology group. van Tuijl, B. A. J., Wais, E. (2014). NL Patent Pending, Harvesting device. van Tuijl, B. A. J., Wais, E., & Edan, Y. (2013). Methodological design of an end-effector for a horticultural robot. In Proc. of 4th Israeli Conference on Robotics (ICR), Israel. Walsh, S. (1984). Gripper design: guidelines for effective results. Manufacturing Engineering, 93, 53e55. Wolf, I., Bar-Or, J., Edan, Y., & Peiper, U. M. (1990). Developing grippers for a melon harvesting robot. In American Society of Agricultural Engineers (ASAE). Paper No. 90e7514. Zaki, A. M., Soliman, A. M., Mahgoub, O. A., & El-Shafei, A. M. (2010). Design and implementation of efficient intelligent robotic gripper. In IEEE International Conference on Modelling, Identification and Control (ICMIC) (pp. 710e716).

Please cite this article in press as: Eizicovits, D., et al., Integration of perception capabilities in gripper design using graspability maps, Biosystems Engineering (2016), http://dx.doi.org/10.1016/j.biosystemseng.2015.12.016