- Email: [email protected]
A Timing Control Model for H-R Synchronization
10th IFAC Symposium on Robot Control International Federation of Automatic Control September 5-7, 2012. Dubrovnik, Croatia
A Timing Control Model for H-R Synchronization Roy Someshwar, Joachim Meyer, Yael Edan
Dept. of Industrial Engineering and Management, Ben-Gurion University of Negev, Beersheba, Israel 84105; e-mail: ({royso, joachim,yael}@bgu.ac.il) Abstract: A timing control model is presented as a possible way to control a Human-Robot system executing a collaborative task in which a high level of synchronization among the agents is desired. The influencing parameters that affect the process of synchronization were analyzed. The performance of the model was evaluated based on costs of the waiting times of each of the agents at the spatial point of handover. The model was applied to two case-studies of dynamic H-R collaborative scenarios. Results indicate that for certain scenarios, the timing control model is preferable over sensor based control. The timing control model provides a balance between speed and moderate accuracy requirements. Keywords: Human-Robot Collaboration, Synchronization, Modeling, Human-Robot Interaction, Robots in Manufacturing Industry, Time-Critical Cooperative Task
1. INTRODUCTION
model that considers the task-specific, need-specific and casespecific requirements of synchronization in H-R collaboration. Such a model can help determine the best mode of collaboration for jointly-efficient synchronization. The general framework for such a model has been presented in (Someshwar et al. 2012), where we describe different control models: timing control, sensor control and adaptive control (Someshwar and Gontar 2012).
Human civilization has developed largely because of humans' ability to collaborate efficiently with others. A Human-Robot (H-R) system, similarly, gains its power and functionality from the quality of the collaboration between the human and the robot. It benefits from the advantages of the robot’s accuracy and high yield and the human’s ability to intelligently respond to dynamic changing situations. H-R collaboration (HRC) research has received considerable attention in the field of assistive robotics for personal care (Choi et al. 2009; Edsinger and Kemp 2007), space robotics (Crandall et al. 2005; Goodrich and Schultz 2007) and social robotics (Glasauer et al. 2010; Hoffman and Breazeal 2007; Hoffman and Breazeal 2006).
The objective of the current paper is the critical analysis of parameters influencing the synchronization process in an H-R system that is controlled by a timing control model. The model is implemented in two case-studies of dynamic H-R collaborative scenarios. Speed and accuracy are two equally important task objectives (sometimes also defined as requirements) for H-R synchronization. However, there is often a trade-off between the two requirements (Someshwar et al. 2012).Processing the large amount of information acquired by the sensors to generate a precise output is time-consuming. This causes delays in the control system, leading to jitters and in some cases inefficient control (Cervin and Eker 2000; Mart 2007). Moreover, the accuracy of sensor data depends on several variables, such as sensor resolution, sensitivity, time-delay, response time etc. So, the information received from a sensor is inherently noisy and may not always be completely reliable.
Lately, the importance and potential of HRC has also been realized in industrial robotics (Stanescu et al. 2008), in contrast to the past when humans were largely excluded from the working sphere of a robot. The current trend of the industrial sector is to develop smart, flexible, and easily customizable robots for diverse tasks involving close humanrobot cooperation, sharing both work and time-space (Duan and Tan 2011; Duan et al. 2012; Tan et al. 2009). For efficient and fluent H-R collaboration, natural harmonization of the human and the robot actions is needed to achieve smooth real-time coordination between them. This is defined as H-R Synchronization. Without synchronization, the joint-efficiency of the collaborative system can be extremely poor. Joint-efficiency is defined as the net throughput of the H-R system (i.e., the team work) for the given task (and not their individual throughput or efficiency).
2. METHODOLOGY 2.1 The Model In the timing control model, the robot actions are controlled by a timer. There are no sensors to control the timing of actions or the action sequence. The robot repeats its predefined set of actions after a fixed interval of time which is pre-defined by the operator. Time is the only parameter that determines the execution of the next robot action. The operator has the option to define (or change) the time interval
However, H-R Synchronization still remains a challenge that is common in every H-R collaborative system (Cakmak et al. 2011; Glasauer et al. 2010). This is because the synchronization process is influenced by several sets of dynamic parameters related to the environment, the task and the agent (Someshwar et al. 2012). This creates the need for a 978-3-902823-11-3/12/$20.00 © 2012 IFAC
698
10.3182/20120905-3-HR-2030.00134
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
between successive events of the robot. The robot can also be switched on and off by the operator. An example of such a scenario from the manufacturing sector will be a pick and place robot, working along with a human in an assembly line. Such kinds of scenarios are very common in the manufacturing and assembly industries
2.4 Tested parameters and scenarios The simulated scenario consisted of a non-buffered, two-agent (human and robot) system executing a dynamic collaborative task that is repetitive (i.e., periodic) in nature with each agent responsible for an exclusive task. Exclusive task implies that each agent is responsible for an individual task that is independent of its partner and collaboration is required only at the pre-specified spatial point of handover at certain interval of time in every action sequence.
2.2 Performance Measures An H-R collaborative task where the human and the robot are physically collaborating with each other requires the accurate anticipation of the spatial and temporal point of handover for an efficient synchronization of the process. This paper deals with the analysis of the timing component of this handover to improve the fluency of synchronization. We assume that another algorithm of the robot control system takes care perfectly of the spatial component of this mutual handover.
In this paper, we focused on the analyses of two types of agent-intrinsic parameters of an H-R collaborative system (Someshwar et al. 2012). They are Experienced or Novice user and System Variables of the collaborating agents (both human and robot).The intrinsic system variable for a robot takes into account all inherent errors in sensor data, mechanical constraints, inertial properties etc.
In such a scenario, if either of the collaborative agents is early or late in arriving at the pre-defined spatial point of handover, one will have to wait for the other. From the timing perspective, the goal of the model is to minimize this waiting time of each of the agents (human and/or robot) to improve the fluency of synchronization. The performance of the model is evaluated by attaching a cost component to these idle/ waiting times for each of the participating agents and then calculating this cost under various dynamic scenarios. Depending upon the needs and requirements of the dynamic scenarios, each of the agents is assigned with a different cost function and different weights of the influencing parameters involved in the collaborative task.
On the other hand, the intrinsic system variable of human takes into account most psychological and neurological aspects of human involved in a time-critical collaborative process including perceptual latency (Seifried et al. 2010), temporal preparation (Bausenhart et al. 2010), and rhythm of operation (Sanabria et al. 2011; Fraenkel 1994) All these system variables of the agents, which play a significant role in the H-R synchronization process (Someshwar et al. 2012), were expressed as the two influencing parameters in the simulation - HSD (human standard deviation) and RSD (robot standard deviation), taking into account the individual system variables of human and robot respectively.
Simulation analyses were performed to determine the critical range of values of the influencing parameters. Additionally, the “critical zones” of the output cost curve were derived. The cost-curve critical zones are the parts of the curve where the timing control model can be an efficient mode of control, providing high joint-efficiency and throughput during the synchronization process resulting in minimal cost.
This approach is possible because the agent-intrinsic parameters (robot and human) are responsible for the inherent variances in the total time to complete one round of predefined action. Hence, in the simulation, these parameters were considered by quantifying them as variances (HSD and RSD) of the human total time (HTT) and robot total time (RTT). The variance of the HTT will be larger if the human is a novice and lower for a trained human. Similarly, a robot with low mechanical constraints will have a lower variance in RTT and a robot with unreliable sensors will have greater variance.
2.3 Analysis Methods There are three common ways to analyze an H-R collaborative system (Someshwar et al. 2012) namely – simulation analysis, analytical analysis and graphical model analysis. In this work, simulation analysis was used to test the influencing parameters of the collaborating system and to analyse the cost of the H-R system in dynamic scenarios. The waiting cost of each of the collaborating agents was considered for evaluation. The simulation was developed in Matlab R2010a.
Similarly, when an averagely trained human works with the robot, we still consider differences in HSD, considering the influence of the human's perceptual latency, temporal preparation etc. on the collaborative task. This is because the level of preparedness or the skill set of the human has a direct influence on the system variables. In the following casestudies section we present more specifically all the influencing parameters that were analysed in the simulation.
By applying the Monte Carlo method, each of the scenarios was simulated for 1000x1000 times for each combination of the influencing parameters (Human standard deviation, HSD, and Robot standard deviation, RSD). The values of each of the variables (Robot total time, RTT and Human total time, HTT) were randomly sampled each time to compute the results. Results indicate the average human waiting cost for 1000x1000 iterations under different scenarios (Figs. 2 and 3).
2.5 Case Studies Two case-studies were evaluated. In both, a typical industrial scenario was considered where the human and the robot collaborate in a repetitive periodic task, sharing both work and time space. This task was selected since it requires both 699
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
speed and moderate accuracy which we claim can be achieved by timing control.
It is also assumed that when one of the collaborating agents does not arrive at this pre-defined spatial point of handover, the other waits at that point until the handover is executed. This time has been defined as the waiting time for the human and the idle/unproductive time for the robot. So, the basic protocol between the collaborating agents is that whoever arrive first waits for the other at the spatial point of handover until the handover is executed successfully. This protocol is suitable for robots which are not meant for multi-tasking or for processes where the system cannot be recalibrated so frequently.
The two case-studies differ in the protocol of the collaboration mode between the agents. In the first case, whoever arrive first waits for the other at the spatial point of handover until the handover is executed successfully. As a result, there is no cumulative error in this mode of collaboration, and hence it is suitable for a robot that cannot be easily recalibrated. In the second case-study, the robot never waits for the human at the point of handover but continues its cycle of periodic movement irrespective of the collaborating partner arriving or not arriving at the right time. The human, on the other hand, waits for the robot if it happens to arrive earlier. This means, if the handover is unsuccessful in the first attempt, then the human waits for the second turn of the robot to repeat the same action. Such a protocol can be very useful for a scenario where a multi-tasking robot is employed that is also responsible for another job other than collaborating with the human. However, this mode of collaboration may give rise to cumulative error, and hence recalibration of the system is necessary when a certain threshold level of waiting time is crossed to maintain the fluency of synchronization.
The agent-intrinsic parameters of the collaborating agents as described in the previous section have been considered in this handover. Some variance is assumed in the total time required by the agent (human and robot) to complete one set of actions. Considering all the factors as mentioned above, the cost of waiting for the human W, is calculated as: W = R(t) – H(t) where R(t) and H(t) are the times taken by the robot and human respectively to complete one round of action. The scenario was simulated for RTT (Robot Total Time) and HTT (Human Total Time) with mean values of 30 sec. The agentintrinsic influencing parameters, responsible for the inherent differences in the total time taken by the agents in every round, were quantified in terms of HSD (Human Standard Deviation) and RSD (Robot Standard Deviation).
3. CASE-STUDY I 3.1 Model Implementation A typical industrial scenario was assumed where a human and a robot work collaboratively (Fig.1). From the right, the robot (B) picks up a metal block from an assembly line (A) and delivers it directly into the hands of the human (C). The human receives it and inspects the quality of the processed block and thereafter places it on another assembly line (D) or in the default section. The process continues repetitively from right to left over time.
3.2 Analyses Let us assume a collaborating scenario where the human is highly trained, therefore having a low variance, HSD=1 and the robot has a lower mechanical constraint giving rise to minor variance in its motion, quantified in terms of variance as RSD=3.
It is assumed that the mean time required for each of the agents (the human and the robot) individually to complete one round of the process is 30 seconds. This means, the robot and human repeat the pre-defined set of actions every 30 seconds, i.e., the physical collaboration at the spatial point of handover should ideally occur every 30 seconds.
Fig2: Average human waiting cost for 1000x1000 operations against HTT for RSD=3 and HSD=1 with Timing based control (dotted line) and RSD=8, HSD=4 with Sensor based control (bold blue line)
Fig1: A typical H-R collaborative scenario (A B C D) that has been considered for the case study.
700
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
Figure 2 shows the average human cost of waiting (W) for a simulation with the scenario settings, calculated for each single event using the equation as aforementioned, averaged over 1000x1000 times of operation (number of collaborative events during a periodic process), and plotted against the HTT, that is the total human time required to complete one round of action.
action depends on the information it received from its sensors that is sensing the state of the human and the robot. The various factors that affect the output of a sensor, such as sensor resolution, range of operation, response time, computation time and sensitivity, have been quantified in the simulation by varying the robot variance, RSD. In the simulation, we considered that the robot is controlled with low quality sensors, and as a result the robot total time, RTT variance is high with RSD = 8.
Analyzing the cost curve represented with dotted line, we find that the human cost is lower when the collaborating human is closer to the mean value. Overall, the cost is lower for HTT values between the range 27 and 35 sec. So, this can be considered as the best zone or “critical zone” of collaboration. In other words, the current collaborative task can have a higher level of fluency with better synchronization if the human takes 27-35 sec to complete the task in every round.
This scenario was simulated and the average human waiting cost was calculated for 1000x1000 iterations as shown by the blue line in Fig 2. The graph clearly shows that the waiting cost in this case is much higher than in the earlier case as represented by the dotted line. The values of the RSD were varied from 6 to 10 with HSD fixed at 3 and the resulting trends were evaluated. In all cases, the nature of the graph did not change, and it remained more or less the same as the blue line shown in the figure. So, in such a scenario, the best collaboration is in the range of 33-40 sec (higher than the mean value) and the cost increases as the human time drops beneath its mean value. This implies that when the robot has an unpredictable speed in its sequence of action, it is always better to collaborate as slow as possible to improve the fluency of synchronization.
Similarly, the synchronization will be worse if the human is works in the range of 23 – 27 sec, with cost reaching its maximum when time taken by the human to complete one round of action is around 26 sec. In other words, when the human is swifter in its action than is required for the activity, the human may feel that he/she is increasing the productivity of the collaborative task, but actually the fluency and hence the throughput of the task decreases. Humans tend to speed up under certain kinds of rhythm (Sanabria et al. 2011). Therefore, a trained person collaborating with a robot may involuntarily speed up the sequence of actions, which can bring him into the range of 2327 sec, which can be regarded as the zone of worse synchronization. This will result in a loss of fluency. Hence, the fluency of H-R synchronization can be maintained if the human avoids speeding up his motion.
This analysis demonstrates that timing-based control can provide better synchronization than sensor based control with poor quality sensors. In addition, the timing based control can be faster than sensor based control because it can eliminate possible delays that occur due to the time required in sensing and acquiring complex information, other than saving the computation time that is required for processing such information. Analyzing the cost curve, it can also be concluded that human waiting time varies little within the critical zone of collaboration. Based on these two arguments, it can be said that timingbased control can provide a balance between the requirements of speed and moderate accuracy.
The overall nature of the cost curve with respect to the change of parameters HSD and RSD was also checked and it was found that as the variance of one of the collaborating agent increases, the human cost also increases proportionately when HTT drops below its mean value. Better synchronization is achieved in such scenarios for the region where HTT is greater than its mean value. On the other hand, as the variance of any of the collaborating agents decreases, better synchronization is achieved for the region where HTT is closer to its mean value.
4. CASE-STUDY II 4.1 Model Implementation
This is not a generalized conclusion that is applicable to all scenarios. In Case-Study II presented below, the fluency of synchronization depends on a different set of conditions, supporting the argument that a task-specific, need-specific and case-specific synchronization model is needed for developing an efficient and natural H-R collaborative system (Someshwar et al. 2012)
All elements and variables of the dynamic environment in this scenario are the same as in Case 1. However, the mode of collaboration between the agents is different. This protocol is different in the sense that in this case, if the robot arrives earlier, it does not wait for the human at the point of handover but continues its periodic movement. As a result, if the handover is unsuccessful in the first attempt, the robot has an unproductive time for that round.
3.3 Comparative Analyses In addition to the timing-based control with RSD=3 and HSD=1, we also looked at a sensor-based control scenario. The basic protocol of collaboration remains the same as earlier. The only difference here is that the robot is not pre-fed with the time interval between successive events. The robot´s
The human, on the other hand, waits for the robot if it happens to arrive earlier. This means, if the handover is not successful in the first attempt, then the human waits for the second turn of the robot to repeat the same action.
701
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
functionality is added to the H-R system, the fluency of synchronization can be drastically improved. Such a scenario was simulated and the result has been plotted using the blue line as shown in Fig 3. Clearly the cost of such a system is much lower than the previous case for a wide range of values. The best zone of collaboration in this case is between 22 and 31 sec, a rather broad range. Normally, a novice user collaborating with the system without having much training can have a high chance of arrhythmic or unpredictable movement. Such a broad range defined as the best zone of collaboration is essentially suitable for a novice user collaborating with the system. Thus, even a novice user can maintain a high fluency of synchronization with the robot if the system has the ability to recalibrate itself. Fig3: The cost curve for the collaborative system with cumulative effect (dotted line) and with system recalibration (blue line)
4. CONCLUSIONS AND FUTURE WORK The timing control model can increase the flexibility and joint-efficiency of human-robot synchronization. It is of course not always the most efficient way to control an H-R system. However, the biggest strength of this model is its balance between speed and moderate accuracy. Additionally, the fact that there are no sensors in the system reduces its cost and increases reliability and trust in the system (no case of sensor failure). The results also indicated that for certain scenarios, the timing control model is preferable over sensor based control.
If missed encounters happen consecutively, the protocol may generate an inherent cumulative error in the process, causing the collaborative process to become completely arrhythmic. 4.2 Analyses This scenario was simulated for HSD=2 and RSD=4 and the cost curve has been plotted using the dotted lines as shown in Fig. 3 above. The HTT within the range 15-20 sec is basically the region when the human is attempting to arrive at the point of handover before the robot and the range from 35-40 sec is the region where in most cases the human arrives late and hence misses the handover in the first turn, thereby waiting for the second turn and adding a cumulative cost to the process. From the graph it is clear that if the human arrives too early to be sure that it does not miss the first turn, the cost is higher than the optimum value, but it is always lower than the cost of handover when it managed to accomplish the task in the second turn.
This model takes into account most of the context-dependent variables and hence it can minimize the time each agent in the interaction has to wait for the other and each agent's need to adapt the actions of the other, thereby providing better fluency of handovers, possibly increasing the net-throughput of the Human-Robot system. This is a small step towards making human robot interaction fluent and natural. The developed model can also be used as an analytical method for analyzing human-robot systems to obtain better synchronization among them. It can be further developed to be used as a tool to determine the conditions under which each kind of synchronization model can be an efficient control method. The current research also aims to develop a combined model that considers all the influencing parameters and maps them with the system requirements and best-fit solution for H-R synchronization.
So, it can be said that if a system does not have the ability to recalibrate itself, it is always better to be on the left side of the graph, i.e., maintain a tendency to arrive faster and before the robot. However, the best zone of collaboration is in the region between 20 and 25 sec for the scenario represented by the cost curve in dotted lines.
The study is a first step towards the development of a comprehensive framework for the specification of humanrobot coordination mechanisms.
It should be noted that this region was considered as the worst zone of collaboration in the cost curve of Case-Study I proving that a task-specific, need-specific and case-specific synchronization model is needed for developing an efficient and natural H-R collaborative system.
ACKNOWLEDGMENTS
The robot may have the ability to recalibrate itself whenever it crosses a threshold level (set by the operator), where the threshold level is defined as the maximum time that is allowed for a single event of handover to be executed. If such
This research was supported by the EU funded Initial Training Network (ITN) in the Marie-Curie People Programme (FP7): INTRO (INTeractive RObotics research network), grant agreement number: 23848TRO research project and partially 702
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
supported by the Paul Ivanier Center for Robotics Research and Production Management, and by the Rabbi W. Gunther Plaut Chair in Manufacturing Engineering, Ben-Gurion University of the Negev.
Hoffman G, Breazeal C (2007) Cost-Based Anticipatory Action Selection for Human–Robot Fluency. IEEE Transactions on Robotics, 23:952-961. Hoffman G, Breazeal C (2006) What lies ahead? Expectation management in human-robot collaboration. Working notes of the AAAI Spring Symposium, pp 1-7
REFERENCES Cakmak M, Srinivasa SS, Lee MK, Sara Kiesler, Jodi Forlizzi (2011) Using spatial and temporal contrast for fluent robot-human hand-overs. Proceedings of the 6th International Conference on Human-Robot Interaction. ACM, pp 489–496
Mart P (2007) Toward Flexible Scheduling of Real-Time Control Tasks : Reviewing Basic Control Models. Proceedings of the 10th International Conference on Hybrid systems: Computation and Control, pp 710-713.
Cervin A, Eker J (2000) An Introduction to Control and Scheduling Co-Design. Proceedings of the 39th IEEE Conference on Decision and Control, 5:4865-70.
Sanabria D, Capizzi M, Correa A (2011) Rhythms that speed you up. Journal of Experimental Psychology. Human Perception and Performance, 37:236-44.
Choi YS, Chen T, Jain A, Anderson C, Glass J.D, Kemp C.C, (2009) Hand it over or set it down: A user study of object delivery with an assistive mobile manipulator. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp 736-743.
Stanescu a. M, Nita A, Moisescu M.A., Sacala I.S. (2008) From industrial robotics towards intelligent robotic systems. IS’08, Proceedings of the 4th International IEEE Conference on Intelligent Systems, pp 6–73. Tan JTC, Duan F, Zhang Y, Watanabe K., Kato R., Arai T. (2009) Human-robot collaboration in cellular manufacturing: Design and development. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 29-34.
Crandall JW, Goodrich MA, Olsen DR, Nielsen CW (2005) Validating Human–Robot Interaction Schemes in Multitasking Environments. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 35:438-449.
Someshwar R, Gontar V (2012) An Adaptive H-R Synchronization Model: Towards Developing “Conscious Robots”. CogSys 2012, 5th International Conference in Cognitive Systems pp 109.
Duan F, Tan J (2011) A new human-robot collaboration assembly system for cellular manufacturing. 30th Chinese Control Conference (CCC), pp 5468-5473.
Someshwar R, Meyer J, Edan Y (2012) Models and Methods for H-R Synchronization. INCOM 2012, 14th IFAC International Symposium on Information Control Problems in Manufacturing.
Duan F, Tan J, Tong J, Kato R (2012) Application of the Assembly Skill Transfer System in an Actual Cellular Manufacturing System. IEEE Transactions on Automation Science and Engineering, 9:31-41. Edsinger A, Kemp CC (2007) Human-Robot Interaction for Cooperative Manipulation: Handing Objects to One Another. RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication, pp 1167-1172. Ferrell CB (1998) A Motivational System for regulating Human-Robot Interaction. Proceedings of the 10th Conference on AAAI, pp 54-62 Glasauer S, Huber M, Basili P (2010) Interacting in time and space: Investigating human-human and human-robot joint action. RO-MAN 2010, pp 252–257. Goodrich M a., Schultz AC (2007) Human-Robot Interaction: A Survey. Foundations and Trends in HumanComputer Interaction, 1:203-275.
703
A Timing Control Model for H-R Synchronization Roy Someshwar, Joachim Meyer, Yael Edan
Dept. of Industrial Engineering and Management, Ben-Gurion University of Negev, Beersheba, Israel 84105; e-mail: ({royso, joachim,yael}@bgu.ac.il) Abstract: A timing control model is presented as a possible way to control a Human-Robot system executing a collaborative task in which a high level of synchronization among the agents is desired. The influencing parameters that affect the process of synchronization were analyzed. The performance of the model was evaluated based on costs of the waiting times of each of the agents at the spatial point of handover. The model was applied to two case-studies of dynamic H-R collaborative scenarios. Results indicate that for certain scenarios, the timing control model is preferable over sensor based control. The timing control model provides a balance between speed and moderate accuracy requirements. Keywords: Human-Robot Collaboration, Synchronization, Modeling, Human-Robot Interaction, Robots in Manufacturing Industry, Time-Critical Cooperative Task
1. INTRODUCTION
model that considers the task-specific, need-specific and casespecific requirements of synchronization in H-R collaboration. Such a model can help determine the best mode of collaboration for jointly-efficient synchronization. The general framework for such a model has been presented in (Someshwar et al. 2012), where we describe different control models: timing control, sensor control and adaptive control (Someshwar and Gontar 2012).
Human civilization has developed largely because of humans' ability to collaborate efficiently with others. A Human-Robot (H-R) system, similarly, gains its power and functionality from the quality of the collaboration between the human and the robot. It benefits from the advantages of the robot’s accuracy and high yield and the human’s ability to intelligently respond to dynamic changing situations. H-R collaboration (HRC) research has received considerable attention in the field of assistive robotics for personal care (Choi et al. 2009; Edsinger and Kemp 2007), space robotics (Crandall et al. 2005; Goodrich and Schultz 2007) and social robotics (Glasauer et al. 2010; Hoffman and Breazeal 2007; Hoffman and Breazeal 2006).
The objective of the current paper is the critical analysis of parameters influencing the synchronization process in an H-R system that is controlled by a timing control model. The model is implemented in two case-studies of dynamic H-R collaborative scenarios. Speed and accuracy are two equally important task objectives (sometimes also defined as requirements) for H-R synchronization. However, there is often a trade-off between the two requirements (Someshwar et al. 2012).Processing the large amount of information acquired by the sensors to generate a precise output is time-consuming. This causes delays in the control system, leading to jitters and in some cases inefficient control (Cervin and Eker 2000; Mart 2007). Moreover, the accuracy of sensor data depends on several variables, such as sensor resolution, sensitivity, time-delay, response time etc. So, the information received from a sensor is inherently noisy and may not always be completely reliable.
Lately, the importance and potential of HRC has also been realized in industrial robotics (Stanescu et al. 2008), in contrast to the past when humans were largely excluded from the working sphere of a robot. The current trend of the industrial sector is to develop smart, flexible, and easily customizable robots for diverse tasks involving close humanrobot cooperation, sharing both work and time-space (Duan and Tan 2011; Duan et al. 2012; Tan et al. 2009). For efficient and fluent H-R collaboration, natural harmonization of the human and the robot actions is needed to achieve smooth real-time coordination between them. This is defined as H-R Synchronization. Without synchronization, the joint-efficiency of the collaborative system can be extremely poor. Joint-efficiency is defined as the net throughput of the H-R system (i.e., the team work) for the given task (and not their individual throughput or efficiency).
2. METHODOLOGY 2.1 The Model In the timing control model, the robot actions are controlled by a timer. There are no sensors to control the timing of actions or the action sequence. The robot repeats its predefined set of actions after a fixed interval of time which is pre-defined by the operator. Time is the only parameter that determines the execution of the next robot action. The operator has the option to define (or change) the time interval
However, H-R Synchronization still remains a challenge that is common in every H-R collaborative system (Cakmak et al. 2011; Glasauer et al. 2010). This is because the synchronization process is influenced by several sets of dynamic parameters related to the environment, the task and the agent (Someshwar et al. 2012). This creates the need for a 978-3-902823-11-3/12/$20.00 © 2012 IFAC
698
10.3182/20120905-3-HR-2030.00134
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
between successive events of the robot. The robot can also be switched on and off by the operator. An example of such a scenario from the manufacturing sector will be a pick and place robot, working along with a human in an assembly line. Such kinds of scenarios are very common in the manufacturing and assembly industries
2.4 Tested parameters and scenarios The simulated scenario consisted of a non-buffered, two-agent (human and robot) system executing a dynamic collaborative task that is repetitive (i.e., periodic) in nature with each agent responsible for an exclusive task. Exclusive task implies that each agent is responsible for an individual task that is independent of its partner and collaboration is required only at the pre-specified spatial point of handover at certain interval of time in every action sequence.
2.2 Performance Measures An H-R collaborative task where the human and the robot are physically collaborating with each other requires the accurate anticipation of the spatial and temporal point of handover for an efficient synchronization of the process. This paper deals with the analysis of the timing component of this handover to improve the fluency of synchronization. We assume that another algorithm of the robot control system takes care perfectly of the spatial component of this mutual handover.
In this paper, we focused on the analyses of two types of agent-intrinsic parameters of an H-R collaborative system (Someshwar et al. 2012). They are Experienced or Novice user and System Variables of the collaborating agents (both human and robot).The intrinsic system variable for a robot takes into account all inherent errors in sensor data, mechanical constraints, inertial properties etc.
In such a scenario, if either of the collaborative agents is early or late in arriving at the pre-defined spatial point of handover, one will have to wait for the other. From the timing perspective, the goal of the model is to minimize this waiting time of each of the agents (human and/or robot) to improve the fluency of synchronization. The performance of the model is evaluated by attaching a cost component to these idle/ waiting times for each of the participating agents and then calculating this cost under various dynamic scenarios. Depending upon the needs and requirements of the dynamic scenarios, each of the agents is assigned with a different cost function and different weights of the influencing parameters involved in the collaborative task.
On the other hand, the intrinsic system variable of human takes into account most psychological and neurological aspects of human involved in a time-critical collaborative process including perceptual latency (Seifried et al. 2010), temporal preparation (Bausenhart et al. 2010), and rhythm of operation (Sanabria et al. 2011; Fraenkel 1994) All these system variables of the agents, which play a significant role in the H-R synchronization process (Someshwar et al. 2012), were expressed as the two influencing parameters in the simulation - HSD (human standard deviation) and RSD (robot standard deviation), taking into account the individual system variables of human and robot respectively.
Simulation analyses were performed to determine the critical range of values of the influencing parameters. Additionally, the “critical zones” of the output cost curve were derived. The cost-curve critical zones are the parts of the curve where the timing control model can be an efficient mode of control, providing high joint-efficiency and throughput during the synchronization process resulting in minimal cost.
This approach is possible because the agent-intrinsic parameters (robot and human) are responsible for the inherent variances in the total time to complete one round of predefined action. Hence, in the simulation, these parameters were considered by quantifying them as variances (HSD and RSD) of the human total time (HTT) and robot total time (RTT). The variance of the HTT will be larger if the human is a novice and lower for a trained human. Similarly, a robot with low mechanical constraints will have a lower variance in RTT and a robot with unreliable sensors will have greater variance.
2.3 Analysis Methods There are three common ways to analyze an H-R collaborative system (Someshwar et al. 2012) namely – simulation analysis, analytical analysis and graphical model analysis. In this work, simulation analysis was used to test the influencing parameters of the collaborating system and to analyse the cost of the H-R system in dynamic scenarios. The waiting cost of each of the collaborating agents was considered for evaluation. The simulation was developed in Matlab R2010a.
Similarly, when an averagely trained human works with the robot, we still consider differences in HSD, considering the influence of the human's perceptual latency, temporal preparation etc. on the collaborative task. This is because the level of preparedness or the skill set of the human has a direct influence on the system variables. In the following casestudies section we present more specifically all the influencing parameters that were analysed in the simulation.
By applying the Monte Carlo method, each of the scenarios was simulated for 1000x1000 times for each combination of the influencing parameters (Human standard deviation, HSD, and Robot standard deviation, RSD). The values of each of the variables (Robot total time, RTT and Human total time, HTT) were randomly sampled each time to compute the results. Results indicate the average human waiting cost for 1000x1000 iterations under different scenarios (Figs. 2 and 3).
2.5 Case Studies Two case-studies were evaluated. In both, a typical industrial scenario was considered where the human and the robot collaborate in a repetitive periodic task, sharing both work and time space. This task was selected since it requires both 699
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
speed and moderate accuracy which we claim can be achieved by timing control.
It is also assumed that when one of the collaborating agents does not arrive at this pre-defined spatial point of handover, the other waits at that point until the handover is executed. This time has been defined as the waiting time for the human and the idle/unproductive time for the robot. So, the basic protocol between the collaborating agents is that whoever arrive first waits for the other at the spatial point of handover until the handover is executed successfully. This protocol is suitable for robots which are not meant for multi-tasking or for processes where the system cannot be recalibrated so frequently.
The two case-studies differ in the protocol of the collaboration mode between the agents. In the first case, whoever arrive first waits for the other at the spatial point of handover until the handover is executed successfully. As a result, there is no cumulative error in this mode of collaboration, and hence it is suitable for a robot that cannot be easily recalibrated. In the second case-study, the robot never waits for the human at the point of handover but continues its cycle of periodic movement irrespective of the collaborating partner arriving or not arriving at the right time. The human, on the other hand, waits for the robot if it happens to arrive earlier. This means, if the handover is unsuccessful in the first attempt, then the human waits for the second turn of the robot to repeat the same action. Such a protocol can be very useful for a scenario where a multi-tasking robot is employed that is also responsible for another job other than collaborating with the human. However, this mode of collaboration may give rise to cumulative error, and hence recalibration of the system is necessary when a certain threshold level of waiting time is crossed to maintain the fluency of synchronization.
The agent-intrinsic parameters of the collaborating agents as described in the previous section have been considered in this handover. Some variance is assumed in the total time required by the agent (human and robot) to complete one set of actions. Considering all the factors as mentioned above, the cost of waiting for the human W, is calculated as: W = R(t) – H(t) where R(t) and H(t) are the times taken by the robot and human respectively to complete one round of action. The scenario was simulated for RTT (Robot Total Time) and HTT (Human Total Time) with mean values of 30 sec. The agentintrinsic influencing parameters, responsible for the inherent differences in the total time taken by the agents in every round, were quantified in terms of HSD (Human Standard Deviation) and RSD (Robot Standard Deviation).
3. CASE-STUDY I 3.1 Model Implementation A typical industrial scenario was assumed where a human and a robot work collaboratively (Fig.1). From the right, the robot (B) picks up a metal block from an assembly line (A) and delivers it directly into the hands of the human (C). The human receives it and inspects the quality of the processed block and thereafter places it on another assembly line (D) or in the default section. The process continues repetitively from right to left over time.
3.2 Analyses Let us assume a collaborating scenario where the human is highly trained, therefore having a low variance, HSD=1 and the robot has a lower mechanical constraint giving rise to minor variance in its motion, quantified in terms of variance as RSD=3.
It is assumed that the mean time required for each of the agents (the human and the robot) individually to complete one round of the process is 30 seconds. This means, the robot and human repeat the pre-defined set of actions every 30 seconds, i.e., the physical collaboration at the spatial point of handover should ideally occur every 30 seconds.
Fig2: Average human waiting cost for 1000x1000 operations against HTT for RSD=3 and HSD=1 with Timing based control (dotted line) and RSD=8, HSD=4 with Sensor based control (bold blue line)
Fig1: A typical H-R collaborative scenario (A B C D) that has been considered for the case study.
700
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
Figure 2 shows the average human cost of waiting (W) for a simulation with the scenario settings, calculated for each single event using the equation as aforementioned, averaged over 1000x1000 times of operation (number of collaborative events during a periodic process), and plotted against the HTT, that is the total human time required to complete one round of action.
action depends on the information it received from its sensors that is sensing the state of the human and the robot. The various factors that affect the output of a sensor, such as sensor resolution, range of operation, response time, computation time and sensitivity, have been quantified in the simulation by varying the robot variance, RSD. In the simulation, we considered that the robot is controlled with low quality sensors, and as a result the robot total time, RTT variance is high with RSD = 8.
Analyzing the cost curve represented with dotted line, we find that the human cost is lower when the collaborating human is closer to the mean value. Overall, the cost is lower for HTT values between the range 27 and 35 sec. So, this can be considered as the best zone or “critical zone” of collaboration. In other words, the current collaborative task can have a higher level of fluency with better synchronization if the human takes 27-35 sec to complete the task in every round.
This scenario was simulated and the average human waiting cost was calculated for 1000x1000 iterations as shown by the blue line in Fig 2. The graph clearly shows that the waiting cost in this case is much higher than in the earlier case as represented by the dotted line. The values of the RSD were varied from 6 to 10 with HSD fixed at 3 and the resulting trends were evaluated. In all cases, the nature of the graph did not change, and it remained more or less the same as the blue line shown in the figure. So, in such a scenario, the best collaboration is in the range of 33-40 sec (higher than the mean value) and the cost increases as the human time drops beneath its mean value. This implies that when the robot has an unpredictable speed in its sequence of action, it is always better to collaborate as slow as possible to improve the fluency of synchronization.
Similarly, the synchronization will be worse if the human is works in the range of 23 – 27 sec, with cost reaching its maximum when time taken by the human to complete one round of action is around 26 sec. In other words, when the human is swifter in its action than is required for the activity, the human may feel that he/she is increasing the productivity of the collaborative task, but actually the fluency and hence the throughput of the task decreases. Humans tend to speed up under certain kinds of rhythm (Sanabria et al. 2011). Therefore, a trained person collaborating with a robot may involuntarily speed up the sequence of actions, which can bring him into the range of 2327 sec, which can be regarded as the zone of worse synchronization. This will result in a loss of fluency. Hence, the fluency of H-R synchronization can be maintained if the human avoids speeding up his motion.
This analysis demonstrates that timing-based control can provide better synchronization than sensor based control with poor quality sensors. In addition, the timing based control can be faster than sensor based control because it can eliminate possible delays that occur due to the time required in sensing and acquiring complex information, other than saving the computation time that is required for processing such information. Analyzing the cost curve, it can also be concluded that human waiting time varies little within the critical zone of collaboration. Based on these two arguments, it can be said that timingbased control can provide a balance between the requirements of speed and moderate accuracy.
The overall nature of the cost curve with respect to the change of parameters HSD and RSD was also checked and it was found that as the variance of one of the collaborating agent increases, the human cost also increases proportionately when HTT drops below its mean value. Better synchronization is achieved in such scenarios for the region where HTT is greater than its mean value. On the other hand, as the variance of any of the collaborating agents decreases, better synchronization is achieved for the region where HTT is closer to its mean value.
4. CASE-STUDY II 4.1 Model Implementation
This is not a generalized conclusion that is applicable to all scenarios. In Case-Study II presented below, the fluency of synchronization depends on a different set of conditions, supporting the argument that a task-specific, need-specific and case-specific synchronization model is needed for developing an efficient and natural H-R collaborative system (Someshwar et al. 2012)
All elements and variables of the dynamic environment in this scenario are the same as in Case 1. However, the mode of collaboration between the agents is different. This protocol is different in the sense that in this case, if the robot arrives earlier, it does not wait for the human at the point of handover but continues its periodic movement. As a result, if the handover is unsuccessful in the first attempt, the robot has an unproductive time for that round.
3.3 Comparative Analyses In addition to the timing-based control with RSD=3 and HSD=1, we also looked at a sensor-based control scenario. The basic protocol of collaboration remains the same as earlier. The only difference here is that the robot is not pre-fed with the time interval between successive events. The robot´s
The human, on the other hand, waits for the robot if it happens to arrive earlier. This means, if the handover is not successful in the first attempt, then the human waits for the second turn of the robot to repeat the same action.
701
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
functionality is added to the H-R system, the fluency of synchronization can be drastically improved. Such a scenario was simulated and the result has been plotted using the blue line as shown in Fig 3. Clearly the cost of such a system is much lower than the previous case for a wide range of values. The best zone of collaboration in this case is between 22 and 31 sec, a rather broad range. Normally, a novice user collaborating with the system without having much training can have a high chance of arrhythmic or unpredictable movement. Such a broad range defined as the best zone of collaboration is essentially suitable for a novice user collaborating with the system. Thus, even a novice user can maintain a high fluency of synchronization with the robot if the system has the ability to recalibrate itself. Fig3: The cost curve for the collaborative system with cumulative effect (dotted line) and with system recalibration (blue line)
4. CONCLUSIONS AND FUTURE WORK The timing control model can increase the flexibility and joint-efficiency of human-robot synchronization. It is of course not always the most efficient way to control an H-R system. However, the biggest strength of this model is its balance between speed and moderate accuracy. Additionally, the fact that there are no sensors in the system reduces its cost and increases reliability and trust in the system (no case of sensor failure). The results also indicated that for certain scenarios, the timing control model is preferable over sensor based control.
If missed encounters happen consecutively, the protocol may generate an inherent cumulative error in the process, causing the collaborative process to become completely arrhythmic. 4.2 Analyses This scenario was simulated for HSD=2 and RSD=4 and the cost curve has been plotted using the dotted lines as shown in Fig. 3 above. The HTT within the range 15-20 sec is basically the region when the human is attempting to arrive at the point of handover before the robot and the range from 35-40 sec is the region where in most cases the human arrives late and hence misses the handover in the first turn, thereby waiting for the second turn and adding a cumulative cost to the process. From the graph it is clear that if the human arrives too early to be sure that it does not miss the first turn, the cost is higher than the optimum value, but it is always lower than the cost of handover when it managed to accomplish the task in the second turn.
This model takes into account most of the context-dependent variables and hence it can minimize the time each agent in the interaction has to wait for the other and each agent's need to adapt the actions of the other, thereby providing better fluency of handovers, possibly increasing the net-throughput of the Human-Robot system. This is a small step towards making human robot interaction fluent and natural. The developed model can also be used as an analytical method for analyzing human-robot systems to obtain better synchronization among them. It can be further developed to be used as a tool to determine the conditions under which each kind of synchronization model can be an efficient control method. The current research also aims to develop a combined model that considers all the influencing parameters and maps them with the system requirements and best-fit solution for H-R synchronization.
So, it can be said that if a system does not have the ability to recalibrate itself, it is always better to be on the left side of the graph, i.e., maintain a tendency to arrive faster and before the robot. However, the best zone of collaboration is in the region between 20 and 25 sec for the scenario represented by the cost curve in dotted lines.
The study is a first step towards the development of a comprehensive framework for the specification of humanrobot coordination mechanisms.
It should be noted that this region was considered as the worst zone of collaboration in the cost curve of Case-Study I proving that a task-specific, need-specific and case-specific synchronization model is needed for developing an efficient and natural H-R collaborative system.
ACKNOWLEDGMENTS
The robot may have the ability to recalibrate itself whenever it crosses a threshold level (set by the operator), where the threshold level is defined as the maximum time that is allowed for a single event of handover to be executed. If such
This research was supported by the EU funded Initial Training Network (ITN) in the Marie-Curie People Programme (FP7): INTRO (INTeractive RObotics research network), grant agreement number: 23848TRO research project and partially 702
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
supported by the Paul Ivanier Center for Robotics Research and Production Management, and by the Rabbi W. Gunther Plaut Chair in Manufacturing Engineering, Ben-Gurion University of the Negev.
Hoffman G, Breazeal C (2007) Cost-Based Anticipatory Action Selection for Human–Robot Fluency. IEEE Transactions on Robotics, 23:952-961. Hoffman G, Breazeal C (2006) What lies ahead? Expectation management in human-robot collaboration. Working notes of the AAAI Spring Symposium, pp 1-7
REFERENCES Cakmak M, Srinivasa SS, Lee MK, Sara Kiesler, Jodi Forlizzi (2011) Using spatial and temporal contrast for fluent robot-human hand-overs. Proceedings of the 6th International Conference on Human-Robot Interaction. ACM, pp 489–496
Mart P (2007) Toward Flexible Scheduling of Real-Time Control Tasks : Reviewing Basic Control Models. Proceedings of the 10th International Conference on Hybrid systems: Computation and Control, pp 710-713.
Cervin A, Eker J (2000) An Introduction to Control and Scheduling Co-Design. Proceedings of the 39th IEEE Conference on Decision and Control, 5:4865-70.
Sanabria D, Capizzi M, Correa A (2011) Rhythms that speed you up. Journal of Experimental Psychology. Human Perception and Performance, 37:236-44.
Choi YS, Chen T, Jain A, Anderson C, Glass J.D, Kemp C.C, (2009) Hand it over or set it down: A user study of object delivery with an assistive mobile manipulator. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp 736-743.
Stanescu a. M, Nita A, Moisescu M.A., Sacala I.S. (2008) From industrial robotics towards intelligent robotic systems. IS’08, Proceedings of the 4th International IEEE Conference on Intelligent Systems, pp 6–73. Tan JTC, Duan F, Zhang Y, Watanabe K., Kato R., Arai T. (2009) Human-robot collaboration in cellular manufacturing: Design and development. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 29-34.
Crandall JW, Goodrich MA, Olsen DR, Nielsen CW (2005) Validating Human–Robot Interaction Schemes in Multitasking Environments. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 35:438-449.
Someshwar R, Gontar V (2012) An Adaptive H-R Synchronization Model: Towards Developing “Conscious Robots”. CogSys 2012, 5th International Conference in Cognitive Systems pp 109.
Duan F, Tan J (2011) A new human-robot collaboration assembly system for cellular manufacturing. 30th Chinese Control Conference (CCC), pp 5468-5473.
Someshwar R, Meyer J, Edan Y (2012) Models and Methods for H-R Synchronization. INCOM 2012, 14th IFAC International Symposium on Information Control Problems in Manufacturing.
Duan F, Tan J, Tong J, Kato R (2012) Application of the Assembly Skill Transfer System in an Actual Cellular Manufacturing System. IEEE Transactions on Automation Science and Engineering, 9:31-41. Edsinger A, Kemp CC (2007) Human-Robot Interaction for Cooperative Manipulation: Handing Objects to One Another. RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication, pp 1167-1172. Ferrell CB (1998) A Motivational System for regulating Human-Robot Interaction. Proceedings of the 10th Conference on AAAI, pp 54-62 Glasauer S, Huber M, Basili P (2010) Interacting in time and space: Investigating human-human and human-robot joint action. RO-MAN 2010, pp 252–257. Goodrich M a., Schultz AC (2007) Human-Robot Interaction: A Survey. Foundations and Trends in HumanComputer Interaction, 1:203-275.
703