- Email: [email protected]
Contributions of feedforward and feedback control in a manual trajectory-tracking task
2nd 2nd IFAC IFAC Conference Conference on on Cyber-Physical Cyber-Physical & & Human-Systems Human-Systems Miami, FL,Conference USA, Dec. 14-15, 2018 2nd IFAC on Cyber-Physical & Human-Systems Miami, FL, USA, Dec. 14-15, 2018 Available online at www.sciencedirect.com Miami, FL,Conference USA, Dec. 14-15, 2018 2nd IFAC on Cyber-Physical & Human-Systems Miami, FL, USA, Dec. 14-15, 2018
ScienceDirect
IFAC PapersOnLine 51-34 (2019) 61–66
Contributions of feedforward and feedback Contributions of feedforward and feedback Contributions of feedforward and feedback control in a manual trajectory-tracking control in a manual trajectory-tracking Contributions of feedforward and feedback control in a manual trajectory-tracking task task control in a manual trajectory-tracking task task Howell ∗∗∗ Eatai Roth ∗∗ ∗∗ Momona Yamagami ∗∗∗ Darrin ∗∗
Momona Yamagami ∗ Darrin Howell Eatai Roth ∗ ∗ Eatai Roth ∗∗ Momona Yamagami Darrin Howell ∗ Samuel A. ∗ Samuel A. Burden Burden ∗ ∗ ∗ Momona Yamagami Howell Eatai Roth ∗∗ SamuelDarrin A. Burden ∗ ∗ Samuel A. Burden ∗ Engineering, University of Washington, ∗ Department of Electrical ∗ Department of Electrical Engineering, University of Washington, Department of Electrical(e-mail: Engineering, University of Washington, Seattle, WA ). {my13,dbhowell,sburden}@uw.edu ). WA 98105 98105 USA USA (e-mail: {my13,dbhowell,sburden}@uw.edu ∗∗∗Seattle, Department of Electrical Engineering, University of Washington, Seattle, WA 98105 USA (e-mail: {my13,dbhowell,sburden}@uw.edu ). ∗∗ of Intelligent Systems Engineering, Indiana University, ∗∗ Department Department of Intelligent Systems Engineering, Indiana University, ∗∗ Seattle, WA 98105 USA (e-mail: {my13,dbhowell,sburden}@uw.edu). Department of Intelligent Systems Indiana ). University, Bloomington, IN 47405 47405 USAEngineering, (e-mail: [email protected] [email protected] ). Bloomington, IN USA (e-mail: ∗∗ Department of Intelligent Systems Indiana ). University, Bloomington, IN 47405 USAEngineering, (e-mail: [email protected] Bloomington, IN 47405 USA (e-mail: [email protected]). Abstract: Abstract: In In joint joint human-cyber-physical human-cyber-physical systems, systems, the the human human operator operator may may rely rely on on aa comcomAbstract: In joint human-cyber-physical systems, the human operator may relyproposes on a combination of reactive (feedback) and predictive (feedforward) control. This paper an bination of reactive (feedback) and predictive (feedforward) control. This paper proposes an Abstract: joint human-cyber-physical systems, the identify human operator relyproposes onand a combination of In reactive (feedback) and predictive (feedforward) control. Thismay paper an experimental and analytical approach to simultaneously the human feedback feedexperimental and analytical approach to simultaneously identify the human feedback and feedbination of reactive and predictive (feedforward) control. This In paper proposes an experimental and analytical approach to simultaneously identify the human feedback and feedforward controllers controllers in(feedback) the context context of human-cyber-physical human-cyber-physical systems (HCPS). our experiments, experiments, forward in the of systems (HCPS). In our experimental and a approach to simultaneously humanaaIn feedback feedforward controllers in1DOF the context of human-cyber-physical systems (HCPS). our experiments, participants play reference-tracking video tasked to guide cursor to follow aa participants play aanalytical 1DOF reference-tracking video game, game,identify taskedthe to guide cursor toand follow forward controllers the context of human-cyber-physical systems (HCPS). our experiments, participants playtrajectory. a in1DOF reference-tracking video game, tasked hypothesis to guide aIncursor to that followthe a pseudo-random For such tasks, the model inversion suggests pseudo-random trajectory. For such tasks, the model inversion hypothesis suggests that the participants playtrajectory. a 1DOF reference-tracking video game, tasked to guideofathe cursor to that followthe a pseudo-random For such the model inversion suggests human operator operator would implement as aatasks, feedforward controller the hypothesis inverse cyber-physicalhuman would implement as feedforward controller the inverse of the cyber-physicalpseudo-random For such the model inversion hypothesis suggests that the human operator trajectory. would implement as atasks, feedforward controller the inverse of individuals the cyber-physicalsystem dynamics. Our results indicate that at lower frequencies (≤ 0.15 Hz), capably system dynamics. Our results indicate that at lower frequencies (≤ 0.15 Hz), individuals capably human operator implement as athat feedforward controller the of individuals the cyber-physicalsystem dynamics. Our results indicate at lower frequencies (≤ inverse 0.15 Hz), capably invert system dynamics to implement aa feedforward controller, but at higher frequencies, invert the the systemwould dynamics to implement feedforward controller, but at higher frequencies, system dynamics. results indicate that at lower frequencies (≤ 0.15 Hz), capably invert the systemofOur dynamics to implement a feedforward controller, but at individuals higherhalf frequencies, the magnitudes the estimated feedforward transformation are approximately those of the magnitudes of the estimated feedforward transformation are approximately half those of invert the model systemof dynamics tosuggests implement a afeedforward controller, but individuals at higher frequencies, the exact magnitudes the estimated feedforward transformation are approximately half those of inverse. This that frequency limit at which are unable the exact model inverse. This suggests that a frequency limit at which individuals are unable the magnitudes of the estimated feedforward transformation are approximately half those of exact model inverse. This suggests that a model frequency limit at which individuals are unable to follow the system dynamics, and thus, the inversion prediction is only applicable at to follow the system dynamics, and thus, the model inversion prediction is only applicable at the exact model inverse. This suggests that a frequency limit at which individuals are unable to follow the system dynamics, and thus, the model inversion prediction is only applicable at lower frequencies. lower frequencies. to follow the system dynamics, and thus, the model inversion prediction is only applicable at lower frequencies. © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. lower frequencies. Keywords: Keywords: Keywords: internal internal inverse inverse model, model, sensorimotor sensorimotor learning, learning, system system identification identification Keywords: internal inverse model, sensorimotor learning, system identification internal inverse model, sensorimotor learning, system identification 1. experts, 1. INTRODUCTION INTRODUCTION experts, and and thus, thus, may may not not generalize generalize to to more more complex complex 1. INTRODUCTION experts, and thus, may not generalize more complex dynamical systems or A could dynamical systems or novices. novices. A similar similartoapproach approach could 1. INTRODUCTION experts, and thus, may not generalize to more complex dynamical systems or novices. A similar approach could apply more broadly to pilot-vehicle systems. From robot-assisted surgery to (semi-)autonomous drivapply more broadly to pilot-vehicle systems. From robot-assisted surgery to (semi-)autonomous driv- apply A similar approach could moresystems broadly or to novices. pilot-vehicle systems. From robot-assisted driv- dynamical ing, systems systems in which whichsurgery controltois is (semi-)autonomous shared between between humans humans In to regulation tracking ing, in control shared In addition addition to feedback feedback regulation on on tracking error, error, rereapply more broadly to pilot-vehicle systems. From robot-assisted surgery to (semi-)autonomous driving, systems in which control is shared between humans and engineered systems—human-cyber-physical systems addition to feedback regulation oninternalized tracking error, research suggests that humans rely on dynamand engineered systems—human-cyber-physical systems In search suggests that humans rely on internalized dynaming, systems inincreasingly which control is shared between humans search and engineered systems—human-cyber-physical systems (HCPS)—are common. Whereas controllers In to feedback regulation oninternalized tracking error, resuggests thatofhumans rely onand dynamicaladdition representations their bodies the environments environments (HCPS)—are increasingly common. Whereas controllers representations of their bodies and the and engineeredincreasingly systems—human-cyber-physical systems ical (HCPS)—are common. Whereas controllers in cyber-physical systems are designed (and therefore search suggests that rely onand internalized dynamrepresentations ofhumans their bodies the environments with which they interact for motion planning (Shadmehr in cyber-physical systems are designed (and therefore ical with which they interact for motion planning (Shadmehr (HCPS)—are increasingly common. in cyber-physical systems are of designed (andcontrollers therefore known), the emergent emergent dynamics theWhereas closed-loop human- with ical of their and the environments which they interact forbodies motion planning (Shadmehr and Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998). known), the dynamics of the closed-loop humanand representations Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998). We We in cyber-physical systems are designed (and operator, therefore known), the emergent dynamics of the closed-loop human- and cyber-physical system is shaped by the human with which they interact for extends motion (Shadmehr Mussa-Ivaldi, 1994; Wolpert andplanning Kawato, 1998). hypothesize that this finding to the devices or We vecyber-physical system is shaped by the human operator, hypothesize that this finding extends to the devices or veknown), the emergent thethe closed-loop humancyber-physical system is shaped by human operator, whose controller controller is not notdynamics known a of priori. Controllers imple- hypothesize and Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998). thatImportantly, this finding extends to themodels devices or We vehicles they use. such internal permit whose is known a priori. Controllers implethey use. Importantly, such internal models permit cyber-physical system shaped by significant theControllers humanvariability operator, whose is operators not is known a priori. imple- hicles mentedcontroller by human human exhibit hypothesize that this allowing finding extends to theto devices or the vehicles they use. Importantly, such models permit feedforward control, the human predict mented by operators exhibit significant variability feedforward control, allowing the internal human to predict the whose controller is not known a priori. Controllers implemented by human operators exhibit significant variability both hicles theyinputs use. Importantly, such internal models permit control, allowing the human to predict the necessary to produce the desired output trajectory, both between between populations populations (e.g. (e.g. novices novices and and experts) experts) and and feedforward inputs to produce the desired output trajectory, mented byindividual human operators exhibit significant variability both populations (e.g. novices and improve experts) and necessary within an individual (e.g. performance performance may improve with feedforward control, allowing the human to predict the necessary to produce desired output trajectory, potentiallyinputs increasing task the performance and decreasing withinbetween an (e.g. may with potentially increasing task performance and decreasing both between (e.g. novices and improve experts) and potentially within an individual (e.g. performance may with experience andpopulations degrade with fatigue) (Abbink et al., al., 2012). 2012). necessary inputs to feedback produce desired output trajectory, increasing task the performance and decreasing reliance on sensory (Gawthrop et al., 2009, 2011; experience and degrade with fatigue) (Abbink et reliance on sensory feedback (Gawthrop et al., 2009, 2011; within an aindividual (e.g. performance may improve with experience and degrade with fatigue) (Abbink et al., 2012). As such, such, predictive model of human human controllers (or potentially increasing task performanceet and decreasing on sensory feedback (Gawthrop al., 2009, 2011; Desmurget and Grafton, 2000). As a predictive model of controllers (or aa reliance Desmurget and Grafton, 2000). experience and degrade with fatigue) (Abbink et al., 2012). As such, a predictive model of human controllers (or a particular an important important Desmurget reliance on sensory feedback (Gawthrop et al., 2009, 2011; and Grafton, 2000). particular human’s human’s controller) controller) could could provide provide an We experimentally experimentally investigate the human human controller controller archiarchiAs ahuman’s predictive model of human controllers (or a We particular controller) could provide an to important tool for cyber-physical-systems tuned perform Desmurget and Grafton, 2000).the investigate toolsuch, for designing designing cyber-physical-systems tuned to perform experimentally investigate the human controller architecture from (Roth et al., 2017; Robinson et al., 2016) particular human’s controller) could provide an to important tool for designing cyber-physical-systems tuned perform We robustly with a human in the loop. tecture from (Roth et al., 2017; Robinson et al., 2016) robustly with a human in the loop. We experimentally investigate the human controller architecture from (Roth et al., 2017; Robinson et al., 2016) containing parallel feedback and feedforward pathways. tool for designing cyber-physical-systems tuned to perform containing parallel feedback and feedforward pathways. robustly with a human in the loop. In this this work, work, we consider trajectory tracking task task in in containing tecture (Roth et al., Robinson et pathways. al., 2016) parallel feedback and feedforward Previousfrom studies have used2017; reference-tracking tasks to robustly with awe human in the loop. In consider aa trajectory tracking studies have used reference-tracking tasks to In this awork, we operator consider learns a trajectory tracking taskdyin Previous which human to control a novel containing parallel feedback and feedforward pathways. studies have used reference-tracking tasks to disentangle feedforward and feedback transforms. (Zhang which a human operator learns to control a novel dy- Previous feedforward and feedback transforms. (Zhang In this awork, we operator consider a trajectory tracking taskdyin disentangle which human learns control a vehicle novel namical system, task similar similar toto steering steering or Previous studies used reference-tracking tasks to disentangle feedforward and 2012). feedback (Zhang et Drop et In contrast these namical system, aa task to aa vehicle or et al., al., 2018; 2018; Drophave et al., al., 2012). In transforms. contrast to to these which a human operator learns to control a novel dynamical system, a task similar to steering a vehicle or guiding a robot remotely. Trajectory tracking tasks have disentangle feedforward and 2012). feedback al., 2018; Drop et al., Inistransforms. contrast to(Zhang these studies, our prescribed trajectory r a pseudo-random guiding a robot remotely. Trajectory tracking tasks have et our prescribed trajectory r is a pseudo-random namical a task similar to steering vehicle or studies, guiding asystem, robot Trajectory trackingahow tasks have long been been task remotely. paradigm for understanding understanding human et al., 2018; Drop et thus al., 2012). to these studies, our prescribed trajectory rInis contrast aparticipants pseudo-random (unpredictable) signal, preventing from long aa task paradigm for how human (unpredictable) signal, thus preventing participants from guiding a robot Trajectory trackinghow tasks have (unpredictable) long been acontrol task remotely. paradigm forweakly understanding human operators linear (or (or nonlinear) systems; studies, our prescribed trajectory r is aparticipants pseudo-random signal,input thus preventing from learning aa particular sequence to achieve the task; operators control linear weakly nonlinear) systems; learning particular input sequence to achieve the task; long been acontrol task models paradigm understanding howsystems; human learning operators linear both (orforweakly nonlinear) control theoretic explain and predict predict manual (unpredictable) signal, thus requires preventing from a particular input sequence to participants achieve the task; rather, control the user the control theoretic models both explain and manual rather, feedforward feedforward control requires the user to to learn learn the operators linear both (or weakly nonlinear) systems; control explainand andJex, predict manual of piloted aircraft (McRuer 1967; Allen learning a particular input sequence to achieve the task; feedforward control requires the user to learn the forward model so that it may be generalized to novel control theoretic of control pilotedmodels aircraft (McRuer and Jex, 1967; Allen rather, forward model so that it may be generalized to novel theoretic models both explainhowever, andJex, predict control of piloted aircraft (McRuer and 1967;manual Allen forward and McRuer, McRuer, 1979). These models, were specifrather, feedforward control requires user tomakes learn the model so that it experimental may be the generalized to novel reference trajectories. Our assay posand 1979). These models, however, were speciftrajectories. Our experimental assay makes poscontrol of piloted aircraft (McRuer and Jex, 1967;specifAllen and McRuer, 1979). Theseregulation models, however, were ically designed to model tasks performed by reference forward model so that it experimental may be generalized to novel reference trajectories. Our assay makes posically designed to model regulation tasks performed by and McRuer, Theseregulation models, however, were specifically designed1979). to model tasks performed by reference trajectories. Our experimental assay makes posically designed to model regulation tasks performed by 61 Copyright ©2018 IFAC 2405-8963 © 2019,IFAC IFAC (International Federation of Automatic Control) Copyright ©2018 61 Hosting by Elsevier Ltd. All rights reserved. Copyright ©2018 IFAC 61 Copyright 61 Control. Peer review©2018 underIFAC responsibility of International Federation of Automatic 10.1016/j.ifacol.2019.01.025 Copyright ©2018 IFAC 61
IFAC CPHS 2018 62 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
sible the simultaneous estimation of concurrent feedback and feedforward transformations. Further, we compare control models between trials in which participants must rely solely on feedback (i.e. disturbance rejection without exogenous reference) and trials in which both feedback and feedforward control contribute to the human response (Yu et al., 2014).
2.2 Separation of Feedforward and Feedback Control Inputs The transfer functions HU R and HU D map the reference and disturbance signals, respectively, to the user input. These transforms are empirically estimated for each frequency (at which we present a reference and/or disturbance) by computing the ratio of the Fourier transforms of U measured signals, U R or D . We may express these empirical transforms as functions of the unknown feedforward and feedback controllers, F and B:
2. PROBLEM FORMULATION 2.1 Trajectory Tracking via Dynamic Model Inversion
U=
In the laboratory, we instantiate the human-cyber-physical interaction as a single-degree-of-freedom, reference-tracking task: a path-following video game in which the human operator is tasked with guiding a cursor, y, along a prescribed path, r (Fig. 1b). The human participant uses a 1 degree-of-freedom sliding joystick to modulate the input u. The cyber-physical system dynamics are prescribed by the model M , transforming human inputs into cursor motion. We hypothesize that the human sensorimotor controller comprises parallel feedforward and feedback pathways, F and B respectively (Fig. 1a). The dynamic inverse model mathematical framework suggests that humans learn the forward model M and implement the inverse model as a feedforward controller, F = M −1 . By introducing a reference trajectory r as well as a disturbance trajectory d together and separately, we are able to separate subject feedforward and feedback models.
−BM F +B R+ D, 1 + BM 1 + BM Hur
(1)
Hud
Y = (U + D)M .
(2)
Or conversely, we estimate the feedforward and feedback controllers as functions of the empirical transforms and the prescribed system model: B=
−Hud , M (1 + Hud )
(3)
Hur + M −1 Hud . (4) 1 + Hud Hypothesis 2.2. The combined feedback and feedforward model predicts user responses better than a model implementing only feedback control. The feedforward pathway is critical in modeling user responses. F =
3. EXPERIMENTAL METHODS 3.1 Overview of Experimental Setup The reference and disturbance signals comprised a scaled and phase-shifted sum of sines with signal frequencies at the prime harmonics of a fundamental frequency 0.05 Hz, spanning 0.1-1.55 Hz (Fig. 2). These were presented to the subject in a window one display unit tall (approx 5.3cm) and 4 display units high. The phase shifts were randomly chosen, and the magnitude of the sine wave at each frequency was scaled to 1/f for the first-order system (i.e. , constant velocity), and 1/f 2 for the secondorder system (i.e. , constant acceleration), such that the larger frequencies had a lower amplitude. The participants were instructed to slide a linear joystick (a 10 kΩ linear potentiometer) to modulate the input U . The linear potentiometer had a 10cm potential traveling distance. Trial trajectories were designed such that the ideal feedforward input trajectory would be constrained to one-third of the allowable range of inputs, thus avoiding saturation of the input signal. Each trial lasted for 40 seconds–two complete periods of the pseudorandom signal–and the user input u and output y were sampled at 60 Hz (u measured using an Arduino Due).
Fig. 1. a. The block diagram of HCPS highlights the feedback and feedforward human controller separation that combines to a measured user input U . Our experimental assays prescribe the reference and disturbance signals, r and d, as well as the cyber-physical model, M , and are designed to enable estimation of the distinct contributions of feedforward and feedback processes. b. A human operator completing the computer trajectory tracking task using a slider.
Table 1. Assays Presented To Human Operator Hypothesis 2.1. The user input for reference tracking with disturbance is consistent with a superposition of the responses to the reference trajectory and the disturbance signals presented individually.
Order Assay # Trials
62
1 r+d 2
2 d 10
3 r+d 2
4 r 10
5 r+d 10
IFAC CPHS 2018 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
63
Fig. 2. The three types of assays used in the study shown as an FFT of the 40 second signal. The resolution of the axis is 0.05 Hz. The blue and red lines indicate frequency content for D and R respectively. Two types of alternating frequencies were prescribed - one where R had frequency content on the even primes and D on odd, and one where R had frequency content on the odd primes and D on even. as a measure of spread. For all trials, circular statistics were used to unwrap the angle of the transfer functions prior to analyzing the data.
Three assay categories were presented to the operator: 1. only r with d = 0; 2. only d, with r = 0; and 3. r + d, with each at alternating frequencies such that for one frequency, either r = 0 or d = 0 (Fig. 2). For trials in which reference and disturbance signals were presented simultaneously, each signal contained alternating frequencies from the above set, even or odd components interleaved. In consecutive trials, the frequency support of r and d would be swapped. The r trajectories were displayed as a gold path having a look-ahead preview of 0.25 seconds; there was no visual representation of the disturbance, d, and consequently no look-ahead. The phase shifts for all frequency components were randomized for each trial, and the three assays were presented in sequence (Table 1).
A feedback only model and a feedback and feedforward model was estimated using the D only and R only trials to determine whether the feedback and feedforward model had a better fit to the data than the feedback only model. The transfer functions were then applied to R and D in the R+D trials which were not a part of the training data, and the percent error between the predicted Up and measured Um was calculated. The IQR of the error was calculated for both models. F and B were calculated as described in Eqns. 3, 4 to determine the extent to which F was similar to the model inverse M −1 . A Wilcoxon Signed-Rank test was performed to quantify this similarity.
Two models were presented sequentially to observe model inversion in a simpler first-order (FO) model and a more complex second-order (SO) model, with, 1 (5) mF O : y˙ = u; MF O : , s 1 . (6) mSO : y¨ − y˙ = u; MSO : 2 s −s
4. RESULTS 4.1 Linearity of HCPS The moderate coherence between the r and r + d trials (median, range Cr,r+d = 0.84, 0.21 − 1.0) and d and r + d trials (median, range Cd,r+d = 0.82, 0.10 − 0.99) demonstrated that r + d trials may be well predicted from r and d trials by a linear least squares function (Fig. 3). This suggests that there may be a linear relationship between the system inputs r, d and the system output y. The Kruskal Wallis test demonstrated that there was a statistical difference in the medians of both HU R and HU D with p < 0.001 and p = 0.014 respectively. The posthoc rank sum test indicated that there was a significant difference in the median coherence between frequencies at 0.35 Hz and lower and 1.15 Hz and higher for HU R , and 0.55 Hz and lower and at 1.55 Hz for HU D . This suggests that there is a lower overall coherence with increasing frequency, especially at 1.15 Hz and above, whereas there is a high coherence at 0.35 Hz and below.
For second order models, the spectrum of the stimuli was truncated to 0.1-0.85 Hz. 3.2 Experimental Data Analysis
Data for seven (7) participants were analyzed with Python 2.7.3. Sampled signals were converted into their frequency representations using the fast Fourier transform (FFT). Coherence was calculated in the time domain using the python function scipy.signal.coherence() between the r, d and r + d trials at frequencies of interest (Fig. 2). The Kruskal-Wallis test was conducted to determine whether the median coherence of each frequency came from the same population, and the rank sum test with a Bonferroni correction was conducted as a post-hoc test from the scipy.stats package. HU D and HU R in the frequency domain were also calculated at each frequency of interest as in Eq. 1 to assess linearity and superposition of the HCPS. A Wilcoxon Signed-Rank test was performed to determine the extent to which the medians of HU D and HU R from the D, R trials were similar to the values calculated from the R + D trials. The interquartile range (IQR) was calculated
Further investigation into the transfer functions HU R and HU D for the first-order system demonstrated a high degree of overlap between the R and R + D trials and D and R + D trials (Fig. 4). The Wilcoxon Signed-Rank test indicated that with α < 0.05, there was no statistical difference in HU R and HU D in either average magnitude 63
IFAC CPHS 2018 64 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
Fig. 3. Coherence values calculated between the r and d only trials and r + d trials demonstrate high coherence between the two for lower frequencies, whereas the coherence at higher frequencies show a larger variability between subjects.
Fig. 4. Transfer functions for D to U : HU D = U/D and R to U : HU R = U/R demonstrate overlap for the transfer functions in the R and D only trials and R + D trials. The shaded blue and red areas represent the IQR of HU D and HU R respectively from the D only and R only trials; the blue triangle represents individual HU D values and the red circle represents individual HU R values calculated from the R + D trials. or angle between the only R or only D trials and the R+D trials. Similarly to the coherence analysis, the higher frequencies tended to have a larger spread in variability across individuals for both the R only and D only trials as well as the R + D trials. The positive results from this superposition test indicates that the HCPS can be thought of and analyzed as a linear system for lower frequencies below 0.35 Hz.
forward and feedback control strategies. We verified that when a feedback only model was fitted to a subsection of the data (R only and D only trials for first-order model) and tested on a separate set of data (R + D trials for first-order model), the error between the predicted Up and measured Um was worse than the error from the fitted feedback and feedforward model at all frequencies (Fig. 5), with the error roughly twice as high for the feedback only model compared to the combined feedback and feedfor4.2 Feedback and Feedforward Combined Controllers Versus ward model. The error linearly increases in the log-log Feedback Only Controller plot, suggesting that for the first-order model, the error is inversely proportional to the amplitude of the shown As proposed in hypothesis 2.2, human operators may con- signal. trol cyber-physical systems with a combination of feed64
IFAC CPHS 2018 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
65
only agrees with the inverse of the cyber-physical system model at low frequencies at 0.15 Hz and below. At higher frequencies at 0.25 Hz and above, we demonstrated that the magnitudes are lower than the expected magnitude of M −1 by roughly a factor of 0.5 (Fig. 6). This was corroborated with our statistical tests, which demonstrated that with an α < 0.05, the median for both models did not have M −1 as the median for either the magnitude or the angle. This result is consistent with the coherence values calculated in Fig. 3, which demonstrated that lower frequencies at 0.35 Hz and below tended to have a higher median coherence and smaller variability than the higher frequencies, suggesting a more linear HCPS at lower frequencies than higher frequencies. The transfer functions for HU D and HU R had similar results where frequencies larger than 0.55 Hz tended to have a much larger variability in both magnitude and angle than the lower frequencies.
Fig. 5. The percent error between the predicted input and actual user input for the feedback only model (blue shaded area) and feedback and feedforward model (red shaded area) demonstrates lower percent error at all frequencies for the feedback and feedforward model. 4.3 Dynamic Model Inversion
This decrease in linearity with increase in frequency may be due to a number of reasons, including the strength of the signal provided and the limits of the neuromuscular system. For this study, we chose to provide human operators with a lower signal amplitude for higher frequencies inversely proportional to the frequency for firstorder, and inversely proportional to the square of the frequency for the second-order system, which will magnify any errors that the human operator makes while tracking these higher frequencies. Another possible explanation is neuromuscular limitations. The human visuomotor limitation suggests that the fastest motor reaction time for humans (as demonstrated by asking a human subject to press a button upon the presentation of a certain visual) is about 5 Hz (Wardle, 1998). Biomechanical limitations may also contribute to the performance drop-off at higher frequencies. Our assay asks our participants to move at speeds up to 20 cm/s. While this is not a slow movement, it is a much slower than the speeds required to complete the reach-out tasks for example, where participants move at speeds of 50 cm/s (Shadmehr and Mussa-Ivaldi, 1994). In addition, the maximum absolute jerk required to perfectly track the reference is 75cm/s2 . As such, we cannot conclude from our results whether the nonlinear response at higher frequencies is due to the breakdown of the model inversion model at higher frequencies or to neuromuscular limitations.
The feedback and feedforward estimates pooled across all individuals and trials demonstrate large deviations in magnitude from M −1 for the feedforward controller for both R and R + D trials (Fig. 6). The Wilcoxon SignedRank test performed on the median FR+D calculated from the R+D trials for all frequencies demonstrated that there was a significant difference between the feedforward controller and M −1 for the magnitudes (first-order: p = 0.006, second-order: p = 0.018). The angle of the experimental feedforward block was significantly different from M −1 for the first order M (p = 0.004), but not for the second order (p = 0.18). Similarly, for the F calculated from the R only and D only trials, there was a significant difference in the magnitudes between the FR,D and M −1 (first-order: p = 0.004, second-order: p = 0.04) and the angle for the first-order model (p = 0.006), while we again failed to verify a difference in the angles for the second-order system (p = 0.74). Looking more closely at whether there is a difference between the median F and expected M −1 magnitude at each frequency, we found that the magnitude was consistently lower than the expected M −1 at all frequencies except 0.1 Hz and 0.15 Hz, as demonstrated by the Wilcoxon SignedRank test (p < 0.05) for both the first and second-order system. The magnitude of the higher frequencies at 0.25 Hz and above were lower than the expected M −1 magnitude by a median factor of 0.54 (range: 0.43 − 0.70) for the first-order system and a median factor of 0.59 (range: 0.41 − 0.69) for the second-order system.
While the feedback and feedforward model can do no worse than the feedback only model in predicting the output of the training dataset, the increased number of variables in the feedback and feedforward model may lead to overfitting and thus, large errors in the validation dataset. Using the final R + D trials as a validation set, we confirmed that the combined feedback and feedforward model had less error predicting the user response at all frequencies as compared to the feedback-only model. This result suggests that we did not overfit, and in fact, both the feedforward and feedback controllers lead to a better model of the HCPS.
5. DISCUSSION We estimated feedback and feedforward elements of human sensorimotor control in a trajectory tracking task. Estimates were obtained for two models (MF O = 1/S and MSO = 1/(S 2 − S)) using two different assays; both estimates agreed to within experimental error for frequencies up to 0.15 Hz. Furthermore, including the feedforward pathway roughly halves the human input prediction error, suggesting both feedforward and feedback elements may be required to predict human control of cyber-physical systems.
6. CONCLUSION This paper proposes a novel experiment design that enables estimation of human feedback and feedforward transformations in trajectory tracking tasks directly from em-
In contrast to the recent results reported in (Zhang et al., 2018), the estimated feedforward transformation 65
IFAC CPHS 2018 66 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
Fig. 6. The experimental values of F and B demonstrate for both first and second order closer model inversion for lower frequencies than higher frequencies in magnitude. The red dotted line represents the model inverse, the shaded red and blue is the IQR for F and B calculated from R only and D only trials, and the dark blue triangle and dark red circle represents F and B calculated from R + D trials for each individual. . pirical frequency responses to reference and disturbance inputs. Our results support the hypothesis that humans learn to use the inverse of the cyber-physical system model in their feedforward control pathway only at frequencies at 0.15 Hz and below, and the estimated feedforward transformation for higher frequencies resembles a scaleddown version of the model inverse. Further investigations are needed to characterize the feedforward transformations learned by human operators.
Gawthrop, P., Loram, I., Lakie, M., and Gollee, H. (2011). Intermittent control: A computational theory of human control. Biological cybernetics, 104(1-2), 31–51. McRuer, D.T. and Jex, H.R. (1967). A review of quasilinear pilot models. IEEE Transactions on Human Factors in Electronics, (3), 231–249. Robinson, R.M., Scobee, D.R., Burden, S.A., and Sastry, S.S. (2016). Dynamic inverse models in humancyber-physical systems. In Micro-and Nanotechnology Sensors, Systems, and Applications VIII, volume 9836, 98361X. International Society for Optics and Photonics. ACKNOWLEDGEMENTS Roth, E., Howell, D., Beckwith, C., and Burden, S.A. (2017). Toward experimental validation of a model for This research is supported the National Science Foundahuman sensorimotor learning and control in teleoperation’s CISE CPS Program (Awards #1565529, 1836819), tion. In Micro-and Nanotechnology Sensors, Systems, the Air Force Office of Scientific Research (Award #FA9550and Applications IX, volume 10194, 101941X. Interna14-1-0398), and the Washington Research Foundation tional Society for Optics and Photonics. Funds for Innovation in Neuroengineering. Shadmehr, R. and Mussa-Ivaldi, F.A. (1994). Adaptive representation of dynamics during learning of a motor REFERENCES task. Journal of Neuroscience, 14(5), 3208–3224. Abbink, D.A., Mulder, M., and Boer, E.R. (2012). Haptic Wardle, D. (1998). The time delay in human vision. The shared control: smoothly shifting control authority? physics teacher, 36(7), 442–444. Cognition, Technology & Work, 14(1), 19–28. Wolpert, D.M. and Kawato, M. (1998). Multiple paired Allen, R.W. and McRuer, D. (1979). The man/machine forward and inverse models for motor control. Neural control interface pursuit control. Automatica, 15(6), networks, 11(7-8), 1317–1329. 683–686. Yu, B., Gillespie, R.B., Freudenberg, J.S., and Cook, J.A. Desmurget, M. and Grafton, S. (2000). Forward modeling (2014). Human control strategies in pursuit tracking allows feedback control for fast reaching movements. with a disturbance input. In 2014 IEEE 53rd Annual Trends in cognitive sciences, 4(11), 423–431. Conference on Decision and Control (CDC), 3795–3800. Drop, F.M., Pool, D.M., Damveld, H.J., van Paassen, IEEE. M.M., B¨ ulthoff, H.H., and Mulder, M. (2012). Identifica- Zhang, X., Wang, S., Hoagg, J.B., and Seigler, T.M. tion of the transition from compensatory to feedforward (2018). The roles of feedback and feedforward as humans behavior in manual control. In 2012 IEEE International learn to control unknown dynamic systems. IEEE Conference on Systems, Man, and Cybernetics (SMC), transactions on cybernetics, 48(2), 543–555. 2008–2013. IEEE. Gawthrop, P., Loram, I., and Lakie, M. (2009). Predictive feedback in human simulated pendulum balancing. Biological cybernetics, 101(2), 131–146. 66
ScienceDirect
IFAC PapersOnLine 51-34 (2019) 61–66
Contributions of feedforward and feedback Contributions of feedforward and feedback Contributions of feedforward and feedback control in a manual trajectory-tracking control in a manual trajectory-tracking Contributions of feedforward and feedback control in a manual trajectory-tracking task task control in a manual trajectory-tracking task task Howell ∗∗∗ Eatai Roth ∗∗ ∗∗ Momona Yamagami ∗∗∗ Darrin ∗∗
Momona Yamagami ∗ Darrin Howell Eatai Roth ∗ ∗ Eatai Roth ∗∗ Momona Yamagami Darrin Howell ∗ Samuel A. ∗ Samuel A. Burden Burden ∗ ∗ ∗ Momona Yamagami Howell Eatai Roth ∗∗ SamuelDarrin A. Burden ∗ ∗ Samuel A. Burden ∗ Engineering, University of Washington, ∗ Department of Electrical ∗ Department of Electrical Engineering, University of Washington, Department of Electrical(e-mail: Engineering, University of Washington, Seattle, WA ). {my13,dbhowell,sburden}@uw.edu ). WA 98105 98105 USA USA (e-mail: {my13,dbhowell,sburden}@uw.edu ∗∗∗Seattle, Department of Electrical Engineering, University of Washington, Seattle, WA 98105 USA (e-mail: {my13,dbhowell,sburden}@uw.edu ). ∗∗ of Intelligent Systems Engineering, Indiana University, ∗∗ Department Department of Intelligent Systems Engineering, Indiana University, ∗∗ Seattle, WA 98105 USA (e-mail: {my13,dbhowell,sburden}@uw.edu). Department of Intelligent Systems Indiana ). University, Bloomington, IN 47405 47405 USAEngineering, (e-mail: [email protected] [email protected] ). Bloomington, IN USA (e-mail: ∗∗ Department of Intelligent Systems Indiana ). University, Bloomington, IN 47405 USAEngineering, (e-mail: [email protected] Bloomington, IN 47405 USA (e-mail: [email protected]). Abstract: Abstract: In In joint joint human-cyber-physical human-cyber-physical systems, systems, the the human human operator operator may may rely rely on on aa comcomAbstract: In joint human-cyber-physical systems, the human operator may relyproposes on a combination of reactive (feedback) and predictive (feedforward) control. This paper an bination of reactive (feedback) and predictive (feedforward) control. This paper proposes an Abstract: joint human-cyber-physical systems, the identify human operator relyproposes onand a combination of In reactive (feedback) and predictive (feedforward) control. Thismay paper an experimental and analytical approach to simultaneously the human feedback feedexperimental and analytical approach to simultaneously identify the human feedback and feedbination of reactive and predictive (feedforward) control. This In paper proposes an experimental and analytical approach to simultaneously identify the human feedback and feedforward controllers controllers in(feedback) the context context of human-cyber-physical human-cyber-physical systems (HCPS). our experiments, experiments, forward in the of systems (HCPS). In our experimental and a approach to simultaneously humanaaIn feedback feedforward controllers in1DOF the context of human-cyber-physical systems (HCPS). our experiments, participants play reference-tracking video tasked to guide cursor to follow aa participants play aanalytical 1DOF reference-tracking video game, game,identify taskedthe to guide cursor toand follow forward controllers the context of human-cyber-physical systems (HCPS). our experiments, participants playtrajectory. a in1DOF reference-tracking video game, tasked hypothesis to guide aIncursor to that followthe a pseudo-random For such tasks, the model inversion suggests pseudo-random trajectory. For such tasks, the model inversion hypothesis suggests that the participants playtrajectory. a 1DOF reference-tracking video game, tasked to guideofathe cursor to that followthe a pseudo-random For such the model inversion suggests human operator operator would implement as aatasks, feedforward controller the hypothesis inverse cyber-physicalhuman would implement as feedforward controller the inverse of the cyber-physicalpseudo-random For such the model inversion hypothesis suggests that the human operator trajectory. would implement as atasks, feedforward controller the inverse of individuals the cyber-physicalsystem dynamics. Our results indicate that at lower frequencies (≤ 0.15 Hz), capably system dynamics. Our results indicate that at lower frequencies (≤ 0.15 Hz), individuals capably human operator implement as athat feedforward controller the of individuals the cyber-physicalsystem dynamics. Our results indicate at lower frequencies (≤ inverse 0.15 Hz), capably invert system dynamics to implement aa feedforward controller, but at higher frequencies, invert the the systemwould dynamics to implement feedforward controller, but at higher frequencies, system dynamics. results indicate that at lower frequencies (≤ 0.15 Hz), capably invert the systemofOur dynamics to implement a feedforward controller, but at individuals higherhalf frequencies, the magnitudes the estimated feedforward transformation are approximately those of the magnitudes of the estimated feedforward transformation are approximately half those of invert the model systemof dynamics tosuggests implement a afeedforward controller, but individuals at higher frequencies, the exact magnitudes the estimated feedforward transformation are approximately half those of inverse. This that frequency limit at which are unable the exact model inverse. This suggests that a frequency limit at which individuals are unable the magnitudes of the estimated feedforward transformation are approximately half those of exact model inverse. This suggests that a model frequency limit at which individuals are unable to follow the system dynamics, and thus, the inversion prediction is only applicable at to follow the system dynamics, and thus, the model inversion prediction is only applicable at the exact model inverse. This suggests that a frequency limit at which individuals are unable to follow the system dynamics, and thus, the model inversion prediction is only applicable at lower frequencies. lower frequencies. to follow the system dynamics, and thus, the model inversion prediction is only applicable at lower frequencies. © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. lower frequencies. Keywords: Keywords: Keywords: internal internal inverse inverse model, model, sensorimotor sensorimotor learning, learning, system system identification identification Keywords: internal inverse model, sensorimotor learning, system identification internal inverse model, sensorimotor learning, system identification 1. experts, 1. INTRODUCTION INTRODUCTION experts, and and thus, thus, may may not not generalize generalize to to more more complex complex 1. INTRODUCTION experts, and thus, may not generalize more complex dynamical systems or A could dynamical systems or novices. novices. A similar similartoapproach approach could 1. INTRODUCTION experts, and thus, may not generalize to more complex dynamical systems or novices. A similar approach could apply more broadly to pilot-vehicle systems. From robot-assisted surgery to (semi-)autonomous drivapply more broadly to pilot-vehicle systems. From robot-assisted surgery to (semi-)autonomous driv- apply A similar approach could moresystems broadly or to novices. pilot-vehicle systems. From robot-assisted driv- dynamical ing, systems systems in which whichsurgery controltois is (semi-)autonomous shared between between humans humans In to regulation tracking ing, in control shared In addition addition to feedback feedback regulation on on tracking error, error, rereapply more broadly to pilot-vehicle systems. From robot-assisted surgery to (semi-)autonomous driving, systems in which control is shared between humans and engineered systems—human-cyber-physical systems addition to feedback regulation oninternalized tracking error, research suggests that humans rely on dynamand engineered systems—human-cyber-physical systems In search suggests that humans rely on internalized dynaming, systems inincreasingly which control is shared between humans search and engineered systems—human-cyber-physical systems (HCPS)—are common. Whereas controllers In to feedback regulation oninternalized tracking error, resuggests thatofhumans rely onand dynamicaladdition representations their bodies the environments environments (HCPS)—are increasingly common. Whereas controllers representations of their bodies and the and engineeredincreasingly systems—human-cyber-physical systems ical (HCPS)—are common. Whereas controllers in cyber-physical systems are designed (and therefore search suggests that rely onand internalized dynamrepresentations ofhumans their bodies the environments with which they interact for motion planning (Shadmehr in cyber-physical systems are designed (and therefore ical with which they interact for motion planning (Shadmehr (HCPS)—are increasingly common. in cyber-physical systems are of designed (andcontrollers therefore known), the emergent emergent dynamics theWhereas closed-loop human- with ical of their and the environments which they interact forbodies motion planning (Shadmehr and Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998). known), the dynamics of the closed-loop humanand representations Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998). We We in cyber-physical systems are designed (and operator, therefore known), the emergent dynamics of the closed-loop human- and cyber-physical system is shaped by the human with which they interact for extends motion (Shadmehr Mussa-Ivaldi, 1994; Wolpert andplanning Kawato, 1998). hypothesize that this finding to the devices or We vecyber-physical system is shaped by the human operator, hypothesize that this finding extends to the devices or veknown), the emergent thethe closed-loop humancyber-physical system is shaped by human operator, whose controller controller is not notdynamics known a of priori. Controllers imple- hypothesize and Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998). thatImportantly, this finding extends to themodels devices or We vehicles they use. such internal permit whose is known a priori. Controllers implethey use. Importantly, such internal models permit cyber-physical system shaped by significant theControllers humanvariability operator, whose is operators not is known a priori. imple- hicles mentedcontroller by human human exhibit hypothesize that this allowing finding extends to theto devices or the vehicles they use. Importantly, such models permit feedforward control, the human predict mented by operators exhibit significant variability feedforward control, allowing the internal human to predict the whose controller is not known a priori. Controllers implemented by human operators exhibit significant variability both hicles theyinputs use. Importantly, such internal models permit control, allowing the human to predict the necessary to produce the desired output trajectory, both between between populations populations (e.g. (e.g. novices novices and and experts) experts) and and feedforward inputs to produce the desired output trajectory, mented byindividual human operators exhibit significant variability both populations (e.g. novices and improve experts) and necessary within an individual (e.g. performance performance may improve with feedforward control, allowing the human to predict the necessary to produce desired output trajectory, potentiallyinputs increasing task the performance and decreasing withinbetween an (e.g. may with potentially increasing task performance and decreasing both between (e.g. novices and improve experts) and potentially within an individual (e.g. performance may with experience andpopulations degrade with fatigue) (Abbink et al., al., 2012). 2012). necessary inputs to feedback produce desired output trajectory, increasing task the performance and decreasing reliance on sensory (Gawthrop et al., 2009, 2011; experience and degrade with fatigue) (Abbink et reliance on sensory feedback (Gawthrop et al., 2009, 2011; within an aindividual (e.g. performance may improve with experience and degrade with fatigue) (Abbink et al., 2012). As such, such, predictive model of human human controllers (or potentially increasing task performanceet and decreasing on sensory feedback (Gawthrop al., 2009, 2011; Desmurget and Grafton, 2000). As a predictive model of controllers (or aa reliance Desmurget and Grafton, 2000). experience and degrade with fatigue) (Abbink et al., 2012). As such, a predictive model of human controllers (or a particular an important important Desmurget reliance on sensory feedback (Gawthrop et al., 2009, 2011; and Grafton, 2000). particular human’s human’s controller) controller) could could provide provide an We experimentally experimentally investigate the human human controller controller archiarchiAs ahuman’s predictive model of human controllers (or a We particular controller) could provide an to important tool for cyber-physical-systems tuned perform Desmurget and Grafton, 2000).the investigate toolsuch, for designing designing cyber-physical-systems tuned to perform experimentally investigate the human controller architecture from (Roth et al., 2017; Robinson et al., 2016) particular human’s controller) could provide an to important tool for designing cyber-physical-systems tuned perform We robustly with a human in the loop. tecture from (Roth et al., 2017; Robinson et al., 2016) robustly with a human in the loop. We experimentally investigate the human controller architecture from (Roth et al., 2017; Robinson et al., 2016) containing parallel feedback and feedforward pathways. tool for designing cyber-physical-systems tuned to perform containing parallel feedback and feedforward pathways. robustly with a human in the loop. In this this work, work, we consider trajectory tracking task task in in containing tecture (Roth et al., Robinson et pathways. al., 2016) parallel feedback and feedforward Previousfrom studies have used2017; reference-tracking tasks to robustly with awe human in the loop. In consider aa trajectory tracking studies have used reference-tracking tasks to In this awork, we operator consider learns a trajectory tracking taskdyin Previous which human to control a novel containing parallel feedback and feedforward pathways. studies have used reference-tracking tasks to disentangle feedforward and feedback transforms. (Zhang which a human operator learns to control a novel dy- Previous feedforward and feedback transforms. (Zhang In this awork, we operator consider a trajectory tracking taskdyin disentangle which human learns control a vehicle novel namical system, task similar similar toto steering steering or Previous studies used reference-tracking tasks to disentangle feedforward and 2012). feedback (Zhang et Drop et In contrast these namical system, aa task to aa vehicle or et al., al., 2018; 2018; Drophave et al., al., 2012). In transforms. contrast to to these which a human operator learns to control a novel dynamical system, a task similar to steering a vehicle or guiding a robot remotely. Trajectory tracking tasks have disentangle feedforward and 2012). feedback al., 2018; Drop et al., Inistransforms. contrast to(Zhang these studies, our prescribed trajectory r a pseudo-random guiding a robot remotely. Trajectory tracking tasks have et our prescribed trajectory r is a pseudo-random namical a task similar to steering vehicle or studies, guiding asystem, robot Trajectory trackingahow tasks have long been been task remotely. paradigm for understanding understanding human et al., 2018; Drop et thus al., 2012). to these studies, our prescribed trajectory rInis contrast aparticipants pseudo-random (unpredictable) signal, preventing from long aa task paradigm for how human (unpredictable) signal, thus preventing participants from guiding a robot Trajectory trackinghow tasks have (unpredictable) long been acontrol task remotely. paradigm forweakly understanding human operators linear (or (or nonlinear) systems; studies, our prescribed trajectory r is aparticipants pseudo-random signal,input thus preventing from learning aa particular sequence to achieve the task; operators control linear weakly nonlinear) systems; learning particular input sequence to achieve the task; long been acontrol task models paradigm understanding howsystems; human learning operators linear both (orforweakly nonlinear) control theoretic explain and predict predict manual (unpredictable) signal, thus requires preventing from a particular input sequence to participants achieve the task; rather, control the user the control theoretic models both explain and manual rather, feedforward feedforward control requires the user to to learn learn the operators linear both (or weakly nonlinear) systems; control explainand andJex, predict manual of piloted aircraft (McRuer 1967; Allen learning a particular input sequence to achieve the task; feedforward control requires the user to learn the forward model so that it may be generalized to novel control theoretic of control pilotedmodels aircraft (McRuer and Jex, 1967; Allen rather, forward model so that it may be generalized to novel theoretic models both explainhowever, andJex, predict control of piloted aircraft (McRuer and 1967;manual Allen forward and McRuer, McRuer, 1979). These models, were specifrather, feedforward control requires user tomakes learn the model so that it experimental may be the generalized to novel reference trajectories. Our assay posand 1979). These models, however, were speciftrajectories. Our experimental assay makes poscontrol of piloted aircraft (McRuer and Jex, 1967;specifAllen and McRuer, 1979). Theseregulation models, however, were ically designed to model tasks performed by reference forward model so that it experimental may be generalized to novel reference trajectories. Our assay makes posically designed to model regulation tasks performed by and McRuer, Theseregulation models, however, were specifically designed1979). to model tasks performed by reference trajectories. Our experimental assay makes posically designed to model regulation tasks performed by 61 Copyright ©2018 IFAC 2405-8963 © 2019,IFAC IFAC (International Federation of Automatic Control) Copyright ©2018 61 Hosting by Elsevier Ltd. All rights reserved. Copyright ©2018 IFAC 61 Copyright 61 Control. Peer review©2018 underIFAC responsibility of International Federation of Automatic 10.1016/j.ifacol.2019.01.025 Copyright ©2018 IFAC 61
IFAC CPHS 2018 62 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
sible the simultaneous estimation of concurrent feedback and feedforward transformations. Further, we compare control models between trials in which participants must rely solely on feedback (i.e. disturbance rejection without exogenous reference) and trials in which both feedback and feedforward control contribute to the human response (Yu et al., 2014).
2.2 Separation of Feedforward and Feedback Control Inputs The transfer functions HU R and HU D map the reference and disturbance signals, respectively, to the user input. These transforms are empirically estimated for each frequency (at which we present a reference and/or disturbance) by computing the ratio of the Fourier transforms of U measured signals, U R or D . We may express these empirical transforms as functions of the unknown feedforward and feedback controllers, F and B:
2. PROBLEM FORMULATION 2.1 Trajectory Tracking via Dynamic Model Inversion
U=
In the laboratory, we instantiate the human-cyber-physical interaction as a single-degree-of-freedom, reference-tracking task: a path-following video game in which the human operator is tasked with guiding a cursor, y, along a prescribed path, r (Fig. 1b). The human participant uses a 1 degree-of-freedom sliding joystick to modulate the input u. The cyber-physical system dynamics are prescribed by the model M , transforming human inputs into cursor motion. We hypothesize that the human sensorimotor controller comprises parallel feedforward and feedback pathways, F and B respectively (Fig. 1a). The dynamic inverse model mathematical framework suggests that humans learn the forward model M and implement the inverse model as a feedforward controller, F = M −1 . By introducing a reference trajectory r as well as a disturbance trajectory d together and separately, we are able to separate subject feedforward and feedback models.
−BM F +B R+ D, 1 + BM 1 + BM Hur
(1)
Hud
Y = (U + D)M .
(2)
Or conversely, we estimate the feedforward and feedback controllers as functions of the empirical transforms and the prescribed system model: B=
−Hud , M (1 + Hud )
(3)
Hur + M −1 Hud . (4) 1 + Hud Hypothesis 2.2. The combined feedback and feedforward model predicts user responses better than a model implementing only feedback control. The feedforward pathway is critical in modeling user responses. F =
3. EXPERIMENTAL METHODS 3.1 Overview of Experimental Setup The reference and disturbance signals comprised a scaled and phase-shifted sum of sines with signal frequencies at the prime harmonics of a fundamental frequency 0.05 Hz, spanning 0.1-1.55 Hz (Fig. 2). These were presented to the subject in a window one display unit tall (approx 5.3cm) and 4 display units high. The phase shifts were randomly chosen, and the magnitude of the sine wave at each frequency was scaled to 1/f for the first-order system (i.e. , constant velocity), and 1/f 2 for the secondorder system (i.e. , constant acceleration), such that the larger frequencies had a lower amplitude. The participants were instructed to slide a linear joystick (a 10 kΩ linear potentiometer) to modulate the input U . The linear potentiometer had a 10cm potential traveling distance. Trial trajectories were designed such that the ideal feedforward input trajectory would be constrained to one-third of the allowable range of inputs, thus avoiding saturation of the input signal. Each trial lasted for 40 seconds–two complete periods of the pseudorandom signal–and the user input u and output y were sampled at 60 Hz (u measured using an Arduino Due).
Fig. 1. a. The block diagram of HCPS highlights the feedback and feedforward human controller separation that combines to a measured user input U . Our experimental assays prescribe the reference and disturbance signals, r and d, as well as the cyber-physical model, M , and are designed to enable estimation of the distinct contributions of feedforward and feedback processes. b. A human operator completing the computer trajectory tracking task using a slider.
Table 1. Assays Presented To Human Operator Hypothesis 2.1. The user input for reference tracking with disturbance is consistent with a superposition of the responses to the reference trajectory and the disturbance signals presented individually.
Order Assay # Trials
62
1 r+d 2
2 d 10
3 r+d 2
4 r 10
5 r+d 10
IFAC CPHS 2018 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
63
Fig. 2. The three types of assays used in the study shown as an FFT of the 40 second signal. The resolution of the axis is 0.05 Hz. The blue and red lines indicate frequency content for D and R respectively. Two types of alternating frequencies were prescribed - one where R had frequency content on the even primes and D on odd, and one where R had frequency content on the odd primes and D on even. as a measure of spread. For all trials, circular statistics were used to unwrap the angle of the transfer functions prior to analyzing the data.
Three assay categories were presented to the operator: 1. only r with d = 0; 2. only d, with r = 0; and 3. r + d, with each at alternating frequencies such that for one frequency, either r = 0 or d = 0 (Fig. 2). For trials in which reference and disturbance signals were presented simultaneously, each signal contained alternating frequencies from the above set, even or odd components interleaved. In consecutive trials, the frequency support of r and d would be swapped. The r trajectories were displayed as a gold path having a look-ahead preview of 0.25 seconds; there was no visual representation of the disturbance, d, and consequently no look-ahead. The phase shifts for all frequency components were randomized for each trial, and the three assays were presented in sequence (Table 1).
A feedback only model and a feedback and feedforward model was estimated using the D only and R only trials to determine whether the feedback and feedforward model had a better fit to the data than the feedback only model. The transfer functions were then applied to R and D in the R+D trials which were not a part of the training data, and the percent error between the predicted Up and measured Um was calculated. The IQR of the error was calculated for both models. F and B were calculated as described in Eqns. 3, 4 to determine the extent to which F was similar to the model inverse M −1 . A Wilcoxon Signed-Rank test was performed to quantify this similarity.
Two models were presented sequentially to observe model inversion in a simpler first-order (FO) model and a more complex second-order (SO) model, with, 1 (5) mF O : y˙ = u; MF O : , s 1 . (6) mSO : y¨ − y˙ = u; MSO : 2 s −s
4. RESULTS 4.1 Linearity of HCPS The moderate coherence between the r and r + d trials (median, range Cr,r+d = 0.84, 0.21 − 1.0) and d and r + d trials (median, range Cd,r+d = 0.82, 0.10 − 0.99) demonstrated that r + d trials may be well predicted from r and d trials by a linear least squares function (Fig. 3). This suggests that there may be a linear relationship between the system inputs r, d and the system output y. The Kruskal Wallis test demonstrated that there was a statistical difference in the medians of both HU R and HU D with p < 0.001 and p = 0.014 respectively. The posthoc rank sum test indicated that there was a significant difference in the median coherence between frequencies at 0.35 Hz and lower and 1.15 Hz and higher for HU R , and 0.55 Hz and lower and at 1.55 Hz for HU D . This suggests that there is a lower overall coherence with increasing frequency, especially at 1.15 Hz and above, whereas there is a high coherence at 0.35 Hz and below.
For second order models, the spectrum of the stimuli was truncated to 0.1-0.85 Hz. 3.2 Experimental Data Analysis
Data for seven (7) participants were analyzed with Python 2.7.3. Sampled signals were converted into their frequency representations using the fast Fourier transform (FFT). Coherence was calculated in the time domain using the python function scipy.signal.coherence() between the r, d and r + d trials at frequencies of interest (Fig. 2). The Kruskal-Wallis test was conducted to determine whether the median coherence of each frequency came from the same population, and the rank sum test with a Bonferroni correction was conducted as a post-hoc test from the scipy.stats package. HU D and HU R in the frequency domain were also calculated at each frequency of interest as in Eq. 1 to assess linearity and superposition of the HCPS. A Wilcoxon Signed-Rank test was performed to determine the extent to which the medians of HU D and HU R from the D, R trials were similar to the values calculated from the R + D trials. The interquartile range (IQR) was calculated
Further investigation into the transfer functions HU R and HU D for the first-order system demonstrated a high degree of overlap between the R and R + D trials and D and R + D trials (Fig. 4). The Wilcoxon Signed-Rank test indicated that with α < 0.05, there was no statistical difference in HU R and HU D in either average magnitude 63
IFAC CPHS 2018 64 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
Fig. 3. Coherence values calculated between the r and d only trials and r + d trials demonstrate high coherence between the two for lower frequencies, whereas the coherence at higher frequencies show a larger variability between subjects.
Fig. 4. Transfer functions for D to U : HU D = U/D and R to U : HU R = U/R demonstrate overlap for the transfer functions in the R and D only trials and R + D trials. The shaded blue and red areas represent the IQR of HU D and HU R respectively from the D only and R only trials; the blue triangle represents individual HU D values and the red circle represents individual HU R values calculated from the R + D trials. or angle between the only R or only D trials and the R+D trials. Similarly to the coherence analysis, the higher frequencies tended to have a larger spread in variability across individuals for both the R only and D only trials as well as the R + D trials. The positive results from this superposition test indicates that the HCPS can be thought of and analyzed as a linear system for lower frequencies below 0.35 Hz.
forward and feedback control strategies. We verified that when a feedback only model was fitted to a subsection of the data (R only and D only trials for first-order model) and tested on a separate set of data (R + D trials for first-order model), the error between the predicted Up and measured Um was worse than the error from the fitted feedback and feedforward model at all frequencies (Fig. 5), with the error roughly twice as high for the feedback only model compared to the combined feedback and feedfor4.2 Feedback and Feedforward Combined Controllers Versus ward model. The error linearly increases in the log-log Feedback Only Controller plot, suggesting that for the first-order model, the error is inversely proportional to the amplitude of the shown As proposed in hypothesis 2.2, human operators may con- signal. trol cyber-physical systems with a combination of feed64
IFAC CPHS 2018 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
65
only agrees with the inverse of the cyber-physical system model at low frequencies at 0.15 Hz and below. At higher frequencies at 0.25 Hz and above, we demonstrated that the magnitudes are lower than the expected magnitude of M −1 by roughly a factor of 0.5 (Fig. 6). This was corroborated with our statistical tests, which demonstrated that with an α < 0.05, the median for both models did not have M −1 as the median for either the magnitude or the angle. This result is consistent with the coherence values calculated in Fig. 3, which demonstrated that lower frequencies at 0.35 Hz and below tended to have a higher median coherence and smaller variability than the higher frequencies, suggesting a more linear HCPS at lower frequencies than higher frequencies. The transfer functions for HU D and HU R had similar results where frequencies larger than 0.55 Hz tended to have a much larger variability in both magnitude and angle than the lower frequencies.
Fig. 5. The percent error between the predicted input and actual user input for the feedback only model (blue shaded area) and feedback and feedforward model (red shaded area) demonstrates lower percent error at all frequencies for the feedback and feedforward model. 4.3 Dynamic Model Inversion
This decrease in linearity with increase in frequency may be due to a number of reasons, including the strength of the signal provided and the limits of the neuromuscular system. For this study, we chose to provide human operators with a lower signal amplitude for higher frequencies inversely proportional to the frequency for firstorder, and inversely proportional to the square of the frequency for the second-order system, which will magnify any errors that the human operator makes while tracking these higher frequencies. Another possible explanation is neuromuscular limitations. The human visuomotor limitation suggests that the fastest motor reaction time for humans (as demonstrated by asking a human subject to press a button upon the presentation of a certain visual) is about 5 Hz (Wardle, 1998). Biomechanical limitations may also contribute to the performance drop-off at higher frequencies. Our assay asks our participants to move at speeds up to 20 cm/s. While this is not a slow movement, it is a much slower than the speeds required to complete the reach-out tasks for example, where participants move at speeds of 50 cm/s (Shadmehr and Mussa-Ivaldi, 1994). In addition, the maximum absolute jerk required to perfectly track the reference is 75cm/s2 . As such, we cannot conclude from our results whether the nonlinear response at higher frequencies is due to the breakdown of the model inversion model at higher frequencies or to neuromuscular limitations.
The feedback and feedforward estimates pooled across all individuals and trials demonstrate large deviations in magnitude from M −1 for the feedforward controller for both R and R + D trials (Fig. 6). The Wilcoxon SignedRank test performed on the median FR+D calculated from the R+D trials for all frequencies demonstrated that there was a significant difference between the feedforward controller and M −1 for the magnitudes (first-order: p = 0.006, second-order: p = 0.018). The angle of the experimental feedforward block was significantly different from M −1 for the first order M (p = 0.004), but not for the second order (p = 0.18). Similarly, for the F calculated from the R only and D only trials, there was a significant difference in the magnitudes between the FR,D and M −1 (first-order: p = 0.004, second-order: p = 0.04) and the angle for the first-order model (p = 0.006), while we again failed to verify a difference in the angles for the second-order system (p = 0.74). Looking more closely at whether there is a difference between the median F and expected M −1 magnitude at each frequency, we found that the magnitude was consistently lower than the expected M −1 at all frequencies except 0.1 Hz and 0.15 Hz, as demonstrated by the Wilcoxon SignedRank test (p < 0.05) for both the first and second-order system. The magnitude of the higher frequencies at 0.25 Hz and above were lower than the expected M −1 magnitude by a median factor of 0.54 (range: 0.43 − 0.70) for the first-order system and a median factor of 0.59 (range: 0.41 − 0.69) for the second-order system.
While the feedback and feedforward model can do no worse than the feedback only model in predicting the output of the training dataset, the increased number of variables in the feedback and feedforward model may lead to overfitting and thus, large errors in the validation dataset. Using the final R + D trials as a validation set, we confirmed that the combined feedback and feedforward model had less error predicting the user response at all frequencies as compared to the feedback-only model. This result suggests that we did not overfit, and in fact, both the feedforward and feedback controllers lead to a better model of the HCPS.
5. DISCUSSION We estimated feedback and feedforward elements of human sensorimotor control in a trajectory tracking task. Estimates were obtained for two models (MF O = 1/S and MSO = 1/(S 2 − S)) using two different assays; both estimates agreed to within experimental error for frequencies up to 0.15 Hz. Furthermore, including the feedforward pathway roughly halves the human input prediction error, suggesting both feedforward and feedback elements may be required to predict human control of cyber-physical systems.
6. CONCLUSION This paper proposes a novel experiment design that enables estimation of human feedback and feedforward transformations in trajectory tracking tasks directly from em-
In contrast to the recent results reported in (Zhang et al., 2018), the estimated feedforward transformation 65
IFAC CPHS 2018 66 Miami, FL, USA, Dec. 14-15, 2018
Momona Yamagami et al. / IFAC PapersOnLine 51-34 (2019) 61–66
Fig. 6. The experimental values of F and B demonstrate for both first and second order closer model inversion for lower frequencies than higher frequencies in magnitude. The red dotted line represents the model inverse, the shaded red and blue is the IQR for F and B calculated from R only and D only trials, and the dark blue triangle and dark red circle represents F and B calculated from R + D trials for each individual. . pirical frequency responses to reference and disturbance inputs. Our results support the hypothesis that humans learn to use the inverse of the cyber-physical system model in their feedforward control pathway only at frequencies at 0.15 Hz and below, and the estimated feedforward transformation for higher frequencies resembles a scaleddown version of the model inverse. Further investigations are needed to characterize the feedforward transformations learned by human operators.
Gawthrop, P., Loram, I., Lakie, M., and Gollee, H. (2011). Intermittent control: A computational theory of human control. Biological cybernetics, 104(1-2), 31–51. McRuer, D.T. and Jex, H.R. (1967). A review of quasilinear pilot models. IEEE Transactions on Human Factors in Electronics, (3), 231–249. Robinson, R.M., Scobee, D.R., Burden, S.A., and Sastry, S.S. (2016). Dynamic inverse models in humancyber-physical systems. In Micro-and Nanotechnology Sensors, Systems, and Applications VIII, volume 9836, 98361X. International Society for Optics and Photonics. ACKNOWLEDGEMENTS Roth, E., Howell, D., Beckwith, C., and Burden, S.A. (2017). Toward experimental validation of a model for This research is supported the National Science Foundahuman sensorimotor learning and control in teleoperation’s CISE CPS Program (Awards #1565529, 1836819), tion. In Micro-and Nanotechnology Sensors, Systems, the Air Force Office of Scientific Research (Award #FA9550and Applications IX, volume 10194, 101941X. Interna14-1-0398), and the Washington Research Foundation tional Society for Optics and Photonics. Funds for Innovation in Neuroengineering. Shadmehr, R. and Mussa-Ivaldi, F.A. (1994). Adaptive representation of dynamics during learning of a motor REFERENCES task. Journal of Neuroscience, 14(5), 3208–3224. Abbink, D.A., Mulder, M., and Boer, E.R. (2012). Haptic Wardle, D. (1998). The time delay in human vision. The shared control: smoothly shifting control authority? physics teacher, 36(7), 442–444. Cognition, Technology & Work, 14(1), 19–28. Wolpert, D.M. and Kawato, M. (1998). Multiple paired Allen, R.W. and McRuer, D. (1979). The man/machine forward and inverse models for motor control. Neural control interface pursuit control. Automatica, 15(6), networks, 11(7-8), 1317–1329. 683–686. Yu, B., Gillespie, R.B., Freudenberg, J.S., and Cook, J.A. Desmurget, M. and Grafton, S. (2000). Forward modeling (2014). Human control strategies in pursuit tracking allows feedback control for fast reaching movements. with a disturbance input. In 2014 IEEE 53rd Annual Trends in cognitive sciences, 4(11), 423–431. Conference on Decision and Control (CDC), 3795–3800. Drop, F.M., Pool, D.M., Damveld, H.J., van Paassen, IEEE. M.M., B¨ ulthoff, H.H., and Mulder, M. (2012). Identifica- Zhang, X., Wang, S., Hoagg, J.B., and Seigler, T.M. tion of the transition from compensatory to feedforward (2018). The roles of feedback and feedforward as humans behavior in manual control. In 2012 IEEE International learn to control unknown dynamic systems. IEEE Conference on Systems, Man, and Cybernetics (SMC), transactions on cybernetics, 48(2), 543–555. 2008–2013. IEEE. Gawthrop, P., Loram, I., and Lakie, M. (2009). Predictive feedback in human simulated pendulum balancing. Biological cybernetics, 101(2), 131–146. 66