ARTICLE IN PRESS
JID: SIMPAT
[m3Gsc;July 12, 2016;15:5]
Simulation Modelling Practice and Theory 0 0 0 (2016) 1–18
Contents lists available at ScienceDirect
Simulation Modelling Practice and Theory journal homepage: www.elsevier.com/locate/simpat
Towards a simulation-based tuning of motion cueing algorithms Sergio Casas∗, Inmaculada Coma, Cristina Portalés, Marcos Fernández Institute on Robotics and Information and Communication Technologies (IRTIC), University of Valencia, Spain
a r t i c l e
i n f o
Article history: Received 5 February 2016 Revised 3 June 2016 Accepted 3 June 2016 Available online xxx Keywords: Motion cueing algorithm Simulation Tuning Heuristics Virtual reality
a b s t r a c t This paper deals with the problem of finding the best values for the parameters of Motion Cueing Algorithms (MCA). MCA are responsible for controlling the movements of robotic motion platforms used to generate the gravito-inertial cues of vehicle simulators. The values of their multiple parameters, or coefficients, are hard to establish and they dramatically change the behaviour of MCA. The problem has been traditionally addressed in a subjective, partially non-systematic, iterative, time-consuming way, by seeking pilot/driver feedback on the generated motion cues. The aim of this paper is to introduce a different approach to solve the problem of MCA tuning, by making use of a simulated motion platform; a series of (human-based) objective metrics relating to the performance of MCA are measured using this simulated device. This simulation-based approach allows for automatic tuning of the MCA, by using a genetic algorithm that is proposed to analyse the results obtained from multiple simulations of the MCA with different parameters. This algorithm is designed to efficiently optimize the simulated MCA parameter space. The proposed solution is assessed with the classical washout MCA, performing a series of tests to validate the correctness of this approach and the suitability of the proposed method to the solution of the MCA tuning problem. Results show that this approach can be an alternative to the traditional subjective tuning method in certain situations, mainly because it provides suitable values for the MCA parameters in a shorter time period, albeit subjective tuning is preferred when time to perform the MCA tuning is not an issue. © 2016 Elsevier B.V. All rights reserved.
1. Introduction and related work The tuning of Motion Cueing Algorithms (MCA) is a real problem that researchers must solve when they use a vehicle simulator with a robotic motion platform, and it is usually addressed in a human-centered, subjective, partially nonsystematic approach. Motion platforms are commonly used in real-time vehicle simulators of different types (aircraft, cars, trucks, ships, etc.) in order to provide the user with the most suitable gravito-inertial cues [1]. It is well known that the inclusion of such motion cues increases the realism of the simulation, eases several control tasks of the vehicle [2,3] and improves the operator’s performance [4–6]. It is important to clarify that the term “operator” refers to “vehicle operator” and may include “pilots” and/or “drivers”. A motion platform consists of a number of actuators that are used in an attempt to reproduce the translational and angular movements of a vehicle. To achieve this, the simulator requires a module to convert the virtual physical forces ∗
Corresponding author. Tel.: +34 963543588. E-mail address:
[email protected] (S. Casas).
http://dx.doi.org/10.1016/j.simpat.2016.06.002 1569-190X/© 2016 Elsevier B.V. All rights reserved.
Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT 2
ARTICLE IN PRESS
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
and angular velocities calculated by the computer model of the simulated vehicle into real motion of the actuators. As the motion platform is constrained by its mechanical limits, the values are usually scaled down and filtered to fit within the bounds of the robotic manipulator. The task of converting and filtering these values is performed by an MCA, commonly called “washout algorithm/filter”. Since motion platforms very rarely can provide accurate motion cues compared to the real-world, the MCA tries to generate suitable motion cues for the users. Thus, an MCA must offer the operator the best motion perception possible while keeping the motion platform within its physical limits. These two goals are often mutually opposed, so the solution to the problem is not simple, and several parameters or coefficients are required to control what kind of movements are scaled/modified to fit the physical limits of the hardware, and how. There are many different types of MCA [7] such as the classical algorithm [8], the adaptive algorithm [9,10], the optimal control approach [11,12], the quasi optimum control technique [13,14] or the spherical washout filter [15]. However, the classical algorithm, with small variations, is still one of the most commonly used [16]. The design of washout algorithms can be complicated. Moreover, it may depend on the type of simulator, the features of the simulated vehicle and the kind of manoeuvres to be reproduced. However, all of them have a set of parameters that (significantly) modify the behaviour of the algorithm [17]. The assignment of values to these parameters is not straightforward, especially with some algorithms (such as the optimal control approach or the adaptive algorithm). Previous research has shown that users’ perception of the motion cues depends significantly on how well the algorithm has been adjusted [18]. Moreover, the use of a motion platform with poor motion cues can produce simulator sickness [19] and can potentially be even worse than no motion. Simulator sickness is the discomfort that some users experience during or/and after a session in a simulated environment [20]. Therefore, to benefit from the advantages of motion platforms, the problem of determining the best values for the MCA parameters has to be solved first. The search for the appropriate MCA and parameter configuration is a difficult task and several researchers have evaluated the impact of different washout algorithms and different parameterizations on operator’s perception and on operator’s performance. Research suggest that subjects prefer motion-based simulators to static set-ups [21,22,23], although the impact of the MCA type on operator’s performance measurements is small [2,21,22,27]. However, it appears that the different MCA compared [2,21,23,16] cause differences in subjective opinions. Regarding MCA parameter configuration, some researchers identify important differences in both performance and operator subjective perception [24,25,26] for different parameter set-ups. However, some authors overlook that, to perform the MCA parameter tuning, a formal evaluation system needs to be defined first in order to compare different instances of the MCA with different parameters and provide a measure of the simulator motion fidelity. Motion fidelity is the degree to which the motion cues match those of a real vehicle. Motion fidelity is also affected by the motion platform hardware, the vehicle model and task-related factors. A widely used criterion for evaluating motion fidelity was developed by Sinacori [1]. MCA motion fidelity can be measured both subjectively and objectively. Subjective evaluation is performed by seeking operator feedback about their impressions of the motion cues. This process can be very subjective. Thus, a subjective fidelity scale has been proposed by Hodge [21]. Objective evaluation is accomplished by measuring objective data from the simulator. These objective data are usually collected in the form of acceleration and angular velocity, although other variables (including those related to the operator performance or control behaviour [22]) can be measured. Parameter tuning is usually performed in a so-called pilot-in-the-loop process with subjective evaluations of the MCA. This approach requires a pilot/driver (or a group of them) and an expert with a great deal of knowledge about the simulated vehicle, the motion platform and the washout algorithm [23]. The process starts with an initial assignment of values to the parameters and an initial assessment by the operators. After this, the subjective impressions of the operators are reported to the expert, who makes the necessary changes to address their complaints, when possible. The process ends when the expert and the operators are satisfied with the set of parameters selected. This process is rather time-consuming as it may require several hours of testing, and the results are subjective because they depend on the selected operators. Besides, although the subjective evaluation can make sense as the main goal is to satisfy the operators, the main drawback is that there is no ending criterion to determine when to stop and finish the adjustment, and the process often completes when the participants are fatigued. Sometimes, an expert system is used as in the PROTEST project [24,25] which helps reduce the effort necessary to complete the process of selecting the most appropriate values for the MCA parameters. The subjective pilot-in-the-loop process is used by several researchers [2,26,27]. In recent years, objective evaluation of motion cueing systems has also been considered. In this regard, the International Civil Aviation Organization (ICAO) has proposed an objective evaluation of flight simulators. ICAO 9625 simulator qualification guidelines [28] includes a method to compare motion systems: the Objective Motion Cueing System Test (OMCT) [29]. This is a frequency-domain test designed to supplement subjective evaluations of simulators’ physical motion. OMCT is a very comprehensive assessment method, but it is designed primary for aircraft (although it can be potentially adapted for other vehicles) and it is not designed to be used for objective automatic MCA tuning, because it does not provide a single performance measure but a list of valid ranges and graphs explaining the performance of the simulator [30]. OMCT, nonetheless, proves that objective evaluation of motion systems is not only necessary, but also feasible. It represents a starting point of a promising research field, where an interesting debate between objective and subjective evaluation and tuning is taking place [31]. To ease the process of tuning a washout algorithm, a simulation-based automatic objective procedure is proposed to find the optimal values of the MCA parameters. The following ideas have been developed. First, a simulated motion Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT
ARTICLE IN PRESS S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 3
platform has been designed and implemented, so that the actual motion platform is not required to test the different algorithm configurations [32]. This virtual motion platform allows testing a large number of different parameter configurations faster than in real time with the actual motion platform. Then, to eliminate the need for the subjective evaluation of the washout algorithm, a characterization of users’ perception of the motion cues provided by the MCA was performed [33]. This characterization provides different objective metrics to evaluate the possible configurations of the algorithm. The next step is to test multiple instances of the washout algorithm and evaluate them using the virtual motion platform and the selected metrics, in order to find the parameters that give the best results according to the selected objective indicators. Since all MCA have a high number of parameters (around 30 in the case of the Reid–Nahon implementation [8] of the classical algorithm) and they have large ranges of variation, the parameter space of this kind of algorithm can be vast. Therefore, the problem of finding the best set of parameters for a given criterion is an optimization problem, which can be solved by seeking suboptimal solutions that try to maximize/minimize the value of the evaluation function. The aim of this paper is to deal with the tuning of MCA by proposing a new method to adjust the parameters of any MCA in a completely novel fashion, reducing the temporal and economic cost of manual tuning by making use of simulation techniques, objective criteria and heuristic approaches. The problem is addressed by implementing a genetic algorithm [34] that looks for the best set of parameters given an objective criterion. As the goal of this work is to demonstrate that the use of a simulated motion platform can be helpful to solve this problem, the focus is not on finding the fastest heuristic to solve this optimization problem, which is a different and challenging problem, but on showing that the solutions provided in this simulation-based automated objective process result in motion cues that are acceptable to operators. In this regard, the authors are aware that subjective tuning of MCA is always operator-tailored, and thus, objective tuning methods could be an alternative in certain situations, but cannot substitute subjective tuning as a general rule. The use of a heuristic approach to solve this kind of problem has only been reported before by Thöndel [35] without giving many details of the algorithm and the results. He uses Matlab–Simulink Optimization Toolbox but he did not detail how the heuristic scheme is adapted to this problem. In addition, the proposed method is partially automatic and the fitness function is not human-based. Nevertheless, heuristic strategies are widely used in similar problems of parameter tuning [36] and many examples of this kind of solution are described by Hoos [37]. The remainder of the paper is organized as follows. Section 2 describes and formalizes the problem. Section 3 describes the proposed solutions. The computation experiments performed and their results are described in Section 4. Finally, Section 5 summarizes the conclusions.
2. Problem description 2.1. The “washout” algorithm As discussed, a washout algorithm filters the motion signals of the simulator and limits their values to ensure that the mechanical bounds of a particular motion platform are not exceeded. The motion sensation is created by limiting translations and rotations to certain frequency bands and by replacing sustained linear translation with tilt motion [18]. Moreover, the MCA also brings the motion platform to its neutral position in anticipation to the next movement, hence the name “washout”. All the MCA schemes available in the literature rely on this basic principle, albeit with different implementation approaches, so this study will focus on the most widely used MCA: the classical washout algorithm. The classical MCA used in this paper follows the Reid–Nahon scheme [8]. There are many different implementations of this algorithm, but this is the most commonly used. The scheme works as follows (see Fig. 1). As the motion perception system of the human body is sensitive to both linear specific force and angular velocity, the inputs to the algorithm are the simulated vehicle specific force ({Fx , Fy , Fz } measured in m/s2 ) and angular velocity ({ωx , ωy , ωz } measured in deg/s or rd/s), both in body-frame. The output is the motion platform 6-component pose {sway, surge, heave, yaw, pitch, roll} that is supposed to create the appropriate sensation on the operator. Sway, surge and heave represent the X, Y, Z translation. Yaw (or heading), pitch and roll, are known as the Euler angles. They describe orientation as three rotations about the axes of a coordinate system. Roll represents rotation around the forward axis, pitch represents rotation around the side axis, and yaw, around the up axis [38]. The algorithm consists of three different channels: a translational, a rotational, and a coordination channel (see Fig. 1). The translational channel is designed to generate translational motion. As the motion platform workspace is limited, this channel only reproduces high frequency displacements. The channel is composed of a Translational Amplifier (TA), which scales the input specific force; a Translational Transformation (TX) which transforms the signals into inertial-frame acceleration, a Translational High-Pass Filter (THPF) which eliminates low-frequency accelerations (hence only high frequency displacements are reproduced in this channel) that would otherwise cause the device to hit its limits, and finally, a Double Integrator (DI) which provides sway (X), surge (Y) and heave (Z) motion platform displacements. The rotational channel is similar to the translational channel, but it operates with angular velocities, instead of specific forces. The principle is similar: to reproduce high-frequency angular displacements and avoid low-frequency displacements. Angular velocities are scaled (RA – Rotational Amplifier), then transformed into Euler angles rates (RX – Rotational Transformation), high-pass filtered (RHPF - Rotational High-Pass Filter) and finally integrated (I - Integrator) to obtain Euler angles Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT 4
ARTICLE IN PRESS
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
Fig. 1. Classical washout scheme.
Fig. 2. Classical washout operation example for one axis.
(yaw, pitch and roll). Yaw is directly output, but pitch and roll need to be combined with the output of the coordination channel, described next. Fig. 2 shows in detail how the translational channel works. Only one translational motion is shown for simplicity. The operation of the rotational channel is fairly similar. The coordination channel tries to simulate low-frequency accelerations (previously eliminated by the translational channel) with slow rotations, by tilting the motion platform. This illusion is called tilt-coordination [39,40] and takes advantage of the somatogravic illusion [41] by using gravity as a way to simulate low-frequency accelerations (see Fig. 3). The basis of the tilt-coordination technique is the “perceptually equivalent situations”, described by Holly [42]. The vestibular system Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT
ARTICLE IN PRESS S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 5
Fig. 3. Tilt coordination and somatogravic illusion.
cannot distinguish between an inertial force due to linear acceleration together with gravity (Fig. 3 A) and a tilt of the head (Fig. 3 B), if angular velocity is not too large. Tilt-coordination can only be performed for lateral (X) and longitudinal (Y) accelerations. The process starts with the same scaling process (TA) of the specific force. Then, a low-pass frequency filter is applied (LPF – Low Pass Filter) in order to eliminate high-frequency signals. This low-frequency acceleration is converted (TC – Tilt Coordination) into pitch and roll tilt angles that are limited by a rate-limiter (RL – Rate Limiter), in order for the trick to not be confused with some angular displacement. This rate-limiter tries to keep the angular velocity associated with tilt-coordination at a level below that which the operator is likely to sense. Since tilt-coordination is a trick designed to deceive the operator’s senses, it is important that the operator not sense this rotation. Most of the processing blocks of the algorithm can be parameterized, and the choice of the parameters significantly modifies its performance. For instance, the translational amplifier (TA) has three parameters, which represent the scaling factors of each translational motion. The ideal value is unity, which means that full-scale accelerations are input into the algorithm. However, this usually causes the motion platform to reach its limits, so values less than unity are often necessary. Similarly, the rotational amplifier (RA) has three parameters, which represent the scaling factors of each rotational motion. The highpass filter in the translational channel (THPF) is a 3-signal filter with nine parameters, as filtering is usually performed with 2nd order linear filters. This class of filter possesses three parameters for each signal: break frequency (also known as cut-off frequency), gain, and damping [43]. Arguably, the most important parameter is the break frequency. This value establishes the limit, above which, signals are able to pass into the double integrator (DI). The functioning of the high-pass filter in the rotational channel (RHPF) is similar, but input signals are Euler angles rates. The filter in the coordination channel (LPF) is a 2-signal low-pass filter (Z translation is not input because there is no equivalent tilt-coordination for Z translation) that acts on specific force with the same type of parameters as in the high-pass filters. Of course, although an ideal filter would act as a switch – removing all undesirable frequency bands – real filters attenuate (but don’t remove completely) undesirable frequencies [43]. The tilt-coordination process is a two-stage process. The first stage (TC) can also be parameterized, by setting the amount of tilt that would provide 1 m/s2 of specific force. Therefore, a tilt scale and a tilt limit need to be defined. The second stage of the tilt-coordination is a rate limiter (RL). This process ensures that the motion platform performs the tilt-coordination at low angular velocities. This is done because the tilt should not be felt as a rotational motion by the operator but as a steady linear acceleration [44] . This process has only one parameter, the rate limit (expressed in °/s), which establishes an angular speed limit for the tilt angles calculated by the TC module. For a more detailed explanation of the classical algorithm and the meaning of its parameters, see [8,18,45]. It is important to emphasize that, although the classical washout is used in the tests, the tuning method that is presented here can be used with any MCA, regardless of how many parameters they have or their meaning. The values of these parameters (see Table 1) affect magnitude distortion and/or phase distortion [1]. Magnitude distortion measures the gain between the vehicle model and the motion platform accelerations. Phase distortion measures the phaseshift between those signals. Scale factors (including filter gains) and tilt limits mainly affect magnitude distortion, while rate limits, break frequencies and dampings, mainly affect phase distortion. However, due to the complexity of the algorithm, reducing the distortion of a Degree of Freedom (DoF) may increase the distortion of other DoF. Therefore, it is necessary to identify a method to find the best set of values for these parameters, given an established objective criterion for the evaluation of the MCA motion fidelity. 2.2. MCA evaluation and objective indicators Unlike other disciplines and research areas, motion cueing lacks a standardized assessment mechanism. In this section, an objective evaluation of washout algorithms is proposed, based on a characterization of the users’ perception when using MCA [33]. The evaluation is performed by comparing the expected motion of the vehicle (calculated by the physics module of the vehicle simulator) with the motion experienced by the operator on the motion platform. The comparison is performed Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
ARTICLE IN PRESS
JID: SIMPAT 6
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18 Table 1 Classical washout parameters. Module
Parameter
Description
TA RA
Scale Scale Break frequency Gain Damping Break frequency Gain Damping Tilt limit Tilt scale Rate limit Break frequency Gain Damping
X, X, X, X, X, X, X, X, X, X, X, X, X, X,
THPF
RHPF TC RL LPF
Y, Z translational scale factors Y, Z angular scale factors Y, Z translational break frequencies Y, Z translational gains Y, Z translational damping factors Y, Z angular break frequencies Y, Z angular gains Y, Z angular damping factors Y maximum tilt-coordination angle Y tilt-coordination scale factors Y maximum tilt-coordination speed Y angular break frequencies Y angular gains Y angular damping factors
Fig. 4. MCA evaluation scheme.
on a signal-by-signal basis, comparing both specific force and angular velocity, which are the usual inputs of an MCA and also the inputs of the human vestibular system responsible for motion perception [46]. As both specific force and angular velocity are measured in the 3-D space, 6 signals: {Fx , Fy , Fz , ωx , ωy , ωz } need to be compared. The objective evaluation can be performed in two ways (see Fig. 4). The first option (option A) compares the real physical state of the operator on the motion platform versus the expected (simulated/virtual) physical state generated by the physics module of the vehicle simulator (this measures the physical validity of the simulator [40]). The second option (option B) compares the perception of the pilot/driver on the motion platform versus the expected perception on a real situation (this corresponds to perceptual validity [40]). In both cases, if the operator is placed on the motion platform at a position that does not match the position of the operator in the real vehicle, a module that calculates the physical magnitudes at the Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
ARTICLE IN PRESS
JID: SIMPAT
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 7
operator position is necessary. Option B needs to define how the motion estimation is performed by the human body, so a human perception model is required. As no consensus has been identified about human perception modelling, this choice can be a controversial issue. In this regard, one of the most acknowledged efforts in this field is MPT (Motion Perception Toolbox), developed at TNO/Delft University as a Matlab-Simulink library [47]. Fig. 4 summarizes the assessment process, including both options. The signal comparison (either with option A or B) is performed by a series of mathematical indicators that compare the 6 signals of the real physical state (the one experienced by the operator) versus the 6 signals of the simulated (or expected) physical state (calculated by the physics module of the simulator), on a signal-by-signal basis. The 6 indicators are later combined into a single objective number that represents the performance of the MCA for the given test, either additively or multiplicatively [33]. The mathematical indicators are detailed and assessed in a previous paper [33] and are not the primary focus of this paper. They are based on the hypothesis that the MCA would take the input signals {Fx , Fy , Fz , ωx , ωy , ωz } and would try to generate output signals (to feed the motion platform) that would be: slightly distorted (by the effect of frequency filtering), somehow scaled down in magnitude (by the input scale processing modules) and slightly delayed (because of the inertia of the motion platform and the effects of some MCA processing modules). The indicators used in this work are: Normalized Average Absolute Difference (NAAD), Average Absolute Scale (AAS), Normalized Pearson Correlation (NPC) and Estimated Delay (ED). A natural way of assessing an MCA is to test it for a specific amount of time (for instance, a lap when using a racing simulator) and analyse the generated signals. With x being the input signal (generated by the simulator), y the signal fed to the motion platform (by the MCA), and n the number of samples of the signals, the formulation of the indicators is the following:
NAAD(x, y ) = 1 +
n 1 · |xi − yi | n · |x p |
(1)
i=1
where xp represents the peak value of the x signal.
AAS (x, y ) =
⎧ 1 ⎫ ⎪ i f c (x, y ) < 1⎪ ⎪ ⎪ ⎨ c(x, y ) ⎬ ⎪ ⎪ ⎩
1 i f c (x, y ) = 1 c (x, y ) i f c (x, y ) > 1
⎪ ⎪ ⎭
(2)
with
c (x, y ) =
n 1 xi · n yi
(3)
i=1
NPC (x, y ) =
1 1 + 1 + PC (x, y ) 2
with
PC (x, y ) =
ED(x, y ) = 1 +
(4)
n
(xi − x¯ ) · (yi − y¯ )
2 n (xi − x¯ )2 · i=1 (yi − y¯ ) i=1
n i=1
∂ (x, y )
(5)
(6)
where ∂ (x, y) represents the offset that needs to be applied to the input signal of the MCA to maximize the Pearson Correlation with respect to the motion generated by the motion platform. All of the indicators can be used independently but they can also be combined to create more sophisticated objective indicators. It can be observed that these indicators are designed to report values greater than or equal to unity, with the latter being the best value possible, and the greater the number the poorer the performance of the MCA. Thus, any optimization method will try to find the set of parameters for the MCA that minimizes the objective indicator used. 2.3. Virtual motion platform A motion platform (MP) is a powered, mechanical and self-contained complex motion system (see Fig. 5). Motion platforms are used for immersive applications and motion simulation in a wide range of scenarios (automotive, maritime, aircraft, etc.) and applications (training, entertainment, medical rehabilitation, industry, etc.). In MCA tuning problem, as many tests over the motion platform are needed in order to find an optimum set of parameters for the MCA, there is a risk of damaging the simulation hardware by making strong movements or by overuse. Moreover, some of these tests can be quite time consuming and, as it is desirable to test as many combinations of the MCA parameters as possible, it would be very advisable to perform these tests faster than real-time. Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT 8
ARTICLE IN PRESS
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
Fig. 5. Example of a 6-DoF motion platform.
Fig. 6. Virtual motion platform corresponding to the 6-DoF motion platform in Figure 5.
For these reasons, the authors propose to perform all the MCA tests on a simulated motion platform instead of using the actual platform. A simulated version of the motion platform will require the same inputs as its real counterpart, will provide (at least) the same outputs and will do the job several times faster than the real hardware. Using this approach, the time saving is limited by the computer hardware in which the simulation is executed. This step is one of the key aspects of the proposed tuning system and constitutes one of the main differentiating points of this work. While subjective tuning with human operators will take several minutes to test the MCA with a certain set of parameters, the simulated motion platform could perform this task in a matter of seconds (as it can perform evaluations faster than the time needed to test the simulator with the actual motion hardware), therefore allowing the examination of hundreds of combinations in the time needed to perform only one on a real motion platform. Moreover, an objective evaluation allows us to achieve repeatability (the same evaluation is provided if the same parameters are tested). Subjective evaluation could provide different results given the same parameters for the MCA, even with the same vehicle operator. The motion platform simulator will require, as input data, the desired angles/lengths for the motion platform actuators, and provide, as outputs, the current angles/lengths of the actuators and the current DoF of the motion platform (this information is not provided by a real motion platform but it needs to be calculated later, so it is useful). Knowing the state of the motion platform, it is possible to calculate the amount of acceleration and angular velocity the operator is experiencing and compare these quantities with the expected ones (the quantities calculated by the vehicle simulator). This is achieved by using the aforementioned objective indicators, which need to be correlated with the operator’s perception. The details of the virtual motion platform used in this work can be found at [48] and [32]. Briefly, the virtual motion platform is a physically-based simulation that takes a CAD model of the motion platform and simulates the way this Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT
ARTICLE IN PRESS S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 9
Fig. 7. Scheme of the MCA-optimization problem.
robotic manipulator moves its actuators when the user changes the input (desired) angles for the actuators. The visualization is performed using the Open Scene Graph library (see Fig. 6), although to achieve greater performance, visualization is disabled unless debugging is required. The physics calculations are performed using a Newtonian–Lagrangian [49] approach implemented with the NVidia PhysX library. The virtual motion platform is detailed at [32] and this paper is the logical continuation of that work. The first paper described the operation and performance of the virtual motion platform, and this paper shows one of the potential uses and benefits of taking this simulated approach.
2.4. Formalization of the problem Before trying to solve the problem, it is important to formalize it in a mathematical way, (although the main focus will be on the practical application of this novel approach). Let us use an instance of a washout algorithm and name it w. This instance belongs to the set of all washout algorithms, called W. Each element w of W has a number of parameters (different in number, shape and meaning for each algorithm), whose values significantly modify the behaviour of w. Let us assume that these parameters are all real numbers (this is the most common situation, and if they are boolean or integer, they can easily be encoded as real numbers) and that the number of modifiable parameters of the w algorithm is n. Each t-uple (of length n) of parameters makes the washout algorithm behave differently. To assess how w works with each t-uple, an evaluation function f is used to provide us with a number representing how the behaviour of w is (using the values of the selected t-uple of parameters). This evaluation or fitness function returns a positive real number, shaped in such a way that the higher the fitness number is, the worst the behaviour of w is (and vice versa). Fig. 7 illustrates the process with a simple example where n = 7. Thus, the goal of the optimization algorithm is to find a t-uple p of parameters that provides the lowest value, o, of the evaluation function for the washout algorithm w. As the values for the parameters have physical meanings, they are restricted to be on a range of values in which they make sense. Therefore, the problem can be limited to a bounded search within the parameter space. The optimization algorithm can be described as follows: Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT 10
ARTICLE IN PRESS
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
Algorithm MCA-Optimization Inputs: w : W // washout algorithm n : N // number of parameters of w that can be modified f : W, Rn ——-> R // evaluation/fitness function li : vector [1..n] of R // lower limits of the parameter ranges ls : vector [1..n] of R // upper limits of the parameter ranges Outputs: p : vector [1..n] of R // optimal t-uple o:R // fitness function value for p Pre-condition: n >0 ∀ i=1..n : li[i] < ls[i] ∀ x ∈ Rn , ∀ w ∈ W : f(w, x) ≥ 1 Post-condition: ∀ x ∈ Rn : f(w, x) > o | w ∈ W, o = f(w, p), li[i] ≤ p[i] ≤ ls[i] with i = 1..n
Obviously, the washout algorithm evaluation depends on several factors: the motion platform, the physics simulation designed to replicate the vehicle dynamics, the fitness function used to assess the algorithm, and, of course, the MCA itself. Thus, the word “optimization” must be understood as a tailored optimization for the evaluation function used, the selected motion platform, the washout algorithm used, and the tested vehicle simulation. As MCA parameters are real numbers, the number of values they can take are infinite. Even constraining them to a few limited ranges, the number of values is still infinite. Therefore, an exhaustive exploration is not feasible and other ways of exploring the Rn space need to be identified. The easiest alternative is to discretize the ranges of the parameter values. In doing so, the problem becomes a combinatorial optimization problem. Nevertheless, the exploration space is still vast. The main problem is that a call to the evaluation function can be very costly, as the common way to test MCA algorithms is to assess them with an execution of the vehicle simulator over several seconds or even minutes. As a motion platform simulator can be used, this process can be accelerated by simulating the motion platform functioning faster than real-time. This is one of the main advantages of the simulation-based solution proposed here. However, it is still necessary to perform thousands, perhaps millions of evaluations, so the problem will be computationally intractable anyway. In fact, if there are n parameters and d different values for each of them, the total number of combinations is dn , the problem is an NP-problem and the original continuous problem is not even being solved, but a simplified discretized one. NP-problems are those that cannot be solved with deterministic polynomial-time complexity. Greedy solutions [50] are not easy to apply either, as the shape of the solution space is unknown and it seems difficult that local decisions based on local criteria will result in global optimal solutions (it makes no sense to evaluate a single parameter without the consideration of the rest). On the contrary, heuristic solutions are of potential benefit here. This kind of approach is useful when the exact solution does not exist or is not feasible, when obtaining the global optimum solution is not really necessary, when the evaluation function is not entirely reliable and when there are strong time limitations. Fortunately, all of these circumstances apply in the washout algorithm tuning problem. Therefore, the use of heuristics is fully justified here as a way to find a satisfactory (quasi-optimal but not optimal) t-uple of parameters for each washout algorithm instance. 3. A simulation-based heuristic solution The easiest method to solve this problem is an exhaustive discretized search. The exhaustive discretized search consists of a brute-force approximation to traverse a discretized version of the entire solution space. The problem is that this approach is not efficient as it needs to test dn possible combinations (t-uples), n being the number of parameters of the MCA, and d the number of subdivisions of each range. As this is not the best strategy to traverse such a potentially large space, another approach is needed. A possible improvement of the aforementioned solution could be the use of a Monte-Carlo algorithm. Monte-Carlo optimization is one of the well-known heuristic schemes used to find a satisfactory solution in an unknown space [51]. As a full sorted traversal of the solution space may be impossible, the main principle is to choose random elements of the solution space and evaluate them. If enough random picks/selections are taken, a large part of the solution space will be covered and a quasi-optimal (or at least satisfactory) solution will be identified. Although the scheme can be customized for each application, and the different implementations may vary, it can be agreed that the more random evaluations are performed, the more reliable the estimation of the optimal solution will be. However, this is precisely the problem. As the solution space is vast, a great deal of random picks will be needed to satisfactorily cover all the search volume. The solution the authors propose to avoid performing an extensive search is to use a genetic algorithm. This scheme has shown to be one of the most successful optimization heuristics in large search spaces. A Darwinian, genetic or evolutive Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT
ARTICLE IN PRESS S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 11
Fig. 8. Reproductive scheme of the MCA-Genetic algorithm.
algorithm, mimics the evolution of natural ecosystems in order to find a quasi-optimal solution for an optimization problem. Although the term “genetic” is the most widely used, the word “Darwinian” is the one that represents best the functioning of the algorithm. The scheme is based on encoding the possible solutions of the problem as a list of chromosomes (a string representing the DNA of the solution) [52]. The algorithm can be adapted to the MCA tuning problem in the following way. Let us consider each t-uple x (of values for the parameters of the washout algorithm w) as an individual of a certain population. Let us encode each individual simply as a sorted sequence of the values of its parameters: x[1] x[2] … x[i] … x[n]. Each scalar value x[i] represents a chromosome, and the entire t-uple represents the DNA of the individual. The evaluation function f(w, x) will be the fitness function. It is important to emphasize that, in this proposed scheme, the evaluation function is not a function itself; it is the result of the objective indicators calculated upon the execution of the motion platform simulator, using the MCA with the parameters stored in the t-uple x. Thus, this is a simulated fitness/evaluation function. Starting with an initial random population P of individuals, the first step is to evaluate them all and sort them by its fitness function in a sorted vector. By Darwin’s natural selection law, only the fittest of them will survive (a fixed percentage). The rest of the population will die. In computational terms, this means that the last part of the P vector will be deleted. Once natural selection has eliminated the less fit (worst simulated solutions), the survivors will reproduce, giving birth to new solutions (Fig. 8). These new individuals must contain DNA information of both parents. As their parents were highly fit individuals, by blending features from two parents, the new child is expected to be a good solution too, and hopefully better. In this step, mutation is introduced, in order for the children to receive the parents’ genetic material with small variations. These mutations can make children fitter or weaker than their parents. The genetic recombination can be sexual (if gender is assigned to each parent) or asexual. Parent “marriages” can be random or guided by the fitness function (the fitter the individual, the more sexually active). The birth of new children can also be accomplished in several ways depending on how their parents’ DNA is combined. For the MCA tuning problem, an asexual random reproduction was selected, in order to foster diversity in the new generations. The recombination of the DNA material is performed by mitosis. This means that for each of the n parameters that form the DNA, the new child will take randomly and equiprobably either the first parent value or the second parent value. In average, new children will have approximately half the material from each of the two parents. Mutations can occur in the recombination process. They consist in randomly changing some of the resulting n values of each individual upon creation. All of the chromosomes of the individual are traversed and some are changed by a random value. The number of mutations is also random and depends of the desired mutation probability. The whole recombination process is best explained in Fig. 8. The reproduction of new individuals is performed as many times as necessary to reinstate the original population size. One interesting modification, which is not present in nature, is the addition of elitism. Elitism is the capability of solutions to survive for more than one generation [53]. The idea behind this concept is not to kill some highly fit individuals, because their children may not always be as fit as their parents. The implementation of this idea for the MCA tuning application will be to retain a very small percentage of the population, by not deleting the first elements in the population Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT 12
ARTICLE IN PRESS
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
vector. Elitism should be kept small, because it can cause population endogamy. Elitism is useful here because each individual requires an entire simulation to be assessed, so it makes sense to retain good solutions that took a while to simulate. The process is repeated until a termination condition is met. This condition can be a number of generations, a number of iterations without improvement, an average population fitness improvement threshold, etc. Nevertheless, the best criterion for this application is a time limit. This is consistent with the idea that an automatic washout algorithm tuning system must conclude in a reasonable amount of time. Otherwise, it would be useless. Moreover, this time limit is necessary if different optimization algorithms (or even different MCA) need to be compared in an unbiased manner. The proposed genetic algorithm is shown in Appendix A. This optimization algorithm implementation depends on four basic parameters that can modify its behaviour. The tuning of these parameters must be performed according to the size of the solution space, the cost of the simulated evaluation function and the available time: (a) Population size: a large size can make the algorithm advance very slowly if the evaluation function is costly, but a very small size would make the algorithm too endogenic. (b) Mortality rate: a high mortality rate can make the algorithm too random. On the contrary, if the value is too low, non-optimal solutions can propagate undesirable chromosomes for several generations and degrade the final solution. (c) Elitism rate: a high elitism rate can make the algorithm too dependent on the initial population. On the contrary, a very small value can make the algorithm overlook some good chromosomes that may not reappear in further generations. (d) Mutation probability: a high mutation probability would ruin the natural selection process, since new generations will be very different from previous ones. On the contrary, a very low mutation rate can make the algorithm too deterministic and dependent on the initial population. Fortunately, these four parameters are easy to tune (much easier than the washout algorithm parameters) and in the several tests performed with this genetic algorithm, it proved to be fairly insensitive to its own parameters, as a satisfactory solution can be obtained for a wide range of values. Besides, it is not necessary to re-tune the genetic algorithm every time the conditions (track, vehicle, task, etc.) of the simulator change. This is not always true for the MCA tuning problem. It is important to emphasize that this proposed optimization method intends to solve the problem of finding the best possible tuning of an MCA, given a specific criterion. It is not the goal of this paper to compare different heuristics to analyse their relative performance and find the best optimization heuristic. The goal is to prove that the tuning of an MCA can be performed using simulation-based techniques instead of the traditional manual approach. 4. Computational experiments and results 4.1. Empirical assessment of the algorithm’s correctness Before analysing the proposed solution against the subjective evaluation, it is first necessary to ensure that the simulation-based solution works properly, and therefore, is able to find the minimum objective value within a certain space of possible solutions. Since the algorithm is heuristic, a formal demonstration to prove that it finds a global minimum makes little sense, as only an exhaustive search can guarantee that. Thus, only an empirical demonstration is performed. The following test uses the classical washout and checks whether the optimization algorithm is able to provide the expected values for the parameters of the MCA. In this test, the exhaustive search and the Monte-Carlo approach are also included, to certify the correctness of the proposed solution, and provide further proof that the virtual motion platform is suitable for this goal. For this experiment, all the classical MCA inputs are set to 0 except for the specific force in the Z axis (set to 9.8 m/s2 ) and the specific force of the Y axis, where a sinusoidal signal, with 10 seconds of duration, a frequency of 1 Hz and an amplitude of 3 m/s2 , is provided. This could represent the typical longitudinal acceleration of a street car. A typical Y-forward, X-right, Z-up axis system is used. Regarding the parameters of the MCA, this test only allows the variation of two of them: the tilt limit of the Y coordination channel (between 0° and 20°), and the break frequency of the Y low-pass filter (between 0 and 10 Hz). The rest of the parameters were set to values considered neutral. Using the NPC indicator, the proposed optimization algorithms should return values near 10 Hz for the break frequency and 20° for the tilt limit. In this case, the NPC indicator was calculated only for the Y translation DoF, to obtain results that are easier to analyze. The comparison used in this test corresponds to option A in Fig. 4. The test was performed with an Intel Core 2 Quad Q8400 at 2.66 GHz, with 4 GB of RAM and Windows 7 operating system. Even with this modest hardware, the simulated motion platform is able to run tests 20 times faster than real-time for the 3-DoF motion platform used in the experiments. The results obtained for each optimization algorithm, with a maximum search time of 500 s (enough time to achieve a satisfactory solution, in this case), can be seen in Table 2. As can be seen in Table 2, all the optimization algorithms return values around 10 Hz for the break frequency and near 20° for the tilt limit value, which are the expected values (both numbers should be close to the top limit of the search intervals), so it can be deduced that the genetic algorithm works properly and the proposed simulation-based technique is correct. The tilt limit does not need to get closer to 20° because the input acceleration is small (0.3 g) and there is no need for further tilt. Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
ARTICLE IN PRESS
JID: SIMPAT
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 13
Table 2 Assessment of the optimization algorithms – 500 s. Algorithm
Exhaustive search Monte-Carlo Genetic
Optimal parameter value Break Freq. (Hz)
Tilt Limit (°)
9.99999 9.98340 9.97510
16.79999 17.71427 16.95966
Best NPC indicator
1.06981 1.06986 1.06980
Table 3 Analysis of the mutation probability. NPC values. Mutation probability
0.0 0.1 0.2 0.3 0.4 0.5 0.75 1.0
Time (s) 100
500
10 0 0
1.29773473 1.29186856 1.28162205 1.28256654 1.28920898 1.28928972 1.29268848 1.31666698
1.28406155 1.26885891 1.26411616 1.26903498 1.26535713 1.28407394 1.28596079 1.29905057
1.27013425 1.26146823 1.25575628 1.26083273 1.26098337 1.28105433 1.28322867 1.28509872
Table 4 Analysis of population size. NPC values. Population size
5 10 20 30 50 100
Time (s) 100
500
10 0 0
1.28885891 1.28162205 1.27167117 1.27251136 1.27345049 1.29309722
1.26954079 1.26411616 1.26251781 1.26961250 1.27823804 1.28097652
1.26089322 1.25575628 1.25220547 1.25338095 1.25580431 1.26064545
4.2. Parameter tuning of the simulation-based solution As the performance of the genetic algorithm depends on the value of its four parameters, it is also important to study the effect of these four parameters (mutation probability, population size, percentage of elitism and percentage of natural selection) over the final minimization process. The first of these series of tests analyses how the probability of mutation affects the algorithm performance. A series of experiments (varying the mutation probability and the execution time) setting the population to 10 individuals, with a 20% of elitism and a survival rate of 40%, were performed. The average NPC obtained for each case (100 executions each) can be seen in Table 3. This time, all the tests were performed with 12 variable parameters of the same classical MCA, the same virtual 3-DoF motion platform and the same computer previously detailed. However, this time the inputs of the MCA are real signals extracted from one lap in the Montmeló Circuit using the rFactor [54] racing simulator with a Formula 3 vehicle. These test signals last around 90 s. Results suggest that the most suitable mutation probability is around 0.2. Higher values make it too random and lower values too inbred. In any case, the algorithm appears to work well for a wide range of values. Table 4 shows the results of a similar experiment varying the size of the population and setting the mutation probability to 0.2, with 20% of elitism and a survival rate of 40%. Results indicate that large population sizes are not always appropriate. As the simulated evaluation function is very costly, populations between 10 and 20 individuals may be enough. Very large populations would take hours of simulation. However, as the available execution time increases, the use of large populations starts to get better. The next experiment studies the most appropriate percentage of elitism, setting the mutation probability to 0.2, with a population size of 20 individuals, and a survival rate of 40%. The indicators obtained are shown in Table 5. Results suggest that a percentage around 20% is ideal for this application. Higher values make the algorithm inbred, and lower values make it too simple because it loses the advantages of remembering good solutions previously identified. In any case, the differences between the various values are not very large, and the algorithm works well even without elitism. Finally, the percentage of natural selection is also analysed, setting the mutation probability to 0.2, with a population size of 20 individuals and 20% of elitism (Table 6). Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
ARTICLE IN PRESS
JID: SIMPAT 14
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18 Table 5 Analysis of the percentage of elitism. NPC values. % of elitism
0 10 20 30 40 50
Time (s) 100
500
10 0 0
1.27230832 1.26859140 1.27167117 1.27258398 1.27633392 1.28290421
1.27153091 1.26408736 1.26251781 1.26417744 1.27080142 1.27348189
1.27033421 1.26537025 1.25220547 1.26790035 1.26890384 1.27128053
Table 6 Analysis of the percentage of natural selection. NPC values. % of natural selection
0 10 20 40 50 75
Time (s) 100
500
10 0 0
1.30067195 1.27094757 1.26837074 1.27167117 1.27829313 1.28706218
1.29120967 1.26326310 1.26558386 1.26251781 1.26970088 1.27038745
1.28862341 1.27653987 1.25673449 1.25220547 1.25937544 1.26987632
Table 7 Genetic algorithm parameters. Parameter
Value
Mutation probability Population size Percentage of elitism Percentage of natural selection
0.2 20 20% 40%
Table 8 Average best NPC for different parameters and times with the genetic algorithm. Time (s)
Number of parameters 2
100 200 300 500 10 0 0 10,0 0 0 Maximum
4
8
1.28283178 1.28691323 1.29114046 1.28192831 1.28347992 1.28836286 1.28012839 1.28038452 1.28152083 1.27906072 1.27898394 1.27349029 1.27639782 1.27472465 1.26992023 1.27556721 1.27012297 1.26502301 standard deviation σ max = 0.005028
18 1.28719175 1.27546432 1.27098391 1.26312893 1.25172981 1.24171954
Results suggest the ideal value is around 40%, although, as in previous cases, the algorithm performs quite well for almost all cases as the obtained NPC values are satisfactory. Having performed this analysis, the parameters of the genetic algorithm can be assigned with some confidence. They are summarized in Table 7. It is also important to emphasize that the algorithm behaves well for a wide range of the values of its parameters, giving only poor results with extreme values. This is important, because the simulation-based solution proposed here tries to offer good values for the parameters of an MCA. As an MCA has up to 30-40 parameters, if the proposed solution also needed a complex parametrisation, the gain would be marginal. Fortunately, the genetic algorithm only needs four parameters and behaves well for most of their ranges. 4.3. Performance tests Next, the results of various performance tests performed with the proposed simulation-based optimization algorithm are detailed. These tests are intended to analyze the behaviour of the proposed simulation-based tuning of MCA. All the following experiments were performed using the same classical MCA, virtual 3-DoF motion platform, input signals and computer previously detailed. The analysis is done using the genetic algorithm with the parameters summarized in Table 7. Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT
ARTICLE IN PRESS S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 15
Table 9 Comparative assessment of the simulation-based tuning method. Preference
Automatic tuning Manual tuning Can’t tell
Time (minutes) 5
20
60
180
18 1 1
11 4 5
7 9 4
2 17 1
The fitness function is, again, the NPC indicator. The values presented (see Table 8) are the average results from a total of 20 trials. This simulation-based genetic approach performs well in multiple of situations, especially when the number of parameters is high. The advantages of the algorithm are more visible when the execution time is high, since the performance improves even in large spaces. Nevertheless, it also performs well with small search times and few parameters. This demonstrates that the solution is suitable, in terms of performance, for all kinds of situations. Moreover, the variability of the algorithm is small, since, although it has a random component, it tends to converge to global optimal solutions. For this reason, the best indicators are clearly obtained with the largest number of variable parameters, and the solution is both scalable and universal.
4.4. Tuning validation To verify if the simulation-based automatic tuning of the MCA parameters is effective and offers advantages over the traditional trial-and-error method, the authors decided to perform a final test, with a 6-DoF motion platform in order to gather the opinions of several vehicle operators (drivers in this case, since a driving simulator was used), comparing the subjective “manual” tuning with automatic tuning. Since the objective indicators have been validated in a previous paper and the correction of the simulation-based optimization algorithm has been proved, at this point the most important thing is to analyze how the time available for performing the tuning process affects the results in both cases, and which of the tuning methods is preferred in each case. Therefore, a comparison between automatic and manual tuning, with different tuning times, is proposed. To do this, a small group of 20 people were invited to test the rFactor racing simulator with a Formula 1 car, on the Montmeló Circuit, using the classical washout algorithm. The subjects were non-professional racing drivers that had been regularly taking part in racing competitions, with experience with open-wheel cars and simulators. The computer used to perform the simulation is an Intel Core i7 - 3700 MHz, with 8 Gb of RAM, an NVidia GTX 690 graphics card, and a Windows 7 operating system. The test was performed in two phases. During the first phase, the drivers and the expert (one driver-expert couple at a time) were prompted to tune the MCA algorithm within a limited amount of time. After this time, the parameters selected for the MCA were stored in an undisclosed location. The time used to achieve that configuration was also recorded. The initial set-up of the MCA was always the same, considered by the expert to be a “neutral” configuration. In a second phase, a week later, the same drivers were invited to test the same simulator in two separate 10-minute runs/trials. One of these two trials was launched with the MCA tuned with their own manual tuning, and the other was launched with the MCA tuned by the automatic objective simulation-based tuning process. The objective tuning was performed by varying 18 parameters of the classical MCA, a combined objective indicator, and with exactly the same time limit that was provided to the human participants. The combined indicator is calculated with an equally-weighted additive combination of the NPC, NAAD and AAS metrics, using option A in Fig. 4. The drivers were not informed of which one of the two trials was the one manually tuned by them, so the test was blind. The question asked to each driver for each couple of trials was: “Which one of the two trials is more convincing in terms of motion cueing?” This was done for four different time limits: 5 min, 20 min, 1 h and 3 h. Results are shown in Table 9, where the number of drivers who (blindly) chose their own manual tuning, the automatic tuning, or they were not able to tell difference, are detailed for each time limit. Results show that manual tuning is preferred when a large amount of time is available. However, automatic tuning is a good tool when a quicker result is necessary. It is important to understand that subjective tuning is “tailored” to the vehicle operator and can be individualized, and thus it is preferable when there is enough time to perform the process, while automatic tuning is a “one size” tuning system. This explains why with only 5 or 20 min, the objective tuning can perform sufficiently well, but with larger time periods available, manual tuning is preferred. It is reasonable to think that results would have probably been different if drivers would have been required to test the MCA tuned by other drivers, instead of testing their own set-ups. In this case, the preference for objective tuning could be, perhaps, slightly higher, although additional experiments should be performed to make such assertion. Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
ARTICLE IN PRESS
JID: SIMPAT 16
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
In this regard, it is important to point out that, if the motion cueing systems need to be used by several pilots/drivers/users, individual subjective tuning may not be the best option, as it is not practical to expend more time tuning the system that using it. In this case, the proposed objective simulation-based tuning can be of great help, as it has the potential to provide an operator-independent solution, or at least a solution suitable for several operators. This is something to keep in mind when dealing with this problem.
5. Conclusions and future work The paper addresses the MCA tuning problem. This is an important problem since the values of their multiple parameters are hard to establish and they dramatically change the behaviour of MCA. This problem is usually addressed in a subjective, manual and time-consuming way. In this paper, a novel – simulation-based, objective and automatic – approach has been proposed and evaluated. The major outcome of this paper is that the proposed simulation-based MCA tuning process works and is an alternative to the existing subjective tuning solution. The only minor drawback is that the optimization algorithm itself has four parameters. As the goal is to find an automatic (and unattended) method to obtain the best values for the parameters of an MCA, it would be preferable that the optimization method would be parameter-less. However, the genetic algorithm works well for a wide range of values of its own parameters, and this issue becomes less important because a fixed set of parameters could be suitable for a wide range of situations, leaving aside the need to find values for these parameters. It is important to emphasize that the genetic algorithm proposed is just one possible solution, and it is not the only contribution of this work. The important result is that it is possible to use simulation techniques to tune motion cueing algorithms, which are, ultimately, used in vehicle simulators. Moreover, the simulation-based tuning is much faster than traditional subjective tuning, as it could provide acceptable results within a much shorter time frame. In this regard, the subjective tuning often takes much longer to achieve, because operators almost always feel tempted to test small variations before giving the final approval to the tuned parameters. Moreover, a simulation-based tuning can be automated, unattended and be applied at any time. On the contrary, subjective tuning requires, at least, an expert and a vehicle operator. Since it may be necessary to re-tune the MCA if the vehicle simulator is changed, if the simulated vehicle is changed, or even if a different track is chosen, the availability of an automated tuning system could provide a decisive advantage. It is important to clarify that this depends on the final application of the simulator. For research, entertainment, and unregulated training applications, the tuning can be automated and the parameters can be changed at any time. For regulated training, the tuned parameters should be locked down, as the simulator needs to be certified by the regulatory authorities. This approach also provides some other advantages, as it is not necessary to operate the real motion platform, saving energy and avoiding an unnecessary wear of the hardware. However, it is important to realize that this approach is presented as an alternative to the subjective tuning, not as a complete substitute, at least in the near future. In fact, this approach shows the potential to provide a general solution, which can be refined with a pilot-in-the-loop approach. A proposed combination could be to use, first, this simulation-based automatic tuning to get a universally acceptable solution, and then use a subjective operator-tailored tuning to fine-tune the obtained solution, if there is enough time to do so, which may or may not be the case. Future work includes looking for different optimization strategies (different than the genetic algorithm) and comparing them with the solution proposed here, looking for the most efficient one (the one that finds the best solution in the shortest time). Other heuristics, such as Simulated Annealing, Black-Box optimization techniques, or different optimization schemes could also be studied. It may also be interesting to compare the performance of this approach for different motion platforms, for different vehicle simulations (different than the driving simulator used in this work), and for different objective metrics. In this last case, it is possible to validate the simulated approach with different metrics and analyze which one of them achieves the best subjective approval. Another interesting suggestion could be to test if the preference for objective and subjective tuning is different when operators try the simulator with MCA tuned by other operators. In addition, it would be very interesting to perform similar experiments with the approach shown in Option B of Fig. 4, using perception models. The problem with perception models is to select a model that represents well this complicated matter, since human perception of motion is still a process partially unknown. Objective indirect evaluation is also a possibility worth studying in the future. Examples of indirect evaluation involve the analysis of how a task assigned to the vehicle operator is performed. This kind of evaluation could be easier to perform, although it has to be proved, for each particular case, that improvements in task performance is caused by improved motion cues.
Appendix A. – Proposed algorithm The optimization (genetic) algorithm, which is designed to perform automatic tuning of MCA, may be implemented as follows: Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
ARTICLE IN PRESS
JID: SIMPAT
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[m3Gsc;July 12, 2016;15:5] 17
Algorithm MCA-Genetic Inputs: w : W // washout algorithm n : N // number of parameters of w that can be modified f : W, Rn ——-> R // evaluation/fitness function li : vector [1..n] of R // lower limits of the parameter ranges ls : vector [1..n] of R // upper limits of the parameter ranges Parameters: populationSize, maxIters : N mortalityRate, elitismRate, mutationProb, maxTime: R Auxiliar: population, parents : list of {params: vector [1..n] of R, eval: R} p1, p2, child : vector [1..n] of R i, j, numDeaths, numLiving: N time, first: R Outputs: p : vector [1..n] of R // optimal t-uple o:R // fitness function value for p Algorithm: i = 0; o = ∞; time = 0.0; startClock(); population = generateRandomPopulation(populationSize, li, ls); do { i = i + 1; // evaluate population for (j=1 to populationSize) do population[j].eval = f(w, population[j].params); sortByEvaluationValue(population); first = population[1].eval; if (first < o) then { o = first; p = population[1].params; } // natural selection numDeaths = populationSize∗ mortalityRate; eliminate(population, numDeaths); parents = population; // elitism numLiving = populationSize∗ elitismRate; numDeaths = size(population) – numLiving; eliminate(population, numDeaths); // reproduction do { p1 = pickRandomParent(parents); p2 = pickRandomParent(parents); child = mitosis(p1, p2); // mutation for (j=1 to n) do generateMutation(child[j], mutationProb); add(population, child); } while (size(population) < populationSize) time = getCurrentClockTime(); } while ((i < maxIters) ^ (time < maxTime))
References [1] [2] [3] [4]
J.B. Sinacori, The Determination of Some Requirements for a Helicopter Flight Research Simulation Facility, (Moffet Field, CA, USA, 1977). L.D. Reid, M.A. Nahon, Response of airline pilots to variations in flight simulator motion algorithms, J. Aircr. 25 (1988) 639–646. J.R. Hall, The Need for Platform Motion in Modern Piloted Flight Training Simulators, Royal Aerospace Establishment, Bedford, United Kingdom, 1989. R.S. Shirley, L.R. Young, Motion cues in man-vehicle control effects of roll-motion cues on human operator’s behavior in compensatory systems with disturbance inputs, IEEE Trans. Man-Mach. Syst. 9 (1968) 121–128.
Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002
JID: SIMPAT 18
ARTICLE IN PRESS
[m3Gsc;July 12, 2016;15:5]
S. Casas et al. / Simulation Modelling Practice and Theory 000 (2016) 1–18
[5] T.W. Showalter, B.L. Parris, The Effects of Motion and g-Seat Cues on Pilot Simulator Performance of Three Piloting Tasks, National Aeronautics and Space Administration, 1980. [6] P.R. Grant, B. Yam, R. Hosman, J.A. Schroeder, Effect of simulator motion on pilot behavior and perception, J. Aircr. 43 (2006) 1914–1924. [7] N.J.I. Garrett, M.C. Best, Driving simulator motion cueing algorithms – a survey of the state of the art, in: Proceedings of the 10th International Symposium on Advanced Vehicle Control (AVEC), Loughborough, UK, 2010, pp. 183–188. [8] L.D. Reid, M.A. Nahon, Flight Simulation Motion-Base Drive Algorithms: Part 1 - Developing and Testing the Equations, UTIAS, University of Toronto, 1985. [9] R.V. Parrish, J.E. Dieudonne, D.J. Martin Jr, Coordinated adaptive washout for motion simulators, J. Aircr. 12 (1975) 44–50. [10] M.A. Nahon, L.D. Reid, J. Kirdeikis, Adaptive simulator motion software with supervisory control, J. Guid. Control Dyn. 15 (1992) 376–383. [11] M. Kurosaki, Optimal washout for control of a moving base simulator, in: Proceedings of the Seventh Triennial World Congress of IFAC (International Federation of Automatic Control), Helsinki, Finland, 1978, pp. 1311–1318. [12] R. Sivan, J. Ish-Shalom, J.K. Huang, An optimal control approach to the design of moving flight simulators, IEEE Trans. Syst. Man Cybern. 12 (1982) 818–827. [13] B. Friedland, C.K. King, M.F. Hutton, Quasi-Optimum Design of a Six Degree of Freedom Moving Base Simulator Control System, NASA Contractor Report, 1973. [14] B. Friedland, Revisiting quasi-optimum control, in: Proceedings of the 2006 American Control Conference, Minneapolis, MN, USA, 2006, pp. 292–297. [15] M. Wentink, W. Bles, R. Hosman, M. Mayrhofer, Design & evaluation of spherical washout algorithm for Desdemona simulator, in: Proceedings of AIAA Modeling and Simulation Technologies Conference and Exhibit, San Francisco, CA, USA, 2005. [16] L. Nehaoua, H. Arioui, S. Espie, H. Mohellebi, Motion cueing algorithms for small driving simulator, in: Proceedings of 2006 IEEE International Conference on Robotics and Automation (ICRA), Orlando, FL, USA, 2006, pp. 3189–3194. [17] L.D. Reid, M.A. Nahon, Flight Simulation Motion-Base Drive Algorithms: Part 2- Selecting the System Parameters, UTIAS, University of Toronto, 1986. [18] M.A. Nahon, L.D. Reid, Simulator motion-drive algorithms - A designer’s perspective, J. Guid. Control Dyn. 13 (1990) 356–362. [19] J.A. Schroeder, Helicopter Flight Simulation Motion Platform Requirements, National Aeronautics and Space Administration, Moffett Field, California, 1999. [20] E.M. Kolasinski, Simulator Sickness in Virtual Environments, Technical Report 1027. U.S. Army Research Institute for Behavioral and Social Sciences, 1995. [21] S.J. Hodge, P. Perfect, G.D. Padfield, M.D. White, Optimising the yaw motion cues available from a short stroke hexapod motion platform, Aeronaut. J. 119 (2015) 1–21. [22] M. Bruenger-Koch, Motion parameter tuning and evaluation for the DLR automotive simulator, in: Proceedings of Driving Simulation Conference North America (DSC-NA), Orlando, FL, USA, 2005, pp. 262–270. [23] M. Fischer, H. Sehammer, G. Palmkvist, Motion cueing for 3-, 6- and 8-degrees-of-freedom motion systems, in: Proceedings of Driving Simulation Conference Europe, Paris, France, 2010, pp. 121–134. [24] P.R. Grant, L.D. Reid, PROTEST: an expert system for tuning simulator washout filters, J. Aircr. 34 (1997) 152–159. [25] P.R. Grant, L.D. Reid, Motion washout filter tuning: rules and requirements, J. Aircr. 34 (1997) 145–151. [26] T. Chapron, J.P. Colinot, The new PSA Peugeot-Citroën advanced driving simulator overall design and motion cue algorithm, in: Proceedings of Driving Simulation Conference, Iowa City, IA, USA, 2007. [27] F. Colombet, M. Dagdelen, G. Reymond, C. Pere, F. Merienne, A. Kemeny, Motion Cueing: what’s the impact on the driver’s behaviour? in: Proceedings of Driving Simulator Conference, Monte-Carlo, Monaco, 2008, pp. 171–182. [28] ICAO, Manual of Criteria for the Qualification of Flight Simulation Training Devices. Volume I - Aeroplanes, International Civil Aviation Organization, 2009). [29] S. Advani, R. Hosman, M. Potter, Objective motion fidelity qualification in flight training simulators, in: Proceedings of AIAA Modeling and Simulation Technologies Conference and Exhibit, Hilton Head, SC, USA, 2007. [30] O. Stroosma, M.M. Van Paassen, M. Mulder, R. Hosman, S. Advani, Applying the objective motion cueing test to a classical washout algorithm, in: Proceedings of AIAA Modeling and Simulation Technologies (MST) Conference, Boston, MA, USA, 2013. [31] R. Hosman, S. Advani, Are Criteria for Motion Cueing and Time Delays Possible? Part 2, in: Proceedings of AIAA Modeling and Simulation Technologies Conference, Boston, MA, USA, 2013. [32] S. Casas, J.M. Alcaraz, R. Olanda, I. Coma, M. Fernández, Towards an extensible simulator of real motion platforms, Simul. Model. Pract. Theory 45 (2014) 50–61. [33] S. Casas, I. Coma, J.V. Riera, M. Fernández, Motion-cuing algorithms: Characterization of users’ perception, Hum. Factors: J. Human Factors Ergon. Soc. 57 (2015) 144–162. [34] J.H. Holland, Genetic algorithms, Sci. Am. 267 (1992) 66–72. [35] E. Thöndel, Design and optimisation of a motion cueing algorithm for a truck simulator, in: Proceedings of European Simulation and Modelling Conference - ESM, Essen, Germany, 2012. [36] L.Y. Kuo, J.Y. Yen, A genetic algorithm-based parameter-tuning algorithm for multi-dimensional motion control of a computer numerical control machine tool, in: Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 216, 2002, pp. 429–438. [37] H.H. Hoos, Automated algorithm configuration and parameter tuning, in: Y. Hamadi, E. Monfroy, F.D.R. Saubion (Eds.). (Springer Berlin Heidelberg, 2012), 37–71. [38] J.M. Van Verth, L.M. Bishop, Essential Mathematics for Games and Interactive Applications: A Programmer’s Guide, Morgan Kaufmann, Burlington, MA, USA, 2004. [39] E.L. Groen, W. Bles, How to use body tilt for the simulation of linear self motion, J. Vestib. Res. 14 (2004) 375–385. [40] G. Reymond, A. Kemeny, Motion cueing in the renault driving simulator, Veh. Syst. Dyn.: Int. J. Veh. Mech. Mobil. 34 (20 0 0) 249–259. [41] P.R. MacNeilage, M.S. Banks, D.R. Berger, H.H. Bulthoff, A Bayesian model of the disambiguation of gravitoinertial force by visual cues, Exper. Brain Res. 179 (2007) 263–290. [42] J.E. Holly, G. McCollum, The shape of self-motion perception–I. Equivalence classification for sustained motions, Neuroscience 70 (1996) 461–486. [43] C.B. Rorabaugh, Digital Filter Designer’s Handbook - Featuring C Routines, TAB Books - McGraw Hill, Blue Ridge (PA), 1993. [44] D.R. Berger, J. Schulte-Pelkum, H.H. Bülthoff, Simulating believable forward accelerations on a Stewart motion platform, ACM Trans. Appl. Percept. 7 (2010) 5:1–5:27. [45] L.D. Reid, M.A. Nahon, Flight Simulation Motion-Base Drive Algorithms:Part 2 - Selecting the System Parameters, UTIAS, University of Toronto, 1985. [46] J. Laurens, J. Droulez, Bayesian processing of vestibular information, Biol. Cybern. 96 (2007) 389–404. [47] M. Wentink, J. Bos, E. Groen, R. Hosman, Develpment of the Motion perception Toolbox, 2006. [48] S. Casas, R. Olanda, M. Fernandez, J.V. Riera, A Faster Than Real-Time Simulator of Motion Platforms, CMMSE, 2012. [49] A. Witkin, D. Baraff, in: SIGGRAPH ’97 Course notes, Carnegie Mellon University - School of Computer Science, 1997. [50] G.T. Heineman, G. Pollice, S. Selkow, Algorithms in a Nutshell, 2nd Edition, O’Reilly Media, Inc., 2016. [51] J.M. Hammersley, D.C. Handscomb, Monte Carlo Methods, London, UK & New York, NY, USA, 1964). [52] E. Alba, et al., Optimization Techniques for Solving Complex Problems, Wiley, New Jersey, NJ, USA, 2009. [53] S. Baluja, R. Caruana, Removing the genetics from the standard genetic algorithm, in: Proceedings of the 12th International Conference on Machine Learning, Lake Tahoe, CA, USA, 1995, pp. 38–46. [54] I. Space, rFactor Racing Simulation, http://www.rfactor.net/ (accessed 03.06.16).
Please cite this article as: S. Casas et al., Towards a simulation-based tuning of motion cueing algorithms, Simulation Modelling Practice and Theory (2016), http://dx.doi.org/10.1016/j.simpat.2016.06.002