Applied Ergonomics 67 (2018) 83e90
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Accident ahead? Difficulties of drivers with and without reading impairment recognising words and pictograms in variable message signs J. Roca a, *, P. Tejero b, B. Insa a a b
, Universitat de Val ERI-Lectura / Dept. Psicologia Evolutiva i Educacio encia, Spain sica, Universitat de Val ERI-Lectura / Dept. Psicologia Ba encia, Spain
a r t i c l e i n f o
a b s t r a c t
Article history: Received 24 February 2017 Received in revised form 5 June 2017 Accepted 20 September 2017
A timely and accurate acquisition of the information provided by variable message signs (VMS) can be crucial while driving. In the current study, we assess the difficulties of adults with dyslexia acquiring the information shown in VMS and provide evidence to discuss the controversial use of pictograms as potential countermeasures. Twenty-two adults with dyslexia and 22 matched controls completed a simulated driving session. The legibility of 12 VMS was assessed, including six text messages (e.g. “ACCIDENT”) and six single pictograms (e.g. the icon for “accident ahead”). On average, participants with dyslexia started reading text messages when they were closer to the VMS. In addition, while approaching text VMS, they dedicated more gazes and manifested worse control of speed. Regarding pictogram VMS, we observed no differences in response distance, accuracy, response duration, or number of gazes. To sum up, the evidence provided reveals that adults with dyslexia, despite potential compensation effects, may still find difficulties reading text messages in VMS (shorter legibility distances, longer reading times, and increased cognitive effort), whereas we found no such differences in the recognition of pictograms (only some difficulties keeping a steady speed). Research on inclusive measures to improve reading in low-skilled or dyslexic drivers must be encouraged. © 2017 Elsevier Ltd. All rights reserved.
Keywords: Traffic sign legibility Dyslexia Cognitive demands Variable message signs Pictograms Driver simulator
1. Introduction
1.1. VMS legibility
Variable message signs (VMS) are a common infrastructure to inform drivers about dynamic regulations and special circumstances affecting traffic, such as variable speed limits, accidents ahead, roadwork sections, queues, or adverse climate conditions. A timely and accurate acquisition of the messages in VMS may be crucial to avoid unnecessary risks and maintain a smooth traffic flow. The current study will analyse the difficulties of individuals with and without reading impairment in accessing the text content of VMS, and will provide new evidence to discuss the use of pictograms as a potential countermeasure.
Although a variety of models are currently found along the roads worldwide, a typical VMS layout may include areas to show text messages and/or pictograms (for example, three lines of 12e18 characters each, and one or two pictograms areas at the horizontal extremes). Usually text characters and pictograms are represented using LED dot matrices, such as a 7x5 matrix for each character and a 32x32 matrix for pictograms, although higher resolutions and more flexible layouts (e.g. full matrix VMS) are also common. These resolutions are usually appropriate to show text messages and pictograms that could be correctly identified from a long distance, depending on factors such as character height, lighting viewing conditions (e.g., day or night), font case, and many others (for reviews see, for example, Garvey, 2002; Nygardhs, 2011; Nygardhs and Helmers, 2007; Ullman et al., 2005). Different factors associated with driver characteristics have also been studied, such as drivers with low-vision or older drivers (Garvey, 2002; Garvey and Mace, 1996). However, despite the abundant previous literature that has analysed the legibility of VMS messages as a function of
, Facultat de Psico* Corresponding author. Dept. Psicologia Evolutiva i Educacio ncia, Avenida Blasco Iba n ~ ez, 21, 46010 Valencia, Spain. logia, Universitat de Vale E-mail address:
[email protected] (J. Roca). https://doi.org/10.1016/j.apergo.2017.09.013 0003-6870/© 2017 Elsevier Ltd. All rights reserved.
84
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90
different perceptual factors, to our knowledge no evidence has been provided so far on the specific difficulties of people with reading impairment. 1.2. Drivers with reading impairment Should VMS designers and operators be concerned about drivers with reading impairment? Several reasons would argue in favour of a positive answer. First, it has been estimated that 7% of the population could be considered as dyslexic, a specific learning disorder impairing accurate or fluent word recognition despite adequate instruction and intelligence and intact sensory abilities (Peterson and Pennington, 2012). If we extrapolated this percentn General de Tra fico, age to the driving population in Spain (Direccio 2016) and in the USA (U.S. Department of Transportation, 2016), they would potentially represent over 1.8 million and over 15 million of drivers, respectively. In addition, many non-dyslexic individuals are actually poor readers, which are performing in the low range along the reading skill continuum and, therefore, they might also encounter difficulties when reading words in VMS. In fact, most of the measures aimed at facilitating text message reading would potentially benefit not only drivers with dyslexia and low-performance readers, but also virtually any driver trying to read the VMS in cognitively demanding conditions (e.g., dense traffic flow, visual clutter, night driving, reduced visibility, among many others). Moreover, are adults with dyslexia still struggling to read or did they overcome their difficulties? Previous research has shown that some of their difficulties may be compensated to a certain extent, and there is evidence of neural adaptation and the deployment of compensatory cognitive strategies after behavioral remediation (Shaywitz et al., 2003; Temple et al., 2003). Nevertheless, it is generally agreed that dyslexia persists into adulthood, even in adults with a higher education level and considerable reading exposure (Afonso et al., 2015; Su arez-Coalla and Cuetos, 2015). The difficulties of adults with dyslexia have been confirmed in different languages (see, for example, Afonso et al., 2015; Bruck, 1990; Callens et al., 2012; Nilssen-Nergård and Hulme, 2014; Parrila rez-Coalla and et al., 2007; Re et al., 2011; Reid et al., 2007; Sua Cuetos, 2015; Tops et al., 2012; Wolff, 2009), and generally manifest as reading errors and, particularly, low speed when reading words (especially with low frequency or long words), pseudowords (i.e., non-existent words aimed to analyse grapheme-phoneme conversion), or full texts, despite the potential help of the meaning context. It should be noted that slow reading might be of particular relevance in the traffic domain, since it may significantly decrease the legibility distance of text messages in VMS or other traffic signs. However, to what extent dyslexia is affecting the legibility of text messages in variable message signs, despite potential compensation effects, has not been specifically addressed in previous research. Regarding pictogram-based messages, the use of pictorial information to complement or substitute text is a frequent practice when designing materials for individuals with low reading ability (Houts et al., 2006). Indeed, dyslexia is traditionally considered as a language-based disorder (Peterson and Pennington, 2012) and, consequently, the processing of pictorial information would be theoretically preserved. However, the evidence regarding the benefit of pictorial information on dyslexia or low-skilled readers has provided conflicted results (see, for example, Holmqvist Olander et al., 2016). In particular, recent evidence in the traffic domain has suggested that individuals with dyslexia might also experience difficulties in responding to or understanding pictorial information in traffic signs (Brachacki et al., 1995; Fisher et al., 2015; Sigmundsson, 2005; Taylor et al., 2016). First, Brachacki
et al. (1995) presented real or false traffic signs to a group of adults with and without dyslexia. Their results revealed worse discrimination performance amongst the participants with dyslexia, and, in contrast to controls, their performance did not seem to be positively associated with driving experience. Sigmundsson (2005) presented a series of traffic signs to a group of young adults with and without dyslexia using a driving simulator. The participants had to negotiate with real-life traffic situations while keeping the track of a computer cursor marking the position of the simulated car relative to the traffic ahead. The traffic signs appeared suddenly in different locations over the traffic scene, and participants had to detect them. Results in this reaction time task showed that dyslexic individuals had significantly slower responses than controls. This was interpreted, in the terms of the magnocellular theory of dyslexia (Stein and Walsh, 1997), as a worse ability to perceive rapid changes in the environment. Fisher et al. (2015) recruited a sample of non-clinical university students who completed a screening test to assess dyslexia symptoms and a selfreport measure of visual stress. They also completed a short simulated driving task that required the appraisal of warning traffic signs to avoid collisions or offence tickets. Regression analysis suggested an association between dyslexia symptoms and speeding in the simulator, although this difference was mainly observed in the dyslexic-prone participants who also reported high visual stress (for further information on visual stress and reading difficulties see, for example, Wilkins and Evans, 2010). Finally, Taylor et al. (2016) also studied non-clinical drivers varying in their reading skill. Their results suggested a negative association between dyslexia symptoms and the scores in a road sign comprehension test. 1.3. Objectives The main objective of the current study is to assess the difficulties of adult drivers with dyslexia acquiring the information shown on VMS. Despite the potential compensation effects, their difficulties may manifest as reduced reading distances or less accurate responses when reading text messages. Besides, the difficulties may also emerge as a worse control of vehicle speed while trying to read the VMS content, or higher visual attentional resources dedicated to the VMS. In addition, a second objective of the current study is to evaluate a potential countermeasure aimed at reducing the expected reading difficulties, in particular, using pictograms as an alternative to text messages. Different outcomes would be predicted depending on the previous literature. According to the general view on dyslexia, non-significant differences in the recognition of pictogram-based VMS would be found between dyslexic and control drivers. In contrast, considering the traffic studies that report difficulties of low-skilled readers with pictorial information, the participants with dyslexia would also perform poorly with the pictogram-based VMS as compared to the control group. 2. Material and methods 2.1. Participants A sample of 44 participants was recruited for the current study. Thirty-two were women. The average age was 28.93 (standard deviation, SD ¼ 10.74; minimum, MIN ¼ 18; maximum, MAX ¼ 49). The average years of education was 14.50 (SD ¼ 1.94; MIN ¼ 10; MAX ¼ 18). Thirty-three hold a B category driving licence, which allows driving motor vehicles with a maximum authorised mass not exceeding 3500 kg and constructed for carrying less than eight passengers plus the driver. They had an average experience of 12.43
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90
years (SD ¼ 10.18; MIN ¼ 1.23; MAX ¼ 31.51). Half of the participants were assigned to a group with dyslexia (DX) and the other half to a control group (CT). All participants were recruited among the respondents of a 10min online screening questionnaire on reading habits and reading difficulties. To recruit potential participants, information about the study and the link to the screening questionnaire was sent to students and staff at the Universitat de Val encia, users of the university disability services, and members of a dyslexia association in the area (AVADIS). In addition, street posters were placed in different parts of the city. Respondents informing a potential prior diagnosis of dyslexia were invited to participate in the study and were submitted to individualized clinical assessment. The evaluation session included the assessment of the current reading performance nchez, 2012), intellectual (PROLEC-SE; Cuetos Vega and Ramos Sa ability (WAIS-IV; Amador, 2013), phonological awareness (Serrano et al., 2003), and a clinical interview. Confirmed participants in the DX group were identified considering the DSM-5 criteria (American Psychiatric Association, 2013). Their current reading performance had to be at least 1.5 standard deviations below the norm group, considering either speed of accuracy in the word reading or pseudoword reading subtests of the PROLEC-SE. In addition, they had to be 18 years old or older and obtain a total IQ of 80 or higher. Potential participants for the CT group reported no previous difficulties in learning to read and matched a DX participant in sex and age (±3 years, with a minimum age of 18 years). Then, they completed a similar clinical assessment session, in which they had to show an appropriate reading performance (1.5 standard deviations below a reference norm group or higher) and obtain a total IQ of 80 or higher. We verified that both groups significantly differed in the reading performance indicators: word reading accuracy (F(1,43) ¼ 4.69, p ¼ 0.036), word reading speed (F(1,43) ¼ 14.37, p < 0.001), pseudoword reading accuracy (F(1,43) ¼ 17.11, p < 0.001), and pseudoword reading speed (F(1,43) ¼ 15.81, p < 0.001). We also checked for differences in phonological awareness, which were not observed considering accuracy (F(1,43) ¼ 2.14, p ¼ 0.150) but were statistically significant in speed (F(1,43) ¼ 11.86, p ¼ 0.001). Finally, we verified that the two groups did not significantly differ in age (F(1,43) ¼ 0.43,
85
p ¼ 0.84), years of education (F(1,43) ¼ 2.01, p ¼ 0.16), total IQ (F(1,43) ¼ 2.31, p ¼ 0.14), driving experience (F(1,43) ¼ 0.11, p ¼ 0.74), or visual acuity (logMAR score, F(1,43) ¼ 0.30, p ¼ 0.59). See Table 1 for descriptive values.
2.2. Stimuli and apparatus We used a research version of the Carnetsoft driving simulator (https://www.rijschool-simulator.nl/), located in a dim light and sound-attenuated room. It consists of three 2400 -inch monitors covering a total angle of 210 of the participant's visual field of view, a control monitor, a steering wheel, three pedals, a gear lever, a comfortable seat, and high-performance computer (Intel Core i73770 3,4 GHz with a Nvidia GTX770 graphic card). The simulator controlled the presentation of the driving task and registered the participant's driving behaviour. A 3D model of a VMS typically found on the Spanish highways was designed for the current study. The VMS consisted of three lines of text with 12 characters each and two areas for pictograms in the extremes. Each text character could be represented in a 7x5 dot matrix and each pictogram in a 32x32 dot matrix. Dimensions, spacing, colours, and other design features followed the specifica n Espan ~ ola de tions of the standard norm UNE-EN 12966 (Asociacio n y Certificacio n, 2015). In addition, to represent the Normalizacio characters of text messages and pictograms, we used the patterns available in CarDim (I.P.S. Vial, S.L.), a computer application aimed at creating sign models for traffic engineering projects. Regarding the content of the VMS, 12 different messages were modelled for the current study (Table 2). Half of the messages were text-type, showing a single word in capital letters, located in the middle line of the VMS, and aligned to the left. The other half were pictogram-type messages, showing a single pictogram with a triangular red border (danger sign) on a black background, placed at the left extreme of the VMS. The pictograms used in the experiment are relatively frequent on Spanish roads, and they are not easily discriminable from each other at a distance. As for the words, each of them matched one of these pictograms. The previous VMS were presented along a 14.40-km circular route in the driving simulator. The route consisted of different
Table 1 Descriptive statistics of sociodemographic data and assessment variables.
Age Years of education Driving experience (years with B license) Visual acuity (LogMAR) PROLEC-SE: word accuracy PROLEC-SE: word speed (s) PROLE-SE: pseudoword accuracy PROLE-SE: pseudoword speed (s) Phonological awareness: Total accuracy Phonological awareness: Total speed (s) WAIS-IV: Full Scale IQ
Group
Mean
Standard error
95% Confidence Interval
Minimum
Maximum
CT DX CT DX CT DX CT DX CT DX CT DX CT DX CT DX CT DX CT DX CT DX
29.27 28.59 14.91 14.09 9.85 8.79 -0.04 -0.05 39.82 39.36 26.59 45.41 38.82 35.09 39.68 81.77 17.09 14.95 268.23 379.05 104.95 100.86
2.35 2.28 0.44 0.37 2.42 2.02 0.01 0.01 0.11 0.18 1.15 4.83 0.30 0.85 1.43 10.49 0.88 1.17 25.04 20.21 1.91 1.90
24.38 23.85 13.99 13.32 4.82 4.59 -0.06 -0.08 39.60 38.99 24.21 35.36 38.20 33.32 36.71 59.96 15.26 12.53 216.16 337.02 100.99 96.91
18 18 12 10 0 0 -0.10 -0.18 38 37 17 25 36 26 28 38 8 4 133 198 87 81
49 49 18 18 32 29 0.08 0.10 40 40 37 139 40 40 61 257 24 23 666 554 117 118
*CT ¼ control group, DX ¼ dyslexia group.
34.17 33.33 15.83 14.86 14.88 12.98 -0.02 -0.02 40.04 39.74 28.98 55.45 39.44 36.86 42.65 103.59 18.92 17.38 320.29 421.08 108.92 104.82
86
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90 Table 2 The 12 VMS shown to the participants during the driving simulation. Each text-type VMS matched a pictogram-type VMS in terms of the message conveyed. Message
Text-type VMS
Pictogram-type VMS
Accident
Soft shoulder
Loose stones
Fog area
Road works
Queues
sections: two parallel sections, each including six straight segments connected with smooth curves, and two U-turn curve sections in the extremes connecting the parallel sections. The parallel sections represented a dual carriageway with two lanes in each direction and some exits (which were not used in the current study). The Uturn curve sections in the extremes were single carriageways. The different VMS were presented at the end of each of the 12 straight segments in the parallel sections. Only two other cars were driving the same route: one positioned 150 m in front of the participant and the other 150 m on the behind. These cars kept constant the distance and thus they did not interact in any way with the participant. During the simulated driving route, we used an eye-tracking device (SMI Eye-Tracking Glasses v1.1) to analyse the gaze of the participant. The eye-tracker recorded gaze data at a sampling rate of 30 Hz and it incorporated a microphone, which was used to record the participant's oral responses while approaching the VMS. 2.3. Procedure The study was conducted according to the international ethical standards and approved by the ethical committee in the University of Valencia. Participants entered the simulation room, read the informed consent form, and filled in a questionnaire to gather sociodemographic data and information on driving habits. Then, they sat in the driving simulator chair, put the eye-tracking glasses on (if necessary, over their own optical correction), and completed a visual acuity test (E chart) to check that they obtained a maximum LogMAR value of 0.1. Next, they were accommodated in the simulator so that their eyes remained at a distance of 38 cm from the central monitor and their gaze fell approximately upon the horizon when looking frontally. At this time, instructions were given and participants were allowed to familiarise themselves with the stimuli. With this aim, we showed the pictogram-type VMS repeatedly until the participants successfully gave all the expected responses. Before starting to drive, the eye-tracking device was
calibrated using a 3-dot pattern and, after calibration, the participants drove some minutes to get familiar with the simulator. Then, they completed three full laps in the experimental route until they found 36 VMS, i.e., the 12 different messages, 6 text-type and 6 pictogram-type, repeated 3 times each and presented in a different random order for each participant. The driving task typically lasted about 30 min. Once started, the simulation did not stop unless the participant was not in a good condition to continue. Three participants experienced simulator sickness and had to abandon the experiment. In addition, some participants were asked to stop shortly at a U-turn curved section, where no VMS was located, to recalibrate the eye-tracker. According to the instructions given, participants had to drive the route obeying the traffic rules and regulations. In particular, they were asked to keep always in the right lane and drive steadily at posted speed limits (120 km/h in the parallel sections where VMS were located and 80 km/h in the U-turn sections at the extremes). In addition, they had to say aloud the content of the different VMS encountered during the route, by reading the text messages and identifying the pictogram messages as soon as they could correctly do it. As previously described, participants completed an initial training task to get familiar with the text messages shown in the VMS and learn the expected response for the pictogram messages (for example, they had to say “accident” when they identified the corresponding pictogram). We did not use the official names of the traffic signs since they are longer than a single word and because it was necessary to standardise the participants’ responses. Once the driving task finished, participants received further information about the study objectives and expected results and were offered a soft drink and a snack. No economical compensation was given for the participation. 2.4. Design and analysis Data from the driving simulator were processed to identify the 36 trials for each participant, i.e. the straight road sections starting
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90
350 m before each VMS and finishing when the participant gave a response or when s/he reached the VMS. Then, the following measures were obtained: (a) Response distance (distance, in meters, separating each VMS and the participant's car when s/he started responding correctly, averaged per participant and condition), (b) Response accuracy (percentage of correct responses per participant and condition, considering as errors the wrong identifications, hesitations, repetitions, rectifications, and omissions), (c) Response duration (time, in milliseconds, of the participants' utterances in correctly responded trials), (d) trial speed variability (standard deviation of the speed registered in each correctly responded trial, averaged per participant and condition), (e) nontrial speed variability (standard deviation of the speed registered outside trial sections, excluding the initial training period and Uturn sections, averaged per participant), (f) gaze number (the number of gazes at each VMS in a correctly responded trial, averaged per participant and condition), and (g) gaze time percentage (percentage of time gazing at each VMS out of the total gazing time registered in the correctly responded trial, averaged per participant and condition). Data analysis was performed using IBM SPSS Statistics v22. Separate one-way between-group ANOVAs were performed to compare the two groups of participants (CT vs. DX) on the sociodemographic and assessment variables. Regarding the measures obtained from the simulator and the eye-tracker, separate two-way mixed ANOVAs were computed with group (CT vs. DX) as a between-group factor and VMS type (text vs. pictogram) as a within-participant factor. The signification level was set at 0.05. 3. Results 3.1. Response to VMS Table 3 shows average results across trials by group and variable message sign type. The analyses performed confirmed the existence of statistically significantly differences between the groups regarding the responses to the VMS, in particular on the response distance and the response duration. First, the analysis of the response distances revealed an interaction effect between group and VMS type (F(1,42) ¼ 27.31, p < 0.001, h2 ¼ 0.40). The analysis of simple effects showed that the response distance was longer in the CT group (mean, M ¼ 130.28; 95% CI [120.06, 140.50]) than in the DX group (M ¼ 109.01; 95% CI [98.79, 119.24]), but only considering the text-type VMS (p ¼ 0.005); whereas differences were not
87
significant within the pictogram-type VMS (CT group: M ¼ 78.60, 95% CI [67.42, 89.78]; DX group: M ¼ 81.95; 95% CI [70.77, 93.13]; p ¼ 0.671). In addition, a main effect of VMS type was observed (F(1,42) ¼ 279.53, p < 0.001, h2 ¼ 0.87), showing that text-type VMS could be read at a longer distance (M ¼ 119.65; 95% CI [112.42, 126.88]) than pictogram-type VMS (M ¼ 80.28; 95% CI [72.37, 88.18]). On the contrary, the main effect of group was found nonsignificant (F(1,42) ¼ 1.58, p ¼ 0.216, h2 ¼ 0.04). Second, regarding response accuracy, both groups of participants obtained high percentages of correct responses (over 90%) and no statistically significant differences were found between them, neither considering the main effect of group (F(1,42) ¼ 1.82, p ¼ 0.184, h2 ¼ 0.04) nor the interaction between group and VMS type (F(1,42) ¼ 1.06, p ¼ 0.309, h2 ¼ 0.03). We only observed a main effect of VMS type (F(1,42) ¼ 20.83, p < 0.001, h2 ¼ 0.33), showing that text-type VMS were associated with higher accuracy (M ¼ 99.74, 95% CI [99.40, 100.10]) than pictogram-type accuracy (M ¼ 91.92; 95% CI [88.51, 95.33]). Third, the analysis of the response durations showed an interaction effect between group and VMS type (F(1,42) ¼ 4.20, p ¼ 0.047, h2 ¼ 0.09). Simple effects revealed that participants’ responses were slower with text-type VMS (M ¼ 555.87; 95% CI [522.24, 589.49]) than with pictogram-type VMS (M ¼ 533.15; 95% CI [499.76, 566.54]), but only in the DX group (p < 0.001); whereas no statistically significant differences were found in the CT group (text-type VMS: M ¼ 550.96; 95% CI [517.57, 584.35]; pictogramtype VMS: M ¼ 558.04; 95% CI [524.42, 591.66]; p ¼ 0.196). Besides, the main effect of VMS type was also statistically significant (F(1,42) ¼ 15.27, p < 0.001, h2 ¼ 0.27), suggesting that overall the text-type VMS (M ¼ 542.05; 95% CI [518.44, 565.66]) were read more slowly than the pictogram-type VMS were responded (M ¼ 556.95; 95% CI [533.18, 580.73]). 3.2. Speed variability The analysis of speed variability while driving through the trial sections in the simulator revealed some relevant results. The main effect of the group variable was statistically significant (F(1,42) ¼ 9.00, p ¼ 0.005, h2 ¼ 0.18), showing higher speed variability in the DX group (M ¼ 0.82; 95% CI [0.67, 0.97]) as compared with the CT group (M ¼ 0.51; 95% CI [0.36, 0.66]), while the participants were approaching the VMS. No interaction between group and VMS type was observed (F(1,42) ¼ 0.04, p ¼ 0.847, h2 ¼ 0.001), which suggests that the reported group effect is similar with either
Table 3 Summary of results in the measures obtained across trials by group and variable message sign type. Group
Response to VMS Response distance (m) Response accuracy (%) Response duration (ms) Speed variability Trial speed variability Gaze indicators Gaze number Gaze time percentage
Text VMS
Pictogram VMS
Mean
St. Error
95% Confidence Interval
Mean
St. Error
95% Confidence Interval
CT DX CT DX CT DX
130.28 109.01 100.00 99.50 558.04 555.87
5.07 5.07 0.25 0.25 16.67 16.67
120.06 98.79 99.50 99.00 524.42 522.24
140.50 119.24 100.50 99.99 591.66 589.49
78.60 81.95 93.94 89.90 550.96 533.15
5.54 5.54 2.40 2.40 16.55 16.55
67.42 70.77 89.12 85.08 517.57 499.76
89.78 93.13 98.76 94.72 584.35 566.54
CT DX
0.46 0.78
0.07 0.07
0.31 0.63
0.61 0.93
0.55 0.87
0.08 0.08
0.40 0.71
0.71 1.03
CT DX CT DX
5.55 7.25 61.84 67.85
0.34 0.35 2.38 2.44
4.85 6.54 57.03 62.92
6.25 7.96 66.65 72.78
7.45 8.29 64.46 70.25
0.52 0.53 2.67 2.73
6.39 7.21 59.07 64.72
8.50 9.37 69.86 75.77
*CT ¼ control group, DX ¼ dyslexia group, St. Error ¼ standard error, VMS ¼ variable message sign.
88
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90
the text-type or the pictogram-type VMS. The main effect of VMS type was also statistically significant (F(1,42) ¼ 15.74, p < 0.001, h2 ¼ 0.27), showing that speed variability is higher with the pictogram-type (M ¼ 0.71; 95% CI [0.60, 0.82]) than the text-type VMS (M ¼ 0.62; 95% CI [0.51, 0.72]). Interestingly, the differences between the groups reported in speed variability were only observed inside the trial sections (i.e., while participants approached the VMS). On the contrary, no statistically significant differences were found in speed variability considering the 120 km/h non-trial sections (CT group: M ¼ 6.91; 95% CI [6.13, 7.69]; DX group: M ¼ 7.43; 95% CI [6.54, 8.32]; F(1,43) ¼ 0.393, p ¼ 0.534, h2 ¼ 0.01). 3.3. Gaze indicators Finally, the analysis of the gaze indicators while approaching the VMS showed some significant differences between the groups of participants. First, there was a statistically significant interaction effect between group and VMS type (F(1,39) ¼ 3.70, p ¼ 0.036, h2 ¼ 0.11). Simple effect analysis revealed that the number of gazes at the VMS was lower in the CT group (M ¼ 5.55; 95% CI [4.85, 6.25]) as compared with the DX group (M ¼ 7.25; 95% CI [6.54, 7.96]), but only considering the text-type VMS (p ¼ 0.001); whereas group differences were non-significant with the pictogram-type VMS (CT group: M ¼ 7.45; 95% CI [6.39, 8.50]; DX group: M ¼ 8.29; 95% CI [7.21, 9.37]; p ¼ 0.262). Additionally, the main group effect was also statistically significant (F(1,39) ¼ 4.49, p ¼ 0.041, h2 ¼ 0.10), suggesting that, in general, participants in the CT group required fewer gazes (M ¼ 6.50; 95% CI [5.65, 7.35]) than participants in the DX group (M ¼ 7.77; 95% CI [6.90, 8.64]) to correctly respond to the VMS. Besides, the main effect of VMS was also statistically significant (F(1,39) ¼ 56.20, p < 0.001, h2 ¼ 0.59): overall, correctly read text-type VMS received fewer gazes (M ¼ 6.40; 95% CI [5.90, 6.90]) than correctly identified pictogram-type VMS (M ¼ 7.87; 95% CI [7.12, 8.62]). Second, there were no significant differences in the percentage of time dedicated to gaze at VMS out of the total gazing time registered. However, there were some unconfirmed tendencies (p < 0.10) in the main effects of group (F(1,39) ¼ 3.13, p ¼ 0.085, h2 ¼ 0.07) and VMS type variables (F(1,39) ¼ 3.19, p ¼ 0.082, h2 ¼ 0.07). These tendencies, if further confirmed, might suggest that participants in the DX group would dedicate a higher percentage of time to gaze at VMS to correctly respond (M ¼ 69.05, 95% CI [64.22, 73.87]) than participants in the CT group (M ¼ 63.15; 95% CI [58.46, 67.86]); and also that, overall, correctly identifying a pictogram-type VMS would require a higher percentage of time (M ¼ 67.36, 95% CI [63.49, 71.28]) than correctly reading a text-type VMS (M ¼ 64.85, 95% CI [61.40, 68.29]). However, even that these tendencies were expected and are consistent with the previous results, data analysis failed to reach the signification level. 4. Discussion The current study aimed to assess the difficulties of adult drivers with dyslexia acquiring the information shown in VMS. In addition, we also evaluated whether using pictograms as an alternative to text messages would be an effective countermeasure to reduce the difficulties of dyslexic individuals reading VMS. First, our results showed that the participants with dyslexia started reading the text messages when they were closer to the VMS (shorter legibility distances). In consequence, our results are consistent with the previous studies suggesting that the difficulties of adult individuals persist into adulthood (e.g., Bruck, 1993; Swanson and Hsieh, 2009), and suggest that these difficulties could interfere in daily activities requiring reading, such as driving
a vehicle and trying to acquire a text message in a VMS. In particular, our results suggest that their difficulties specially affect the time required to recognise the words (thus reducing the legibility distance), but not so the reading accuracy. This pattern of results (i.e., difficulties more apparent in speed than in accuracy) is consistent with the previous literature on adult dyslexia in Spanish rez-Coalla and Cuetos, 2015) and other (Afonso et al., 2015; Sua languages (Bruck, 1990; Callens et al., 2012; Nilssen-Nergård and Hulme, 2014; Parrila et al., 2007; Re et al., 2011; Reid et al., 2007; Tops et al., 2012; Wolff, 2009). However, in our study both groups of participants made few reading errors, probably because we used common words that were presented repeatedly during the experiment. Therefore, participants found no particular difficulties recognising accurately these words, although the dyslexic participants required more time to do it. In contrast, results from another study in our laboratory (Tejero et al., in press), in which the task was to read names of 64 cities, town, or villages displayed on direction traffic signs with no repetition and no previous presentation, suggested that adults with dyslexia were less accurate than normally reading adults, particularly with low-frequency and long names. It could be argued that adults with dyslexia might have adopted a more conservative response criterion than controls to avoid making mistakes, and consequently they would have waited until they were completely sure of the recognition of the word before responding. However, our results seem inconsistent with this interpretation, since the participants with dyslexia required more time to pronounce aloud the words in text-type VMS as compared to the pictograms, which suggests that they initiated reading before being completely confident of the recognition of the word. In addition, no significant differences in accuracy have been reported in the current study, which is not supporting a potential trade-off between speed and accuracy. Second, results in the current study revealed that, when the participants with dyslexia were approaching the VMS on a trial section, they dedicated more gazes at the VMS before being able to give a correct response and they manifested worse control of speed in the driving simulator. This suggests that trying to read the text messages in the VMS involved higher cognitive demands for these participants than for control participants. Driving a vehicle is a multi-tasking activity: when participants approach a VMS and have to read the message, they have also to keep an appropriate level of performance in the driving task. In the task we used in the driving simulator, participants were asked to keep steadily the posted speed. Control participants showed no particular difficulty performing correctly both the reading and the driving task. In contrast, participants with dyslexia found more challenging to keep a steady speed when they were approaching the VMS and had to read. Importantly, no differences between the groups were found outside the trial sections, which suggests that the worse control of speed observed in the participants with dyslexia would have essentially manifested when they were driving and trying to read the VMS at the same time, but not when they were merely driving through the non-trial sections. Some previous studies analysed the effect of increased cognitive demands on dyslexic individuals, generally assessing the effect of dual tasks on balance control (Legrand et al., 2012; Nicolson and Fawcett, 1990; Vieira et al., 2009). In particular, Legrand et al. (2012) and Vieira et al. (2009) observed that dyslexic children were significantly more unstable than non-dyslexic children, especially when they had to keep balance and reading in comparison to when they had to keep balance and just make saccades (but not reading). Consistently with our results, they suggested that the attention consumed by the reading task could be responsible for the loss of postural control in both groups of children. Similarly, in our study, we propose that the attention consumed by reading a
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90
VMS would be responsible for the observed worse control of speed. These results are also consistent with what was found in the already mentioned study by Tejero et al. (in press), in which the dyslexic participants showed a control of speed worse than normally reading participants when trying to read direction traffic signs containing names of cities, towns, or villages, but not when only driving was required. Finally, regarding the pictogram-based VMS, we observed no differences between dyslexic and control participants in the response distance, the number of errors, the response duration, or the number of gazes at the VMS. These results would not seem totally consistent with the previous traffic studies reporting difficulties of dyslexic individuals and low-skilled readers in responding to pictorial information in traffic signs (Brachacki et al., 1995; Fisher et al., 2015; Sigmundsson, 2005; Taylor et al., 2016). However, there are clear differences between our study and the previous research that may account for the diverging results. First, the study by Brachacki et al. (1995) was not specifically aimed at assessing the legibility distance. Participants had only to discriminate between real and false traffic signs on a computer screen to evaluate a potential deficit in symbolic, non-verbal learning (i.e., the ability to learn the meaning of pictorial symbols). Therefore, Brachacki's and our study were assessing quite different cognitive processes and a slower rate in learning symbolic signs is not inconsistent with a preserved legibility distance or, as the authors acknowledge in their paper, an appropriate understanding of the traffic signs. Regarding the comprehension of traffic signs, Taylor et al. (2016) provided some evidence that the higher the selfreported symptoms of dyslexia in non-clinical drivers, the worse the comprehension of traffic signs and the lower situation awareness. But, as opposed to our research, the study by Taylor et al. was neither specifically assessing the legibility of traffic signs. Moreover, Sigmundsson (2005) used a simple reaction-time task in which participants had to detect the sudden presentation of traffic signs over a traffic scene. In contrast, we embedded, in a more ecological way, the VMS in the traffic scene, simulating a progressive approach to the sign. Therefore, we think that our method is more appropriate to assess the legibility of traffic signs, whereas Sigmundsson actually evaluated the ability to react to sudden changes in the visual scene, with the aim to test a prediction coming from the magnocellular theory of dyslexia (Stein and Walsh, 1997). Finally, Fisher et al. (2015) observed that the self-reported symptoms of dyslexia in non-clinical population were correlated with lower compliance of posted speed limits. However, the regression analysis only confirmed that this association interacted with visual stress, suggesting that the relationship between dyslexia and speed compliance was mainly observed in participants also reporting high visual stress. In conclusion, despite the apparently diverging evidence, our study is not supporting a difference between dyslexic adults and controls in the recognition distance of pictogram-based VMS. Still, it should be noted that no interaction between group and VMS type was observed on speed variability in our study, which suggests that the previously reported main effect of group is similar with either the text-type or the pictogram-type VMS. This means that the adults with dyslexia were also experiencing some difficulties keeping a steady speed when trying to identify the pictograms in the VMS and therefore they were also dedicating more cognitive resources than control participants. 4.1. VMS type comparison The current paper is focused on the differences between adults with dyslexia and a control group. However, our results also report significant differences between the two VMS types (text vs.
89
pictograms) considering the full sample of participants. Overall, text-type VMS could be read at a longer distance and with higher accuracy, received fewer gazes, and approaching speed variability was lower. The only advantage of text-type VMS was in the average response duration (i.e. the utterance of the response), which was slightly slower for text-type VMS. Overall, these results suggest that short text messages may outperform matched pictograms in terms of legibility, which is apparently inconsistent with the previous literature suggesting that pictorial information would advantage equivalent text messages in different aspects (for reviews, see Bartłomiejczyk, 2013; Horberry et al., 2004; but see also Shinar and Vogelzang, 2013; Zahabi et al., 2017). Apart from the noticeable differences in the relative size of text and pictograms in VMS, in the current study we asked the participants to read aloud the contents of the VMS, what may give advantage to text messages because they are directly delivering the expected response. Still, we acquired new data in our laboratory (Roca et al., 2017) with a variation of the simulated driving task, in which a manual response is now required (i.e., using two levers to classify the messages, according to their meaning). Results in the latter experiment with a manual response replicate our findings with oral responses and point out that, in the particular case of VMS, short text messages can outperform pictograms in terms of legibility, most likely for the different text and pictogram sizes that VMS are able to accommodate (for a more detailed discussion on this topic, see Roca et al., 2017). 4.2. Limitations First, the current study reports data from a driving simulator and, therefore, convergent evidence in real-setting studies must be obtained to confirm our results. In addition, it should be noted that VMS in real settings usually include both text and pictograms at the same time. In contrast, our study only considered text or pictogram messages, but no combined messages. When text and pictograms are included in real settings, harmonised operation guidelines generally recommend avoiding redundancy, which implies that the meaning of a pictogram should not also be shown in text (Arbaiza and Lucas-Alba, 2012). Still, combined messages may act as a unified stimulus and the processing of a pictogram may be influenced by the text and vice versa. The evidence regarding whether this influence would be beneficial or detrimental is conflicted (see, for example, Holmqvist Olander et al., 2016) and it is not the aim of the current study to address this particular issue. 4.3. Conclusions The evidence provided in the current study reveals that, despite the potential deployment of compensatory strategies, adults with dyslexia may still find difficulties reading the text messages in VMS. These difficulties would manifest as shorter legibility distances and longer reading times, as well as an increased cognitive effort dedicated to acquire the contents of VMS (as manifested as worse speed control and more gazes at the VMS). In consequence, the current paper highlights the importance of considering the particular difficulties of the drivers with reading impairment when designing and operating VMS. In particular, research on applicable, inclusive measures, which would improve reading in low-skilled or dyslexic drivers without affecting non-reading impaired drivers, must be encouraged. Regarding the potential use of pictogram-based messages to substitute the information provided by text in VMS, our data revealed no significant differences between adults with dyslexia and control participants in the legibility distance or the verbal response times. Therefore, our results suggest that pictorial
90
J. Roca et al. / Applied Ergonomics 67 (2018) 83e90
information could be a potential countermeasure to reduce the reported group differences to acquire text information. However, this result must be considered with care, because we observed that participants with dyslexia also showed increased cognitive effort when trying to identify the pictograms in the VMS. Further research will be useful to clarify to what extent the processing of pictogrambased information is more demanding for adult drivers with dyslexia than control participants. Acknowledgements Funding: This work was partially funded by the Spanish fico (Directorate-General for Traffic) [grant n General de Tra Direccio number DGT-SPIP2015-01829], and the Spanish Ministerio de Economía y Competitividad (Ministry of Economy, Industry and Competitiveness) [grant number PSI2013-43862-P]. ~ alizacio n We would like to thank Inventarios y Proyectos de Sen Vial (I.P.S. Vial S.L.) for their altruistic collaboration by facilitating the use of the software CarDim to design the models of the traffic signs used in the current project. We would also like to thank Antonio rez Pen ~ a in the Ministerio de Fomento (Ministry of Development) Pe for the useful information provided regarding the design of traffic signs. References rez-Coalla, P., Cuetos, F., 2015. Spelling impairments in Spanish Afonso, O., Sua dyslexic adults. Front. Psychol. 6 (MAR), 1e10. https://doi.org/10.3389/fpsyg. 2015.00466. Amador, J., 2013. Escala de inteligencia de Wechsler para adultos-IV (WAIS-IV). 4 n. Retrieved from. http://diposit.ub.edu/dspace/bitstream/2445/33834/1/ Edicio Escala de inteligencia de Wechsler para adultos-WAIS-IV.pdf. American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental Disorders, fifth ed. American Psychiatric Association, Washington, DC. Arbaiza, A., Lucas-Alba, A., 2012. Variable Message Signs Harmonisation PRINCIPLES OF VMS MESSAGES DESIGN Supporting guideline. n Espan ~ ola de Normalizacio n y Certificacio n, 2015. Sen ~ alizacio n vertical en Asociacio carretera. Paneles de mensaje variable. Norma EN 12966:2014. AENOR, Madrid. Bartłomiejczyk, M., 2013. Text and image in traffic signs. Linguist. Silesiana 34, 111e131. Brachacki, G.W., Nicolson, R.I., Fawcett, A.J., 1995. Impaired recognition of traffic signs in adults with dyslexia. J. Learn. Disabil. 28 (5), 297e301. https://doi.org/ 10.1177/002221949502800505. Bruck, M., 1990. Word-recognition skills of adults with childhood diagnoses of dyslexia. Dev. Psychol. 26 (3), 439e454. https://doi.org/10.1037/0012-1649.26.3. 439. Bruck, M., 1993. Word recognition and component phonological processing skills of adults with childhood diagnosis of dyslexia. Dev. Rev. Spec. Phonological Process. Learn. Disabil. https://doi.org/10.1006/drev.1993.1012. Callens, M., Tops, W., Brysbaert, M., 2012. Cognitive profile of students Who enter higher education with an indication of Dyslexia. PLoS One 7 (6) e38081. https:// doi.org/10.1371/journal.pone.0038081. Cuetos Vega, F., Ramos S anchez, J.L., 2012. PROLEC-SE. TEA Ediciones S.A, Madrid. n General de Tra fico, 2016. Anuario Estadístico General 2015. Direccio n Direccio fico, Madrid. Retrieved from. http://www.dgt.es/es/seguridadGeneral de Tra vial/estadisticas-e-indicadores/publicaciones/anuario-estadistico-general/. Fisher, C., Chekaluk, E., Irwin, J., 2015. Impaired driving performance as evidence of a magnocellular deficit in dyslexia and visual stress. Dyslexia 21 (4), 350e360. https://doi.org/10.1002/dys.1504. Garvey, P.M., 2002. Synthesis on the Legibility of Variable Message Signing - United States Access Board. Retrieved February 14, 2017, from. https://www.accessboard.gov/research/completed-research/synthesis-on-the-legibility-ofvariable-message-signing. Garvey, P.M., Mace, D.J., 1996. Changeable message sign visibility. Natl. Tech. Inf. Serv. 137. € m, M., 2016. The effect of illusHolmqvist Olander, M., Wennås Brante, E., Nystro tration on improving text comprehension in dyslexic adults. Dyslexia 23 (1), 42e65. https://doi.org/10.1002/dys.1545. Horberry, T., Castro, C., Martos, F., Mertova, P., 2004. An introduction to transport signs and an overview of this book. In: Castro, C., Horberry, T. (Eds.), The Human Factors of Transport Signs. CRC Press, pp. 1e15. https://doi.org/10.1201/ 9780203457412.ch1. Houts, P.S., Doak, C.C., Doak, L.G., Loscalzo, M.J., 2006. The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Educ. Couns. https://doi.org/10.1016/j. pec.2005.05.004.
-Mazars, K., Lemoine, C., Ge rard, C.L., Bucci, M.P., Legrand, A., Bui-Quoc, E., Dore 2012. Effect of a dual task on postural control in dyslexic children. PLoS One 7 (4) e35301. https://doi.org/10.1371/journal.pone.0035301. Nicolson, R.I., Fawcett, A.J., 1990. Automaticity: a new framework for dyslexia research? Cognition 35 (2), 159e182. https://doi.org/10.1016/0010-0277(90) 90013-A. Nilssen-Nergård, T., Hulme, C., 2014. Developmental dyslexia in adults: behavioural manifestations and cognitive correlates. Dyslexia 20 (3), 191e207. https://doi. org/10.1002/dys.1477. Nygardhs, S., 2011. Literature Review on Variable Message Signs (VMS) 2006e2009. VTI. Nygardhs, S., Helmers, G., 2007. VMSevariable Message Signs a Literature Review. Parrila, R., Georgiou, G., Corkett, J., 2007. University students with a significant history of reading difficulties: what is and is not compensated? Except. Educ. 17 (2), 195e220. https://doi.org/10.1371/journal.pone.0106550. Peterson, R.L., Pennington, B.F., 2012. Developmental dyslexia. Lancet 379 (9830), 1997e2007. https://doi.org/10.1016/S0140-6736(12)60198-6. Re, A.M., Tressoldi, P.E., Cornoldi, C., Lucangeli, D., 2011. Which tasks best discriminate between dyslexic university students and controls in a transparent language? Dyslexia 17 (3), 227e241. https://doi.org/10.1002/dys.431. Reid, A.A., Szczerbinski, M., Iskierka-Kasperek, E., Hansen, P., 2007. Cognitive profiles of adult developmental dyslexics: theoretical implications. Dyslexia 13 (1), 1e24. https://doi.org/10.1002/dys.321. Roca, J., Insa, B., Tejero, P., 2017. Legibility of Text and Pictograms in Variable Message Signs: Can Text Outperform Pictograms? (Manuscript submitted for publication). Serrano, F., Defior, S., Martos, F., 2003. To be or not to be phonologically aware: a reflection about metalinguistic skills of student of teacher. In: Joshi, R.M., Leong, C.K., Kaczmarek, B.L.J. (Eds.), Literacy Acquisition: the Rol of Phonology, Morphology and Orthography. IOS Press, Amsterdam, NL. Shaywitz, S.E., Shaywitz, B.A., Fulbright, R.K., Skudlarski, P., Mencl, W.E., Constable, R.T., Gore, J.C., 2003. Neural systems for compensation and persistence: young adult outcome of childhood reading disability. Biol. Psychiatry 54 (1), 25e33. https://doi.org/10.1016/S0006-3223(03)01836-X. Shinar, D., Vogelzang, M., 2013. Comprehension of traffic signs with symbolic versus text displays. Transp. Res. Part F Traffic Psychol. Behav. 18, 72e82. https://doi. org/10.1016/j.trf.2012.12.012. Sigmundsson, H., 2005. Do visual processing deficits cause problem on response time task for dyslexics? Brain Cognit. 58 (2), 213e216. https://doi.org/10.1016/j. bandc.2004.11.007. Stein, J., Walsh, V., 1997. To see but not to read; the magnocellular theory of dyslexia. Trends Neurosci. 20 (4), 147e152. Retrieved from. http://www.ncbi. nlm.nih.gov/pubmed/9106353. Su arez-Coalla, P., Cuetos, F., 2015. Reading difficulties in Spanish adults with dyslexia. Ann. Dyslexia 65 (1), 33e51. https://doi.org/10.1007/s11881-015-01013. Swanson, H.L., Hsieh, C.-J., 2009. Reading disabilities in adults: a selective metaanalysis of the literature. Rev. Educ. Res. 79 (4), 1362e1390. https://doi.org/ 10.3102/0034654309350931. Taylor, B., Chekaluk, E., Irwin, J., 2016. Reading the situation: the relationship between dyslexia and situational awareness for road sign information. Transp. Res. Part F Traffic Psychol. Behav. 36, 6e13. https://doi.org/10.1016/j.trf.2015.11. 005. Tejero, P., Insa, B., Roca, J. (in press). Difficulties of Drivers with Dyslexia when Reading Direction Traffic Signs: Analysis of Reading, Driving and Eye Gaze Performance. Journal of Learning Disabilities. Temple, E., Deutsch, G.K., Poldrack, R.A., Miller, S.L., Tallal, P., Merzenich, M.M., Gabrieli, J.D.E., 2003. Neural deficits in children with dyslexia ameliorated by behavioral remediation: evidence from functional MRI. Proc. Natl. Acad. Sci. U. S. A. 100 (5), 2860e2865. https://doi.org/10.1073/pnas.0030098100. Tops, W., Callens, M., Lammertyn, J., van Hees, V., Brysbaert, M., 2012. Identifying students with dyslexia in higher education. Ann. Dyslexia 62 (3), 186e203. https://doi.org/10.1007/s11881-012-0072-6. U.S. Department of Transportation, 2016. Highway Statistics 2015. Distribution of Licensed Drivers - 2015 (Table DL-20). Retrieved from. https://www.fhwa.dot. gov/policyinformation/statistics/2015/dl20.cfm. Ullman, B., Ullman, G., Dudek, C., Ramirez, E., 2005. Legibility distances of smaller letters in changeable message signs with light-emitting diodes. Transp. Res. Rec. 1918 (1), 56e62. https://doi.org/10.3141/1918-07. Vieira, S., Quercia, P., Michel, C., Pozzo, T., Bonnetblanc, F., 2009. Cognitive demands impair postural control in developmental dyslexia: a negative effect that can be compensated. Neurosci. Lett. 462 (2), 125e129. https://doi.org/10.1016/j.neulet. 2009.06.093. Wilkins, A.J., Evans, B.J.W., 2010. Visual stress, its treatment with spectral filters, and its relationship to visually induced motion sickness. Appl. Ergon. 41 (4), 509e515. https://doi.org/10.1016/j.apergo.2009.01.011. Wolff, U., 2009. Phonological and surface subtypes among university students with dyslexia. Int. J. Disabil. Dev. Educ. 56 (1), 73e91. https://doi.org/10.1080/ 10349120802682083. Zahabi, M., Machado, P., Pankok, C., Lau, M.Y., Liao, Y.-F., Hummer, J., Kaber, D.B., 2017. The role of driver age in performance and attention allocation effects of roadway sign count, format and familiarity. Appl. Ergon. 63, 17e30. https://doi. org/10.1016/j.apergo.2017.04.001.