Journal of Orthopaedic Science xxx (2018) 1e5
Contents lists available at ScienceDirect
Journal of Orthopaedic Science journal homepage: http://www.elsevier.com/locate/jos
Original Article
Flexible flatfoot of 6e13-year-old children: A cross-sectional study Jichao Yin a, b, Hongmou Zhao c, Guihua Zhuang a, *, Xiaojun Liang c, Xinglv Hu b, Yi Zhu e, Rongqiang Zhang d, Xiaochen Fan c, Yi Cao c a
Department of Epidemiology and Biostatistics, School of Public Health, Xi'an Jiaotong University Health Science Center, Xi'an, Shaanxi, 710061, China Xi'an Hospital Of Traditional Chinese Medicine, China c Foot and Ankle Surgery Department, Honghui Hospital of Xi'an Jiaotong University College of Medicine, China d Public Health of Department of Shaanxi University of Chinese Medicine, China e The Second Affiliated Hospital of Hainan Medical University, China b
a r t i c l e i n f o
a b s t r a c t
Article history: Received 2 April 2017 Received in revised form 23 January 2018 Accepted 7 February 2018 Available online xxx
Background: This cross-sectional study aims to investigate the flexible flatfoot (FFF) prevalence and related factors in school-aged children. Methods: A total of 1059 children aged 6e13 years were included. Dynamic footprints according to the FootScan system were collected from both feet. The relationship of FFF with age, gender, side, and body mass index (BMI) was investigated. Results: FFF percentage decreased from 39.5% at 6 years to 11.8% at 12 years and reached a plateau at 12 e13 years. Overweight (OR 1.35, 95%CI 1.03e1.77, P ¼ 0.03) and obese (OR 2.43, 95%CI 1.81e3.26, P< 0.01) showed a positive correlation with percentage of FFF children. No correlation was found between FFF prevalence and gender or side. Conclusions: FFF prevalence decreases with age and reaches a plateau at 12e13 years. Moreover, FFF prevalence is positively correlated with increased BMI and body height. © 2018 The Japanese Orthopaedic Association. Published by Elsevier B.V. All rights reserved.
1. Introduction The medial longitudinal arch is one of the most significant foot arches, which are defined by the position of bones and stabilized by active and passive soft tissue structures. The morphologic characteristics of flexible flatfoot (FFF) are the heel valgus and the “collapse” of the medial longitudinal arch [1]. During the early age, the pediatric FFF in full weight-bearing position is considered a normal physiological condition, but it represents a pathological deformity requiring additional diagnostics and treatment in later age [2,3]. However, identifying the correct time for the prognosis and intervention of pathological pediatric flatfoot remains conflicting in the literature [4]. Several previous studies focused on flatfoot in school-aged children [5e8]. Most studies confirmed the
* Corresponding author. Department of Epidemiology and Biostatistics, School of Public Health, Xi'an Jiaotong University Health Science Center, No.76 West Yanta Road, Xi'an, Shaanxi, 710061, China. Fax: þ86 29 85251042. E-mail addresses:
[email protected] (J. Yin),
[email protected] (H. Zhao),
[email protected] (G. Zhuang),
[email protected] (X. Liang),
[email protected] (X. Hu),
[email protected] (Y. Zhu), 372590580@ qq.com (R. Zhang),
[email protected] (X. Fan),
[email protected] (Y. Cao).
FFF prevalence associated with age, but the prevalence varied greatly among studies [5,6,8]. Some studies collected bilateral feet of included children [9], and others collected one foot of each child [5,6,8,10]. All previous studies collected footprints in status standing with one or two feet [5,6,8,10]. The objective of the current study was to investigate the FFF prevalence and relative factors in school-aged children. 2. Materials and methods This cross-sectional epidemiological study was conducted between October 2013 and July 2015 in three elementary schools in Xi'an, China, and approved by the research board of the Xi'an Health and Family Planning Commission. The study was registered in the Chinese Clinical Trial Registry (No. chiCTR-EOC-16009772). 2.1. Study cohort Children from first to eight grades in elementary school were included in the study. Age, gender, body height, body weight, body mass index (BMI), place of residence and foot contact area were recorded. The exclusion criteria were (1) children with major
https://doi.org/10.1016/j.jos.2018.02.004 0949-2658/© 2018 The Japanese Orthopaedic Association. Published by Elsevier B.V. All rights reserved.
Please cite this article in press as: Yin J, et al., Flexible flatfoot of 6e13-year-old children: A cross-sectional study, Journal of Orthopaedic Science (2018), https://doi.org/10.1016/j.jos.2018.02.004
2
J. Yin et al. / Journal of Orthopaedic Science xxx (2018) 1e5
medical diseases that could affect physical fitness and (2) children with musculoskeletal disorders that could affect the measurements of gait and footprints. 2.2. Measurements The footprints according to the FootScan® 7.x plate system (RSscan International, Olen, Belgium) were collected from both feet with the child walking through the plate in normal speed. The footprints are combined figures of all foot contact phase. The sampling frequency by walking was set to 250 Hz. Staheli's arch index (SAI) was used to measure the FFF from the 2D pressure images saved in the FootScan system [11]. First, the medial tangential line was marked by the most medial points at the metatarsal and heel. Second, a heel line was drawn from the most medial point of the heel, perpendicular to the medial tangential line, to the lateral border of the heel. Heel width (a) was defined as the length of the foot section on the heel line. Finally, a midfoot line was drawn from the midpoint of the medial tangential line and parallel to the heel line (Fig. 1A and B). The midfoot width (b) was defined as the length of the foot section on the midfoot line. SAI is the ratio of midfoot width to heel width (b/a). According to SAI method, the FFF means the SAI ratio more than 1.0 [11]. The SAI measurement was reported with good diagnostic ability and excellent inter-rater reliability (intraclass correlation coefficients [ICC] ¼ 0.95) and testeretest reliability (ICC ¼ 0.96) [5]. 2.3. Statistical analysis Descriptive statistics were calculated as mean ± standard deviation (SD). FFF percentage was calculated with feet number, and
that of FFF children was calculated with children number. Children with a single flatfoot were included in the FFF children group. Statistical analysis of the included data was performed using the Student t test or Pearson chi-square test. Results from logistic regression analyses were presented as odds ratio (OR) and 95% confidence interval (CI). The pooled continuous variables were analyzed with use of weighted mean difference. The level of significance was set to a ¼ 0.05. The statistical analyses were performed with SPSS 17.0 software (SPSS Inc., Chicago, Illinois) and Review Manager 5.0 (Cochrane Collaboration). 3. Results 3.1. FFF prevalence The study included 1059 children aged 6e13 years (mean age 9.5 ± 2.0 years). Among the children, 575 were boys and 484 were girls, with a total of 2118 feet. The mean FFF percentage was 19.3% (409/2118), and the mean FFF children percentage was 26.6% (282/ 1059). No correlation was found between FFF prevalence and gender (OR 1.01, 95%CI 0.81e1.26, P ¼ 0.91), between that and side (OR 1.21, 95%CI 0.97e1.50, P ¼ 0.07), and between that and place of residence (OR 0.84, 95%CI 0.68e1.05, P ¼ 0.13) (Table 1). 3.2. Association between FFF and age FFF percentage decreased from 39.5% at 6 years to 11.8% at 12 years. The percentage reached a significant difference between age groups (P<0.01). The percentage of FFF children decreased by age, and it was in consistence with the FFF percentage. However, the decrease reached a plateau at 12e13 years (Fig. 2A). The percentage
Fig. 1. Measure of SAI ¼ b/a in normal foot (A) and flatfoot (B).
Please cite this article in press as: Yin J, et al., Flexible flatfoot of 6e13-year-old children: A cross-sectional study, Journal of Orthopaedic Science (2018), https://doi.org/10.1016/j.jos.2018.02.004
J. Yin et al. / Journal of Orthopaedic Science xxx (2018) 1e5
3
of bilateral FFF children decreased by age, but the decrease was not remarkable in single FFF children (Fig. 2B).
flatfoot group, and no significant difference in mean weight was found among the groups. The included children were grouped into underweight (-SD), healthy (between ±SD), overweight (þSD), and obese (þ2SD) by BMI data according to the World Health Organization (WHO) criteria [12]. The percentage of FFF children was higher in the overweight (OR 1.35, 95%CI 1.03e1.77, P ¼ 0.03) and obese groups (OR 2.43, 95%CI 1.81e3.26, P<0.01) than in the healthy group. The percentage of FFF children in the underweight group reached no significant difference while comparing healthy group (OR 0.64, 95%CI 0.38e1.07, P ¼ 0.09). The included children were grouped into under height (SD), healthy (between ± SD), over height (þSD), and very tall (þ2SD) by height data according to the WHO criteria [12]. The percentage of FFF children was higher in the very tall group (OR 1.76, 95%CI 1.17e2.39, P<0.01) and lower in the under height group (OR 0.71, 95%CI 0.51e0.99, P ¼ 0.04) than in the healthy group. No significant difference was reached between over height group and normal group (OR 0.77, 95%CI 0.55e1.07, P ¼ 0.11). The mean height of included children in FFF group was smaller than the normal group (P<0.01, Table 2). However, if excluded the age factor, the mean height reached no significant difference between the two groups (P ¼ 0.08, Fig. 3).
3.3. Association between FFF and BMI
4. Discussion
BMI showed a positive correlation with FFF prevalence in children aged 6e13 years (Table 2). The mean height was small in the
Increasing evidence shows that flatfoot can lead to significant symptoms and impairment of health-related quality of life
Table 1 The occurrence of flatfoot in 1059 children aged 6 to 13 years. Children
Age (y) 6 7 8 9 10 11 12 13 Boys Girls Left Right Urban Rural
Feet
Normal
Flatfoot
P value
Normal
Flatfoot
P value
42 76 110 116 124 135 108 66 417 360 e e 469 308
39 42 49 39 38 37 23 15 158 124 e e 167 115
0.00
98 172 246 254 272 294 231 142 928 783 839 872 1041 670
64 64 72 56 52 50 31 20 222 185 220 187 231 176
0.00
0.50 e 0.74
0.91 0.07 0.13
Note: Categorical data were compared by the Chi-square test.
Fig. 2. The flatfoot percentages and the flatfoot children percentage in all age categories (A); the percentages of healthy feet children, single and bilateral flatfeet children in all age categories (B).
Please cite this article in press as: Yin J, et al., Flexible flatfoot of 6e13-year-old children: A cross-sectional study, Journal of Orthopaedic Science (2018), https://doi.org/10.1016/j.jos.2018.02.004
4
J. Yin et al. / Journal of Orthopaedic Science xxx (2018) 1e5
Table 2 Correlation between body mass index and flatfoot in different ages (mean ± SD). Age (y)
Weight (Kg) Normal
6 7 8 9 10 11 12 13 Total
24.5 24.4 27.2 30.0 35.3 37.7 46.0 54.8 35.3
± ± ± ± ± ± ± ± ±
4.7 3.3 5.7 5.1 8.8 7.9 12.0 14.8 12.4
Height (cm) Flatfoot 26.2 25.7 30.1 32.9 35.0 42.3 51.2 55.9 34.7
± ± ± ± ± ± ± ± ±
5.5 5.6 7.8 7.0 8.7 12.1 12.9 14.5 12.6
P value
Normal
0.14 0.12 0.01 0.01 0.85 0.01 0.06 0.76 0.49
121.7 120.4 127.4 132.6 138.2 144.4 151.0 159.5 138.0
± ± ± ± ± ± ± ± ±
BMI (Kg/m2) Flatfoot
6.2 5.4 6.9 6.6 7.4 8.0 10.0 9.8 14.1
122.3 122.2 127.0 134.8 137.5 146.3 153.6 157.5 134.5
± ± ± ± ± ± ± ± ±
7.8 7.0 6.4 7.0 7.3 7.1 6.8 7.2 13.4
P value
Normal
0.70 0.12 0.73 0.08 0.61 0.19 0.24 0.46 0.00
16.4 16.8 16.6 17.0 18.3 18.0 19.7 21.3 18.0
± ± ± ± ± ± ± ± ±
2.4 1.8 2.3 2.0 3.2 2.8 3.1 4.2 3.13
Flatfoot 17.5 17.1 18.6 18.0 18.3 19.5 21.5 22.3 18.7
± ± ± ± ± ± ± ± ±
2.8 2.6 4.1 2.5 3.3 4.1 4.4 4.3 3.76
P value 0.06 0.46 0.00 0.01 1.00 0.01 0.02 0.41 0.00
Note: Categorical data were compared by the Student t test.
Fig. 3. The forest plot of pooled mean height in different age groups between flatfoot and normal children.
[2,13e15]. However, most pediatric FFF is physiological and will turn to normal in time. Identifying the correct time boundary of physiological and pathological FFF in children is difficult because many other potential confounders may be correlated with FFF besides age. Evans and Rome [16] systematically reviewed the literature and concluded that the available prevalence of pediatric FFF estimates was limited by variable sampling, assessment measures, and age groups, thereby resulting in disparate findings (0.6%e 77.9%). Accordingly, large sample data from different areas of the world are needed to clear this issue. The results showed a decreased FFF prevalence and a negative correlation with age. This result was also confirmed by other studies [5,6,8,17,18]. However, FFF prevalence varied greatly in previous studies. Sadeghi-Demneh et al. [8] reported the decrease to be from 26% at 7 years to 14% at 14 years. Rao et al. [18] reported the decrease to be from 14.9% at 6 years to 2.5% at 13 years. According to the current results, FFF prevalence decreased from 39.5% at 6 years to 11.8% at 12 years and reached a plateau at 12e13 years (Fig. 2A). Three studies reported FFF prevalence with age higher than 12 years, and no correlation between age and FFF percentage was found in all three studies [6e8]. Waseda et al. [19] found that the arch height ratio elevated from 11 to 13 years in boys and from 10 to 12 years in girls. However, surgical intervention of pediatric FFF in the literature was earlier than this age. Subtalar joint arthroeresis was used in children with FFF as young as 3e4 years old [20,21]. We do agree with the following treatment suggestion of pediatric flatfoot by Evans and Rome [16] only footwear and BMI control may be enough if the FFF child function is normal, footwear or orthoses should be used if the FFF child has pain and function is not normal, and all FFF children should be monitored to ensure that the foot does not get “flatter” over time. The medial longitudinal arch of FFF children younger than 12e13 years may be a physiological condition and become normal in time or with assisted conservative treatment. However, the flatfoot may be a physiological condition
after this age, and surgical intervention can be used after conservative treatment. We found that the prevalence of bilateral FFF children decreased by age, but the decrease was not remarkable in single FFF children (Fig. 2B). This result may be due to the medial longitudinal arches not developing at the same time and to some bilateral FFF changing to single FFF children and some single FFF children changing to normal. Chen et al. [22] evaluated 1748 preschool-aged children and found that the prevalence of bilateral FFF decreased from 54.5% at 3 years to 21.0% at 6 years, and that the prevalence of unilateral FFF increased from 14.2% to 17.5%. Our study also showed that the mean height in flatfoot group is smaller than that in normal children group as showed in Table 2. That is just because of the percentage of FFF decreases with age, so, there are more flatfoot children with small age. As we can see that the mean weight in each age group between flatfoot and normal children are similar. Some studies reported a higher FFF prevalence in boys than in girls [5,17,22,23]. However, no correlation was found between FFF prevalence and gender in the current study. This finding is ndez et al. [25] consistent in other literature [7,8,24]. Pita-Ferna revealed that the prevalence of flatfoot was significantly higher in females than in males (OR 3.54; 95%CI 1.26e10.01) in adults. The mean FFF percentage of left feet was 20.8% and that of right feet was 17.8% in our study, but still no significant difference was reached. This result is consistent with that in adults [25]. Many previous studies confirmed the positive correlation between FFF prevalence in school-aged children and BMI [5,6,8,10]. This positive correlation was also confirmed in preschool-aged children [8], adolescents [26], and adults [25]. Nevertheless, negative results were still observed [18,27]. The definitions of overweight and obesity in previous studies were based on different classification methods. To define overweight, some studies used BMI 85th [5,15] or 90th [17] percentile, Cole's classification [6,28,29], classification of the Taiwan Department of Health [22], and even the WHO classification system for adults [8]. In current
Please cite this article in press as: Yin J, et al., Flexible flatfoot of 6e13-year-old children: A cross-sectional study, Journal of Orthopaedic Science (2018), https://doi.org/10.1016/j.jos.2018.02.004
J. Yin et al. / Journal of Orthopaedic Science xxx (2018) 1e5
study, we used the WHO classification of BMI for children distinguished by age and gender [12]. Our results confirmed the positive relationship between FFF prevalence and BMI. However, the OR of obese decreased with age (from 3.25 to 1.78), and the OR of overweight increased with age. This result may be due to the medial longitudinal arches developing quickly in children with normal BMI, and it implies that being overweight plays a role in impeding the development of the medial longitudinal arch. Also, we found that the percentage of FFF children was positively correlated with the height increase in same age children regardless of weight factor. This is might because of the height increasing dose not so much in accordance with the development of foot arch, and even stunt the development of foot arch. Although the mean height was higher in normal group than FFF group for the included children, this may because of a relatively more FFF in small age children. If the age factor was excluded, the difference disappears. We confirmed the relationship between FFF prevalence in 6e13year-old school-aged children and age and BMI. These data were the initial ones collected from the FootScan system. The footprints collected from walking can reflect the real condition of foot arch, because the strength of leg muscles in walking may be different from standing, especially the tibialis posterior muscle will increase the height of the foot arch. Nevertheless, this study has its limitations. Other positive affect factors, such as joint laxity [22,24,18], and physical activity [30] reported in the literature were not evaluated in the current study. As these issues remain controversial, well-designed studies are still needed. In conclusion, FFF prevalence decreased with age and reached a plateau at 12e13 years. No correlation was found between FFF prevalence and gender or side. FFF prevalence was positively correlative with increased BMI and body height. Declaration of conflicting interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Acknowledgment This study was funded by the Social Public Relations Project of Shaanxi Province (grant number: 2014K11-05-02) References [1] Kanatli U, Yetkin H, Cila E. Footprint and radiographic analysis of the feet. J Pediatr Orthop 2001 Mar-Apr;21(2):225e8. [2] Moraleda L, Mubarak SJ. Flexible flatfoot: differences in the relative alignment of each segment of the foot between symptomatic and asymptomatic patients. J Pediatr Orthop 2011 Jun;31(4):421e8. [3] Vukasinovi c ZS, Spasovski DV, Matanovi c DD, Zivkovi c ZM, Stevanovi c VB, Janici c RR. Flatfoot in children. Acta Chir Iugosl 2011;58(3):103e6. [4] Evans AM. The flat-footed child-to treat or not to treat: what is the clinician to do? J Am Podiatr Med Assoc 2008 Sep-Oct;98(5):386e93. [5] Chang CH, Chen YC, Yang WT, Ho PC, Hwang AW, Chen CH, Chang JH, Chang LW. Flatfoot diagnosis by a unique bimodal distribution of footprint index in children. PLoS One 2014 Dec 31;9(12). e115808. [6] Chen JP, Chung MJ, Wang MJ. Flatfoot prevalence and foot dimensions of 5- to 13-year-old children in Taiwan. Foot Ankle Int 2009 Apr;30(4):326e32.
5
[7] Leung AK, Cheng JC, Mak AF. A cross-sectional study on the development of foot arch function of 2715 Chinese children. Prosthet Orthot Int 2005 Dec;29(3):241e53. [8] Sadeghi-Demneh E, Jafarian F, Melvin JM, Azadinia F, Shamsi F, Jafarpishe M. Flatfoot in school-age children: prevalence and associated factors. Foot Ankle Spec 2015 Jun;8(3):186e93. [9] Chen KC, Tung LC, Yeh CJ, Yang JF, Kuo JF, Wang CH. Change in flatfoot of preschool-aged children: a 1-year follow-up study. Eur J Pediatr 2013 Feb;172(2):255e60. [10] Sadeghi-Demneh E, Azadinia F, Jafarian F, Shamsi F, Melvin JM, Jafarpishe M, Rezaeian Z. Flatfoot and obesity in school-age children: a cross-sectional study. Clin Obes 2016 Feb;6(1):42e50. [11] Staheli LT, Chew DE, Corbett M. The longitudinal arch. A survey of eight hundred and eighty-two feet in normal children and adults. J Bone Joint Surg Am 1987 Mar;69(3):426e8. [12] WHO. Growth reference 5e19 years. Geneva: World Health Organization [online]. Available from: LIRL: http://www.who.int/growthref/who2007_bmi_ for_age/en/ [Accessed 20 February 2018]. [13] Benedetti MG, Ceccarelli F, Berti L, Luciani D, Catani F, Boschi M, Giannini S. Diagnosis of flexible flatfoot in children: a systematic clinical approach. Orthopedics 2011 Jan 1;34(2):94. [14] Kothari A, Stebbins J, Zavatsky AB, Theologis T. Health-related quality of life in children with flexible flatfeet: a cross-sectional study. J Child Orthop 2014 Dec;8(6):489e96. pez Lo pez D, Bouza Prego Mde L, Requeijo Constenla A, Saleta Canosa JL, [15] Lo Bautista Casasnovas A, Tajes FA. The impact of foot arch height on quality of life in 6-12 year olds. Colomb Med (Cali) 2014 Dec 30;45(4):168e72. [16] Evans AM, Rome KA. Cochrane review of the evidence for non-surgical interventions for flexible pediatric flat feet. Eur J Phys Rehabil Med 2011 Mar;47(1):69e89. [17] Pfeiffer M, Kotz R, Ledl T, Hauser G, Sluga M. Prevalence of flat foot in preschool-aged children. Pediatrics 2006 Aug;118(2):634e9. [18] Rao UB, Joseph B. The influence of footwear on the prevalence of flat foot. A survey of 2300 children. J Bone Joint Surg Br 1992 Jul;74(4):525e7. [19] Waseda A, Suda Y, Inokuchi S, Nishiwaki Y, Toyama Y. Standard growth of the foot arch in childhood and adolescenceederived from the measurement results of 10,155 children. Foot Ankle Surg 2014 Sep;20(3):208e14. [20] Koning PM, Heesterbeek PJ, de Visser E. Subtalar arthroereisis for pediatric flexible pes planovalgus: fifteen years experience with the cone-shaped implant. J Am Podiatr Med Assoc 2009 Sep-Oct;99(5):447e53. [21] Metcalfe SA, Bowling FL, Reeves ND. Subtalar joint arthroereisis in the management of pediatric flexible flatfoot: a critical review of the literature. Foot Ankle Int 2011 Dec;32(12):1127e39. [22] Chen KC, Yeh CJ, Tung LC, Yang JF, Yang SF, Wang CH. Relevant factors influencing flatfoot in preschool-aged children. Eur J Pediatr 2011 Jul;170(7): 931e6. [23] Echarri JJ, Forriol F. The development in footprint morphology in 1851 Congolese children from urban and rural areas, and the relationship between this and wearing shoes. J Pediatr Orthop B 2003 Mar;12(2):141e6. [24] Lin CJ, Lai KA, Kuan TS, Chou YL. Correlating factors and clinical significance of flexible flatfoot in preschool children. J Pediatr Orthop 2001 May-Jun;21(3): 378e82. lez-Martín C, Seoane-Pillado T, Lo pez-Calvin ~ o B, [25] Pita-Fern andez S, Gonza rtega-Díaz S, Gil-Guille n V. Validity of footprint analysis to determine Pe flatfoot using clinical diagnosis as the gold standard in a random sample aged 40 years and older. J Epidemiol 2015;25(2):148e54. [26] Tenenbaum S, Hershkovich O, Gordon B, Bruck N, Thein R, Derazne E, Tzur D, Shamiss A, Afek A. Flexible pes planus in adolescents: body mass index, body height, and genderean epidemiological study. Foot Ankle Int 2013 Jun;34(6): 811e7. [27] Kothari A, Bhuva S, Stebbins J, Zavatsky AB, Theologis T. An investigation into the aetiology of flexible flat feet: the role of subtalar joint morphology. Bone Joint J 2016 Apr;98-B(4):564e8. [28] Mickle KJ, Steele JR, Munro BJ. The feet of overweight and obese young children: are they flat or fat? Obesity (Silver Spring) 2006 Nov;14(11): 1949e53. [29] Riddiford-Harland DL, Steele JR, Baur LA. Are the feet of obese children fat or flat? Revisiting the debate. Int J Obes (Lond) 2011 Jan;35(1):115e20. [30] Pauk J, Ezerskiy V, Raso JV, Rogalski M. Epidemiologic factors affecting plantar arch development in children with flat feet. J Am Podiatr Med Assoc 2012 Mar-Apr;102(2):114e21.
Please cite this article in press as: Yin J, et al., Flexible flatfoot of 6e13-year-old children: A cross-sectional study, Journal of Orthopaedic Science (2018), https://doi.org/10.1016/j.jos.2018.02.004