Intersite variations of the Gillette Gait Index

Available online at www.sciencedirect.com

Gait & Posture 28 (2008) 483–487 www.elsevier.com/locate/gaitpost

Intersite variations of the Gillette Gait Index Mark L. McMulkin a,*, Bruce A. MacWilliams b,c a

Walter E. Griffin and Agnes M. Griffin Motion Analysis Laboratory, Shriners Hospitals for Children, 911 West 5th Avenue, Spokane, WA 99204, USA b Motion Analysis Laboratory, Shriners Hospitals for Children, Fairfax Road, Virginia Street, Salt Lake City, UT 84103, USA c Department of Orthopaedic Surgery, University of Utah, Salt Lake City, UT, USA Received 20 December 2007; received in revised form 12 February 2008; accepted 5 March 2008

Abstract The Gillette Gait Index (GGI) is a tool used to measure pathologic gait severity and assess outcomes. The purpose of this study is to assess the variation in calculated GGI values with different sets of control data. Five able bodied control sets from four labs were used to establish the basis of the GGI. Gait data from three pediatric patients seen pre- and post-operatively at one lab and one adult control subject that visited each lab were input to calculate GGI values. Differences in underlying control data created large differences in computed GGI values for both pathologic and able bodied subjects. Initial pre-operative GGI values calculated for the three patients with cerebral palsy using different control data sets varied widely with differences as large as 1129 and had magnitudes of improvement differing by as much as 800 (or 21%). GGI value differences greater than 250 were determined from an able bodied control subject seen at each lab, both when examining a single trial with different control sets, and when examining different trials of the same individual collected from different labs using a single control set. These results highlight the importance of the underlying control set for establishing mean values and variance in the GGI and suggest that if GGI values are compared longitudinally or between sites these comparisons should be based on a single control dataset. # 2008 Elsevier B.V. All rights reserved. Keywords: Gillette Gait Index; Principle component analysis; Gait analysis

1. Introduction The Gillette Gait Index (GGI) is a tool used to measure pathologic gait severity and assess outcomes [1–6]. The GGI is a multivariate index combining 16 gait variables including temporal distance and pelvic, hip, knee, and ankle kinematic parameters to derive a single measure of overall gait function. This tool requires an able bodied gait data set to establish control means and variance in each of the variables. These data are used in a principle component analysis to create a set of eigen values and eigen vectors needed to calculate the GGI. Control data sets vary between labs for many reasons including differences in marker alignment techniques, software, data reduction (e.g. reconstruction and filtering), processing (e.g. identifying gait events), and natural variations between control subjects. It has been * Corresponding author. Tel.: +1 509 623 0413; fax: +1 509 623 0474. E-mail address: [email protected] (M.L. McMulkin). 0966-6362/$ – see front matter # 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.gaitpost.2008.03.002

suggested that GGI values of clinical subjects be computed from the able bodied control set collected in the same lab to reflect these inherent variations. While the GGI has been shown to be reliable within a single control dataset [3,4], the extent that GGI values may differ when using different underlying control sets is unknown. It is important to establish the magnitude of this variance if the GGI is to be used in multicenter studies, when informally comparing patients between labs, or when comparing to published values. The purpose of this study is to assess the variation in calculated GGI values when different sets of control data are used to establish the basis of the GGI. First, the GGI is calculated for the same patients with pathologic gait, but using different sets of control data as the foundations. Second, the variation is assessed by calculated GGI values of the same single able bodied subject seen at multiple sites when sets of control data from those same sites are used to establish the basis of the GGI.

484

M.L. McMulkin, B.A. MacWilliams / Gait & Posture 28 (2008) 483–487

Table 1 Summary of control data sets used for Gillette Gait Index (GGI) calculations Set #

1 2 3 4 5

Lab #

1 2 3 4 4

Year(s)

1996 1996–1997 1996 1999 2005

n

52 30 23 44 32

SCoV

Age (years) Mean (S.D.)

Range

9.5 10.0 8.9 11.5 10.9

5.1–16.9 5.0–14.1 4.2–16.8 4.2–19.1 6.2–16.4

(3.0) (3.0) (3.1) (4.2) (2.9)

9.4 8.0 13.5 10.8 4.9

SCoV is the sum of the coefficients of variations (CoVs) of each of the 16 variables which contribute to the GGI.

2. Methods 2.1. Control data This retrospective study was approved by the University of Utah Institutional Review Board. Five separate control sets were gathered from four laboratories; one laboratory contributed two control sets which were gathered six years apart (Table 1). All laboratories used similar hardware (Vicon motion capture system), marker models [7] and methodologies (Vicon processing software) to determine their data sets. One of the contributed sets of control data was developed by randomly selecting a single side from a single trial of each able bodied control subject. Additional trials and sides from this control group were not available. Therefore, for the remaining four control data sets, single sides from a single trial of each control subject were randomly selected. The sample size of control subjects varied between the five data sets. A measure of overall variance inherent in each control set was computed by summing the coefficient of variation (CoV) for the 16 GGI variables. The eigen value and eigen vector pairs of each control data set were computed and validated independently by two researchers. These calculations for each control data set allowed a GGI value to be calculated for other gait trials. 2.2. Comparison of pathological subjects Three patients with cerebral palsy from one unit (represented by control set 3) were selected to represent different levels of severity consisting of one patient in each of the Gross Motor Function Classification System (GMFCS) Levels I, II, and III. The subjects selected had both pre- and post-operative assessments. The pre-operative ages of the subjects were 5.6 (GMFCS I), 6.6 (GMFCS II), and 13.6 years (GMFCS III). The hardware, marker model and processing methods were similar to those used to establish the able bodied control databases. A single representative trial and side were selected for each pre- and post-operative session. The GGI was then calculated for the three patients with cerebral palsy with each of the five control sets. Both absolute values and changes in GGI between pre- and post-operative assessment were calculated and analyzed.

2.3. Comparison of a single able bodied subject As a second analysis, gait evaluation data was utilized from a single able bodied subject that had traveled to each of the four laboratories during a variability study [8,9]. Gait data from this subject (age 25 years) was collected during an initial visit to each lab in 1999. As a response to the intersite variability determined by this exercise, a set of standardized training videos were created and reviewed by clinicians in these labs. The same subject then revisited the labs for follow-up evaluations in 2001. For the present study, the right side from a single trial collected at each of the four laboratories contributing the control sets was evaluated for both pre- and post-training gait evaluations. The GGI of each trial (four initial and four follow-up) was computed with each of the five sets of laboratory control data. One initial trial and one follow-up trial were compared to the two sets of control data from the same lab.

3. Results Initial pre-operative GGI values calculated for the three patients with cerebral palsy using different control data sets varied widely with differences as large as 1129 in the Level III patient (Table 2). While all subjects consistently showed improvements following surgeries, magnitudes of improvement differed by as much as 800: Level III patient using control set 2 improved from a GGI score of 714 to 356 (358 improvement) while using control set 5 improved from a GGI score of 1843 to 685 (1158 improvement). Improvement was also expressed as percent change in GGI. The largest difference between control data sets was 21%: Level II patient using control set 4 improved from a GGI score of 1006 to 191, an 81% improvement, while using control set 5 improved from a GGI score of 904 to 359, a 60% improvement. Initial GGI values calculated for a single trial of the able bodied control subject seen in each lab varied with differences as great as 260 using different control data sets (Table 3). The follow-up GGI values in this case varied as much as 312. When examining different trials of the same individual collected from different labs using a single control set differences varied from 79 to 268 on the initial visit. On the follow-up visit this variability decreased, ranging from 22 to 90.

4. Discussion While the GGI is a valuable tool for assessing outcomes, care needs to be taken when comparisons are made between sites. In this study, large variations in both absolute values and changes subsequent to surgical intervention were found. These differences reflect differences in the variance of the underlying control sets. Change was qualitatively more

M.L. McMulkin, B.A. MacWilliams / Gait & Posture 28 (2008) 483–487

485

Table 2 Gillette Gait Index values calculated for three patients with Cerebral Palsy (Gross Motor Functional Classification System Levels I, II, III) when five different able bodied control data sets were used as basis Three patients

Set 1

Set 2

Set 3

Set 4

Level I initial Level I FU D Level I %D Level I

247 111 136 55

729 227 502 69

278 125 154 55

353 160 193 55

Set 5 558 160 397 71

Low 247 111 502 71

High 729 227 136 55

Max Diff 482 116 366 16

Level II initial Level II FU D Level II %D Level II

298 72 226 76

990 217 773 78

851 198 653 77

1006 191 816 81

904 359 545 60

298 72 816 81

1006 359 226 60

708 287 590 21

Level III initial Level III FU D Level III %D Level III

1182 431 751 64

714 356 358 50

1054 390 665 63

1181 349 831 70

1843 685 1158 63

714 349 1158 70

1843 685 358 50

1129 336 800 20

FU: follow-up; D: change between initial and follow up studies.

Table 3 Gillette Gait Index (GGI) values for a single able bodied subject

The GGI of one trial from each lab is computed with different sets of control data in each column; thus columns represent a consistent trial, rows represent different trials analyzed with a consistent control set. Average and maximum differences are shown for both control sets and trials. Values derived using native control sets (e.g. a trial collected in Lab 2 with GGI derived from the Lab 2 control set) are highlighted in bold. The grayed box indicates the mean of the 4 Lab  5 control set matrix. Bottom right boxes indicates the maximum difference between native data.

consistent when expressed as a percent change from the initial value. However, percent change between sites should still be used with caution. For example, it has been proposed that a 10% change in the GGI pre- to post-treatment can be considered an improvement [5,6]. Results from this study demonstrated that improvement varied by as much as 21% for the same patient pre- to post-operative. The three sample patients used here all had large improvements with surgery, however it is feasible that in patients with lesser improvements, one data set may predict (for example) a 5% change while another may predict 25% change thereby bringing into

question whether or not the surgery resulted in an actual improvement. It is unlikely that data would have been more consistent even if trials from the same patients could have been collected in each lab so that each lab’s GGI values would be based on its own clinical data. Variations greater than 250 were found on an able bodied control adult subject seen at each lab, both when examining a single trial with different control sets, and when examining different trials of the same individual collected from different labs using a single control set. Perhaps most telling, this study also determined

486

M.L. McMulkin, B.A. MacWilliams / Gait & Posture 28 (2008) 483–487

differences greater than 250 between native data, defined as the GGI computed from a trial and control set from the same lab. According to this result, intersite variations upwards of 250 in the GGI may be expected in able bodied individuals. It might have been anticipated that native data would be more consistent and demonstrate the lowest GGI values because the data collected within a lab would reflect the same variations inherent in the control set, but this was not the case. There are several sources of variability when considering GGI calculations between and within labs for subjects seen multiple times. This study illustrates the underlying control data set creates variability in the GGI. Sources of variability for gait analysis in general include between lab variability [8,9] and within lab test–retest variability [10]. Not all the variance can be attributed to intersite data collection and processing differences. The data sets 4 and 5 were collected at the same site six years apart and still show marked differences. There was a change in motion analysis hardware equipment and clinicians conducting the analysis, but the protocol and processing method/software were largely the same. No studies have quantified changes in gait analysis variability due to equipment updates or clinician turnover. It appears care must also be taken if a site updates able bodied control data. GGI values calculated from an older control data set should not be compared to an updated control data set; only GGI values using the same control data set (even within a single lab) should be used to compare pre/post treatment. This brings into question the ability to use the GGI for long term retrospective or longitudinal studies, particularly over periods where staff, hardware, model application or processing changes may have occurred. It may be possible to reduce intersite differences in the GGI values between labs by implementing standardized teaching and training in marker alignment [9] and associated data collection procedures. Subsequent to standardized training, mean GGI values increased from 97 to 159. The higher GGI values likely reflect that the variance in the control sets better represents the pre-training data; all except set 5 were collected prior to training. Improved consistency in the data does seem indicated by the maximum differences across data sets (trials from different labs tested with a single control set) which reduced from 127 initially to 46 following standardized training. Since trials apparently were collected more consistently between sites, GGI values calculated from a single control set are more consistent. The GGI values for the single able bodied subject may generally seem large. This might be explained by an age difference between the test subject (an adult) and the control data subjects (from children). It is more likely explained by the fact that reported mean normative GGI values are calculated by using the same control subjects that are used to establish the GGI [1,4]. By using the same control subjects, the average GGI of any data set is predetermined based upon the number of subjects and is equal to 16  (n 1)/n, or approximately 16. When the GGI is calculated for able bodied subjects not used in the control set to establish the GGI, values will be higher.

Control data sets 1 and 3 seem to lead to the lowest absolute GGI values followed by sets 4 and 5 and then set 2 for the single able bodied subject (Table 3). This does not mean sets 1 and 3 are superior, but may in fact indicate that these sets have larger variance in the able bodied controls. This is supported by the summed CoV of the variables that contribute to the GGI. Conversely, sets 2 and 5 have tighter variance, magnifying the GGI. Further support for this notion is that sets 4 and 5 are from the same lab, but acquired six years apart. The quality of data collection and processing are perceived to be higher in set 5 leading to lower variance overall than set 4 data, but higher average GGI values. In fact, the summed CoV decreased from data set 4 to data set 5 (Table 1). This highlights the importance of keeping control data current to reflect current laboratory practices. There are several limitations to the current study. A random sampling of three patients does not give a true range of expected differences. The large variations determined by this small sample only highlight the importance of the underlying control sets in determining the GGI. Similarly, a sampling of five data sets from four labs does not determine the true range of expected deviations between labs. Finally, the GGI is derived from a principle component analysis which determines independent variable combinations from the input variables. Examining the CoV in the input variables does not necessarily reflect the variability of the independent variable combinations. If this were the case, the control sets with the highest CoVs would always predict the lowest GGI scores for the same input data. This study determined that differences in underlying control data created large differences in computed GGI values for both pathologic and able bodied subjects. This finding has substantial indications for researchers or clinicians comparing GGI values between sites. For multi-site studies, it does not appear that each lab should use its own control data to calculate GGI values since results from this study demonstrated large differences in the index when different control sets are applied to the same patient trial. It does appear that with some standardized training, the control data from a single lab might be used in multi-site studies. Alternatively, GGI results from subjects seen at multiple sites might be established with an eigen set derived from a conglomerate of control data contributed from each site. Finally, this study suggests that to calculate an accurate estimate of gait pathology it is imperative to update control sets to reflect current laboratory practices. By extension, this also implies that for retrospective studies or longitudinal patient care that it may not be possible to directly compare GGI values over time periods in which significant changes in collection techniques have occurred.

Acknowledgments The authors wish to thank Nancy Scarborough and the Shriners Hospitals for Children, Houston lab, Roy Davis and

M.L. McMulkin, B.A. MacWilliams / Gait & Posture 28 (2008) 483–487

the Shriners Hospitals for Children, Greenville lab, and George Gorton and the Shriners Hospitals for Children, Springfield lab for their collaboration. Forms of support: Neither of the authors received financial support for this study. Conflict of interest None. References [1] Schutte LM, Narayanan U, Stout JL, Selber P, Gage JR, Schwartz MH. An index for quantifying deviations from normal gait. Gait Posture 2000;11:25–31. [2] Tervo RC, Azuma S, Stout J, Novacheck T. Correlation between physical functioning and gait measures in children with cerebral palsy. Dev Med Child Neurol 2002;44:185–90.

487

[3] Bothner KE, Fischer R, Alderink G. Assessment of reliability of the normalcy index for children with cerebral palsy.. Proc GCMAS 2003;53–54.. [4] Romei M, Galli M, Motta F, Schwartz M, Crivellini M. Use of the normalcy index for the evaluation of gait pathology. Gait Posture 2004;19:85–90. [5] Schwartz MH, Viehweger E, Stout J, Novacheck TF, Gage JR. Comprehensive treatment of ambulatory children with cerebral palsy: an outcome assessment. J Pediatr Orthop 2004;24:45–53. [6] Postans N, Roberts A, Stewart C. Outcome of multi-level surgery and selective dorsal rhizotomy assessed using the Gillette Gait Index. Gait Posture 2006;24(S2):S143–4. [7] Davis R, Ounpuu S, Tyburski D, Gage J. A gait analysis data collection and reduction technique. Hum Mov Sci 1991;10:575–87. [8] Gorton G, Hebert D, Goode B. Assessment of the kinematic variability between twelve shriners motion analysis laboratories. Gait Posture 2001;13(3):247. [9] Gorton G, Hebert D, Goode B. Assessment of the kinematic variability between twelve shriners motion analysis laboratories. Part 2. Short term follow up. Gait Posture 2002;16(S1):S65–6. [10] Schwartz MH, Trost JP, Wervey RA. Measurement and management of errors in quantitative gait data. Gait Posture 2004;20:196–203.