- Email: [email protected]
Preoperative Validation of the Patient-Reported Outcomes Measurement Information System in Patients With Articular Cartilage Defects of the Knee
Preoperative Validation of the Patient-Reported Outcomes Measurement Information System (PROMIS) in Patients With Articular Cartilage Defects of the Knee Alan G. Shamrock, M.D., Brian R. Wolf, M.D., M.S., Shannon F. Ortiz, Ph.D., Kyle R. Duchman, M.D., Matthew J. Bollier, M.D., Christopher N. Carender, M.D., and Robert W. Westermann, M.D
Purpose: To validate the Patient-Reported Outcomes Measurement Information System (PROMIS) physical function computer adaptive test (PF CAT) with current patient-reported outcome (PRO) instruments in patients with cartilage injuries of the knee. Methods: Patients scheduled for osteochondral autograft or allograft transplant, microfracture, autologous chondrocyte implantation, allograft cartilage resurfacing, and chondroplasty were prospectively enrolled in the study and completed PROMIS PF CAT, Knee Injury and Osteoarthritis Outcome Score (KOOS activities of daily living, pain, symptoms, sport, and quality of life), Short Form-36 Health Survey (SF-36 physical function [PF] and Physical Component Summary), and EuroQol-5 Dimension questionnaires. The Spearman correlation coefficient was used to compare instruments. Instrument correlations were defined as excellent (>0.7), excellent to good (0.61-0.69), good (0.4 to 0.6), and poor (<0.39), with significance defined as P < .05. Results: A total of 293 knees in 275 patients (54.5% male) undergoing 319 cartilage procedures were analyzed. The most commonly performed cartilage procedure was chondroplasty (n ¼ 118; 37.0%), followed by microfracture (n ¼ 100; 31.3%). The mean age was 34.0 14.7 and the mean body mass index was 30.0 6.9. The PROMIS PF CAT had an excellent correlation with the SF-36 PF (r ¼ 0.819; P < .001), SF-36 Physical Component Summary (r ¼ 0.766; P < .001), KOOS activities of daily living (r ¼ 0.733; P < .001), KOOS Sport (r ¼ 0.709; P < .001), and EuroQol-5 Dimension (r ¼ 0.752; P < .001) instruments; an excellent-good correlation with the KOOS pain (r ¼ 0.662; P < .001), and KOOS quality of life (r ¼ 0.640; P < .001) scores; and a good correlation with the KOOS symptoms (r ¼ 0.519; P < .001) scale. The PROMIS PF CAT had no floor or ceiling effects and the smallest question burden (mean 4.17 0.93 questions). Dimensionality analyses demonstrated that the smallest amount of unexplained variance was present in the PROMIS PF CAT (3.6%). Conclusion: The PROMIS PF CAT is an effective tool for preoperative outcome assessment in patients with cartilage defects of the knee. It correlates strongly with legacy PRO measures of physical function with no ceiling and floor effects and a minimal time burden for completion. Further study is warranted to determine postoperative performance and to evaluate the responsiveness of PROMIS to change within a patient. Level of Evidence: III; Prognostic retrospective comparative study.
I
solated cartilage defects of the knee are a common source of pain and pose a challenge for treating
From the Department of Orthopaedics and Rehabilitation, University of Iowa Hospitals and Clinics, Iowa City, IA. The authors report that they have no conflicts of interest in the authorship and publication of this article. Full ICMJE author disclosure forms are available for this article online, as supplementary material. Received February 23, 2019; accepted August 14, 2019. Address correspondence to Alan Shamrock M.D., Department of Orthopaedics and Rehabilitation, University of Iowa Hospitals and Clinics, 200 Hawkins Dr, Iowa City, IA 52242. E-mail: [email protected] Ó 2019 by the Arthroscopy Association of North America 0749-8063/19215/$36.00 https://doi.org/10.1016/j.arthro.2019.08.043
orthopaedic surgeons given the limited ability of articular cartilage to heal.1,2 Clinical management decisions for cartilage defects are multifactorial and often depend on the size, location, and grade of the lesion, as well as the age and activity level of the patient.3 Cartilage restoration is a current area of intensive basic science and clinic research, with patient-reported outcomes (PROs) frequently used to assess preoperative disability and postoperative improvement. Although clinically useful, the most frequently used legacy general health, disease-specific, and anatomic site-specific questionnaires are burdensome to administer and complete, with patients expected to read and answer a large number of questions.
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol
-,
No
-
(Month), 2019: pp 1-5
1
2
A. G. SHAMROCK ET AL.
In 2004, the National Institutes of Health created the Patient-Reported Outcomes Measurement Information System (PROMIS), which consists of a large database of questions in many health domains administered using item-response theory.4 This provides a dynamic testing environment where a patient’s answer to a selected question is scored, and the next-most informative question is then given.5 In theory, this method of testing provides a patient-specific assessment with fewer questions without sacrificing precision. Furthermore, PROMIS Physical Function (PF) tools can be given in Computer Adaptive Test (CAT) form consisting of a 121-question (PROMIS PF CAT v1.2) or 165question (PROMIS PF CAT v2.0) dataset from which 4-12 questions are selected.4,6 The PROMIS PF CAT is appealing because multiple studies have demonstrated that a reduction of question burden and electronic administration improves PRO completion rates.6-10 Although PROMIS PF CAT has been validated as an efficient outcome measurement tool in orthopaedic procedures such as meniscus surgery, anterior cruciate ligament (ACL) reconstruction, shoulder arthroplasty, and rotator cuff repair, it has not been validated in patients undergoing surgery to address articular cartilage defects of the knee.8,11-16 The purpose of this study was to validate the PROMIS PF CAT with current PRO instruments in patients with cartilage injuries of the knee. We hypothesized that the PROMIS PF CAT (1) would have an excellent correlation with PRO instruments used to measure physical function such as Short Form-36 physical function (SF-36 PF), Knee injury and Osteoarthritis Outcomes Score (KOOS) sport, and KOOS activities of daily living (ADL), (2) would have fewer ceiling or floor effects, and (3) would require fewer questions to complete compared with other commonly used PRO tools.
Methods This study was approved by the participating institution’s Institutional Review Board. Patients older than 10 years of age who were scheduled for osteochondral autograft or allograft transfer, microfracture, chondroplasty, autologous chondrocyte implantation, and allograft cartilage resurfacing (BioCartilage; Arthrex, Naples, FL) from January 2015 to June 2018 were prospectively enrolled in the study. Written English-language proficiency was required for inclusion in the study. Subjects undergoing concomitant ACL reconstruction, collateral ligament repair/reconstruction, meniscal repair, meniscal transplant, high tibial osteotomy, distal femoral osteotomy, or bilateral procedures under the same general anesthetic were excluded from analysis. All patients underwent surgery by 1 of 5 sports medicine fellowshipetrained orthopaedic surgeons. Patients were included if multiple cartilage procedures were performed in the same knee
if each specific procedure was recorded. Enrolled patients completed PROMIS PF CAT v2.0, KOOS (KOOS ADL, pain, symptoms, sport, and quality of life [QOL]), SF-36 (SF-36 PF and Physical Component Summary [PCS]), and EuroQol-5 Dimension (EQ-5D) questionnaires using a computer at their preoperative clinic visits. The order of questionnaire application was randomized to control for response fatigue. Additional demographic data included age, sex, and body mass index. Procedural information extracted included intraoperative diagnoses and any concomitant procedures performed. Construct validity was determined by examining the correlation of PROMIS PF CAT with other PRO measurement tools. The Spearman or Pearson correlation coefficient was used to compare instruments. Instrument correlations were defined as excellent (>0.7), excellent to good (0.61-0.69), good (0.4-0.6), and poor (<0.39), with significance set to P < .05.15 The mean number of answers required to complete each questionnaire was recorded. Ceiling and floor effects were measured by determining the percentage of patients that achieved the highest and lowest possible scores. They represent an inability of the instrument to measure individuals whose scores exist outside the instrument’s measurement range. If >15% of subjects achieved the highest or lowest possible scores on a PRO tool, ceiling or floor effects were considered to be present. PRO dimensionality was measured to quantify the extent to which an instrument represented a specific health domain such as physical function. Statistical analysis was performed using SAS software (SAS version 9.4; SAS Institute, Cary, NC), with statistical significance defined as P < .05.
Results A total of 293 knees in 275 patients (54.5% male) undergoing 319 cartilage procedures were analyzed (Table 1). Chondroplasty was performed in 118 patients (37.0%), microfracture was performed in 100 patients (31.3%), cartilage resurfacing in 56 patients (17.6%), osteochondral transplant in 44 (22 autograft and 22 allograft) patients (13.8%), and 1 patient (0.3%) underwent autologous chondrocyte implantation (Fig 1). The most common concomitant procedure performed was partial meniscectomy/ debridement (n ¼ 106), followed by partial synovectomy (n ¼ 63), loose body removal (n ¼ 39), Table 1. Demographic Information of the Study cohort Patients Knees Male, n (%) Age (y), Mean SD (range) BMI, Mean SD (range)
275 293 150 (54.5%) 34.0 14.7 (12-70) 30.0 6.9 (17.8-57.1)
BMI, body mass index; SD, standard deviation.
PROMIS VALIDATION IN KNEE CARTILAGE INJURY
3
Fig 1. Pie graph detailing articular cartilage procedures performed in the study cohort.
Fulkerson osteotomy (n ¼ 25), and medial patellofemoral ligament reconstruction (n ¼ 22). The PROMIS PF CAT had an excellent correlation with the SF-36 PF (r ¼ 0.819, P < .001), SF-36 PCS (r ¼ 0.766, P < .001), KOOS ADL (r ¼ 0.733, P < .001), KOOS Sport (r ¼ 0.709, P < .001), and EQ-5D (r ¼ 0.752, P < .001) instruments; an excellent to good correlation with the KOOS pain (r ¼ 0.662, P < .001), and KOOS QOL (r ¼ 0.640, P < .001) scores; and a good correlation with the KOOS symptoms (r ¼ 0.519, P < .001) scale. The PROMIS PF CAT had no floor or ceiling effects with no patients achieving either the highest or lowest possible score (Table 2). The average number of questions answered was fewest in the PROMIS PF CAT (4.17 0.93 questions). Dimensionality analyses demonstrated that the smallest amount of unexplained variance was present in the PROMIS PF CAT (3.6%), suggesting unidimensionality in quantifying physical function. The largest amount of unexplained variance was seen in the KOOS Sport subscale (16.0%). The SF-36 PF and KOOS ADL exhibited variances of 5.0% and 6.0%, respectively.
Discussion The results of this study provide evidence of construct validity of the PROMIS PF CAT instrument in the preoperative evaluation of patients undergoing cartilage restoration procedures of the knee. Specifically, excellent correlation was noted between PROMIS PF CAT and PRO instruments designed to measure knee pathology, particularly physical function. Additionally, no floor or ceiling effects were observed with a decreased question burden (mean 4.17 0.93 questions) required for completion. These findings support the use of
PROMIS PF CAT in this patient population as a valid outcome instrument, given the importance for PROs to efficiently and effectively measure patient-perceived disability. The PROMIS PF CAT instrument was found to have an excellent correlation with the more traditional PROs designed to assess physical function such as SF-36 PF, KOOS ADL, and KOOS sport. Additionally, an excellent correlation was noted with the SF-36 PCS scale, whereas questionnaires designed to assess quality of life or general well-being exhibited weaker relationships with PROMIS PF CAT. This trend of excellent correlation relationships between PROMIS PF CAT and PROs with physical function domains has been previously demonstrated. In a cohort of patients undergoing ACL reconstruction, Scott et al.14 found an excellent correlation between preoperative PROMIS PF CAT scores and SF-36, KOOS ADL, and KOOS sport instruments. Similar to the findings in this study, general-health PROs demonstrated a weaker association. Hancock et al.8,16 validated PROMIS PF CAT for preoperative evaluation in healthy patients undergoing both meniscal surgery and ACL reconstruction. The authors further demonstrated excellent correlation between PROMIS PF CAT and physical functionespecific PROs and concluded that PROMIS PF CAT maintained construct validity with a decreased question burden. To complete the PROMIS PF CAT survey, patients indicated for cartilage restoration procedures in the current study answered an average of 4.17 0.93 questions. In contrast, patients answered 36 questions for SF-36 (including 12 PF questions), 42 questions for KOOS (including 4 QOL, 5 sport, and 17 ADL questions), and 6 questions for EQ-5D questionnaires. This
4
A. G. SHAMROCK ET AL.
Table 2. Floor and ceiling Effects of PRO Measures specifically Measuring the Physical Function Domain PROMIS PF CAT SF-36 PF KOOS ADL KOOS sport
Ceiling Effects 0 (0.0%) 12 (4.11%) 0 (0.0%) 34 (11.64%)
Floor Effects 0 (0.0%) 4 (1.37%) 7 (2.40%) 2 (0.68%)
Values are provided as the number and percentage of patient with the highest (ceiling effect) and lowest (floor effect) scores.
number of answers required for questionnaire completion is an accurate surrogate for this metric, but certainly more complex or longer questions may take more time. Last, patients in this study were only examined before surgery. The performance of PROMIS PF CAT after cartilage restoration of the knee remains unknown. Further study is warranted to determine postoperative performance and to evaluate the instrument’s responsiveness to change within a patient.
Conclusions decreased question burden is an attractive feature of PROMIS PF CAT because multiple studies have demonstrated an association between fewer questions and increased PRO completion rates.6-10 This study is in agreement with previous studies that have validated PROMIS questionnaires in various orthopaedic procedures, with decreased question burden being a consistent finding.8,11-16 Previous studies have shown that ceiling effects using PROMIS instruments may be present in populations of young and healthy patients undergoing elective surgery.17 We found no ceiling or floor effects with the PROMIS PF CAT and no ceiling effects with the KOOS ADL instruments. In contrast, the SF-36 PF scale had 12 ceiling and 4 floor effects, the KOOS sport had 34 ceiling and 2 floor effects, and the KOOS ADL had 7 floor effects. No PRO instrument had >15% floor or ceiling effects which was required to state the presence of either measure. Previous studies have also demonstrated zero or minimal floor and ceiling effects with the PROMIS PF CAT.6,8,14,16 Additionally, this study contradicts the suspicion of ceiling effects in a healthy population17 and confirms the findings in more recent studies that further dispel the concern for ceiling effects in this population.6,8 Our findings suggest that the PROMIS PF CAT is a precise measure of knee physical function without ceiling effects in a population of otherwise healthy patients undergoing knee cartilage restoration procedures. Limitations There are several limitations of this study, including those exhibited in other PRO measurement tool studies, specifically poor patient understanding and language barriers. We did not note any English-language barriers; however, unfamiliarity with computer use may prolong testing and affect scores in some populations. Additionally, the small sample size may not be representative of the entire population undergoing knee cartilage procedures. Also, we did not exclude all concomitant procedures, including partial meniscectomy and medial patellofemoral ligament reconstruction, which may be a source of confounding. Furthermore, the time necessary for survey completion was not recorded. The authors assume that the mean
The PROMIS PF CAT is an effective tool for preoperative outcome assessment in patients with isolated cartilage defects of the knee. It correlates strongly with currently accepted PRO measures of physical function with no ceiling and floor effects and a minimal time burden for completion.
References 1. Curl WW, Krome J, Gordon ES, Rushing J, Smith BP, Poehling GG. Cartilage injuries: A review of 31,516 knee arthroscopies. Arthroscopy 1997;13:456-460. 2. Montgomery SR, Foster BD, Ngo SS, et al. Trends in the surgical treatment of articular cartilage defects of the knee in the United States. Knee Surg Sports Traumatol Arthrosc 2014;22:2070-2075. 3. Hancock KJ, Westermann RR, Shamrock AG, Duchman KR, Wolf BR, Amendola A. Trends in knee articular cartilage treatments: An American Board of Orthopaedic Surgery Database Study. J Knee Surg 2019;32: 85-90. 4. Cella D, Gershon R, Lai JS, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res 2007;16: 133-141 (Suppl 1). 5. Fries JF, Bruce B, Cella D. The promise of PROMIS: Using item response theory to improve assessment of patientreported outcomes. Clin Exp Rheumatol 2005;23:S53-S57 (Suppl 39). 6. Hung M1, Stuart AR, Higgins TF, Saltzman CL, Kubiak EN. Computerized adaptive testing using the PROMIS Physical Function Item Bank reduces test burden with less ceiling effects compared with the short Musculoskeletal Function Assessment in orthopaedic trauma patients. J Orthop Trauma 2014;28:439-443. 7. Döring AC, Nota SP, Hageman MG, Ring DC. Measurement of upper extremity disability using the PatientReported Outcomes Measurement Information System. J Hand Surg Am 2014;39:1160-1165. 8. Hancock KJ, Glass N, Anthony CA, et al. Performance of PROMIS for healthy patients undergoing meniscal surgery. J Bone Joint Surg Am 2017;99:954-958. 9. Jayakumar P, Overbeek C, Vranceanu AM, et al. The use of computer adaptive tests in outcome assessments following upper limb trauma. Bone Joint J 2018;100-B: 693-702. 10. Schamber EM1, Takemoto SK, Chenok KE, Bozic KJ. Barriers to completion of patient reported outcome measures. J Arthroplasty 2013;28:1449-1453.
PROMIS VALIDATION IN KNEE CARTILAGE INJURY 11. Beckmann JT, Hung M, Bounsanga J, Wylie JD, Granger EK, Tashjian RZ. Psychometric evaluation of the PROMIS Physical Function Computerized Adaptive Test in comparison to the American Shoulder and Elbow Surgeons score and Simple Shoulder Test in patients with rotator cuff disease. J Shoulder Elbow Surg 2015;24: 1961-1967. 12. Dowdle SB, Glass N, Anthony CA, Hettrich CM. Use of PROMIS for patients undergoing primary total shoulder arthroplasty. Orthop J Sports Med 2017;5:1-4. 13. Patterson BM, Orvets ND, Aleem AW, et al. Correlation of Patient-Reported Outcomes Measurement Information System (PROMIS) scores with legacy patient-reported outcome scores in patients undergoing rotator cuff repair. J Shoulder Elbow Surg 2018;27: S17-S23.
5
14. Scott EJ, Westermann R, Glass NA, Hettrich C, Wolf BR, Bollier MJ. Performance of the PROMIS in Patients After Anterior Cruciate Ligament Reconstruction. Orthop J Sports Med 2018;6:1-7. 15. Anthony CA, Glass N, Hancock K, Bollier M, Hettrich CM, Wolf BR. Preoperative performance of the patientreported outcomes measurement information system in patients with rotator cuff pathology. Arthroscopy 2017;33: 1770-1774 e1. 16. Hancock KJ, Glass N, Anthony CA, et al. PROMIS: A valid and efficient outcomes instrument for patients with ACL tears. Knee Surg Sports Traumatol Arthrosc 2019;27: 100-104. 17. Fries J, Rose M, Krishnan E. The PROMIS of better outcome assessment: Responsiveness, floor and ceiling effects, and Internet administration. J Rheumatol 2011;38:1759-1764.
Purpose: To validate the Patient-Reported Outcomes Measurement Information System (PROMIS) physical function computer adaptive test (PF CAT) with current patient-reported outcome (PRO) instruments in patients with cartilage injuries of the knee. Methods: Patients scheduled for osteochondral autograft or allograft transplant, microfracture, autologous chondrocyte implantation, allograft cartilage resurfacing, and chondroplasty were prospectively enrolled in the study and completed PROMIS PF CAT, Knee Injury and Osteoarthritis Outcome Score (KOOS activities of daily living, pain, symptoms, sport, and quality of life), Short Form-36 Health Survey (SF-36 physical function [PF] and Physical Component Summary), and EuroQol-5 Dimension questionnaires. The Spearman correlation coefficient was used to compare instruments. Instrument correlations were defined as excellent (>0.7), excellent to good (0.61-0.69), good (0.4 to 0.6), and poor (<0.39), with significance defined as P < .05. Results: A total of 293 knees in 275 patients (54.5% male) undergoing 319 cartilage procedures were analyzed. The most commonly performed cartilage procedure was chondroplasty (n ¼ 118; 37.0%), followed by microfracture (n ¼ 100; 31.3%). The mean age was 34.0 14.7 and the mean body mass index was 30.0 6.9. The PROMIS PF CAT had an excellent correlation with the SF-36 PF (r ¼ 0.819; P < .001), SF-36 Physical Component Summary (r ¼ 0.766; P < .001), KOOS activities of daily living (r ¼ 0.733; P < .001), KOOS Sport (r ¼ 0.709; P < .001), and EuroQol-5 Dimension (r ¼ 0.752; P < .001) instruments; an excellent-good correlation with the KOOS pain (r ¼ 0.662; P < .001), and KOOS quality of life (r ¼ 0.640; P < .001) scores; and a good correlation with the KOOS symptoms (r ¼ 0.519; P < .001) scale. The PROMIS PF CAT had no floor or ceiling effects and the smallest question burden (mean 4.17 0.93 questions). Dimensionality analyses demonstrated that the smallest amount of unexplained variance was present in the PROMIS PF CAT (3.6%). Conclusion: The PROMIS PF CAT is an effective tool for preoperative outcome assessment in patients with cartilage defects of the knee. It correlates strongly with legacy PRO measures of physical function with no ceiling and floor effects and a minimal time burden for completion. Further study is warranted to determine postoperative performance and to evaluate the responsiveness of PROMIS to change within a patient. Level of Evidence: III; Prognostic retrospective comparative study.
I
solated cartilage defects of the knee are a common source of pain and pose a challenge for treating
From the Department of Orthopaedics and Rehabilitation, University of Iowa Hospitals and Clinics, Iowa City, IA. The authors report that they have no conflicts of interest in the authorship and publication of this article. Full ICMJE author disclosure forms are available for this article online, as supplementary material. Received February 23, 2019; accepted August 14, 2019. Address correspondence to Alan Shamrock M.D., Department of Orthopaedics and Rehabilitation, University of Iowa Hospitals and Clinics, 200 Hawkins Dr, Iowa City, IA 52242. E-mail: [email protected] Ó 2019 by the Arthroscopy Association of North America 0749-8063/19215/$36.00 https://doi.org/10.1016/j.arthro.2019.08.043
orthopaedic surgeons given the limited ability of articular cartilage to heal.1,2 Clinical management decisions for cartilage defects are multifactorial and often depend on the size, location, and grade of the lesion, as well as the age and activity level of the patient.3 Cartilage restoration is a current area of intensive basic science and clinic research, with patient-reported outcomes (PROs) frequently used to assess preoperative disability and postoperative improvement. Although clinically useful, the most frequently used legacy general health, disease-specific, and anatomic site-specific questionnaires are burdensome to administer and complete, with patients expected to read and answer a large number of questions.
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol
-,
No
-
(Month), 2019: pp 1-5
1
2
A. G. SHAMROCK ET AL.
In 2004, the National Institutes of Health created the Patient-Reported Outcomes Measurement Information System (PROMIS), which consists of a large database of questions in many health domains administered using item-response theory.4 This provides a dynamic testing environment where a patient’s answer to a selected question is scored, and the next-most informative question is then given.5 In theory, this method of testing provides a patient-specific assessment with fewer questions without sacrificing precision. Furthermore, PROMIS Physical Function (PF) tools can be given in Computer Adaptive Test (CAT) form consisting of a 121-question (PROMIS PF CAT v1.2) or 165question (PROMIS PF CAT v2.0) dataset from which 4-12 questions are selected.4,6 The PROMIS PF CAT is appealing because multiple studies have demonstrated that a reduction of question burden and electronic administration improves PRO completion rates.6-10 Although PROMIS PF CAT has been validated as an efficient outcome measurement tool in orthopaedic procedures such as meniscus surgery, anterior cruciate ligament (ACL) reconstruction, shoulder arthroplasty, and rotator cuff repair, it has not been validated in patients undergoing surgery to address articular cartilage defects of the knee.8,11-16 The purpose of this study was to validate the PROMIS PF CAT with current PRO instruments in patients with cartilage injuries of the knee. We hypothesized that the PROMIS PF CAT (1) would have an excellent correlation with PRO instruments used to measure physical function such as Short Form-36 physical function (SF-36 PF), Knee injury and Osteoarthritis Outcomes Score (KOOS) sport, and KOOS activities of daily living (ADL), (2) would have fewer ceiling or floor effects, and (3) would require fewer questions to complete compared with other commonly used PRO tools.
Methods This study was approved by the participating institution’s Institutional Review Board. Patients older than 10 years of age who were scheduled for osteochondral autograft or allograft transfer, microfracture, chondroplasty, autologous chondrocyte implantation, and allograft cartilage resurfacing (BioCartilage; Arthrex, Naples, FL) from January 2015 to June 2018 were prospectively enrolled in the study. Written English-language proficiency was required for inclusion in the study. Subjects undergoing concomitant ACL reconstruction, collateral ligament repair/reconstruction, meniscal repair, meniscal transplant, high tibial osteotomy, distal femoral osteotomy, or bilateral procedures under the same general anesthetic were excluded from analysis. All patients underwent surgery by 1 of 5 sports medicine fellowshipetrained orthopaedic surgeons. Patients were included if multiple cartilage procedures were performed in the same knee
if each specific procedure was recorded. Enrolled patients completed PROMIS PF CAT v2.0, KOOS (KOOS ADL, pain, symptoms, sport, and quality of life [QOL]), SF-36 (SF-36 PF and Physical Component Summary [PCS]), and EuroQol-5 Dimension (EQ-5D) questionnaires using a computer at their preoperative clinic visits. The order of questionnaire application was randomized to control for response fatigue. Additional demographic data included age, sex, and body mass index. Procedural information extracted included intraoperative diagnoses and any concomitant procedures performed. Construct validity was determined by examining the correlation of PROMIS PF CAT with other PRO measurement tools. The Spearman or Pearson correlation coefficient was used to compare instruments. Instrument correlations were defined as excellent (>0.7), excellent to good (0.61-0.69), good (0.4-0.6), and poor (<0.39), with significance set to P < .05.15 The mean number of answers required to complete each questionnaire was recorded. Ceiling and floor effects were measured by determining the percentage of patients that achieved the highest and lowest possible scores. They represent an inability of the instrument to measure individuals whose scores exist outside the instrument’s measurement range. If >15% of subjects achieved the highest or lowest possible scores on a PRO tool, ceiling or floor effects were considered to be present. PRO dimensionality was measured to quantify the extent to which an instrument represented a specific health domain such as physical function. Statistical analysis was performed using SAS software (SAS version 9.4; SAS Institute, Cary, NC), with statistical significance defined as P < .05.
Results A total of 293 knees in 275 patients (54.5% male) undergoing 319 cartilage procedures were analyzed (Table 1). Chondroplasty was performed in 118 patients (37.0%), microfracture was performed in 100 patients (31.3%), cartilage resurfacing in 56 patients (17.6%), osteochondral transplant in 44 (22 autograft and 22 allograft) patients (13.8%), and 1 patient (0.3%) underwent autologous chondrocyte implantation (Fig 1). The most common concomitant procedure performed was partial meniscectomy/ debridement (n ¼ 106), followed by partial synovectomy (n ¼ 63), loose body removal (n ¼ 39), Table 1. Demographic Information of the Study cohort Patients Knees Male, n (%) Age (y), Mean SD (range) BMI, Mean SD (range)
275 293 150 (54.5%) 34.0 14.7 (12-70) 30.0 6.9 (17.8-57.1)
BMI, body mass index; SD, standard deviation.
PROMIS VALIDATION IN KNEE CARTILAGE INJURY
3
Fig 1. Pie graph detailing articular cartilage procedures performed in the study cohort.
Fulkerson osteotomy (n ¼ 25), and medial patellofemoral ligament reconstruction (n ¼ 22). The PROMIS PF CAT had an excellent correlation with the SF-36 PF (r ¼ 0.819, P < .001), SF-36 PCS (r ¼ 0.766, P < .001), KOOS ADL (r ¼ 0.733, P < .001), KOOS Sport (r ¼ 0.709, P < .001), and EQ-5D (r ¼ 0.752, P < .001) instruments; an excellent to good correlation with the KOOS pain (r ¼ 0.662, P < .001), and KOOS QOL (r ¼ 0.640, P < .001) scores; and a good correlation with the KOOS symptoms (r ¼ 0.519, P < .001) scale. The PROMIS PF CAT had no floor or ceiling effects with no patients achieving either the highest or lowest possible score (Table 2). The average number of questions answered was fewest in the PROMIS PF CAT (4.17 0.93 questions). Dimensionality analyses demonstrated that the smallest amount of unexplained variance was present in the PROMIS PF CAT (3.6%), suggesting unidimensionality in quantifying physical function. The largest amount of unexplained variance was seen in the KOOS Sport subscale (16.0%). The SF-36 PF and KOOS ADL exhibited variances of 5.0% and 6.0%, respectively.
Discussion The results of this study provide evidence of construct validity of the PROMIS PF CAT instrument in the preoperative evaluation of patients undergoing cartilage restoration procedures of the knee. Specifically, excellent correlation was noted between PROMIS PF CAT and PRO instruments designed to measure knee pathology, particularly physical function. Additionally, no floor or ceiling effects were observed with a decreased question burden (mean 4.17 0.93 questions) required for completion. These findings support the use of
PROMIS PF CAT in this patient population as a valid outcome instrument, given the importance for PROs to efficiently and effectively measure patient-perceived disability. The PROMIS PF CAT instrument was found to have an excellent correlation with the more traditional PROs designed to assess physical function such as SF-36 PF, KOOS ADL, and KOOS sport. Additionally, an excellent correlation was noted with the SF-36 PCS scale, whereas questionnaires designed to assess quality of life or general well-being exhibited weaker relationships with PROMIS PF CAT. This trend of excellent correlation relationships between PROMIS PF CAT and PROs with physical function domains has been previously demonstrated. In a cohort of patients undergoing ACL reconstruction, Scott et al.14 found an excellent correlation between preoperative PROMIS PF CAT scores and SF-36, KOOS ADL, and KOOS sport instruments. Similar to the findings in this study, general-health PROs demonstrated a weaker association. Hancock et al.8,16 validated PROMIS PF CAT for preoperative evaluation in healthy patients undergoing both meniscal surgery and ACL reconstruction. The authors further demonstrated excellent correlation between PROMIS PF CAT and physical functionespecific PROs and concluded that PROMIS PF CAT maintained construct validity with a decreased question burden. To complete the PROMIS PF CAT survey, patients indicated for cartilage restoration procedures in the current study answered an average of 4.17 0.93 questions. In contrast, patients answered 36 questions for SF-36 (including 12 PF questions), 42 questions for KOOS (including 4 QOL, 5 sport, and 17 ADL questions), and 6 questions for EQ-5D questionnaires. This
4
A. G. SHAMROCK ET AL.
Table 2. Floor and ceiling Effects of PRO Measures specifically Measuring the Physical Function Domain PROMIS PF CAT SF-36 PF KOOS ADL KOOS sport
Ceiling Effects 0 (0.0%) 12 (4.11%) 0 (0.0%) 34 (11.64%)
Floor Effects 0 (0.0%) 4 (1.37%) 7 (2.40%) 2 (0.68%)
Values are provided as the number and percentage of patient with the highest (ceiling effect) and lowest (floor effect) scores.
number of answers required for questionnaire completion is an accurate surrogate for this metric, but certainly more complex or longer questions may take more time. Last, patients in this study were only examined before surgery. The performance of PROMIS PF CAT after cartilage restoration of the knee remains unknown. Further study is warranted to determine postoperative performance and to evaluate the instrument’s responsiveness to change within a patient.
Conclusions decreased question burden is an attractive feature of PROMIS PF CAT because multiple studies have demonstrated an association between fewer questions and increased PRO completion rates.6-10 This study is in agreement with previous studies that have validated PROMIS questionnaires in various orthopaedic procedures, with decreased question burden being a consistent finding.8,11-16 Previous studies have shown that ceiling effects using PROMIS instruments may be present in populations of young and healthy patients undergoing elective surgery.17 We found no ceiling or floor effects with the PROMIS PF CAT and no ceiling effects with the KOOS ADL instruments. In contrast, the SF-36 PF scale had 12 ceiling and 4 floor effects, the KOOS sport had 34 ceiling and 2 floor effects, and the KOOS ADL had 7 floor effects. No PRO instrument had >15% floor or ceiling effects which was required to state the presence of either measure. Previous studies have also demonstrated zero or minimal floor and ceiling effects with the PROMIS PF CAT.6,8,14,16 Additionally, this study contradicts the suspicion of ceiling effects in a healthy population17 and confirms the findings in more recent studies that further dispel the concern for ceiling effects in this population.6,8 Our findings suggest that the PROMIS PF CAT is a precise measure of knee physical function without ceiling effects in a population of otherwise healthy patients undergoing knee cartilage restoration procedures. Limitations There are several limitations of this study, including those exhibited in other PRO measurement tool studies, specifically poor patient understanding and language barriers. We did not note any English-language barriers; however, unfamiliarity with computer use may prolong testing and affect scores in some populations. Additionally, the small sample size may not be representative of the entire population undergoing knee cartilage procedures. Also, we did not exclude all concomitant procedures, including partial meniscectomy and medial patellofemoral ligament reconstruction, which may be a source of confounding. Furthermore, the time necessary for survey completion was not recorded. The authors assume that the mean
The PROMIS PF CAT is an effective tool for preoperative outcome assessment in patients with isolated cartilage defects of the knee. It correlates strongly with currently accepted PRO measures of physical function with no ceiling and floor effects and a minimal time burden for completion.
References 1. Curl WW, Krome J, Gordon ES, Rushing J, Smith BP, Poehling GG. Cartilage injuries: A review of 31,516 knee arthroscopies. Arthroscopy 1997;13:456-460. 2. Montgomery SR, Foster BD, Ngo SS, et al. Trends in the surgical treatment of articular cartilage defects of the knee in the United States. Knee Surg Sports Traumatol Arthrosc 2014;22:2070-2075. 3. Hancock KJ, Westermann RR, Shamrock AG, Duchman KR, Wolf BR, Amendola A. Trends in knee articular cartilage treatments: An American Board of Orthopaedic Surgery Database Study. J Knee Surg 2019;32: 85-90. 4. Cella D, Gershon R, Lai JS, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res 2007;16: 133-141 (Suppl 1). 5. Fries JF, Bruce B, Cella D. The promise of PROMIS: Using item response theory to improve assessment of patientreported outcomes. Clin Exp Rheumatol 2005;23:S53-S57 (Suppl 39). 6. Hung M1, Stuart AR, Higgins TF, Saltzman CL, Kubiak EN. Computerized adaptive testing using the PROMIS Physical Function Item Bank reduces test burden with less ceiling effects compared with the short Musculoskeletal Function Assessment in orthopaedic trauma patients. J Orthop Trauma 2014;28:439-443. 7. Döring AC, Nota SP, Hageman MG, Ring DC. Measurement of upper extremity disability using the PatientReported Outcomes Measurement Information System. J Hand Surg Am 2014;39:1160-1165. 8. Hancock KJ, Glass N, Anthony CA, et al. Performance of PROMIS for healthy patients undergoing meniscal surgery. J Bone Joint Surg Am 2017;99:954-958. 9. Jayakumar P, Overbeek C, Vranceanu AM, et al. The use of computer adaptive tests in outcome assessments following upper limb trauma. Bone Joint J 2018;100-B: 693-702. 10. Schamber EM1, Takemoto SK, Chenok KE, Bozic KJ. Barriers to completion of patient reported outcome measures. J Arthroplasty 2013;28:1449-1453.
PROMIS VALIDATION IN KNEE CARTILAGE INJURY 11. Beckmann JT, Hung M, Bounsanga J, Wylie JD, Granger EK, Tashjian RZ. Psychometric evaluation of the PROMIS Physical Function Computerized Adaptive Test in comparison to the American Shoulder and Elbow Surgeons score and Simple Shoulder Test in patients with rotator cuff disease. J Shoulder Elbow Surg 2015;24: 1961-1967. 12. Dowdle SB, Glass N, Anthony CA, Hettrich CM. Use of PROMIS for patients undergoing primary total shoulder arthroplasty. Orthop J Sports Med 2017;5:1-4. 13. Patterson BM, Orvets ND, Aleem AW, et al. Correlation of Patient-Reported Outcomes Measurement Information System (PROMIS) scores with legacy patient-reported outcome scores in patients undergoing rotator cuff repair. J Shoulder Elbow Surg 2018;27: S17-S23.
5
14. Scott EJ, Westermann R, Glass NA, Hettrich C, Wolf BR, Bollier MJ. Performance of the PROMIS in Patients After Anterior Cruciate Ligament Reconstruction. Orthop J Sports Med 2018;6:1-7. 15. Anthony CA, Glass N, Hancock K, Bollier M, Hettrich CM, Wolf BR. Preoperative performance of the patientreported outcomes measurement information system in patients with rotator cuff pathology. Arthroscopy 2017;33: 1770-1774 e1. 16. Hancock KJ, Glass N, Anthony CA, et al. PROMIS: A valid and efficient outcomes instrument for patients with ACL tears. Knee Surg Sports Traumatol Arthrosc 2019;27: 100-104. 17. Fries J, Rose M, Krishnan E. The PROMIS of better outcome assessment: Responsiveness, floor and ceiling effects, and Internet administration. J Rheumatol 2011;38:1759-1764.