- Email: [email protected]
Measuring knee arthroplasty outcomes
KNEE ARTHROPLASTY
Measuring knee arthroplasty outcomes
to a treatment or control group and are followed prospectively. Level 2 evidence consists of prospective comparative cohort studies where patients are not randomized. Meta-analysis of Level 2 studies also falls into this category. Level 3 evidence is represented by retrospective cohort studies, case control studies (presence or absence of disease or treatment) and meta-analysis of Level 3 studies. Case series are Level 4 and case reports Level 5 evidence. Level 5 evidence also includes expert opinions and personal observations. A good study design starts with a hypothesis that investigates an anticipated association between variables (or no association, as in a null hypothesis).1 Next, a representative study population needs to be chosen to answer the research question. Inclusion and exclusion criteria determine the characteristics of the population, and an appropriate sample size needs to be calculated to be able to reach statistical significance in outcomes. A blinded (concealed from the investigator) random allocation procedure should be used to create the different groups. Blinding when applying the intervention and measuring the outcome is essential to avoid bias. There are several outcome scores for knee arthritis, including the Oxford Knee Score (OKS), Knee Society Score (KSS), Knee Injury and Osteoarthritis Outcome Score (KOOS), Western Ontario and McMaster Universities Arthritis Index (WOMAC) and International Knee Documentation Committee (IKDC). Outcome measures need to be chosen carefully and when a scoring system is used, validation of the scoring system is essential. Finally, statistical analysis can prove or disprove the hypothesis. Other considerations are the ethical implications (Ethics Committee study approval), the need for informed consent for the study population and last but not least, patient safety.
Paul Nicolai
Abstract Outcome measures are increasingly used by clinicians, healthcare providers and implant manufacturers for a variety of reasons. Clinical outcome studies have traditionally been used but the level of evidence presented has been variable. In recent decades joint registries have provided useful data on implant survival, but implant revision as an outcome measure has restrictions as many factors contribute to it. Patient reported outcome scores have been introduced more recently and provide important information about health gain after surgical intervention. In response to guidelines from the National Institute for Clinical Excellence, the Orthopaedic Data Evaluation Panel assesses and rates implants, and Beyond Compliance advises implant manufacturers of the level of risk of newly introduced or modified prostheses.
Keywords Beyond Compliance; clinical studies; National Joint Registry; Orthopaedic Data Evaluation Panel; outcome measures; patient reported outcome scores
Introduction The analysis of results has become an essential part of orthopaedic practice. Outcome scores can be used to assess the effect of a disease or condition, the effect of disease treatment and its cost effectiveness of treatments. They can also be used for research purposes, teaching and peer comparison. Outcome analysis allows surgeons to make informed choices of the use of prostheses and surgical techniques based on evidence. There is increasing pressure on hospital trusts to assess their performance data and in the future clinical commissioning may depend on outcome scores for implants and treatments. Surgeons’ mortality rates are openly published and available to see by the general population. Transparency is increasingly important. There are a variety of scoring instruments available, all with advantages, limitations and varying degrees of evidence. Correct interpretation of results is essential, and combining results from different instruments may be necessary to come to the right conclusions. This article discusses clinical outcome studies, joint registries, patient reported outcome scores (PROMs), the Orthopaedic Data Evaluation Panel (ODEP) and Beyond Compliance.
Joint registries In October 1975 the first national knee arthroplasty register was established in Sweden. Other countries followed suit and the British National Joint Registry (NJR) was set up by the Department of Health and Welsh Government in 2002. Since 2008, the management of the NJR has been in the hands of the Healthcare Quality Improvement Partnership (HQIP), a consortium comprising the Academy of Medical Royal Colleges, the Royal College of Nursing and National Voices. Northern Ireland joined in 2013 and the Isle of Man in 2015. Data on hip and knee joint replacements have been collected since April 2003, ankle joint replacements since April 2010 and elbow and shoulder joint replacements since April 2012. The NJR collects information on all hip, knee, ankle, elbow and shoulder replacement operations, using revision for any reason as an end-point. The registry monitors the performance of joint replacement implants, investigates different types of surgical methods and improves clinical standards; it benefits patients, clinicians and the orthopaedic sector as a whole. The goals of the NJR are to monitor the outcomes achieved by brand of prosthesis, hospital and surgeon. Information is published for patients and the healthcare organization in general, including clinicians, commissioning groups and implant companies. By engaging patients there is better awareness of choice. The NJR data allow for identification of best practice and presents healthcare providers with information on implant quality and cost-effectiveness. In recent years NJR data have been used to determine a national tariff for new best
Clinical outcome studies Clinical studies are probably the most traditional of methods to investigate outcomes. Best evidence can be obtained using a randomized controlled trial or a meta-analysis of randomized studies (Level 1). In these studies patients are randomly assigned
Paul Nicolai FRCS (Tr & Orth) Consultant Orthopaedic Surgeon, West Suffolk Hospital, Bury St Edmunds, Suffolk, UK. Conflicts of interest: none declared.
ORTHOPAEDICS AND TRAUMA --:-
1
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
Management feedback provides healthcare provider management with essential indicators of their hospital’s performance. It includes compliance rates of data entry, consent rates and knee outcome data (funnel plots showing revision and mortality). Revision rates for individual surgeons are reported anonymously, but can be cross-checked against surgeon-level reports (clinician feedback).
practice payment in knee replacement. To-date, almost two million implants have been added to the registry. In 2011, the National Joint Registry acquired mandatory status as a National Audit, which means that any provider carrying out hip, knee, ankle, elbow or shoulder surgery is mandated to submit 100% of eligible primary and revision procedures to the NJR. This includes NHS and private sector healthcare providers. By crossreferencing NJR data with data held in Hospital Episodes Statistics (HES) and the Patient Episode Database Wales (PEDW), accuracy of data entry can be monitored. However, errors associated with the use of HES data have been widely documented. In a paper from Jayatilaka et al. significant discrepancies were found when NJR data were compared with a surgeon’s prospectively collected personal logbook.2 Independent sector hospitals do not submit data to either HES or PEDW, so accuracy there is more difficult to monitor. NJR analysis showed that in 2013, 77% of NHS providers reported 95% or more of the joint replacements they undertook, 16% of NHS providers reported between 80% and 95% and only 7% of NHS providers reported less than 80%. A key NJR strategy is to improve data quality by monitoring data accuracy and completeness, in order to facilitate validation. The NJR publishes results in different formats: The 12th Annual Report (Figure 1) was published in 2015 (www.njrcentre.org.uk), and analyzed data between 2003 and 2014. Its key conclusion was that first-time knee replacement operations continue to be hugely successful, with overall life of implants meeting national standards (95%þ lasting ten or more years). Knee brands offer comparable results and patient factors have a significant bearing on how long the implant will last, with younger patients reporting higher revision rates than their older counterparts. NJR data also highlight an interesting debate about unicompartmental knee replacements. Critics identify higher revision rates compared to total knee replacements, but proponents argue that the functional results of unicompartmental knee replacements are superior to those of total knee replacement. It is well known that approximately 20% of total knee replacements patients are unhappy with the results of surgery, but this phenomenon is not obvious in unicompartmental knee replacement patients. StatsOnline is an NJR facility to view and download hospital statistics regarding joint replacements. It shows number of operations submitted to the NJR, broken down by type of replacement and based on the operation date. It also shows NJR consent rate. ReportsOnline gives information on case total, case mix and implant usage. Outlier monitoring is a process where prostheses and surgical outcomes are compared to an expected revision rate. When results fall below an expected performance (‘outlier’), an investigation is initiated and further action is recommended. Clinician feedback is only available to surgeons performing knee and hip replacements. It gives more information regarding the procedures recorded in a surgeon’s name, and analyses it against hospital data and national benchmarks. It is good practice for orthopaedic departments to openly discuss surgeon-level reports in a yearly meeting.
ORTHOPAEDICS AND TRAUMA --:-
Patient reported outcome scores (PROMs) Joint registry data provide information about revision rates only, so it was clear there was a requirement for outcome evaluation seen from a patient’s perspective, about how a joint replacement operation affects their life in terms of function and quality of life. This was recognized by the Department of Health, and since April 2009 patients’ health status data have been collected before and after procedures, including knee replacements. These are called patient reported outcome scores, and they aim to calculating health gains after surgical treatment. Data are collected by patients through short self-completed questionnaires pre-operatively and six months post-operatively for knee replacements. From January 2012, regular publications were released using statistical analysis. In April 2013, NHS England took over the responsibility for the national PROMs programme from the Department of Health. NHS England commissioned the Health and Social Care Information Centre (HSCIC) to collect, process and publish PROMs data, and this is now called NHS Digital. PROMs reports can be found on the NHS Digital website (www.digital.nhs.uk). NHS Digital uses three PROMs scoring systems for knee replacement patients: the EQ-5DÔ index, EQ Visual Analogue Scale (EQ VAS) and the Oxford Knee Score (OKS): The EQ-5DÔ is a trademark of the EuroQol Group, and asks patients about their mobility, self-care, usual activities, pain/ discomfort and anxiety/depression on the day of completion of the questionnaire. Each question gives five options to choose from. The EQ-5D has been validated in a patient population in six countries with patient groups suffering from a variety of chronic conditions, including arthritis.3 The EQ Visual Analogue Scale asks patients how good or bad their health is, which is scored on a visual 100-point scale. Zero is the worst and 100 is the best possible health. Patients mark their score on the scale. The answer to this question is not necessarily associated with the knee replacement treatment, but more an indicator of general health. The Oxford Knee Score consists of 12 questions on daily activities to assess residual pain and function in knee replacement patients. It was developed at the University of Oxford in 1998 and has been validated.4 In large scale studies it has been ranked the best disease-specific patient reported outcome score for assessing outcome after knee replacement.5 The questions concern pain, personal hygiene restrictions as a result of knee pain, use of car or public transport, walking, getting up from a chair, limping, kneeling, night pain, (house) work interruption due to knee pain, knee instability, shopping and stairs. Each question has five options, scored from 0 to 4. A score from 0 to 19 may indicate severe knee arthritis, possibly requiring surgical intervention.
2
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
Figure 1 The 12th Annual Report for the National Joint Registry for England, Wales, Northern Ireland and the Isle of Man.
To be able to compare results across large patient groups in a meaningful way it has to be considered that every patient population is different. The Department of Health has developed a complex method to adjust for case-mix differences. There are specific adjustments for knee replacement patients, based on variations in patient characteristics and other factors that are beyond provider control. Examples of factors include age, gender, ethnicity, social status, home circumstances, previous knee surgery, comorbidity, depression and pre-operative score. Adjusted health gain is published at quarterly intervals in February, May, August and November on the NHS Digital website. In the latest report, published in August 2016, data were analzsed between 1st April 2015 and 31st March 2016. For knee replacements, 93.7% of patients had an improved Oxford Knee Score after six months, 80.8% had an improved EQ-5D index and
ORTHOPAEDICS AND TRAUMA --:-
55.3% had an improved EQ VAS. These results are slightly worse compared to hip replacement surgery, but better than surgery for groin hernia or varicose veins. On the website, the adjusted health gain for each hospital can be found, including an outlier summary (Figure 2). In a study from Baker et al. published in 2012, PROMs data were linked to National Joint Registry data.6 In a large cohort of 22,691 total knee replacements surgical factors that influenced improvements in Oxford Knee Score and EQ-5D were investigated. The study found that implant brand and hospital type affected patient reported outcome scores. However, the effect of these surgical factors was small compared to patient factors, and the most important factors affecting outcome were the preoperative score and the patient’s general health status. In another study from Partridge et al., PROMs data initiated changes
3
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
Figure 2 NHS Digital website page showing national EQ-5D results with an outlier summary.
in surgical technique (fat pad preservation) and change of implant.7 The effect of these changes demonstrated an average Oxford Knee Score improvement from 14 to 17.3.
states that “Only an artificial hip or hip resurfacing with a replacement rate of less than 1 in 20 at 10 years should be used.” A similar guideline is not available for knee replacements. ODEP aims to provide hip and knee benchmarks to comply with NICE guidelines. It started to provide ratings for primary hip implants in 2002 and hip resurfacing in 2004. In 2014, total knee replacements were also included in the rating process. The panel was set up by the National Health Purchasing and Supply Agency, now called the NHS Supply Chain. It is an independent panel of experts, including experienced UK hip and knee surgeons and non-clinical implant experts. Implant
Orthopaedic Data Evaluation Panel (ODEP) The Orthopaedic Data Evaluation Panel (ODEP) was set up to enable compliance with implant guidelines from the National Institute for Clinical Excellence (NICE). NICE have been publishing guidelines since 2000, and the latest technology appraisal guidance was published on 26th February 2014 (www.nice.org. uk/Guidance/TA304). This relates to hip replacements and
ORTHOPAEDICS AND TRAUMA --:-
4
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
prostheses. Based on standardized data collection and implant surveillance, it agrees a suitable rate at which the implant should be introduced in clinical practice. Beyond Compliance was set up following implant related adverse effects leading to a recall of more than one make of joint replacement in the last few years. The Medicines and Healthcare products Regulatory Agency (MHRA) regulates medical devices to ensure their quality, safety and efficacy. It analzsed the reasons for the implant failures and the main conclusion was that large numbers of the affected prosthesis were implanted soon after the product was launched, before sufficient data were available to demonstrate that the implant was fit for widespread clinical practice. In response, the British Orthopaedic Association and MHRA introduced the Beyond Compliance process. Beyond Compliance is supported by a Steering Committee (responsible for the overall process and funding) and an Advisory Group. This group consists of experienced hip and knee surgeons (from the Orthopaedic Data Evaluation Panel), bioengineers, statisticians and toxicologists. When new hip or knee implants are marketed, a technical file must be submitted by the manufacturer to a notified body of examination before a CE mark can be affixed. The CE allows the implant to be freely marketed in the European Union. The technical file should contain design features and testing data, clinical data and a plan for postmarketing surveillance. A CE mark is no guarantee for the safety or risk assessment of the new implant. Beyond Compliance fills the gap between a CE mark and ODEP/NJR outcomes. Implant companies can register newly developed or modified prostheses with Beyond Compliance. Significant modifications include a change in materials used, a change in the bearing surface and a change in fixation. The Advisory Committee analyses the presented evidence and allocates a risk level. There are five levels of risk, and the manufacturer will receive confidential information of the risk level. Based on the risk level, the Advisory Committee agrees a step-by-step introduction of the implant in clinical practice, the speed of which depends on the perceived risk level. Post-marketing surveillance includes evaluation of clinical data, adverse events and recognition of early problems. There is regular feedback to implant manufacturers and participating surgeons. Beyond Compliance also requests patients’ consent to allow analysis in case of implant removal. Forensic examination in a recognized retrieval centre provides valuable information about implant performance and failure. All UK implant retrieval centres met in February 2016 to agree on how to obtain excellence in implant retrieval analysis. Implants registered with Beyond Compliance receive a Preentry A* ODEP rating; prostheses registered with the National Joint Registry but not under Beyond Compliance are rated ‘preentry’.
manufacturers are invited to submit evidence regarding implant survivorship using standardized forms. Evidence needs to be submitted for every type of knee replacement prosthesis, and hence cruciate-retaining and posteriorly-stabilized implants of the same brand need separate submissions. The panel reviews and discusses the submissions based on the presented strength of evidence of implant survivorship. If the evidence is compliant with the NICE guidance then an ODEP rating is awarded. Although the NICE benchmark is set at 10-year performance, the panel also assesses implants at three, five and seven years, to track and rate them. Failure to show progression in terms of numbers of cases presented and survivorship targets will result in removal of an implant’s ODEP rating. An ODEP rating consists of a number and a letter; an optional star has been added after the 2014 NICE guidelines. The number reflects the number of years of performance evidence presented. The letter represents the strength of the submitted evidence. The letter A represents strong evidence with a minimum number of cases submitted. The minimum numbers are set by the panel and increase with follow-up, so at 10 years 500 cases need to be submitted, but lower numbers are allowed at shorter follow-up. The letter B is awarded for acceptable evidence, both of survivorship and number of cases presented, and sufficient data to demonstrate compliance. Initially the benchmark was 90% survivorship at 10 years, but since the 2014 NICE guidance a star is awarded for 95% survivorship. That means a 10A* rating fully complies with NICE guidelines. If evidence is presented of a sufficient number of cases with 90% survivorship at five years, a 5A rating is awarded; the product is seen as being on track to achieve the 10-year benchmark. Figure 2 shows all ODEP ratings. There are products that are registered with ODEP that do not have sufficient evidence for the three year benchmark. These implants receive a “pre-entry” listing and often relate to new prostheses or prostheses who have only recently been registered with ODEP. Clinical evaluation is in place and evidence will be submitted according to the ODEP time points (Figure 3). Implant ratings can be seen on the ODEP website (www.odep. org.uk) and provide surgeons with another tool to consider the choice of implants they use. It is a simple assessment reflecting national clinical best practice guidelines. It is a commonly used benchmark not just in the United Kingdom, but globally.8
Beyond Compliance The Beyond Compliance Advisory Group advises implant manufacturers of the level of risk of newly introduced or modified
Summary There are numerous ways to measure outcome in knee or hip arthroplasty, and this article presents an overview of the different instruments in use in the UK. All have advantages and limitations, but all provide a unique amount of information that enables surgeons, healthcare providers and implant manufacturers to optimize clinical care. It is important to base clinical practice
Figure 3 The ODEP ratings for hip and knee prostheses.
ORTHOPAEDICS AND TRAUMA --:-
5
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
on available evidence, and the methods described in this article allow all parties to make sensible choices about best practice and cost-effectiveness. A
5 Dunbar MJ, Robertsson O, Ryd L, Lidgren L. Appropriate questionnaires for knee arthroplasty. Results of a survey of 3600 patients from the Swedish Knee Arthroplasty Registry. J Bone Joint Surg Br 2001; 83: 339e44. 6 Baker PN, Deehan DJ, Lees D, et al. The effect of surgical factors on early patient-reported outcome measures (PROMS) following total knee replacement. J Bone Joint Surg Br 2012 Aug; 94: 1058e66. 7 Partridge T, Carluke I, Emmerson K, Partington P, Reed M. Improving patient reported outcome measures (PROMs) in total knee replacement by changing implant and preserving the infrapatella fatpad: a quality improvement project. BMJ Qual Improv Rep 2016 May 5; 5. 8 Poolman RW, Verhaar JA, Schreurs BW, et al. Finding the right hip implant for patient and surgeon: the Dutch strategy e empowering patients. Hip Int 2015 MareApr; 25: 131e7.
REFERENCES 1 Kendall JM. Designing a research project: randomised controlled trials and their principles. Emerg Med J 2003; 20: 164e8. 2 Jayatilaka MLT, Perry D, Cope MR. Accuracy of the national joint registry data at a surgeon level: a comparison of three years of data entry between the National Joint Registry and a individual surgeon’s e-logbook. BJJ 2013; 95-B. SUPP 27/14. 3 Janssen MF, Pickard AS, Golicki D, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res 2013; 22: 1717e27. 4 Xie F, Ye H, Zhang Y, Lin X, Lei T, Li SC. Extension from in-patients to out-patients: validity and reliability of the Oxford Knee Score in measuring health outcomes in patients with knee osteoarthritis. Int J Rheum Dis 2011 May; 14: 206e10.
ORTHOPAEDICS AND TRAUMA --:-
6
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
Measuring knee arthroplasty outcomes
to a treatment or control group and are followed prospectively. Level 2 evidence consists of prospective comparative cohort studies where patients are not randomized. Meta-analysis of Level 2 studies also falls into this category. Level 3 evidence is represented by retrospective cohort studies, case control studies (presence or absence of disease or treatment) and meta-analysis of Level 3 studies. Case series are Level 4 and case reports Level 5 evidence. Level 5 evidence also includes expert opinions and personal observations. A good study design starts with a hypothesis that investigates an anticipated association between variables (or no association, as in a null hypothesis).1 Next, a representative study population needs to be chosen to answer the research question. Inclusion and exclusion criteria determine the characteristics of the population, and an appropriate sample size needs to be calculated to be able to reach statistical significance in outcomes. A blinded (concealed from the investigator) random allocation procedure should be used to create the different groups. Blinding when applying the intervention and measuring the outcome is essential to avoid bias. There are several outcome scores for knee arthritis, including the Oxford Knee Score (OKS), Knee Society Score (KSS), Knee Injury and Osteoarthritis Outcome Score (KOOS), Western Ontario and McMaster Universities Arthritis Index (WOMAC) and International Knee Documentation Committee (IKDC). Outcome measures need to be chosen carefully and when a scoring system is used, validation of the scoring system is essential. Finally, statistical analysis can prove or disprove the hypothesis. Other considerations are the ethical implications (Ethics Committee study approval), the need for informed consent for the study population and last but not least, patient safety.
Paul Nicolai
Abstract Outcome measures are increasingly used by clinicians, healthcare providers and implant manufacturers for a variety of reasons. Clinical outcome studies have traditionally been used but the level of evidence presented has been variable. In recent decades joint registries have provided useful data on implant survival, but implant revision as an outcome measure has restrictions as many factors contribute to it. Patient reported outcome scores have been introduced more recently and provide important information about health gain after surgical intervention. In response to guidelines from the National Institute for Clinical Excellence, the Orthopaedic Data Evaluation Panel assesses and rates implants, and Beyond Compliance advises implant manufacturers of the level of risk of newly introduced or modified prostheses.
Keywords Beyond Compliance; clinical studies; National Joint Registry; Orthopaedic Data Evaluation Panel; outcome measures; patient reported outcome scores
Introduction The analysis of results has become an essential part of orthopaedic practice. Outcome scores can be used to assess the effect of a disease or condition, the effect of disease treatment and its cost effectiveness of treatments. They can also be used for research purposes, teaching and peer comparison. Outcome analysis allows surgeons to make informed choices of the use of prostheses and surgical techniques based on evidence. There is increasing pressure on hospital trusts to assess their performance data and in the future clinical commissioning may depend on outcome scores for implants and treatments. Surgeons’ mortality rates are openly published and available to see by the general population. Transparency is increasingly important. There are a variety of scoring instruments available, all with advantages, limitations and varying degrees of evidence. Correct interpretation of results is essential, and combining results from different instruments may be necessary to come to the right conclusions. This article discusses clinical outcome studies, joint registries, patient reported outcome scores (PROMs), the Orthopaedic Data Evaluation Panel (ODEP) and Beyond Compliance.
Joint registries In October 1975 the first national knee arthroplasty register was established in Sweden. Other countries followed suit and the British National Joint Registry (NJR) was set up by the Department of Health and Welsh Government in 2002. Since 2008, the management of the NJR has been in the hands of the Healthcare Quality Improvement Partnership (HQIP), a consortium comprising the Academy of Medical Royal Colleges, the Royal College of Nursing and National Voices. Northern Ireland joined in 2013 and the Isle of Man in 2015. Data on hip and knee joint replacements have been collected since April 2003, ankle joint replacements since April 2010 and elbow and shoulder joint replacements since April 2012. The NJR collects information on all hip, knee, ankle, elbow and shoulder replacement operations, using revision for any reason as an end-point. The registry monitors the performance of joint replacement implants, investigates different types of surgical methods and improves clinical standards; it benefits patients, clinicians and the orthopaedic sector as a whole. The goals of the NJR are to monitor the outcomes achieved by brand of prosthesis, hospital and surgeon. Information is published for patients and the healthcare organization in general, including clinicians, commissioning groups and implant companies. By engaging patients there is better awareness of choice. The NJR data allow for identification of best practice and presents healthcare providers with information on implant quality and cost-effectiveness. In recent years NJR data have been used to determine a national tariff for new best
Clinical outcome studies Clinical studies are probably the most traditional of methods to investigate outcomes. Best evidence can be obtained using a randomized controlled trial or a meta-analysis of randomized studies (Level 1). In these studies patients are randomly assigned
Paul Nicolai FRCS (Tr & Orth) Consultant Orthopaedic Surgeon, West Suffolk Hospital, Bury St Edmunds, Suffolk, UK. Conflicts of interest: none declared.
ORTHOPAEDICS AND TRAUMA --:-
1
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
Management feedback provides healthcare provider management with essential indicators of their hospital’s performance. It includes compliance rates of data entry, consent rates and knee outcome data (funnel plots showing revision and mortality). Revision rates for individual surgeons are reported anonymously, but can be cross-checked against surgeon-level reports (clinician feedback).
practice payment in knee replacement. To-date, almost two million implants have been added to the registry. In 2011, the National Joint Registry acquired mandatory status as a National Audit, which means that any provider carrying out hip, knee, ankle, elbow or shoulder surgery is mandated to submit 100% of eligible primary and revision procedures to the NJR. This includes NHS and private sector healthcare providers. By crossreferencing NJR data with data held in Hospital Episodes Statistics (HES) and the Patient Episode Database Wales (PEDW), accuracy of data entry can be monitored. However, errors associated with the use of HES data have been widely documented. In a paper from Jayatilaka et al. significant discrepancies were found when NJR data were compared with a surgeon’s prospectively collected personal logbook.2 Independent sector hospitals do not submit data to either HES or PEDW, so accuracy there is more difficult to monitor. NJR analysis showed that in 2013, 77% of NHS providers reported 95% or more of the joint replacements they undertook, 16% of NHS providers reported between 80% and 95% and only 7% of NHS providers reported less than 80%. A key NJR strategy is to improve data quality by monitoring data accuracy and completeness, in order to facilitate validation. The NJR publishes results in different formats: The 12th Annual Report (Figure 1) was published in 2015 (www.njrcentre.org.uk), and analyzed data between 2003 and 2014. Its key conclusion was that first-time knee replacement operations continue to be hugely successful, with overall life of implants meeting national standards (95%þ lasting ten or more years). Knee brands offer comparable results and patient factors have a significant bearing on how long the implant will last, with younger patients reporting higher revision rates than their older counterparts. NJR data also highlight an interesting debate about unicompartmental knee replacements. Critics identify higher revision rates compared to total knee replacements, but proponents argue that the functional results of unicompartmental knee replacements are superior to those of total knee replacement. It is well known that approximately 20% of total knee replacements patients are unhappy with the results of surgery, but this phenomenon is not obvious in unicompartmental knee replacement patients. StatsOnline is an NJR facility to view and download hospital statistics regarding joint replacements. It shows number of operations submitted to the NJR, broken down by type of replacement and based on the operation date. It also shows NJR consent rate. ReportsOnline gives information on case total, case mix and implant usage. Outlier monitoring is a process where prostheses and surgical outcomes are compared to an expected revision rate. When results fall below an expected performance (‘outlier’), an investigation is initiated and further action is recommended. Clinician feedback is only available to surgeons performing knee and hip replacements. It gives more information regarding the procedures recorded in a surgeon’s name, and analyses it against hospital data and national benchmarks. It is good practice for orthopaedic departments to openly discuss surgeon-level reports in a yearly meeting.
ORTHOPAEDICS AND TRAUMA --:-
Patient reported outcome scores (PROMs) Joint registry data provide information about revision rates only, so it was clear there was a requirement for outcome evaluation seen from a patient’s perspective, about how a joint replacement operation affects their life in terms of function and quality of life. This was recognized by the Department of Health, and since April 2009 patients’ health status data have been collected before and after procedures, including knee replacements. These are called patient reported outcome scores, and they aim to calculating health gains after surgical treatment. Data are collected by patients through short self-completed questionnaires pre-operatively and six months post-operatively for knee replacements. From January 2012, regular publications were released using statistical analysis. In April 2013, NHS England took over the responsibility for the national PROMs programme from the Department of Health. NHS England commissioned the Health and Social Care Information Centre (HSCIC) to collect, process and publish PROMs data, and this is now called NHS Digital. PROMs reports can be found on the NHS Digital website (www.digital.nhs.uk). NHS Digital uses three PROMs scoring systems for knee replacement patients: the EQ-5DÔ index, EQ Visual Analogue Scale (EQ VAS) and the Oxford Knee Score (OKS): The EQ-5DÔ is a trademark of the EuroQol Group, and asks patients about their mobility, self-care, usual activities, pain/ discomfort and anxiety/depression on the day of completion of the questionnaire. Each question gives five options to choose from. The EQ-5D has been validated in a patient population in six countries with patient groups suffering from a variety of chronic conditions, including arthritis.3 The EQ Visual Analogue Scale asks patients how good or bad their health is, which is scored on a visual 100-point scale. Zero is the worst and 100 is the best possible health. Patients mark their score on the scale. The answer to this question is not necessarily associated with the knee replacement treatment, but more an indicator of general health. The Oxford Knee Score consists of 12 questions on daily activities to assess residual pain and function in knee replacement patients. It was developed at the University of Oxford in 1998 and has been validated.4 In large scale studies it has been ranked the best disease-specific patient reported outcome score for assessing outcome after knee replacement.5 The questions concern pain, personal hygiene restrictions as a result of knee pain, use of car or public transport, walking, getting up from a chair, limping, kneeling, night pain, (house) work interruption due to knee pain, knee instability, shopping and stairs. Each question has five options, scored from 0 to 4. A score from 0 to 19 may indicate severe knee arthritis, possibly requiring surgical intervention.
2
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
Figure 1 The 12th Annual Report for the National Joint Registry for England, Wales, Northern Ireland and the Isle of Man.
To be able to compare results across large patient groups in a meaningful way it has to be considered that every patient population is different. The Department of Health has developed a complex method to adjust for case-mix differences. There are specific adjustments for knee replacement patients, based on variations in patient characteristics and other factors that are beyond provider control. Examples of factors include age, gender, ethnicity, social status, home circumstances, previous knee surgery, comorbidity, depression and pre-operative score. Adjusted health gain is published at quarterly intervals in February, May, August and November on the NHS Digital website. In the latest report, published in August 2016, data were analzsed between 1st April 2015 and 31st March 2016. For knee replacements, 93.7% of patients had an improved Oxford Knee Score after six months, 80.8% had an improved EQ-5D index and
ORTHOPAEDICS AND TRAUMA --:-
55.3% had an improved EQ VAS. These results are slightly worse compared to hip replacement surgery, but better than surgery for groin hernia or varicose veins. On the website, the adjusted health gain for each hospital can be found, including an outlier summary (Figure 2). In a study from Baker et al. published in 2012, PROMs data were linked to National Joint Registry data.6 In a large cohort of 22,691 total knee replacements surgical factors that influenced improvements in Oxford Knee Score and EQ-5D were investigated. The study found that implant brand and hospital type affected patient reported outcome scores. However, the effect of these surgical factors was small compared to patient factors, and the most important factors affecting outcome were the preoperative score and the patient’s general health status. In another study from Partridge et al., PROMs data initiated changes
3
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
Figure 2 NHS Digital website page showing national EQ-5D results with an outlier summary.
in surgical technique (fat pad preservation) and change of implant.7 The effect of these changes demonstrated an average Oxford Knee Score improvement from 14 to 17.3.
states that “Only an artificial hip or hip resurfacing with a replacement rate of less than 1 in 20 at 10 years should be used.” A similar guideline is not available for knee replacements. ODEP aims to provide hip and knee benchmarks to comply with NICE guidelines. It started to provide ratings for primary hip implants in 2002 and hip resurfacing in 2004. In 2014, total knee replacements were also included in the rating process. The panel was set up by the National Health Purchasing and Supply Agency, now called the NHS Supply Chain. It is an independent panel of experts, including experienced UK hip and knee surgeons and non-clinical implant experts. Implant
Orthopaedic Data Evaluation Panel (ODEP) The Orthopaedic Data Evaluation Panel (ODEP) was set up to enable compliance with implant guidelines from the National Institute for Clinical Excellence (NICE). NICE have been publishing guidelines since 2000, and the latest technology appraisal guidance was published on 26th February 2014 (www.nice.org. uk/Guidance/TA304). This relates to hip replacements and
ORTHOPAEDICS AND TRAUMA --:-
4
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
prostheses. Based on standardized data collection and implant surveillance, it agrees a suitable rate at which the implant should be introduced in clinical practice. Beyond Compliance was set up following implant related adverse effects leading to a recall of more than one make of joint replacement in the last few years. The Medicines and Healthcare products Regulatory Agency (MHRA) regulates medical devices to ensure their quality, safety and efficacy. It analzsed the reasons for the implant failures and the main conclusion was that large numbers of the affected prosthesis were implanted soon after the product was launched, before sufficient data were available to demonstrate that the implant was fit for widespread clinical practice. In response, the British Orthopaedic Association and MHRA introduced the Beyond Compliance process. Beyond Compliance is supported by a Steering Committee (responsible for the overall process and funding) and an Advisory Group. This group consists of experienced hip and knee surgeons (from the Orthopaedic Data Evaluation Panel), bioengineers, statisticians and toxicologists. When new hip or knee implants are marketed, a technical file must be submitted by the manufacturer to a notified body of examination before a CE mark can be affixed. The CE allows the implant to be freely marketed in the European Union. The technical file should contain design features and testing data, clinical data and a plan for postmarketing surveillance. A CE mark is no guarantee for the safety or risk assessment of the new implant. Beyond Compliance fills the gap between a CE mark and ODEP/NJR outcomes. Implant companies can register newly developed or modified prostheses with Beyond Compliance. Significant modifications include a change in materials used, a change in the bearing surface and a change in fixation. The Advisory Committee analyses the presented evidence and allocates a risk level. There are five levels of risk, and the manufacturer will receive confidential information of the risk level. Based on the risk level, the Advisory Committee agrees a step-by-step introduction of the implant in clinical practice, the speed of which depends on the perceived risk level. Post-marketing surveillance includes evaluation of clinical data, adverse events and recognition of early problems. There is regular feedback to implant manufacturers and participating surgeons. Beyond Compliance also requests patients’ consent to allow analysis in case of implant removal. Forensic examination in a recognized retrieval centre provides valuable information about implant performance and failure. All UK implant retrieval centres met in February 2016 to agree on how to obtain excellence in implant retrieval analysis. Implants registered with Beyond Compliance receive a Preentry A* ODEP rating; prostheses registered with the National Joint Registry but not under Beyond Compliance are rated ‘preentry’.
manufacturers are invited to submit evidence regarding implant survivorship using standardized forms. Evidence needs to be submitted for every type of knee replacement prosthesis, and hence cruciate-retaining and posteriorly-stabilized implants of the same brand need separate submissions. The panel reviews and discusses the submissions based on the presented strength of evidence of implant survivorship. If the evidence is compliant with the NICE guidance then an ODEP rating is awarded. Although the NICE benchmark is set at 10-year performance, the panel also assesses implants at three, five and seven years, to track and rate them. Failure to show progression in terms of numbers of cases presented and survivorship targets will result in removal of an implant’s ODEP rating. An ODEP rating consists of a number and a letter; an optional star has been added after the 2014 NICE guidelines. The number reflects the number of years of performance evidence presented. The letter represents the strength of the submitted evidence. The letter A represents strong evidence with a minimum number of cases submitted. The minimum numbers are set by the panel and increase with follow-up, so at 10 years 500 cases need to be submitted, but lower numbers are allowed at shorter follow-up. The letter B is awarded for acceptable evidence, both of survivorship and number of cases presented, and sufficient data to demonstrate compliance. Initially the benchmark was 90% survivorship at 10 years, but since the 2014 NICE guidance a star is awarded for 95% survivorship. That means a 10A* rating fully complies with NICE guidelines. If evidence is presented of a sufficient number of cases with 90% survivorship at five years, a 5A rating is awarded; the product is seen as being on track to achieve the 10-year benchmark. Figure 2 shows all ODEP ratings. There are products that are registered with ODEP that do not have sufficient evidence for the three year benchmark. These implants receive a “pre-entry” listing and often relate to new prostheses or prostheses who have only recently been registered with ODEP. Clinical evaluation is in place and evidence will be submitted according to the ODEP time points (Figure 3). Implant ratings can be seen on the ODEP website (www.odep. org.uk) and provide surgeons with another tool to consider the choice of implants they use. It is a simple assessment reflecting national clinical best practice guidelines. It is a commonly used benchmark not just in the United Kingdom, but globally.8
Beyond Compliance The Beyond Compliance Advisory Group advises implant manufacturers of the level of risk of newly introduced or modified
Summary There are numerous ways to measure outcome in knee or hip arthroplasty, and this article presents an overview of the different instruments in use in the UK. All have advantages and limitations, but all provide a unique amount of information that enables surgeons, healthcare providers and implant manufacturers to optimize clinical care. It is important to base clinical practice
Figure 3 The ODEP ratings for hip and knee prostheses.
ORTHOPAEDICS AND TRAUMA --:-
5
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006
KNEE ARTHROPLASTY
on available evidence, and the methods described in this article allow all parties to make sensible choices about best practice and cost-effectiveness. A
5 Dunbar MJ, Robertsson O, Ryd L, Lidgren L. Appropriate questionnaires for knee arthroplasty. Results of a survey of 3600 patients from the Swedish Knee Arthroplasty Registry. J Bone Joint Surg Br 2001; 83: 339e44. 6 Baker PN, Deehan DJ, Lees D, et al. The effect of surgical factors on early patient-reported outcome measures (PROMS) following total knee replacement. J Bone Joint Surg Br 2012 Aug; 94: 1058e66. 7 Partridge T, Carluke I, Emmerson K, Partington P, Reed M. Improving patient reported outcome measures (PROMs) in total knee replacement by changing implant and preserving the infrapatella fatpad: a quality improvement project. BMJ Qual Improv Rep 2016 May 5; 5. 8 Poolman RW, Verhaar JA, Schreurs BW, et al. Finding the right hip implant for patient and surgeon: the Dutch strategy e empowering patients. Hip Int 2015 MareApr; 25: 131e7.
REFERENCES 1 Kendall JM. Designing a research project: randomised controlled trials and their principles. Emerg Med J 2003; 20: 164e8. 2 Jayatilaka MLT, Perry D, Cope MR. Accuracy of the national joint registry data at a surgeon level: a comparison of three years of data entry between the National Joint Registry and a individual surgeon’s e-logbook. BJJ 2013; 95-B. SUPP 27/14. 3 Janssen MF, Pickard AS, Golicki D, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res 2013; 22: 1717e27. 4 Xie F, Ye H, Zhang Y, Lin X, Lei T, Li SC. Extension from in-patients to out-patients: validity and reliability of the Oxford Knee Score in measuring health outcomes in patients with knee osteoarthritis. Int J Rheum Dis 2011 May; 14: 206e10.
ORTHOPAEDICS AND TRAUMA --:-
6
Ó 2016 Published by Elsevier Ltd.
Please cite this article in press as: Nicolai P, Measuring knee arthroplasty outcomes, Orthopaedics and Trauma (2016), http://dx.doi.org/10.1016/ j.mporth.2016.09.006