- Email: [email protected]
PEDro's bias: summary quality scores should not be used in meta-analysis
Journal of Clinical Epidemiology 66 (2013) 75e77
ORIGINAL ARTICLES
PEDro’s bias: summary quality scores should not be used in meta-analysis Bruno R. da Costaa, Roger Hilfikera,b, Matthias Eggera,* a
Institute of Social and Preventive Medicine, University of Bern, Finkenhubelweg 11, 3012 Bern, Switzerland b Department of Physiotherapy, University of Applied Sciences Western Switzerland, Sion, Switzerland Accepted 20 August 2012
Systematic reviews of randomized controlled trials and statistical combination of results from different trials in meta-analysis have risen to such prominence that few question their usefulness in evidence-based clinical practice. Although systematic reviews are indeed generally useful, the results of meta-analyses must be cautiously interpreted [1]. If the estimates of the trials included in the meta-analysis deviate from the truth in a systematic fashion, the summary estimate from the meta-analysis will likely be biased as well. There is strong empirical evidence showing that the meta-analyses of clinical trials in which the allocation of patients to treatment groups was not concealed, or in which the assessment of outcomes was not blinded, overestimate the treatment effects [2e5]. It is now widely accepted that the quality of a trial should be assessed before including it in a meta-analysis. Likewise, it is a good practice to ascertain if the results differ between trials at greater or lesser risk of bias. As yet, however, there is no consensus as to how this assessment should be done. In this commentary, we discuss the scoring of trials using quality scales, using the example of a scale widely used in trials of physiotherapy. We contrast the use of summary scores with an alternative approach, which is based on an assessment of individual components such as concealment of allocation and blinding. Quality scales assess several criteria related to the design, conduct, and analysis of trials, and each earns points that are aggregated into an overall score. The score determines the classification of the study as one of the higher or lower methodological quality, with the implication that bias has been prevented to a greater or lesser degree. Chalmers et al. [6] in 1981 were among the first to develop such a scale, with possible summary scores ranging from 0 to 44. Since then, many other scales were developed,
* Corresponding author. Tel.: þ41 (0)31 631 35 11; fax: þ41 (0)31 631 35 20. E-mail address: [email protected] (M. Egger). 0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2012.08.003
including the widely used Physiotherapy Evidence Database (PEDro) scale [7e10]. Table 1 details the 11 items included in the PEDro scale. Most relate to the design and conduct of the trial, but three are concerned with reporting eligibility criteria (item 1), between-group statistical comparisons (item 10), and measures of variability (item 11). Notably, only two of the three items on reporting quality contribute points to the total score: ‘‘eligibility criteria specified’’ does not. The total score is therefore 10 rather than 11. Although the PEDro scale was developed for clinical trials of physiotherapy, it does not contain items that are specific to this field. For example, efforts to reduce bias owing to differential expertise of therapists are not considered by the PEDro scale. A physiotherapist’s training, experience, and personal preferences are likely to influence the efficacy of randomized therapies. Such bias may be reduced if patients are randomly allocated to practitioners who have expertise in the intervention under investigation [11]. The PEDro scale is routinely used to assess the quality of the randomized controlled trials that are continuously added to the freely available Physiotherapy Evidence Database [7,8] (also see http://www.pedro.org.au/). The database currently covers more than 15,000 clinical trials and makes their quality scores available. The PEDro scale is also widely used in published systematic reviews and meta-analyses. A search of Medline in March 2012 identified 126 reports that mentioned the PEDro scale in the abstract. Twenty-three used the PEDro score as an eligibility criterion, or to stratify analyses by trial quality, with the threshold score for eligibility or high quality ranging from three to seven. Of note, the database allows searches to be restricted to trials with a minimum PEDro score defined by the user and thus encourages the exclusion of trials below a given threshold. The Cochrane Collaboration recommends a component approach to the assessment of trial quality and risk of bias, rather than the use of quality scales such as PEDro [12]. It was wondered whether the approach chosen might affect the conclusions of Cochrane reviews. To examine this, the
76
B.R. da Costa et al. / Journal of Clinical Epidemiology 66 (2013) 75e77
Table 1. Items of the Physiotherapy Evidence Database scale Item No. 1 2 3 4 5 6 7 8 9
Description Eligibility criteria were specified Subjects were randomly allocated to groups Allocation was concealed The groups were similar at baseline regarding the most important prognostic indicators There was blinding of all subjects There was blinding of all therapists who administered the therapy There was blinding of all assessors who measured at least one key outcome Measures of at least one key outcome were obtained from more than 85% of the subjects initially allocated to groups All subjects for whom outcome measures were available received the treatment or control condition as allocated or, where this was not the case, data for at least one key outcome was analyzed by ‘‘intention to treat’’ The results of between-group statistical comparisons are reported for at least one key outcome The study provides both point measures and measures of variability for at least one key outcome
10 11
Item 1 (eligibility criteria) does not contribute to total score.
review of transcutaneous electrostimulation for osteoarthritis of the knee was revisited [13]. The PEDro scores were available for 15 of the 16 trials included in the meta-analysis, with scores ranging from three to eight. These 15 trials contributed 17 comparisons. The Cochrane review had identified only one trial of clearly high quality [14], with adequate generation of random sequence, concealment of allocation, and blinding of study participants and therapists. This trial received a high PEDro score of eight; however, other trials that were also highly scored by PEDro lacked adequate concealment of allocation and blinding of patients or therapists [15e19]. A high PEDro score does thus not mean that the trial was adequately randomized and blinded. The Cochrane meta-analysis was repeated [13] using the scores from the PEDro database to identify high-quality trials. Fig. 1 shows the summary effect sizes for pain for all trials, for trials of higher quality using different PEDro thresholds, and for the high-quality trial identified in the original review [13]. With the PEDro scale, the beneficial effect of electrostimulation became more prominent as the quality of the trials increased. In contrast, the estimate from the high-quality trial was close to zero, indicating little benefit of electrostimulation. The previous review concluded Std. Mean Difference (95%CI)
Summary score approach
No. of comparisons
PEDro score ≥ 7
-1.28 (-2.20 to -0.37)
4
PEDro score ≥ 5
-0.96 (-1.56 to -0.37)
9
-0.38 (-0.93 to 0.17)
1
-0.90 (-1.30 to -0.51)
17
Component approach Adequate randomisation and blinding All trials
-2.5
0 Favours electrostimulation
1 Favours control
Fig. 1. Standardized mean differences (SMDs) in pain scales from trials of transcutaneous electrostimulation for osteoarthritis of the knee. Trials of high-methodological quality are either defined based on PEDro summary scores or adequate randomization and blinding. The SMDs were combined in inverse-variance random-effects metaanalysis. Adapted from a Cochrane review [13]. PEDro, Physiotherapy Evidence Database.
that the evidence was ‘‘inconclusive, hampered by the inclusion of only small trials of questionable quality’’ [13]. It seems likely that based on the results from the PEDro scale, many reviewers would conclude with the contrary opinion that there was robust evidence from high-quality trials that electrostimulation had a clinically relevant, beneficial effect on pain in osteoarthritis of the knee. Greenland [20] described quality scoring as ‘‘perhaps the most insidious form of subjectivity masquerading as objectivity in meta-analysis’’: the effects of quality dimensions that are important in a given study and context are diluted or confounded by the summary quality score, sometimes to the point that quality effects are no longer evident, or that effects are reversed, as in our example. The PEDro scale and many other quality scales include items that are not in fact related to the methodology and extent bias was avoided in a trial, but to the quality of reporting. Furthermore, items that are important for some interventions or outcomes may not be relevant in other situations but will receive the same consideration. For example, blinding of study participants will be crucial for pain but irrelevant for all-cause mortality. Greenland’s arguments are supported by our case study and other empirical evidence. J€uni et al. [22] used 25 different scales to assess the quality of trials comparing lowemolecular weight heparin with standard heparin. In 25 separate meta-analyses (one for each quality scale), the results from high- and low-quality trials were compared, with very different results depending on which quality scale was used. Colle et al. [21] found similar effects across 16 different scales for trials of exercise for low back pain. Finally, Wood et al. [4] showed in a study of 146 meta-analyses that blinding was associated with exaggerated effects in trials with subjective outcomes, but not in trials with objective outcomes. The assembly of the PEDro database of many thousands of trials, dating back to the 1930s, is a tremendous achievement and an extremely valuable resource for the evaluation of the evidence underpinning physiotherapy interventions. Unfortunately, the PEDro database’s inappropriate emphasis on the use of summary scores from a quality scale makes the database less useful than it might be. It is likely that bias is introduced into systematic reviews and meta-analyses
B.R. da Costa et al. / Journal of Clinical Epidemiology 66 (2013) 75e77
when these scores are used as the main criteria on which the inclusion or exclusion of trials is based. We suggest that the use of summary scores should be discouraged, and that the PEDro database be restricted to presenting the scores for individual items of the scale. The addition of items, for example on the prevention of differential expertise bias, and the removal of items not related to the risk of bias, might further enhance the value of this important initiative.
References [1] Egger M, Davey Smith G. Misleading meta-analysis. BMJ 1995;311:753e4. [2] Schulz KF, Chalmers I, Hayes RJ, Altman D. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408e12. [3] Nuesch E, Reichenbach S, Trelle S, Rutjes AW, Liewald K, Sterchi R, et al. The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum 2009;61:1633e41. [4] Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 2008;336:601e5. [5] Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:e1e37. [6] Chalmers TC, Smith H, Blackburn B, Silverman B, Schroeder B, Reitman D, et al. A method for assessing the quality of a randomized control trial. Control Clin Trials 1981;2:31e49. [7] Moseley AM, Herbert RD, Maher CG, Sherrington C, Elkins MR. Reported quality of randomized controlled trials of physiotherapy interventions has improved over time. J Clin Epidemiol 2011;64: 594e601. [8] Herbert R, Moseley A, Sherrington C. PEDro: a database of randomised controlled trials in physiotherapy. Health Inf Manag 1998; 28(4):186e8. [9] Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995;16:62e73.
77
[10] Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther 2008;88:156e75. [11] Johnston BC, da Costa BR, Devereaux PJ, Akl EA, Busse JW. The use of expertise-based randomized controlled trials to assess spinal manipulation and acupuncture for low back pain: a systematic review. Spine (Phila Pa 1976) 2008;33(8):914e8. [12] Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. [13] Rutjes AW, Nuesch E, Sterchi R, Kalichman L, Hendriks E, Osiri M, et al. Transcutaneous electrostimulation for osteoarthritis of the knee. Cochrane Database Syst Rev 2009;4:CD002823. [14] Garland D, Holt P, Harrington JT, Caldwell J, Zizic T, Cholewczynski J. A 3-month, randomized, double-blind, placebocontrolled study to evaluate the safety and efficacy of a highly optimized, capacitively coupled, pulsed electrical stimulator in patients with osteoarthritis of the knee. Osteoarthritis Cartilage 2007;15: 630e7. [15] Law PP, Cheing GL. Optimal stimulation frequency of transcutaneous electrical nerve stimulation on people with knee osteoarthritis. J Rehabil Med 2004;36(5):220e5. [16] Quirk RA, Newman RJ, Newman KJ. An evaluation of interferential therapy, shortwave diathermy and exercise in the treatment of osteoarthritis of the knee. Physiotherapy 1985;71(8):266. [17] Law PP, Cheing GL, Tsui AY. Does transcutaneous electrical nerve stimulation improve the physical performance of people with knee osteoarthritis? J Clin Rheumatol 2004;10(6):295e9. [18] Grimmer K. A controlled double blind study comparing the effects of strong burst mode TENS and high rate TENS on painful osteoarthritis of the knee. Aust J Physiother 1992;38(1):49e56. [19] Cetin N, Aytar A, Atalay A, Akman MN. Comparing hot pack, short-wave diathermy, ultrasound, and TENS on isokinetic strength, pain, and functional status of women with osteoarthritic knees: a singleblind, randomized, controlled trial. Am J Phys Med Rehabil 2008; 87(6):443e51. [20] Greenland S. Quality scores are useless and potentially misleading. Am J Epidemiol 1994;140:300e1. [21] Colle F, Rannou F, Revel M, Fermanian J, Poiraudeau S. Impact of quality scales on levels of evidence inferred from a systematic review of exercise therapy and low back pain. Arch Phys Med Rehabil 2002;83:1745e52. [22] J€uni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282:1054e60.
ORIGINAL ARTICLES
PEDro’s bias: summary quality scores should not be used in meta-analysis Bruno R. da Costaa, Roger Hilfikera,b, Matthias Eggera,* a
Institute of Social and Preventive Medicine, University of Bern, Finkenhubelweg 11, 3012 Bern, Switzerland b Department of Physiotherapy, University of Applied Sciences Western Switzerland, Sion, Switzerland Accepted 20 August 2012
Systematic reviews of randomized controlled trials and statistical combination of results from different trials in meta-analysis have risen to such prominence that few question their usefulness in evidence-based clinical practice. Although systematic reviews are indeed generally useful, the results of meta-analyses must be cautiously interpreted [1]. If the estimates of the trials included in the meta-analysis deviate from the truth in a systematic fashion, the summary estimate from the meta-analysis will likely be biased as well. There is strong empirical evidence showing that the meta-analyses of clinical trials in which the allocation of patients to treatment groups was not concealed, or in which the assessment of outcomes was not blinded, overestimate the treatment effects [2e5]. It is now widely accepted that the quality of a trial should be assessed before including it in a meta-analysis. Likewise, it is a good practice to ascertain if the results differ between trials at greater or lesser risk of bias. As yet, however, there is no consensus as to how this assessment should be done. In this commentary, we discuss the scoring of trials using quality scales, using the example of a scale widely used in trials of physiotherapy. We contrast the use of summary scores with an alternative approach, which is based on an assessment of individual components such as concealment of allocation and blinding. Quality scales assess several criteria related to the design, conduct, and analysis of trials, and each earns points that are aggregated into an overall score. The score determines the classification of the study as one of the higher or lower methodological quality, with the implication that bias has been prevented to a greater or lesser degree. Chalmers et al. [6] in 1981 were among the first to develop such a scale, with possible summary scores ranging from 0 to 44. Since then, many other scales were developed,
* Corresponding author. Tel.: þ41 (0)31 631 35 11; fax: þ41 (0)31 631 35 20. E-mail address: [email protected] (M. Egger). 0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2012.08.003
including the widely used Physiotherapy Evidence Database (PEDro) scale [7e10]. Table 1 details the 11 items included in the PEDro scale. Most relate to the design and conduct of the trial, but three are concerned with reporting eligibility criteria (item 1), between-group statistical comparisons (item 10), and measures of variability (item 11). Notably, only two of the three items on reporting quality contribute points to the total score: ‘‘eligibility criteria specified’’ does not. The total score is therefore 10 rather than 11. Although the PEDro scale was developed for clinical trials of physiotherapy, it does not contain items that are specific to this field. For example, efforts to reduce bias owing to differential expertise of therapists are not considered by the PEDro scale. A physiotherapist’s training, experience, and personal preferences are likely to influence the efficacy of randomized therapies. Such bias may be reduced if patients are randomly allocated to practitioners who have expertise in the intervention under investigation [11]. The PEDro scale is routinely used to assess the quality of the randomized controlled trials that are continuously added to the freely available Physiotherapy Evidence Database [7,8] (also see http://www.pedro.org.au/). The database currently covers more than 15,000 clinical trials and makes their quality scores available. The PEDro scale is also widely used in published systematic reviews and meta-analyses. A search of Medline in March 2012 identified 126 reports that mentioned the PEDro scale in the abstract. Twenty-three used the PEDro score as an eligibility criterion, or to stratify analyses by trial quality, with the threshold score for eligibility or high quality ranging from three to seven. Of note, the database allows searches to be restricted to trials with a minimum PEDro score defined by the user and thus encourages the exclusion of trials below a given threshold. The Cochrane Collaboration recommends a component approach to the assessment of trial quality and risk of bias, rather than the use of quality scales such as PEDro [12]. It was wondered whether the approach chosen might affect the conclusions of Cochrane reviews. To examine this, the
76
B.R. da Costa et al. / Journal of Clinical Epidemiology 66 (2013) 75e77
Table 1. Items of the Physiotherapy Evidence Database scale Item No. 1 2 3 4 5 6 7 8 9
Description Eligibility criteria were specified Subjects were randomly allocated to groups Allocation was concealed The groups were similar at baseline regarding the most important prognostic indicators There was blinding of all subjects There was blinding of all therapists who administered the therapy There was blinding of all assessors who measured at least one key outcome Measures of at least one key outcome were obtained from more than 85% of the subjects initially allocated to groups All subjects for whom outcome measures were available received the treatment or control condition as allocated or, where this was not the case, data for at least one key outcome was analyzed by ‘‘intention to treat’’ The results of between-group statistical comparisons are reported for at least one key outcome The study provides both point measures and measures of variability for at least one key outcome
10 11
Item 1 (eligibility criteria) does not contribute to total score.
review of transcutaneous electrostimulation for osteoarthritis of the knee was revisited [13]. The PEDro scores were available for 15 of the 16 trials included in the meta-analysis, with scores ranging from three to eight. These 15 trials contributed 17 comparisons. The Cochrane review had identified only one trial of clearly high quality [14], with adequate generation of random sequence, concealment of allocation, and blinding of study participants and therapists. This trial received a high PEDro score of eight; however, other trials that were also highly scored by PEDro lacked adequate concealment of allocation and blinding of patients or therapists [15e19]. A high PEDro score does thus not mean that the trial was adequately randomized and blinded. The Cochrane meta-analysis was repeated [13] using the scores from the PEDro database to identify high-quality trials. Fig. 1 shows the summary effect sizes for pain for all trials, for trials of higher quality using different PEDro thresholds, and for the high-quality trial identified in the original review [13]. With the PEDro scale, the beneficial effect of electrostimulation became more prominent as the quality of the trials increased. In contrast, the estimate from the high-quality trial was close to zero, indicating little benefit of electrostimulation. The previous review concluded Std. Mean Difference (95%CI)
Summary score approach
No. of comparisons
PEDro score ≥ 7
-1.28 (-2.20 to -0.37)
4
PEDro score ≥ 5
-0.96 (-1.56 to -0.37)
9
-0.38 (-0.93 to 0.17)
1
-0.90 (-1.30 to -0.51)
17
Component approach Adequate randomisation and blinding All trials
-2.5
0 Favours electrostimulation
1 Favours control
Fig. 1. Standardized mean differences (SMDs) in pain scales from trials of transcutaneous electrostimulation for osteoarthritis of the knee. Trials of high-methodological quality are either defined based on PEDro summary scores or adequate randomization and blinding. The SMDs were combined in inverse-variance random-effects metaanalysis. Adapted from a Cochrane review [13]. PEDro, Physiotherapy Evidence Database.
that the evidence was ‘‘inconclusive, hampered by the inclusion of only small trials of questionable quality’’ [13]. It seems likely that based on the results from the PEDro scale, many reviewers would conclude with the contrary opinion that there was robust evidence from high-quality trials that electrostimulation had a clinically relevant, beneficial effect on pain in osteoarthritis of the knee. Greenland [20] described quality scoring as ‘‘perhaps the most insidious form of subjectivity masquerading as objectivity in meta-analysis’’: the effects of quality dimensions that are important in a given study and context are diluted or confounded by the summary quality score, sometimes to the point that quality effects are no longer evident, or that effects are reversed, as in our example. The PEDro scale and many other quality scales include items that are not in fact related to the methodology and extent bias was avoided in a trial, but to the quality of reporting. Furthermore, items that are important for some interventions or outcomes may not be relevant in other situations but will receive the same consideration. For example, blinding of study participants will be crucial for pain but irrelevant for all-cause mortality. Greenland’s arguments are supported by our case study and other empirical evidence. J€uni et al. [22] used 25 different scales to assess the quality of trials comparing lowemolecular weight heparin with standard heparin. In 25 separate meta-analyses (one for each quality scale), the results from high- and low-quality trials were compared, with very different results depending on which quality scale was used. Colle et al. [21] found similar effects across 16 different scales for trials of exercise for low back pain. Finally, Wood et al. [4] showed in a study of 146 meta-analyses that blinding was associated with exaggerated effects in trials with subjective outcomes, but not in trials with objective outcomes. The assembly of the PEDro database of many thousands of trials, dating back to the 1930s, is a tremendous achievement and an extremely valuable resource for the evaluation of the evidence underpinning physiotherapy interventions. Unfortunately, the PEDro database’s inappropriate emphasis on the use of summary scores from a quality scale makes the database less useful than it might be. It is likely that bias is introduced into systematic reviews and meta-analyses
B.R. da Costa et al. / Journal of Clinical Epidemiology 66 (2013) 75e77
when these scores are used as the main criteria on which the inclusion or exclusion of trials is based. We suggest that the use of summary scores should be discouraged, and that the PEDro database be restricted to presenting the scores for individual items of the scale. The addition of items, for example on the prevention of differential expertise bias, and the removal of items not related to the risk of bias, might further enhance the value of this important initiative.
References [1] Egger M, Davey Smith G. Misleading meta-analysis. BMJ 1995;311:753e4. [2] Schulz KF, Chalmers I, Hayes RJ, Altman D. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408e12. [3] Nuesch E, Reichenbach S, Trelle S, Rutjes AW, Liewald K, Sterchi R, et al. The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum 2009;61:1633e41. [4] Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 2008;336:601e5. [5] Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:e1e37. [6] Chalmers TC, Smith H, Blackburn B, Silverman B, Schroeder B, Reitman D, et al. A method for assessing the quality of a randomized control trial. Control Clin Trials 1981;2:31e49. [7] Moseley AM, Herbert RD, Maher CG, Sherrington C, Elkins MR. Reported quality of randomized controlled trials of physiotherapy interventions has improved over time. J Clin Epidemiol 2011;64: 594e601. [8] Herbert R, Moseley A, Sherrington C. PEDro: a database of randomised controlled trials in physiotherapy. Health Inf Manag 1998; 28(4):186e8. [9] Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995;16:62e73.
77
[10] Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther 2008;88:156e75. [11] Johnston BC, da Costa BR, Devereaux PJ, Akl EA, Busse JW. The use of expertise-based randomized controlled trials to assess spinal manipulation and acupuncture for low back pain: a systematic review. Spine (Phila Pa 1976) 2008;33(8):914e8. [12] Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. [13] Rutjes AW, Nuesch E, Sterchi R, Kalichman L, Hendriks E, Osiri M, et al. Transcutaneous electrostimulation for osteoarthritis of the knee. Cochrane Database Syst Rev 2009;4:CD002823. [14] Garland D, Holt P, Harrington JT, Caldwell J, Zizic T, Cholewczynski J. A 3-month, randomized, double-blind, placebocontrolled study to evaluate the safety and efficacy of a highly optimized, capacitively coupled, pulsed electrical stimulator in patients with osteoarthritis of the knee. Osteoarthritis Cartilage 2007;15: 630e7. [15] Law PP, Cheing GL. Optimal stimulation frequency of transcutaneous electrical nerve stimulation on people with knee osteoarthritis. J Rehabil Med 2004;36(5):220e5. [16] Quirk RA, Newman RJ, Newman KJ. An evaluation of interferential therapy, shortwave diathermy and exercise in the treatment of osteoarthritis of the knee. Physiotherapy 1985;71(8):266. [17] Law PP, Cheing GL, Tsui AY. Does transcutaneous electrical nerve stimulation improve the physical performance of people with knee osteoarthritis? J Clin Rheumatol 2004;10(6):295e9. [18] Grimmer K. A controlled double blind study comparing the effects of strong burst mode TENS and high rate TENS on painful osteoarthritis of the knee. Aust J Physiother 1992;38(1):49e56. [19] Cetin N, Aytar A, Atalay A, Akman MN. Comparing hot pack, short-wave diathermy, ultrasound, and TENS on isokinetic strength, pain, and functional status of women with osteoarthritic knees: a singleblind, randomized, controlled trial. Am J Phys Med Rehabil 2008; 87(6):443e51. [20] Greenland S. Quality scores are useless and potentially misleading. Am J Epidemiol 1994;140:300e1. [21] Colle F, Rannou F, Revel M, Fermanian J, Poiraudeau S. Impact of quality scales on levels of evidence inferred from a systematic review of exercise therapy and low back pain. Arch Phys Med Rehabil 2002;83:1745e52. [22] J€uni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282:1054e60.