- Email: [email protected]
External validity of a randomised clinical trial of temporomandibular disorders: analysis of the patients who refused to participate in research
British Journal of Oral and Maxillofacial Surgery (2003) 41, 129–131 © 2003 The British Association of Oral and Maxillofacial Surgeons. Published by Elsevier Science Ltd. All rights reserved. doi:10.1016/S0266-4356(03)00037-8, available online at www.sciencedirect.com
SHORT COMMUNICATION External validity of a randomised clinical trial of temporomandibular disorders: analysis of the patients who refused to participate in research H. Yuasa, ∗ K. Kurita, † P.-L. Westesson ‡ ∗ The
Second Department of Oral and Maxillofacial Surgery; †The First Department of Oral and Maxillofacial Surgery, School of Dentistry, Aichi-Gakuin University, Japan; ‡Division of Neuroradiololgy, Department of Radiology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA SUMMARY. Purpose: To assess the external validity of a randomised clinical trial (RCT) of a painful condition. Method: Consecutive patients with painful temporomandibular disorders (TMDs) were invited to participate in a clinical trial. Patients who refused to participate were compared to those who agreed to participate in this study with respect to degree of symptoms at time of presentation. Results: The patients who refused to participate had more pain, and their condition interfered more with their daily life than those who accepted the invitation to participate. Conclusion: Selection bias in RCTs of painful conditions can skew the results, and external validity should be analysed before the results are generalised. © 2003 The British Association of Oral and Maxillofacial Surgeons. Published by Elsevier Science Ltd. All rights reserved.
who agree to be randomised are enrolled. Other eligible patients are dropped and usually not observed further.3 Differences between patients who are enrolled and those who refuse to participate may limit the generalisability of the results. Temporomandibular disorders (TMDs) are common, painful and affect up to 20% of the population. Many treatments have been attempted but there is not a single, generally accepted treatment, that has been shown to be effective over untreated controls. Further, the natural course of one of the most commonest form of TMD, namely, disc displacement without reduction, has proved to be favourable with around two-thirds being free of symptoms or considerable improved after 2 years without treatment.4 In this study, we analysed the differences between those who agreed to participate in an RCT for treatment of painful TMD and compared them with those who refused to be randomised. To elucidate the external validity and generalisability of the results, we analysed the difference between patients who accepted the process and those who refused to be randomised.
INTRODUCTION Clinical trials are essential to establish evidence-based medical treatment. The randomised controlled trial (RCT) is viewed as the best study design because of its internal validity, which assures comparability among selected regimens. However, when the enrolment rate of participants is low and there is the difference between patients who accepted the process, the external validity of the extrapolation of the findings to the target population is not guaranteed.1 It is the purpose of RCTs to create groups of patients who are balanced with respect to known and unknown characteristics.2 The practical impact of clinical trials depends, among other things, on their external validity; that is, whether the results can be generalised to a broader group of patients. If those patients who refuse to participate therefore are different from those who agree to participate, the results cannot be generalised and the value of the clinical trial is limited. The first step in recruitment for a clinical trial is to find out the eligibility of the patient considered for study. When a patient fulfils the entry criteria of the trial, informed consent is sought. The patient is then told the nature of his or her condition and the available treatment options with their advantages and disadvantages. The patient is asked whether he or she agrees to be enrolled into the clinical trial and be randomised to treatment. Only those patients
PATIENTS AND METHODS This study was based on a consecutive series of patients aged between 16 and 69 years with unilateral symptoms of TMD. Patients with temporomandibular joint (TMJ) 129
130
British Journal of Oral and Maxillofacial Surgery Table 1 Classification of dysfunction of the temporomandibular joint (TMJ) without the use of radiographic evaluation
Table 2 Degree of pain at rest and interference with daily life as evaluated by the 20 patients using the 100 mm visual analogue scale Characteristics
Degree of dysfunction of TMJ No Slight Moderate Severe
Maximal mouth opening (mm)
Value of visual analogue scales
40 or more 35–39 30–34 29 or less
0 1–33 34–66 67–100
dysfunction were evaluated first at the Department of Oral and Maxillofacial Surgery in this hospital. These patients were classified into four groups according to our criteria (Table 1) and those whose symptoms were classified as moderate or severe were referred for further magnetic resonance (MR) examination. The patients diagnosed as having TMJ disc displacement without reduction on MR imaging were informed and recorded in the following RCT: no treatment group (control group), after having given informed consent patients were followed up without treatment for at least 4 weeks; treatment group (non-steroidal anti-inflammatory drugs (NSAIDs) and mouth opening exercise), NSAIDs (ampiroxicam 13.5 mg) were given orally once a day. Subjects were instructed to mouth opening exercises four times a day after each meal and during bathing. The NSAIDs and mouth opening exercises were continued for at least 4 weeks until all signs of TMD had changed to none or slight dysfunction (Table 1). All those who did not accept this RCT were considered to have refused. We estimated that the RCT would require 30 patients in each group when a two-sided test was applied to the data (α = 0.05, β = 0.2), so the trial ended when the number of patients randomised reached 60. At the end of RCT, 18 patients had refused to be enrolled in the RCT and were given no active treatment. It is ethically permissible to have a no treatment group in clinical research on TMD.5 A detailed description of the underlying study is published elsewhere.6 This study was approved by the Human Research Committee at our hospital, and the consent forms were written according to the Helsinki Declaration for Scientific Studies in Medicine. Statistical analysis The Mann–Whitney U-test with the stratification factors was used to compare the characteristics of the patients who agreed to participate with those of the group who refused. Probabilities of less than 0.05 using two-tailed tests were considered significant. The patients were initially evaluated for pain at rest and interference with their daily life secondary to the TMD using visual analogue scales on which they marked the de-
Pain at rest Interference with daily life
Accepted randomisation (n = 60)
Refused randomisation (n = 18)
P value
0 (0–12) 43 (20–60)
19 (0–50) 61 (49–77)
0.03 0.003
gree of pain and the degree of interference with daily life. Many other factors were also evaluated for the underlying study, but there were no other significant differences between the patients who agreed to participate in the study and those who refused. These other variables are not reported in this study.
RESULTS During the study period, 662 new patients were screened for the study and 78 patients fulfilled the inclusion criteria and were invited to participate. Sixty patients agreed to be randomised and were included in the study, whereas 18 patients refused to participate. The patients who refused to participate had more pain and reported more interference with their daily life than those who agreed to participate (Tables 1 and 2). The patients who agreed to participate in the study had median rest pain of 0, range 0–12, whereas the patients who refused to participate had a median of 19, range 0–50.
DISCUSSION This study showed that patients who refused to participate in a relatively short-term RCT had more pain at rest and more interference with their daily life than those who accepted to participate. The external validity of this RCT was, therefore, limited and the results may not be generalisable.7 It has been stated in analysis of this subject that the results of those who refused to participate in an RCT should be analysed, but in most papers they are not. It is particularly difficult in multicentre studies. The degree of difference between the patients who accepted and those who refused to participate in this clinical trial is remarkable, as it was a short-term study (4 weeks) testing a relatively non-invasive method of treatment. It is likely that clinical trials that persist over a longer period of time and involve more serious conditions, and more invasive methods of treatment, would result in an even greater difference between the patients who agreed to participate and those who refused. It is, therefore, an important part of every clinical trial to present the difference between those who agree and those who refuse to participate. Only after
External validity in clinical trial
this analysis has been presented can the generalisability of the results be judged. REFERENCES 1. Yuasa H, Yoda T et al. Consent for enrolment in randomised controlled trials: a questionnaire study. Asian J Oral Maxillofac Surg 2002; 14: 125–131. 2. Schmoor C, Olschewski M, Schumacher M. Randomised and non-randomised patients in clinical trials: experiences with comprehensive cohort studies. Stat Med 1996; 15: 263–271. 3. Olschewski M, Schumacher M, Davis KB. Analysis of randomised and nonrandomised patients in clinical trials using the comprehensive cohort follow-up study design. Control Clin Trials 1992; 13: 226–239. 4. Kurita K, Westesson P-L, Yuasa H, Toyama M, Machida J, Ogi N. Natural history of disc displacement without reduction. J Dent Res 1998; 77: 361–365. 5. Schiffman EL. The role of the randomised clinical trial in evaluating management strategies for temporomandibular disorders. In: Fricton JR, Dubner R, eds. Orofacial Pain and Temporomandibular Disorders (Advance in Pain Research and Therapy, Vol. 21). New York: Raven Press, 1995: 415–463. 6. Yuasa H, Kurita K for The Treatment Group on Temporomandibular Disorders, Nagoya, Japan. Randomised clinical trial of primary
131
treatment for temporomandibular joint disc displacement without reduction and without osseous changes: a combination of NSAIDs and mouth-opening exercise versus no treatment. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2001; 91: 671–675. 7. Yuasa H. The CONSORT statement: explanation and elaboration. Consolidated Standards of Reporting Trials. Ann Intern Med 2002; 136: 926–927.
The Authors H. Yuasa DDS, PhD The Second Department of Oral and Maxillofacial Surgery K. Kurita DDS, PhD The First Department of Oral and Maxillofacial Surgery, School of Dentistry, Aichi-Gakuin University, Japan P.-L. Westesson MD, PhD, DDS Division of Neuroradiololgy, Department of Radiology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA Correspondence and requests for offprints to: Hidemichi Yuasa DDS, PhD, 8U-102 room, a-banrafure hosigaoka, 1-23-4, Hosigaoka, Chikusa-ku, Nagoya 464-0801, Japan. Tel: +81 52 781 4045; E-mail: [email protected] Accepted 21 January 2003
SHORT COMMUNICATION External validity of a randomised clinical trial of temporomandibular disorders: analysis of the patients who refused to participate in research H. Yuasa, ∗ K. Kurita, † P.-L. Westesson ‡ ∗ The
Second Department of Oral and Maxillofacial Surgery; †The First Department of Oral and Maxillofacial Surgery, School of Dentistry, Aichi-Gakuin University, Japan; ‡Division of Neuroradiololgy, Department of Radiology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA SUMMARY. Purpose: To assess the external validity of a randomised clinical trial (RCT) of a painful condition. Method: Consecutive patients with painful temporomandibular disorders (TMDs) were invited to participate in a clinical trial. Patients who refused to participate were compared to those who agreed to participate in this study with respect to degree of symptoms at time of presentation. Results: The patients who refused to participate had more pain, and their condition interfered more with their daily life than those who accepted the invitation to participate. Conclusion: Selection bias in RCTs of painful conditions can skew the results, and external validity should be analysed before the results are generalised. © 2003 The British Association of Oral and Maxillofacial Surgeons. Published by Elsevier Science Ltd. All rights reserved.
who agree to be randomised are enrolled. Other eligible patients are dropped and usually not observed further.3 Differences between patients who are enrolled and those who refuse to participate may limit the generalisability of the results. Temporomandibular disorders (TMDs) are common, painful and affect up to 20% of the population. Many treatments have been attempted but there is not a single, generally accepted treatment, that has been shown to be effective over untreated controls. Further, the natural course of one of the most commonest form of TMD, namely, disc displacement without reduction, has proved to be favourable with around two-thirds being free of symptoms or considerable improved after 2 years without treatment.4 In this study, we analysed the differences between those who agreed to participate in an RCT for treatment of painful TMD and compared them with those who refused to be randomised. To elucidate the external validity and generalisability of the results, we analysed the difference between patients who accepted the process and those who refused to be randomised.
INTRODUCTION Clinical trials are essential to establish evidence-based medical treatment. The randomised controlled trial (RCT) is viewed as the best study design because of its internal validity, which assures comparability among selected regimens. However, when the enrolment rate of participants is low and there is the difference between patients who accepted the process, the external validity of the extrapolation of the findings to the target population is not guaranteed.1 It is the purpose of RCTs to create groups of patients who are balanced with respect to known and unknown characteristics.2 The practical impact of clinical trials depends, among other things, on their external validity; that is, whether the results can be generalised to a broader group of patients. If those patients who refuse to participate therefore are different from those who agree to participate, the results cannot be generalised and the value of the clinical trial is limited. The first step in recruitment for a clinical trial is to find out the eligibility of the patient considered for study. When a patient fulfils the entry criteria of the trial, informed consent is sought. The patient is then told the nature of his or her condition and the available treatment options with their advantages and disadvantages. The patient is asked whether he or she agrees to be enrolled into the clinical trial and be randomised to treatment. Only those patients
PATIENTS AND METHODS This study was based on a consecutive series of patients aged between 16 and 69 years with unilateral symptoms of TMD. Patients with temporomandibular joint (TMJ) 129
130
British Journal of Oral and Maxillofacial Surgery Table 1 Classification of dysfunction of the temporomandibular joint (TMJ) without the use of radiographic evaluation
Table 2 Degree of pain at rest and interference with daily life as evaluated by the 20 patients using the 100 mm visual analogue scale Characteristics
Degree of dysfunction of TMJ No Slight Moderate Severe
Maximal mouth opening (mm)
Value of visual analogue scales
40 or more 35–39 30–34 29 or less
0 1–33 34–66 67–100
dysfunction were evaluated first at the Department of Oral and Maxillofacial Surgery in this hospital. These patients were classified into four groups according to our criteria (Table 1) and those whose symptoms were classified as moderate or severe were referred for further magnetic resonance (MR) examination. The patients diagnosed as having TMJ disc displacement without reduction on MR imaging were informed and recorded in the following RCT: no treatment group (control group), after having given informed consent patients were followed up without treatment for at least 4 weeks; treatment group (non-steroidal anti-inflammatory drugs (NSAIDs) and mouth opening exercise), NSAIDs (ampiroxicam 13.5 mg) were given orally once a day. Subjects were instructed to mouth opening exercises four times a day after each meal and during bathing. The NSAIDs and mouth opening exercises were continued for at least 4 weeks until all signs of TMD had changed to none or slight dysfunction (Table 1). All those who did not accept this RCT were considered to have refused. We estimated that the RCT would require 30 patients in each group when a two-sided test was applied to the data (α = 0.05, β = 0.2), so the trial ended when the number of patients randomised reached 60. At the end of RCT, 18 patients had refused to be enrolled in the RCT and were given no active treatment. It is ethically permissible to have a no treatment group in clinical research on TMD.5 A detailed description of the underlying study is published elsewhere.6 This study was approved by the Human Research Committee at our hospital, and the consent forms were written according to the Helsinki Declaration for Scientific Studies in Medicine. Statistical analysis The Mann–Whitney U-test with the stratification factors was used to compare the characteristics of the patients who agreed to participate with those of the group who refused. Probabilities of less than 0.05 using two-tailed tests were considered significant. The patients were initially evaluated for pain at rest and interference with their daily life secondary to the TMD using visual analogue scales on which they marked the de-
Pain at rest Interference with daily life
Accepted randomisation (n = 60)
Refused randomisation (n = 18)
P value
0 (0–12) 43 (20–60)
19 (0–50) 61 (49–77)
0.03 0.003
gree of pain and the degree of interference with daily life. Many other factors were also evaluated for the underlying study, but there were no other significant differences between the patients who agreed to participate in the study and those who refused. These other variables are not reported in this study.
RESULTS During the study period, 662 new patients were screened for the study and 78 patients fulfilled the inclusion criteria and were invited to participate. Sixty patients agreed to be randomised and were included in the study, whereas 18 patients refused to participate. The patients who refused to participate had more pain and reported more interference with their daily life than those who agreed to participate (Tables 1 and 2). The patients who agreed to participate in the study had median rest pain of 0, range 0–12, whereas the patients who refused to participate had a median of 19, range 0–50.
DISCUSSION This study showed that patients who refused to participate in a relatively short-term RCT had more pain at rest and more interference with their daily life than those who accepted to participate. The external validity of this RCT was, therefore, limited and the results may not be generalisable.7 It has been stated in analysis of this subject that the results of those who refused to participate in an RCT should be analysed, but in most papers they are not. It is particularly difficult in multicentre studies. The degree of difference between the patients who accepted and those who refused to participate in this clinical trial is remarkable, as it was a short-term study (4 weeks) testing a relatively non-invasive method of treatment. It is likely that clinical trials that persist over a longer period of time and involve more serious conditions, and more invasive methods of treatment, would result in an even greater difference between the patients who agreed to participate and those who refused. It is, therefore, an important part of every clinical trial to present the difference between those who agree and those who refuse to participate. Only after
External validity in clinical trial
this analysis has been presented can the generalisability of the results be judged. REFERENCES 1. Yuasa H, Yoda T et al. Consent for enrolment in randomised controlled trials: a questionnaire study. Asian J Oral Maxillofac Surg 2002; 14: 125–131. 2. Schmoor C, Olschewski M, Schumacher M. Randomised and non-randomised patients in clinical trials: experiences with comprehensive cohort studies. Stat Med 1996; 15: 263–271. 3. Olschewski M, Schumacher M, Davis KB. Analysis of randomised and nonrandomised patients in clinical trials using the comprehensive cohort follow-up study design. Control Clin Trials 1992; 13: 226–239. 4. Kurita K, Westesson P-L, Yuasa H, Toyama M, Machida J, Ogi N. Natural history of disc displacement without reduction. J Dent Res 1998; 77: 361–365. 5. Schiffman EL. The role of the randomised clinical trial in evaluating management strategies for temporomandibular disorders. In: Fricton JR, Dubner R, eds. Orofacial Pain and Temporomandibular Disorders (Advance in Pain Research and Therapy, Vol. 21). New York: Raven Press, 1995: 415–463. 6. Yuasa H, Kurita K for The Treatment Group on Temporomandibular Disorders, Nagoya, Japan. Randomised clinical trial of primary
131
treatment for temporomandibular joint disc displacement without reduction and without osseous changes: a combination of NSAIDs and mouth-opening exercise versus no treatment. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2001; 91: 671–675. 7. Yuasa H. The CONSORT statement: explanation and elaboration. Consolidated Standards of Reporting Trials. Ann Intern Med 2002; 136: 926–927.
The Authors H. Yuasa DDS, PhD The Second Department of Oral and Maxillofacial Surgery K. Kurita DDS, PhD The First Department of Oral and Maxillofacial Surgery, School of Dentistry, Aichi-Gakuin University, Japan P.-L. Westesson MD, PhD, DDS Division of Neuroradiololgy, Department of Radiology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA Correspondence and requests for offprints to: Hidemichi Yuasa DDS, PhD, 8U-102 room, a-banrafure hosigaoka, 1-23-4, Hosigaoka, Chikusa-ku, Nagoya 464-0801, Japan. Tel: +81 52 781 4045; E-mail: [email protected] Accepted 21 January 2003