- Email: [email protected]
Randomized controlled trials: the control group dilemma revisited
Trial Methodology
Randomized controlled trials: the control group dilemma revisited A. Hart Statistics Group, Faculty of Science, University of Central Lancashire, Preston, UK
SUMMARY. In some randomised controlled trials the nature of the therapy means that subjects cannot or should not be blinded. Such studies need careful design. Particular attention needs to be given to the choice of control group and the nature of the informed consent obtained from subjects, because these affect the precise research question being addressed.A survey of published studies was carried to investigate how these issues had been tackled.The paper summarizes key findings from the survey. If the research question is about the specific effect of a therapy sometimes a good case can be made for a second control group which is ‘attention-controlled’.There is a need for more detailed justifications of such design decisions in published studies. © 2001 Harcourt Publishers Ltd
INTRODUCTION
Anna Hart, Statistics Group, Faculty of Science, University of Central Lancashire, Preston PR1 2HE, UK.Tel.: +44 1772 893 732; fax: + 44 1772 892 996; E-mail [email protected]
therefore, seek out the therapy themselves. Moreover, a clinician may be unwilling to administer a ‘fraudulent control condition’ which misrepresents the treatment. The choice of successful control involves ‘distinguishing between nonspecific factors and unspecified therapeutic ingredients. Nonspecific factors....can include social support, a therapeutic alliance, a plausible explanation for the client’s problem, and a credible treatment rationale’. It is often assumed that, where possible, a RCT should be double-blind. However, the very nature of some therapies means that blinding of subjects is impossible, or even undesirable. A principal reason for subject-blinding is to control for psychological components of the placebo effect, which are seen as confounds for treatment effects. The ‘true treatment effect’ is then defined as the ‘perceived treatment...minus non-specific effects’.5 Some researchers regard this as naïve, because psychological effects can be an important components of overall treatment effects. Furthermore, the size of specific treatment effects might depend on non-specific factors such as therapist, clinical setting, patient’s beliefs and information given.6,7 A study that minimized placebo effects might, therefore, bear little resemblance to clinical practice where placebos may well be maximized. The aim of a study is often to quantify the ‘effectiveness of therapy X’. However, this is achieved
The randomized controlled trial (RCT) is generally regarded as the most rigorous and methodologically pure method for evaluation of therapies, but behavioural medicine can present a challenge for trial designers. Black1 identifies several problems of RCTs including the self-defeating nature of a RCT if the effectiveness of the intervention depends on the subject’s beliefs and preferences, and the practical difficulties of maintaining a prolonged prospective study. In the context of nutritional epidemiology, Willett2 comments on the potential long duration of RCTs, problems with compliance over the duration of the study, and ‘contamination’ between treatment and control groups. They both suggest that rigorous observational studies might be considered where the gold standard can cause too many problems in a clinical setting. Cleophas et al.3 make a similar point in a discussion of problems associated with a placebo control group. Schwartz et al.4 discuss ethical and scientific issues concerned with the choice of a control group in psychosocial and behavioural medicine. As they state, ‘the rationale for placebo control conditions is to protect against the perpetuation of spurious or superstitious therapy procedures’. However, problems can be caused by the resentful demoralization of control group subjects who know that they are not receiving the therapy of interest, and who may,
Complementary Therapies in Medicine (2001), 9, 40–44 © 2001 Harcourt Publishers Ltd doi: 10.1054/ctim.2000.0414, available online at http://www.idealibrary.com on
40
RCTs: the control group dilemma revisited
under a set of defined conditions, including the information given to the subjects, and normally in comparison with either a control group or with an alternative therapy. The specific research question being asked, therefore, depends on the nature of the control group and on the knowledge and beliefs of the subjects. The latter factor is related to the issue of informed consent, which is an area of some contention.8,9 For example, if subjects knew that they were in a control group, and fully understood the role of such a group, they may respond differently than if they had received exactly the same treatment but without that knowledge and ensuing beliefs. The choice of control group is, therefore, extremely important. Changing the choice of control group or the nature of the informed consent will change the research question being addressed and hence also the information acquired about the therapy being studied.7 This issue is very important for researchers. It is also important for clinicians. Evidence-based medicine requires practitioners to critically appraise the evidence presented in papers. They cannot do this effectively unless they have a clear understanding of the research question being addressed, and this in turn requires an understanding of the role of the control group. The difficulties faced in behavioural medicine are likely to plague alternative and complementary medicine. This paper describes the results of a brief survey of controlled trials where such difficulties have been faced, and discusses some of the issues raised. The specific question addressed was ‘In RCTs where subject-blinding was impossible, and where some outcome measures were subjective, what were the choices of control group?’
METHODS A survey of published studies was carried out. This comprised a hand search of papers published between 1997 and 1999 in the following journals: Journal of Human Nutrition and Dietetics, American Journal of Clinical Nutrition, British Journal of Nutrition, British Journal of Rheumatology, Rheumatology, Arthritis and Rheumatism, British Journal of Health Psychology, British Journal of Clinical Psychology and Journal of Psychosomatic Research (Disability and Rehabilitation was also searched but no papers meeting the criteria were found). Criteria for selection were a randomized study comparing a ‘treatment’ with at least one other treatment or placebo, where the randomization unit was the subject (i.e. not hospital, ward or town), the impossibility of blinding of subjects to the treatment they were receiving, and at least one psychological or subjective outcome measure. This comprised a ‘convenience sample’, covering a range of areas.
41
RESULTS A total of 22 descriptions of studies was found. The papers are summarized in Table 1. Eleven of the studies investigated the treatment as an adjunct to usual care,10–20 three compared the treatment with usual care or waitlist controls;21–23 two compared the treatment with another therapy,24,25 four had an ‘attention-controlled’ comparison group,26–29 and the remaining two compared more than two groups30,31 (an attention-controlled group is one that is designed to receive the same amount of attention and social contact as the treatment group, but without the treatment under investigation). This classification is approximate but generally reflects a spectrum ranging from studies of the total effect of a therapy or intervention to investigation of specific non-psychological treatment effects. The table also indicates whether or not the paper makes a clear statement about informed consent being obtained from all subjects, and highlights other issues identified in the papers. It could be argued that sham acupuncture procedures do exist, and so the study of acupuncture15 does not meet the inclusion criteria. However, these researchers were not convinced of the existence of a suitable sham treatment, believing that ‘a standardized reliable model has not been developed to date’. This study of acupuncture (unlike some others) is, therefore, included. One study of chronic fatigue syndrome11 specifically noted that a longer term study was needed to determine whether apparently beneficial effects of self help were maintained in the long term. Many studies used objective as well as subjective outcome measures and one study mentioned this as a deliberate design strategy.29 It is possible that treatment effects were confounded with centre or therapist effects in a study of cognitive-behaviour therapy for pain.31 Possible confounds or contamination were noted in other studies. In particular in a study of spa therapy for osteoarthritis the authors say:22 . . . the inevitable more frequent contact of the spa patients with spa staff could also introduce a bias since such contact could well result in additional explanation and education. Therefore such attention by spa medical staff could be considered as an individual component of the spa therapy which could contribute to improvement. This study is, therefore, a pragmatic one which studied the ‘spa treatment’ rather than the ‘spa water’. In a study of feeding support for the elderly in a hospital ward, the possibility of contamination was admitted.12 The authors also noted the multifaceted nature of the treatment, but comment, ‘at this stage we plan to determine whether feeding support improves nutritional intake and whether this affects outcome, rather than focus on which components of feeding support are involved’. The difficulty
Unhealthy diet Chronic Fatigue Syndrome
Malnutrition in acutely ill older in-patients Chronic myositis Rheumatoid arthritis
Osteoarthritis of the knee Rheumatoid arthritis
High-risk parasuicide
Psychosis Post angioplasty Chronic heart failure Rheumatoid arthritis Osteoarthritis Pre-menstrual Syndrome
Agoraphobia
Chronic fatigue syndrome Poor performance/concentration at school Osteoarthritis of the knee
Post-operative analgesia Challenging behaviour
Obesity/unhealthy diet Sickle cell disease pain
12
13 14
15 16
17
18 19 20 21 22 23
24
25 26
27
28 29
30 31
Problem
10 11
Paper
Dietary advice Cognitive behaviour therapy
Tape for relaxation Multi-sensory environment
Self-care education
Cognitive behaviour therapy Breakfast
Cognitive-behaviour therapy
Manual assisted cognitive-behaviour therapy Cognitive-behaviour therapy plus standard care Behaviourally oriented programme Exercise In-patient multi-disciplinary team care Spa therapy Cognitive therapy
Acupuncture Educational programme
Training programme Leaflet
Feeding support
Educational intervention Self help booklet and advice
Research intervention
?? Yes
Partly Yes
Yes
?? Yes
??
?? Yes Yes Yes Yes Yes
??
Yes Yes
Yes ??
Yes
Partly Yes
Informed consent
Attention control tape of information Non-complex sensory environment matched for social contact and attention Alternative advice Or No advice Attention control Or Nothing
Attention controlled public education
Relaxation Snack plus attention control
Exposure
Usual care Usual care Usual care Routine out-patient care Usual treatment Waitlist controls
Usual care
Usual care Usual care
Usual way of life Usual care
Usual care
Nothing No treatment
Control groups
Require replication with no-intervention control. Consent from parents or primary caregivers Crossover design Problems of self-reporting noted Effects probably confounded with centre/therapist effects.
Informed consent from parents Control group was not a true placebo.
Need for a 3rd group Need to compare CBT with an equally credible psychological therapy e.g. relaxation Both groups had same amount of exposure. If exposure worked well then there is no scope to show differential CBT effect.
Treatment group had more attention – an extra phone call. Need for placebo control group Controls were told that they could have the intervention, if useful, after the study. Control group of normal volunteers included – no intervention
Subjects not informed of real objective of study Need to see if results are maintained in the longterm Contamination between groups likely Need for 3rd group – attention-controlled
Comments
Table 1 Summary of studies found in the survey. For informed consent Yes means the paper states that informed consent was obtained, ?? means that no such statement was found, and Partly means that consent was not fully informed.
42 Complementary Therapies in Medicine
RCTs: the control group dilemma revisited
of teasing out the specific effective components of a treatment was noted elsewhere. For example, in the comparison of acupuncture and normal care,15 the authors make the following observation: This adjunctive design, however, may not totally control for the placebo effect and, therefore, cannot conclusively determine how much improvement is attributable to the acupuncture treatment versus the effects that stem from the entire treatment experience. The authors describe the research question as an ‘intermediate’ one. It is suggested here, and elsewhere12,22,23,28 that a third group or further study would be needed to study fully the treatment effect. In the study of exercise for chronic heart failure20 the authors note that it ‘could not be ruled out that results obtained were due to non-specific treatment factors’. Fourteen studies reported that informed consent had been obtained from the subjects or, where appropriate, the adults responsible for them.11–13,15,16,19–23,26,27,29,31 In one study about the effect of an information leaflet14 and in four psychologically oriented studies it was not clear that informed consent had been obtained.17,18,24,25 Moreover, in a study of dietary advice30 ‘volunteers were asked to take part in a Healthy Eating study’, the implication being that subjects were not fully informed about the nature and purpose of the study. This is explicitly stated in the report of a different study of diet10 where ‘before intervention subjects were not informed that the study focused on fruit and vegetables, and control subjects were only informed after the study was completed’. A similar strategy may have been used in a study of pre-operative relaxation28 since consent was described as ‘for participation in a study of whether listening to a tape before surgery would help patients to recover’.
43
Moreover, for these designs, it is useful if the effectiveness of the other treatment has already been established. The nature of informed consent appears to vary considerably from study to study. It is not helpful merely to report that ‘informed consent was obtained from all subjects’. Without details of precisely what the subjects were told and when, it is impossible precisely to define the research question being asked, and to replicate the study, or assess its clinical relevance. There seems to be a case, in some studies, for informed consent not being ‘fully informed’. Subjects should be told as much as possible, but not information that would invalidate the study. Where subject blinding is not feasible it is important to establish whether non-specific effects are possible confounds or an integral part of the treatment being studied. This will help to determine whether there is a need for placebo control, possibly including attention control. In some studies three study groups are worth considering, provided resources allow this and there are no ethical objections. There are likely to be several aspects to a therapy or treatment, only some of which can be investigated in a single study. Moreover, there are different types of question that one can ask about a therapy. Researchers should be fully aware of which questions they are trying to answer, which they are ignoring, and why. For effective evidence-based medicine all such reasoning should be presented in papers.
ACKNOWLEDGEMENTS The author is grateful to the anonymous referees whose constructive criticism has improved the quality of this paper. Views expressed are those of the author.
REFERENCES
DISCUSSION As a general rule, although not always stated explicitly, the ultimate aim of most of the researchers cited here appeared to be an investigation of specific treatment effects. The more pragmatic designs were often seen as a step in this direction. As a general principle, however, more precise statements of the research questions and more detailed justifications of the designs would have been helpful. There are two reasons for this. Firstly, it is necessary to establish the exact objective and to assess the integrity of the study, and secondly it helps other researchers with their designs. When the therapy is being compared with an alternative treatment it is important to decide whether the aim is to show the superiority, noninferiority or the equivalence of the new treatment, since the latter cases might require a larger study.3
1. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312: 1215–1218. 2. Willett WC. Nutritional Epidemiology. In: Rothman KJ, Greenland S (eds) Modern Epidemiology. Philadelphia: Lippincott-Raven, 1998. 3. Cleophas TJM, Meulen Jvd, Kalmansohn RB. Clinical Trials: specific problems associated with the use of a placebo control group. British Journal of Clinical Pharmacology 1997; 43: 219–221. 4. Schwartz CE, Chesney MA, Irvine MJ, Keefe FJ. The Control Group Dilemma in Clinical Research: Applications for Psychosocial and Behavioural Medicine. Psychosomatic Medicine 1997; 59: 362–371. 5. Ernst E, Resch KL. Concept of true and perceived placebo effects. BMJ 1995; 311: 511–553. 6. Kleijnen J, de Craen AJ, van Everdingen J, Krol L. Placebo effects in double-blind clinical trials: a review of interactions with medications. Lancet 1994; 344: 1347–1349. 7. Vickers AJ, de Craen AJ. Why use placebos in clinical trials? A narrative review of the methodological literature. Journal of Clinical Epidemiology 2000; 53(2) 157–161.
44
Complementary Therapies in Medicine
8. Letters. Informed consent. BMJ 1997; 314: 1477–1483. 9. Letters. Informed Consent. BMJ 1997; 315: 247 10. Cox DN, Anderson AS, Reynolds J et al. Take Five, a nutrition education intervention to increase fruit and vegetable intakes: impact on consumer choice and nutrient intakes. British Journal of Nutrition 1998; 80: 123–131. 11. Chalder T, Wallace P, Wessely S. Self-help treatment of chronic fatigue in the community; A randomized controlled trial. British Journal of Health Psychology 1997; 2: 189–197. 12. Hickson M, Nicholl C, Bulpitt C et al. The Design of the Feeding Support Trial – does intensive feeding support improve nutritional status and outcome in acutely ill older in-patients? Journal of Human Nutrition and Dietetics 1999; 12: 53–59. 13. Wiesinger GF, Quittan M, Aringer M et al. Improvement of physical fitness and muscle strength in polymyositis/dermatomyositis patients by a training programme. British Journal of Rheumatology 1998; 37: 196–200. 14. Barlow JH, Wright CC. Knowledge in patients with rheumatoid arthritis: a longer term follow-up of a randomized controlled study of patient education leaflets. British Journal of Rheumatology 1998; 37: 373–376. 15. Berman BM, Singh BB, Lao L et al. A randomized trial of acupuncture as an adjunctive therapy in osteoarthritis of the knee. Rheumatology 1999; 38: 346–354. 16. Helliwell PS, O’Hara M, Holdsworth J et al. A 12month randomized controlled trial of patient education on radiographic changes and quality of life in early rheumatoid arthritis. Rheumatology 1999; 38: 303–308. 17. MacLeod AK, Tata P, Evans K et al. Recovery of positive future thinking within a high-risk parasuicide group: Results from a pilot randomized controlled trial. British Journal of Clinical Psychology 1998; 37: 371–379. 18. Freeman D, Garety P, Fowler D et al. The London-East Anglia randomized controlled trial of cognitivebehaviour therapy for pyschosis IV: Self-esteem and persecutory delusions. British Journal of Clinical Psychology 1998; 37: 415–430. 19. Lisspers J, Sundin O, Hofman-Bang C et al. Behavioural effects of a comprehensive, multifactorial program for lifestyle change after percutaneous transluminal coronary angioplasty: a prospective, randomized, controlled study. Journal of Psychosomatic Research 1999; 46(2): 143–154.
20. Weilinga RP, Erdman RAM, Huisveld IA et al. Effect of exercise training on quality of life in patients with chronic heart failure. Journal of Psychosomatic Research 1998; 45(5): 459–464. 21. Vlieland TPMV, Breedveld FC, Hazes JMW. The twoyear follow-up of a randomized comparison of in-patient multidisciplinary team care and routine out-patient care for active rheumatoid arthritis. British Journal of Rheumatology 1997; 36: 82–85. 22. Nguyen M, Revel M, Dougados M. Prolonged effects of 3 week therapy in a spa resort on lumbar spine, knee and hip osteoarthritis: follow-up after 6 months. A randomized controlled trial. British Journal of Rheumatology 1997; 36: 77–81. 23. Blake F, Salkovskis P, Gath D, Day A, Garrod A. Cognitive therapy for premenstrual syndrome: a controlled study. Journal of Psychosomatic Research 1998; 45(4): 307–318. 24. Burke M, Drummond LM, Johnston DW. Treatment for agoraphobic women: Exposure or cognitive-behaviour therapy? British Journal of Clinical Psychology 1997; 36: 409–420. 25. Deale A, Chalder T, Wessely S. Illness beliefs and treatment outcome in chronic fatigue syndrome. Journal of Psychosomatic Research 1998; 45(1): 77–83. 26. Powell CA, Walker SP, Chang SM, GranthamMcGregor SM. Nutrition and education: a randomized trial of the effects of breakfast in rural primary school children. American Journal of Clinical Nutrition 1998; 68: 873–9. 27. Mazzuca SA, Brandt KD, Katz BP et al. Effects of selfcare education on the health status of inner-city patients with osteoarthritis of the knee. Arthritis and Rheumatism 1997; 40(8): 1466–1474. 28. Manyande A, Salmon P. Effects of pre-operative relaxation on post-operative analgesia: Immediate increase and delayed reaction. British Journal of Health Psychology 1998; 3: 215–224. 29. Martin NT, Gaffan EA, Williams T. Behavioural effects of long-term multi-sensory stimulation. British Journal of Clinical Psychology 1998; 37: 69–82. 30. Drummond S, Kirk T. The effect of different types of dietary advice on body composition in a group of Scottish men. Journal of Human Nutrition and Dietetics 1998; 11: 473–485. 31. Thomas VJ, Dixon AL, Milligan P. Cognitive-behaviour therapy for the management of sickle cell disease pain: An evaluation of a community-based intervention. British Journal of Health Psychology 1999; 4: 209–229.
Randomized controlled trials: the control group dilemma revisited A. Hart Statistics Group, Faculty of Science, University of Central Lancashire, Preston, UK
SUMMARY. In some randomised controlled trials the nature of the therapy means that subjects cannot or should not be blinded. Such studies need careful design. Particular attention needs to be given to the choice of control group and the nature of the informed consent obtained from subjects, because these affect the precise research question being addressed.A survey of published studies was carried to investigate how these issues had been tackled.The paper summarizes key findings from the survey. If the research question is about the specific effect of a therapy sometimes a good case can be made for a second control group which is ‘attention-controlled’.There is a need for more detailed justifications of such design decisions in published studies. © 2001 Harcourt Publishers Ltd
INTRODUCTION
Anna Hart, Statistics Group, Faculty of Science, University of Central Lancashire, Preston PR1 2HE, UK.Tel.: +44 1772 893 732; fax: + 44 1772 892 996; E-mail [email protected]
therefore, seek out the therapy themselves. Moreover, a clinician may be unwilling to administer a ‘fraudulent control condition’ which misrepresents the treatment. The choice of successful control involves ‘distinguishing between nonspecific factors and unspecified therapeutic ingredients. Nonspecific factors....can include social support, a therapeutic alliance, a plausible explanation for the client’s problem, and a credible treatment rationale’. It is often assumed that, where possible, a RCT should be double-blind. However, the very nature of some therapies means that blinding of subjects is impossible, or even undesirable. A principal reason for subject-blinding is to control for psychological components of the placebo effect, which are seen as confounds for treatment effects. The ‘true treatment effect’ is then defined as the ‘perceived treatment...minus non-specific effects’.5 Some researchers regard this as naïve, because psychological effects can be an important components of overall treatment effects. Furthermore, the size of specific treatment effects might depend on non-specific factors such as therapist, clinical setting, patient’s beliefs and information given.6,7 A study that minimized placebo effects might, therefore, bear little resemblance to clinical practice where placebos may well be maximized. The aim of a study is often to quantify the ‘effectiveness of therapy X’. However, this is achieved
The randomized controlled trial (RCT) is generally regarded as the most rigorous and methodologically pure method for evaluation of therapies, but behavioural medicine can present a challenge for trial designers. Black1 identifies several problems of RCTs including the self-defeating nature of a RCT if the effectiveness of the intervention depends on the subject’s beliefs and preferences, and the practical difficulties of maintaining a prolonged prospective study. In the context of nutritional epidemiology, Willett2 comments on the potential long duration of RCTs, problems with compliance over the duration of the study, and ‘contamination’ between treatment and control groups. They both suggest that rigorous observational studies might be considered where the gold standard can cause too many problems in a clinical setting. Cleophas et al.3 make a similar point in a discussion of problems associated with a placebo control group. Schwartz et al.4 discuss ethical and scientific issues concerned with the choice of a control group in psychosocial and behavioural medicine. As they state, ‘the rationale for placebo control conditions is to protect against the perpetuation of spurious or superstitious therapy procedures’. However, problems can be caused by the resentful demoralization of control group subjects who know that they are not receiving the therapy of interest, and who may,
Complementary Therapies in Medicine (2001), 9, 40–44 © 2001 Harcourt Publishers Ltd doi: 10.1054/ctim.2000.0414, available online at http://www.idealibrary.com on
40
RCTs: the control group dilemma revisited
under a set of defined conditions, including the information given to the subjects, and normally in comparison with either a control group or with an alternative therapy. The specific research question being asked, therefore, depends on the nature of the control group and on the knowledge and beliefs of the subjects. The latter factor is related to the issue of informed consent, which is an area of some contention.8,9 For example, if subjects knew that they were in a control group, and fully understood the role of such a group, they may respond differently than if they had received exactly the same treatment but without that knowledge and ensuing beliefs. The choice of control group is, therefore, extremely important. Changing the choice of control group or the nature of the informed consent will change the research question being addressed and hence also the information acquired about the therapy being studied.7 This issue is very important for researchers. It is also important for clinicians. Evidence-based medicine requires practitioners to critically appraise the evidence presented in papers. They cannot do this effectively unless they have a clear understanding of the research question being addressed, and this in turn requires an understanding of the role of the control group. The difficulties faced in behavioural medicine are likely to plague alternative and complementary medicine. This paper describes the results of a brief survey of controlled trials where such difficulties have been faced, and discusses some of the issues raised. The specific question addressed was ‘In RCTs where subject-blinding was impossible, and where some outcome measures were subjective, what were the choices of control group?’
METHODS A survey of published studies was carried out. This comprised a hand search of papers published between 1997 and 1999 in the following journals: Journal of Human Nutrition and Dietetics, American Journal of Clinical Nutrition, British Journal of Nutrition, British Journal of Rheumatology, Rheumatology, Arthritis and Rheumatism, British Journal of Health Psychology, British Journal of Clinical Psychology and Journal of Psychosomatic Research (Disability and Rehabilitation was also searched but no papers meeting the criteria were found). Criteria for selection were a randomized study comparing a ‘treatment’ with at least one other treatment or placebo, where the randomization unit was the subject (i.e. not hospital, ward or town), the impossibility of blinding of subjects to the treatment they were receiving, and at least one psychological or subjective outcome measure. This comprised a ‘convenience sample’, covering a range of areas.
41
RESULTS A total of 22 descriptions of studies was found. The papers are summarized in Table 1. Eleven of the studies investigated the treatment as an adjunct to usual care,10–20 three compared the treatment with usual care or waitlist controls;21–23 two compared the treatment with another therapy,24,25 four had an ‘attention-controlled’ comparison group,26–29 and the remaining two compared more than two groups30,31 (an attention-controlled group is one that is designed to receive the same amount of attention and social contact as the treatment group, but without the treatment under investigation). This classification is approximate but generally reflects a spectrum ranging from studies of the total effect of a therapy or intervention to investigation of specific non-psychological treatment effects. The table also indicates whether or not the paper makes a clear statement about informed consent being obtained from all subjects, and highlights other issues identified in the papers. It could be argued that sham acupuncture procedures do exist, and so the study of acupuncture15 does not meet the inclusion criteria. However, these researchers were not convinced of the existence of a suitable sham treatment, believing that ‘a standardized reliable model has not been developed to date’. This study of acupuncture (unlike some others) is, therefore, included. One study of chronic fatigue syndrome11 specifically noted that a longer term study was needed to determine whether apparently beneficial effects of self help were maintained in the long term. Many studies used objective as well as subjective outcome measures and one study mentioned this as a deliberate design strategy.29 It is possible that treatment effects were confounded with centre or therapist effects in a study of cognitive-behaviour therapy for pain.31 Possible confounds or contamination were noted in other studies. In particular in a study of spa therapy for osteoarthritis the authors say:22 . . . the inevitable more frequent contact of the spa patients with spa staff could also introduce a bias since such contact could well result in additional explanation and education. Therefore such attention by spa medical staff could be considered as an individual component of the spa therapy which could contribute to improvement. This study is, therefore, a pragmatic one which studied the ‘spa treatment’ rather than the ‘spa water’. In a study of feeding support for the elderly in a hospital ward, the possibility of contamination was admitted.12 The authors also noted the multifaceted nature of the treatment, but comment, ‘at this stage we plan to determine whether feeding support improves nutritional intake and whether this affects outcome, rather than focus on which components of feeding support are involved’. The difficulty
Unhealthy diet Chronic Fatigue Syndrome
Malnutrition in acutely ill older in-patients Chronic myositis Rheumatoid arthritis
Osteoarthritis of the knee Rheumatoid arthritis
High-risk parasuicide
Psychosis Post angioplasty Chronic heart failure Rheumatoid arthritis Osteoarthritis Pre-menstrual Syndrome
Agoraphobia
Chronic fatigue syndrome Poor performance/concentration at school Osteoarthritis of the knee
Post-operative analgesia Challenging behaviour
Obesity/unhealthy diet Sickle cell disease pain
12
13 14
15 16
17
18 19 20 21 22 23
24
25 26
27
28 29
30 31
Problem
10 11
Paper
Dietary advice Cognitive behaviour therapy
Tape for relaxation Multi-sensory environment
Self-care education
Cognitive behaviour therapy Breakfast
Cognitive-behaviour therapy
Manual assisted cognitive-behaviour therapy Cognitive-behaviour therapy plus standard care Behaviourally oriented programme Exercise In-patient multi-disciplinary team care Spa therapy Cognitive therapy
Acupuncture Educational programme
Training programme Leaflet
Feeding support
Educational intervention Self help booklet and advice
Research intervention
?? Yes
Partly Yes
Yes
?? Yes
??
?? Yes Yes Yes Yes Yes
??
Yes Yes
Yes ??
Yes
Partly Yes
Informed consent
Attention control tape of information Non-complex sensory environment matched for social contact and attention Alternative advice Or No advice Attention control Or Nothing
Attention controlled public education
Relaxation Snack plus attention control
Exposure
Usual care Usual care Usual care Routine out-patient care Usual treatment Waitlist controls
Usual care
Usual care Usual care
Usual way of life Usual care
Usual care
Nothing No treatment
Control groups
Require replication with no-intervention control. Consent from parents or primary caregivers Crossover design Problems of self-reporting noted Effects probably confounded with centre/therapist effects.
Informed consent from parents Control group was not a true placebo.
Need for a 3rd group Need to compare CBT with an equally credible psychological therapy e.g. relaxation Both groups had same amount of exposure. If exposure worked well then there is no scope to show differential CBT effect.
Treatment group had more attention – an extra phone call. Need for placebo control group Controls were told that they could have the intervention, if useful, after the study. Control group of normal volunteers included – no intervention
Subjects not informed of real objective of study Need to see if results are maintained in the longterm Contamination between groups likely Need for 3rd group – attention-controlled
Comments
Table 1 Summary of studies found in the survey. For informed consent Yes means the paper states that informed consent was obtained, ?? means that no such statement was found, and Partly means that consent was not fully informed.
42 Complementary Therapies in Medicine
RCTs: the control group dilemma revisited
of teasing out the specific effective components of a treatment was noted elsewhere. For example, in the comparison of acupuncture and normal care,15 the authors make the following observation: This adjunctive design, however, may not totally control for the placebo effect and, therefore, cannot conclusively determine how much improvement is attributable to the acupuncture treatment versus the effects that stem from the entire treatment experience. The authors describe the research question as an ‘intermediate’ one. It is suggested here, and elsewhere12,22,23,28 that a third group or further study would be needed to study fully the treatment effect. In the study of exercise for chronic heart failure20 the authors note that it ‘could not be ruled out that results obtained were due to non-specific treatment factors’. Fourteen studies reported that informed consent had been obtained from the subjects or, where appropriate, the adults responsible for them.11–13,15,16,19–23,26,27,29,31 In one study about the effect of an information leaflet14 and in four psychologically oriented studies it was not clear that informed consent had been obtained.17,18,24,25 Moreover, in a study of dietary advice30 ‘volunteers were asked to take part in a Healthy Eating study’, the implication being that subjects were not fully informed about the nature and purpose of the study. This is explicitly stated in the report of a different study of diet10 where ‘before intervention subjects were not informed that the study focused on fruit and vegetables, and control subjects were only informed after the study was completed’. A similar strategy may have been used in a study of pre-operative relaxation28 since consent was described as ‘for participation in a study of whether listening to a tape before surgery would help patients to recover’.
43
Moreover, for these designs, it is useful if the effectiveness of the other treatment has already been established. The nature of informed consent appears to vary considerably from study to study. It is not helpful merely to report that ‘informed consent was obtained from all subjects’. Without details of precisely what the subjects were told and when, it is impossible precisely to define the research question being asked, and to replicate the study, or assess its clinical relevance. There seems to be a case, in some studies, for informed consent not being ‘fully informed’. Subjects should be told as much as possible, but not information that would invalidate the study. Where subject blinding is not feasible it is important to establish whether non-specific effects are possible confounds or an integral part of the treatment being studied. This will help to determine whether there is a need for placebo control, possibly including attention control. In some studies three study groups are worth considering, provided resources allow this and there are no ethical objections. There are likely to be several aspects to a therapy or treatment, only some of which can be investigated in a single study. Moreover, there are different types of question that one can ask about a therapy. Researchers should be fully aware of which questions they are trying to answer, which they are ignoring, and why. For effective evidence-based medicine all such reasoning should be presented in papers.
ACKNOWLEDGEMENTS The author is grateful to the anonymous referees whose constructive criticism has improved the quality of this paper. Views expressed are those of the author.
REFERENCES
DISCUSSION As a general rule, although not always stated explicitly, the ultimate aim of most of the researchers cited here appeared to be an investigation of specific treatment effects. The more pragmatic designs were often seen as a step in this direction. As a general principle, however, more precise statements of the research questions and more detailed justifications of the designs would have been helpful. There are two reasons for this. Firstly, it is necessary to establish the exact objective and to assess the integrity of the study, and secondly it helps other researchers with their designs. When the therapy is being compared with an alternative treatment it is important to decide whether the aim is to show the superiority, noninferiority or the equivalence of the new treatment, since the latter cases might require a larger study.3
1. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312: 1215–1218. 2. Willett WC. Nutritional Epidemiology. In: Rothman KJ, Greenland S (eds) Modern Epidemiology. Philadelphia: Lippincott-Raven, 1998. 3. Cleophas TJM, Meulen Jvd, Kalmansohn RB. Clinical Trials: specific problems associated with the use of a placebo control group. British Journal of Clinical Pharmacology 1997; 43: 219–221. 4. Schwartz CE, Chesney MA, Irvine MJ, Keefe FJ. The Control Group Dilemma in Clinical Research: Applications for Psychosocial and Behavioural Medicine. Psychosomatic Medicine 1997; 59: 362–371. 5. Ernst E, Resch KL. Concept of true and perceived placebo effects. BMJ 1995; 311: 511–553. 6. Kleijnen J, de Craen AJ, van Everdingen J, Krol L. Placebo effects in double-blind clinical trials: a review of interactions with medications. Lancet 1994; 344: 1347–1349. 7. Vickers AJ, de Craen AJ. Why use placebos in clinical trials? A narrative review of the methodological literature. Journal of Clinical Epidemiology 2000; 53(2) 157–161.
44
Complementary Therapies in Medicine
8. Letters. Informed consent. BMJ 1997; 314: 1477–1483. 9. Letters. Informed Consent. BMJ 1997; 315: 247 10. Cox DN, Anderson AS, Reynolds J et al. Take Five, a nutrition education intervention to increase fruit and vegetable intakes: impact on consumer choice and nutrient intakes. British Journal of Nutrition 1998; 80: 123–131. 11. Chalder T, Wallace P, Wessely S. Self-help treatment of chronic fatigue in the community; A randomized controlled trial. British Journal of Health Psychology 1997; 2: 189–197. 12. Hickson M, Nicholl C, Bulpitt C et al. The Design of the Feeding Support Trial – does intensive feeding support improve nutritional status and outcome in acutely ill older in-patients? Journal of Human Nutrition and Dietetics 1999; 12: 53–59. 13. Wiesinger GF, Quittan M, Aringer M et al. Improvement of physical fitness and muscle strength in polymyositis/dermatomyositis patients by a training programme. British Journal of Rheumatology 1998; 37: 196–200. 14. Barlow JH, Wright CC. Knowledge in patients with rheumatoid arthritis: a longer term follow-up of a randomized controlled study of patient education leaflets. British Journal of Rheumatology 1998; 37: 373–376. 15. Berman BM, Singh BB, Lao L et al. A randomized trial of acupuncture as an adjunctive therapy in osteoarthritis of the knee. Rheumatology 1999; 38: 346–354. 16. Helliwell PS, O’Hara M, Holdsworth J et al. A 12month randomized controlled trial of patient education on radiographic changes and quality of life in early rheumatoid arthritis. Rheumatology 1999; 38: 303–308. 17. MacLeod AK, Tata P, Evans K et al. Recovery of positive future thinking within a high-risk parasuicide group: Results from a pilot randomized controlled trial. British Journal of Clinical Psychology 1998; 37: 371–379. 18. Freeman D, Garety P, Fowler D et al. The London-East Anglia randomized controlled trial of cognitivebehaviour therapy for pyschosis IV: Self-esteem and persecutory delusions. British Journal of Clinical Psychology 1998; 37: 415–430. 19. Lisspers J, Sundin O, Hofman-Bang C et al. Behavioural effects of a comprehensive, multifactorial program for lifestyle change after percutaneous transluminal coronary angioplasty: a prospective, randomized, controlled study. Journal of Psychosomatic Research 1999; 46(2): 143–154.
20. Weilinga RP, Erdman RAM, Huisveld IA et al. Effect of exercise training on quality of life in patients with chronic heart failure. Journal of Psychosomatic Research 1998; 45(5): 459–464. 21. Vlieland TPMV, Breedveld FC, Hazes JMW. The twoyear follow-up of a randomized comparison of in-patient multidisciplinary team care and routine out-patient care for active rheumatoid arthritis. British Journal of Rheumatology 1997; 36: 82–85. 22. Nguyen M, Revel M, Dougados M. Prolonged effects of 3 week therapy in a spa resort on lumbar spine, knee and hip osteoarthritis: follow-up after 6 months. A randomized controlled trial. British Journal of Rheumatology 1997; 36: 77–81. 23. Blake F, Salkovskis P, Gath D, Day A, Garrod A. Cognitive therapy for premenstrual syndrome: a controlled study. Journal of Psychosomatic Research 1998; 45(4): 307–318. 24. Burke M, Drummond LM, Johnston DW. Treatment for agoraphobic women: Exposure or cognitive-behaviour therapy? British Journal of Clinical Psychology 1997; 36: 409–420. 25. Deale A, Chalder T, Wessely S. Illness beliefs and treatment outcome in chronic fatigue syndrome. Journal of Psychosomatic Research 1998; 45(1): 77–83. 26. Powell CA, Walker SP, Chang SM, GranthamMcGregor SM. Nutrition and education: a randomized trial of the effects of breakfast in rural primary school children. American Journal of Clinical Nutrition 1998; 68: 873–9. 27. Mazzuca SA, Brandt KD, Katz BP et al. Effects of selfcare education on the health status of inner-city patients with osteoarthritis of the knee. Arthritis and Rheumatism 1997; 40(8): 1466–1474. 28. Manyande A, Salmon P. Effects of pre-operative relaxation on post-operative analgesia: Immediate increase and delayed reaction. British Journal of Health Psychology 1998; 3: 215–224. 29. Martin NT, Gaffan EA, Williams T. Behavioural effects of long-term multi-sensory stimulation. British Journal of Clinical Psychology 1998; 37: 69–82. 30. Drummond S, Kirk T. The effect of different types of dietary advice on body composition in a group of Scottish men. Journal of Human Nutrition and Dietetics 1998; 11: 473–485. 31. Thomas VJ, Dixon AL, Milligan P. Cognitive-behaviour therapy for the management of sickle cell disease pain: An evaluation of a community-based intervention. British Journal of Health Psychology 1999; 4: 209–229.