- Email: [email protected]
What is the best airway clearance technique in cystic fibrosis?
Paediatric Respiratory Reviews 14S (2013) 10–12
Contents lists available at SciVerse ScienceDirect
Paediatric Respiratory Reviews
What is the best airway clearance technique in cystic fibrosis? Eleanor Main * Senior Lecturer in Physiotherapy and Course Director: UCL MSc, Diploma & Certificate in Advanced Physiotherapy. Portex Department, 6th Floor, Cardiac Wing, Institute of Child Health, University College London, 30 Guilford Street, London WCIN 1EH
A R T I C L E I N F O
S U M M A R Y
Keywords: Physiotherapy Cystic fibrosis Airway clearance Research Preference trials
The global development of airway clearance techniques (ACTs) for cystic fibrosis (CF) and corresponding research spans over four decades. Five Cochrane reviews synthesising the evidence from a plethora of early short and medium term studies have not uncovered any superior method. Four recent long term RCT studies exposed fundamental shortcomings in the standard RCT trial design and the insensitivity of FEV1 in physiotherapy studies. Strong patient preference, lack of blinding and the requirement for effortful and demanding participation over long intervals will continue to derail efforts to find the best ACT for CF, unless they are addressed in future clinical trials. ß 2013 Elsevier Ltd. All rights reserved.
Cystic Fibrosis (CF) is a life-shortening inherited disease, characterised by thick sticky mucus in the internal organs, including the lungs and digestive system. The lung disease caused by retained secretions, inflammation and infection is the primary cause of morbidity and death in patients with CF. For almost half a century, airway clearance techniques (ACTs) have been used to remove tenacious airway secretions and improve health and survival in these patients. Over this era, different ACTs have been invented, modified, retained or rejected. Some involve airway oscillation, others do not, some are independently performed, and others require a power source or physical assistance. Some have strong geographical dominance, often more closely related to the origin of the technique and the strength of marketing there, than to best evidence for efficacy. The best ACT in CF has yet to be identified. This period has also seen an illuminating evolution in the quality and rigor of research conducted to identify the best ACT. Physiotherapy studies have evolved from a plethora of early short term studies, typically cross-over design, typically between a day and a fortnight in length, with very small participant numbers, in which the outcomes were often sputum clearance or lung function testing. Results were inevitably not valid for, or generalisable to chronic disease conditions and this generated calls for further research in the form of properly conducted, long-term, randomised controlled trials (RCTs) with standardised and meaningful outcome measures. Cochrane Reviews are widely regarded as the most reliable synthesis of best available evidence. There are currently five Cochrane Systematic Reviews strictly relevant to ACTs for
* Tel.: +0207 905 2754; fax: +0207 829 8634. E-mail address: [email protected]. 1526-0542/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.prrv.2013.01.008
CF, published between 2000 and 2011.1–6 These Cochrane reviews, which excluded poor quality studies and integrated the best research data through meta-analysis, have not yet yielded a best ACT for CF. All conclude there is currently insufficient evidence to suggest any one technique is any better than another. Some offer a vague recommendation that doing any ACT is better than doing nothing, and that patients seem to prefer independently performed techniques, for example Positive Expiratory Pressure mask (PEP) therapy rather than those that require assistance, for example postural drainage and percussion (PD&P). Efforts to break the deadlock in evidence have continued. Leaders in this clinical field have recently undertaken welldesigned and well-funded long term RCTs which seemed to address all the concerns of the past. Ironically, these 4 international long term ACT studies published between 2010–2012 have only served to expose a fundamental obstacle in physiotherapy ACT studies which has not been acknowledged to date. They individually and collectively provide some important insights into the future of Physiotherapy CF research.7–10 During 2012, results were reported by McIlwaine et al., from a Canadian research team trying to identify whether PEP, the ACT used most frequently by Canadian CF patients, would be any better than High Frequency Chest Wall Oscillation (HFCWO), used by American patients.8 The trial was conducted over 1 year through 12 CF centres in Canada. Results suggested that HFCWO was associated with a significantly larger number of respiratory exacerbations per year, and treatments were significantly more time-consuming and less flexible than PEP. Not an overwhelming endorsement for the ‘Vest’, but this should not distract from another important aspect of this trial which finds resonance in the remaining three studies and which requires urgent further consideration.
E. Main / Paediatric Respiratory Reviews 14S (2013) 10–12
In the study, 107 patients were approached, were fully informed about the trial and agreed to participate, but 16 of these dropped out at or just before randomisation, with a further 3 dropping out during the trial (18%). The substantive majority dropped out before they had officially started using either technique. Most of these participants were quite transparent about the fact that they would have preferred allocation to a different treatment arm. Similarly, Sontag et al in 2010 published results of their long term ACT study conducted between 1999 and 2002 at 20 centres across America.10 The three arm parallel RCT compared HFCWO with the flutter device (FD) and PD&P over 3 years. Enrolment goals of 180 patients were not met and subsequently, 56 of the 166 participants enrolled dropped out (34%), a third of these dropping out at or around randomisation and the remainder dropping out during the trial. The trial was terminated early for two reasons. Firstly, the number of dropouts in different groups was so disproportionate (35, 16 and 5 in the PD&P, FD and HFCWO groups respectively), that it undermined the validity of any analysis. Early dropouts were largely related to preference for an alternative treatment. The extreme bias in pattern of late dropouts also strongly suggested preference for specific techniques, since there were no other variables between groups. Authors reported that ‘satisfaction with the therapy’ was an independent predictor of withdrawing. The dropout rate also increased with age, suggesting that preferences became more entrenched with maturity, along with the confidence to withdraw from a treatment that was not preferred. Ironically the HFCWO treatment, which was favoured most by patients in terms of perceived efficacy, fared least well, showing significantly faster decline in FEF25-75 compared with the other techniques. Preference does not guarantee efficacy. The second reason for terminating the trial early was that the anticipated decline in FEV1, (-2% predicted per annum), upon which the study was powered, failed to materialise. Observed FEV1 decline was only -1.23%, leaving the study underpowered and undermining the analysis. The authors argued that FEV1, despite its wide recognition as a gold standard measure of lung function, was not a helpful clinical trial outcome in this study. Another large RCT was published in 2010, by Pryor et al in the UK.9 They compared 5 different, independently performed ACTs over one year. While the comparisons between techniques were largely unremarkable, showing no important differences between groups, this study too suffered from large dropout rates (29%), with more than half of these patients (13/22) admitting they did not like the regimen to which they had been randomised. They either reverted to their original regimen (N=9) or to a different one of their choosing (N=4). These 18% of the cohort remained in the intention-to-treat analysis, compromising the validity of results. Once again the reported decline in FEV1 (-1.8%) was less than anticipated (-2%), and authors suggested FEV1 was unlikely to assist as a sensitive health care indicator. The last of these studies, again by McIlwaine et al., in 2010, reported on the first year of their 2 year crossover trial comparing PD&P with Autogenic Drainage (AD).7 They too terminated their trial early having encountered an intractable problem with dropouts which, this time, manifested in an unusual and unforeseen way. In this study, more than half of the participants (58%) in the AD group (10/17) refused to cross over to the PD&P arm after one year of study. Those who were willing to cross-over began to incorporate AD into their PD&P treatments during the second year, again invalidating the study. Once again, the issue of preference was so important it destabilised and unhinged a well-designed and well conducted research study. The study was powered on a predicted FEV1 decline of -2.3%. Instead, both groups showed an improvement in FEV1 over the first year.
11
This pattern of dropouts in all four recent, well designed, international long term RCTs, is too important to ignore. Dropouts, largely as a result of strong preferences by patients for a particular ACT, entirely derailed two of these studies, resulting in their premature termination and compromised the remaining two. This issue is gaining important recognition in specific areas of clinical research.11–15 Randomised Controlled Trials (RCT), either parallel or cross-over designs, have been around for over 130 years and are arguably the most rigorous way of deciding if one intervention is better than another. Many designs however, are not fit for purpose in all circumstances, and there are some important reasons why the RCT is problematic in physiotherapy intervention studies in CF. The success of the RCT rests on three principles, two of which are essentially incompatible with physiotherapy ACT research questions. These are random allocation and blinding. The third principle is that if the first two are met, probability theory can be used to express the likelihood that any difference between treatment groups happened by chance. Physiotherapy treatments are notoriously difficult to ‘blind’ by concealment or placebo. They are very physical, they may involve hands or devices, special breathing, hard exercise, vibrations or oscillations, and they are certainly time-consuming, effortful and demanding. They cannot be put into unmarked bottles or nebulisers or tablets or aerosols. Patients with CF have experiential knowledge about ACTs and they care about what they are experiencing, especially if they are required to do it daily and for life. An inability to conceal treatment from patients undermines a fundamental strength of the RCT design and begins to open a door to an uncontrollable preference bias. In addition, while random allocation of participants to treatment groups almost entirely eliminates selection bias and other uncontrolled variables in treatment assignment, it is conceptually like tossing a coin. Deciding on health care by coin toss is really only palatable for patients if there is genuine equipoise about the relative benefits AND they don’t know or care what they are getting OR if they only are required to endure the relative burden of treatment for a short time. Brewin et al. recognized that clinical trials in which patients are: 1. required to sustain an effortful and demanding role and 2. are likely to have strong preferences for one treatment, need to be considered and conducted differently.11 There are few interventions for which these two arguments against a standard RCT are met more strongly than ACTs for CF. The difficult process of clearing mucous from the lungs will, since the introduction of newborn screening, have begun at birth, and continue throughout the abbreviated lifetime of the person with CF. They will experience the full burden of daily time consuming ACTs. They will suffer the advice, guidance and tuition, often inconsistent, between different therapists in different hospitals. They will explore the relative advantages and disadvantages of different techniques through stable disease and exacerbations, through mild disease and debilitating disease, with and without personal modifications. They will work out which device is too noisy, or too difficult to clean, or too difficult to transport, or too expensive. They will Google and Facebook and Tweet each other about their experiences. They will suffer the hopes and disappointments of potential new treatments, some of which offer such small incremental gains that they hardly seem worth it, some of which are too expensive for Health Authorities to consider providing for their patients. They will find new devices and machines and charlatans with miracle cures on the internet and observe with interest treatments that come into and out of fashion for no particular reason. In their lives they will receive countless invitations to participate in research trials to decide on the best new thing. They may begin this journey with optimistic altruism and progress
12
E. Main / Paediatric Respiratory Reviews 14S (2013) 10–12
to cynical conditional participation in trials that genuinely seem to offer something important, such as the new gene therapy trials. In short, patients with CF will quite naturally develop and modify their strong preferences about how they participate in managing their chronic lung disease. And these preferences matter a great deal. They matter because people’s own beliefs about treatments are known to be the most important determinant of whether and how they are taken.16 They matter because, as observed above in all 4 recent long term ACT RCTs, preferences can seriously undermine the validity of the research, because it is too difficult to control for the post-randomisation effects of these preferences on treatment outcomes. In practical terms, these effects can manifest in noncompliance or dropout. The seasoned CF trial participant knows that continued participation in any trial cannot be enforced. They may sign up because they have an interest in a particular treatment, but drop out immediately if they do not get the preferred arm. If this happens for one arm more than the other, as in the Sontag study, the bias introduced cannot be corrected. If this happens halfway through a cross-over study, as happened in the McIlwaine study above, the trial is equally doomed. Both these trials were destined to fail, for reasons beyond the control of the research team, who faithfully adhered to rigorous principles in RCT trial design.7,10. It is now recognized that the problematic effects of preference are most apparent in 1. Unblinded trials in which patients are aware of the treatment they are receiving and 2. Trials in which participants are required to sustain an effortful and demanding role.15,17 Asking a sentient adult with CF to disregard his or her inherent treatment preference is counterintuitive, and is unlikely to be sustainable for an extended interval. Patients with CF will simply not risk languishing for a year or more, in a trial arm that they instinctively disagree with. Life for them, is literally too short. Two things are certain; preference needs to be accounted for in long term physiotherapy airway clearance studies, and FEV1 can no longer be the gold standard outcome measure in these trials. It is beyond the scope of this paper to describe in detail how preference may be accommodated in research, while maintaining rigor and validity in statistical analysis, but a number of authors have proposed excellent ways in which this can be done. The Rucker design, described in 1989, offered a two stage trial design, in which the separate effects of preference, selection and treatment could be identified.18 In this design, all participants are randomized into two groups: those that are permitted to select their treatment and those that are not. The latter are randomized into two treatment groups as usual. The former are asked to select their preferred treatment and are allocated that treatment. Where there is no strong preference, they are reallocated to the standard randomization group. Variations on this have also been described, from simply ‘recording’ preference before normal full randomization, so that they can be taken into account during statistical analysis, to degrees of partial randomization.18–21 This is a rare area of clinical research in which the ‘perfect storm’ of strong preference, lack of blinding and the requirement for effortful and demanding participation over long intervals will continue to unseat any serious effort to find the best ACT for patients with CF. Unless future studies take these fundamental
concepts into consideration, it is very unlikely that an answer to the question heading this manuscript will be forthcoming any time soon and we will continue to see the collapse of well conducted RCTs. This will be a spectacular waste of time and money and a terrible disservice to patients with CF. New research design approaches will provide an opportunity to assess the true magnitude of treatment effects as well preference effects after randomisation in physiotherapy ACT studies. CONFLICT OF INTEREST None Declared References 1. Bradley JM, Moran FM, Elborn JS. Evidence for physical therapies (airway clearance and physical training) in cystic fibrosis: an overview of five Cochrane systematic reviews. Respir Med 2006;100:191–201. 2. Elkins MR, Jones A, Schans C. Positive expiratory pressure physiotherapy for airway clearance in people with cystic fibrosis. Cochrane Database Syst Rev 2004. CD003147. 3. Main E, Prasad A, Schans C. Conventional chest physiotherapy compared to other airway clearance techniques for cystic fibrosis. Cochrane Database Syst Rev 2005. CD002011. 4. Morrison L, Agnew J. Oscillating devices for airway clearance in people with cystic fibrosis. Cochrane Database Syst Rev 2009. CD006842. 5. Robinson KA, McKoy N, Saldanha I, Odelola OA. Active cycle of breathing technique for cystic fibrosis. Cochrane Database Syst Rev 2010. CD007862. 6. van der Schans C, Prasad A, Main E. Chest physiotherapy compared to no chest physiotherapy for cystic fibrosis. Cochrane Database Syst Rev 2000. CD001401. 7. McIlwaine M, Wong LT, Chilvers M, Davidson GF. Long-term comparative trial of two different physiotherapy techniques; postural drainage with percussion and autogenic drainage, in the treatment of cystic fibrosis. Pediatr Pulmonol 2010;45:1064–9. 8. McIlwaine M, Agnew JL, Alarie N, Lands L, Ratjen F, Milner R, et al. Canadian national airway clearance study: positive expiratory pressure mask versus high frequency chest wall oscillation. Journal of Cystic Fibrosis 2012;11(Supplement 1(0)):S23. 9. Pryor JA, Tannenbaum E, Scott SF, Burgess J, Cramer D, Gyi K, et al. Beyond postural drainage and percussion: Airway clearance in people with cystic fibrosis. J Cyst Fibros 2010;9:187–92. 10. Sontag MK, Quittner AL, Modi AC, Koenig JM, Giles D, Oermann CM, et al. Lessons learned from a randomized trial of airway secretion clearance techniques in cystic fibrosis. Pediatr Pulmonol 2010;45:291–300. 11. Brewin CR, Bradley C. Patient preferences and randomised clinical trials. BMJ 1989;299:313–5. 12. Udell JA, Redelmeier DA. Patient preferences and the ironic nature of randomized trials. Med Decis Making 2011;31:226–8. 13. McPherson K. Do patients’ preferences matter? BMJ 2008;337:a2034. 14. Tilbrook H. Patients’ preferences within randomised trials: systematic review and patient level meta-analysis. BMJ 2008;337:a1864. 15. Silverman WA, Altman DG. Patients’ preferences and randomised trials. Lancet 1996;347:171–4. 16. Shaw J, Baker M. ‘‘Expert patient’’–dream or nightmare? BMJ 2004;328: 723–4. 17. King M, Nazareth I, Lampe F, Bower P, Chandler M, Morou M, et al. Impact of participant and physician intervention preferences on randomized trials: a systematic review. JAMA 2005;293:1089–99. 18. Rucker G. A two-stage trial design for testing treatment, self-selection and treatment preference effects. Stat Med 1989;8:477–85. 19. McCaffery KJ, Turner R, Macaskill P, Walter SD, Chan SF, Irwig L. Determining the impact of informed choice: separating treatment effects from the effects of choice and selection in randomized trials. Med Decis Making 2011;31:229–36. 20. McPherson K, Chalmers I. Incorporating patient preferences into clinical trials. Information about patients’ preference must be obtained first. BMJ 1998;317: 78–9. 21. Walter SD, Turner RM, Macaskill P, McCaffery KJ, Irwig L. Optimal allocation of participants for the estimation of selection, preference and treatment effects in the two-stage randomised trial design. Stat Med 2012;31:1307–22.
Contents lists available at SciVerse ScienceDirect
Paediatric Respiratory Reviews
What is the best airway clearance technique in cystic fibrosis? Eleanor Main * Senior Lecturer in Physiotherapy and Course Director: UCL MSc, Diploma & Certificate in Advanced Physiotherapy. Portex Department, 6th Floor, Cardiac Wing, Institute of Child Health, University College London, 30 Guilford Street, London WCIN 1EH
A R T I C L E I N F O
S U M M A R Y
Keywords: Physiotherapy Cystic fibrosis Airway clearance Research Preference trials
The global development of airway clearance techniques (ACTs) for cystic fibrosis (CF) and corresponding research spans over four decades. Five Cochrane reviews synthesising the evidence from a plethora of early short and medium term studies have not uncovered any superior method. Four recent long term RCT studies exposed fundamental shortcomings in the standard RCT trial design and the insensitivity of FEV1 in physiotherapy studies. Strong patient preference, lack of blinding and the requirement for effortful and demanding participation over long intervals will continue to derail efforts to find the best ACT for CF, unless they are addressed in future clinical trials. ß 2013 Elsevier Ltd. All rights reserved.
Cystic Fibrosis (CF) is a life-shortening inherited disease, characterised by thick sticky mucus in the internal organs, including the lungs and digestive system. The lung disease caused by retained secretions, inflammation and infection is the primary cause of morbidity and death in patients with CF. For almost half a century, airway clearance techniques (ACTs) have been used to remove tenacious airway secretions and improve health and survival in these patients. Over this era, different ACTs have been invented, modified, retained or rejected. Some involve airway oscillation, others do not, some are independently performed, and others require a power source or physical assistance. Some have strong geographical dominance, often more closely related to the origin of the technique and the strength of marketing there, than to best evidence for efficacy. The best ACT in CF has yet to be identified. This period has also seen an illuminating evolution in the quality and rigor of research conducted to identify the best ACT. Physiotherapy studies have evolved from a plethora of early short term studies, typically cross-over design, typically between a day and a fortnight in length, with very small participant numbers, in which the outcomes were often sputum clearance or lung function testing. Results were inevitably not valid for, or generalisable to chronic disease conditions and this generated calls for further research in the form of properly conducted, long-term, randomised controlled trials (RCTs) with standardised and meaningful outcome measures. Cochrane Reviews are widely regarded as the most reliable synthesis of best available evidence. There are currently five Cochrane Systematic Reviews strictly relevant to ACTs for
* Tel.: +0207 905 2754; fax: +0207 829 8634. E-mail address: [email protected]. 1526-0542/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.prrv.2013.01.008
CF, published between 2000 and 2011.1–6 These Cochrane reviews, which excluded poor quality studies and integrated the best research data through meta-analysis, have not yet yielded a best ACT for CF. All conclude there is currently insufficient evidence to suggest any one technique is any better than another. Some offer a vague recommendation that doing any ACT is better than doing nothing, and that patients seem to prefer independently performed techniques, for example Positive Expiratory Pressure mask (PEP) therapy rather than those that require assistance, for example postural drainage and percussion (PD&P). Efforts to break the deadlock in evidence have continued. Leaders in this clinical field have recently undertaken welldesigned and well-funded long term RCTs which seemed to address all the concerns of the past. Ironically, these 4 international long term ACT studies published between 2010–2012 have only served to expose a fundamental obstacle in physiotherapy ACT studies which has not been acknowledged to date. They individually and collectively provide some important insights into the future of Physiotherapy CF research.7–10 During 2012, results were reported by McIlwaine et al., from a Canadian research team trying to identify whether PEP, the ACT used most frequently by Canadian CF patients, would be any better than High Frequency Chest Wall Oscillation (HFCWO), used by American patients.8 The trial was conducted over 1 year through 12 CF centres in Canada. Results suggested that HFCWO was associated with a significantly larger number of respiratory exacerbations per year, and treatments were significantly more time-consuming and less flexible than PEP. Not an overwhelming endorsement for the ‘Vest’, but this should not distract from another important aspect of this trial which finds resonance in the remaining three studies and which requires urgent further consideration.
E. Main / Paediatric Respiratory Reviews 14S (2013) 10–12
In the study, 107 patients were approached, were fully informed about the trial and agreed to participate, but 16 of these dropped out at or just before randomisation, with a further 3 dropping out during the trial (18%). The substantive majority dropped out before they had officially started using either technique. Most of these participants were quite transparent about the fact that they would have preferred allocation to a different treatment arm. Similarly, Sontag et al in 2010 published results of their long term ACT study conducted between 1999 and 2002 at 20 centres across America.10 The three arm parallel RCT compared HFCWO with the flutter device (FD) and PD&P over 3 years. Enrolment goals of 180 patients were not met and subsequently, 56 of the 166 participants enrolled dropped out (34%), a third of these dropping out at or around randomisation and the remainder dropping out during the trial. The trial was terminated early for two reasons. Firstly, the number of dropouts in different groups was so disproportionate (35, 16 and 5 in the PD&P, FD and HFCWO groups respectively), that it undermined the validity of any analysis. Early dropouts were largely related to preference for an alternative treatment. The extreme bias in pattern of late dropouts also strongly suggested preference for specific techniques, since there were no other variables between groups. Authors reported that ‘satisfaction with the therapy’ was an independent predictor of withdrawing. The dropout rate also increased with age, suggesting that preferences became more entrenched with maturity, along with the confidence to withdraw from a treatment that was not preferred. Ironically the HFCWO treatment, which was favoured most by patients in terms of perceived efficacy, fared least well, showing significantly faster decline in FEF25-75 compared with the other techniques. Preference does not guarantee efficacy. The second reason for terminating the trial early was that the anticipated decline in FEV1, (-2% predicted per annum), upon which the study was powered, failed to materialise. Observed FEV1 decline was only -1.23%, leaving the study underpowered and undermining the analysis. The authors argued that FEV1, despite its wide recognition as a gold standard measure of lung function, was not a helpful clinical trial outcome in this study. Another large RCT was published in 2010, by Pryor et al in the UK.9 They compared 5 different, independently performed ACTs over one year. While the comparisons between techniques were largely unremarkable, showing no important differences between groups, this study too suffered from large dropout rates (29%), with more than half of these patients (13/22) admitting they did not like the regimen to which they had been randomised. They either reverted to their original regimen (N=9) or to a different one of their choosing (N=4). These 18% of the cohort remained in the intention-to-treat analysis, compromising the validity of results. Once again the reported decline in FEV1 (-1.8%) was less than anticipated (-2%), and authors suggested FEV1 was unlikely to assist as a sensitive health care indicator. The last of these studies, again by McIlwaine et al., in 2010, reported on the first year of their 2 year crossover trial comparing PD&P with Autogenic Drainage (AD).7 They too terminated their trial early having encountered an intractable problem with dropouts which, this time, manifested in an unusual and unforeseen way. In this study, more than half of the participants (58%) in the AD group (10/17) refused to cross over to the PD&P arm after one year of study. Those who were willing to cross-over began to incorporate AD into their PD&P treatments during the second year, again invalidating the study. Once again, the issue of preference was so important it destabilised and unhinged a well-designed and well conducted research study. The study was powered on a predicted FEV1 decline of -2.3%. Instead, both groups showed an improvement in FEV1 over the first year.
11
This pattern of dropouts in all four recent, well designed, international long term RCTs, is too important to ignore. Dropouts, largely as a result of strong preferences by patients for a particular ACT, entirely derailed two of these studies, resulting in their premature termination and compromised the remaining two. This issue is gaining important recognition in specific areas of clinical research.11–15 Randomised Controlled Trials (RCT), either parallel or cross-over designs, have been around for over 130 years and are arguably the most rigorous way of deciding if one intervention is better than another. Many designs however, are not fit for purpose in all circumstances, and there are some important reasons why the RCT is problematic in physiotherapy intervention studies in CF. The success of the RCT rests on three principles, two of which are essentially incompatible with physiotherapy ACT research questions. These are random allocation and blinding. The third principle is that if the first two are met, probability theory can be used to express the likelihood that any difference between treatment groups happened by chance. Physiotherapy treatments are notoriously difficult to ‘blind’ by concealment or placebo. They are very physical, they may involve hands or devices, special breathing, hard exercise, vibrations or oscillations, and they are certainly time-consuming, effortful and demanding. They cannot be put into unmarked bottles or nebulisers or tablets or aerosols. Patients with CF have experiential knowledge about ACTs and they care about what they are experiencing, especially if they are required to do it daily and for life. An inability to conceal treatment from patients undermines a fundamental strength of the RCT design and begins to open a door to an uncontrollable preference bias. In addition, while random allocation of participants to treatment groups almost entirely eliminates selection bias and other uncontrolled variables in treatment assignment, it is conceptually like tossing a coin. Deciding on health care by coin toss is really only palatable for patients if there is genuine equipoise about the relative benefits AND they don’t know or care what they are getting OR if they only are required to endure the relative burden of treatment for a short time. Brewin et al. recognized that clinical trials in which patients are: 1. required to sustain an effortful and demanding role and 2. are likely to have strong preferences for one treatment, need to be considered and conducted differently.11 There are few interventions for which these two arguments against a standard RCT are met more strongly than ACTs for CF. The difficult process of clearing mucous from the lungs will, since the introduction of newborn screening, have begun at birth, and continue throughout the abbreviated lifetime of the person with CF. They will experience the full burden of daily time consuming ACTs. They will suffer the advice, guidance and tuition, often inconsistent, between different therapists in different hospitals. They will explore the relative advantages and disadvantages of different techniques through stable disease and exacerbations, through mild disease and debilitating disease, with and without personal modifications. They will work out which device is too noisy, or too difficult to clean, or too difficult to transport, or too expensive. They will Google and Facebook and Tweet each other about their experiences. They will suffer the hopes and disappointments of potential new treatments, some of which offer such small incremental gains that they hardly seem worth it, some of which are too expensive for Health Authorities to consider providing for their patients. They will find new devices and machines and charlatans with miracle cures on the internet and observe with interest treatments that come into and out of fashion for no particular reason. In their lives they will receive countless invitations to participate in research trials to decide on the best new thing. They may begin this journey with optimistic altruism and progress
12
E. Main / Paediatric Respiratory Reviews 14S (2013) 10–12
to cynical conditional participation in trials that genuinely seem to offer something important, such as the new gene therapy trials. In short, patients with CF will quite naturally develop and modify their strong preferences about how they participate in managing their chronic lung disease. And these preferences matter a great deal. They matter because people’s own beliefs about treatments are known to be the most important determinant of whether and how they are taken.16 They matter because, as observed above in all 4 recent long term ACT RCTs, preferences can seriously undermine the validity of the research, because it is too difficult to control for the post-randomisation effects of these preferences on treatment outcomes. In practical terms, these effects can manifest in noncompliance or dropout. The seasoned CF trial participant knows that continued participation in any trial cannot be enforced. They may sign up because they have an interest in a particular treatment, but drop out immediately if they do not get the preferred arm. If this happens for one arm more than the other, as in the Sontag study, the bias introduced cannot be corrected. If this happens halfway through a cross-over study, as happened in the McIlwaine study above, the trial is equally doomed. Both these trials were destined to fail, for reasons beyond the control of the research team, who faithfully adhered to rigorous principles in RCT trial design.7,10. It is now recognized that the problematic effects of preference are most apparent in 1. Unblinded trials in which patients are aware of the treatment they are receiving and 2. Trials in which participants are required to sustain an effortful and demanding role.15,17 Asking a sentient adult with CF to disregard his or her inherent treatment preference is counterintuitive, and is unlikely to be sustainable for an extended interval. Patients with CF will simply not risk languishing for a year or more, in a trial arm that they instinctively disagree with. Life for them, is literally too short. Two things are certain; preference needs to be accounted for in long term physiotherapy airway clearance studies, and FEV1 can no longer be the gold standard outcome measure in these trials. It is beyond the scope of this paper to describe in detail how preference may be accommodated in research, while maintaining rigor and validity in statistical analysis, but a number of authors have proposed excellent ways in which this can be done. The Rucker design, described in 1989, offered a two stage trial design, in which the separate effects of preference, selection and treatment could be identified.18 In this design, all participants are randomized into two groups: those that are permitted to select their treatment and those that are not. The latter are randomized into two treatment groups as usual. The former are asked to select their preferred treatment and are allocated that treatment. Where there is no strong preference, they are reallocated to the standard randomization group. Variations on this have also been described, from simply ‘recording’ preference before normal full randomization, so that they can be taken into account during statistical analysis, to degrees of partial randomization.18–21 This is a rare area of clinical research in which the ‘perfect storm’ of strong preference, lack of blinding and the requirement for effortful and demanding participation over long intervals will continue to unseat any serious effort to find the best ACT for patients with CF. Unless future studies take these fundamental
concepts into consideration, it is very unlikely that an answer to the question heading this manuscript will be forthcoming any time soon and we will continue to see the collapse of well conducted RCTs. This will be a spectacular waste of time and money and a terrible disservice to patients with CF. New research design approaches will provide an opportunity to assess the true magnitude of treatment effects as well preference effects after randomisation in physiotherapy ACT studies. CONFLICT OF INTEREST None Declared References 1. Bradley JM, Moran FM, Elborn JS. Evidence for physical therapies (airway clearance and physical training) in cystic fibrosis: an overview of five Cochrane systematic reviews. Respir Med 2006;100:191–201. 2. Elkins MR, Jones A, Schans C. Positive expiratory pressure physiotherapy for airway clearance in people with cystic fibrosis. Cochrane Database Syst Rev 2004. CD003147. 3. Main E, Prasad A, Schans C. Conventional chest physiotherapy compared to other airway clearance techniques for cystic fibrosis. Cochrane Database Syst Rev 2005. CD002011. 4. Morrison L, Agnew J. Oscillating devices for airway clearance in people with cystic fibrosis. Cochrane Database Syst Rev 2009. CD006842. 5. Robinson KA, McKoy N, Saldanha I, Odelola OA. Active cycle of breathing technique for cystic fibrosis. Cochrane Database Syst Rev 2010. CD007862. 6. van der Schans C, Prasad A, Main E. Chest physiotherapy compared to no chest physiotherapy for cystic fibrosis. Cochrane Database Syst Rev 2000. CD001401. 7. McIlwaine M, Wong LT, Chilvers M, Davidson GF. Long-term comparative trial of two different physiotherapy techniques; postural drainage with percussion and autogenic drainage, in the treatment of cystic fibrosis. Pediatr Pulmonol 2010;45:1064–9. 8. McIlwaine M, Agnew JL, Alarie N, Lands L, Ratjen F, Milner R, et al. Canadian national airway clearance study: positive expiratory pressure mask versus high frequency chest wall oscillation. Journal of Cystic Fibrosis 2012;11(Supplement 1(0)):S23. 9. Pryor JA, Tannenbaum E, Scott SF, Burgess J, Cramer D, Gyi K, et al. Beyond postural drainage and percussion: Airway clearance in people with cystic fibrosis. J Cyst Fibros 2010;9:187–92. 10. Sontag MK, Quittner AL, Modi AC, Koenig JM, Giles D, Oermann CM, et al. Lessons learned from a randomized trial of airway secretion clearance techniques in cystic fibrosis. Pediatr Pulmonol 2010;45:291–300. 11. Brewin CR, Bradley C. Patient preferences and randomised clinical trials. BMJ 1989;299:313–5. 12. Udell JA, Redelmeier DA. Patient preferences and the ironic nature of randomized trials. Med Decis Making 2011;31:226–8. 13. McPherson K. Do patients’ preferences matter? BMJ 2008;337:a2034. 14. Tilbrook H. Patients’ preferences within randomised trials: systematic review and patient level meta-analysis. BMJ 2008;337:a1864. 15. Silverman WA, Altman DG. Patients’ preferences and randomised trials. Lancet 1996;347:171–4. 16. Shaw J, Baker M. ‘‘Expert patient’’–dream or nightmare? BMJ 2004;328: 723–4. 17. King M, Nazareth I, Lampe F, Bower P, Chandler M, Morou M, et al. Impact of participant and physician intervention preferences on randomized trials: a systematic review. JAMA 2005;293:1089–99. 18. Rucker G. A two-stage trial design for testing treatment, self-selection and treatment preference effects. Stat Med 1989;8:477–85. 19. McCaffery KJ, Turner R, Macaskill P, Walter SD, Chan SF, Irwig L. Determining the impact of informed choice: separating treatment effects from the effects of choice and selection in randomized trials. Med Decis Making 2011;31:229–36. 20. McPherson K, Chalmers I. Incorporating patient preferences into clinical trials. Information about patients’ preference must be obtained first. BMJ 1998;317: 78–9. 21. Walter SD, Turner RM, Macaskill P, McCaffery KJ, Irwig L. Optimal allocation of participants for the estimation of selection, preference and treatment effects in the two-stage randomised trial design. Stat Med 2012;31:1307–22.