- Email: [email protected]
Improving transparency of clinical trials
TIPS-1186; No. of Pages 3
Science & Society
Improving transparency of clinical trials Rafael Dal-Re´ Clinical Research, BUC (Biosciences UAM+CSIC) Program, International Campus of Excellence, Universidad Auto´noma de Madrid, Madrid, Spain
Recent data reveal that subtle selective publication affects critical aspects of trial reporting, in some cases altering the interpretation of results. Timely prospective registration could help deter selective reporting and clinical trial stakeholders from government authorities to journal editors should work together to foster prospective registration of trials.
Introduction Within-study selective reporting is both common and subtle in clinical trials. In 96 trials recently published in 19 high-impact journals (impact factor 10; 55% published in the New England Journal of Medicine, The Lancet, or the Journal of the American Medical Association) only 52% of primary efficacy end points were included in both the registry and the articles and reported numerically identical (concordant) results [1]. More important is that six of the primary efficacy end points that were discordant alter the interpretation of the trial results. In other words, 6% of trials published in the most respected journals conveyed erroneous messages regarding primary efficacy end points [1]. If these trials were published in the highest-quality journals, we should wonder what would have been the results obtained in analyses of the medium- or low-ranked journals that are responsible for the publication of the vast majority of trial results, many of them being eventually included in systematic reviews and meta-analyses. Thus, for instance, of papers published between 2008 and 2012 in five core journals of clinical geriatrics (impact factor 2.2–5.3), only 29% of 140 reported on all study outcomes compared with those included on trials registry [2]. Beyond primary end points: examples of subtle but important selective reporting Other aspects that affect interpretation of results, such as selective reporting of analyses and of serious adverse events (SAEs), are important to appropriately appraise the relevance of trial findings. A systematic review conducted to assess the occurrence of selective reporting of analyses by comparing the trial protocol to the trial report, or by comparing different sections within a trial article, showed high discrepancy rates, ranging from 7% to 88% in statistical analyses, from 46% to 82% in adjusted versus Corresponding author: Dal-Re´, R. ([email protected]). Keywords: misconduct; publication; editors; outcome-reporting bias; trial registration; ClinicalTrials.gov. 0165-6147/ ß 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.tips.2014.10.012
unadjusted analyses, and from 61% to 100% in subgroup analyses [3]. Data from randomized clinical trials approved by research ethics committees (RECs) showed that whereas 28% of protocols included prespecified subgroup analyses, 48% of articles reported subgroup analyses [4]. More striking is that authors in 33% of articles reporting subgroup analyses affirmed that subgroup analyses were planned, something that was not supported in 35% of the corresponding protocols [4]. Finally, Table 1 depicts recent analyses describing discrepancies observed between information included in articles and that in trials registries regarding SAEs, including deaths. These discrepancies could have a variety of reasons, including, among others, proper deviation from a prespecified protocol, additional analyses proposed by reviewers, lack of space in the journal, error, carelessness, and selective publication of outcomes. The first two reasons should be revealed to readers in the article or as supplemental information. This is especially relevant with regard to primary end points. When claiming that a particular molecular entity shows benefit, authors must fully disclose all subgroup analyses and SAEs. Without full disclosure, readers may not be aware that the compound provides benefit only to a subgroup of patients or be left with the false impression that the compound has a superior safety profile to a comparator – of critical importance if this is the main difference reported between agents. With no space limitations in the online era (any information can be provided as supplemental information), it is unacceptable that these types of information are concealed to readers – an attitude that is considered to be a form of misconduct (http://www.wame.org/about/recommendations-on-publication-ethics-policie). Promoters of prospective registration of trials It is expected that prospective registration of trials will help deter and detect selective publication and selective reporting of trial outcomes. It will make non-publication and changes in the critical aspects of trials (e.g., primary end points) visible. Selective reporting of outcomes deserves the attention of many investigators, yet prospective registration has not been adopted by the whole clinical trial community, although trial investigators have to comply with US and EU clinical trials regulations on drugs, biologics, and medical devices. The Declaration of Helsinki (http://www.wma.net/ es/30publications/10policies/b3/17c.pdf) demands the registration of all clinical trials since 2008; this, however, has been ignored by many investigators. A trial should be registered at or before (http:// www.icmje.org/) or within 21 days of the enrollment of Trends in Pharmacological Sciences xx (2014) 1–3
1
TIPS-1186; No. of Pages 3
Science & Society
Trends in Pharmacological Sciences xxx xxxx, Vol. xxx, No. x
Table 1. Discrepancies observed in the reporting of serious adverse events (SAEs) and deaths in articles and trials registry Outcome No of SAEs per trial arm No of participants with SAEs Report of deaths in the registry No of SAEs reported Report of deaths and suicides Report of deaths
Finding Reported in 99% of trial registry summaries but in only 63% of articles 87% of trials reported more SAEs in the registry than in the article 28% reported different numbers in articles compared with trials registry data 57% of SAEs were reported in articles compared with the corresponding clinical trial registries’ summaries 62% and 53% of cases of deaths or suicide were not reported in articles, respectively In only 56% of trials was the information reported in both the articles and the registry; in 33% of these, there were discordant death counts
the first trial participant (http://www.gpo.gov/fdsys/pkg/ PLAW-110publ85/pdf/PLAW-110publ85.pdf). Recent data showed that timely registration of trials by investigators is improving. An analysis on ClinicalTrials.gov found that there is a trend of increasing timely registration (from 56% in 2006 to 72% in 2011) for Phase II or higher trials [5]. In this respect, supporting retrospective registration, as do the AllTrials campaign (http://www.alltrials.net) and journals such as PLoS Medicine (http://blogs.plos. org/speakingofmedicine/2013/08/13/full-registration-andreporting-of-all-trials-at-plos-medicine/) and Trials (http:// www.trialsjournal.com/), although of utmost importance to investigators and clinicians, has no benefit in deterring selective reporting of outcomes. On the contrary, it could convey a false message of timely registration if the journal does not inform readers of the date on which the trial was registered, so the reader can check it against that of recruitment information, which should be provided in the article. Top-ranked journals belonging to the International Committee of Medical Journal Editors are still publishing results from trials that were not registered in a timely manner [5] and readers are unaware of this unless they check the trial registration date in the registry, something that seldom occurs. There are initiatives aiming to increase registration of clinical trials. For instance, prospective registration started to be (or will be) mandatory between January 2013 and January 2015 for 11 journals belonging to the International Society of Physiotherapy Journal Editors [6]. In India, editors of 11 biomedical journals have required since 2010 that trials must be prospectively registered for publication of the results [7]. However, only 28% of journal editors ask for registration of clinical trials [8]. Some editors might not understand its benefit or believe that asking for it will place their journal at a disadvantage in publishing trials. In this respect, some editors argue that registration will penalize trials from developing countries because there are no registries in these countries [8]. This is a spurious claim: there are primary registries in the World Health Organization Registry Network in countries such as Brazil, China, Cuba, India, Iran, Sri Lanka, and Thailand (http:// www.who.int/ictrp/network/en/); furthermore, ClinicalTrials.gov hosts studies from 187 countries (of the 193 United Nations member states), which clearly underscores how widespread the use of this registry is among investigators worldwide. Health authorities could implement regulations aimed to ensure the registration and public disclosure of trial results. The UK Research Health Authority started by 2
Refs [11] [13] [14]
[15]
requesting registration of all clinical trials to be approved by a REC from 30 September 2013 onward. Registration should occur within 6 weeks of the first participant being recruited in the UK. From September 2014, when submitting a new trial to a REC for review sponsors must explicitly declare that the trial has been registered; from April 2015, sponsors seeking review for a new trial will have to state that all other trials in the recruitment phase have also been registered (http://www.hra.nhs.uk/documents/ 2014/10/hra-research-transparency-registration-key-messages-q.pdf). It is somewhat frustrating to observe that many trials are not registered and many of those registered show serious deficiencies in the quality of the provided information; for instance, a sample of 400 recently registered clinical trials showed that only 58% of primary outcomes were appropriately described and only 52% of drugs or biologics trials had complete information on intervention specifics [9]. Looking forward Although in the near future selective reporting of trial outcomes could most likely be only efficiently prevented by the implementation of specific measures by journal editors [10], prospective registration should be encouraged since it is the first step toward achieving full trial transparency. If some articles reporting the results of trials that were appropriately registered include distorted results, we should wonder about the quality of the results reported in articles on trials that were not registered but which are acceptable for publication by 72% of journals [8]. Both the lack of adequate and timely registration and the lack of (or lack of timely) publication of trial results [5,6,11,12] suggest that many investigators and research and academic institutions still have not internalized the need for full trial transparency. If this presumption is correct, the only way to have most investigators comply with prospective registration would be through the critical involvement of funders, RECs, and journal editors. These stakeholders could halt trial conduct and publication unless adequate and timely prospective registration of the trial is proven. Scientific associations can play their role too, through adequately and regularly informing all members about the benefit that transparency brings to the clinical trial enterprise and asking for a clear commitment from their members toward prospective registration (and publication). Finally, publishers are starting to be committed toward clinical trial transparency. Thus, for instance, whereas Nature Publishing Group
TIPS-1186; No. of Pages 3
Science & Society (http://www.nature.com/authors/gta.pdf) requires prospective registration of trials, Elsevier (http://www.elsevier.com/ connect/homepage/alltrials-campaign-supports-greaterclinical-trial-transparency) is assessing how to insert trial registration numbers into the peer-review processes and Wiley (http://authorservices.wiley.com/bauthor/publicationethics.asp#_Toc149460099) is encouraging all journal editors to require trial registration. The steps taken by the UK Health Research Authority, some journal editors and publishers, and many other stakeholders could be followed by their peers in other countries and health-care disciplines. The question is whether this will occur, to what extent and when. Meanwhile, we have to admit that some investigators will report what they believe is relevant to have their trial published, but will conceal or alter important information that all clinical trial stakeholders have the moral duty (and should work) to have it openly disclose to article readers. Disclaimer statement The opinions expressed in this article are those of the author and may not reflect the opinions of the organization for which he works.
References 1 Becker, J.E. et al. (2014) Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 311, 1063–1065 2 Mann, E. et al. (2014) Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing 43, 872–876 3 Dwan, K. et al. (2014) Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 11, e1001666
Trends in Pharmacological Sciences xxx xxxx, Vol. xxx, No. x 4 Kasenda, B. et al. (2014) Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications. BMJ 349, g4539 5 Huser, V. and Cimino, J.J. (2013) Evaluating adherence to the International Committee of Medical Journal Editors’ policy of mandatory, timely clinical trial registration. J. Am. Med. Inform. Assoc. 20, e169–e174 6 Costa, L.O. et al. (2013) Clinical trial registration in physiotherapy journals: recommendations from the International Society of Physiotherapy Journal Editors. Man. Ther. 18, 1–3 7 Satyanarayana, K. et al. (2008) Statement on publishing clinical trials in Indian biomedical journals. Natl. Med. J. India 21, 105–106 8 Wager, E. et al. (2013) Hardly worth the effort’’? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ 347, f5248 9 Viergever, R.F. et al. (2014) The quality of registration of clinical trials: still a problem. PLoS ONE 9, e84727 10 Dal-Re´, R. and Caplan, A.L. (2014) Time to ensure that clinical trial appropriate results are actually published. Eur. J. Clin. Pharmacol. 70, 491–493 11 Riveros, C. et al. (2014) Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Med. 10, e1001566 12 Saito, H. and Gill, C.J. (2014) How frequently do the results from completed US clinical trials enter the public domain? A statistical analysis of the ClinicalTrials.gov database. PLoS ONE 9, e101826 13 Hartung, D.M. et al. (2014) Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann. Intern. Med. 160, 477–483 14 Hughes, S. et al. (2014) Differences in reporting serious adverse events in industry sponsored clinical trial registries and journal articles on antidepressant and antipsychotic drugs: a cross sectional study. BMJ Open 4, e005535 15 Earley, A. et al. (2013) Haphazard reporting of deaths in clinical trials: a review of cases of ClinicalTrials.gov records and matched publications – a cross-sectional study. BMJ Open 3, e001963
3
Science & Society
Improving transparency of clinical trials Rafael Dal-Re´ Clinical Research, BUC (Biosciences UAM+CSIC) Program, International Campus of Excellence, Universidad Auto´noma de Madrid, Madrid, Spain
Recent data reveal that subtle selective publication affects critical aspects of trial reporting, in some cases altering the interpretation of results. Timely prospective registration could help deter selective reporting and clinical trial stakeholders from government authorities to journal editors should work together to foster prospective registration of trials.
Introduction Within-study selective reporting is both common and subtle in clinical trials. In 96 trials recently published in 19 high-impact journals (impact factor 10; 55% published in the New England Journal of Medicine, The Lancet, or the Journal of the American Medical Association) only 52% of primary efficacy end points were included in both the registry and the articles and reported numerically identical (concordant) results [1]. More important is that six of the primary efficacy end points that were discordant alter the interpretation of the trial results. In other words, 6% of trials published in the most respected journals conveyed erroneous messages regarding primary efficacy end points [1]. If these trials were published in the highest-quality journals, we should wonder what would have been the results obtained in analyses of the medium- or low-ranked journals that are responsible for the publication of the vast majority of trial results, many of them being eventually included in systematic reviews and meta-analyses. Thus, for instance, of papers published between 2008 and 2012 in five core journals of clinical geriatrics (impact factor 2.2–5.3), only 29% of 140 reported on all study outcomes compared with those included on trials registry [2]. Beyond primary end points: examples of subtle but important selective reporting Other aspects that affect interpretation of results, such as selective reporting of analyses and of serious adverse events (SAEs), are important to appropriately appraise the relevance of trial findings. A systematic review conducted to assess the occurrence of selective reporting of analyses by comparing the trial protocol to the trial report, or by comparing different sections within a trial article, showed high discrepancy rates, ranging from 7% to 88% in statistical analyses, from 46% to 82% in adjusted versus Corresponding author: Dal-Re´, R. ([email protected]). Keywords: misconduct; publication; editors; outcome-reporting bias; trial registration; ClinicalTrials.gov. 0165-6147/ ß 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.tips.2014.10.012
unadjusted analyses, and from 61% to 100% in subgroup analyses [3]. Data from randomized clinical trials approved by research ethics committees (RECs) showed that whereas 28% of protocols included prespecified subgroup analyses, 48% of articles reported subgroup analyses [4]. More striking is that authors in 33% of articles reporting subgroup analyses affirmed that subgroup analyses were planned, something that was not supported in 35% of the corresponding protocols [4]. Finally, Table 1 depicts recent analyses describing discrepancies observed between information included in articles and that in trials registries regarding SAEs, including deaths. These discrepancies could have a variety of reasons, including, among others, proper deviation from a prespecified protocol, additional analyses proposed by reviewers, lack of space in the journal, error, carelessness, and selective publication of outcomes. The first two reasons should be revealed to readers in the article or as supplemental information. This is especially relevant with regard to primary end points. When claiming that a particular molecular entity shows benefit, authors must fully disclose all subgroup analyses and SAEs. Without full disclosure, readers may not be aware that the compound provides benefit only to a subgroup of patients or be left with the false impression that the compound has a superior safety profile to a comparator – of critical importance if this is the main difference reported between agents. With no space limitations in the online era (any information can be provided as supplemental information), it is unacceptable that these types of information are concealed to readers – an attitude that is considered to be a form of misconduct (http://www.wame.org/about/recommendations-on-publication-ethics-policie). Promoters of prospective registration of trials It is expected that prospective registration of trials will help deter and detect selective publication and selective reporting of trial outcomes. It will make non-publication and changes in the critical aspects of trials (e.g., primary end points) visible. Selective reporting of outcomes deserves the attention of many investigators, yet prospective registration has not been adopted by the whole clinical trial community, although trial investigators have to comply with US and EU clinical trials regulations on drugs, biologics, and medical devices. The Declaration of Helsinki (http://www.wma.net/ es/30publications/10policies/b3/17c.pdf) demands the registration of all clinical trials since 2008; this, however, has been ignored by many investigators. A trial should be registered at or before (http:// www.icmje.org/) or within 21 days of the enrollment of Trends in Pharmacological Sciences xx (2014) 1–3
1
TIPS-1186; No. of Pages 3
Science & Society
Trends in Pharmacological Sciences xxx xxxx, Vol. xxx, No. x
Table 1. Discrepancies observed in the reporting of serious adverse events (SAEs) and deaths in articles and trials registry Outcome No of SAEs per trial arm No of participants with SAEs Report of deaths in the registry No of SAEs reported Report of deaths and suicides Report of deaths
Finding Reported in 99% of trial registry summaries but in only 63% of articles 87% of trials reported more SAEs in the registry than in the article 28% reported different numbers in articles compared with trials registry data 57% of SAEs were reported in articles compared with the corresponding clinical trial registries’ summaries 62% and 53% of cases of deaths or suicide were not reported in articles, respectively In only 56% of trials was the information reported in both the articles and the registry; in 33% of these, there were discordant death counts
the first trial participant (http://www.gpo.gov/fdsys/pkg/ PLAW-110publ85/pdf/PLAW-110publ85.pdf). Recent data showed that timely registration of trials by investigators is improving. An analysis on ClinicalTrials.gov found that there is a trend of increasing timely registration (from 56% in 2006 to 72% in 2011) for Phase II or higher trials [5]. In this respect, supporting retrospective registration, as do the AllTrials campaign (http://www.alltrials.net) and journals such as PLoS Medicine (http://blogs.plos. org/speakingofmedicine/2013/08/13/full-registration-andreporting-of-all-trials-at-plos-medicine/) and Trials (http:// www.trialsjournal.com/), although of utmost importance to investigators and clinicians, has no benefit in deterring selective reporting of outcomes. On the contrary, it could convey a false message of timely registration if the journal does not inform readers of the date on which the trial was registered, so the reader can check it against that of recruitment information, which should be provided in the article. Top-ranked journals belonging to the International Committee of Medical Journal Editors are still publishing results from trials that were not registered in a timely manner [5] and readers are unaware of this unless they check the trial registration date in the registry, something that seldom occurs. There are initiatives aiming to increase registration of clinical trials. For instance, prospective registration started to be (or will be) mandatory between January 2013 and January 2015 for 11 journals belonging to the International Society of Physiotherapy Journal Editors [6]. In India, editors of 11 biomedical journals have required since 2010 that trials must be prospectively registered for publication of the results [7]. However, only 28% of journal editors ask for registration of clinical trials [8]. Some editors might not understand its benefit or believe that asking for it will place their journal at a disadvantage in publishing trials. In this respect, some editors argue that registration will penalize trials from developing countries because there are no registries in these countries [8]. This is a spurious claim: there are primary registries in the World Health Organization Registry Network in countries such as Brazil, China, Cuba, India, Iran, Sri Lanka, and Thailand (http:// www.who.int/ictrp/network/en/); furthermore, ClinicalTrials.gov hosts studies from 187 countries (of the 193 United Nations member states), which clearly underscores how widespread the use of this registry is among investigators worldwide. Health authorities could implement regulations aimed to ensure the registration and public disclosure of trial results. The UK Research Health Authority started by 2
Refs [11] [13] [14]
[15]
requesting registration of all clinical trials to be approved by a REC from 30 September 2013 onward. Registration should occur within 6 weeks of the first participant being recruited in the UK. From September 2014, when submitting a new trial to a REC for review sponsors must explicitly declare that the trial has been registered; from April 2015, sponsors seeking review for a new trial will have to state that all other trials in the recruitment phase have also been registered (http://www.hra.nhs.uk/documents/ 2014/10/hra-research-transparency-registration-key-messages-q.pdf). It is somewhat frustrating to observe that many trials are not registered and many of those registered show serious deficiencies in the quality of the provided information; for instance, a sample of 400 recently registered clinical trials showed that only 58% of primary outcomes were appropriately described and only 52% of drugs or biologics trials had complete information on intervention specifics [9]. Looking forward Although in the near future selective reporting of trial outcomes could most likely be only efficiently prevented by the implementation of specific measures by journal editors [10], prospective registration should be encouraged since it is the first step toward achieving full trial transparency. If some articles reporting the results of trials that were appropriately registered include distorted results, we should wonder about the quality of the results reported in articles on trials that were not registered but which are acceptable for publication by 72% of journals [8]. Both the lack of adequate and timely registration and the lack of (or lack of timely) publication of trial results [5,6,11,12] suggest that many investigators and research and academic institutions still have not internalized the need for full trial transparency. If this presumption is correct, the only way to have most investigators comply with prospective registration would be through the critical involvement of funders, RECs, and journal editors. These stakeholders could halt trial conduct and publication unless adequate and timely prospective registration of the trial is proven. Scientific associations can play their role too, through adequately and regularly informing all members about the benefit that transparency brings to the clinical trial enterprise and asking for a clear commitment from their members toward prospective registration (and publication). Finally, publishers are starting to be committed toward clinical trial transparency. Thus, for instance, whereas Nature Publishing Group
TIPS-1186; No. of Pages 3
Science & Society (http://www.nature.com/authors/gta.pdf) requires prospective registration of trials, Elsevier (http://www.elsevier.com/ connect/homepage/alltrials-campaign-supports-greaterclinical-trial-transparency) is assessing how to insert trial registration numbers into the peer-review processes and Wiley (http://authorservices.wiley.com/bauthor/publicationethics.asp#_Toc149460099) is encouraging all journal editors to require trial registration. The steps taken by the UK Health Research Authority, some journal editors and publishers, and many other stakeholders could be followed by their peers in other countries and health-care disciplines. The question is whether this will occur, to what extent and when. Meanwhile, we have to admit that some investigators will report what they believe is relevant to have their trial published, but will conceal or alter important information that all clinical trial stakeholders have the moral duty (and should work) to have it openly disclose to article readers. Disclaimer statement The opinions expressed in this article are those of the author and may not reflect the opinions of the organization for which he works.
References 1 Becker, J.E. et al. (2014) Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 311, 1063–1065 2 Mann, E. et al. (2014) Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing 43, 872–876 3 Dwan, K. et al. (2014) Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 11, e1001666
Trends in Pharmacological Sciences xxx xxxx, Vol. xxx, No. x 4 Kasenda, B. et al. (2014) Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications. BMJ 349, g4539 5 Huser, V. and Cimino, J.J. (2013) Evaluating adherence to the International Committee of Medical Journal Editors’ policy of mandatory, timely clinical trial registration. J. Am. Med. Inform. Assoc. 20, e169–e174 6 Costa, L.O. et al. (2013) Clinical trial registration in physiotherapy journals: recommendations from the International Society of Physiotherapy Journal Editors. Man. Ther. 18, 1–3 7 Satyanarayana, K. et al. (2008) Statement on publishing clinical trials in Indian biomedical journals. Natl. Med. J. India 21, 105–106 8 Wager, E. et al. (2013) Hardly worth the effort’’? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ 347, f5248 9 Viergever, R.F. et al. (2014) The quality of registration of clinical trials: still a problem. PLoS ONE 9, e84727 10 Dal-Re´, R. and Caplan, A.L. (2014) Time to ensure that clinical trial appropriate results are actually published. Eur. J. Clin. Pharmacol. 70, 491–493 11 Riveros, C. et al. (2014) Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Med. 10, e1001566 12 Saito, H. and Gill, C.J. (2014) How frequently do the results from completed US clinical trials enter the public domain? A statistical analysis of the ClinicalTrials.gov database. PLoS ONE 9, e101826 13 Hartung, D.M. et al. (2014) Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann. Intern. Med. 160, 477–483 14 Hughes, S. et al. (2014) Differences in reporting serious adverse events in industry sponsored clinical trial registries and journal articles on antidepressant and antipsychotic drugs: a cross sectional study. BMJ Open 4, e005535 15 Earley, A. et al. (2013) Haphazard reporting of deaths in clinical trials: a review of cases of ClinicalTrials.gov records and matched publications – a cross-sectional study. BMJ Open 3, e001963
3