- Email: [email protected]
Is There a Need for Resident Training in Clinical Trial Design?
International Journal of
Radiation Oncology biology
physics
www.redjournal.org
BRIEF REPORT AND OPINION
Is There a Need for Resident Training in Clinical Trial Design? Abigail T. Berman, MD,* Judy A. Shea, PhD,y Cordelia Baffic, BA,* and Neha Vapiwala, MD* *Department of Radiation Oncology, yOffice of Evaluation and Assessment Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania Received Nov 27, 2013. Accepted for publication Dec 14, 2013.
Introduction The success rate of radiation oncology (RO) clinical trials (CTs) is poor, with the majority of trials not achieving the projected accrual goals (1) and thereby lacking the power to achieve statistical significance (2). Academic radiation oncologists, as principal investigators (PIs) of such trials, are responsible for their design. However, there is no widely available formal training, nor is there a requirement to demonstrate competency in designing trials. This suggests a possible role for CT design instruction during RO residency (3, 4), as many trainees, if not already engaged in trials, ultimately will be future PIs. The ACGME requires RO residents to have statistics instruction (5); yet a survey showed that 30% had no statistics education during residency, and of those who did, 98% had less than 10 hours (6). In addition, the new ACGME Milestone Project is still in development, and CT training may be a valuable component for residents to achieve the highest level of performance in research-specific milestones (7, 8). We conducted a targeted needs assessment of all RO residents to determine whether residents are designing CTs and to identify the challenges to being an educated participant in the design process.
Methods A survey was developed by a RO faculty, resident, and education researcher. It was administered anonymously to participants of the Reprint requests to: Abigail T. Berman, MD, University of Pennsylvania School of Medicine, Hospital of the University of Pennsylvania, TRC-2, 3400 Civic Center Boulevard, Philadelphia, PA 19104. Tel: (215) 900-7257; E-mail: [email protected] Conflict of interest: none. Int J Radiation Oncol Biol Phys, Vol. 88, No. 4, pp. 969e970, 2014 0360-3016/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ijrobp.2013.12.023
Second Annual Clinical Trials Course at the University of Pennsylvania before the course (“live”), and was also sent as a Webbased (“online”) survey to all RO residents nationwide who did not complete it live. The survey contained multiple choice and rating scale questions.
Results A total of 146 residents completed the survey; 94 of a potential of 413 residents polled across the country (23%) responded online, and 52 of 54 (96%) participants responded live (overall response rate, 31%). Residents were in their postgraduate year 1 (1%), 2 (23%), 3 (29%), 4 (27%), or 5 (20%). Although 10% of residents had served as a PI to-date, 3 times as many (30%) were currently designing or were about to open a trial. Most often, trials being designed were phase 1 (43%) or 2 (43%), and less commonly phase 3 (11%). Only 25% of participants had undergone formal instruction in CT design, 40% of whom received it at their own institutions. This instruction usually was exclusively delivered as lectures (80%) and less frequently as interactive workshops (31%). Residents did not report high comfort levels with designing phase 1, 2, or 3 CT (Fig. 1); and the majority of residents (60%, 76%, and 69%) indicated that they would like instruction in phase 1, 2, and 3 CT design, respectively. The challenges to starting a CT most frequently reported were “protocol writing/finding mentorship” (44%) and “financial support” (30%). Only 15% and 7% identified “statistical” and “coordinator” support as obstacles, respectively. In addition to AcknowledgmentdWe would like to thank Anand Shah, MD, and Steve Hahn, MD, for their vision in developing a course on clinical trial design.
970
Berman et al.
International Journal of Radiation Oncology Biology Physics
Fig. 1. Residents’ comfort level with designing clinical trials (CTs), on a scale from 1 (least comfortable) to 7 (most comfortable). Categorized as low (1,2), moderate (3-5), or high (6,7) level of comfort. lectures on CT fundamentals, residents chose which of the following additional topics would be most helpful: 61% selected statistics, 13% radiobiology and biomarkers, 13% medical oncology, and 6% physics. When choosing the ideal course format, 32% preferred 2 days live, split evenly between didactics and workshops, to develop their own protocol; 26% 1 day live; 23% Web based; and 19% Web based with a 1-day live workshop.
Conclusions Increasing numbers of RO residents are designing CTs as PIs or are seeking to become involved, yet few have had formal education on the topic. The foremost barrier to starting a CT appears to be lack of mentorship, and statistics is the most desired topic for specialized training. There is a clear need for resources to train residents on how to properly design CTs.
References 1. Cheng SK, Dietrich MS, Dilts DM. Predicting accrual achievement: Monitoring accrual milestones of NCI-CTEP-sponsored clinical trials. Clin Cancer Res 2011;17:1947-1955.
2. BIO / BioMedTracker. Clinical Trial Success Rates Study [online]. Available at: http://www.biotech-now.org/wp-content/uploads/2011/02/ bio-ceo-biomedtracker-bio-study-handout-final-2-15-2011.pdf. Accessed January 14, 2014. 3. Green ML. Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-based medicine: A critical review of curricula. Acad Med 1999;74:686-694. 4. Windish DM, Huot SJ, Green ML. Medicine residents’ understanding of the biostatistics and results in the medical literature. JAMA 2007; 298:1010-1022. 5. Accreditation Council of Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Radiation Oncology 2009. Available at: http://dconnect.acgme.org/acWebsite/downloads/RRC_ progReq/430_radiation_oncology_01012009_f07012011.pdf. Accessed December 10, 2013. 6. Gondi V, Bernard JR Jr., Jabbari S, et al. Results of the 2005 to 2008 Association of Residents in Radiation Oncology surveys of chief residents in the United States: Didactics and research experience. Am J Clin Oncol 2012;35:32-39. 7. Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach 2007 Sep;29(7):648-654. 8. Accreditation Council of Graduate Medical Education. The next accreditation system. 2013. Available at: http://www.acgme.org/acgmeweb/tabid/ 435/ProgramandInstitutionalAccreditation/NextAccreditationSystem.aspx. Accessed January 14, 2014.
Radiation Oncology biology
physics
www.redjournal.org
BRIEF REPORT AND OPINION
Is There a Need for Resident Training in Clinical Trial Design? Abigail T. Berman, MD,* Judy A. Shea, PhD,y Cordelia Baffic, BA,* and Neha Vapiwala, MD* *Department of Radiation Oncology, yOffice of Evaluation and Assessment Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania Received Nov 27, 2013. Accepted for publication Dec 14, 2013.
Introduction The success rate of radiation oncology (RO) clinical trials (CTs) is poor, with the majority of trials not achieving the projected accrual goals (1) and thereby lacking the power to achieve statistical significance (2). Academic radiation oncologists, as principal investigators (PIs) of such trials, are responsible for their design. However, there is no widely available formal training, nor is there a requirement to demonstrate competency in designing trials. This suggests a possible role for CT design instruction during RO residency (3, 4), as many trainees, if not already engaged in trials, ultimately will be future PIs. The ACGME requires RO residents to have statistics instruction (5); yet a survey showed that 30% had no statistics education during residency, and of those who did, 98% had less than 10 hours (6). In addition, the new ACGME Milestone Project is still in development, and CT training may be a valuable component for residents to achieve the highest level of performance in research-specific milestones (7, 8). We conducted a targeted needs assessment of all RO residents to determine whether residents are designing CTs and to identify the challenges to being an educated participant in the design process.
Methods A survey was developed by a RO faculty, resident, and education researcher. It was administered anonymously to participants of the Reprint requests to: Abigail T. Berman, MD, University of Pennsylvania School of Medicine, Hospital of the University of Pennsylvania, TRC-2, 3400 Civic Center Boulevard, Philadelphia, PA 19104. Tel: (215) 900-7257; E-mail: [email protected] Conflict of interest: none. Int J Radiation Oncol Biol Phys, Vol. 88, No. 4, pp. 969e970, 2014 0360-3016/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ijrobp.2013.12.023
Second Annual Clinical Trials Course at the University of Pennsylvania before the course (“live”), and was also sent as a Webbased (“online”) survey to all RO residents nationwide who did not complete it live. The survey contained multiple choice and rating scale questions.
Results A total of 146 residents completed the survey; 94 of a potential of 413 residents polled across the country (23%) responded online, and 52 of 54 (96%) participants responded live (overall response rate, 31%). Residents were in their postgraduate year 1 (1%), 2 (23%), 3 (29%), 4 (27%), or 5 (20%). Although 10% of residents had served as a PI to-date, 3 times as many (30%) were currently designing or were about to open a trial. Most often, trials being designed were phase 1 (43%) or 2 (43%), and less commonly phase 3 (11%). Only 25% of participants had undergone formal instruction in CT design, 40% of whom received it at their own institutions. This instruction usually was exclusively delivered as lectures (80%) and less frequently as interactive workshops (31%). Residents did not report high comfort levels with designing phase 1, 2, or 3 CT (Fig. 1); and the majority of residents (60%, 76%, and 69%) indicated that they would like instruction in phase 1, 2, and 3 CT design, respectively. The challenges to starting a CT most frequently reported were “protocol writing/finding mentorship” (44%) and “financial support” (30%). Only 15% and 7% identified “statistical” and “coordinator” support as obstacles, respectively. In addition to AcknowledgmentdWe would like to thank Anand Shah, MD, and Steve Hahn, MD, for their vision in developing a course on clinical trial design.
970
Berman et al.
International Journal of Radiation Oncology Biology Physics
Fig. 1. Residents’ comfort level with designing clinical trials (CTs), on a scale from 1 (least comfortable) to 7 (most comfortable). Categorized as low (1,2), moderate (3-5), or high (6,7) level of comfort. lectures on CT fundamentals, residents chose which of the following additional topics would be most helpful: 61% selected statistics, 13% radiobiology and biomarkers, 13% medical oncology, and 6% physics. When choosing the ideal course format, 32% preferred 2 days live, split evenly between didactics and workshops, to develop their own protocol; 26% 1 day live; 23% Web based; and 19% Web based with a 1-day live workshop.
Conclusions Increasing numbers of RO residents are designing CTs as PIs or are seeking to become involved, yet few have had formal education on the topic. The foremost barrier to starting a CT appears to be lack of mentorship, and statistics is the most desired topic for specialized training. There is a clear need for resources to train residents on how to properly design CTs.
References 1. Cheng SK, Dietrich MS, Dilts DM. Predicting accrual achievement: Monitoring accrual milestones of NCI-CTEP-sponsored clinical trials. Clin Cancer Res 2011;17:1947-1955.
2. BIO / BioMedTracker. Clinical Trial Success Rates Study [online]. Available at: http://www.biotech-now.org/wp-content/uploads/2011/02/ bio-ceo-biomedtracker-bio-study-handout-final-2-15-2011.pdf. Accessed January 14, 2014. 3. Green ML. Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-based medicine: A critical review of curricula. Acad Med 1999;74:686-694. 4. Windish DM, Huot SJ, Green ML. Medicine residents’ understanding of the biostatistics and results in the medical literature. JAMA 2007; 298:1010-1022. 5. Accreditation Council of Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Radiation Oncology 2009. Available at: http://dconnect.acgme.org/acWebsite/downloads/RRC_ progReq/430_radiation_oncology_01012009_f07012011.pdf. Accessed December 10, 2013. 6. Gondi V, Bernard JR Jr., Jabbari S, et al. Results of the 2005 to 2008 Association of Residents in Radiation Oncology surveys of chief residents in the United States: Didactics and research experience. Am J Clin Oncol 2012;35:32-39. 7. Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach 2007 Sep;29(7):648-654. 8. Accreditation Council of Graduate Medical Education. The next accreditation system. 2013. Available at: http://www.acgme.org/acgmeweb/tabid/ 435/ProgramandInstitutionalAccreditation/NextAccreditationSystem.aspx. Accessed January 14, 2014.