- Email: [email protected]
Does the Oral “Mock Board” Examination Still Have a Role as a Training Tool?
ARTICLE IN PRESS
Radiology Resident Education
Does the Oral “Mock Board” Examination Still Have a Role as a Training Tool? Colin Strickland, MD, Alexandria Jensen, BA, BS, Tatum McArthur, MD Rationale and Objectives: The American Board of Radiology has adopted a new standardized board examination and the traditional oral examination has been abandoned. Although many programs have changed their educational efforts to reflect the new test format, some faculty members and residents have expressed a desire to keep an oral examination as a component of education and evaluation in radiology residency programs. Materials and Methods: An oral comprehensive examination including all the appropriate subspecialties was administered to each second year and third year resident in our training program by faculty members. Both the resident examinees and faculty examiners were surveyed after the examination to gauge the perceived value of the experience. Results: Residents were divided in their perceptions of the fairness and utility of an oral examination as a tool to aid in board preparation and as an assessment of their knowledge and communication skill. Faculty members were universal in their endorsement of the oral examination and suggested continued use of the technique. Conclusions: Residents and faculty members have differing perceptions of an oral examination delivered during training to assess knowledge and communication skill. The value of an oral examination in providing actionable feedback to trainees and the possibility of detecting struggling residents made it useful in our training program, and it thus it has been implemented for future years. Whether resident performance measured by this technique is predictive of success on American Board of Radiology examinations remains unclear. Key Words: Education; examination; residency. © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.
INTRODUCTION
R
adiology residency programs have undergone significant changes in their approach to board preparation with the advent of the American Board of Radiology (ABR) computerized Core Examination, which was introduced in 2013. Traditionally, oral examinations and casebased reviews have been a major component of board preparation and continue to be used by residency programs. The most recent survey of academic chief residents shows a continued reliance on internal board review preparation, with 81% of programs incorporating this approach (1). The use of a mock oral board examination to assess fund of knowledge and communication was common in years past (2) but seems to be less so with the changed board examination format. Detailed descriptions of how best to conduct a mock radiology oral examination exist in the literature (3) and have served as the template for similar exercises performed at many training Acad Radiol 2017; ■:■■–■■ From the Division of Musculoskeletal Radiology, Department of Radiology, University of Colorado School of Medicine, 12631 E. 17th Avenue, Aurora, CO 80045 (C.S., T.MA.); Department of Biostatistics & Informatics, Colorado School of Public Health, CO (A.J.). Received December 31, 2016; revised June 11, 2017; accepted June 19, 2017. Address correspondence to: C.S. e-mail: [email protected] © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.acra.2017.06.012
programs, including University of Colorado. In addition to preparing residents for a high-stakes oral assessment of their medical knowledge by replicating the experience of being shown cases with little or no guidance, the mock board examination also focused on how to “take a case” and is reflective of what radiologists do in clinical practice. At our institution before the new ABR examination format change, mock oral examinations were given to members of the second, third, and fourth year classes each year. Although the examination served as a final preparation for most senior residents, it was a useful way to generate feedback to the more junior participants and for the program to identify struggling trainees. A growing perception in our program has been that it is increasingly difficult to identify residents in need of remediation, and a cohesive style of communication when describing cases in conference was lacking among junior residents. In years past, this skill developed in junior residents as they modeled these skills from their more senior resident colleagues. For the reasons described above, we set out to revisit the notion of having a formal comprehensive oral examination to recapture the benefits that this process had brought to the program and residents in years past. MATERIALS AND METHODS Second and third year residents were included in the comprehensive oral examination, which was given in two sessions. 1
STRICKLAND ET AL
The second year residents took the examination in May 2016. Taking the oral examination 1 year before the Core Examination would provide feedback on performance and identify specific areas in need of remediation. The third year residents took the oral examination in June of 2016, immediately after the Core Examination. The first year residents were excluded as they have a separate call-readiness examination assessment at the end of the academic year. The fourth year residents were also excluded because of their involvement in mini-fellowships, and assessment of their performance in those areas could be better assessed by the corresponding clinical division. Eight second year residents and eight third year residents participated in the oral examination, each resident being examined on one of the 2 days offered. Each half-day oral comprehensive examination session began with a 15-minute orientation for the residents being examined to discuss the day’s schedule and format. The May 2016 session for the second year residents included the following subspecialties: chest, neuroradiology, abdominal imaging (including fluoroscopy), ultrasound, pediatrics, musculoskeletal imaging, interventional radiology, and nuclear medicine. Eight examiners participated, administering testing sessions to each resident. All of the examiners had either previously been an ABR examiner or had participated in previous departmental mock examinations. Therefore, each had experience administering cases in the oral format. The June session for the third year residents included the same specialties, with the addition of breast imaging and cardiac imaging sections. Because of an acute faculty shortage in the nuclear medicine division, that subspecialty was not included in the second session, leaving nine examiners and subspecialties in total. For both sessions, each faculty examiner had a computer with imaging cases they had personally selected to test important diagnoses and concepts for their specific subspecialty. No specific guidance was given to the faculty in the selection of cases nor was a specific bank of cases provided. Instead, faculty members were encouraged to choose cases of clinical relevance and on topics critical to the understanding of each subspecialty. Each session was 20 minutes in length, with 5-minute breaks to allow examinees to move between offices and for faculty to write out comments and complete grading sheets. The overall format for the examination closely paralleled that described in the literature (3) and what had been used at our institution in years past when the ABR oral board examination was still being administered. After each session was completed, the faculty examiners met and discussed the performance of each resident with grading on a 1–5 scale (1—unacceptable, 2—needs improvement, 3—at expected level of training, 4—exceeds expectations, and 5—ready for practice) for each subspecialty. Second and third year residents were graded differently to reflect their level of training. Although some of the same cases were shown to both groups, the level of detail expected in the resident discussion and understanding of treatment options was different between the two groups. The grading sheets and a summary 2
Academic Radiology, Vol ■, No ■■, ■■ 2017
of faculty comments from this meeting were given to each participating resident by the Associate Program Director during the following weeks. The Residency Program Director fully endorsed the oral comprehensive examination and was given a summary of each resident’s performance at the conclusion of the examination for inclusion in their file. An anonymous survey was given to each examiner and examinee to complete immediately at the end of the session to gather feedback for the program before residents and faculty had the opportunity to discuss their experience with one another.
RESULTS All 16 residents who participated in the oral comprehensive examination (eight second year residents and eight third year residents) as well as all examining faculty completed surveys at the conclusion of their session. Although 100% of surveyed faculty had a positive impression of the examination (ie, totally or somewhat agreed) with the statement “The experience of the oral comprehensive examination was useful to me in assessing strengths and weaknesses,” only 50% of surveyed residents felt the same way (Fig 1). Specific resident comments were mixed, with some of the participants reporting that the experience was needed in our curriculum, very useful as it helped identified areas of improvement, and enjoyable, whereas others found the examination futile given the new computerized format. In addition, although 100% of surveyed faculty had a positive response (ie, totally or somewhat agreed) with the statement “This examination is a good reflection of resident fund of knowledge in radiology,” only 53.3% of surveyed residents felt the same way (Fig 2). When asked during which year(s) of residency should an oral comprehensive examination be given, faculty tended choose both the second and the third years. In contrast, 30% of all residents responded that the examination should not be administered at all (Fig 3). It is also worth noting that more than half of faculty examiners also somewhat agreed with the statement: “The residents are less well prepared to discuss cases in this format than in years past when the oral board examination was held” (Fig 4).
DISCUSSION Programs are searching for objective ways to assess residents during training. During radiology residency, residents are evaluated for competence in the six Accreditation Council for Graduate Medical Education Milestones, which include Patient Care and Technical Skills, Medical Knowledge, Systembased Practice, Practice-based Learning and Improvement, Professionalism, and Interpersonal and Communication Skills (4). In the published milestones, simulation or objective structured clinical examination is listed as a possible method of
Academic Radiology, Vol ■, No ■■, ■■ 2017
ORAL “MOCK BOARD” EXAMINATION
Figure 1. (a) Faculty response to the question: “The experience of an oral comprehensive examination was useful to me in assessing the strengths and weaknesses of the examined residents.” (b) Resident response to the question: “The experience of an oral comprehensive examination was useful to me in assessing my strengths and weaknesses.” (c) Selected resident free-text comments about the oral comprehensive examination. (d) Selected faculty free text comments about the oral comprehensive examination.
Figure 2. (a) Faculty response to the question: “This examination is a good reflection of resident fund of knowledge in radiology.” (b) Resident response to the question: “This examination is a good reflection of my fund of knowledge in radiology.”
3
STRICKLAND ET AL
Academic Radiology, Vol ■, No ■■, ■■ 2017
Figure 3. (a) Faculty response to the question: “Which year(s) of residency should include an oral comprehensive examination? (Please circle all that apply or none).” (b) Resident response to the question: “Which year(s) of residency should include an oral comprehensive examination? (Please circle all that apply or none).”
Figure 4. Faculty response to the question: “The residents are less well prepared to discuss cases in this format than in years past when the oral board examination was held.”
assessment under each of the six competencies. With the oral comprehensive examination, the residents are being evaluated on all of the required milestones, especially Patient Care and Technical Skills, Medical Knowledge, and Interpersonal and Communication Skills. The oral examination provides another layer of assessment of critical elements of 4
competency in radiology, as we are assessing what we do in practice every day. Our residency training program includes daily didactic lectures, and each clinical service holds a 30-minute daily case conference for residents. A weekly residency-wide case conference is also given by a resident to classmates and faculty.
Academic Radiology, Vol ■, No ■■, ■■ 2017
Despite these numerous case conference activities, the reintroduction of a formal oral comprehensive examination in our residency program was seen as a needed improvement by the Program Director and many faculty members who felt that the examination format was helpful in assessing resident knowledge and communication. Resident perceptions were more mixed, with some residents reporting that the experience was helpful, whereas others described it as an onerous additional exercise of questionable value given the new computerized examination format and abundance of existing case conferences. It is important to consider the role that reactive devaluation may have played in resident survey responses. Several residents expressed dismay at the new requirement of an oral comprehensive examination and, thus, their survey responses likely reflected that sentiment. It is difficult to draw simple conclusions from our data as the expressed sentiments split along lines dividing those taking the examination from those tasked with its administration. For faculty, the absence of a formal oral examination component in resident training in the past few years has been seen more as an aberration and departure from decades of tradition in training and examination. It is important as well to recognize that the impressions of our faculty examiners are certainly shaped by the experience they themselves had as an examinee at the ABR oral boards and each can easily recall a time when oral examination was a standard part of most training programs. In addition, selection bias of the participating faculty is important to consider when drawing conclusions from survey data. The examination was administered by those faculty who volunteered their time and energy to the project such that their survey responses likely reflect the shared belief in that group that an oral testing format is of value. There is a rich history of residency level oral examination in the specialty of diagnostic radiology, with authors describing benefits to trainees in terms of boards preparation, selfassessment, and practice in presenting skills (3). The departure from an oral examination format by the ABR has been met with some consternation. In fact, 91% of surveyed Program Directors of radiology residency programs in 2014–2015 felt that the ABR Oral Examination was superior to the Core Examination in testing readiness for clinical practice (5). This is striking despite the immense effort and academic rigor that has gone into both development of the new examination (6,7) and efforts by the ABR to disseminate information about the new test format and how programs and residents may best prepare for it (8). It may be too early to tell if this attitude will persist as more graduates of the current testing structure enter clinical practice and take on leadership roles in academic departments. It certainly seems worth considering what (if any) role a formal oral examination process may continue to play in residency training programs.
ORAL “MOCK BOARD” EXAMINATION
Training programs in other specialties have experimented with a variety of oral examination formats including a process during which senior residents are evaluated publicly with more junior residents in attendance. One such General Surgery training program reported this format as a valuable educational tool for both those directly participating in and those simply observing the “public mock oral” examination (9). Given the critical importance of communication and consultation in the field of Diagnostic Radiology, a format such as this during which more junior trainees observe and later model an accepted approach to cases could prove useful. Residents of course already participate in daily case conferences and get ample opportunity to describe cases and formulate differential diagnoses. Despite this, it is important to note that more than half of the faculty in our survey reported that they felt residents were less well prepared to discuss cases in a formal oral examination format than in past years. What may have changed in radiology residency training programs is that with the shift to a new computer-based examination format, a clear standard of how to “take a case” that was expertly exhibited by senior residents in past years has been missing as an ideal to which more junior residents can aspire. Whether the reintroduction of a similar experience into our residency program at the University of Colorado will reverse this trend (and ultimately how much that matters) will be the subject or further research.
REFERENCES 1. Hammer MM, Shetty AS, Cizman Z, et al. Results of the 2015 survey of the American Alliance of Academic Chief Residents in Radiology. Acad Radiol 2015; 22:1308–1316. 2. Khan FA, Bhalla S, Jost RG. Results of the 2001 survey of the American Association of Academic Chief Residents in Radiology. Acad Radiol 2002; 9:89–97. 3. Canon CL, Mulligan S, Koehler RE. Mock radiology oral examination. Acad Radiol 2005; 12:368–372. 4. Accreditation Council for Graduate Medical Education (ACGME). The Diagnostic radiology milestone project. ACGME website. Published July 2015. Available at: https://www.acgme.org/Portals/0/PDFs/Milestones/ DiagnosticRadiologyMilestones.pdf. Accessed October 31, 2016. 5. Rozenshtein A, Heitkamp DE, Muhammed TL, et al. “What Program Directors Think” III: results of the 2014/2015 annual surveys of the Association of Program Directors in Radiology (APDR). Acad Radiol 2016; 23:861– 869. 6. Hollingsworth CL, Wriston CC, Bisset GS, et al. American Board of Radiology certifying examination: oral versus computer-based format. AJR Am J Roentgenol 2010; 195:820–824. [Comparative Study Research Support, Non-U.S. Gov’t]. 7. Pressman BD, Hoffman TR. ACR white paper: task force on timing of oral boards in diagnostic radiology. J Am Coll Radiol 2008; 5:1112– 1117. 8. Becker GJ, Bosma JL, Guiberteau MJ, et al. ABR examinations: the why, what, and how. Radiology 2013; 268:219–227. 9. Aboulian A, Schwartz S, Kaji AH, et al. The public mock oral: a useful tool for examinees and the audience in preparation for the American Board of Surgery Certifying Examination. J Surg Educ 2010; 67:33–36.
5
Radiology Resident Education
Does the Oral “Mock Board” Examination Still Have a Role as a Training Tool? Colin Strickland, MD, Alexandria Jensen, BA, BS, Tatum McArthur, MD Rationale and Objectives: The American Board of Radiology has adopted a new standardized board examination and the traditional oral examination has been abandoned. Although many programs have changed their educational efforts to reflect the new test format, some faculty members and residents have expressed a desire to keep an oral examination as a component of education and evaluation in radiology residency programs. Materials and Methods: An oral comprehensive examination including all the appropriate subspecialties was administered to each second year and third year resident in our training program by faculty members. Both the resident examinees and faculty examiners were surveyed after the examination to gauge the perceived value of the experience. Results: Residents were divided in their perceptions of the fairness and utility of an oral examination as a tool to aid in board preparation and as an assessment of their knowledge and communication skill. Faculty members were universal in their endorsement of the oral examination and suggested continued use of the technique. Conclusions: Residents and faculty members have differing perceptions of an oral examination delivered during training to assess knowledge and communication skill. The value of an oral examination in providing actionable feedback to trainees and the possibility of detecting struggling residents made it useful in our training program, and it thus it has been implemented for future years. Whether resident performance measured by this technique is predictive of success on American Board of Radiology examinations remains unclear. Key Words: Education; examination; residency. © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.
INTRODUCTION
R
adiology residency programs have undergone significant changes in their approach to board preparation with the advent of the American Board of Radiology (ABR) computerized Core Examination, which was introduced in 2013. Traditionally, oral examinations and casebased reviews have been a major component of board preparation and continue to be used by residency programs. The most recent survey of academic chief residents shows a continued reliance on internal board review preparation, with 81% of programs incorporating this approach (1). The use of a mock oral board examination to assess fund of knowledge and communication was common in years past (2) but seems to be less so with the changed board examination format. Detailed descriptions of how best to conduct a mock radiology oral examination exist in the literature (3) and have served as the template for similar exercises performed at many training Acad Radiol 2017; ■:■■–■■ From the Division of Musculoskeletal Radiology, Department of Radiology, University of Colorado School of Medicine, 12631 E. 17th Avenue, Aurora, CO 80045 (C.S., T.MA.); Department of Biostatistics & Informatics, Colorado School of Public Health, CO (A.J.). Received December 31, 2016; revised June 11, 2017; accepted June 19, 2017. Address correspondence to: C.S. e-mail: [email protected] © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.acra.2017.06.012
programs, including University of Colorado. In addition to preparing residents for a high-stakes oral assessment of their medical knowledge by replicating the experience of being shown cases with little or no guidance, the mock board examination also focused on how to “take a case” and is reflective of what radiologists do in clinical practice. At our institution before the new ABR examination format change, mock oral examinations were given to members of the second, third, and fourth year classes each year. Although the examination served as a final preparation for most senior residents, it was a useful way to generate feedback to the more junior participants and for the program to identify struggling trainees. A growing perception in our program has been that it is increasingly difficult to identify residents in need of remediation, and a cohesive style of communication when describing cases in conference was lacking among junior residents. In years past, this skill developed in junior residents as they modeled these skills from their more senior resident colleagues. For the reasons described above, we set out to revisit the notion of having a formal comprehensive oral examination to recapture the benefits that this process had brought to the program and residents in years past. MATERIALS AND METHODS Second and third year residents were included in the comprehensive oral examination, which was given in two sessions. 1
STRICKLAND ET AL
The second year residents took the examination in May 2016. Taking the oral examination 1 year before the Core Examination would provide feedback on performance and identify specific areas in need of remediation. The third year residents took the oral examination in June of 2016, immediately after the Core Examination. The first year residents were excluded as they have a separate call-readiness examination assessment at the end of the academic year. The fourth year residents were also excluded because of their involvement in mini-fellowships, and assessment of their performance in those areas could be better assessed by the corresponding clinical division. Eight second year residents and eight third year residents participated in the oral examination, each resident being examined on one of the 2 days offered. Each half-day oral comprehensive examination session began with a 15-minute orientation for the residents being examined to discuss the day’s schedule and format. The May 2016 session for the second year residents included the following subspecialties: chest, neuroradiology, abdominal imaging (including fluoroscopy), ultrasound, pediatrics, musculoskeletal imaging, interventional radiology, and nuclear medicine. Eight examiners participated, administering testing sessions to each resident. All of the examiners had either previously been an ABR examiner or had participated in previous departmental mock examinations. Therefore, each had experience administering cases in the oral format. The June session for the third year residents included the same specialties, with the addition of breast imaging and cardiac imaging sections. Because of an acute faculty shortage in the nuclear medicine division, that subspecialty was not included in the second session, leaving nine examiners and subspecialties in total. For both sessions, each faculty examiner had a computer with imaging cases they had personally selected to test important diagnoses and concepts for their specific subspecialty. No specific guidance was given to the faculty in the selection of cases nor was a specific bank of cases provided. Instead, faculty members were encouraged to choose cases of clinical relevance and on topics critical to the understanding of each subspecialty. Each session was 20 minutes in length, with 5-minute breaks to allow examinees to move between offices and for faculty to write out comments and complete grading sheets. The overall format for the examination closely paralleled that described in the literature (3) and what had been used at our institution in years past when the ABR oral board examination was still being administered. After each session was completed, the faculty examiners met and discussed the performance of each resident with grading on a 1–5 scale (1—unacceptable, 2—needs improvement, 3—at expected level of training, 4—exceeds expectations, and 5—ready for practice) for each subspecialty. Second and third year residents were graded differently to reflect their level of training. Although some of the same cases were shown to both groups, the level of detail expected in the resident discussion and understanding of treatment options was different between the two groups. The grading sheets and a summary 2
Academic Radiology, Vol ■, No ■■, ■■ 2017
of faculty comments from this meeting were given to each participating resident by the Associate Program Director during the following weeks. The Residency Program Director fully endorsed the oral comprehensive examination and was given a summary of each resident’s performance at the conclusion of the examination for inclusion in their file. An anonymous survey was given to each examiner and examinee to complete immediately at the end of the session to gather feedback for the program before residents and faculty had the opportunity to discuss their experience with one another.
RESULTS All 16 residents who participated in the oral comprehensive examination (eight second year residents and eight third year residents) as well as all examining faculty completed surveys at the conclusion of their session. Although 100% of surveyed faculty had a positive impression of the examination (ie, totally or somewhat agreed) with the statement “The experience of the oral comprehensive examination was useful to me in assessing strengths and weaknesses,” only 50% of surveyed residents felt the same way (Fig 1). Specific resident comments were mixed, with some of the participants reporting that the experience was needed in our curriculum, very useful as it helped identified areas of improvement, and enjoyable, whereas others found the examination futile given the new computerized format. In addition, although 100% of surveyed faculty had a positive response (ie, totally or somewhat agreed) with the statement “This examination is a good reflection of resident fund of knowledge in radiology,” only 53.3% of surveyed residents felt the same way (Fig 2). When asked during which year(s) of residency should an oral comprehensive examination be given, faculty tended choose both the second and the third years. In contrast, 30% of all residents responded that the examination should not be administered at all (Fig 3). It is also worth noting that more than half of faculty examiners also somewhat agreed with the statement: “The residents are less well prepared to discuss cases in this format than in years past when the oral board examination was held” (Fig 4).
DISCUSSION Programs are searching for objective ways to assess residents during training. During radiology residency, residents are evaluated for competence in the six Accreditation Council for Graduate Medical Education Milestones, which include Patient Care and Technical Skills, Medical Knowledge, Systembased Practice, Practice-based Learning and Improvement, Professionalism, and Interpersonal and Communication Skills (4). In the published milestones, simulation or objective structured clinical examination is listed as a possible method of
Academic Radiology, Vol ■, No ■■, ■■ 2017
ORAL “MOCK BOARD” EXAMINATION
Figure 1. (a) Faculty response to the question: “The experience of an oral comprehensive examination was useful to me in assessing the strengths and weaknesses of the examined residents.” (b) Resident response to the question: “The experience of an oral comprehensive examination was useful to me in assessing my strengths and weaknesses.” (c) Selected resident free-text comments about the oral comprehensive examination. (d) Selected faculty free text comments about the oral comprehensive examination.
Figure 2. (a) Faculty response to the question: “This examination is a good reflection of resident fund of knowledge in radiology.” (b) Resident response to the question: “This examination is a good reflection of my fund of knowledge in radiology.”
3
STRICKLAND ET AL
Academic Radiology, Vol ■, No ■■, ■■ 2017
Figure 3. (a) Faculty response to the question: “Which year(s) of residency should include an oral comprehensive examination? (Please circle all that apply or none).” (b) Resident response to the question: “Which year(s) of residency should include an oral comprehensive examination? (Please circle all that apply or none).”
Figure 4. Faculty response to the question: “The residents are less well prepared to discuss cases in this format than in years past when the oral board examination was held.”
assessment under each of the six competencies. With the oral comprehensive examination, the residents are being evaluated on all of the required milestones, especially Patient Care and Technical Skills, Medical Knowledge, and Interpersonal and Communication Skills. The oral examination provides another layer of assessment of critical elements of 4
competency in radiology, as we are assessing what we do in practice every day. Our residency training program includes daily didactic lectures, and each clinical service holds a 30-minute daily case conference for residents. A weekly residency-wide case conference is also given by a resident to classmates and faculty.
Academic Radiology, Vol ■, No ■■, ■■ 2017
Despite these numerous case conference activities, the reintroduction of a formal oral comprehensive examination in our residency program was seen as a needed improvement by the Program Director and many faculty members who felt that the examination format was helpful in assessing resident knowledge and communication. Resident perceptions were more mixed, with some residents reporting that the experience was helpful, whereas others described it as an onerous additional exercise of questionable value given the new computerized examination format and abundance of existing case conferences. It is important to consider the role that reactive devaluation may have played in resident survey responses. Several residents expressed dismay at the new requirement of an oral comprehensive examination and, thus, their survey responses likely reflected that sentiment. It is difficult to draw simple conclusions from our data as the expressed sentiments split along lines dividing those taking the examination from those tasked with its administration. For faculty, the absence of a formal oral examination component in resident training in the past few years has been seen more as an aberration and departure from decades of tradition in training and examination. It is important as well to recognize that the impressions of our faculty examiners are certainly shaped by the experience they themselves had as an examinee at the ABR oral boards and each can easily recall a time when oral examination was a standard part of most training programs. In addition, selection bias of the participating faculty is important to consider when drawing conclusions from survey data. The examination was administered by those faculty who volunteered their time and energy to the project such that their survey responses likely reflect the shared belief in that group that an oral testing format is of value. There is a rich history of residency level oral examination in the specialty of diagnostic radiology, with authors describing benefits to trainees in terms of boards preparation, selfassessment, and practice in presenting skills (3). The departure from an oral examination format by the ABR has been met with some consternation. In fact, 91% of surveyed Program Directors of radiology residency programs in 2014–2015 felt that the ABR Oral Examination was superior to the Core Examination in testing readiness for clinical practice (5). This is striking despite the immense effort and academic rigor that has gone into both development of the new examination (6,7) and efforts by the ABR to disseminate information about the new test format and how programs and residents may best prepare for it (8). It may be too early to tell if this attitude will persist as more graduates of the current testing structure enter clinical practice and take on leadership roles in academic departments. It certainly seems worth considering what (if any) role a formal oral examination process may continue to play in residency training programs.
ORAL “MOCK BOARD” EXAMINATION
Training programs in other specialties have experimented with a variety of oral examination formats including a process during which senior residents are evaluated publicly with more junior residents in attendance. One such General Surgery training program reported this format as a valuable educational tool for both those directly participating in and those simply observing the “public mock oral” examination (9). Given the critical importance of communication and consultation in the field of Diagnostic Radiology, a format such as this during which more junior trainees observe and later model an accepted approach to cases could prove useful. Residents of course already participate in daily case conferences and get ample opportunity to describe cases and formulate differential diagnoses. Despite this, it is important to note that more than half of the faculty in our survey reported that they felt residents were less well prepared to discuss cases in a formal oral examination format than in past years. What may have changed in radiology residency training programs is that with the shift to a new computer-based examination format, a clear standard of how to “take a case” that was expertly exhibited by senior residents in past years has been missing as an ideal to which more junior residents can aspire. Whether the reintroduction of a similar experience into our residency program at the University of Colorado will reverse this trend (and ultimately how much that matters) will be the subject or further research.
REFERENCES 1. Hammer MM, Shetty AS, Cizman Z, et al. Results of the 2015 survey of the American Alliance of Academic Chief Residents in Radiology. Acad Radiol 2015; 22:1308–1316. 2. Khan FA, Bhalla S, Jost RG. Results of the 2001 survey of the American Association of Academic Chief Residents in Radiology. Acad Radiol 2002; 9:89–97. 3. Canon CL, Mulligan S, Koehler RE. Mock radiology oral examination. Acad Radiol 2005; 12:368–372. 4. Accreditation Council for Graduate Medical Education (ACGME). The Diagnostic radiology milestone project. ACGME website. Published July 2015. Available at: https://www.acgme.org/Portals/0/PDFs/Milestones/ DiagnosticRadiologyMilestones.pdf. Accessed October 31, 2016. 5. Rozenshtein A, Heitkamp DE, Muhammed TL, et al. “What Program Directors Think” III: results of the 2014/2015 annual surveys of the Association of Program Directors in Radiology (APDR). Acad Radiol 2016; 23:861– 869. 6. Hollingsworth CL, Wriston CC, Bisset GS, et al. American Board of Radiology certifying examination: oral versus computer-based format. AJR Am J Roentgenol 2010; 195:820–824. [Comparative Study Research Support, Non-U.S. Gov’t]. 7. Pressman BD, Hoffman TR. ACR white paper: task force on timing of oral boards in diagnostic radiology. J Am Coll Radiol 2008; 5:1112– 1117. 8. Becker GJ, Bosma JL, Guiberteau MJ, et al. ABR examinations: the why, what, and how. Radiology 2013; 268:219–227. 9. Aboulian A, Schwartz S, Kaji AH, et al. The public mock oral: a useful tool for examinees and the audience in preparation for the American Board of Surgery Certifying Examination. J Surg Educ 2010; 67:33–36.
5