- Email: [email protected]
Evaluation of American Board of Orthodontics certification protocols in postgraduate orthodontic programs in the United States and Canada
ORIGINAL ARTICLE
Evaluation of American Board of Orthodontics certification protocols in postgraduate orthodontic programs in the United States and Canada Jae Hyun Park,a Raphael R. Putrus,b Dawn P. Pruzansky,b and John Grubbb Mesa, Ariz, and Seoul, South Korea Introduction: The objective of this study was to identify the board certification protocols that hospital and university-based postgraduate orthodontic programs have in place to prepare residents for the American Board of Orthodontics (ABO) certification examination. Methods: An electronic survey was sent to the program directors of each of the 72 postgraduate orthodontic programs in the United States and Canada. The survey consisted of 49 questions about demographics, resident case assignment protocols, and ABO examination preparation methods. Results: The response rate was 81%. Most programs were 30 to 36 months in length (72.7%). Many residents had a case load of 51 to 75 during their first year (50.9%), with an average maximum case load of 70 to 109. There was a positive correlation with both the number of cases that first-year residents start and the length of the program (Spearman correlation coefficient 5 0.379; P \0.01) when compared with maximum case load. Approximately 72% of the programs do not offer a written mock board examination; however, 72% reported offering a clinical mock board examination. ABO cases are identified within the first 6 months of most programs. About 88% of respondents believe that residents take advantage of the banking system, and that over the past 5 years ABO Initial Certification Examination applications have increased. Conclusions: Most program directors (89.1%) believe that their program length is sufficient for board preparation. Subjects tested in the written examination are integrated into the didactic curriculum and strengthened with ongoing literature reviews, with a passing rate over 90%. Clinical examination preparation varies, with most programs requiring a mock board examination for graduation. Total participation in both the Initial Certification Examination and banking has increased since 2010; better follow-up protocols are needed to track residents after graduation. (Am J Orthod Dentofacial Orthop 2017;151:463-70)
C
urrently, there is no standardized protocol in place at postgraduate orthodontic residency programs to prepare students for the initial American Board of Orthodontics (ABO) certification process. However, a refinement of the ABO certification process began in 2005, making it possible for orthodontic residents to attain initial certification and increasing the desire among
a Postgraduate Orthodontic Program, Arizona School of Dentistry & Oral Health, A. T. Still University, Mesa, Ariz; Graduate School of Dentistry, Kyung Hee University, Seoul, South Korea. b Postgraduate Orthodontic Program, Arizona School of Dentistry & Oral Health, A. T. Still University, Mesa, Ariz. All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest, and none were reported. Address correspondence to: Jae Hyun Park, Postgraduate Orthodontic Program, Arizona School of Dentistry & Oral Health, A. T. Still University, 5835 E Still Cir, Mesa, AZ 85206; e-mail, [email protected]. Submitted, May 2016; revised and accepted, August 2016. 0889-5406/$36.00 Ó 2017 by the American Association of Orthodontists. All rights reserved. http://dx.doi.org/10.1016/j.ajodo.2016.08.022
residents to become certified.1,2 The Gateway Offer, implemented from 2005 to 2007, granted a 5-year ABO certificate to all interested practicing orthodontists who had successfully passed the written examination.3 During this time, a pilot study was conducted to determine whether residents were able to finish their cases with board-quality results.4 Once this was confirmed, a new Initial Certification Examination (ICE) was offered, starting with the class of 2007. This allowed residents to submit cases treated in their programs, promoting ABO certification early in their careers. In 2010, the “banking” process was introduced to allow residents to accumulate cases after graduation if they could not meet the case requirements during their program.5 The banking process is initiated after a candidate presents 3 cases; the total process must be completed in 10 years. An exemption request can be submitted for those initially presenting fewer than 3 cases with the understanding that 6 cases must still be completed in 10 years.6 463
Park et al
464
The Commission on Dental Accreditation (CODA), in its Accreditation Standards for Advanced Specialty Education Programs in Orthodontics and Dentofacial Orthopedics, standard 2-11, stated: “The program director and faculty must prepare students/residents to pursue certification by the American Board of Orthodontics,” but it is not up to the CODA or the American Association of Orthodontists to specify how that is accomplished.7 Because the ICE process makes board certification more attainable for recent graduates, it is the responsibility of each orthodontic residency program to develop its own protocol, employ knowledgeable faculty, and administer a program of adequate length to prepare its residents for the ICE process.8 CODA accreditation standards require all directors of advanced specialty education programs to be certified by an American Dental Association recognized certifying board in the specialty. This action was the initial step in increasing the standards for ABO certification.9 Since then, there have been numerous revisions to the certification process. Surveys by residents in 2003 and 2007 showed that 81% to 87% of the orthodontic resident respondents planned to become ABO certified.1,10 In addition to completing a CODA-approved orthodontic specialty program, an orthodontic resident must complete the following to become ABO certified: the ABO written examination, a board case oral examination, a case report examination, and a case report oral examination. A minimum of 3 to a maximum of 6 qualifying cases are completed by the candidate during residency and are used for the case report examination and the case report oral examination portion of the ICE process.11 An update to the ABO initial certification process has made it possible for graduating residents to achieve board certification using cases treated under faculty supervision. This change has made it far more important to have certified faculty members with knowledge of the certification process and defined criteria for assessing case difficulty. The combination of shortages in recruitment and retention of orthodontic educators and a diminished desire by orthodontic residents to become educators has created a need for programs to develop a standardized protocol for teaching residents and keeping faculty up to date with the certification process.10,12 The length of orthodontic residency programs is a hotly debated topic, and there is an ongoing discussion about whether 24-month programs are long enough to sufficiently educate an orthodontic resident in the didactic and clinical regimens necessary to be professionally competent.13-15 Some proponents of 3-year programs believe that the increase in program length would allow residents to treat more cases
March 2017 Vol 151 Issue 3
with better case demographics, thereby increasing their practical understanding before they take the ABO examination.13,15 The current curriculum guidelines set by the American Association of Orthodontists for orthodontic residency programs are broad and allow for great variations between programs.16,17 Therefore, the objective of this study was to assess whether the board-certification protocols that are in place in hospital and university-based postgraduate orthodontic programs are adequate to prepare residents for the ABO certification examination. In addition, this information may identify methods to increase the number of residents becoming board certified through the ICE process. MATERIAL AND METHODS
After we received approval from the A. T. Still University Institutional Review Board, an electronic survey (Qualtrics, Provo, Utah) was created. An e-mail link to the survey was sent to the program directors or chairpersons of each of the 72 postgraduate orthodontic programs in the United States and Canada, requesting their anonymous participation in the survey. The e-mail link was personalized to each program so that follow-up e-mails and calls could be sent only to those who had not previously responded and so that multiple responses were not allowed. The survey consisted of 49 questions using a logic section format in which only applicable questions, based on previous responses, were asked to respondents. These logic sections included director and program demographics, resident case assignment protocols, and ABO written and clinical examination preparation methods. All questions were multiple choice, although some questions allowed users to enter additional comments. Over a 3-month period, 2 e-mail requests were sent, followed by a final e-mail request to each person who had not yet responded. Four e-mail addresses were bounced back, bringing the total number of surveys sent to 68. Fifty-five programs participated in the survey, resulting in an 81% response rate. Survey data were compiled using Excel (Microsoft, Redmond, Wash). Statistical analysis
Descriptive statistics, including means (standard deviations), ranges, and counts (percentages), were calculated. Spearman correlation coefficients (rs) were calculated to estimate the strength of the relationships between program characteristics and outcomes of interest. The criterion for statistical significance was P\0.05, 2-tailed. SPSS software (version 23; IBM, Armonk, NY) was used to analyze the data.
American Journal of Orthodontics and Dentofacial Orthopedics
Park et al
465
Table I. Demographic characteristic of survey respon-
Table II. Characteristics of program clinical compo-
dents and programs
nents
Characteristic % of respondents (n) Sex Male 85.5 (47) Female 14.5 (8) Age range (y) 36-45 20.1 (11) 46-55 20.1 (11) 56-65 25.3 (14) 66 and over 34.5 (19) ABO certification status Certified 87.3 (48) Noncertified 12.7 (7) Full-time (FT) clinical Total FT FT ABO certified faculty members faculty faculty 0-2 21.8 (12) 58.3 (32) 3-4 49.1 (27) 30.9 (17) 5 and over 29 (16) 10.2 (6) Part-time (PT) clinical Total PT PT ABO certified faculty members faculty faculty 0-10 40 (22) 76.2 (42) 11-20 38 (21) 16.3 (9) 21 and over 10.8 (6) 7.2 (4) Are you an ABO examiner? Yes 32.7 (18) No 67.3 (37) Length of postgraduate orthodontic program (mo) 24-29 25.5 (14) 30-36 72.7 (40) Over 36 1.8 (1) Number of residents per class 1-3 12.7 (7) 4-6 65.5 (36) 7-10 20.0 (11) 11 and over 1.8 (1) Degrees offered MS only 7.3 (4) Certificate only 16.4 (9) MS and certificate 72.7 (40) Other 9.1 (5)
Characteristic % of respondents (n) Average case load of first-year residents Up to 50 starts 34.5 (19) 51-75 starts 50.9 (28) Over 75 starts 14.5 (8) Time of incoming resident's starting cases First month of residency 27.3 (15) Second month of residency 47.3 (26) Third month of residency 20.0 (11) Percentage of patients started and finished by the same resident 0%-30% 32.2 (18) 31%-60% 21.7 (12) 61%-90% 45.0 (25) Maximum case load of residents* 60-89 32.7 (18) 90-109 34.5 (19) 110-129 12.7 (7) 130 and over 20.0 (11) Maximum case load reached First year of residency 5.5 (3) Second year of residency 76.4 (42) Third year of residency 12.7 (7) Residents always accept new patients 5.5 (3) Length of program sufficient for residents to complete ABO board cases Yes 87.3 (48) No 12.7 (7) Ideal length of program 30 months 5.5 (3) 36 months 5.5 (3)
RESULTS
Of the 55 respondents, 85.5% were male, and 14.5% were female. Approximately 87% of the directors who responded reported their board certification status, and 33% were board examiners. Most programs were 30 to 36 months in length (72.7%), and a majority had 4 to 6 residents per class (65.5%). The number of residents accepted per year was positively correlated with the number of full-time ABO certified faculty members (rs 5 0.364; P \0.01), suggesting that the larger programs employed more full-time ABO certified faculty. Most directors thought that their program length was sufficient for residents to complete ABO cases (89.1%), and 72.7% of programs offered a combined MS/certificate degree. A majority of both full-
*Includes new and transfer patients.
time and part-time faculty members were board certified (Table I). The most common reason provided for why faculty members were not board certified was: “It was not stressed during their residency,” followed by “inadequate patients” and “not interested.” “Stricter ABO case requirements at the time of graduation” was the least common reason. Some respondents stated that board certification was a requirement to teach in their program, so this question did not apply to them. Many residents had a case load of 51 to 75 during their first year (50.9%), with most cases being started in the second month of residency (47.3%) (Table II). The average maximum case load ranged from 70 to 109, with this number reached by the second year of residency. There was a positive correlation with both the number of cases first-year residents start and the length of the postgraduate orthodontic program (rs 5 0.379; P \0.01) when compared with maximum case load. In other words, a longer program is associated with more cases started, hence a larger case load.
American Journal of Orthodontics and Dentofacial Orthopedics
March 2017 Vol 151 Issue 3
Park et al
466
Table III. Written ABO examination preparation % of respondents Characteristics (n) When does the program begin preparing residents to take the written board examination? 1-6 months into the program 58.5 (31) 6-12 months into the program 17 (9) 13-18 months into the program 17 (9) 19-24 months into the program 7.5 (4) How does the program prepare residents for the written board examination? Program preparation course 47.3 (26) CD-ABO preparation course 9.1 (5) Literature review course 78.2 (43) Other (curriculum is geared toward 30.9 (17) examination) Does the program administer a mock board examination? Yes 28.3 (15) No 71.7 (38) Is it a graduation requirement to pass the written examination? Yes 43.4 (23) No 56.6 (30) What is the pass rate for the past 5 years? 76%-90% pass 1.9 (1) Over 91% pass 98.1 (52) What is the consequence for failing the written examination? Retake examination before graduation 34 (18) Take additional examination administered by 5.7 (3) program No remediation 34 (18) Other (not experienced) 26.4 (14) CD, College of diplomats.
A resident's maximum case load was negatively correlated with the numbers of both full-time and part-time clinical faculty members who are ABO certified (rs 5 0.280; rs 5 0.315; both, P \0.05), suggesting that the number of patients a resident can treat may be limited by the number of faculty present for supervision. There was also a negative correlation between the number of cases residents started in their first year and the time when incoming residents begin bonding their cases (rs 5 0.304; P \0.05), showing that residents can bond more cases when they start during the first few months of their program. A variable factor in residency programs is case distribution; programs attempt to ensure an equal distribution of various case types to each resident. Case distribution methods varied from Discrepancy Index score to Angle classification and random distribution. About 38% of respondents distributed patients based on “other” criteria, ranging from extraction need to degree of difficulty. Most cases were distributed by the clinical director (58.2%). Pretreatment Discrepancy Index scores were predominantly performed by the residents (89.1%).
March 2017 Vol 151 Issue 3
A majority of programs begin examination preparation within the first year of residency, with 58.5% preparing during the first 6 months (Table III). Ongoing literature review is the most popular method of examination preparation (78.2%), and 47.3% of programs offer a formal in-house preparation course. Of those that selected “other” for the examination preparation method, 16% reported in the comments section that the curriculum is geared toward the examination, so an additional preparation course is not necessary. Approximately 72% of programs do not offer a written mock board examination. Although the majority of programs do not require a passing grade for graduation, 98% of programs reported a pass rate over 90%. Since the failure rate is low, many programs do not have a remediation plan in place, although 34% of programs prefer that the resident take the examination again before graduation. In contrast to the written examination, about 72% reported offering a clinical mock board examination graded according to ABO standards, which includes the Cast-Radiograph Evaluation form and the Case Management Form, by calibrated faculty. ABO cases are identified within the first 6 months of most programs. The most common time frame to monitor ABO case progress is “quarterly” (21.8%), and it is incorporated into overall resident evaluations in 64.2% of programs. Case progress is evaluated by numerous methods in addition to the resident evaluations, including case presentations, discussions with attending faculty and self-evaluation by the resident. Outside ABO certified guests (70.9%) or in-house faculty (27.3%) explain the certification process to the residents. Residents present their completed cases to faculty (63.6%), coresidents (49.1%), and faculty panels (25.5%). About 88% of respondents believe that residents take advantage of the banking system and that over the past 5 years, ABO ICE applications have increased (Table IV). The average percentage of residents who apply for the ABO ICE process immediately after graduation showed a positive correlation with the number of full-time clinical faculty members who are ABO certified (rs 5 0.330; P \0.05); this may imply a push toward certification by the faculty members. “Not interested” and “insufficient number of patients” were the most common reasons that a new graduate did not pursue ABO certification, followed by “required classifications difficult to attain.” “Inadequate faculty support,” “international status,” and “other” (most commonly cited as “cost”) were the least common reasons. After graduation, the most common reasons that a resident did not complete the ICE,
American Journal of Orthodontics and Dentofacial Orthopedics
Park et al
Table IV. ICE preparation % of respondents Characteristic (n) Do you administer a mock ABO clinical examination? Yes 71.7 (38) No 28.3 (15) How is ABO case progress evaluated? Discussion during resident evaluation 41.8 (23) Presentation of ABO cases, including records 58.2 (32) Discussion with attending faculty 69.1 (38) Self-evaluation by resident 47.3 (26) Other (1 faculty member) 10.9 (6) Does the program invite lecturers to explain the certification process? ABO advocate 70.9 (39) Other (in-house) 27.3 (15) No 16.4 (9) Does the program offer any reimbursement for ABO examination expenses? ABO registration fee 14.5 (8) ABO written examination fee 36.4 (20) ABO clinical examination fee 9.1 (5) CD-ABO preparation course 100 (55) Other 12.7 (7) No 52.7 (29) Residents are required to present their cases to Faculty 63.6 (35) Coresidents 49.1 (27) Calibrated faculty panel in a defense type of 25.5 (14) setting Other 25.5 (14) Residents identify ABO cases 1-3 months into program 34.5 (19) 4-6 months into program 34.5 (19) Other (up to 1 year) 15.6 (7) Do residents take advantage of the ICE banking system? Yes 87.5 (42) No 12.5 (6) CD, College of diplomats.
with the banking system in place, were “lack of longterm job stability,” “corporate environment not ideal for ABO preparation,” and “starting a practice/lack of patient pool.” The least common reason was “not interested.” DISCUSSION
The ABO certification process incorporates both course-work knowledge and clinical competency for the graduating orthodontic resident. It is the ABO's belief that through this process, “a national, standardized expectation of clinical competency” will be maintained.2,18 In 1974, the written examination was first offered immediately after graduate training; it is the first step in the certification pathway.19 Currently, the examination consists of 240 multiple choice questions on
467
biomedical sciences, clinical sciences, and orthodontics. Most programs require residents to take this examination during the second year of residency, and the passing rate is high. Grading of the examination follows a standardized scale with a range of 100 to 1000 points; 500 points is the minimum passing score, and 580 is the national average.20 Since the examination topics are based on CODA requirements, many programs do not offer a separate preparation course. The program curriculum and literature reviews prepare the residents for the variety of orthodontic topics that may be encountered. The high pass rate indicates that residency programs provide the necessary didactic education to prepare for the written examination. The Gateway process allowed residents to submit cases treated during their program, in the hope of increasing ABO certification.2 Five years were allowed to complete the clinical examination after acceptance of the offer. The board has since revised the regulations regarding timing and case requirements. Currently, the ICE consists of a board case oral examination, a case report examination, and a case report oral examination; 6 cases of varying criteria are required.11 The clinical aspect of the examination necessitates a presentation and understanding by the resident of all aspects of his or her treated orthodontic cases from diagnostic records to final treatment outcomes. The examiner evaluates all aspects of the case being presented, and ABO standards of excellence must be met throughout the entirety of the case. Compared with the high participation and success of the written examination, it seems that the postgraduate orthodontic programs diverge in regard to their ICE status. The importance placed on ABO status, number of ABO certified faculty members, patient case load and distribution, resident motivation, and accountability varies from program to program. Although preparation for the written examination seems standardized across the programs, the clinical component adds the complication of variability, since all residents do not encounter the same malocclusions in their patient population. The extent to which the programs attempt to control and distribute this variability is of utmost importance when compiling potential board patients. The ABO may be able to help with this by introducing some flexibility into the case report examination. For example, the ABO might consider an option to substitute a nonsurgical treatment of a full-step Class III for a Class II. If the ABO standards and certification processes are to be practiced and understood by orthodontic residents before their certification, several aspects must be included in the teaching protocol. First, using certification-required knowledge as a curriculum
American Journal of Orthodontics and Dentofacial Orthopedics
March 2017 Vol 151 Issue 3
Park et al
468
guideline would help establish ABO standards as the base expectation for all residents. Since the certification process is all encompassing, a written protocol should outline the course work and clinical competency expected of the resident and his or her ability to remain current on evidence-based literature. The protocol should be clear regarding the objectives and carefully outline what is expected for ABO certification. Additionally, the resident should have the ability to verbally defend his or her case management approach with course-work knowledge and current literature. A successful protocol for preparing residents for the certification process must include faculty responsibilities. Trends toward fewer full-time faculty and more part-time faculty require the protocol to clearly define the qualifications required of each faculty member.8 Faculty members should be versed in the current certification process as well as ABO standards and mandated training so that they can maintain their knowledge base. Every 3 years, the ABO directors attempt to visit their constituent postgraduate orthodontic programs and discuss Discrepancy Index scoring and CastRadiograph Evaluation forms and to present sample cases.21 A protocol that defines the required faculty training and regulation process using the ABO certification guidelines would benefit residents and the program alike. As a result, there would be fewer concerns over the support provided by multiple part-time faculty members related to subjective matters. A majority of the faculty members in our study, both full-time and part-time, reported they were ABO certified. This criterion is the first step in establishing a clinical protocol for resident treatment of potential ABO cases. ABO case progress was most commonly monitored quarterly, and some respondents reported that 1 faculty member oversees this component of the program. As the knowledge of board requirements increases, the residents may be able to treat more cases to the board standard of care under the guidance of competent clinical faculty. This promotes treatment consistent with board standards, which may translate to an increased pool of patients that qualify as ICE cases. Most directors believe that the length of their program is sufficient for residents to complete board cases. During the implementation of changes to the certification process, a pilot study by Dykhouse et al4 found that orthodontic residents were capable of treating, managing, and presenting cases according to the ABO standard of care. Organized by the ABO, the study recruited 50 incoming orthodontic residents from the class of 2002 to present their best qualifying resident-treated cases alongside the traditional certification candidates in 2006. The results of the study showed that there was no
March 2017 Vol 151 Issue 3
significant difference in the pass rates between graduating residents and traditional orthodontic examinees. Orthodontic residents have shown that it is possible to acquire the ability to treat to the ABO standard within the allotted residency period.4 Our study highlights that although most directors believe their program is long enough to promote ABO certification and that residents are banking cases, there is no concrete protocol to monitor the completion rate of ABO certification. Most programs do not track residents after graduation to follow up on ABO progress, so the assumption reported in our survey that recent graduates completed their ICE in “1 to 2 years” cannot be verified. According to the ABO, “cases presented which are evaluated as ‘Complete’ will be banked, or documented, at the ABO central office. The examinee will have ten (10) years and two (2) more attempts to collect the balance of cases. Residency cases may only be used towards the balance if the cases are submitted within the first 24 month period after graduation.”11 It seems that the more cases are completed in residency, the more likely are the residents to follow through with the process. Long-term data are needed to determine whether the banking program has resulted in more recent graduates attaining ABO certification. With the majority of patients started in the first 3 months of residency, this allows ample time to finish and work up cases to ABO standards. However, after graduation, most residents are not permitted to return to the clinic to finish ABO cases because of university policies and malpractice insurance regulations. The burden lies with the university to provide boardeligible cases at the onset of the residency, so that they can be treated within the time frame of the program. Once the patient pool is in place and the faculty members serve as advisors and mentors, the process becomes more streamlined. According to our study, distribution methods vary between programs, with random distribution the most common method. Discrepancy Index scores are typically calculated by residents and may be the method by which board cases are identified. However, all potential cases should be reviewed and rescored by attending faculty. Only 18% of 136 orthodontic residents polled in 2007 believed that certification should be mandatory for licensure.1 Unlike other specialties, the ABO certification process is still entirely voluntary and unrestricting if not achieved. Currently, there are 3419 ABO certified orthodontists; approximately 41% of active American Association of Orthodontists members are board certified.21 Other specialties require certification to maintain hospital privileges, but with the majority of orthodontic residents planning to go into private practice,
American Journal of Orthodontics and Dentofacial Orthopedics
Park et al
there is no requirement for certification. Even though this is the case, 95% of graduating orthodontists take the ABO written examination, and 81% to 87% plan to become certified.1,10,16 These statistics reinforce the fact that there is a desire among residents to become ABO certified and that more should be done by current orthodontic specialty programs to prepare their residents for the certification process. At the 2016 ABO Educators Symposium, the results of the September 2015 and February 2016 clinical examinations were discussed. Of 1367 cases submitted for grading, 94% were complete (passing). The most common reason for being scored as incomplete (nonpassing) was treatment related (68%), with the Cast-Radiograph Evaluation, Case Management Form, or both being unacceptable. The average grade on the Cast-Radiograph Evaluation was 22, with the most points taken off for alignment (4.2 points), buccolingual inclination (3.8 points), and marginal ridges (3.5 points). On the Case Management Form, tracings (33%) and superimpositions (38%) were the most common components scored as unacceptable.20 Since tracings and superimpositions seem to be difficult components of the Case Management Form, increased concentration on these areas during residency may be needed. Incorporation of superimpositions into routine case presentations, as well as didactic lessens on hand tracing and digital tracing should improve residents’ understanding of these components. The banking component of the ICE process is in its infancy, so although it is promising, there are no longterm data to support or refute the benefit of the banking system. Since its inception in 2010, total participation and banking have increased steadily, with an average of 4 cases banked per examinee.20 Unfortunately, the lack of communication once a resident has graduated makes it difficult to track his or her board certification progress. Participation per graduation year has increased, but the status should be monitored in more detail. More information is needed on the success of the banking program, and there should be new graduate follow up. Interestingly, 1744 ABO members will be up for recertification after 2020.20 We must consider the most common reasons that board-certified orthodontists would choose not to take the First Certification Renewal Examination after 2020 and ensure maximum participation. New graduates are facing more hardships as they enter the work force, so they may delay board certification as they start their practices and families. Increased student loan debt and fewer employment opportunities translate to a longer transitional period as newer graduates are slower to purchase practices and start-ups.21
469
The high doctor turnover in corporate environments and low patient numbers in start-ups make it difficult to collect ideal board cases within the allotted time frame. The newer guidelines and banking process may make it easier to select cases in residency, decreasing the burden of finding appropriate patients once in private or corporate practice. Another barrier to certification is the cost of the examination. Although some programs (36.4%) include the cost of the written examination in their programs, only 9% are willing to reimburse the clinical examination fee. Although cost was the least common reason for not obtaining certification during residency, the question was asked from the program director's perspective. In reality, the cost of the examination plus travel and lodging expenses may be a deterrent for some new graduates. To avoid undue time constraints on program directors and chairs, the survey was designed to be as short as possible, but this limited the depth of detail that could be obtained from the results. Many additional questions could have been created to collect information about each program and program director, such as training or treatment philosophy. Each of these variables could be a confounding factor, influencing the practices of the program. Another potential limitation of the study was selection bias, introduced because of the voluntary nature of the survey. Some respondents may have chosen not to participate simply because the importance of board certification is not stressed in their programs. This selection bias may work in the opposite way, too. Program directors who promote board certification may be eager to share their views of the process and therefore be more likely to respond. The results must be carefully considered so as not to mislead the reader into thinking that the collected responses truly represent the entire population sampled. Lastly, this survey was distributed to program directors. The opinions of residents as to why they have decided whether to pursue board certification are still needed. Although graduate surveys have been distributed, they typically ask about all aspects of a program and do not delve into board certification beyond a few questions. Future research should be focused on a pathway to maximize the number of board-eligible cases finished in residency. In addition, hardships facing new graduates after the economic downturn and increasing numbers of corporate practices need to be considered. Many new graduates are interested in board certification, but “lack of long term job stability,” “a corporate environment not ideal for ABO preparation,” and “starting a practice/lack of patient pool” are reported barriers to
American Journal of Orthodontics and Dentofacial Orthopedics
March 2017 Vol 151 Issue 3
Park et al
470
success. We need to be aware of these changing trends and give our younger colleagues the tools to effectively achieve board status. CONCLUSIONS
1. 2.
3.
4.
6.
7.
Most program directors believe that their program length is sufficient for board preparation. Subjects tested by the written examination are integrated into the didactic curriculum and strengthened with ongoing literature reviews, having a passing rate over 90%. Clinical examination preparation varies from program to program, with most programs requiring a mock board examination for graduation. Total participation in ICE and banking has increased since 2010; better follow-up protocols are needed to track residents after graduation.
8. 9. 10.
11.
12. 13.
ACKNOWLEDGMENTS
We thank Cale Forgues for help with the literature review, Michael L. Riolo for manuscript review, and Curtis Bay for statistical analyses.
15. 16.
REFERENCES 1. Noble J, Hechter FJ, Karaiskos NE, Lekic N, Wiltshire WA. Future practice plans of orthodontic residents in the United States. Am J Orthod Dentofacial Orthop 2009;135:357-60. 2. Owens SE Jr, Dykhouse VJ, Moffitt AH, Grubb JE, Greco PM, English JD, et al. The new American Board of Orthodontics certification process: further clarification. Am J Orthod Dentofacial Orthop 2005;128:541-4. 3. Riolo ML, Owens SE, Dykhouse VJ, Moffitt AH, Grubb JE, Greco PM, et al. A change in the certification process by the American Board of Orthodontics. Am J Orthod Dentofacial Orthop 2005;127:278-81. 4. Dykhouse VJ, Moffitt AH, Grubb JE, Greco PM, English JD, Briss BS, et al. A report of the ABO resident clinical outcome study (the pilot study). Am J Orthod Dentofacial Orthop 2006;130:656-61. 5. Greco PM, English JD, Briss BS, Jamieson SA, Kastrop MC, Castelein PT, et al. Banking cases for the American Board of
March 2017 Vol 151 Issue 3
14.
17. 18.
19.
20.
21.
Orthodontics’ initial certification examination. Am J Orthod Dentofacial Orthop 2010;137:575-6. American Board of Orthodontics. Banking process. Available at: https://www.americanboardortho.com/media/4821/exam-excep tion-request-form.pdf. Accessed June 7, 2016. Commission on Dental Accreditation. Accreditation Standards for Advanced Specialty Education Programs in Orthodontics and Dentofacial Orthopedics. Available at: http://www.ada.org//media/ CODA/Files/ortho.ashx. Accessed June 7, 2016. Park JH. Retirement planning and practice operations in the aftermath of the recession. PCSO Bull 2015;87:6-7. Neumann LM, Nix JA. Trends in dental specialty education and practice, 1990-99. J Dent Educ 2002;66:1338-47. Bruner MK, Hilger KK, Silveir AM, Butters JM. Graduate orthodontic education: the resident’s perspective. Am J Orthod Dentofacial Orthop 2005;128:277-82. American Board of Orthodontics. Examination process. Available at: https://www.americanboardortho.com/orthodontic-professionals/ about-board-certification/clinical-examination/examination-require ments/initial-certification-examination/. Accessed April 7, 2016. Larson B. Faculty recruitment and retention: challenge or crisis. Am J Orthod Dentofacial Orthop 1998;113:122-3. Moskowitz EM. A mandatory 3-year orthodontic postgraduate program and the American Board of Orthodontics: what’s the connection? Am J Orthod Dentofacial Orthop 2003;124:2-3. Lindauer SJ. Is the time really right for mandatory 3-year orthodontic residency programs? Am J Orthod Dentofacial Orthop 2008;133:2-3. Cangialosi TJ. The time is right for a 3-year orthodontic curriculum. Am J Orthod Dentofacial Orthop 2007;132:135-6. Burk T, Orellana M. Assessment of graduate orthodontic programs in North America. J Dent Educ 2013;77:463-75. Keim RG, Sinclair PM. Orthodontic graduate education survey, 1983-2000. Am J Orthod Dentofacial Orthop 2002;121:2-8. Castelein PT, DeLeon E Jr, Dugoni SA, Chung CH, Tadlock LP, Barone ND, et al. Certification renewal process of the American Board of Orthodontics. Am J Orthod Dentofacial Orthop 2015; 147(Supp 5):S232-3. Cangialosi TJ, Riolo ML, Owens SE, Dykhouse VJ, Moffitt AH, Grubb JE, et al. The American Board of Orthodontics and specialty certification: the first 50 years. Am J Orthod Dentofacial Orthop 2004;126:3-6. ABO directors. Clinical challenges. Presented at the Charles Burstone-American Board of Orthodontics 2016 Educators Symposium, April 2, 2016; St Louis, Mo. Pruzansky D, Ellis B, Park JH. Influence of student-loan debt on orthodontic residents and recent graduates. J Clin Orthod 2016; 50:23-32.
American Journal of Orthodontics and Dentofacial Orthopedics
Evaluation of American Board of Orthodontics certification protocols in postgraduate orthodontic programs in the United States and Canada Jae Hyun Park,a Raphael R. Putrus,b Dawn P. Pruzansky,b and John Grubbb Mesa, Ariz, and Seoul, South Korea Introduction: The objective of this study was to identify the board certification protocols that hospital and university-based postgraduate orthodontic programs have in place to prepare residents for the American Board of Orthodontics (ABO) certification examination. Methods: An electronic survey was sent to the program directors of each of the 72 postgraduate orthodontic programs in the United States and Canada. The survey consisted of 49 questions about demographics, resident case assignment protocols, and ABO examination preparation methods. Results: The response rate was 81%. Most programs were 30 to 36 months in length (72.7%). Many residents had a case load of 51 to 75 during their first year (50.9%), with an average maximum case load of 70 to 109. There was a positive correlation with both the number of cases that first-year residents start and the length of the program (Spearman correlation coefficient 5 0.379; P \0.01) when compared with maximum case load. Approximately 72% of the programs do not offer a written mock board examination; however, 72% reported offering a clinical mock board examination. ABO cases are identified within the first 6 months of most programs. About 88% of respondents believe that residents take advantage of the banking system, and that over the past 5 years ABO Initial Certification Examination applications have increased. Conclusions: Most program directors (89.1%) believe that their program length is sufficient for board preparation. Subjects tested in the written examination are integrated into the didactic curriculum and strengthened with ongoing literature reviews, with a passing rate over 90%. Clinical examination preparation varies, with most programs requiring a mock board examination for graduation. Total participation in both the Initial Certification Examination and banking has increased since 2010; better follow-up protocols are needed to track residents after graduation. (Am J Orthod Dentofacial Orthop 2017;151:463-70)
C
urrently, there is no standardized protocol in place at postgraduate orthodontic residency programs to prepare students for the initial American Board of Orthodontics (ABO) certification process. However, a refinement of the ABO certification process began in 2005, making it possible for orthodontic residents to attain initial certification and increasing the desire among
a Postgraduate Orthodontic Program, Arizona School of Dentistry & Oral Health, A. T. Still University, Mesa, Ariz; Graduate School of Dentistry, Kyung Hee University, Seoul, South Korea. b Postgraduate Orthodontic Program, Arizona School of Dentistry & Oral Health, A. T. Still University, Mesa, Ariz. All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest, and none were reported. Address correspondence to: Jae Hyun Park, Postgraduate Orthodontic Program, Arizona School of Dentistry & Oral Health, A. T. Still University, 5835 E Still Cir, Mesa, AZ 85206; e-mail, [email protected]. Submitted, May 2016; revised and accepted, August 2016. 0889-5406/$36.00 Ó 2017 by the American Association of Orthodontists. All rights reserved. http://dx.doi.org/10.1016/j.ajodo.2016.08.022
residents to become certified.1,2 The Gateway Offer, implemented from 2005 to 2007, granted a 5-year ABO certificate to all interested practicing orthodontists who had successfully passed the written examination.3 During this time, a pilot study was conducted to determine whether residents were able to finish their cases with board-quality results.4 Once this was confirmed, a new Initial Certification Examination (ICE) was offered, starting with the class of 2007. This allowed residents to submit cases treated in their programs, promoting ABO certification early in their careers. In 2010, the “banking” process was introduced to allow residents to accumulate cases after graduation if they could not meet the case requirements during their program.5 The banking process is initiated after a candidate presents 3 cases; the total process must be completed in 10 years. An exemption request can be submitted for those initially presenting fewer than 3 cases with the understanding that 6 cases must still be completed in 10 years.6 463
Park et al
464
The Commission on Dental Accreditation (CODA), in its Accreditation Standards for Advanced Specialty Education Programs in Orthodontics and Dentofacial Orthopedics, standard 2-11, stated: “The program director and faculty must prepare students/residents to pursue certification by the American Board of Orthodontics,” but it is not up to the CODA or the American Association of Orthodontists to specify how that is accomplished.7 Because the ICE process makes board certification more attainable for recent graduates, it is the responsibility of each orthodontic residency program to develop its own protocol, employ knowledgeable faculty, and administer a program of adequate length to prepare its residents for the ICE process.8 CODA accreditation standards require all directors of advanced specialty education programs to be certified by an American Dental Association recognized certifying board in the specialty. This action was the initial step in increasing the standards for ABO certification.9 Since then, there have been numerous revisions to the certification process. Surveys by residents in 2003 and 2007 showed that 81% to 87% of the orthodontic resident respondents planned to become ABO certified.1,10 In addition to completing a CODA-approved orthodontic specialty program, an orthodontic resident must complete the following to become ABO certified: the ABO written examination, a board case oral examination, a case report examination, and a case report oral examination. A minimum of 3 to a maximum of 6 qualifying cases are completed by the candidate during residency and are used for the case report examination and the case report oral examination portion of the ICE process.11 An update to the ABO initial certification process has made it possible for graduating residents to achieve board certification using cases treated under faculty supervision. This change has made it far more important to have certified faculty members with knowledge of the certification process and defined criteria for assessing case difficulty. The combination of shortages in recruitment and retention of orthodontic educators and a diminished desire by orthodontic residents to become educators has created a need for programs to develop a standardized protocol for teaching residents and keeping faculty up to date with the certification process.10,12 The length of orthodontic residency programs is a hotly debated topic, and there is an ongoing discussion about whether 24-month programs are long enough to sufficiently educate an orthodontic resident in the didactic and clinical regimens necessary to be professionally competent.13-15 Some proponents of 3-year programs believe that the increase in program length would allow residents to treat more cases
March 2017 Vol 151 Issue 3
with better case demographics, thereby increasing their practical understanding before they take the ABO examination.13,15 The current curriculum guidelines set by the American Association of Orthodontists for orthodontic residency programs are broad and allow for great variations between programs.16,17 Therefore, the objective of this study was to assess whether the board-certification protocols that are in place in hospital and university-based postgraduate orthodontic programs are adequate to prepare residents for the ABO certification examination. In addition, this information may identify methods to increase the number of residents becoming board certified through the ICE process. MATERIAL AND METHODS
After we received approval from the A. T. Still University Institutional Review Board, an electronic survey (Qualtrics, Provo, Utah) was created. An e-mail link to the survey was sent to the program directors or chairpersons of each of the 72 postgraduate orthodontic programs in the United States and Canada, requesting their anonymous participation in the survey. The e-mail link was personalized to each program so that follow-up e-mails and calls could be sent only to those who had not previously responded and so that multiple responses were not allowed. The survey consisted of 49 questions using a logic section format in which only applicable questions, based on previous responses, were asked to respondents. These logic sections included director and program demographics, resident case assignment protocols, and ABO written and clinical examination preparation methods. All questions were multiple choice, although some questions allowed users to enter additional comments. Over a 3-month period, 2 e-mail requests were sent, followed by a final e-mail request to each person who had not yet responded. Four e-mail addresses were bounced back, bringing the total number of surveys sent to 68. Fifty-five programs participated in the survey, resulting in an 81% response rate. Survey data were compiled using Excel (Microsoft, Redmond, Wash). Statistical analysis
Descriptive statistics, including means (standard deviations), ranges, and counts (percentages), were calculated. Spearman correlation coefficients (rs) were calculated to estimate the strength of the relationships between program characteristics and outcomes of interest. The criterion for statistical significance was P\0.05, 2-tailed. SPSS software (version 23; IBM, Armonk, NY) was used to analyze the data.
American Journal of Orthodontics and Dentofacial Orthopedics
Park et al
465
Table I. Demographic characteristic of survey respon-
Table II. Characteristics of program clinical compo-
dents and programs
nents
Characteristic % of respondents (n) Sex Male 85.5 (47) Female 14.5 (8) Age range (y) 36-45 20.1 (11) 46-55 20.1 (11) 56-65 25.3 (14) 66 and over 34.5 (19) ABO certification status Certified 87.3 (48) Noncertified 12.7 (7) Full-time (FT) clinical Total FT FT ABO certified faculty members faculty faculty 0-2 21.8 (12) 58.3 (32) 3-4 49.1 (27) 30.9 (17) 5 and over 29 (16) 10.2 (6) Part-time (PT) clinical Total PT PT ABO certified faculty members faculty faculty 0-10 40 (22) 76.2 (42) 11-20 38 (21) 16.3 (9) 21 and over 10.8 (6) 7.2 (4) Are you an ABO examiner? Yes 32.7 (18) No 67.3 (37) Length of postgraduate orthodontic program (mo) 24-29 25.5 (14) 30-36 72.7 (40) Over 36 1.8 (1) Number of residents per class 1-3 12.7 (7) 4-6 65.5 (36) 7-10 20.0 (11) 11 and over 1.8 (1) Degrees offered MS only 7.3 (4) Certificate only 16.4 (9) MS and certificate 72.7 (40) Other 9.1 (5)
Characteristic % of respondents (n) Average case load of first-year residents Up to 50 starts 34.5 (19) 51-75 starts 50.9 (28) Over 75 starts 14.5 (8) Time of incoming resident's starting cases First month of residency 27.3 (15) Second month of residency 47.3 (26) Third month of residency 20.0 (11) Percentage of patients started and finished by the same resident 0%-30% 32.2 (18) 31%-60% 21.7 (12) 61%-90% 45.0 (25) Maximum case load of residents* 60-89 32.7 (18) 90-109 34.5 (19) 110-129 12.7 (7) 130 and over 20.0 (11) Maximum case load reached First year of residency 5.5 (3) Second year of residency 76.4 (42) Third year of residency 12.7 (7) Residents always accept new patients 5.5 (3) Length of program sufficient for residents to complete ABO board cases Yes 87.3 (48) No 12.7 (7) Ideal length of program 30 months 5.5 (3) 36 months 5.5 (3)
RESULTS
Of the 55 respondents, 85.5% were male, and 14.5% were female. Approximately 87% of the directors who responded reported their board certification status, and 33% were board examiners. Most programs were 30 to 36 months in length (72.7%), and a majority had 4 to 6 residents per class (65.5%). The number of residents accepted per year was positively correlated with the number of full-time ABO certified faculty members (rs 5 0.364; P \0.01), suggesting that the larger programs employed more full-time ABO certified faculty. Most directors thought that their program length was sufficient for residents to complete ABO cases (89.1%), and 72.7% of programs offered a combined MS/certificate degree. A majority of both full-
*Includes new and transfer patients.
time and part-time faculty members were board certified (Table I). The most common reason provided for why faculty members were not board certified was: “It was not stressed during their residency,” followed by “inadequate patients” and “not interested.” “Stricter ABO case requirements at the time of graduation” was the least common reason. Some respondents stated that board certification was a requirement to teach in their program, so this question did not apply to them. Many residents had a case load of 51 to 75 during their first year (50.9%), with most cases being started in the second month of residency (47.3%) (Table II). The average maximum case load ranged from 70 to 109, with this number reached by the second year of residency. There was a positive correlation with both the number of cases first-year residents start and the length of the postgraduate orthodontic program (rs 5 0.379; P \0.01) when compared with maximum case load. In other words, a longer program is associated with more cases started, hence a larger case load.
American Journal of Orthodontics and Dentofacial Orthopedics
March 2017 Vol 151 Issue 3
Park et al
466
Table III. Written ABO examination preparation % of respondents Characteristics (n) When does the program begin preparing residents to take the written board examination? 1-6 months into the program 58.5 (31) 6-12 months into the program 17 (9) 13-18 months into the program 17 (9) 19-24 months into the program 7.5 (4) How does the program prepare residents for the written board examination? Program preparation course 47.3 (26) CD-ABO preparation course 9.1 (5) Literature review course 78.2 (43) Other (curriculum is geared toward 30.9 (17) examination) Does the program administer a mock board examination? Yes 28.3 (15) No 71.7 (38) Is it a graduation requirement to pass the written examination? Yes 43.4 (23) No 56.6 (30) What is the pass rate for the past 5 years? 76%-90% pass 1.9 (1) Over 91% pass 98.1 (52) What is the consequence for failing the written examination? Retake examination before graduation 34 (18) Take additional examination administered by 5.7 (3) program No remediation 34 (18) Other (not experienced) 26.4 (14) CD, College of diplomats.
A resident's maximum case load was negatively correlated with the numbers of both full-time and part-time clinical faculty members who are ABO certified (rs 5 0.280; rs 5 0.315; both, P \0.05), suggesting that the number of patients a resident can treat may be limited by the number of faculty present for supervision. There was also a negative correlation between the number of cases residents started in their first year and the time when incoming residents begin bonding their cases (rs 5 0.304; P \0.05), showing that residents can bond more cases when they start during the first few months of their program. A variable factor in residency programs is case distribution; programs attempt to ensure an equal distribution of various case types to each resident. Case distribution methods varied from Discrepancy Index score to Angle classification and random distribution. About 38% of respondents distributed patients based on “other” criteria, ranging from extraction need to degree of difficulty. Most cases were distributed by the clinical director (58.2%). Pretreatment Discrepancy Index scores were predominantly performed by the residents (89.1%).
March 2017 Vol 151 Issue 3
A majority of programs begin examination preparation within the first year of residency, with 58.5% preparing during the first 6 months (Table III). Ongoing literature review is the most popular method of examination preparation (78.2%), and 47.3% of programs offer a formal in-house preparation course. Of those that selected “other” for the examination preparation method, 16% reported in the comments section that the curriculum is geared toward the examination, so an additional preparation course is not necessary. Approximately 72% of programs do not offer a written mock board examination. Although the majority of programs do not require a passing grade for graduation, 98% of programs reported a pass rate over 90%. Since the failure rate is low, many programs do not have a remediation plan in place, although 34% of programs prefer that the resident take the examination again before graduation. In contrast to the written examination, about 72% reported offering a clinical mock board examination graded according to ABO standards, which includes the Cast-Radiograph Evaluation form and the Case Management Form, by calibrated faculty. ABO cases are identified within the first 6 months of most programs. The most common time frame to monitor ABO case progress is “quarterly” (21.8%), and it is incorporated into overall resident evaluations in 64.2% of programs. Case progress is evaluated by numerous methods in addition to the resident evaluations, including case presentations, discussions with attending faculty and self-evaluation by the resident. Outside ABO certified guests (70.9%) or in-house faculty (27.3%) explain the certification process to the residents. Residents present their completed cases to faculty (63.6%), coresidents (49.1%), and faculty panels (25.5%). About 88% of respondents believe that residents take advantage of the banking system and that over the past 5 years, ABO ICE applications have increased (Table IV). The average percentage of residents who apply for the ABO ICE process immediately after graduation showed a positive correlation with the number of full-time clinical faculty members who are ABO certified (rs 5 0.330; P \0.05); this may imply a push toward certification by the faculty members. “Not interested” and “insufficient number of patients” were the most common reasons that a new graduate did not pursue ABO certification, followed by “required classifications difficult to attain.” “Inadequate faculty support,” “international status,” and “other” (most commonly cited as “cost”) were the least common reasons. After graduation, the most common reasons that a resident did not complete the ICE,
American Journal of Orthodontics and Dentofacial Orthopedics
Park et al
Table IV. ICE preparation % of respondents Characteristic (n) Do you administer a mock ABO clinical examination? Yes 71.7 (38) No 28.3 (15) How is ABO case progress evaluated? Discussion during resident evaluation 41.8 (23) Presentation of ABO cases, including records 58.2 (32) Discussion with attending faculty 69.1 (38) Self-evaluation by resident 47.3 (26) Other (1 faculty member) 10.9 (6) Does the program invite lecturers to explain the certification process? ABO advocate 70.9 (39) Other (in-house) 27.3 (15) No 16.4 (9) Does the program offer any reimbursement for ABO examination expenses? ABO registration fee 14.5 (8) ABO written examination fee 36.4 (20) ABO clinical examination fee 9.1 (5) CD-ABO preparation course 100 (55) Other 12.7 (7) No 52.7 (29) Residents are required to present their cases to Faculty 63.6 (35) Coresidents 49.1 (27) Calibrated faculty panel in a defense type of 25.5 (14) setting Other 25.5 (14) Residents identify ABO cases 1-3 months into program 34.5 (19) 4-6 months into program 34.5 (19) Other (up to 1 year) 15.6 (7) Do residents take advantage of the ICE banking system? Yes 87.5 (42) No 12.5 (6) CD, College of diplomats.
with the banking system in place, were “lack of longterm job stability,” “corporate environment not ideal for ABO preparation,” and “starting a practice/lack of patient pool.” The least common reason was “not interested.” DISCUSSION
The ABO certification process incorporates both course-work knowledge and clinical competency for the graduating orthodontic resident. It is the ABO's belief that through this process, “a national, standardized expectation of clinical competency” will be maintained.2,18 In 1974, the written examination was first offered immediately after graduate training; it is the first step in the certification pathway.19 Currently, the examination consists of 240 multiple choice questions on
467
biomedical sciences, clinical sciences, and orthodontics. Most programs require residents to take this examination during the second year of residency, and the passing rate is high. Grading of the examination follows a standardized scale with a range of 100 to 1000 points; 500 points is the minimum passing score, and 580 is the national average.20 Since the examination topics are based on CODA requirements, many programs do not offer a separate preparation course. The program curriculum and literature reviews prepare the residents for the variety of orthodontic topics that may be encountered. The high pass rate indicates that residency programs provide the necessary didactic education to prepare for the written examination. The Gateway process allowed residents to submit cases treated during their program, in the hope of increasing ABO certification.2 Five years were allowed to complete the clinical examination after acceptance of the offer. The board has since revised the regulations regarding timing and case requirements. Currently, the ICE consists of a board case oral examination, a case report examination, and a case report oral examination; 6 cases of varying criteria are required.11 The clinical aspect of the examination necessitates a presentation and understanding by the resident of all aspects of his or her treated orthodontic cases from diagnostic records to final treatment outcomes. The examiner evaluates all aspects of the case being presented, and ABO standards of excellence must be met throughout the entirety of the case. Compared with the high participation and success of the written examination, it seems that the postgraduate orthodontic programs diverge in regard to their ICE status. The importance placed on ABO status, number of ABO certified faculty members, patient case load and distribution, resident motivation, and accountability varies from program to program. Although preparation for the written examination seems standardized across the programs, the clinical component adds the complication of variability, since all residents do not encounter the same malocclusions in their patient population. The extent to which the programs attempt to control and distribute this variability is of utmost importance when compiling potential board patients. The ABO may be able to help with this by introducing some flexibility into the case report examination. For example, the ABO might consider an option to substitute a nonsurgical treatment of a full-step Class III for a Class II. If the ABO standards and certification processes are to be practiced and understood by orthodontic residents before their certification, several aspects must be included in the teaching protocol. First, using certification-required knowledge as a curriculum
American Journal of Orthodontics and Dentofacial Orthopedics
March 2017 Vol 151 Issue 3
Park et al
468
guideline would help establish ABO standards as the base expectation for all residents. Since the certification process is all encompassing, a written protocol should outline the course work and clinical competency expected of the resident and his or her ability to remain current on evidence-based literature. The protocol should be clear regarding the objectives and carefully outline what is expected for ABO certification. Additionally, the resident should have the ability to verbally defend his or her case management approach with course-work knowledge and current literature. A successful protocol for preparing residents for the certification process must include faculty responsibilities. Trends toward fewer full-time faculty and more part-time faculty require the protocol to clearly define the qualifications required of each faculty member.8 Faculty members should be versed in the current certification process as well as ABO standards and mandated training so that they can maintain their knowledge base. Every 3 years, the ABO directors attempt to visit their constituent postgraduate orthodontic programs and discuss Discrepancy Index scoring and CastRadiograph Evaluation forms and to present sample cases.21 A protocol that defines the required faculty training and regulation process using the ABO certification guidelines would benefit residents and the program alike. As a result, there would be fewer concerns over the support provided by multiple part-time faculty members related to subjective matters. A majority of the faculty members in our study, both full-time and part-time, reported they were ABO certified. This criterion is the first step in establishing a clinical protocol for resident treatment of potential ABO cases. ABO case progress was most commonly monitored quarterly, and some respondents reported that 1 faculty member oversees this component of the program. As the knowledge of board requirements increases, the residents may be able to treat more cases to the board standard of care under the guidance of competent clinical faculty. This promotes treatment consistent with board standards, which may translate to an increased pool of patients that qualify as ICE cases. Most directors believe that the length of their program is sufficient for residents to complete board cases. During the implementation of changes to the certification process, a pilot study by Dykhouse et al4 found that orthodontic residents were capable of treating, managing, and presenting cases according to the ABO standard of care. Organized by the ABO, the study recruited 50 incoming orthodontic residents from the class of 2002 to present their best qualifying resident-treated cases alongside the traditional certification candidates in 2006. The results of the study showed that there was no
March 2017 Vol 151 Issue 3
significant difference in the pass rates between graduating residents and traditional orthodontic examinees. Orthodontic residents have shown that it is possible to acquire the ability to treat to the ABO standard within the allotted residency period.4 Our study highlights that although most directors believe their program is long enough to promote ABO certification and that residents are banking cases, there is no concrete protocol to monitor the completion rate of ABO certification. Most programs do not track residents after graduation to follow up on ABO progress, so the assumption reported in our survey that recent graduates completed their ICE in “1 to 2 years” cannot be verified. According to the ABO, “cases presented which are evaluated as ‘Complete’ will be banked, or documented, at the ABO central office. The examinee will have ten (10) years and two (2) more attempts to collect the balance of cases. Residency cases may only be used towards the balance if the cases are submitted within the first 24 month period after graduation.”11 It seems that the more cases are completed in residency, the more likely are the residents to follow through with the process. Long-term data are needed to determine whether the banking program has resulted in more recent graduates attaining ABO certification. With the majority of patients started in the first 3 months of residency, this allows ample time to finish and work up cases to ABO standards. However, after graduation, most residents are not permitted to return to the clinic to finish ABO cases because of university policies and malpractice insurance regulations. The burden lies with the university to provide boardeligible cases at the onset of the residency, so that they can be treated within the time frame of the program. Once the patient pool is in place and the faculty members serve as advisors and mentors, the process becomes more streamlined. According to our study, distribution methods vary between programs, with random distribution the most common method. Discrepancy Index scores are typically calculated by residents and may be the method by which board cases are identified. However, all potential cases should be reviewed and rescored by attending faculty. Only 18% of 136 orthodontic residents polled in 2007 believed that certification should be mandatory for licensure.1 Unlike other specialties, the ABO certification process is still entirely voluntary and unrestricting if not achieved. Currently, there are 3419 ABO certified orthodontists; approximately 41% of active American Association of Orthodontists members are board certified.21 Other specialties require certification to maintain hospital privileges, but with the majority of orthodontic residents planning to go into private practice,
American Journal of Orthodontics and Dentofacial Orthopedics
Park et al
there is no requirement for certification. Even though this is the case, 95% of graduating orthodontists take the ABO written examination, and 81% to 87% plan to become certified.1,10,16 These statistics reinforce the fact that there is a desire among residents to become ABO certified and that more should be done by current orthodontic specialty programs to prepare their residents for the certification process. At the 2016 ABO Educators Symposium, the results of the September 2015 and February 2016 clinical examinations were discussed. Of 1367 cases submitted for grading, 94% were complete (passing). The most common reason for being scored as incomplete (nonpassing) was treatment related (68%), with the Cast-Radiograph Evaluation, Case Management Form, or both being unacceptable. The average grade on the Cast-Radiograph Evaluation was 22, with the most points taken off for alignment (4.2 points), buccolingual inclination (3.8 points), and marginal ridges (3.5 points). On the Case Management Form, tracings (33%) and superimpositions (38%) were the most common components scored as unacceptable.20 Since tracings and superimpositions seem to be difficult components of the Case Management Form, increased concentration on these areas during residency may be needed. Incorporation of superimpositions into routine case presentations, as well as didactic lessens on hand tracing and digital tracing should improve residents’ understanding of these components. The banking component of the ICE process is in its infancy, so although it is promising, there are no longterm data to support or refute the benefit of the banking system. Since its inception in 2010, total participation and banking have increased steadily, with an average of 4 cases banked per examinee.20 Unfortunately, the lack of communication once a resident has graduated makes it difficult to track his or her board certification progress. Participation per graduation year has increased, but the status should be monitored in more detail. More information is needed on the success of the banking program, and there should be new graduate follow up. Interestingly, 1744 ABO members will be up for recertification after 2020.20 We must consider the most common reasons that board-certified orthodontists would choose not to take the First Certification Renewal Examination after 2020 and ensure maximum participation. New graduates are facing more hardships as they enter the work force, so they may delay board certification as they start their practices and families. Increased student loan debt and fewer employment opportunities translate to a longer transitional period as newer graduates are slower to purchase practices and start-ups.21
469
The high doctor turnover in corporate environments and low patient numbers in start-ups make it difficult to collect ideal board cases within the allotted time frame. The newer guidelines and banking process may make it easier to select cases in residency, decreasing the burden of finding appropriate patients once in private or corporate practice. Another barrier to certification is the cost of the examination. Although some programs (36.4%) include the cost of the written examination in their programs, only 9% are willing to reimburse the clinical examination fee. Although cost was the least common reason for not obtaining certification during residency, the question was asked from the program director's perspective. In reality, the cost of the examination plus travel and lodging expenses may be a deterrent for some new graduates. To avoid undue time constraints on program directors and chairs, the survey was designed to be as short as possible, but this limited the depth of detail that could be obtained from the results. Many additional questions could have been created to collect information about each program and program director, such as training or treatment philosophy. Each of these variables could be a confounding factor, influencing the practices of the program. Another potential limitation of the study was selection bias, introduced because of the voluntary nature of the survey. Some respondents may have chosen not to participate simply because the importance of board certification is not stressed in their programs. This selection bias may work in the opposite way, too. Program directors who promote board certification may be eager to share their views of the process and therefore be more likely to respond. The results must be carefully considered so as not to mislead the reader into thinking that the collected responses truly represent the entire population sampled. Lastly, this survey was distributed to program directors. The opinions of residents as to why they have decided whether to pursue board certification are still needed. Although graduate surveys have been distributed, they typically ask about all aspects of a program and do not delve into board certification beyond a few questions. Future research should be focused on a pathway to maximize the number of board-eligible cases finished in residency. In addition, hardships facing new graduates after the economic downturn and increasing numbers of corporate practices need to be considered. Many new graduates are interested in board certification, but “lack of long term job stability,” “a corporate environment not ideal for ABO preparation,” and “starting a practice/lack of patient pool” are reported barriers to
American Journal of Orthodontics and Dentofacial Orthopedics
March 2017 Vol 151 Issue 3
Park et al
470
success. We need to be aware of these changing trends and give our younger colleagues the tools to effectively achieve board status. CONCLUSIONS
1. 2.
3.
4.
6.
7.
Most program directors believe that their program length is sufficient for board preparation. Subjects tested by the written examination are integrated into the didactic curriculum and strengthened with ongoing literature reviews, having a passing rate over 90%. Clinical examination preparation varies from program to program, with most programs requiring a mock board examination for graduation. Total participation in ICE and banking has increased since 2010; better follow-up protocols are needed to track residents after graduation.
8. 9. 10.
11.
12. 13.
ACKNOWLEDGMENTS
We thank Cale Forgues for help with the literature review, Michael L. Riolo for manuscript review, and Curtis Bay for statistical analyses.
15. 16.
REFERENCES 1. Noble J, Hechter FJ, Karaiskos NE, Lekic N, Wiltshire WA. Future practice plans of orthodontic residents in the United States. Am J Orthod Dentofacial Orthop 2009;135:357-60. 2. Owens SE Jr, Dykhouse VJ, Moffitt AH, Grubb JE, Greco PM, English JD, et al. The new American Board of Orthodontics certification process: further clarification. Am J Orthod Dentofacial Orthop 2005;128:541-4. 3. Riolo ML, Owens SE, Dykhouse VJ, Moffitt AH, Grubb JE, Greco PM, et al. A change in the certification process by the American Board of Orthodontics. Am J Orthod Dentofacial Orthop 2005;127:278-81. 4. Dykhouse VJ, Moffitt AH, Grubb JE, Greco PM, English JD, Briss BS, et al. A report of the ABO resident clinical outcome study (the pilot study). Am J Orthod Dentofacial Orthop 2006;130:656-61. 5. Greco PM, English JD, Briss BS, Jamieson SA, Kastrop MC, Castelein PT, et al. Banking cases for the American Board of
March 2017 Vol 151 Issue 3
14.
17. 18.
19.
20.
21.
Orthodontics’ initial certification examination. Am J Orthod Dentofacial Orthop 2010;137:575-6. American Board of Orthodontics. Banking process. Available at: https://www.americanboardortho.com/media/4821/exam-excep tion-request-form.pdf. Accessed June 7, 2016. Commission on Dental Accreditation. Accreditation Standards for Advanced Specialty Education Programs in Orthodontics and Dentofacial Orthopedics. Available at: http://www.ada.org//media/ CODA/Files/ortho.ashx. Accessed June 7, 2016. Park JH. Retirement planning and practice operations in the aftermath of the recession. PCSO Bull 2015;87:6-7. Neumann LM, Nix JA. Trends in dental specialty education and practice, 1990-99. J Dent Educ 2002;66:1338-47. Bruner MK, Hilger KK, Silveir AM, Butters JM. Graduate orthodontic education: the resident’s perspective. Am J Orthod Dentofacial Orthop 2005;128:277-82. American Board of Orthodontics. Examination process. Available at: https://www.americanboardortho.com/orthodontic-professionals/ about-board-certification/clinical-examination/examination-require ments/initial-certification-examination/. Accessed April 7, 2016. Larson B. Faculty recruitment and retention: challenge or crisis. Am J Orthod Dentofacial Orthop 1998;113:122-3. Moskowitz EM. A mandatory 3-year orthodontic postgraduate program and the American Board of Orthodontics: what’s the connection? Am J Orthod Dentofacial Orthop 2003;124:2-3. Lindauer SJ. Is the time really right for mandatory 3-year orthodontic residency programs? Am J Orthod Dentofacial Orthop 2008;133:2-3. Cangialosi TJ. The time is right for a 3-year orthodontic curriculum. Am J Orthod Dentofacial Orthop 2007;132:135-6. Burk T, Orellana M. Assessment of graduate orthodontic programs in North America. J Dent Educ 2013;77:463-75. Keim RG, Sinclair PM. Orthodontic graduate education survey, 1983-2000. Am J Orthod Dentofacial Orthop 2002;121:2-8. Castelein PT, DeLeon E Jr, Dugoni SA, Chung CH, Tadlock LP, Barone ND, et al. Certification renewal process of the American Board of Orthodontics. Am J Orthod Dentofacial Orthop 2015; 147(Supp 5):S232-3. Cangialosi TJ, Riolo ML, Owens SE, Dykhouse VJ, Moffitt AH, Grubb JE, et al. The American Board of Orthodontics and specialty certification: the first 50 years. Am J Orthod Dentofacial Orthop 2004;126:3-6. ABO directors. Clinical challenges. Presented at the Charles Burstone-American Board of Orthodontics 2016 Educators Symposium, April 2, 2016; St Louis, Mo. Pruzansky D, Ellis B, Park JH. Influence of student-loan debt on orthodontic residents and recent graduates. J Clin Orthod 2016; 50:23-32.
American Journal of Orthodontics and Dentofacial Orthopedics