- Email: [email protected]
Development of a national course on research methodology for Canadian residents in obstetrics and gynecology 1
Graduate Education Development of a national course on research methodology for Canadian residents in obstetrics and gynecology Robert L. Reid, MD, Dean A. Van Vugt, PhD, and Philip M. Hahn, MSc Objective: To report our experience developing and implementing an introductory course on research methods for Canadian obstetrics and gynecology residents. Methods: A program entitled “An Introduction to Research,” originating at Queen’s University, developed into an annual series of regional courses across Canada, under the auspices of the Association of Professors of Obstetrics and Gynaecology of Canada. Didactic lectures, interactive workshops, and online computer demonstrations introduced participants to the basic elements of clinical research. Results: Since its inception, over 1000 participants have attended the program. Nearly all of the 296 respondents to a course evaluation agreed that the program was well organized, presented material at an appropriate level, and was useful. Conclusion: This course ensured that residents in obstetrics and gynecology across Canada were given a basic level of research training, as required by the Royal College of Physicians and Surgeons of Canada. (Obstet Gynecol 1999; 93:308 –11. © 1999 by The American College of Obstetricians and Gynecologists.)
and gynecology programs found that of 142 responding programs that required residency research, only 48% had formal instruction in conducting research.1 In Canada, a survey returned by all 16 residency program directors in obstetrics and gynecology revealed that 12 programs had research guidelines for residents, but only six had required courses in research or critical appraisal (Daya S, et al. Canadian survey of opportunities for learning research/critical appraisal skills during residency training: Perspective of residents versus program directors. Abstract: Association of Professors of Obstetrics and Gynaecology Annual Meeting, Toronto, Ontario, Dec 3– 4, 1994). We concluded that opportunities to learn how to conduct research were limited, ill preparing residents in this area. This finding was particularly surprising, because excellence in critical appraisal depends largely on knowledge of research methods and pitfalls. By acquiring research skills and understanding the research process, one should be able to better evaluate the medical literature, distinguish strong evidence from weak, and be more judicious in therapy recommendations to individual patients.2 In this article we report our experience developing and implementing an introductory course on research methods for Canadian obstetrics and gynecology residents.
Methods
Critical appraisal skills are essential in any residency training program, yet emphasis on research training is highly variable in postgraduate programs in North America. A national survey of United States obstetrics
In our department, resident training in clinical research methods began as a weekly 1-hour lecture series given throughout the academic year. Because many residents were bound to clinical activities in the hospital setting, attendance was sporadic. Research was encouraged, but at that time it was not required for completion of the program. Residents anticipating nonacademic clinical careers might have lacked incentive to attend. A decision was made in the winter of 1991 to formalize teaching of research methods and to offer a course to residents from the province of Ontario. Residents were given protected time to attend, and attendance was insured by combining sessions over 2 days at a site away from the hospital or university setting. We adhered to five principles that we believe set this program apart from other university research methods courses.
From the Division of Reproductive Endocrinology, Department of Obstetrics and Gynaecology, Queen’s University, Kingston, Ontario, Canada. This course was sponsored by educational grants from Ortho Pharmaceutical, Ortho-McNeil and then Janssen-Ortho Inc. from its inception May 2, 1991 through December 1, 1997. The authors acknowledge Michael Sand of Ortho Pharmaceutical (presently with Digital Health Inc., Markham, Ontario, Canada) for providing essential support for this program in its developmental phases.
1. Sequestered setting. The course was located away from distractions and responsibilities of clinical service, providing the best opportunity to learn. 2. Multidisciplinary faculty. Three faculty members with different areas of expertise, a clinical investigator (RLR), a basic scientist (DVV), and a research associate with skills in research design, biostatistics, and computing (PMH), designed a course with a
308 0029-7844/99/$20.00 PII S0029-7844(98)00367-6
Obstetrics & Gynecology
Table 1. Course Presentations 1. Research in my residency. Why? When? How? (RLR)* • Where do research ideas originate? • Finding the right mentor and the time. 2. Study design. Key concepts. (PMH)* • Pros and cons of case-control, cohort, randomized control trial and crossover designs. 3. Literature search today. (DVV)*† • Strengths and limitations of textbooks, journals, and computer databases. • Online demonstration of MEDLINE search strategy. 4. Sample size determination. Give me more power. (PMH)* • Development of a simple sample size estimate for continuous and nominal data. • Introduction to meta-analysis. 5. Workshop: Study design. (RLR)‡ • Interactive exercises exploring design choices. • Reinforcement of concepts delivered in sessions 2 and 4. 6. Research models: Humans, animals, cells, and computers. (DVV)* • Choosing the most appropriate experimental subject. • Animal rights versus animal welfare groups. 7. Basic statistics for critical appraisal. (PMH)*‡ • Descriptive statistics: standard deviation, standard error, confidence intervals. 2 • T test, analysis of variance, x analysis. • Sensitivity, specificity, positive and negative predictive values in relation to disease prevalence. 8. More basic statistics. Measures of clinical benefit. (PMH)*‡ • Relative versus absolute risk reduction, number needed to treat, and the odds ratio. • Misuse of the relative risk measure. 9. The research proposal. I have a great idea. Now what? (RLR)* • How to draft a proposal from hypothesis statement to budget. • Research ethics boards. 10. Critical appraisal of the literature. (DVV)* • Getting the most out of your journal club. • Strategies for evaluating scientific papers. 11. Obstetrics and gynecology in cyberspace. (PMH)*† • Brief overview of the internet and its effect on the doctor-patient relationship. • Minimum hardware requirements. • Online demonstration of obstetrics and gynecology-related sites; SOGC and ACOG homepages. 12. Evidence-Based Medicine. Where to find it. (PMH)*† • The Cochrane Collaboration. • Online demonstration of The Cochrane Library and the Centre for Evidence-Based Medicine. • Evidence-based medicine journal. 13. The publication process. The dos and the don’ts. (RLR)* • Helpful steps for submitting a paper to a medical journal. 14. Effective presentations. How not to give a paper. (RLR)* • Introducing quality into presentation content and style. • Dealing with the media. 15. Workshop: Presentations. Projecting your image. (DVV)‡ • Audio-visual presentation methods. • The problem projector. • Computer demonstration of presentation software. 16. Workshop: Critical Appraisal Exercises. (PMH)*‡ • Examples from the medical literature highlighting flaws in design and analysis. SOGC 5 Society of Obstetricians and Gynaecologists of Canada. * Didactic slide presentation. † Online computer demonstration. ‡ Interactive session.
broad spectrum of topics. The importance of collaboration in research was stressed throughout the program, and the faculty served as role models. 3. Adherence to basic concepts. The instructors assumed that the residents had no previous knowledge or research experience. 4. Examples from obstetrics and gynecology. Most ex-
VOL. 93, NO. 2, FEBRUARY 1999
amples were common, easily understood clinical situations from our specialty, so participants could focus on the principles being taught. 5. Production of syllabus. A collection of all the slides presented was contained in a syllabus, which allowed participants to concentrate on concepts without having to take notes.
Reid et al
Research Methodology Course
309
A list of the course’s most recent presentations is provided in Table 1. Eight of the 16 presentations were strictly didactic. Slides were presented for 40 minutes followed by a 5-minute question period. The program incorporated three 1-hour workshops after the didactic sessions, which introduced participants to particular topics, such as study design. The most recent program also included three online computer demonstrations, which involved an internet connection, notebook computer, and data projector. A MEDLINE search was demonstrated, as well as searching the world wide web for evidence-based medicine sites, such as homepages of ACOG, the Society of Obstetricians and Gynaecologists of Canada, and the Cochrane Collaboration. After positive evaluations from the participants in the 1991 pilot program, we approached Association of Professors of Obstetrics and Gynaecology for a national endorsement. Consensus was reached to conduct the program during the next 3 years (1992–1994) in three regions of Canada: the Maritimes, Ontario, and western Canada. Faculty from each region were invited to take the course to renew their knowledge, increase their teaching skills in this area, or monitor the course. After the first year, regional faculty were invited to deliver presentations designed by the original speakers. We anticipated that individual obstetrics and gynecology departments would eventually conduct identical or similar programs at their own centers. William Fraser of Laval University translated the course material into French, for residents of Quebe´c, after attending a program in September 1992 given to residents and faculty at Queen’s University. After 1994, the program was extended for 3 more years. In 1995, the program was held in all four regions across Canada including Quebe´c. In 1996 and 1997, because of major funding cutbacks, the course was held only once, in Toronto, Ontario, in association with the annual Association of Professors of Obstetrics and Gynaecology meeting. A standard evaluation form was collected from participants at the end of each course. General ratings were sought on the organizational aspect of the program, the level of material presented, and overall usefulness. Each presentation was rated 1 to 5 on the ordinal scale (1 5 poor, 3 5 adequate, 5 5 outstanding). A comparison of presentation scores by original and regional faculty during the same academic year was analyzed with the Kruskal-Wallis test using SAS version 6.08 (SAS Institute, Cary, NC). This comparison was conducted to determine whether material composed by the authors could be presented effectively by other university faculty. Ratings of the French program in Quebe´c are not contained in this report. A pharmaceutical company sponsored travel and accommodation for two regional faculty presenters and
310 Reid et al
Research Methodology Course
the three original faculty. Meeting rooms, health breaks, lunches, and course materials, including slides, syllabae and two text books3,4 were provided. Funding for resident travel, accommodation, and maintenance was the responsibility of each institution.
Results Since its inception, 450 residents and 87 faculty members have attended this program. Additionally, 14 basic science graduate students, eight research assistants, two fellows, and 24 pharmaceutical representatives participated. In addition to the Association of Professors of Obstetrics and Gynaecology program, 425 participants have attended similar programs at Queen’s University between 1992 and 1998, for a total of 1014 participants. For the English-speaking program between 1991 and 1997, we collected 296 evaluations from 409 participants, a return rate of 72.4%. Not all respondents answered all questions, so the denominators are less than 296. All 272 agreed that the course was well organized and well presented. Ninety-four percent (256 of 272) of the respondents claimed that the level of material presented was appropriate for the their level of experience. Fifteen participants (5.5%) thought that the level of material was too basic: six were residents, two having already authored original publications; five were basic science graduate students; three were faculty; and one did not list a position. One resident with graduate-level training claimed that the level of material was too advanced. Only two participants stated that the material was not useful. One resident wanted more material on the effective use of biostatistics and the other participant did not give a reason. The average overall course rating ranged from 4.2 of 5 (n 5 21 respondents) for the 1995 western regional program to 4.6 (n 5 27) for the 1996 central regional program. Since 1993, 13 regional English-speaking faculty have given 28 presentations designed by the original faculty. Ratings for regional speakers and the original faculty were similar. In 1993, the presentation on the publication process was given three times, once by the originator and then by east and west coast regional faculty. The average score for the originator was 4.1 (n 5 11) compared with 3.9 (n 5 8, n 5 23) for both regional presenters (P 5 .714, Kruskal-Wallis test). In 1994, a comparison of the presentations on basic statistics showed equally high grades (originator score 4.2 [n 5 17] compared with regional scores 4.1 [n 5 16] and 3.9 [n 5 10] [P 5 .318]).
Discussion Since its inception in May 1991, this introductory course on clinical research methods has been well received by
Obstetrics & Gynecology
residents and faculty in obstetrics and gynecology throughout Canada. For some it provides their first exposure to research ideas, and for others it serves as a solid review. The collegial atmosphere allows residents to share experiences from other institutions. They leave knowing that choosing a mentor with the proper resources and background training is the critical first step to success. The program reinforces the importance of an association initially with biostatisticians or epidemiologists to ensure valid study design, including sample size estimation. By establishing a foundation in the basics of research methods, this course serves as a stepping stone for those pursuing academic medicine. Exposure to research training early in a residency program is likely to increase one’s opportunity to become an independent researcher upon completing training.5 Faculty members who attend leave with material that can be applied to their own training programs. Course material was effectively presented by regional faculty with appropriate training; however, presenting prepackaged course material has inherent difficulties and drawbacks. Subtle details are sometimes lost in translation, and occasionally misinformation is conveyed. One objective of this national initiative was for each university department to begin offering an identical or similar introductory level course on research methods to their residents in the 1994–1995 academic year. This plan relied on a central source for developing, updating, and disseminating material, as well as for continued funding. Funding cutbacks eliminated regional programs in 1996 and 1997, and decentralization to individual departments has not yet been done. Several centers did not send residents to the centralized program because of travel costs. Future programs might involve alternative delivery systems, such as teleconferencing. A recent review showed negligible gains in behavior toward critical use of medical literature by residents who were taught critical appraisal and evidence-based medicine.6 Lack of resident evaluation, thus incentive, is a possible explanation for ineffective teaching of critical appraisal skills. Accreditation standards of The Royal College of Physicians and Surgeons of Canada (September 1997) stated that residency programs must provide opportunity for residents to learn biostatistics and critical appraisal of research methodology and medical literature. This directive, if accompanied by proper evaluation, should provide incentive to learn and develop proper research methods, which we hope will translate to better critical appraisal skills and an evidencebased approach to learning and practicing medicine. The effectiveness of the present course has not been measured. A randomized, double-blind study might not be feasible, since it would necessitate that half the residents would not receive instruction in this impor-
VOL. 93, NO. 2, FEBRUARY 1999
tant area, an unacceptable prospect to most participants and educators. A single-group, before-after design could measure immediate gains in knowledge; however, long-term assessment of retained knowledge and research productivity, in relation to a teaching intervention, would likely be confounded by co-intervention. As pointed out by David Sackett and Julie Parkes, “because medical education occurs in a maelstrom of information, advice, and admonishment, every critical appraisal course is contaminated, and co-intervention is the rule. As a consequence, it is difficult or impossible to isolate the contribution of instruction in critical appraisal to the way that its graduates practice medicine.”7 This course ensures that residents in obstetrics and gynaecology throughout Canada are exposed to a basic level of research training. Continuation of this program in its present form requires a continuing commitment from faculty, sufficient financial support, departmental commitment to the costs and inconvenience of having residents away, and ongoing endorsement by Association of Professors of Obstetrics and Gynaecology.
References 1. Sulak PJ, Croop JA, Hillis A, Kuehl TJ. Resident research in obstetrics and gynecology: Development of a program with comparison to a national survey of residency programs. Am J Obstet Gynecol 1992;167:498 –502. 2. Evidence-Based Medicine Working Group. Evidence-based medicine: A new approach to teaching the practice of medicine. JAMA 1992;268:2420 –5. 3. Norman GR, Streiner DL. PDQ statistics. Toronto, Ontario, Canada: B.C. Decker Inc, 1986. 4. Streiner DL, Norman GR, Munroe Blum H. PDQ epidemiology. Toronto, Ontario, Canada: B.C. Decker Inc, 1989. 5. Hillman BJ, Witzte DB, Jafardo LLL, Fulginiti JV. Research and training in academic radiology departments: A survey of department chairmen. Invest Radiol 1990;25:587–91. 6. Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: A critical appraisal. Can Med Assoc J 1998;158:177– 81. 7. Sackett DL, Parkes J. Teaching critical appraisal: No quick fixes. Can Med Assoc J 1998;158:203– 4.
Address reprint requests to:
Robert L. Reid, MD Department of Obstetrics and Gynecology Queen’s University Kingston, Ontario K7L 3N6 Canada E-mail: [email protected] Received April 1, 1998. Received in revised form July 13, 1998. Accepted July 30, 1998. Copyright © 1999 by The American College of Obstetricians and Gynecologists. Published by Elsevier Science Inc.
Reid et al
Research Methodology Course
311
and gynecology programs found that of 142 responding programs that required residency research, only 48% had formal instruction in conducting research.1 In Canada, a survey returned by all 16 residency program directors in obstetrics and gynecology revealed that 12 programs had research guidelines for residents, but only six had required courses in research or critical appraisal (Daya S, et al. Canadian survey of opportunities for learning research/critical appraisal skills during residency training: Perspective of residents versus program directors. Abstract: Association of Professors of Obstetrics and Gynaecology Annual Meeting, Toronto, Ontario, Dec 3– 4, 1994). We concluded that opportunities to learn how to conduct research were limited, ill preparing residents in this area. This finding was particularly surprising, because excellence in critical appraisal depends largely on knowledge of research methods and pitfalls. By acquiring research skills and understanding the research process, one should be able to better evaluate the medical literature, distinguish strong evidence from weak, and be more judicious in therapy recommendations to individual patients.2 In this article we report our experience developing and implementing an introductory course on research methods for Canadian obstetrics and gynecology residents.
Methods
Critical appraisal skills are essential in any residency training program, yet emphasis on research training is highly variable in postgraduate programs in North America. A national survey of United States obstetrics
In our department, resident training in clinical research methods began as a weekly 1-hour lecture series given throughout the academic year. Because many residents were bound to clinical activities in the hospital setting, attendance was sporadic. Research was encouraged, but at that time it was not required for completion of the program. Residents anticipating nonacademic clinical careers might have lacked incentive to attend. A decision was made in the winter of 1991 to formalize teaching of research methods and to offer a course to residents from the province of Ontario. Residents were given protected time to attend, and attendance was insured by combining sessions over 2 days at a site away from the hospital or university setting. We adhered to five principles that we believe set this program apart from other university research methods courses.
From the Division of Reproductive Endocrinology, Department of Obstetrics and Gynaecology, Queen’s University, Kingston, Ontario, Canada. This course was sponsored by educational grants from Ortho Pharmaceutical, Ortho-McNeil and then Janssen-Ortho Inc. from its inception May 2, 1991 through December 1, 1997. The authors acknowledge Michael Sand of Ortho Pharmaceutical (presently with Digital Health Inc., Markham, Ontario, Canada) for providing essential support for this program in its developmental phases.
1. Sequestered setting. The course was located away from distractions and responsibilities of clinical service, providing the best opportunity to learn. 2. Multidisciplinary faculty. Three faculty members with different areas of expertise, a clinical investigator (RLR), a basic scientist (DVV), and a research associate with skills in research design, biostatistics, and computing (PMH), designed a course with a
308 0029-7844/99/$20.00 PII S0029-7844(98)00367-6
Obstetrics & Gynecology
Table 1. Course Presentations 1. Research in my residency. Why? When? How? (RLR)* • Where do research ideas originate? • Finding the right mentor and the time. 2. Study design. Key concepts. (PMH)* • Pros and cons of case-control, cohort, randomized control trial and crossover designs. 3. Literature search today. (DVV)*† • Strengths and limitations of textbooks, journals, and computer databases. • Online demonstration of MEDLINE search strategy. 4. Sample size determination. Give me more power. (PMH)* • Development of a simple sample size estimate for continuous and nominal data. • Introduction to meta-analysis. 5. Workshop: Study design. (RLR)‡ • Interactive exercises exploring design choices. • Reinforcement of concepts delivered in sessions 2 and 4. 6. Research models: Humans, animals, cells, and computers. (DVV)* • Choosing the most appropriate experimental subject. • Animal rights versus animal welfare groups. 7. Basic statistics for critical appraisal. (PMH)*‡ • Descriptive statistics: standard deviation, standard error, confidence intervals. 2 • T test, analysis of variance, x analysis. • Sensitivity, specificity, positive and negative predictive values in relation to disease prevalence. 8. More basic statistics. Measures of clinical benefit. (PMH)*‡ • Relative versus absolute risk reduction, number needed to treat, and the odds ratio. • Misuse of the relative risk measure. 9. The research proposal. I have a great idea. Now what? (RLR)* • How to draft a proposal from hypothesis statement to budget. • Research ethics boards. 10. Critical appraisal of the literature. (DVV)* • Getting the most out of your journal club. • Strategies for evaluating scientific papers. 11. Obstetrics and gynecology in cyberspace. (PMH)*† • Brief overview of the internet and its effect on the doctor-patient relationship. • Minimum hardware requirements. • Online demonstration of obstetrics and gynecology-related sites; SOGC and ACOG homepages. 12. Evidence-Based Medicine. Where to find it. (PMH)*† • The Cochrane Collaboration. • Online demonstration of The Cochrane Library and the Centre for Evidence-Based Medicine. • Evidence-based medicine journal. 13. The publication process. The dos and the don’ts. (RLR)* • Helpful steps for submitting a paper to a medical journal. 14. Effective presentations. How not to give a paper. (RLR)* • Introducing quality into presentation content and style. • Dealing with the media. 15. Workshop: Presentations. Projecting your image. (DVV)‡ • Audio-visual presentation methods. • The problem projector. • Computer demonstration of presentation software. 16. Workshop: Critical Appraisal Exercises. (PMH)*‡ • Examples from the medical literature highlighting flaws in design and analysis. SOGC 5 Society of Obstetricians and Gynaecologists of Canada. * Didactic slide presentation. † Online computer demonstration. ‡ Interactive session.
broad spectrum of topics. The importance of collaboration in research was stressed throughout the program, and the faculty served as role models. 3. Adherence to basic concepts. The instructors assumed that the residents had no previous knowledge or research experience. 4. Examples from obstetrics and gynecology. Most ex-
VOL. 93, NO. 2, FEBRUARY 1999
amples were common, easily understood clinical situations from our specialty, so participants could focus on the principles being taught. 5. Production of syllabus. A collection of all the slides presented was contained in a syllabus, which allowed participants to concentrate on concepts without having to take notes.
Reid et al
Research Methodology Course
309
A list of the course’s most recent presentations is provided in Table 1. Eight of the 16 presentations were strictly didactic. Slides were presented for 40 minutes followed by a 5-minute question period. The program incorporated three 1-hour workshops after the didactic sessions, which introduced participants to particular topics, such as study design. The most recent program also included three online computer demonstrations, which involved an internet connection, notebook computer, and data projector. A MEDLINE search was demonstrated, as well as searching the world wide web for evidence-based medicine sites, such as homepages of ACOG, the Society of Obstetricians and Gynaecologists of Canada, and the Cochrane Collaboration. After positive evaluations from the participants in the 1991 pilot program, we approached Association of Professors of Obstetrics and Gynaecology for a national endorsement. Consensus was reached to conduct the program during the next 3 years (1992–1994) in three regions of Canada: the Maritimes, Ontario, and western Canada. Faculty from each region were invited to take the course to renew their knowledge, increase their teaching skills in this area, or monitor the course. After the first year, regional faculty were invited to deliver presentations designed by the original speakers. We anticipated that individual obstetrics and gynecology departments would eventually conduct identical or similar programs at their own centers. William Fraser of Laval University translated the course material into French, for residents of Quebe´c, after attending a program in September 1992 given to residents and faculty at Queen’s University. After 1994, the program was extended for 3 more years. In 1995, the program was held in all four regions across Canada including Quebe´c. In 1996 and 1997, because of major funding cutbacks, the course was held only once, in Toronto, Ontario, in association with the annual Association of Professors of Obstetrics and Gynaecology meeting. A standard evaluation form was collected from participants at the end of each course. General ratings were sought on the organizational aspect of the program, the level of material presented, and overall usefulness. Each presentation was rated 1 to 5 on the ordinal scale (1 5 poor, 3 5 adequate, 5 5 outstanding). A comparison of presentation scores by original and regional faculty during the same academic year was analyzed with the Kruskal-Wallis test using SAS version 6.08 (SAS Institute, Cary, NC). This comparison was conducted to determine whether material composed by the authors could be presented effectively by other university faculty. Ratings of the French program in Quebe´c are not contained in this report. A pharmaceutical company sponsored travel and accommodation for two regional faculty presenters and
310 Reid et al
Research Methodology Course
the three original faculty. Meeting rooms, health breaks, lunches, and course materials, including slides, syllabae and two text books3,4 were provided. Funding for resident travel, accommodation, and maintenance was the responsibility of each institution.
Results Since its inception, 450 residents and 87 faculty members have attended this program. Additionally, 14 basic science graduate students, eight research assistants, two fellows, and 24 pharmaceutical representatives participated. In addition to the Association of Professors of Obstetrics and Gynaecology program, 425 participants have attended similar programs at Queen’s University between 1992 and 1998, for a total of 1014 participants. For the English-speaking program between 1991 and 1997, we collected 296 evaluations from 409 participants, a return rate of 72.4%. Not all respondents answered all questions, so the denominators are less than 296. All 272 agreed that the course was well organized and well presented. Ninety-four percent (256 of 272) of the respondents claimed that the level of material presented was appropriate for the their level of experience. Fifteen participants (5.5%) thought that the level of material was too basic: six were residents, two having already authored original publications; five were basic science graduate students; three were faculty; and one did not list a position. One resident with graduate-level training claimed that the level of material was too advanced. Only two participants stated that the material was not useful. One resident wanted more material on the effective use of biostatistics and the other participant did not give a reason. The average overall course rating ranged from 4.2 of 5 (n 5 21 respondents) for the 1995 western regional program to 4.6 (n 5 27) for the 1996 central regional program. Since 1993, 13 regional English-speaking faculty have given 28 presentations designed by the original faculty. Ratings for regional speakers and the original faculty were similar. In 1993, the presentation on the publication process was given three times, once by the originator and then by east and west coast regional faculty. The average score for the originator was 4.1 (n 5 11) compared with 3.9 (n 5 8, n 5 23) for both regional presenters (P 5 .714, Kruskal-Wallis test). In 1994, a comparison of the presentations on basic statistics showed equally high grades (originator score 4.2 [n 5 17] compared with regional scores 4.1 [n 5 16] and 3.9 [n 5 10] [P 5 .318]).
Discussion Since its inception in May 1991, this introductory course on clinical research methods has been well received by
Obstetrics & Gynecology
residents and faculty in obstetrics and gynecology throughout Canada. For some it provides their first exposure to research ideas, and for others it serves as a solid review. The collegial atmosphere allows residents to share experiences from other institutions. They leave knowing that choosing a mentor with the proper resources and background training is the critical first step to success. The program reinforces the importance of an association initially with biostatisticians or epidemiologists to ensure valid study design, including sample size estimation. By establishing a foundation in the basics of research methods, this course serves as a stepping stone for those pursuing academic medicine. Exposure to research training early in a residency program is likely to increase one’s opportunity to become an independent researcher upon completing training.5 Faculty members who attend leave with material that can be applied to their own training programs. Course material was effectively presented by regional faculty with appropriate training; however, presenting prepackaged course material has inherent difficulties and drawbacks. Subtle details are sometimes lost in translation, and occasionally misinformation is conveyed. One objective of this national initiative was for each university department to begin offering an identical or similar introductory level course on research methods to their residents in the 1994–1995 academic year. This plan relied on a central source for developing, updating, and disseminating material, as well as for continued funding. Funding cutbacks eliminated regional programs in 1996 and 1997, and decentralization to individual departments has not yet been done. Several centers did not send residents to the centralized program because of travel costs. Future programs might involve alternative delivery systems, such as teleconferencing. A recent review showed negligible gains in behavior toward critical use of medical literature by residents who were taught critical appraisal and evidence-based medicine.6 Lack of resident evaluation, thus incentive, is a possible explanation for ineffective teaching of critical appraisal skills. Accreditation standards of The Royal College of Physicians and Surgeons of Canada (September 1997) stated that residency programs must provide opportunity for residents to learn biostatistics and critical appraisal of research methodology and medical literature. This directive, if accompanied by proper evaluation, should provide incentive to learn and develop proper research methods, which we hope will translate to better critical appraisal skills and an evidencebased approach to learning and practicing medicine. The effectiveness of the present course has not been measured. A randomized, double-blind study might not be feasible, since it would necessitate that half the residents would not receive instruction in this impor-
VOL. 93, NO. 2, FEBRUARY 1999
tant area, an unacceptable prospect to most participants and educators. A single-group, before-after design could measure immediate gains in knowledge; however, long-term assessment of retained knowledge and research productivity, in relation to a teaching intervention, would likely be confounded by co-intervention. As pointed out by David Sackett and Julie Parkes, “because medical education occurs in a maelstrom of information, advice, and admonishment, every critical appraisal course is contaminated, and co-intervention is the rule. As a consequence, it is difficult or impossible to isolate the contribution of instruction in critical appraisal to the way that its graduates practice medicine.”7 This course ensures that residents in obstetrics and gynaecology throughout Canada are exposed to a basic level of research training. Continuation of this program in its present form requires a continuing commitment from faculty, sufficient financial support, departmental commitment to the costs and inconvenience of having residents away, and ongoing endorsement by Association of Professors of Obstetrics and Gynaecology.
References 1. Sulak PJ, Croop JA, Hillis A, Kuehl TJ. Resident research in obstetrics and gynecology: Development of a program with comparison to a national survey of residency programs. Am J Obstet Gynecol 1992;167:498 –502. 2. Evidence-Based Medicine Working Group. Evidence-based medicine: A new approach to teaching the practice of medicine. JAMA 1992;268:2420 –5. 3. Norman GR, Streiner DL. PDQ statistics. Toronto, Ontario, Canada: B.C. Decker Inc, 1986. 4. Streiner DL, Norman GR, Munroe Blum H. PDQ epidemiology. Toronto, Ontario, Canada: B.C. Decker Inc, 1989. 5. Hillman BJ, Witzte DB, Jafardo LLL, Fulginiti JV. Research and training in academic radiology departments: A survey of department chairmen. Invest Radiol 1990;25:587–91. 6. Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: A critical appraisal. Can Med Assoc J 1998;158:177– 81. 7. Sackett DL, Parkes J. Teaching critical appraisal: No quick fixes. Can Med Assoc J 1998;158:203– 4.
Address reprint requests to:
Robert L. Reid, MD Department of Obstetrics and Gynecology Queen’s University Kingston, Ontario K7L 3N6 Canada E-mail: [email protected] Received April 1, 1998. Received in revised form July 13, 1998. Accepted July 30, 1998. Copyright © 1999 by The American College of Obstetricians and Gynecologists. Published by Elsevier Science Inc.
Reid et al
Research Methodology Course
311