- Email: [email protected]
“SURGENT”—Student e-Learning for Reality: The Application of Interactive Visual Images to Problem-Based Learning in Undergraduate Surgery
EDUCATION
“SURGENT”—Student e-Learning for Reality: The Application of Interactive Visual Images to Problem-Based Learning in Undergraduate Surgery Mark Corrigan, Michelle Reardon, Connor Shields, and Henry Redmond Department of Academic Surgery, Cork University Hospital, Cork, Ireland Information technology has the potential to transform surgical education. Combining symbolic, iconic, and enactive teaching modalities to construct an authentic conceptual model potentially can transform a primarily didactic learning experience into an interactive Web-enhanced one. This study sought to assess the introduction of a Web-based module to complement traditional surgical undergraduate curricula. Adopting the clinical case as its fundamental educational approach, an online resource simulating surgical clinical decision making (“SURGENT,” http://www.surgent.ie) was developed, which consists of the interpretation of clinical photographs, laboratory data, and X-rays as well as the formulation of a management plan. Evaluation was in both a qualitative and a quantitative fashion. An anonymous postcourse survey (73% response) of 117 final medical students was used to by researchers to evaluate access, process, and outcome criteria. SURGENT was used by 98% of students, with 69% spending more than 30 minutes per session on the program. First-class honors in the final surgical clinical examination improved from 11% to 20% (p ⫽ 0.01) as compared with the previous control year. A Web-enhanced interactive surgical module in an undergraduate course can convey successfully information and understanding beyond the textbook. It is intended that SURGENT will supplement textbooks and ward experience, allowing students to develop their clinical decision-making skills. (J Surg 65:120-125. © 2008 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: SURGENT, information technology, e-learn-
ing, virtual learning, internet based learning COMPETENCY: Medical Knowledge, Practice Based Learning, Systems Based Practice
Correspondence: Inquiries to Mark Anthony Corrigan, Department of Surgery, Cork University Hospital, Cork, Ireland; fax: 00353 87 94 11 555; e-mail: [email protected]
120
INTRODUCTION Current trends in medical education suggest a move toward problem-based learning (PBL), with an emphasis on studentcentred education and use of information technologies. One of the main motivating factors for these changes was The General Medical Council’s publication entitled “Recommendations on Undergraduate Medical Education.” This publication states that learning through curiosity, the exploration of knowledge, and the critical evaluation of evidence should be promoted to ensure a capacity for self-education.1 The same document recommends that learning systems should be informed by modern educational theory and should draw on the wide range of technological resources available. The idea of “learning through curiosity” and “learning by experience” are concepts discussed by the educationalist Donald Schon. In his seminal work “The Reflective Practitioner,” he acknowledges that although “we do not have a full understanding of things before we act, but, hopefully, we can avoid major problems while ’testing the water.”2 Additionally when confronted by a situation unique to the practitioner he/she “reflects on the phenomenon before him, and on the prior understandings which have been implicit in his behaviour.” This sentiment is echoed by his contemporary M. K. Smith when describing a practitioners approach to a new situation: “We have to fall back on routines in which previous thought and sentiment has been sedimented. It is here that the full importance of reflection-on-action becomes revealed. . . . the space afforded by recording, supervision and conversation with our peers allows us to approach these. Reflection requires space in the present and the promise of space in the future.”3 These concepts of safely “testing the water” and of providing students with those “prior understandings” necessary for effective clinical thought are fundamental in the development of new teaching strategies. The primary goal of PBL, which is based on constructivist educational theory, is the integration and synthesis of knowledge, skills, and attitudes to effective critical thinking. The constructivist approach to teaching argues that understanding is
Journal of Surgical Education • © 2008 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/08/$30.00 doi:10.1016/j.jsurg.2007.11.011
not solely built up from received pieces of information, but it is a dynamic process whereby it is assembled actively and reshaped constantly by the individual according to new experiences.4,5 A combination of language, images, and actions facilitates this process.6 Clinical scenarios in a realistic environment using patient contact, end-of-bed notes, laboratory results, and radiological images should be encouraged. To this end, symbolic (language), iconic (images), and enactive (action) modalities are used to construct an authentic conceptual model that can then be applied to future clinical problems. However, the traditional application of PBL involves the application of only the symbolic and enactive modalities, failing to incorporate iconic imagery. Consequently, although PBL has been shown to be more nurturing and enjoyable compared with conventional instruction,7 it has been demonstrated that PBL students have scored lower in basic science examinations, which highlights an apparent gap in cognitive knowledge that has the potential to affect practice outcomes. The increased use of information technology (IT) supports a student-centred educational approach that promotes student autonomy and flexibility.8 IT has the potential to engender students with a desire to manage their own learning, which is a valuable step toward lifelong learning. The student dictates the time, duration, frequency, depth, and direction of interactive study. It can also offer opportunities for the student to enquire into their own competence, promoting self-awareness. Every student cannot observe every patient. Interactive information technology was employed in this project to broaden the students’ exposure to a greater variety of patients in a variety of clinical contexts. Applying the constructivist theory as its fundamental educational approach, an online resource simulating surgical clinical decision making (“SURGENT,” http:// www.surgent.ie) was developed by the surgical faculty. This resource was designed to reflect the daily problems and experi-
ences faced by surgical nonconsultant hospital doctors (NCHDs) and to allow students access to these experiences. This study was designed to determine whether information technology can be used to design and implement a system capable of delivering constructivist ethos, enhanced by iconic imagery, to surgical teaching. Additionally, it sought to evaluate the response of students to the integration of such a system to the surgical curriculum and its effect on student examination performance.
MATERIALS AND METHODS Database The relational database MySQL (Structured Query Language) was employed to store patient data. Interactions between the user and the database were regulated by a series of specially written scripts, encoded in the scripting language PHP (Hypertext Preprocessor) and embedded in HTML (Hypertext Markup Language). Interface The online interface was constructed using HTML with embedded PHP and written using the Macromedia HomeSite interface (version 5.5; Adobe Systems Incorporated, San Jose, California). A dropdown menu was employed to facilitate ease of navigation. To monitor usage of the system, access was protected through the use of an individual username and password, with automated free registration available 24 hours a day. Sections were created to represent the standard divisions used to examine and teach medical students. These sections comprised long-cases, short-cases, equipment, radiology, operative, and definitions (Fig. 1). Long cases were designed to enable the user to order relevant hematological, biochemical, microbiological,
FIGURE 1. The SURGENT frontpage is divided into educational sections: Long Cases, Short Cases, Equipment, Radiology, Operative and Definitions, as observed from this screen shot. Access to these sections is through a user-friendly dropdown menu. Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
121
Ethical Approval Ethical approval for the collection of patient data and its use in an online teaching program was sought and granted by the Clinical Research Ethics Committee of the Cork Teaching Hospitals. Assessment of Program
FIGURE 2. “Long Cases” enable the user to order any relevant tests from the hematology, biochemistry, microbiology, and radiological departments. Furthermore, the user may request the patients’ vital signs at any stage in the case. Both test results and vital signs will change in accordance with the actions of the user. The user may also order blood products from the blood bank.
and radiological tests (Fig. 2). According to the users’ actions, the results of these tests will change, providing clinical feedback to the student. Similarly, the user may request their patients’ vital signs. These will also change according to the users’ actions. Commonly examined conditions were incorporated in the “short cases” section, with frequently preformed operations and regularly used equipment included in the “operative” and “equipment” sections. All images used were kept below the size of 100 Kb to help facilitate those users without broadband access. Data Collection After ethical approval, 2 NCHDs were supplied with digital cameras (5 megapixels) and asked to record digitally, where feasible, all cases observed over a 6-month period. All data pertaining to the clinical case were recorded, including patient images, history, laboratory results, radiology films, and operative images. Images were taken only with fully informed consent from the patient in every case. A selection of the collected data was then compiled in a series of problem-based interactive cases. All images were anonymous, and any identifying features on patient images, films, or reports were removed before use using Adobe Photoshop CS (version 8.0; Adobe Systems Incorporated). Problem-based vignettes were constructed in a fashion that facilitated the dynamic interaction between software and user, with laboratory reports and radiological films changing according to the treatment plan of the student. Initially 305 cases and definitions were designed using a total of 225 images. Approval for the website was granted by the international health resource bodies “OMNI” and “Health on the Net Foundation” with the program having first met the rigorous standards of these respective bodies. 122
Both qualitative and quantitative data were collected to evaluate the project. First, a commercially validated set of monitoring scripts was employed to determine system usage (net applications). This set was monitored over a 12-month period from December 2004 to November 2005, and information on several parameters was collected. The IP (Internet Protocol) addresses of those involved in SURGENT’s development were removed from analysis to avoid bias. Only usage of the system by University College Cork students was included in statistical analysis. To determine patterns of use through self reporting, an anonymous electronic postcourse survey of final medical students was also used to evaluate access, process, and outcome criteria. Finally, in addition to the classic essay format of the written final surgical examination 2004, an extra problem-based vignette, based on the SURGENT template, was added and the scores were recorded for use as a control group. A similarly styled question was included in the 2005 examination, and scores were recorded to allow comparison of overall grades between the 2 years. Data analysis was performed using MINITAB (version 13-32; Minitab, Inc., State College, Pennsylvania). Both descriptive and inferential (Mann-Whitney) tests were used to describe program usage and to compare surgical scores across both written and clinical examinations for the years 2004 and 2005.
RESULTS Use of Program Overall, 115 of the 117 students in final medical 2004-2005 registered to use the online program, which constituted a 98% uptake. The total number of logins over the 12-month period was 1356. The use of the program was significantly higher during the months of April (23% ⫽ 308 logins) and May (21% ⫽ 279 logins) (Fig. 3). The day of the week affected the pattern of use, with highest use on Mondays (19% ⫽ 259 logins) and lowest on Saturdays (12% ⫽ 157 logins). Mean time spent on the program was 71 minutes. The most regularly accessed sections were “LongCases” (36%) and “Short-Cases” (28%) (Fig. 4). Questionnaire In total, 95 of the 117 members of final medical year 2005 replied to the electronic questionnaire, providing a response
Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
FIGURE 3. Use of the program per month. Use of the program was significantly higher during the months of April (23% ⫽ 308 separate logins) and May (21% ⫽ 279 logins). Correspondingly, use of the system fell outside the university term.
rate of 73%. Overall, 100% of respondents used the program, and 66% (56) reported spending more than 30 minutes per session on the program, with 47% (45) reporting 1 session per week and 37% (35) between 2 and 5 sessions per week. When asked what part of the final medical syllabus the program helped with the most, 41% (39) reported “Clinical Exam,” 34% (32) “Written Exam,” and 25% (24) “Clinical Attachments.” When asked to identify what features were most beneficial when using the program, 29% (28) responded “Question and Answer” format, whereas an equal number (28) felt that “Photos” constituted the most beneficial feature, and 15 (15%) felt the management plan to be valuable. All respondents felt that the program should be kept for future years. When questioned about the use of other Web-based teaching packages, 30% (32) reported using these to supplement their study. Examination The number of overall first-class honors increased from 11 (2004) to 20 (2005) (p ⫽ 0.01). No significant change occurred in the number of overall honors awarded between the years 2004 and 2005. Similarly no significant change occurred in the number of those participants who failed, with 3 students failing in both years.
DISCUSSION This study demonstrates that problem-based learning, enhanced with visual features, and in an interactive, dynamic environment is educationally well founded, popular, and to the benefit of students. As reported in the questionnaire, most students spent over 30 minutes per session online. This average is borne out with the data collected electronically, which reports a mean time online of 71 minutes per session. The design of an elaborate computer-based teaching program is of little use if it is not used by the students; however, this study establishes that such programs are popular with students. Additional analyses of the program reveal particular patterns of use. Some patterns are predictable, such as a peak in the months preceding examinations followed by a lull in the summer months. The daily pattern of use, however, raises the question of possible limited student access to Internet facilities. The Monday peak and weekend dip would support previously reported figures of Internet access among university students.9,10 Another result of interest was the report from 30% of respondents regarding their use of online teaching websites to supplement their study. This result raises several issues relating to the both limited “off-campus” Internet access for medical students
FIGURE 4. Use of SURGENT sections: The “Default” and “frontpage” sections represent those pages required for initial logging in. Excluding these sections, the most regularly accessed sections were “Long-Cases” (36%) and “Short-Cases” (28%). Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
123
and the monitoring of the accuracy of sites used to supplement study, which is an issue of increasing importance in light of the rapid growth of the Internet as a medium for education.11 The purpose of designing this program was to introduce visual or “iconic” elements to problem-based learning. Students deemed “Photos” and “Questions and Answers Format” as the 2 program features of most benefit. The third feature considered to be of most assistance was the “Management Plan and Feedback.” This response is of interest when one recalls the work of Bruner6,12 who reported the 3 modalities necessary to construct cognitive structures: “iconic,” “symbolic,” and “enactive.” In this study, iconic is represented by the visual images, or “Photos”; symbolic is represented by words or “Questions and Answers”; and enactive is observed in the “Management Plan and Feedback.” Students reported all 3 modalities to be among the most beneficial features of SURGENT, which emphasizes their importance in student learning. Although the authors recognize that a multitude of extraneous variables influence the grades achieved by a medical year, a formal meta-analysis would be required to explore fully the impact of such variables. Bearing this in mind, the results of this study are encouraging with a statistically significant improvement in the number of first-class honors achieved. Another possible reason for this increase may be the appointment of a full-time clinical tutor; however as both events occurred within similar time lines, it is difficult to assess each independently. Furthermore, a statistically significant impact on the overall failure and honors rate was not demonstrated. A thorough assessment of this impact will entail more observation of the program over many years. Additionally, this study has assessed the impact of an IT-based, modified, constructivist program on a small number of senior medical students. Full evaluation of such a system demands rigorous review of its effect across a range of different medical education systems. Programs such as SURGENT are not intended to replace teacher–student relationships but to supplement those interactions through the introduction of the student to “independent practice.” Through the work of others,13,14 it is known that the optimal learning environment is not the authentic simulation of an environment but the simplification of that environment, populating that world with features approximating to those found in reality. These computer-based worlds introduce the visual features hereto lacking from PBL, succeeding in combining the 3 modalities of iconic, symbolic, and enactive. These modalities can facilitate the construction of students’ own conceptual models and in doing so prepare them for solving the type of clinical problems they will be faced with in the early stage of their careers. Programs such as SURGENT assist in facilitating the educational paradigm shift toward more interactive approaches to teaching, thus increasing students’ responsibility for their own learning. Such approaches have been called for by educationalists such as Zane Berge15 in their advocacy of better integration of constructivist theory in integrated learning. University College Cork is currently in the process of curriculum reform. First-year students have just completed 1 year in 124
a new, system-based, integrated, problem-assisted learning program. Students in years 2 through 5 are engaged in a more traditional discipline-based program. SURGENT currently targets our fourth- and final-year students, complimenting clinical attachments in providing problem-based learning. It is relevant and motivating. It facilitates integration of knowledge gained from various disciplines, allowing opportunities to apply that knowledge, gaining a deeper understanding of the patient and their surgical condition. Since completion of this study, use of the program SURGENT has increased rapidly and is now accessed by students in 58 different countries with up to 35,000 page views per day (unpublished data). As a result, additional assessment will now take place in universities from 4 different continents. Furthermore, in conjunction with the lateral expansion of SURGENT to include other specialties, it is intended to develop case scenarios at increasing levels of complexity to allow junior students graded exposure, as part of a problem-initiated approach. By adding data as the student progresses, it is hoped that SURGENT can facilitate the vertical integration of course material.
REFERENCES 1. General Medical Council. Tomorrow’s Doctors. Recom-
mendations on Undergraduate Medical Education. London: GMC; 1993. 2. Schon DA. The Reflective Practitioner: How Professionals
Think in Action. New York: Basic Books; 1983. 3. Smith MK. Local Education. Buckingham, UK: Open
University Press; 1994. 4. von Glaserfeld. Constructivism in education. In: Husen
N, Postlethwaite T, eds. International Encyclopedia of Education. Oxford, UK: Pergamon; 1989:162-163. 5. Ernest P. Varities of constructivism: their metaphors, ep-
idemiologies and pedagogical implications. In: Preece P, Fox R, eds. Constructivism: Perspectives. Exeter, UK: University of Exeter; 1997:73-92. 6. Bruner JS. The Process of Education. London: Harvard
University Press; 1978. 7. Albanese MA, Mitchell S. Problem-based learning: a re-
view of literature on its outcomes and implementation issues. Acad Med. 1993;68:52-81. 8. Kerfoot BP, Masser BA, Hafler JP. Influence of new edu-
cational technology on problem-based learning at Harvard Medical School. Med Educ. 2005;39:380-387. 9. Burr L, Spennemann DHR. Patterns of user behavior in
university online forums. Internat J Instructional Technol Distance Learn. 2004;1:11-28.
Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
10. Healy DG, Fleming FJ, Gilhooley D, et al. Electronic
13. Wakefield J. Educational Psychology. Learning to be a Prob-
learning can facilitate student performance in undergraduate surgical education: a prospective observational study. BMC Med Educ. 2005;5:23.
lem Solver. Boston, MA: Houghton Mifflin Company; 1996.
11. Curran VR, Fleet L. A review of evaluation outcomes of
14. Weeks KW, Lyne P, Torrance C. Written drug dosage
web-based continuing medical education. Med Educ. 2005;39:561-567.
errors made by students: the threat to clinical effectiveness and the need for a new approach. Clin Effectiveness Nurs. 2000;4:20-29.
12. Bruner JS. Toward a Theory of Instruction. Cambridge,
15. Berge ZL, Clark T. Virtual Schools: Planning For Success.
UK; Belknap/Harvard; 1975.
New York, NY: Teachers College Press; 2005.
Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
125
“SURGENT”—Student e-Learning for Reality: The Application of Interactive Visual Images to Problem-Based Learning in Undergraduate Surgery Mark Corrigan, Michelle Reardon, Connor Shields, and Henry Redmond Department of Academic Surgery, Cork University Hospital, Cork, Ireland Information technology has the potential to transform surgical education. Combining symbolic, iconic, and enactive teaching modalities to construct an authentic conceptual model potentially can transform a primarily didactic learning experience into an interactive Web-enhanced one. This study sought to assess the introduction of a Web-based module to complement traditional surgical undergraduate curricula. Adopting the clinical case as its fundamental educational approach, an online resource simulating surgical clinical decision making (“SURGENT,” http://www.surgent.ie) was developed, which consists of the interpretation of clinical photographs, laboratory data, and X-rays as well as the formulation of a management plan. Evaluation was in both a qualitative and a quantitative fashion. An anonymous postcourse survey (73% response) of 117 final medical students was used to by researchers to evaluate access, process, and outcome criteria. SURGENT was used by 98% of students, with 69% spending more than 30 minutes per session on the program. First-class honors in the final surgical clinical examination improved from 11% to 20% (p ⫽ 0.01) as compared with the previous control year. A Web-enhanced interactive surgical module in an undergraduate course can convey successfully information and understanding beyond the textbook. It is intended that SURGENT will supplement textbooks and ward experience, allowing students to develop their clinical decision-making skills. (J Surg 65:120-125. © 2008 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: SURGENT, information technology, e-learn-
ing, virtual learning, internet based learning COMPETENCY: Medical Knowledge, Practice Based Learning, Systems Based Practice
Correspondence: Inquiries to Mark Anthony Corrigan, Department of Surgery, Cork University Hospital, Cork, Ireland; fax: 00353 87 94 11 555; e-mail: [email protected]
120
INTRODUCTION Current trends in medical education suggest a move toward problem-based learning (PBL), with an emphasis on studentcentred education and use of information technologies. One of the main motivating factors for these changes was The General Medical Council’s publication entitled “Recommendations on Undergraduate Medical Education.” This publication states that learning through curiosity, the exploration of knowledge, and the critical evaluation of evidence should be promoted to ensure a capacity for self-education.1 The same document recommends that learning systems should be informed by modern educational theory and should draw on the wide range of technological resources available. The idea of “learning through curiosity” and “learning by experience” are concepts discussed by the educationalist Donald Schon. In his seminal work “The Reflective Practitioner,” he acknowledges that although “we do not have a full understanding of things before we act, but, hopefully, we can avoid major problems while ’testing the water.”2 Additionally when confronted by a situation unique to the practitioner he/she “reflects on the phenomenon before him, and on the prior understandings which have been implicit in his behaviour.” This sentiment is echoed by his contemporary M. K. Smith when describing a practitioners approach to a new situation: “We have to fall back on routines in which previous thought and sentiment has been sedimented. It is here that the full importance of reflection-on-action becomes revealed. . . . the space afforded by recording, supervision and conversation with our peers allows us to approach these. Reflection requires space in the present and the promise of space in the future.”3 These concepts of safely “testing the water” and of providing students with those “prior understandings” necessary for effective clinical thought are fundamental in the development of new teaching strategies. The primary goal of PBL, which is based on constructivist educational theory, is the integration and synthesis of knowledge, skills, and attitudes to effective critical thinking. The constructivist approach to teaching argues that understanding is
Journal of Surgical Education • © 2008 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/08/$30.00 doi:10.1016/j.jsurg.2007.11.011
not solely built up from received pieces of information, but it is a dynamic process whereby it is assembled actively and reshaped constantly by the individual according to new experiences.4,5 A combination of language, images, and actions facilitates this process.6 Clinical scenarios in a realistic environment using patient contact, end-of-bed notes, laboratory results, and radiological images should be encouraged. To this end, symbolic (language), iconic (images), and enactive (action) modalities are used to construct an authentic conceptual model that can then be applied to future clinical problems. However, the traditional application of PBL involves the application of only the symbolic and enactive modalities, failing to incorporate iconic imagery. Consequently, although PBL has been shown to be more nurturing and enjoyable compared with conventional instruction,7 it has been demonstrated that PBL students have scored lower in basic science examinations, which highlights an apparent gap in cognitive knowledge that has the potential to affect practice outcomes. The increased use of information technology (IT) supports a student-centred educational approach that promotes student autonomy and flexibility.8 IT has the potential to engender students with a desire to manage their own learning, which is a valuable step toward lifelong learning. The student dictates the time, duration, frequency, depth, and direction of interactive study. It can also offer opportunities for the student to enquire into their own competence, promoting self-awareness. Every student cannot observe every patient. Interactive information technology was employed in this project to broaden the students’ exposure to a greater variety of patients in a variety of clinical contexts. Applying the constructivist theory as its fundamental educational approach, an online resource simulating surgical clinical decision making (“SURGENT,” http:// www.surgent.ie) was developed by the surgical faculty. This resource was designed to reflect the daily problems and experi-
ences faced by surgical nonconsultant hospital doctors (NCHDs) and to allow students access to these experiences. This study was designed to determine whether information technology can be used to design and implement a system capable of delivering constructivist ethos, enhanced by iconic imagery, to surgical teaching. Additionally, it sought to evaluate the response of students to the integration of such a system to the surgical curriculum and its effect on student examination performance.
MATERIALS AND METHODS Database The relational database MySQL (Structured Query Language) was employed to store patient data. Interactions between the user and the database were regulated by a series of specially written scripts, encoded in the scripting language PHP (Hypertext Preprocessor) and embedded in HTML (Hypertext Markup Language). Interface The online interface was constructed using HTML with embedded PHP and written using the Macromedia HomeSite interface (version 5.5; Adobe Systems Incorporated, San Jose, California). A dropdown menu was employed to facilitate ease of navigation. To monitor usage of the system, access was protected through the use of an individual username and password, with automated free registration available 24 hours a day. Sections were created to represent the standard divisions used to examine and teach medical students. These sections comprised long-cases, short-cases, equipment, radiology, operative, and definitions (Fig. 1). Long cases were designed to enable the user to order relevant hematological, biochemical, microbiological,
FIGURE 1. The SURGENT frontpage is divided into educational sections: Long Cases, Short Cases, Equipment, Radiology, Operative and Definitions, as observed from this screen shot. Access to these sections is through a user-friendly dropdown menu. Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
121
Ethical Approval Ethical approval for the collection of patient data and its use in an online teaching program was sought and granted by the Clinical Research Ethics Committee of the Cork Teaching Hospitals. Assessment of Program
FIGURE 2. “Long Cases” enable the user to order any relevant tests from the hematology, biochemistry, microbiology, and radiological departments. Furthermore, the user may request the patients’ vital signs at any stage in the case. Both test results and vital signs will change in accordance with the actions of the user. The user may also order blood products from the blood bank.
and radiological tests (Fig. 2). According to the users’ actions, the results of these tests will change, providing clinical feedback to the student. Similarly, the user may request their patients’ vital signs. These will also change according to the users’ actions. Commonly examined conditions were incorporated in the “short cases” section, with frequently preformed operations and regularly used equipment included in the “operative” and “equipment” sections. All images used were kept below the size of 100 Kb to help facilitate those users without broadband access. Data Collection After ethical approval, 2 NCHDs were supplied with digital cameras (5 megapixels) and asked to record digitally, where feasible, all cases observed over a 6-month period. All data pertaining to the clinical case were recorded, including patient images, history, laboratory results, radiology films, and operative images. Images were taken only with fully informed consent from the patient in every case. A selection of the collected data was then compiled in a series of problem-based interactive cases. All images were anonymous, and any identifying features on patient images, films, or reports were removed before use using Adobe Photoshop CS (version 8.0; Adobe Systems Incorporated). Problem-based vignettes were constructed in a fashion that facilitated the dynamic interaction between software and user, with laboratory reports and radiological films changing according to the treatment plan of the student. Initially 305 cases and definitions were designed using a total of 225 images. Approval for the website was granted by the international health resource bodies “OMNI” and “Health on the Net Foundation” with the program having first met the rigorous standards of these respective bodies. 122
Both qualitative and quantitative data were collected to evaluate the project. First, a commercially validated set of monitoring scripts was employed to determine system usage (net applications). This set was monitored over a 12-month period from December 2004 to November 2005, and information on several parameters was collected. The IP (Internet Protocol) addresses of those involved in SURGENT’s development were removed from analysis to avoid bias. Only usage of the system by University College Cork students was included in statistical analysis. To determine patterns of use through self reporting, an anonymous electronic postcourse survey of final medical students was also used to evaluate access, process, and outcome criteria. Finally, in addition to the classic essay format of the written final surgical examination 2004, an extra problem-based vignette, based on the SURGENT template, was added and the scores were recorded for use as a control group. A similarly styled question was included in the 2005 examination, and scores were recorded to allow comparison of overall grades between the 2 years. Data analysis was performed using MINITAB (version 13-32; Minitab, Inc., State College, Pennsylvania). Both descriptive and inferential (Mann-Whitney) tests were used to describe program usage and to compare surgical scores across both written and clinical examinations for the years 2004 and 2005.
RESULTS Use of Program Overall, 115 of the 117 students in final medical 2004-2005 registered to use the online program, which constituted a 98% uptake. The total number of logins over the 12-month period was 1356. The use of the program was significantly higher during the months of April (23% ⫽ 308 logins) and May (21% ⫽ 279 logins) (Fig. 3). The day of the week affected the pattern of use, with highest use on Mondays (19% ⫽ 259 logins) and lowest on Saturdays (12% ⫽ 157 logins). Mean time spent on the program was 71 minutes. The most regularly accessed sections were “LongCases” (36%) and “Short-Cases” (28%) (Fig. 4). Questionnaire In total, 95 of the 117 members of final medical year 2005 replied to the electronic questionnaire, providing a response
Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
FIGURE 3. Use of the program per month. Use of the program was significantly higher during the months of April (23% ⫽ 308 separate logins) and May (21% ⫽ 279 logins). Correspondingly, use of the system fell outside the university term.
rate of 73%. Overall, 100% of respondents used the program, and 66% (56) reported spending more than 30 minutes per session on the program, with 47% (45) reporting 1 session per week and 37% (35) between 2 and 5 sessions per week. When asked what part of the final medical syllabus the program helped with the most, 41% (39) reported “Clinical Exam,” 34% (32) “Written Exam,” and 25% (24) “Clinical Attachments.” When asked to identify what features were most beneficial when using the program, 29% (28) responded “Question and Answer” format, whereas an equal number (28) felt that “Photos” constituted the most beneficial feature, and 15 (15%) felt the management plan to be valuable. All respondents felt that the program should be kept for future years. When questioned about the use of other Web-based teaching packages, 30% (32) reported using these to supplement their study. Examination The number of overall first-class honors increased from 11 (2004) to 20 (2005) (p ⫽ 0.01). No significant change occurred in the number of overall honors awarded between the years 2004 and 2005. Similarly no significant change occurred in the number of those participants who failed, with 3 students failing in both years.
DISCUSSION This study demonstrates that problem-based learning, enhanced with visual features, and in an interactive, dynamic environment is educationally well founded, popular, and to the benefit of students. As reported in the questionnaire, most students spent over 30 minutes per session online. This average is borne out with the data collected electronically, which reports a mean time online of 71 minutes per session. The design of an elaborate computer-based teaching program is of little use if it is not used by the students; however, this study establishes that such programs are popular with students. Additional analyses of the program reveal particular patterns of use. Some patterns are predictable, such as a peak in the months preceding examinations followed by a lull in the summer months. The daily pattern of use, however, raises the question of possible limited student access to Internet facilities. The Monday peak and weekend dip would support previously reported figures of Internet access among university students.9,10 Another result of interest was the report from 30% of respondents regarding their use of online teaching websites to supplement their study. This result raises several issues relating to the both limited “off-campus” Internet access for medical students
FIGURE 4. Use of SURGENT sections: The “Default” and “frontpage” sections represent those pages required for initial logging in. Excluding these sections, the most regularly accessed sections were “Long-Cases” (36%) and “Short-Cases” (28%). Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
123
and the monitoring of the accuracy of sites used to supplement study, which is an issue of increasing importance in light of the rapid growth of the Internet as a medium for education.11 The purpose of designing this program was to introduce visual or “iconic” elements to problem-based learning. Students deemed “Photos” and “Questions and Answers Format” as the 2 program features of most benefit. The third feature considered to be of most assistance was the “Management Plan and Feedback.” This response is of interest when one recalls the work of Bruner6,12 who reported the 3 modalities necessary to construct cognitive structures: “iconic,” “symbolic,” and “enactive.” In this study, iconic is represented by the visual images, or “Photos”; symbolic is represented by words or “Questions and Answers”; and enactive is observed in the “Management Plan and Feedback.” Students reported all 3 modalities to be among the most beneficial features of SURGENT, which emphasizes their importance in student learning. Although the authors recognize that a multitude of extraneous variables influence the grades achieved by a medical year, a formal meta-analysis would be required to explore fully the impact of such variables. Bearing this in mind, the results of this study are encouraging with a statistically significant improvement in the number of first-class honors achieved. Another possible reason for this increase may be the appointment of a full-time clinical tutor; however as both events occurred within similar time lines, it is difficult to assess each independently. Furthermore, a statistically significant impact on the overall failure and honors rate was not demonstrated. A thorough assessment of this impact will entail more observation of the program over many years. Additionally, this study has assessed the impact of an IT-based, modified, constructivist program on a small number of senior medical students. Full evaluation of such a system demands rigorous review of its effect across a range of different medical education systems. Programs such as SURGENT are not intended to replace teacher–student relationships but to supplement those interactions through the introduction of the student to “independent practice.” Through the work of others,13,14 it is known that the optimal learning environment is not the authentic simulation of an environment but the simplification of that environment, populating that world with features approximating to those found in reality. These computer-based worlds introduce the visual features hereto lacking from PBL, succeeding in combining the 3 modalities of iconic, symbolic, and enactive. These modalities can facilitate the construction of students’ own conceptual models and in doing so prepare them for solving the type of clinical problems they will be faced with in the early stage of their careers. Programs such as SURGENT assist in facilitating the educational paradigm shift toward more interactive approaches to teaching, thus increasing students’ responsibility for their own learning. Such approaches have been called for by educationalists such as Zane Berge15 in their advocacy of better integration of constructivist theory in integrated learning. University College Cork is currently in the process of curriculum reform. First-year students have just completed 1 year in 124
a new, system-based, integrated, problem-assisted learning program. Students in years 2 through 5 are engaged in a more traditional discipline-based program. SURGENT currently targets our fourth- and final-year students, complimenting clinical attachments in providing problem-based learning. It is relevant and motivating. It facilitates integration of knowledge gained from various disciplines, allowing opportunities to apply that knowledge, gaining a deeper understanding of the patient and their surgical condition. Since completion of this study, use of the program SURGENT has increased rapidly and is now accessed by students in 58 different countries with up to 35,000 page views per day (unpublished data). As a result, additional assessment will now take place in universities from 4 different continents. Furthermore, in conjunction with the lateral expansion of SURGENT to include other specialties, it is intended to develop case scenarios at increasing levels of complexity to allow junior students graded exposure, as part of a problem-initiated approach. By adding data as the student progresses, it is hoped that SURGENT can facilitate the vertical integration of course material.
REFERENCES 1. General Medical Council. Tomorrow’s Doctors. Recom-
mendations on Undergraduate Medical Education. London: GMC; 1993. 2. Schon DA. The Reflective Practitioner: How Professionals
Think in Action. New York: Basic Books; 1983. 3. Smith MK. Local Education. Buckingham, UK: Open
University Press; 1994. 4. von Glaserfeld. Constructivism in education. In: Husen
N, Postlethwaite T, eds. International Encyclopedia of Education. Oxford, UK: Pergamon; 1989:162-163. 5. Ernest P. Varities of constructivism: their metaphors, ep-
idemiologies and pedagogical implications. In: Preece P, Fox R, eds. Constructivism: Perspectives. Exeter, UK: University of Exeter; 1997:73-92. 6. Bruner JS. The Process of Education. London: Harvard
University Press; 1978. 7. Albanese MA, Mitchell S. Problem-based learning: a re-
view of literature on its outcomes and implementation issues. Acad Med. 1993;68:52-81. 8. Kerfoot BP, Masser BA, Hafler JP. Influence of new edu-
cational technology on problem-based learning at Harvard Medical School. Med Educ. 2005;39:380-387. 9. Burr L, Spennemann DHR. Patterns of user behavior in
university online forums. Internat J Instructional Technol Distance Learn. 2004;1:11-28.
Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
10. Healy DG, Fleming FJ, Gilhooley D, et al. Electronic
13. Wakefield J. Educational Psychology. Learning to be a Prob-
learning can facilitate student performance in undergraduate surgical education: a prospective observational study. BMC Med Educ. 2005;5:23.
lem Solver. Boston, MA: Houghton Mifflin Company; 1996.
11. Curran VR, Fleet L. A review of evaluation outcomes of
14. Weeks KW, Lyne P, Torrance C. Written drug dosage
web-based continuing medical education. Med Educ. 2005;39:561-567.
errors made by students: the threat to clinical effectiveness and the need for a new approach. Clin Effectiveness Nurs. 2000;4:20-29.
12. Bruner JS. Toward a Theory of Instruction. Cambridge,
15. Berge ZL, Clark T. Virtual Schools: Planning For Success.
UK; Belknap/Harvard; 1975.
New York, NY: Teachers College Press; 2005.
Journal of Surgical Education • Volume 65/Number 2 • March/April 2008
125