- Email: [email protected]
Surgical instruction for general practitioners: How, who and how often?
Journal of Plastic, Reconstructive & Aesthetic Surgery (2010) 63, 1156e1162
Surgical instruction for general practitioners: How, who and how often?* Anne M. Collins a, Paul F. Ridgway b,*, Mohammed S.U. Hassan c, Christy W.K. Chou d, Arnold D. Hill a, Brian Kneafsey e a
Department of Surgery, Royal College of Surgeons in Ireland, Dublin, Ireland Department of Surgical Oncology, Princess Margaret Hospital, University Health Network, University of Toronto, 3-130, 610 University Avenue, Toronto, Ontario M5G 2M9, Canada c Department of Plastic Surgery, University Hospital, Birmingham, UK d Department of Plastic Surgery, University Hospital of North Durham, UK e Department of Plastic Surgery, Beaumont Hospital, Dublin, Ireland b
Received 9 February 2009; accepted 11 May 2009
KEYWORDS Construct validity; Minor surgery; Plastic surgeon; Continuing medical education; General practitioner
Summary Educational programmes, designed to meet the training needs of General Practitioners (GPs) performing minor surgical procedures, have previously been shown to increase their surgical workload. The change in the level of competence following these programmes has not been assessed. The aims of this study were two-fold: to evaluate the vertical mattress suture for construct validity and to assess the impact of plastic surgery training on the surgical skill of GPs. Thirty non-consultant hospital doctors and 27 self-selected GPs were included. Using a modified objective structured assessment of technical skills (OSATS) scoring system, construct validity of the vertical mattress suture was confirmed. The median total OSATS score was 16 points (26.7%) in the novice group (medical registrars), 38.5 points (64.2%) in the intermediate group (surgical SHOs) and 59 points (98.3%) in the expert group (surgical registrars, p < 0.001, KruskaleWallis test). Objective assessment in the GP group immediately following practical instruction revealed a median overall improvement of 31.7% (19 points) in total OSATS scores (p < 0.001, Friedman non-parametric test, F). At six months follow-up all course participants had improved compared to their baseline. A median overall improvement of 13 points (21.7%) was noted (p < 0.001, F). However, the majority (80%, n Z 20) had deteriorated from the standard set immediately after the course with a median overall reduction in total OSATS scores of six points (10%, p Z 0.001, F).
* Meetings at which this work was presented: This paper was presented at the Winter meeting of the British Association of Plastic, Reconstructive and Aesthetic Surgeons in the Royal College of Surgeons in London and was awarded the Kilner prize for the best overall paper. * Corresponding author. Tel.: þ1 (416) 946 4501. E-mail address: [email protected] (P.F. Ridgway).
1748-6815/$ - see front matter ª 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.bjps.2009.05.023
Surgical instruction for general practitioners
1157
Plastic surgery training is immediately efficacious in improving the technical proficiency of GPs. Through objective assessment of a standardised suture task we demonstrated a low rate of educational decay of 10% over a six-month period. ª 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved.
General practitioners (GPs) perform a range of minor surgical procedures.1 In general, these include any procedures carried out under local anaesthetic that are easy to perform, involve easily accessible structures, are of short duration and are not usually associated with significant post-operative complications.2 The waiting time for GPs performing such procedures is short. They are carried out at a time convenient for patients, in a less intimidating environment than the hospital, and are associated with high levels of patient satisfaction. Such activity has been shown to be cost effective and reduces hospital waiting lists for minor surgery.3 Formal surgical training is not a component of either the Irish College of General Practitioners or the Royal College of General Practitioners core curriculum. The performance of minor surgical procedures is regarded as an area of special interest for individual GPs. Such skills are considered relevant but not core to general practice. The acquisition of surgical skills by GPs is, therefore, largely unstructured and no formal appraisial system for technical proficiency exists. It is the responsibility of the individual GP to identify their own strengths and weaknesses and obtain additional training should they so require. Basic skin surgery courses afford an opportunity to do so. Similar to surgical skills laboratories, such courses are associated with a less stressful learning environment when compared to the operating room. They allow for repetitive and deliberate practice, participatory rather than observational learning and the identification and correction of basic performance flaws.4 It is well recognised that surgical competence encompasses many factors as well as technical proficiency.5 Correct wound closure, however, is an essential skill and is particularly important in the setting of general practice. Poor suturing technique and excessive suture tension are directly related to adverse wound healing and increased scarring.6 One of the most common methods for wound closure is the simple interrupted suture. The vertical mattress suture is more complex requiring a second superficial bite in the vertical plane to evert the skin edges.7 By using objective assessment of a suture task, our aims were to evaluate the immediate impact of plastic surgery training on the surgical skill of GPs and the durability of skill acquisition. The selected suture task should demonstrate construct validity. This is the extent to which a task differentiates between different experience levels. A task is deemed construct valid if a more experienced trainee consistently performs better than a less experienced trainee. Too simple a task may bias against determining construct validity. The vertical mattress suture, for example, has previously been shown to have a broader spread of performance than the simple suture and the potential to discriminate between individuals in an expert group.8
A common limitation of continuing medical education programs, such as one-day skin surgery courses, is the time delay between when a skill is learned and when it is next performed. Although technical proficiency may improve immediately following training, how much is retained is uncertain. The ultimate value of skills courses should therefore be measured not by performance immediately after training but by performance after a time delay.9
Methods This was a prospective cohort design study with two components: I. Evaluation of the vertical mattress suture for construct validity. II. Assessment of the impact of plastic surgery training on the surgical skill of GPs.
Evaluation of the vertical mattress suture for construct validity In this component of the study, ten medical registrars with no previous surgical training (novice surgical but medically trained group), ten surgical senior house officers (SHOs; intermediate group) and ten surgical registrars (experts) from Beaumont hospital, Dublin, Ireland were selected randomly to participate. Pigskin is widely accepted as a suitable model for human skin10 and was used in this study. A 4 0.5 cm wound was made and participants were instructed to treat it as a wound on the arm. Plastic surgery instruments and a range of suture material [3-0 vicryl, 3-0 nylon, 3-0 prolene (Ethicon) and 3-0 novafil (Tyco Healthcare)] was provided (Figure 1). Participants were instructed to insert a single vertical mattress suture in the centre of the wound using the instruments and suture they felt were most appropriate for the task. No antecedent training was provided to reduce familiarty bias. If participants were unfamiliar with the vertical mattress suture, brief verbal instruction was provided. All participants were scored in a blinded fashion by a single surgeon (AMC) using a modified Objective Structured Assessment of Technical Skills (OSATS) scoring system. OSATS was devised in 1997 and was initially validated for bench model simulation.11 It has also been used by our group to validate a hybrid surgical stimulator12 and to complement metric assessment used in suturing studies.13 OSATS is comprised of two individual scores: a task specific checklist score and an overall global rating score of operative performance. A twenty-point task specific
1158
Figure 1 task.
A.M. Collins et al.
Setup for assessment of vertical mattress suture
checklist was devised for the vertical mattress suture by identifying its integral components, including appropriate knowledge of the task, correct choice of instruments and suture material, correct position of the needle in the needle holder and competent instrument handling. In addition the needle must enter the skin at a right angle, exit at a point equidistant from the insertion point for both bites and be reverse mounted for the second superficial bite. The skin should be approximated with the appropriate level of tension with a square knot and meticulous tissue handling should be evident throughout the task. Participants scored one point for each item correctly performed on the checklist, giving a maximum score of 20. The global rating score of operative performance measures overall ability to perform the procedure. It assesses general surgical competence and has a broad applicability. Eight parameters were assessed including respect for tissue, time and motion, familiarity with and handling of instruments, flow and knowledge of the procedure, overall performance and the quality of the final product. Each parameter was graded on a 5-point Likert scale (Likert, 1932) from zero (very poor) to five (clearly superior performance), yielding a maximum score of 40.
Assessment of the impact of plastic surgery training on the surgical skill of GPs The second part of the study involved 27 self-selected GPs who were evaluated in the setting of a skin surgery course in Dublin in December 2007. The course faculty included an associate specialist in plastic surgery, a consultant dermatologist, a senior plastic surgery specialist registrar and two plastic surgery senior house officers. Prior to practical assessment all participating GPs completed a questionnaire documenting their experience level, previous surgical training and how frequently minor procedures are performed in their practice. In addition, the GPs were questioned on the different types of mattress sutures, indications for use and whether they are used in their practices. Each GP was then asked to perform the same vertical mattress suture task as described in the construct validity
component of the study and was scored in the same manner by the same surgeon (AMC). Participants subsequently completed an intensive skin surgery course and were instructed on a 5.4:1 ratio. The course faculty continually rotated between the groups. In total, 300 min of practical instruction were provided incorporating instrument and tissue handling, suture selection, biopsy and excision of lesions and simple, mattress and subcuticular suturing. 150 min were dedicated to suturing technique. At the conclusion of the course, participants were asked to repeat the mattress suture task and were scored in the same manner by the same surgeon as described previously. A practice kit [disposable suture packs (Vernon-Carus Ltd, Lancashire) and sutures (3-0 nylon, Ethicon)] was also offered to participants. Six months later the course participants were invited to participate in a follow-up study. Once again they were questioned on the different types of mattress suture and indications for their use. In addition it was noted whether they had performed mattress sutures in the preceding six months, and whether they had availed of or used the suture pack offered to them. A study-specific questionnaire was also completed and scored on a 5-point Likert scale. The perceived improvement in surgical skill, level of satisfaction with the course, the likelihood of recommending the course to a colleague and of benefitting from repeating it was documented. Finally the mattress suture task was reassessed and participants were scored in the same manner by the same surgeon (AMC) who performed the other assessments. All results were recorded on a database.
Statistics Data was tested for normality using ShapiroeWilk tests. Non-parametric analysis was used. KruskaleWallis test (KWT) was performed to evaluate for construct validity of the vertical mattress suture. ManneWhitney U (MWU), Wilcoxon Signed Ranks (WSR) and Friedman tests (F) were used to compare GP scores at baseline, immediately postcourse and six months later. Correlations between the results of the tasks and previous operative experience were made using Spearman’s non-parametric correlation. p Values, where appropriate, were taken to be significant at 0.05. All statistics were calculated using the Statistical Package for Social Sciences (SPSS, version16 Chicago, IL).
Results Evaluation of the vertical mattress suture for construct validity Construct validity of the vertical mattress suture was confirmed. The novice group (medical registrars) had a median overall score of 16 points (26.7%), the intermediate group (surgical SHOs) had a median score of 38.5 points (64.2%) and the expert group (surgical registrars) had a median score of 59 points (98.3%, p < 0.001, KWT, Figure 2). When taken individually, the median checklist and global rating scores correlated with an increasing level of experience (p < 0.001, KWT, Table 1).
Surgical instruction for general practitioners
1159 Table 1 Evaluation of construct validity of the vertical mattress suture. Median scores for medical registrars (novice group), surgical SHOs (intermediate group) and surgical registrars (expert group)
Novice Intermediate Expert p (KWT)
TSC (%)
OP (%)
Overall (%)
5.5 (27.5) 14.5 (72.5) 19 (95) 0.000002*
11 (27.5) 24.5 (61.3) 40 (100) 0.000002*
16 (26.7) 38.5 (64.2) 59 (98.3) 0.000002*
TSC: task specific checklist score, OP: global rating score of operative performance, Overall: total OSATS score (TSC þ OP). *Significance is at the 0.05 level.
Figure 2 Boxplot of construct validity of vertical mattress suture. Total scores for novice, intermediate and expert groups. Maximum score was 60. The heavy line is the median. The bars represent the range of the data.
Assessment of the impact of plastic surgery training on the surgical skill of GPs Twenty-seven GPs enrolled in the course. Two were not available to participate in the follow-up study and were excluded from statistical analysis. The median experience level was ten years. The majority of participants (72%, n Z 18) perform minor surgical procedures regularly but only 12% (n Z 3) had previously received formal surgical training. Although 88% (n Z 22) had heard of mattress sutures, only 32% (n Z 8) had previously experience performing them, 16% (n Z 4) were familiar with the indications for their use and 8% (n Z 2) were aware of the different types. Assessment immediately following practical instruction revealed an improvement in all participants compared to baseline. The median overall improvement in total OSATS scores was 19 points, which is equivalent to an increase of 31.7% (p < 0.001, F). The median percentage improvement in task specific checklist scores was 45% (p < 0.001) and the median percentage improvement in global rating scores was 20% (p < 0.001, WSR). The follow-up rate was 93%. Ninety-six percent (n Z 24) recalled the different types of mattress sutures and 72% (n Z 18) were aware of the indications for their use. Forty percent (n Z 10) had used mattress sutures in the preceding six months. Of the 80% (n Z 20) that availed of the suture pack, only 20% (n Z 4) had used it to practice their technique. By comparing the baseline (pre-course) total OSATS scores with those at six months follow-up, a median overall improvement of 13 points (21.7%) was noted (p < 0.001, F, Table 2, Figure 3). The median percentage improvement in task specific checklist and global rating scores over the same time period was 35% and 17.5% respectively (p < 0.001, p < 0.001, WSR, Table 3). At six months follow-up all course participants had improved technical proficiency compared to their baseline. However, the majority (80%, n Z 20) had deteriorated below the standard they set immediately after the course. The median overall reduction in total OSATS scores was six
points (10%, p Z 0.001, F, Table 2). The median percentage reduction in task specific checklist and global rating scores were 10% and 2.5% respectively (p < 0.001, p Z 0.005, WSR, Table 3). Of the 80% (n Z 20) of GPs whose proficiency was below the standard they set immediately after practical instruction, 70% had performed minor procedures regularly over the preceding six months but only 25% had used mattress sutures. With regard to level of confidence in performing minor procedures, 45% (n Z 9) acknowledged they had a lower level of confidence compared to immediately postcourse. The median reduction in total OSATS score in this group was ten points (16.7%) The remaining 55% (n Z 11) felt they had the same level of confidence at six months follow-up. The median deterioration in this group was five points (8.3%). The difference between these two groups was not statistically significant (p Z 0.15, MWU). A minority of GPs (20%, n Z 5) improved on their postcourse total OSATS scores at six months follow-up, with a median improvement of four points (6.7%). All of this cohort performed minor procedures regularly and had used mattress sutures over the preceding six months. Moreover, there was a significant correlation between use of mattress sutures in the six months following the course and the task specific checklist score at the follow-up visit. Spearman’s correlation coefficient was 0.65 (p Z 0.008). There was a high level of satisfaction among participants with the training received at the skin surgery course. Ninety-two percent (n Z 23) felt their surgical skills improved and 84% (n Z 21) felt more confident in dealing with the surgical cases encountered in their practice. 96% (n Z 24) felt they would benefit from taking the course again and all participants would recommend the course to a colleague.
Discussion The majority of GPs performing minor procedures do not have any formal surgical training. In order to obtain a place on the Health Authority minor surgery list in the United Kingdom, GPs must attend a course approved by regional advisors in general practice and three practical sessions covering the necessary range of procedures with approved teachers.14 Despite this, a high level of dissatisfaction with teaching in minor surgery exists.15 GPs are often unwilling to excise complex lesions such as sebaceous cysts16 and
1160 Table 2
Mean Median p (F)
A.M. Collins et al. Comparison of total OSATS scores pre-course, post-course and at six months follow-up Pre-course OSATS (%)
Post-course OSATS (%)
Six-month OSATS (%)
Change: pre- to post-course (%)
Change: pre-course to six months (%)
Change: post-course to six months (%)
23.9 (39.9) 25 (41.7)
41.8 (69.7) 44 (73.3)
36 (60) 38 (63.33)
17.9 (29.8) 19 (31.6) 0.0000000008*
12.1 (20.2) 13 (21.7) 0.00000006*
5.8 (9.7) 6 (10) 0.001*
F Z Friedman non-parametric test. *Significance is at the 0.05 level.
lesions on the face.15 A structured training programme would improve the technical proficiency of GPs, thus allowing more GPs tackle a wider range of cases. Structured assessment has been shown to function effectively as a method of formally assessing the surgical skills of family medicine residents,17 and of general surgery8 and plastic surgery trainees.18 This type of assessment allowed us to identify and correct deficiencies in suturing performance and ensure basic principals were followed by participating GPs. Instruction was then provided in specific areas, and the resultant improvement in suturing competence was demonstrated. Such instruction has been shown to decrease the frequency of technical errors and the types of errors made.5 Although simple interrupted sutures are more commonly used, the more complex vertical mattress suture was selected as the task for the purposes of evaluation. The increased complexity of the latter allows for more extensive assessment of surgical competence. Interobserver relaibility was not specifically tested in our construct as it has been tested and found to be acceptable using a similar OSATS model. 8 Our findings demonstrated a wider spread of performance in the novice and intermediate groups than in the expert group. Others have suggested the mattress suture has the potential to discriminate between individuals in an expert group.8 These conclusions were not investigated specifically by our study design.
Figure 3 Boxplot of total OSATS scores (in percentages) of GPs pre-course, immediately post-course and at six months follow-up. Maximum score was 100 (%). The heavy line is the median. The bars represent the range of the data.
It has previously been shown that less experienced surgeons benefit more from instruction than more experienced ones.19 We therefore postulated that a short period of intense instruction by plastic surgeons would improve the level of surgical proficiency of a group of GPs, of whom only 12% had previously received formal surgical training. Other educational programmes, designed to meet the training needs of GPs performing minor procedures, have been shown to increase their surgical workload, but the change in the level of competency following these programmes was not assessed.20,21 Through assesment of a standardised suture task, we have shown that participation at a one-day skin surgery course increases the level of competence among GPs by 31.7% immediately and by 21.7% at six months follow-up. The one-day skin surgery course evaluated in this study is typical of what is offered in the majority of continuing medical education programmes; short and intensive. It has recently been suggested that practice interspersed with periods of rest (distributed) is superior to the massed method of teaching that predominates in skills courses (practice delivered in a continuous block with little or no rest).9 Therefore, a structured minor surgery training programme delivered over a period of weeks is desirable, in order to maintain the level of proficiency obtained as a result of additional training and to reduce the rate of educational decay. With regard to self-assessment, our findings correlate with other reports suggesting that confidence in performing a procedure, while important, does not necessarily correlate with ability.22 56% (n Z 14) of GPs assessed claimed to have the same level of confidence six months later as they did immediately after the course. The majority of this cohorts’ (78.6%, n Z 11) proficiency had deteriorated over the six-month follow-up period. In order to facilitate practicing of the skills learned on the course, each participant was offered a practice kit, consisting of a disposable suture set and sutures, at the conclusion of the course. The majority (80%, n Z 20) availed of the pack, the remainder cited possession of their own instruments as the reason for declining. However only 20% (n Z 4) used the pack provided, thus confirming this intervention as unsatisfactory. The limitations of this study include the small sample size. The sample was also biased, as this group of GPs were self-selected and therefore motivated. However, we feel that this self-funded, enthusiastic group is ideal to study. In theory they should benefit most from practical instruction and demonstrate high levels of retention. Through objective assessment of a suture task, an
Surgical instruction for general practitioners Table 3
1161
Comparison of pre-course, post-course and six month follow-up checklist and global rating scores
Mean Median Percentiles 25 50 75
TSC 1 (%)
TSC 2 (%)
TSC 3 (%)
OP 1 (%)
OP 2 (%)
OP 3 (%)
8.1 (40.6) 8 (40)
16.5 (82.4) 17 (85)
13.7 (68.5) 15 (75)
15.8 (39.5) 17 (42.5)
25.4 (63.4) 25 (62.5)
22.2 (55.5) 24 (60)
6 (30) 8 (40) 11 (55)
15 (75) 17 (85) 18 (90)
11 (55) 15 (75) 17 (85)
9 (22.5) 17 (42.5) 20.5 (51.3)
23 (57.5) 25 (62.5) 28 (70)
17.5 (43.8) 24 (60) 27 (67.5)
TSC 1: task specific checklist score prior to practical instruction, TSC 2: task specific checklist score directly after practical instruction, TSC 3: task specific checklist at six months follow-up, OP 1: global rating score prior to practical instruction, OP 2: global rating score directly after practical instruction, OP 3: global rating score at six months follow-up.
improvement was noted in all course participants following practical instruction. At six months follow-up, an improvement was also noted in all participants compared to their baseline (pre-course). However, the majority (80%, n Z 20) had deteriorated at six months follow-up compared to the standard set immediately after the course. The rate of educational decay over this period was only 10%, reflecting a satisfactory level of skill retention. Finally, in addition to suturing, numerous other aspects of minor surgery were taught during the course. Only one of these skills, the vertical mattress suture, was selected for objective assessment. We felt that the complexity of this task would allow for comprehensive assessment of surgical competence in GPs, in particular through global rating scores, which measure overall surgical proficiency and are applicable to all surgical procedures. Skin surgery courses instruct on the most important aspects of minor surgery in general practice. Through objective assessment of a standardised suture task, we have shown that plastic surgery training is immediately efficacious in improving the technical proficiency of GPs and is associated with a low rate of educational decay. The introduction of a structured minor surgical training programme into the existing general practice curriculum, would further reduce the rate of educational decay, in addition to reducing hospital waiting lists for minor surgical procedures.
Conflicts of interest None.
Funding None.
Ethical approval Not required.
References 1. Brown JS, Smith RR, Cantor T, et al. General practitioners as providers of minor surgery e a success story? Br J Gen Pract 1997;47:205e10.
2. Menon NK. Minor surgery in general practice. Practitioner 1986;230:917e9. 3. O’Cathain A, Brazier JE, Milner PC, et al. Cost effectiveness of minor surgery in general practice: a prospective comparison with hospital practice. Br J Gen Pract 1992;42:13e7. 4. Xeroulis GJ, Park J, Moulton CA, et al. Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 2007;141: 442e9. 5. Khan MS, Bann SD, Darzi A, et al. Use of suturing as a measure of technical competence. Ann Plast Surg 2003;50:304e8. discussion 08e9. 6. Speer DP. The influence of suture technique on early wound healing. J Surg Res 1979;27:385e91. 7. Taylor B, Bayat A. Basic plastic surgery techniques and principles: how to suture. studentBMJ 2003;11:182e4. 8. Mackay S, Datta V, Chang A, et al. Multiple objective measures of skill (MOMS): a new approach to the assessment of technical ability in surgical trainees. Ann Surg 2003;238:291e300. 9. Moulton CA, Dubrowski A, Macrae H, et al. Teaching surgical skills: what kind of practice makes perfect? A randomized, controlled trial. Ann Surg 2006;244:400e9. 10. Bowling J, Botting J. Porcine sebaceous cyst model: an inexpensive, reproducible skin surgery simulator. Dermatol Surg 2005;31:953e6. 11. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273e8. 12. Broe D, Ridgway PF, Johnson S, et al. Construct validation of a novel hybrid surgical simulator. Surg Endosc 2006;20:900e4. 13. Ridgway PF, Ziprin P, Datta VK, et al. Laboratory-based validation of a novel suture technique for wound closure. Ann Plast Surg 2002;49:291e6. 14. Association GPCatBM. Minor surgery in general practice: Guidance from the General Practitioners Committee and the Royal College of General Practitioners in association with the Royal College of Surgeons of England, the Royal College of Surgeons in Edinburgh, British Society for Dermatological Surgery and the Joint Committee on Postgraduate Training for General Practice 2001. 15. Thompson AM, Park KG, Kelly DR, et al. Training for minor surgery in general practice: is it adequate? J R Coll Surg Edinb 1997;42:89e91. 16. Pringle M, Hasler J, De Marco P. Training for minor surgery in general practice during preregistration surgical posts. BMJ 1991;302:830e2. 17. Friedlich M, MacRae H, Oandasan I, et al. Structured assessment of minor surgical skills (SAMSS) for family medicine residents. Acad Med 2001;76:1241e6. 18. Khan MS, Bann SD, Darzi AW, et al. Assessing surgical skill using bench station models. Plast Reconstr Surg 2007;120:793e800.
1162 19. Seki S. Techniques for better suturing. Br J Surg 1988;75:1181e4. 20. Gmajnic R, Pribic S, Lukic A, et al. Effect of surgical training course on performance of minor surgical procedures in family medicine physicians’ offices: an observational study. Croat Med J 2008;49:358e63.
A.M. Collins et al. 21. Maguire N. Effect of a skills programme on minor surgical workload in general practice. Ir Med J 2000;93:136e8. 22. Gordon MJ. A review of the validity and accuracy of selfassessments in health professions training. Acad Med 1991;66: 762e9.
Surgical instruction for general practitioners: How, who and how often?* Anne M. Collins a, Paul F. Ridgway b,*, Mohammed S.U. Hassan c, Christy W.K. Chou d, Arnold D. Hill a, Brian Kneafsey e a
Department of Surgery, Royal College of Surgeons in Ireland, Dublin, Ireland Department of Surgical Oncology, Princess Margaret Hospital, University Health Network, University of Toronto, 3-130, 610 University Avenue, Toronto, Ontario M5G 2M9, Canada c Department of Plastic Surgery, University Hospital, Birmingham, UK d Department of Plastic Surgery, University Hospital of North Durham, UK e Department of Plastic Surgery, Beaumont Hospital, Dublin, Ireland b
Received 9 February 2009; accepted 11 May 2009
KEYWORDS Construct validity; Minor surgery; Plastic surgeon; Continuing medical education; General practitioner
Summary Educational programmes, designed to meet the training needs of General Practitioners (GPs) performing minor surgical procedures, have previously been shown to increase their surgical workload. The change in the level of competence following these programmes has not been assessed. The aims of this study were two-fold: to evaluate the vertical mattress suture for construct validity and to assess the impact of plastic surgery training on the surgical skill of GPs. Thirty non-consultant hospital doctors and 27 self-selected GPs were included. Using a modified objective structured assessment of technical skills (OSATS) scoring system, construct validity of the vertical mattress suture was confirmed. The median total OSATS score was 16 points (26.7%) in the novice group (medical registrars), 38.5 points (64.2%) in the intermediate group (surgical SHOs) and 59 points (98.3%) in the expert group (surgical registrars, p < 0.001, KruskaleWallis test). Objective assessment in the GP group immediately following practical instruction revealed a median overall improvement of 31.7% (19 points) in total OSATS scores (p < 0.001, Friedman non-parametric test, F). At six months follow-up all course participants had improved compared to their baseline. A median overall improvement of 13 points (21.7%) was noted (p < 0.001, F). However, the majority (80%, n Z 20) had deteriorated from the standard set immediately after the course with a median overall reduction in total OSATS scores of six points (10%, p Z 0.001, F).
* Meetings at which this work was presented: This paper was presented at the Winter meeting of the British Association of Plastic, Reconstructive and Aesthetic Surgeons in the Royal College of Surgeons in London and was awarded the Kilner prize for the best overall paper. * Corresponding author. Tel.: þ1 (416) 946 4501. E-mail address: [email protected] (P.F. Ridgway).
1748-6815/$ - see front matter ª 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.bjps.2009.05.023
Surgical instruction for general practitioners
1157
Plastic surgery training is immediately efficacious in improving the technical proficiency of GPs. Through objective assessment of a standardised suture task we demonstrated a low rate of educational decay of 10% over a six-month period. ª 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved.
General practitioners (GPs) perform a range of minor surgical procedures.1 In general, these include any procedures carried out under local anaesthetic that are easy to perform, involve easily accessible structures, are of short duration and are not usually associated with significant post-operative complications.2 The waiting time for GPs performing such procedures is short. They are carried out at a time convenient for patients, in a less intimidating environment than the hospital, and are associated with high levels of patient satisfaction. Such activity has been shown to be cost effective and reduces hospital waiting lists for minor surgery.3 Formal surgical training is not a component of either the Irish College of General Practitioners or the Royal College of General Practitioners core curriculum. The performance of minor surgical procedures is regarded as an area of special interest for individual GPs. Such skills are considered relevant but not core to general practice. The acquisition of surgical skills by GPs is, therefore, largely unstructured and no formal appraisial system for technical proficiency exists. It is the responsibility of the individual GP to identify their own strengths and weaknesses and obtain additional training should they so require. Basic skin surgery courses afford an opportunity to do so. Similar to surgical skills laboratories, such courses are associated with a less stressful learning environment when compared to the operating room. They allow for repetitive and deliberate practice, participatory rather than observational learning and the identification and correction of basic performance flaws.4 It is well recognised that surgical competence encompasses many factors as well as technical proficiency.5 Correct wound closure, however, is an essential skill and is particularly important in the setting of general practice. Poor suturing technique and excessive suture tension are directly related to adverse wound healing and increased scarring.6 One of the most common methods for wound closure is the simple interrupted suture. The vertical mattress suture is more complex requiring a second superficial bite in the vertical plane to evert the skin edges.7 By using objective assessment of a suture task, our aims were to evaluate the immediate impact of plastic surgery training on the surgical skill of GPs and the durability of skill acquisition. The selected suture task should demonstrate construct validity. This is the extent to which a task differentiates between different experience levels. A task is deemed construct valid if a more experienced trainee consistently performs better than a less experienced trainee. Too simple a task may bias against determining construct validity. The vertical mattress suture, for example, has previously been shown to have a broader spread of performance than the simple suture and the potential to discriminate between individuals in an expert group.8
A common limitation of continuing medical education programs, such as one-day skin surgery courses, is the time delay between when a skill is learned and when it is next performed. Although technical proficiency may improve immediately following training, how much is retained is uncertain. The ultimate value of skills courses should therefore be measured not by performance immediately after training but by performance after a time delay.9
Methods This was a prospective cohort design study with two components: I. Evaluation of the vertical mattress suture for construct validity. II. Assessment of the impact of plastic surgery training on the surgical skill of GPs.
Evaluation of the vertical mattress suture for construct validity In this component of the study, ten medical registrars with no previous surgical training (novice surgical but medically trained group), ten surgical senior house officers (SHOs; intermediate group) and ten surgical registrars (experts) from Beaumont hospital, Dublin, Ireland were selected randomly to participate. Pigskin is widely accepted as a suitable model for human skin10 and was used in this study. A 4 0.5 cm wound was made and participants were instructed to treat it as a wound on the arm. Plastic surgery instruments and a range of suture material [3-0 vicryl, 3-0 nylon, 3-0 prolene (Ethicon) and 3-0 novafil (Tyco Healthcare)] was provided (Figure 1). Participants were instructed to insert a single vertical mattress suture in the centre of the wound using the instruments and suture they felt were most appropriate for the task. No antecedent training was provided to reduce familiarty bias. If participants were unfamiliar with the vertical mattress suture, brief verbal instruction was provided. All participants were scored in a blinded fashion by a single surgeon (AMC) using a modified Objective Structured Assessment of Technical Skills (OSATS) scoring system. OSATS was devised in 1997 and was initially validated for bench model simulation.11 It has also been used by our group to validate a hybrid surgical stimulator12 and to complement metric assessment used in suturing studies.13 OSATS is comprised of two individual scores: a task specific checklist score and an overall global rating score of operative performance. A twenty-point task specific
1158
Figure 1 task.
A.M. Collins et al.
Setup for assessment of vertical mattress suture
checklist was devised for the vertical mattress suture by identifying its integral components, including appropriate knowledge of the task, correct choice of instruments and suture material, correct position of the needle in the needle holder and competent instrument handling. In addition the needle must enter the skin at a right angle, exit at a point equidistant from the insertion point for both bites and be reverse mounted for the second superficial bite. The skin should be approximated with the appropriate level of tension with a square knot and meticulous tissue handling should be evident throughout the task. Participants scored one point for each item correctly performed on the checklist, giving a maximum score of 20. The global rating score of operative performance measures overall ability to perform the procedure. It assesses general surgical competence and has a broad applicability. Eight parameters were assessed including respect for tissue, time and motion, familiarity with and handling of instruments, flow and knowledge of the procedure, overall performance and the quality of the final product. Each parameter was graded on a 5-point Likert scale (Likert, 1932) from zero (very poor) to five (clearly superior performance), yielding a maximum score of 40.
Assessment of the impact of plastic surgery training on the surgical skill of GPs The second part of the study involved 27 self-selected GPs who were evaluated in the setting of a skin surgery course in Dublin in December 2007. The course faculty included an associate specialist in plastic surgery, a consultant dermatologist, a senior plastic surgery specialist registrar and two plastic surgery senior house officers. Prior to practical assessment all participating GPs completed a questionnaire documenting their experience level, previous surgical training and how frequently minor procedures are performed in their practice. In addition, the GPs were questioned on the different types of mattress sutures, indications for use and whether they are used in their practices. Each GP was then asked to perform the same vertical mattress suture task as described in the construct validity
component of the study and was scored in the same manner by the same surgeon (AMC). Participants subsequently completed an intensive skin surgery course and were instructed on a 5.4:1 ratio. The course faculty continually rotated between the groups. In total, 300 min of practical instruction were provided incorporating instrument and tissue handling, suture selection, biopsy and excision of lesions and simple, mattress and subcuticular suturing. 150 min were dedicated to suturing technique. At the conclusion of the course, participants were asked to repeat the mattress suture task and were scored in the same manner by the same surgeon as described previously. A practice kit [disposable suture packs (Vernon-Carus Ltd, Lancashire) and sutures (3-0 nylon, Ethicon)] was also offered to participants. Six months later the course participants were invited to participate in a follow-up study. Once again they were questioned on the different types of mattress suture and indications for their use. In addition it was noted whether they had performed mattress sutures in the preceding six months, and whether they had availed of or used the suture pack offered to them. A study-specific questionnaire was also completed and scored on a 5-point Likert scale. The perceived improvement in surgical skill, level of satisfaction with the course, the likelihood of recommending the course to a colleague and of benefitting from repeating it was documented. Finally the mattress suture task was reassessed and participants were scored in the same manner by the same surgeon (AMC) who performed the other assessments. All results were recorded on a database.
Statistics Data was tested for normality using ShapiroeWilk tests. Non-parametric analysis was used. KruskaleWallis test (KWT) was performed to evaluate for construct validity of the vertical mattress suture. ManneWhitney U (MWU), Wilcoxon Signed Ranks (WSR) and Friedman tests (F) were used to compare GP scores at baseline, immediately postcourse and six months later. Correlations between the results of the tasks and previous operative experience were made using Spearman’s non-parametric correlation. p Values, where appropriate, were taken to be significant at 0.05. All statistics were calculated using the Statistical Package for Social Sciences (SPSS, version16 Chicago, IL).
Results Evaluation of the vertical mattress suture for construct validity Construct validity of the vertical mattress suture was confirmed. The novice group (medical registrars) had a median overall score of 16 points (26.7%), the intermediate group (surgical SHOs) had a median score of 38.5 points (64.2%) and the expert group (surgical registrars) had a median score of 59 points (98.3%, p < 0.001, KWT, Figure 2). When taken individually, the median checklist and global rating scores correlated with an increasing level of experience (p < 0.001, KWT, Table 1).
Surgical instruction for general practitioners
1159 Table 1 Evaluation of construct validity of the vertical mattress suture. Median scores for medical registrars (novice group), surgical SHOs (intermediate group) and surgical registrars (expert group)
Novice Intermediate Expert p (KWT)
TSC (%)
OP (%)
Overall (%)
5.5 (27.5) 14.5 (72.5) 19 (95) 0.000002*
11 (27.5) 24.5 (61.3) 40 (100) 0.000002*
16 (26.7) 38.5 (64.2) 59 (98.3) 0.000002*
TSC: task specific checklist score, OP: global rating score of operative performance, Overall: total OSATS score (TSC þ OP). *Significance is at the 0.05 level.
Figure 2 Boxplot of construct validity of vertical mattress suture. Total scores for novice, intermediate and expert groups. Maximum score was 60. The heavy line is the median. The bars represent the range of the data.
Assessment of the impact of plastic surgery training on the surgical skill of GPs Twenty-seven GPs enrolled in the course. Two were not available to participate in the follow-up study and were excluded from statistical analysis. The median experience level was ten years. The majority of participants (72%, n Z 18) perform minor surgical procedures regularly but only 12% (n Z 3) had previously received formal surgical training. Although 88% (n Z 22) had heard of mattress sutures, only 32% (n Z 8) had previously experience performing them, 16% (n Z 4) were familiar with the indications for their use and 8% (n Z 2) were aware of the different types. Assessment immediately following practical instruction revealed an improvement in all participants compared to baseline. The median overall improvement in total OSATS scores was 19 points, which is equivalent to an increase of 31.7% (p < 0.001, F). The median percentage improvement in task specific checklist scores was 45% (p < 0.001) and the median percentage improvement in global rating scores was 20% (p < 0.001, WSR). The follow-up rate was 93%. Ninety-six percent (n Z 24) recalled the different types of mattress sutures and 72% (n Z 18) were aware of the indications for their use. Forty percent (n Z 10) had used mattress sutures in the preceding six months. Of the 80% (n Z 20) that availed of the suture pack, only 20% (n Z 4) had used it to practice their technique. By comparing the baseline (pre-course) total OSATS scores with those at six months follow-up, a median overall improvement of 13 points (21.7%) was noted (p < 0.001, F, Table 2, Figure 3). The median percentage improvement in task specific checklist and global rating scores over the same time period was 35% and 17.5% respectively (p < 0.001, p < 0.001, WSR, Table 3). At six months follow-up all course participants had improved technical proficiency compared to their baseline. However, the majority (80%, n Z 20) had deteriorated below the standard they set immediately after the course. The median overall reduction in total OSATS scores was six
points (10%, p Z 0.001, F, Table 2). The median percentage reduction in task specific checklist and global rating scores were 10% and 2.5% respectively (p < 0.001, p Z 0.005, WSR, Table 3). Of the 80% (n Z 20) of GPs whose proficiency was below the standard they set immediately after practical instruction, 70% had performed minor procedures regularly over the preceding six months but only 25% had used mattress sutures. With regard to level of confidence in performing minor procedures, 45% (n Z 9) acknowledged they had a lower level of confidence compared to immediately postcourse. The median reduction in total OSATS score in this group was ten points (16.7%) The remaining 55% (n Z 11) felt they had the same level of confidence at six months follow-up. The median deterioration in this group was five points (8.3%). The difference between these two groups was not statistically significant (p Z 0.15, MWU). A minority of GPs (20%, n Z 5) improved on their postcourse total OSATS scores at six months follow-up, with a median improvement of four points (6.7%). All of this cohort performed minor procedures regularly and had used mattress sutures over the preceding six months. Moreover, there was a significant correlation between use of mattress sutures in the six months following the course and the task specific checklist score at the follow-up visit. Spearman’s correlation coefficient was 0.65 (p Z 0.008). There was a high level of satisfaction among participants with the training received at the skin surgery course. Ninety-two percent (n Z 23) felt their surgical skills improved and 84% (n Z 21) felt more confident in dealing with the surgical cases encountered in their practice. 96% (n Z 24) felt they would benefit from taking the course again and all participants would recommend the course to a colleague.
Discussion The majority of GPs performing minor procedures do not have any formal surgical training. In order to obtain a place on the Health Authority minor surgery list in the United Kingdom, GPs must attend a course approved by regional advisors in general practice and three practical sessions covering the necessary range of procedures with approved teachers.14 Despite this, a high level of dissatisfaction with teaching in minor surgery exists.15 GPs are often unwilling to excise complex lesions such as sebaceous cysts16 and
1160 Table 2
Mean Median p (F)
A.M. Collins et al. Comparison of total OSATS scores pre-course, post-course and at six months follow-up Pre-course OSATS (%)
Post-course OSATS (%)
Six-month OSATS (%)
Change: pre- to post-course (%)
Change: pre-course to six months (%)
Change: post-course to six months (%)
23.9 (39.9) 25 (41.7)
41.8 (69.7) 44 (73.3)
36 (60) 38 (63.33)
17.9 (29.8) 19 (31.6) 0.0000000008*
12.1 (20.2) 13 (21.7) 0.00000006*
5.8 (9.7) 6 (10) 0.001*
F Z Friedman non-parametric test. *Significance is at the 0.05 level.
lesions on the face.15 A structured training programme would improve the technical proficiency of GPs, thus allowing more GPs tackle a wider range of cases. Structured assessment has been shown to function effectively as a method of formally assessing the surgical skills of family medicine residents,17 and of general surgery8 and plastic surgery trainees.18 This type of assessment allowed us to identify and correct deficiencies in suturing performance and ensure basic principals were followed by participating GPs. Instruction was then provided in specific areas, and the resultant improvement in suturing competence was demonstrated. Such instruction has been shown to decrease the frequency of technical errors and the types of errors made.5 Although simple interrupted sutures are more commonly used, the more complex vertical mattress suture was selected as the task for the purposes of evaluation. The increased complexity of the latter allows for more extensive assessment of surgical competence. Interobserver relaibility was not specifically tested in our construct as it has been tested and found to be acceptable using a similar OSATS model. 8 Our findings demonstrated a wider spread of performance in the novice and intermediate groups than in the expert group. Others have suggested the mattress suture has the potential to discriminate between individuals in an expert group.8 These conclusions were not investigated specifically by our study design.
Figure 3 Boxplot of total OSATS scores (in percentages) of GPs pre-course, immediately post-course and at six months follow-up. Maximum score was 100 (%). The heavy line is the median. The bars represent the range of the data.
It has previously been shown that less experienced surgeons benefit more from instruction than more experienced ones.19 We therefore postulated that a short period of intense instruction by plastic surgeons would improve the level of surgical proficiency of a group of GPs, of whom only 12% had previously received formal surgical training. Other educational programmes, designed to meet the training needs of GPs performing minor procedures, have been shown to increase their surgical workload, but the change in the level of competency following these programmes was not assessed.20,21 Through assesment of a standardised suture task, we have shown that participation at a one-day skin surgery course increases the level of competence among GPs by 31.7% immediately and by 21.7% at six months follow-up. The one-day skin surgery course evaluated in this study is typical of what is offered in the majority of continuing medical education programmes; short and intensive. It has recently been suggested that practice interspersed with periods of rest (distributed) is superior to the massed method of teaching that predominates in skills courses (practice delivered in a continuous block with little or no rest).9 Therefore, a structured minor surgery training programme delivered over a period of weeks is desirable, in order to maintain the level of proficiency obtained as a result of additional training and to reduce the rate of educational decay. With regard to self-assessment, our findings correlate with other reports suggesting that confidence in performing a procedure, while important, does not necessarily correlate with ability.22 56% (n Z 14) of GPs assessed claimed to have the same level of confidence six months later as they did immediately after the course. The majority of this cohorts’ (78.6%, n Z 11) proficiency had deteriorated over the six-month follow-up period. In order to facilitate practicing of the skills learned on the course, each participant was offered a practice kit, consisting of a disposable suture set and sutures, at the conclusion of the course. The majority (80%, n Z 20) availed of the pack, the remainder cited possession of their own instruments as the reason for declining. However only 20% (n Z 4) used the pack provided, thus confirming this intervention as unsatisfactory. The limitations of this study include the small sample size. The sample was also biased, as this group of GPs were self-selected and therefore motivated. However, we feel that this self-funded, enthusiastic group is ideal to study. In theory they should benefit most from practical instruction and demonstrate high levels of retention. Through objective assessment of a suture task, an
Surgical instruction for general practitioners Table 3
1161
Comparison of pre-course, post-course and six month follow-up checklist and global rating scores
Mean Median Percentiles 25 50 75
TSC 1 (%)
TSC 2 (%)
TSC 3 (%)
OP 1 (%)
OP 2 (%)
OP 3 (%)
8.1 (40.6) 8 (40)
16.5 (82.4) 17 (85)
13.7 (68.5) 15 (75)
15.8 (39.5) 17 (42.5)
25.4 (63.4) 25 (62.5)
22.2 (55.5) 24 (60)
6 (30) 8 (40) 11 (55)
15 (75) 17 (85) 18 (90)
11 (55) 15 (75) 17 (85)
9 (22.5) 17 (42.5) 20.5 (51.3)
23 (57.5) 25 (62.5) 28 (70)
17.5 (43.8) 24 (60) 27 (67.5)
TSC 1: task specific checklist score prior to practical instruction, TSC 2: task specific checklist score directly after practical instruction, TSC 3: task specific checklist at six months follow-up, OP 1: global rating score prior to practical instruction, OP 2: global rating score directly after practical instruction, OP 3: global rating score at six months follow-up.
improvement was noted in all course participants following practical instruction. At six months follow-up, an improvement was also noted in all participants compared to their baseline (pre-course). However, the majority (80%, n Z 20) had deteriorated at six months follow-up compared to the standard set immediately after the course. The rate of educational decay over this period was only 10%, reflecting a satisfactory level of skill retention. Finally, in addition to suturing, numerous other aspects of minor surgery were taught during the course. Only one of these skills, the vertical mattress suture, was selected for objective assessment. We felt that the complexity of this task would allow for comprehensive assessment of surgical competence in GPs, in particular through global rating scores, which measure overall surgical proficiency and are applicable to all surgical procedures. Skin surgery courses instruct on the most important aspects of minor surgery in general practice. Through objective assessment of a standardised suture task, we have shown that plastic surgery training is immediately efficacious in improving the technical proficiency of GPs and is associated with a low rate of educational decay. The introduction of a structured minor surgical training programme into the existing general practice curriculum, would further reduce the rate of educational decay, in addition to reducing hospital waiting lists for minor surgical procedures.
Conflicts of interest None.
Funding None.
Ethical approval Not required.
References 1. Brown JS, Smith RR, Cantor T, et al. General practitioners as providers of minor surgery e a success story? Br J Gen Pract 1997;47:205e10.
2. Menon NK. Minor surgery in general practice. Practitioner 1986;230:917e9. 3. O’Cathain A, Brazier JE, Milner PC, et al. Cost effectiveness of minor surgery in general practice: a prospective comparison with hospital practice. Br J Gen Pract 1992;42:13e7. 4. Xeroulis GJ, Park J, Moulton CA, et al. Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 2007;141: 442e9. 5. Khan MS, Bann SD, Darzi A, et al. Use of suturing as a measure of technical competence. Ann Plast Surg 2003;50:304e8. discussion 08e9. 6. Speer DP. The influence of suture technique on early wound healing. J Surg Res 1979;27:385e91. 7. Taylor B, Bayat A. Basic plastic surgery techniques and principles: how to suture. studentBMJ 2003;11:182e4. 8. Mackay S, Datta V, Chang A, et al. Multiple objective measures of skill (MOMS): a new approach to the assessment of technical ability in surgical trainees. Ann Surg 2003;238:291e300. 9. Moulton CA, Dubrowski A, Macrae H, et al. Teaching surgical skills: what kind of practice makes perfect? A randomized, controlled trial. Ann Surg 2006;244:400e9. 10. Bowling J, Botting J. Porcine sebaceous cyst model: an inexpensive, reproducible skin surgery simulator. Dermatol Surg 2005;31:953e6. 11. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273e8. 12. Broe D, Ridgway PF, Johnson S, et al. Construct validation of a novel hybrid surgical simulator. Surg Endosc 2006;20:900e4. 13. Ridgway PF, Ziprin P, Datta VK, et al. Laboratory-based validation of a novel suture technique for wound closure. Ann Plast Surg 2002;49:291e6. 14. Association GPCatBM. Minor surgery in general practice: Guidance from the General Practitioners Committee and the Royal College of General Practitioners in association with the Royal College of Surgeons of England, the Royal College of Surgeons in Edinburgh, British Society for Dermatological Surgery and the Joint Committee on Postgraduate Training for General Practice 2001. 15. Thompson AM, Park KG, Kelly DR, et al. Training for minor surgery in general practice: is it adequate? J R Coll Surg Edinb 1997;42:89e91. 16. Pringle M, Hasler J, De Marco P. Training for minor surgery in general practice during preregistration surgical posts. BMJ 1991;302:830e2. 17. Friedlich M, MacRae H, Oandasan I, et al. Structured assessment of minor surgical skills (SAMSS) for family medicine residents. Acad Med 2001;76:1241e6. 18. Khan MS, Bann SD, Darzi AW, et al. Assessing surgical skill using bench station models. Plast Reconstr Surg 2007;120:793e800.
1162 19. Seki S. Techniques for better suturing. Br J Surg 1988;75:1181e4. 20. Gmajnic R, Pribic S, Lukic A, et al. Effect of surgical training course on performance of minor surgical procedures in family medicine physicians’ offices: an observational study. Croat Med J 2008;49:358e63.
A.M. Collins et al. 21. Maguire N. Effect of a skills programme on minor surgical workload in general practice. Ir Med J 2000;93:136e8. 22. Gordon MJ. A review of the validity and accuracy of selfassessments in health professions training. Acad Med 1991;66: 762e9.