- Email: [email protected]
Predictive validity of the medical specialty preference inventory
Journal of Vocational Behavior 74 (2009) 128–133
Contents lists available at ScienceDirect
Journal of Vocational Behavior journal homepage: www.elsevier.com/locate/jvb
Predictive validity of the medical specialty preference inventory Kevin W. Glavin a,*, George V. Richard b, Erik J. Porfeli a a b
Northeastern Ohio Universities College of Medicine and Pharmacy, Department of Behavioral Sciences, 4209 St. Rt. 44, P.O. Box 95, Rootstown, OH 44272, USA Association of American Medical Colleges, 2450 N Street, N.W., Washington, DC 20037, USA
a r t i c l e
i n f o
Article history: Received 9 November 2008 Available online 6 December 2008
Keywords: Predictive validity Medical specialty preference inventory Medical interests MSPI Medical specialty choice Medical students
a b s t r a c t Medical schools can assist students by providing them with quality career counseling to help them choose a medical specialty. Many schools use interest inventories to help identify students’ specialty interests. This study examined the predictive validity of one such inventory, the Medical Specialty Preference Inventory (MSPI). In a longitudinal design, we used discriminant function analysis to examine how well students’ scores on the MSPI fit their chosen medical specialty one year later. The MSPI correctly predicted students’ future medical specialty choice 58.1% of the time. These results can help career advisors interpret MSPI scores, and identify students’ most likely medical specialty choice, as well as their second most likely choice. Ó 2008 Elsevier Inc. All rights reserved.
1. Introduction Medical students’ specialty choice constitutes an important personal decision with far reaching consequences for individuals, and their families. Obtaining a medical degree requires the student to invest significant personal and economic resourses, and to delay transitions to work and family roles. State governments invest enormous resources to subsidize medical education at state institutions, and students mortgage their future to pay for the high costs of a medical education. Consequently, medical care providers, medical school faculty and staff, and students’ families encourage medical students to think proactively and carefully about their future medical specialty choice. Medical students choose a specialty for a variety of reasons, including experiences and exposure in medical school, academic performance in relevant clinical clerkships, personality attributes, and ratings of the content of medical practice (Reed, Jernstedt, & Reber, 2001). Unfortunately, students must often make this life-changing choice with inadequate information. Medical specialty counseling has developed over time to assist students in making specialty choices. Some career advisors use the Medical Specialty Preference Inventory (MSPI: Zimny, 1979) to facilitate the decision-making process. The MSPI is an assessment of students’ specialty interests and is used to inform their choice of medical specialty. In addition, the instrument can be used to assist physicians who are considering a medical specialty change at some point during residency. The original MSPI consisted of 199 items pertaining to medical activities and settings. Zimny (1979) asked a sample of physicians in each of six major medical specialties (Family Medicine, Internal Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry, and Surgery) to rate the extent to which each item from the MSPI was characteristic of general practice in their specialties. This method avoided the inherent problem in using generic interests, which may co-vary across specialties, and therefore, fail to distinguish clearly between them. The authors considered this approach to be favorable because it relates the characteristics of the practice of the specialties themselves rather than the characteristics of the physicians who
* Corresponding author. E-mail address: [email protected] (K.W. Glavin). 0001-8791/$ - see front matter Ó 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jvb.2008.11.004
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
129
practice in those specialties. Subsequently, it compares the medical activities that students prefer to the activities that physician’s in that specialty perform. This approach has received support from the literature. Savickas, Brizzi, Brisbin, and Pethtel (1988) compared the predictive validity of the MSPI with another medical specialty preference inventory, the Medical Specialty Preference Scales (MSPS: Gough, 1979). Whereas the MSPI uses practitioner-based scales and contains items related to actual medical practice, the MSPS uses student-based scales and contains generic interest items. The results from two consecutive medical student cohorts found that the MSPI successfully predicted a student’s specialty 59% of the time, whereas the MSPS successfully predicted a student’s specialty 19% of the time. Furthermore, the prediction rate for the MSPS only slightly exceeded what one would expect to see by chance alone. These results suggest that practitioner-based scales containing items related to specific medical practice predict students’ medical specialty choice better than student-based scales containing generic interest items. The current study employed the MSPI 2nd edition, which Zimny revised and updated in 2002 (Zimny, 2002). This revision brought about a number of important changes including a reduction in the number of items and subscales. Although previous research demonstrated the strong predictive validity of the original version of the MSPI, the predictive power of the updated version has not been tested. Therefore, this study investigated the predictive validity of the MSPI 2nd edition. 2. Methodology 2.1. Participants The participants were 506 medical students who completed the MSPI on the Association of American Medical Colleges’ Careers in Medicine (CiM) website, which can be found at http://www.aamc.org/students/cim/. Careers in Medicine is a comprehensive career planning program available to all US and Canadian medical schools. To gain access to the site students must obtain a free token from their school liaison, usually the school’s associate or assistant dean of student affairs. At registration users are presented with an IRB-approved informed consent statement that indicates any data stored in the CiM site may be used for research. While most schools actively provide access to the program, there are small number of schools that do not participate as actively as others. Registered users of the site represent approximately 55–65% of all enrolled medical students in US and Canadian medical schools. Once registered, students are free to use all of the confidential resources available on the site. Use of this site is voluntary as is the completion of the MSPI. The data were gathered from January 2005 to December 2006, while the respondents were in their final year of medical school, and contained 190 male students, and 316 female students. Respondents identified their race as follows: Caucasian (67%), Asian American/Pacific Islander (12%), African American (9%), Indian American (7%), Native American (1%), Other (2%), and Unknown (2%). Upon completion of medical school, and approximately one year later, students entered residency training for their chosen specialty. The time between completing the MSPI and reporting residency choice was approximately one year. Information about specialty choice was obtained from the AAMC’s GME Track system containing resident census information for all training programs in the US. Those who were specified as active residents in one of six medical specialties (Family Medicine, Internal Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry, and Surgery) were selected for the purposes of this study. 2.2. Instrument The MSPI measures medical interests, and is used to assist students in choosing a specialty. A 2002 revision of the MSPI resulted in a reduction in the number of items to 150. Of the 150 items, 104 are used for scoring purposes. The remaining 46 provide additional data intended for use in constructing future scales. MSPI items reflect job specific tasks that relate directly to medical practice. Students rate each item on a seven-point scale which reflects their degree of desirability for each item. A score of 1–2 indicates low desirability, 3–5 indicates moderate desirability, and 6–7 indicates high desirability. Differences between students’ subscale scores and specialists’ scores determine students’ preference for each medical specialty. Students whose subscale scores are similar to specialist scores for the same subscales will receive a high preference score for that particular specialty. Scores are calculated for: Family Medicine (FAM), Obstetrics Gynecology (OBGYN), Surgery, Psychiatry (PSY), Pediatrics (PED), and Internal Medicine (MED). Higher scores indicate a greater preference for a specialty, while lower scores indicate less preference for a specialty. The instrument was self-administered on the internet, and preference scores were reported to participants immediately upon completion. Zimny (1979) reported on the reliability and validity indices used in the initial development of the MSPI. Reliability was estimated in two separate analyses. In the first analysis, using the Spearman Brown formula, reliability estimates ranged from .74 (Pediatrics) to .93 (Family Medicine), and in the second analysis estimates ranged from .66 (Pediatrics) to .91 (Surgery). Most reliability estimates in both analyses fell in the .80s and .90s. Zimny also conducted a study on the predictive validity of the MSPI using National Resident Matching Program (NRMP) data (1980). The NRMP contains data about the initial specialty choice of medical students as they enter into residency training. He found that the level of predictive accuracy over all specialties ranged from about 50 to 55%. This range represents a level of prediction well above the conservative chance expectancy level of 17% accuracy, and indicates that a substantial relationship exists between specialty preference scores on the MSPI and subsequent specialty choice.
130
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
To test the predictive validity of the MSPI, scores for each of the six medical specialties were compared to students’ residency choices. Six v2-tests were conducted to test the predictive validity of the six MSPI scale scores. The general hypothesis was that students’ pattern of scores across the six MSPI scores would be predictive of their medical specialty choice one year later. Students were classified into three MSPI groups for each of the six specialties. Group one included those students whose score for a particular specialty was their highest score relative to the other five specialties, and this score was at least 3.5 points greater than their next highest scale score. The researchers used 3.5 points because this represented approximately half of the average difference between students’ highest and second highest scores. Group two included those students whose score for a particular specialty was their highest score relative to the other five specialties, but this score was not at least 3.5 points greater than their next highest scale score. Group three included the remaining students. The aim was to discern if students choose, for example, family medicine for their residency with greater frequency if they scored highest on the family medicine subscale of the MSPI, and if the frequency of this specialty choice increased as this score became more pronounced (by more than 3.5 points) relative to the other five scale scores. 3. Results Results showed that the overall hit rate (i.e., rate of correctly classified students based upon their interests and specialty choice), for groups one and two combined, was 46%. The overall hit rates by medical specialty were as follows; Family Medicine 41%, OBGYN 53%, Pediatrics 46%, Psychiatry 56%, Internal Medicine 27%, Surgery-General 88%. Of the participants who chose Family Medicine as their medical specialty, 27.7% of them met both criteria (i.e., group 1), 13.3% met the second criteria only (i.e., group 2), and the remainder, 59%, met neither criteria (i.e., group 3), v2 (2, N = 506) = 81.09 p < 0.001. Of the participants who chose OBGYN as their medical specialty, 44.2% of them met both criteria, 9.3% met the second criteria only, and the remainder, 46.5%, met neither criteria, v2 (2, N = 506) = 82.92 p < 0.001. Of the participants who chose Pediatrics as their medical specialty, 23.2% of them met both criteria, 23.2% met the second criteria only, and the remainder, 53.7%, met neither criteria, v2 (2, N = 506) = 60.9 p < 0.001. Of the participants who chose Psychiatry as their medical specialty, 48.8% of them met both criteria, 7% met the second criteria only, and the remainder, 44.2%, met neither criteria, v2 (2, N = 506) = 172.94 p < 0.001. Of the participants who chose Internal Medicine as their medical specialty, 11.2% of them met both criteria, 15.4% met the second criteria only, and the remainder, 73.4%, met neither criteria, v2 (2, N = 506) = 59.36 p < 0.001. Of the participants who chose Surgery-General as their medical specialty, 78.1% of them met both criteria, 9.6% met the second criteria only, and the remainder, 12.3%, met neither criteria, v2 (2, N = 506) = 117.9 p < 0.001. Although these v2 results are promising, they mainly speak to bivariate associations between categories of MSPI subscale scores and specialty choice. They do not permit a simultaneous comparison of the discriminatory power of the six subscales in combination. Discriminant function analysis was employed to test if, and to what extent, the entire MSPI predicts medical specialty choice and to what extent each of the subscales contributes to its discriminatory power while controlling for the others. Fig. 1 shows the mean scores for each of the subscales for each of the medical specialty choices. Each line represents residents who chose that specialty, and their mean scores on each of the six preference scales. For example, students who chose
75.00
Preference Score
70.00
Medical Specialty Choice
65.00
FAM MED
60.00
OBGYN PED
55.00
PSY SUR
50.00 45.00 FAM
MED
OBGYN
PED
PSY
SUR
Medical Specialty Preference Scales Fig. 1. Mean scores for medical specialty preference scales by specialty choice. *FAM, Family Medicine; MED, Internal Medicine; Pediatrics, Pediatrics, Psychiatry, Psychiatry, SUR, Surgery-General.
131
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
Family Medicine, on average, scored 67.73 on Family Medicine, 65.1 on Internal Medicine, 67.13 on OBGYN, 67.43 on Pediatrics, 55.68 on Psychiatry, and 60.35 on Surgery-General. The most differentiated group profiles occur for those students who selected Psychiatry and Surgery-General specialties. Students who chose Psychiatry tended to score highest on the Psychiatry subscale and lowest on the Surgery-General subscale, relative to the other four subscales, and relative to all the other students in the other medical specialties. The least differentiated profile occurred for Family Medicine. Students choosing OBGYN and Pediatrics had the most elevated scores, while students choosing Psychiatry had the least elevated scores. Medical residents in Family Medicine and Internal Medicine demonstrated mid-range elevation scores, when compared to the residents in other medical specialties. A direct discriminant analysis was performed using six interest variables as predictors of membership in six medical specialty groups. Predictors were scores on Family Medicine, Internal Medicine, OBGYN, Pediatrics, Psychiatry, and SurgeryGeneral MSPI scales. The criterion variable was the medical specialty the participant entered. The researchers used 70% of the original dataset to run the discriminant analysis, and reserved the remaining 30% for cross-validation purposes. No cases were found to be missing data, n = 365. The results yielded five significant discriminant functions, with a combined v2 (30) = 613.29, p < 0.001. Functions two through five yielded a v2 (20) = 364.293, p < 0.001. Functions three through five yielded a v2 (12) = 227.175, p < 0.001. Functions four through five yielded a v2 (6) = 114.957, p < 0.001. Function five yielded a v2 (2) = 34.107, p < 0.001. Table 1 displays bivariate correlations, means, and standard deviations for each of the predictor variables. Strong intercorrelations exist between Family Medicine and Internal Medicine (.81), and OBGYN and Pediatrics (.75). These correlations give cause for concern because they violate one of the assumptions of discriminant function analysis. The effect sizes for the discriminant functions were as follows; .5, .32, .27, .2, and .09, for functions 1, 2, 3, 4, and 5, respectively. Therefore, the five functions accounted for 50%, 32%, 27%, 20%, and 9% of the total relationship between predictors and groups. The amount of between-group variance accounted for by each function was as follows; 45.8%, 21.3%, 16.8%, 11.6%, and 4.6% for functions 1, 2, 3, 4, and 5, respectively. The classification results displayed in Table 2 demonstrate that the MSPI may be used to correctly classify students’ specialty choice 58.1% of the time, which exceeds what we would expect to see by chance alone (i.e., 20%). The results show that the MSPI best predicts participants who chose Surgery-General, with a 74% correct classification rate, followed by psychiatry, with a hit rate of 71%. Although the MSPI’s predictive power is weakest for family medicine, the hit rate (47%) remains well above chance. The pattern of misclassifications for Family Medicine and Pediatrics specialties suggest that students who select these specialties may exhibit similar patterns of interests. The same can be said for Surgery-General and Internal Medicine. The patterns of misclassifications were inconsistent for Psychiatry relative to any other discipline, which suggests that participants who chose this specialty appear to have a relatively unique interest profile. The stability of the classification procedure was checked by a cross-validation run. Approximately 30% of the cases were withheld from calculation of the classification functions in this run. For the 70% of the cases from which the functions were derived, there was a 58% correct classification rate. For the cross-validation cases, classification was 57%. This result suggests there exists a high degree of consistency in the classification scheme, and suggests that the results are not dependent on the sample data used to test the discriminatory power of the MSPI. 4. Discussion This study examined the predictive validity of the MSPI, which demonstrated an overall hit rate of approximately 46% when using a v2 analysis. The highest hit rate occurred for Surgery-General, and the lowest hit rate occurred for Psychiatry. These findings are significantly better than what we would expect to see by chance alone, which was determined to be 20%. Interest inventories based on six categories generally report hit rates between 35 and 45% (Holland, Magoon, & Spokane, 1981). The researchers used discriminant function analysis to move beyond simple bivariate associations to test the simultaneous discriminatory power of the six MSPI scales to predict medical specialty choice. Fig. 1 demonstrates that residents within the six medical specialties typically exhibited unique patterns of preference scores based on the elevation and variability of those scores. Table 2 suggests that in most specialties there exists a secondary, or perhaps tertiary, specialty that is typically chosen by participants who do not choose the predicted specialty on the basis of their interests. One can think of
Table 1 Inter-correlations, means, and standard deviations for medical specialty preference scores.
1. 2. 3. 4. 5. 6. *
Family Internal OBGYN Pediatrics Psychiatry Surgery p < .001.
Mean
SD
1
2
3
4
5
6
60.37 63.38 65.97 65.65 55.11 64.33
11.20 8.81 7.63 7.20 10.34 10.77
1.00
0.81* 1.00
0.48* 0.36* 1.00
0.48* 0.45* 0.75* 1.00
0.35* 0.24* 0.31* 0.47* 1.00
0.00* 0.37* 0.06 0.00 -0.55* 1.00
132
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
Table 2 Classifications by medical specialty. Predicted Medical specialty
Chosen medical specialty
Family Medicine Internal Medicine OBGYN Pediatrics Psychiatry Surgery-General
Family Medicine
Internal Medicine
OBGYN
Pediatrics
Psychiatry
Surgery-General
46.7 8.3 8.6 15.2 17.6 2
10 55.8 8.6 12.1 0 14
8.3 7.5 60 6.1 2.9 6
18.3 12.5 5.7 53 8.8 4
6.7 2.5 0 4.5 70.6 0
10 13.3 17.1 9.1 0 74
Note: Numbers are row percentages and bolded numbers reflect the percent of correct classifications for each medical specialty.
those secondary and tertiary specialties as a typical alternative. For example, while participants who scored highest in Family Medicine chose to enter Family Medicine 46.7% of the time, another 18.3% actually entered Pediatrics. This makes intuitive sense given that both Family Medicine and Pediatrics share some commonalities, such as the breadth of problems they deal with, and in the role they play as primary-care providers. Participants with high OBGYN scores chose OBGYN 60% of the time, but another 17.1% chose to enter General-Surgery. Both of these specialties share commonality in their surgical applications, and so these results make sense also. While most of the specialties seem to have interest companions, psychiatry seems to be fairly distinct with respect to interests. On a larger level, these results hint at an underlying conceptual structure of medical specialties on the basis of interest that could be explored in future research. The two discriminant functions accounted for over 67% of the between-group variance for the six medical specialties, and further help to show how the specialties are similar, or dissimilar, to each other. The first function best differentiates between Surgery-General and Psychiatry. We would expect to see a statistically derived difference in the interest profiles of students choosing these two specialties because of the different activities each entail. Whereas Psychiatry involves a heavy emphasis on interpersonal issues, and relies on ‘‘talking cures” (i.e., counseling practices) to address patients’ health disturbances, Surgery-General generally focuses less on interpersonal issues, and relies more heavily on physical interventions to address health disturbances. As mentioned earlier, Psychiatry appears to share the least amount of interests with the other five specialties. Psychiatrists concentrate on mental health issues, whereas the other specialties attend to physical ailments. Family Medicine and Pediatrics appear to be the least differentiated. Both specialties address a wide range of health problems, and only differ in the population of patients they see. Family physicians attend to all types of patients, while Pediatricians primarily treat children, but both ultimately interact with all family members. OBGYN appears to be different from the other specialties, possibly because this specialty attends to a specific set of health problems and mainly treats females. The other specialties may see a greater variety of patients and health problems. The results of this study have implications for career practitioners who advise medical students. Practitioners should first examine students’ MSPI scores for each of the medical specialties and identify their highest score. Next, they should determine the distinctiveness of the highest score, by subtracting the second highest score from the highest score. If the highest score is at least 3.5 points greater than the second highest score, students should be advised to consider the medical specialty associated with their highest score. If the difference between the highest score and the second highest score is less than 3.5 points, students should be advised to consider the next most likely alternative. Practitioners can use the results displayed in Table 2 to identify which medical specialty students will choose if they do not choose the specialty they were predicted to enter. For example, students who do not enter Family Medicine are most likely to enter Pediatrics. A case study using a student’s scores, and chosen medical specialty, helps to illustrate how practitioners can interpret MSPI profiles. The MPSI scores for a female student were as follows: Family Medicine = 83, Internal Medicine = 70, OBGYN = 63, Pediatrics = 62, Psychiatry = 57, and Surgery-General = 52. This student chose to enter Family Medicine. The MSPI results show that the student scored highest on Family Medicine, and that this score is at least 3.5 points greater than the second highest score. Therefore, this student would be best advised to consider choosing Family Medicine. Table 2 suggests that individuals who score high on Family Medicine, but do not choose it, tend to choose Pediatrics. Another case study illustrates a student whose highest MSPI score did not match his chosen medical specialty. The MPSI scores for a male student were as follows: Family Medicine = 71, Internal Medicine = 74, OBGYN = 55, Pediatrics = 53, Psychiatry = 42, and Surgery-General = 65. This student chose to enter Surgery-General, despite the fact that this is his third highest score. The MSPI results show that the student scored highest on Internal Medicine. However, the difference between the highest and second highest scores is less than 3.5 points. This student did not enter Internal Medicine. Table 2 suggests that individuals who score highest on Internal Medicine, but do not choose it, tend to choose Surgery-General. 4.1. Limitations The results presented herein are limited to medical students who have made a specialty choice based on one of the six major specialties, which include; Family Medicine, Internal Medicine, Pediatrics, OBGYN, Surgery-General, and Psychiatry. These specialties only account for about 60% of all possible medical specialty choices. Future research should examine the predictive validity of the MSPI for a larger number of medical specialty choices. The results are also limited to the degree
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
133
that we only studied students in their first year of residency. This presents a problem because some students, especially those in Internal Medicine, use a specialty as an entry point for another specialty. Therefore, follow-up studies should compare MSPI scores to students’ medical specialty choice in their second and third years of residency. The self-selection process by which students participated in this research may have impacted the results. Although the MSPI is available to all medical students, on average, only 52% of students in each cohort completed the instrument. The researchers are aware that this process of self-selection may lead to a sample that is not representative of the overall population. There may be factors that differentiate those students who participated from those who did not, and these factors may have influenced the results. References Gough, H. (1979). Medical specialty preference scales: A report for counselors. Palo Alto, CA: Consulting Psychologists Press. Holland, J. L., Magoon, T. M., & Spokane, A. R. (1981). Counseling psychology: Career interventions, research, and theory. Annual Review of Psychology, 32, 270–305. Reed, V. A., Jernstedt, G. C., & Reber, E. S. (2001). Understanding and improving medical student specialty choice: A synthesis of the literature using decision theory as a referent. Teaching and Learning in Medicine, 13(2), 117–129. Savickas, M. L., Brizzi, J. S., Brisbin, L. A., & Pethtel, L. L. (1988). Predictive validity of two medical specialty preferences inventories. Measurement and Evaluation in Counseling and Development, 21, 106–112. Zimny, G. H. (1979). Manual for the medical specialty preference inventory. St. Louis, MO: St. Louis University School of Medicine. Zimny, G. H. (1980). Predictive validity of the medical specialty preference inventory. Medical Education, 14, 414–418. Zimny, G. H. (2002). Updating the medical specialty preference inventory. St. Louis University School of Medicine. Unpublished manuscript.
Contents lists available at ScienceDirect
Journal of Vocational Behavior journal homepage: www.elsevier.com/locate/jvb
Predictive validity of the medical specialty preference inventory Kevin W. Glavin a,*, George V. Richard b, Erik J. Porfeli a a b
Northeastern Ohio Universities College of Medicine and Pharmacy, Department of Behavioral Sciences, 4209 St. Rt. 44, P.O. Box 95, Rootstown, OH 44272, USA Association of American Medical Colleges, 2450 N Street, N.W., Washington, DC 20037, USA
a r t i c l e
i n f o
Article history: Received 9 November 2008 Available online 6 December 2008
Keywords: Predictive validity Medical specialty preference inventory Medical interests MSPI Medical specialty choice Medical students
a b s t r a c t Medical schools can assist students by providing them with quality career counseling to help them choose a medical specialty. Many schools use interest inventories to help identify students’ specialty interests. This study examined the predictive validity of one such inventory, the Medical Specialty Preference Inventory (MSPI). In a longitudinal design, we used discriminant function analysis to examine how well students’ scores on the MSPI fit their chosen medical specialty one year later. The MSPI correctly predicted students’ future medical specialty choice 58.1% of the time. These results can help career advisors interpret MSPI scores, and identify students’ most likely medical specialty choice, as well as their second most likely choice. Ó 2008 Elsevier Inc. All rights reserved.
1. Introduction Medical students’ specialty choice constitutes an important personal decision with far reaching consequences for individuals, and their families. Obtaining a medical degree requires the student to invest significant personal and economic resourses, and to delay transitions to work and family roles. State governments invest enormous resources to subsidize medical education at state institutions, and students mortgage their future to pay for the high costs of a medical education. Consequently, medical care providers, medical school faculty and staff, and students’ families encourage medical students to think proactively and carefully about their future medical specialty choice. Medical students choose a specialty for a variety of reasons, including experiences and exposure in medical school, academic performance in relevant clinical clerkships, personality attributes, and ratings of the content of medical practice (Reed, Jernstedt, & Reber, 2001). Unfortunately, students must often make this life-changing choice with inadequate information. Medical specialty counseling has developed over time to assist students in making specialty choices. Some career advisors use the Medical Specialty Preference Inventory (MSPI: Zimny, 1979) to facilitate the decision-making process. The MSPI is an assessment of students’ specialty interests and is used to inform their choice of medical specialty. In addition, the instrument can be used to assist physicians who are considering a medical specialty change at some point during residency. The original MSPI consisted of 199 items pertaining to medical activities and settings. Zimny (1979) asked a sample of physicians in each of six major medical specialties (Family Medicine, Internal Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry, and Surgery) to rate the extent to which each item from the MSPI was characteristic of general practice in their specialties. This method avoided the inherent problem in using generic interests, which may co-vary across specialties, and therefore, fail to distinguish clearly between them. The authors considered this approach to be favorable because it relates the characteristics of the practice of the specialties themselves rather than the characteristics of the physicians who
* Corresponding author. E-mail address: [email protected] (K.W. Glavin). 0001-8791/$ - see front matter Ó 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jvb.2008.11.004
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
129
practice in those specialties. Subsequently, it compares the medical activities that students prefer to the activities that physician’s in that specialty perform. This approach has received support from the literature. Savickas, Brizzi, Brisbin, and Pethtel (1988) compared the predictive validity of the MSPI with another medical specialty preference inventory, the Medical Specialty Preference Scales (MSPS: Gough, 1979). Whereas the MSPI uses practitioner-based scales and contains items related to actual medical practice, the MSPS uses student-based scales and contains generic interest items. The results from two consecutive medical student cohorts found that the MSPI successfully predicted a student’s specialty 59% of the time, whereas the MSPS successfully predicted a student’s specialty 19% of the time. Furthermore, the prediction rate for the MSPS only slightly exceeded what one would expect to see by chance alone. These results suggest that practitioner-based scales containing items related to specific medical practice predict students’ medical specialty choice better than student-based scales containing generic interest items. The current study employed the MSPI 2nd edition, which Zimny revised and updated in 2002 (Zimny, 2002). This revision brought about a number of important changes including a reduction in the number of items and subscales. Although previous research demonstrated the strong predictive validity of the original version of the MSPI, the predictive power of the updated version has not been tested. Therefore, this study investigated the predictive validity of the MSPI 2nd edition. 2. Methodology 2.1. Participants The participants were 506 medical students who completed the MSPI on the Association of American Medical Colleges’ Careers in Medicine (CiM) website, which can be found at http://www.aamc.org/students/cim/. Careers in Medicine is a comprehensive career planning program available to all US and Canadian medical schools. To gain access to the site students must obtain a free token from their school liaison, usually the school’s associate or assistant dean of student affairs. At registration users are presented with an IRB-approved informed consent statement that indicates any data stored in the CiM site may be used for research. While most schools actively provide access to the program, there are small number of schools that do not participate as actively as others. Registered users of the site represent approximately 55–65% of all enrolled medical students in US and Canadian medical schools. Once registered, students are free to use all of the confidential resources available on the site. Use of this site is voluntary as is the completion of the MSPI. The data were gathered from January 2005 to December 2006, while the respondents were in their final year of medical school, and contained 190 male students, and 316 female students. Respondents identified their race as follows: Caucasian (67%), Asian American/Pacific Islander (12%), African American (9%), Indian American (7%), Native American (1%), Other (2%), and Unknown (2%). Upon completion of medical school, and approximately one year later, students entered residency training for their chosen specialty. The time between completing the MSPI and reporting residency choice was approximately one year. Information about specialty choice was obtained from the AAMC’s GME Track system containing resident census information for all training programs in the US. Those who were specified as active residents in one of six medical specialties (Family Medicine, Internal Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry, and Surgery) were selected for the purposes of this study. 2.2. Instrument The MSPI measures medical interests, and is used to assist students in choosing a specialty. A 2002 revision of the MSPI resulted in a reduction in the number of items to 150. Of the 150 items, 104 are used for scoring purposes. The remaining 46 provide additional data intended for use in constructing future scales. MSPI items reflect job specific tasks that relate directly to medical practice. Students rate each item on a seven-point scale which reflects their degree of desirability for each item. A score of 1–2 indicates low desirability, 3–5 indicates moderate desirability, and 6–7 indicates high desirability. Differences between students’ subscale scores and specialists’ scores determine students’ preference for each medical specialty. Students whose subscale scores are similar to specialist scores for the same subscales will receive a high preference score for that particular specialty. Scores are calculated for: Family Medicine (FAM), Obstetrics Gynecology (OBGYN), Surgery, Psychiatry (PSY), Pediatrics (PED), and Internal Medicine (MED). Higher scores indicate a greater preference for a specialty, while lower scores indicate less preference for a specialty. The instrument was self-administered on the internet, and preference scores were reported to participants immediately upon completion. Zimny (1979) reported on the reliability and validity indices used in the initial development of the MSPI. Reliability was estimated in two separate analyses. In the first analysis, using the Spearman Brown formula, reliability estimates ranged from .74 (Pediatrics) to .93 (Family Medicine), and in the second analysis estimates ranged from .66 (Pediatrics) to .91 (Surgery). Most reliability estimates in both analyses fell in the .80s and .90s. Zimny also conducted a study on the predictive validity of the MSPI using National Resident Matching Program (NRMP) data (1980). The NRMP contains data about the initial specialty choice of medical students as they enter into residency training. He found that the level of predictive accuracy over all specialties ranged from about 50 to 55%. This range represents a level of prediction well above the conservative chance expectancy level of 17% accuracy, and indicates that a substantial relationship exists between specialty preference scores on the MSPI and subsequent specialty choice.
130
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
To test the predictive validity of the MSPI, scores for each of the six medical specialties were compared to students’ residency choices. Six v2-tests were conducted to test the predictive validity of the six MSPI scale scores. The general hypothesis was that students’ pattern of scores across the six MSPI scores would be predictive of their medical specialty choice one year later. Students were classified into three MSPI groups for each of the six specialties. Group one included those students whose score for a particular specialty was their highest score relative to the other five specialties, and this score was at least 3.5 points greater than their next highest scale score. The researchers used 3.5 points because this represented approximately half of the average difference between students’ highest and second highest scores. Group two included those students whose score for a particular specialty was their highest score relative to the other five specialties, but this score was not at least 3.5 points greater than their next highest scale score. Group three included the remaining students. The aim was to discern if students choose, for example, family medicine for their residency with greater frequency if they scored highest on the family medicine subscale of the MSPI, and if the frequency of this specialty choice increased as this score became more pronounced (by more than 3.5 points) relative to the other five scale scores. 3. Results Results showed that the overall hit rate (i.e., rate of correctly classified students based upon their interests and specialty choice), for groups one and two combined, was 46%. The overall hit rates by medical specialty were as follows; Family Medicine 41%, OBGYN 53%, Pediatrics 46%, Psychiatry 56%, Internal Medicine 27%, Surgery-General 88%. Of the participants who chose Family Medicine as their medical specialty, 27.7% of them met both criteria (i.e., group 1), 13.3% met the second criteria only (i.e., group 2), and the remainder, 59%, met neither criteria (i.e., group 3), v2 (2, N = 506) = 81.09 p < 0.001. Of the participants who chose OBGYN as their medical specialty, 44.2% of them met both criteria, 9.3% met the second criteria only, and the remainder, 46.5%, met neither criteria, v2 (2, N = 506) = 82.92 p < 0.001. Of the participants who chose Pediatrics as their medical specialty, 23.2% of them met both criteria, 23.2% met the second criteria only, and the remainder, 53.7%, met neither criteria, v2 (2, N = 506) = 60.9 p < 0.001. Of the participants who chose Psychiatry as their medical specialty, 48.8% of them met both criteria, 7% met the second criteria only, and the remainder, 44.2%, met neither criteria, v2 (2, N = 506) = 172.94 p < 0.001. Of the participants who chose Internal Medicine as their medical specialty, 11.2% of them met both criteria, 15.4% met the second criteria only, and the remainder, 73.4%, met neither criteria, v2 (2, N = 506) = 59.36 p < 0.001. Of the participants who chose Surgery-General as their medical specialty, 78.1% of them met both criteria, 9.6% met the second criteria only, and the remainder, 12.3%, met neither criteria, v2 (2, N = 506) = 117.9 p < 0.001. Although these v2 results are promising, they mainly speak to bivariate associations between categories of MSPI subscale scores and specialty choice. They do not permit a simultaneous comparison of the discriminatory power of the six subscales in combination. Discriminant function analysis was employed to test if, and to what extent, the entire MSPI predicts medical specialty choice and to what extent each of the subscales contributes to its discriminatory power while controlling for the others. Fig. 1 shows the mean scores for each of the subscales for each of the medical specialty choices. Each line represents residents who chose that specialty, and their mean scores on each of the six preference scales. For example, students who chose
75.00
Preference Score
70.00
Medical Specialty Choice
65.00
FAM MED
60.00
OBGYN PED
55.00
PSY SUR
50.00 45.00 FAM
MED
OBGYN
PED
PSY
SUR
Medical Specialty Preference Scales Fig. 1. Mean scores for medical specialty preference scales by specialty choice. *FAM, Family Medicine; MED, Internal Medicine; Pediatrics, Pediatrics, Psychiatry, Psychiatry, SUR, Surgery-General.
131
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
Family Medicine, on average, scored 67.73 on Family Medicine, 65.1 on Internal Medicine, 67.13 on OBGYN, 67.43 on Pediatrics, 55.68 on Psychiatry, and 60.35 on Surgery-General. The most differentiated group profiles occur for those students who selected Psychiatry and Surgery-General specialties. Students who chose Psychiatry tended to score highest on the Psychiatry subscale and lowest on the Surgery-General subscale, relative to the other four subscales, and relative to all the other students in the other medical specialties. The least differentiated profile occurred for Family Medicine. Students choosing OBGYN and Pediatrics had the most elevated scores, while students choosing Psychiatry had the least elevated scores. Medical residents in Family Medicine and Internal Medicine demonstrated mid-range elevation scores, when compared to the residents in other medical specialties. A direct discriminant analysis was performed using six interest variables as predictors of membership in six medical specialty groups. Predictors were scores on Family Medicine, Internal Medicine, OBGYN, Pediatrics, Psychiatry, and SurgeryGeneral MSPI scales. The criterion variable was the medical specialty the participant entered. The researchers used 70% of the original dataset to run the discriminant analysis, and reserved the remaining 30% for cross-validation purposes. No cases were found to be missing data, n = 365. The results yielded five significant discriminant functions, with a combined v2 (30) = 613.29, p < 0.001. Functions two through five yielded a v2 (20) = 364.293, p < 0.001. Functions three through five yielded a v2 (12) = 227.175, p < 0.001. Functions four through five yielded a v2 (6) = 114.957, p < 0.001. Function five yielded a v2 (2) = 34.107, p < 0.001. Table 1 displays bivariate correlations, means, and standard deviations for each of the predictor variables. Strong intercorrelations exist between Family Medicine and Internal Medicine (.81), and OBGYN and Pediatrics (.75). These correlations give cause for concern because they violate one of the assumptions of discriminant function analysis. The effect sizes for the discriminant functions were as follows; .5, .32, .27, .2, and .09, for functions 1, 2, 3, 4, and 5, respectively. Therefore, the five functions accounted for 50%, 32%, 27%, 20%, and 9% of the total relationship between predictors and groups. The amount of between-group variance accounted for by each function was as follows; 45.8%, 21.3%, 16.8%, 11.6%, and 4.6% for functions 1, 2, 3, 4, and 5, respectively. The classification results displayed in Table 2 demonstrate that the MSPI may be used to correctly classify students’ specialty choice 58.1% of the time, which exceeds what we would expect to see by chance alone (i.e., 20%). The results show that the MSPI best predicts participants who chose Surgery-General, with a 74% correct classification rate, followed by psychiatry, with a hit rate of 71%. Although the MSPI’s predictive power is weakest for family medicine, the hit rate (47%) remains well above chance. The pattern of misclassifications for Family Medicine and Pediatrics specialties suggest that students who select these specialties may exhibit similar patterns of interests. The same can be said for Surgery-General and Internal Medicine. The patterns of misclassifications were inconsistent for Psychiatry relative to any other discipline, which suggests that participants who chose this specialty appear to have a relatively unique interest profile. The stability of the classification procedure was checked by a cross-validation run. Approximately 30% of the cases were withheld from calculation of the classification functions in this run. For the 70% of the cases from which the functions were derived, there was a 58% correct classification rate. For the cross-validation cases, classification was 57%. This result suggests there exists a high degree of consistency in the classification scheme, and suggests that the results are not dependent on the sample data used to test the discriminatory power of the MSPI. 4. Discussion This study examined the predictive validity of the MSPI, which demonstrated an overall hit rate of approximately 46% when using a v2 analysis. The highest hit rate occurred for Surgery-General, and the lowest hit rate occurred for Psychiatry. These findings are significantly better than what we would expect to see by chance alone, which was determined to be 20%. Interest inventories based on six categories generally report hit rates between 35 and 45% (Holland, Magoon, & Spokane, 1981). The researchers used discriminant function analysis to move beyond simple bivariate associations to test the simultaneous discriminatory power of the six MSPI scales to predict medical specialty choice. Fig. 1 demonstrates that residents within the six medical specialties typically exhibited unique patterns of preference scores based on the elevation and variability of those scores. Table 2 suggests that in most specialties there exists a secondary, or perhaps tertiary, specialty that is typically chosen by participants who do not choose the predicted specialty on the basis of their interests. One can think of
Table 1 Inter-correlations, means, and standard deviations for medical specialty preference scores.
1. 2. 3. 4. 5. 6. *
Family Internal OBGYN Pediatrics Psychiatry Surgery p < .001.
Mean
SD
1
2
3
4
5
6
60.37 63.38 65.97 65.65 55.11 64.33
11.20 8.81 7.63 7.20 10.34 10.77
1.00
0.81* 1.00
0.48* 0.36* 1.00
0.48* 0.45* 0.75* 1.00
0.35* 0.24* 0.31* 0.47* 1.00
0.00* 0.37* 0.06 0.00 -0.55* 1.00
132
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
Table 2 Classifications by medical specialty. Predicted Medical specialty
Chosen medical specialty
Family Medicine Internal Medicine OBGYN Pediatrics Psychiatry Surgery-General
Family Medicine
Internal Medicine
OBGYN
Pediatrics
Psychiatry
Surgery-General
46.7 8.3 8.6 15.2 17.6 2
10 55.8 8.6 12.1 0 14
8.3 7.5 60 6.1 2.9 6
18.3 12.5 5.7 53 8.8 4
6.7 2.5 0 4.5 70.6 0
10 13.3 17.1 9.1 0 74
Note: Numbers are row percentages and bolded numbers reflect the percent of correct classifications for each medical specialty.
those secondary and tertiary specialties as a typical alternative. For example, while participants who scored highest in Family Medicine chose to enter Family Medicine 46.7% of the time, another 18.3% actually entered Pediatrics. This makes intuitive sense given that both Family Medicine and Pediatrics share some commonalities, such as the breadth of problems they deal with, and in the role they play as primary-care providers. Participants with high OBGYN scores chose OBGYN 60% of the time, but another 17.1% chose to enter General-Surgery. Both of these specialties share commonality in their surgical applications, and so these results make sense also. While most of the specialties seem to have interest companions, psychiatry seems to be fairly distinct with respect to interests. On a larger level, these results hint at an underlying conceptual structure of medical specialties on the basis of interest that could be explored in future research. The two discriminant functions accounted for over 67% of the between-group variance for the six medical specialties, and further help to show how the specialties are similar, or dissimilar, to each other. The first function best differentiates between Surgery-General and Psychiatry. We would expect to see a statistically derived difference in the interest profiles of students choosing these two specialties because of the different activities each entail. Whereas Psychiatry involves a heavy emphasis on interpersonal issues, and relies on ‘‘talking cures” (i.e., counseling practices) to address patients’ health disturbances, Surgery-General generally focuses less on interpersonal issues, and relies more heavily on physical interventions to address health disturbances. As mentioned earlier, Psychiatry appears to share the least amount of interests with the other five specialties. Psychiatrists concentrate on mental health issues, whereas the other specialties attend to physical ailments. Family Medicine and Pediatrics appear to be the least differentiated. Both specialties address a wide range of health problems, and only differ in the population of patients they see. Family physicians attend to all types of patients, while Pediatricians primarily treat children, but both ultimately interact with all family members. OBGYN appears to be different from the other specialties, possibly because this specialty attends to a specific set of health problems and mainly treats females. The other specialties may see a greater variety of patients and health problems. The results of this study have implications for career practitioners who advise medical students. Practitioners should first examine students’ MSPI scores for each of the medical specialties and identify their highest score. Next, they should determine the distinctiveness of the highest score, by subtracting the second highest score from the highest score. If the highest score is at least 3.5 points greater than the second highest score, students should be advised to consider the medical specialty associated with their highest score. If the difference between the highest score and the second highest score is less than 3.5 points, students should be advised to consider the next most likely alternative. Practitioners can use the results displayed in Table 2 to identify which medical specialty students will choose if they do not choose the specialty they were predicted to enter. For example, students who do not enter Family Medicine are most likely to enter Pediatrics. A case study using a student’s scores, and chosen medical specialty, helps to illustrate how practitioners can interpret MSPI profiles. The MPSI scores for a female student were as follows: Family Medicine = 83, Internal Medicine = 70, OBGYN = 63, Pediatrics = 62, Psychiatry = 57, and Surgery-General = 52. This student chose to enter Family Medicine. The MSPI results show that the student scored highest on Family Medicine, and that this score is at least 3.5 points greater than the second highest score. Therefore, this student would be best advised to consider choosing Family Medicine. Table 2 suggests that individuals who score high on Family Medicine, but do not choose it, tend to choose Pediatrics. Another case study illustrates a student whose highest MSPI score did not match his chosen medical specialty. The MPSI scores for a male student were as follows: Family Medicine = 71, Internal Medicine = 74, OBGYN = 55, Pediatrics = 53, Psychiatry = 42, and Surgery-General = 65. This student chose to enter Surgery-General, despite the fact that this is his third highest score. The MSPI results show that the student scored highest on Internal Medicine. However, the difference between the highest and second highest scores is less than 3.5 points. This student did not enter Internal Medicine. Table 2 suggests that individuals who score highest on Internal Medicine, but do not choose it, tend to choose Surgery-General. 4.1. Limitations The results presented herein are limited to medical students who have made a specialty choice based on one of the six major specialties, which include; Family Medicine, Internal Medicine, Pediatrics, OBGYN, Surgery-General, and Psychiatry. These specialties only account for about 60% of all possible medical specialty choices. Future research should examine the predictive validity of the MSPI for a larger number of medical specialty choices. The results are also limited to the degree
K.W. Glavin et al. / Journal of Vocational Behavior 74 (2009) 128–133
133
that we only studied students in their first year of residency. This presents a problem because some students, especially those in Internal Medicine, use a specialty as an entry point for another specialty. Therefore, follow-up studies should compare MSPI scores to students’ medical specialty choice in their second and third years of residency. The self-selection process by which students participated in this research may have impacted the results. Although the MSPI is available to all medical students, on average, only 52% of students in each cohort completed the instrument. The researchers are aware that this process of self-selection may lead to a sample that is not representative of the overall population. There may be factors that differentiate those students who participated from those who did not, and these factors may have influenced the results. References Gough, H. (1979). Medical specialty preference scales: A report for counselors. Palo Alto, CA: Consulting Psychologists Press. Holland, J. L., Magoon, T. M., & Spokane, A. R. (1981). Counseling psychology: Career interventions, research, and theory. Annual Review of Psychology, 32, 270–305. Reed, V. A., Jernstedt, G. C., & Reber, E. S. (2001). Understanding and improving medical student specialty choice: A synthesis of the literature using decision theory as a referent. Teaching and Learning in Medicine, 13(2), 117–129. Savickas, M. L., Brizzi, J. S., Brisbin, L. A., & Pethtel, L. L. (1988). Predictive validity of two medical specialty preferences inventories. Measurement and Evaluation in Counseling and Development, 21, 106–112. Zimny, G. H. (1979). Manual for the medical specialty preference inventory. St. Louis, MO: St. Louis University School of Medicine. Zimny, G. H. (1980). Predictive validity of the medical specialty preference inventory. Medical Education, 14, 414–418. Zimny, G. H. (2002). Updating the medical specialty preference inventory. St. Louis University School of Medicine. Unpublished manuscript.