- Email: [email protected]
Can success and failure be predicted for baccalaureate graduates on the computerized NCLEX-RN?
Can Success and Failure be Predicted for Baccalaureate Graduates on the Computerized NCLEX-RN? LISA A. SELDOMRIDGE, PHD, RN,* AND MARY C. DIBARTOLO, PHD, RNC†
The current shortage of nurses and declining national pass rate on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) has heightened educators’ interest in identifying students at risk for failure. A retrospective descriptive study was conducted at a rural, public baccalaureate nursing program to determine variables that best predict NCLEX-RN success and failure. Data collected from 1998 through 2002 (N ⴝ 186) included entry as native or transfer student, preadmission grade point average (GPA), GPA after completing one semester of nursing courses, cumulative GPA at graduation, grades earned in prerequisite and core nursing courses, test averages in beginning and advanced medical/surgical nursing courses, and performance on the National League for Nursing Comprehensive Achievement Test for Baccalaureate Students (NLNCATBS). Logistic regression analysis revealed that a combination of test average in advanced medical/ surgical nursing and percentile score on the NLNCATBS predicted 94.7 percent of NCLEX-RN passes and 33.3 percent of failures. The combination of NLNCATBS score and pathophysiology grade predicted 93.3 percent of NCLEX-RN passes and 50 percent of failures. Although success could be accurately predicted across all models, predicting failure was far more difficult. (Index words: NCLEX-RN predictors; Nursing licensure; Baccalaureate education) J Prof Nurs 20:361-368, 2004. © 2004 Elsevier Inc. All rights reserved.
T
HE CURRENT nursing shortage, acknowledged as one of significant magnitude, is projected to worsen, reaching a 20 percent deficit by the year 2020 (Nelson, 2002). This crisis, coupled with the climate of health care reform, has placed unprecedented pressure on nursing programs to increase the supply of qualified graduates. Entry of competent nurses into the work*Associate Professor of Nursing, Salisbury University Department of Nursing, Salisbury, MD. †Assistant Professor of Nursing, Salisbury University Department of Nursing, Salisbury, MD. Address correspondence and reprint requests to Dr. Seldomridge: Salisbury University Department of Nursing, 1101 Camden Ave., Salisbury, MD 21801. E-mail: [email protected] 8755-7223/$30.00 © 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.profnurs.2004.08.005
force is facilitated by first-time success on the National Council Licensure Examination for Registered Nurses (NCLEX-RN). NCLEX-RN pass rates remain an important criterion for measuring effectiveness of a nursing program (Arathuzik & Aber, 1998). Acceptable NCLEX-RN pass rates are necessary to receive continued state board of nursing approval, maintain accreditation, attract qualified students, and uphold a program’s reputation and obligation to protect the public from unsafe care (Beeson & Kissling, 2001). Recent data depict a steady decline in NCLEX-RN pass rates for first-time candidates (Maryland Board of Nursing, 2002; National Council of State Boards of Nursing, 2002a), from an 87.7 percent pass rate in 1997 to an 82.2 percent pass rate in 2002. Examining the relationships among admissions criteria, performance in nursing courses, and NCLEX-RN results assists programs in strengthening admissions and progression policies so that students selected to begin nursing courses have the best possibility of completing the typically rigorous curriculum. Once enrolled in nursing courses, early identification of at-risk students is useful so they may receive interventions to improve likelihood of initial NCLEX-RN success. This is essential in an environment where limited clinical placements, qualified faculty, and financial resources can greatly restrict enrollment capacity (Byrd, Garza, & Neiswiadomy, 1999). Initial NCLEX-RN success obviously helps students avoid the consequences of failure, such as the negative impact on self-esteem and financial setbacks when career plans are delayed. Predicting NCLEX-RN success is vital for educators and administrators because potential applicants use the passing rate to evaluate and compare programs. Furthermore, nursing programs that exhibit difficulty with maintaining a satisfactory pass rate may also face the possibility of probationary status with the state board of nursing (Endres, 1997). The purposes of this retrospective study were to identify the best models for predicting NCLEX-RN success and failure at three points in the nursing curriculum: preadmission, after the completion of 1 year of nursing courses, and immediately prior to gradua-
Journal of Professional Nursing, Vol 20, No 6 (November–December), 2004: pp 361-368
361
362
SELDOMRIDGE AND DIBARTOLO
tion. Additionally, this study aims to answer an elusive question—to what extent, if any, can NCLEX-RN success and failure be accurately predicted?
Review of the Literature
Since the NCLEX-RN was instituted as the nationally approved licensing examination for registered nurses in 1982, a multitude of research has been conducted to identify factors associated with the greatest likelihood of success. Studies have explored a variety of academic and nonacademic predictors. Academic predictors have included high school rank, Scholastic Aptitude Test (SAT) scores, American College Test (ACT) scores, entering GPA, freshman and prenursing GPA, nursing GPA, GPA upon graduation, achievement in specific nursing courses, science course grades, Mosby AssessTest scores, National League for Nursing Achievement Test scores, and Watson-Glaser Critical Thinking Appraisal scores. Nonacademic predictors of success on NCLEX-RN have ranged from age, sex, ethnicity, primary language, and test anxiety to selfesteem and self-efficacy. Although several studies were reported after the initial state board exam was revised to become the NCLEX-RN in 1982, few predictive studies of NCLEX-RN success in baccalaureate students have been conducted since the examination was modified in 1988 to the pass/fail format. Even fewer studies have been conducted since the format was changed to computer adaptive testing (CAT) in April of 1994. Beeson and Kissling (2001) found that students who passed the NCLEX-RN had fewer grades of C or below in nursing courses, scored higher on the Mosby AssessTest and had significantly higher average GPAs than those who failed. Beeman and Waterhouse (2001) identified seven significant predictor variables, with the total number of C⫹ or lower grades earned in nursing didactic courses as the best predictor, followed by grades in several individual nursing courses. Waterhouse and Beeman (2003) tested the accuracy of a simple risk appraisal instrument in predicting NCLEX-RN success; nearly 61 percent of failures were correctly classified; 72 percent were accurately classified overall. In a recent study, final course grade for a didactic, senior-level medical/surgical nursing course and cumulative program GPA were the only program variables consistently associated with NCLEX-RN success (Daley, Kirkpatrick, Frazier, Chung, & Moser, 2003). Related studies have recently investigated the Health Education Systems, Inc. (HESI) Exit Examina-
tion as a predictor of NCLEX-RN success. The HESI Exit Examination is a computerized comprehensive examination that uses a proprietary mathematical model to offer immediate feedback to students in the form of a probability score. The exam has been used as both a measure of preparedness to take the NCLEX-RN and a benchmark for remediation (Morrison, Free, & Newman, 2002). Studies have found HESI exam scores to be highly predictive of NCLEX-RN success (Lauchner, Newman, & Britt, 1999; Newman, Britt, & Lauchner, 2000; Nibert & Young, 2001) with results in the 96 to 99 percent range. It is evident from the literature that while the number and variety of variables that may affect NCLEX-RN success has steadily increased, the accuracy of predictions of NCLEX-RN success has declined overall since 1988 when studies were generally able to correctly classify up to 86 percent of students (Waterhouse & Beeman, 2003). Since that time, fewer studies have been published; this may reflect the difficulty of predicting success based exclusively on pass/ fail scores that usually fall into the “pass” category. Furthermore, the adoption of the computerized testing format in 1994 has complicated the search for the best set of predictors by requiring that students have some expertise in computerized test-taking (Beeman & Kissling, 2001). The implementation of a modified test plan and a more rigorous passing standard in 1998 by the National Council of State Boards of Nursing has also made comparisons among cohorts difficult. The following research questions were developed to guide this study: ●
●
●
● ●
Which prerequisites to the nursing major are the best predictors of success/failure on the NCLEX-RN? Which variables occurring after completion of junior-year nursing courses best predict success/failure on the NCLEX-RN? Which variables occurring between junior-year nursing courses and graduation best predict success/failure on the NCLEX-RN? Which overall combination of variables best predicts success/failure on the NCLEX-RN? Can success or failure on the NCLEX-RN be accurately predicted? Methodology
The sample consisted of graduates of the traditional baccalaureate nursing program at a rural, mid-Atlantic
363
NCLEX-RN PREDICTORS
public institution from 1998 through 2002. To minimize the potential effects of periodic changes in the NCLEX-RN, the sample was limited to students taking the computerized version according to the modified test plan and passing standards instituted in April 1998. After approval from the University Committee on Human Volunteers, data were collected from student records and official reports of NCLEX-RN results. One graduate was excluded from the sample due to missing NCLEX-RN results. The final sample consisted of 186 students, 174 women and 12 men, evenly divided between native students (beginning their college careers at this university, n ⫽ 95 or 51 percent) and transfer students (n ⫽ 91 or 49 percent). Of the 186 subjects, 150 (80.6 percent) passed NCLEX-RN on the first attempt, and 36 (19.3 percent) did not. The dependent variable was the NCLEX-RN outcome, noted as pass or fail. A total of 13 independent variables were selected as predictors of NCLEX-RN success or failure based on a thorough review of the literature. The independent variables, divided into three groups for analysis by time period, included preadmission indicators used to make decisions about starting the nursing curriculum, indicators available after completing junior-year nursing courses, and indicators available prior to graduation. Independent variables of relevance to the admissions process were grades in specific prerequisites, including Anatomy and Physiology I, Pathophysiology, and Chemistry (4 credits each) and Statistics (3 credits); overall pattern of performance in these prerequisites measured by the number of grades of C or lower (0-4); and cumulative GPA prior to beginning nursing courses. GPAs and course grades were based on a scale of 0 to 4 (0 ⫽ F, 4 ⫽ A). SAT scores were excluded from this study because they were not available for transfer students who comprised half the sample. It is important to note that pathophysiology is the only prerequisite course in which a minimum grade of C is required. Independent variables at the conclusion of the junior year included cumulative GPA after completion of the first semester of junior nursing courses (selected because of an observed decline in academic performance in this particular semester), number of Cs in junior nursing courses (0-9), and test averages (out of 100 percent) in two medical/surgical didactic nursing courses (Adult Health I & II, 3 credits each). Test averages, not examined in any previous research as a predictor variable, were selected as precise measures of performance in a testing setting. Test averages from the medical/surgical didactic courses were used because of
their history of discriminating between passing and failing students. Independent variables selected for study in the senior year prior to graduation included number of Cs in all nursing courses (0-17) and percentile score on the National League for Nursing Comprehensive Achievement Test for Baccalaureate Students (NLNCATBS)—a paper and pencil test taken 2 weeks before graduation that compares individual achievement to a national sampling of students completing baccalaureate nursing programs. DATA ANALYSIS
All data were entered into SPSSX version 11.0 (SPSS, Chicago, IL) for analysis. Descriptive statistics were used to characterize the sample and to identify possible predictors of NCLEX-RN outcome. Pearsonproduct moment correlation coefficients were computed for NCLEX-RN success with continuous variables. The two-sample t test was used as a screening procedure to identify possible predictors of NCLEX-RN success. Group differences were also examined using the nonparametric Mann-Whitney test because some variables were not normally distributed. Since the results of this comparison were of the same magnitude of significance as the two-sample t test, only the latter are reported here. Logistic regression, a multivariate technique, was chosen to combine continuous and categorical predictor variables to predict outcome on a categorical variable (Hosmer & Lemeshow, 1989) such as the NCLEX-RN.
Results
Student characteristics and their association with NCLEX-RN performance are summarized in Tables 1 and 2. Percentile score on the NLNCATBS showed the highest correlation with NCLEX-RN success, (r ⫽ .452, P ⫽ .000), followed by grade in Pathophysiology (r ⫽ .377, P ⫽ .000), test average in the advanced medical/surgical course (r ⫽ .307, P ⫽ .000), and test average in the introductory adult medical/surgical course (r ⫽ .303, P ⫽ .000). Patterns of low grades in prerequisites (r ⫽ ⫺.245, P ⫽ .002) and nursing courses (r ⫽ ⫺.342, P ⫽ .000) were negatively correlated with NCLEX-RN success. Those successful on NCLEX-RN had significantly higher test averages in both medical/surgical nursing courses, had higher GPAs (upon entry to nursing courses, at the end of 1 semester of nursing courses, and on program comple-
364 TABLE
SELDOMRIDGE AND DIBARTOLO
tistical significance. Students who passed the NCLEX-RN were correctly classified 100 percent of the time based on their grade in pathophysiology. However, students who failed the NCLEX-RN were correctly classified only 2.8 percent of the time. The overall accuracy in prediction was 81.2 percent. For each letter grade increase in Pathophysiology, the odds of passing the NCLEX-RN improved by nearly 5 times. The second model, using data available after completion of junior-year nursing courses, included test averages from two medical/surgical didactic courses (Adult Health I & II), number of Cs in junior-year nursing courses (maximum ⫽ 9) and overall GPA after 1 semester of nursing courses. Test averages from both medical/surgical didactic courses predicted NCLEX-RN pass 98.7 percent of the time but correctly predicted NCLEX-RN failure only 5.6 percent of the time. The overall prediction accuracy was 80.6 percent. For every 1-point increase in each test average, the odds of passing NCLEX-RN improved by one time. Those passing NCLEX-RN had higher mean test averages in both courses compared to those who failed NCLEX-RN. Students who failed NCLEX-RN had twice the number of Cs in the junior-year nursing courses as students who passed NCLEX-RN. Of the students with five or more Cs (n ⫽ 11) in junior-year nursing courses, 50 percent passed NCLEX-RN, while those without any Cs (n ⫽ 45) had a pass rate of 100 percent. The third model examined predictors available on completion of the senior year, including test averages in both medical/surgical courses, number of Cs in all nursing courses (maximum ⫽ 17), and percentile score on the standardized NLNCATBS. Score on the NLNCATBS, which entered the model on Step 1, correctly predicted 94.7 percent of NCLEX-RN passes and 25 percent of NCLEX-RN failures, with an overall pre-
1. Descriptive Statistics and Correlations with Passing the NCLEX-RN (n ⫽ 186) Variable
Mean
Grade-Anatomy/physiology 2.87 GradePathophysiology 2.98 Grade-Chemistry I 2.88 Grade-Statistics 3.07 Number of Cs or lower in pre-requisites 1.16 Test average-Adult Health I 79.01 Test average-Adult Health II 76.56 GPA-preadmission 3.07 GPA-after one semester of nursing 3.04 Number of Cs in junior nursing courses 2.2 Number of Cs or lower in all nursing courses 4.0 GPA-Final 3.11 Percentile score on NLNCATBS 48.4
Standard Deviation
Correlation with NCLEX-RN
.77
.131
.74 .81 .81
.377** .009 .126
1.13
⫺.222**
5.83
.303**
5.13 .41
.307** .195**
.43
.309**
1.61
⫺.306**
3.30 .34
⫺.342** .258**
25.58
.452**
**p ⬍ .01, two-tailed.
tion), and performed at a higher level on the NLNCATBS. Logistic regression analyses were performed to determine the most parsimonious set of predictors for each of three groupings of variables and when all variables were entered into the model. These results are reported in Table 3. Variables were selected for inclusion using the stepwise method with a significance level of .15 for entry into or removal from the model. The first model tested preadmission variables: grade in Pathophysiology, preadmission GPA, and number of Cs or lower in Anatomy & Physiology, Statistics, Chemistry I, and Pathophysiology. Performance in Pathophysiology was the only variable to achieve staTABLE
2. Bivariate Comparisons Between NCLEX-RN Outcome and Continuous Variables NCLEX-RN Outcome
Two Sample t Test Results
Variable
Passed (n ⫽ 150) Mean (SD)
Failed (n ⫽ 36) Mean (SD)
t(df ⫽ 184) Value
P
Preadmission GPA Grade Pathophysiology Number of Cs or lower in prerequisites Test average Adult Health I Test average Adult Health II GPA after one semester of nursing courses Number of Cs-junior nursing courses Number of Cs-all nursing courses Final GPA Percentile score NLNCATBS
3.11 (.42) 3.12 (.68) 1.03 (1.07) 79.87 (5.77) 77.34 (5.12) 3.09 (.42) 1.81 (1.59) 3.43 (3.09) 3.15 (.35) 54.06 (24.09)
2.91 (.32) 2.42 (.73) 1.67 (1.24) 75.41 (4.62) 73.35 (3.80) 2.79 (.37) 3.06 (1.26) 6.23 (3.16) 2.93 (.24) 24.86 (16.74)
⫺2.70 (184) ⫺5.52 (184) 3.09 (184) ⫺4.32 (184) ⫺4.38 (184) ⫺3.97 (184) 4.37 (184) 4.95 (184) ⫺3.62 (184) ⫺6.88 (184)
.008 .000 .002 .000 .000 .000 .000 .000 .000 .000
365
NCLEX-RN PREDICTORS
TABLE
3. Stepwise Logistic Regression Results: Odds Ratios and Confidence Intervals
Regression Models and Variables
Preadmission Grade in Pathophysiology Constant End of junior year Test average advanced med/surg Test average intro med/surg Constant End of final semester NLNCATBS score Test average advanced med/surg Constant All time periods NLNCATBS score Grade in Pathophysiology Constant
Odds Ratio
95% Confidence Intervals
4.47 .066
(2.37, 8.45)**
1.12 1.09 .000
(1.00, 1.26)** (.99, 1.21)
1.057
(1.032, 1.08)**
1.117 .000
(1.01, 1.24)*
1.057 3.15 .022
(1.032, 1.08)** (1.56, 6.36)**
NOTE. Advanced med/surg ⫽ Adult Health II nursing course; intro med/surg ⫽ Adult Health I nursing course. Odds ratios reflect one-unit change in the independent variable *p ⱕ .05, **p ⱕ .001.
diction accuracy of 81.2 percent. Test average for Adult Health II entered on Step 2, correctly predicting 94 percent of NCLEX-RN passes and 33.3 percent of NCLEX-RN failures, with an overall accuracy of 82.3 percent. Score on the NLNCATBS contributed most to the prediction (P ⫽ .000). For every 10-point increase in percentile score, the odds of passing NCLEX-RN increased by nearly 11 times. Upon program completion, students without any grades of C in nursing courses (n ⫽ 32) had an NCLEX-RN pass rate of 99.3 percent. Neither of the two students with the highest number of C grades (13 of a possible 17) passed the licensure examination on the first attempt. The final model for NCLEX-RN success used variables from all three time periods. At the end of Step 1, score on the NLNCATBS correctly predicted success on the NCLEX-RN 94.7 percent of the time and correctly predicted failure 25 percent of the time. Grade in Pathophysiology entered the model at Step 2, improving the prediction for NCLEX-RN failures to 50 percent; prediction of success dropped slightly to 93.3 percent. Overall success in predicting NCLEX-RN performance was 84.9 percent. No other variables entered the model. Discussion
The purposes of this study were to identify the best models for predicting NCLEX-RN outcome based on
data from several time points (on preadmission, after completion of junior-level nursing courses, and immediately prior to graduation) and to determine the extent to which NCLEX-RN success or failure can be accurately predicted. The results of this investigation suggest that while several variables were very accurate in predicting success (grade in Pathophysiology, test averages in two medical/surgical nursing didactic courses, and score on a National League for Nursing achievement test), they were much less accurate in predicting failure. The most accurate combination was the score on the NLNCATBS and Pathophysiology grade; this predicted failures 50 percent of the time. These findings are consistent with other recent studies (Beeman & Waterhouse, 2001; Beeson & Kissling, 2001; Byrd et al., 1999; Daley et al, 2003) that also reported a substantial disparity in accuracy of predicting success versus failure. The pass/fail nature of NCLEX-RN limits the use of more sophisticated statistical procedures, and the disproportionately low numbers who fail NCLEX-RN compared to those who pass also adversely impacts the degree to which failure can be predicted. Unless schools admit an exceedingly high number of unqualified applicants and/or provide such low-quality education that more than 20 percent of students fail the licensing exam (Hanks, 1999), predicting NCLEX-RN failure will continue to be a challenge. At this and many other universities, students are selected during their sophomore year to enter upper division nursing courses. Identifying predictors of success at this point aids in admission decisions, particularly during times of a burgeoning applicant pool for a limited number of spaces. In the preadmission phase, grade in Pathophysiology was the only significant predictor variable, confirming the need to require a minimum grade of C to be considered. Although GPA in prerequisite courses was not a statistically significant predictor of NCLEX-RN success, there is support for requiring a GPA of at least 2.5 of 4.0. As noted in this and other studies (Beeson & Kissling, 2001; Daley et al., 2003; Endres, 1997; Roncoli, Lisanti, & Falcone, 2000), students who passed NCLEX-RN had significantly higher GPAs than those who failed. Because pattern of performance in science courses appears to be important in NCLEX-RN success, it might be valuable to use GPA in science courses rather than overall GPA as one of the admission criteria. When and where students take science courses may reveal important information about their background and abilities. Often students who perceive themselves as weak in science take them during summer or winter sessions when they
366
have no other academic commitments or enroll at community colleges where such courses are believed to be less rigorous. After completion of the junior nursing courses, test averages in the introductory and advanced medical/ surgical courses best predicted NCLEX-RN success (98.7 percent accuracy). Although test averages only predicted failure 5.6 percent of the time, they were more powerful predictors of NCLEX-RN success than the number of Cs in nursing courses. Moreover, students who failed NCLEX-RN had significantly lower test averages and nearly double the number of Cs in nursing courses compared with those who passed NCLEX-RN. Daley et al. (2003) also found that those who passed NCLEX-RN had significantly higher didactic medical/surgical course grades than those who failed. To progress in the nursing curriculum at Salisbury University, students must achieve an average of 70 percent in didactic courses and a minimum grade of C in clinical courses; furthermore, they may only repeat one nursing course. Very few students receive grades of C or lower in clinical nursing courses, largely due to the more subjective nature of performance in practice settings (Walsh & Seldomridge, 2004). The combination of these factors reduces the number of students with weaker grades who complete the nursing curriculum and take NCLEX-RN and may explain why test averages in the medical/surgical courses were superior to the number of C grades in predicting NCLEX-RN success in this study. Regarding senior level indicators, performance on NLNCATBS taken 2 weeks before graduation was the best predictor of both NCLEX-RN success (94.7 percent) and failure (25 percent). When combined with test average from the advanced medical/surgical course, accuracy of prediction improved to 94 percent for those who passed NCLEX-RN and to 33.3 percent for those who failed. However, it should be noted that the NLNCATBS is not based on the NCLEX-RN test plan and measures overall achievement at the end of the program (National League for Nursing, 1997). Additionally, its usefulness as a predictor is limited because the results are not available until several weeks after graduation. When entering all study variables into the model, the NLNCATBS and grade in Pathophysiology predicted 93.3 percent of the NCLEX-RN successes and 50 percent of the failures. The results of this investigation suggest that success on NCLEX-RN can be predicted with a high degree of accuracy but that failure cannot. While the notion of identifying potential NCLEX-RN failures with more certainty is desirable for implementation
SELDOMRIDGE AND DIBARTOLO
of remedial strategies, specific predictors of failure are much less convincing due to the methodological problems with analysis of this relatively small group. In the meantime, results of this study suggest that faculty be alert to students with Cs in Pathophysiology and a pattern of marginal test performance in medical/surgical didactic courses in the junior year so that remedial interventions can be implemented. By examining additional data, such as test averages from all nursing didactic courses, which were not available for this study, it may be possible to improve the accuracy of predicting NCLEX-RN failure. However, even if prediction of failure were improved, there are no clear data to indicate that categorizing students at risk for NCLEX-RN failure motivates them (Hanks, 1999). In fact, singling out students may frighten them unnecessarily or become a self-fulfilling prophecy. Therefore, it may be more prudent to provide an intervention for all students as early in the curriculum as possible, which continues until graduation. However, more research is needed to elucidate whether the use of remedial strategies should be directed only to the students identified as at risk or used with all students and which strategies are the most effective at which points in the curriculum. Currently at the authors’ institution, beginning with the first semester of nursing courses, students subscribe to a computer software package for tutoring and testing. It is hoped that requiring periodic use of this software will assist students with mastery of course content, improve their comfort with the computerized testing format, and ultimately provide them with detailed information about their readiness for NCLEX-RN. Admission criteria have also been revised based on the findings of this study. Applicants are grouped into pools based first on their entry status (native or transfer) and then on the strength of their academic performance in approximately 60 credits of prerequisites. Data such as number of credits and grades in courses taken at 4-year versus community colleges, and patterns of withdrawals or repeating science courses are taken into consideration when ranking candidates. Native students with the best overall credentials are offered admission first, followed by transfer students, until all spaces are filled. Admission is usually denied if students have more than one D or F in a science course. A minimum grade of C in Pathophysiology continues to be required. The effect of these new standards will not be known until summer 2005 when two cohorts admitted under them will have
367
NCLEX-RN PREDICTORS
taken the licensing exam, producing a sample large enough for analysis. While it may be wise for programs to revise admission criteria, the realities of the nursing shortage must also be considered. Supply and demand in the health care industry have always influenced admission and progression policies. When there is a surplus of job opportunities, many programs experience an influx of applications and can afford to be more selective. Programs, especially those receiving public funding, are under pressure to increase enrollments to meet the demand for qualified nurses.A careful balance must be maintained to insure selection of applicants who can be successful in the curriculum and on the licensing examination. Admission and progression policies should be reviewed regularly and revised as needed based on new research findings. Certain limitations must be acknowledged when interpreting the findings of this study. First, the subjects were all enrolled in the traditional undergraduate baccalaureate program at one mid-Atlantic public, comprehensive university so the results may not be generalizable to other programs. Second, subjects took the NLNCATBS, a paper and pencil standardized test designed to measure overall student knowledge at the end of the curriculum (NLN, 1997). It is just one source of data about student performance and does not reflect the emphasis of the NCLEX-RN test plan. Third, potential problems with multicollinearity could occur because several predictor variables were highly correlated (Tabachnick & Fidell, 1996).
Conclusion
Predicting success and failure on the NCLEX-RN will continue to be an area of inquiry despite some conflicting results and difficulties in data comparisons across curricula and populations. Failing the licensure examination adversely affects students, educational programs, and society and is particularly troubling given the current nursing shortage. Early identification of factors that restrict or support academic achievement is a priority for nurse educators (Jeffreys, 1998). Increased diversity among learners creates special challenges for nursing programs, and as “student populations become more diverse, predictors might change or previously unidentified variables may become more important in predicting completion of a nursing program and NCLEX-RN success” (Beeson & Kissling, 2001). While there is no one formula to be applied
across curricula, it is apparent that predicting success in the post-1998 pass/fail era of NCLEX-RN is much more accurate than predicting failure. The challenge of identifying students at risk for failure using logistic regression will continue because it is “sensitive to the relative sizes of the two component groups and will always favor classification into the larger group” (Hosmer & Lemeshow, 1989). Comparison across studies and cohorts is also complicated by periodic revisions to the NCLEX-RN test plan, question format, and passing standard. While one could easily assume that the more data available about student performance, the greater the accuracy of prediction, even with data through graduation the accuracy of predicting NCLEX-RN failure in this study was 50 percent— about as scientific as the toss of a coin. Although it remains difficult, if not impossible, to arrive at a comprehensive model to predict NCLEX-RN failure because of the complex interaction of demographic, academic, and psychosocial variables that cannot possibly be addressed in any one study, replication of this research by other baccalaureate programs would enhance generalizability of the models proposed thus far. Variables not addressed in this study that may have accounted for variability in results include readiness to take the exam, experience with computer testing, emotional distress, fatigue/illness, family responsibilities, work obligations, age, ethnic background, and primary language. Postgraduation issues of interest are length of time between graduation and taking the NCLEX-RN, estimated hours of study time, use of specific study strategies, and attitude toward taking the examination. These factors are undoubtedly relevant to some degree, with elapsed time between graduation and taking the NCLEX-RN of particular importance in light of a recent study by the National Council of State Boards of Nursing (2002b), which found the more time between graduation and initial NCLEX-RN testing, the greater likelihood of failure. Research to identify additional preadmission academic factors, including credit load each semester and number of course withdrawals or repeats, as well as nonacademic factors such as hours per week of employment, employment in health-related fields, and nature of family responsibilities, is warranted. Examining test averages from all required didactic nursing courses would provide a pattern of test performance that might improve prediction of failure and assist students in actively identifying the need for remediation as opposed to being singled out by faculty. Faculty knowledge about the NCLEX-RN test plan, test items, passing
368
SELDOMRIDGE AND DIBARTOLO
standards, and how this is communicated to students has yet to be explored. Evaluation of the type and timing of interventions to improve NCLEX-RN perfor-
mance is needed to discover which are most effective or whether targeted interventions should be implemented at all.
References Arathuzik, D., & Aber, C. (1998). Factors associated with National Council Licensure Examination-Registered Nurse success. Journal of Professional Nursing, 14, 119-126. Beeman, P. B., & Waterhouse, J. K. (2001). NCLEX-RN performance: Predicting success on the computerized examination. Journal of Professional Nursing, 17, 158-165. Beeson, S. A., & Kissling, G. (2001). Predicting success for baccalaureate graduates on the NCLEX-RN. Journal of Professional Nursing, 17, 121-127. Byrd, G., Garza, C., & Nieswiadomy, R. (1999). Predictors of successful completion of a baccalaureate nursing program. Nurse Educator, 24, 33-37. Daley, L. K., Kirkpatrick, B. L., Frazier, S. K., Chung, M. L., & Moser, D. K. (2003). Predictors of NCLEX-RN success in a baccalaureate nursing program as a foundation for remediation. Journal of Nursing Education, 42, 390-398. Endres, D. (1997). A comparison of predictors of success on NCLEX-RN for African American, foreign-born, and white baccalaureate graduates. Journal of Nursing Education, 36, 365-371. Hanks, C. (1999). How well does a test predict an outcome? Computers in Nursing, 17, 241-243. Hosmer, D. W., & Lemeshow, S. (1989). Applied logistic regression. New York: Wiley. Jeffreys, M. R. (1998). Predicting nontraditional student retention and academic achievement. Nurse Educator, 23, 42-48. Lauchner, K., Newman, M., & Britt, R. (1999). Predicting licensure success with a computerized comprehensive nursing exam: The HESI Exit Exam. Computers in Nursing, 17, 120-125. Maryland Board of Nursing. (2002). NCLEX-RN Firsttime candidate performance, fiscal year 2002. Retrieved February 2, 2004, from http://www.mbon.org/main.php?v⫽ norm&p⫽0&c⫽education/nclex_rn_stats_fy02.html.
Morrison, S., Free, K. W., & Newman, M. (2002). Do progression and remediation policies improve NLCEX-RN pass rates? Nurse Educator, 27, 94-96. National Council of State Boards of Nursing, Inc. (2002a). Number of candidates taking NCLEX-RN examination and percent passing, 2002, by type of candidate. Retrieved February 3, 2004, from http://www.ncsbn.org/research _stats/nclex.asp. National Council of State Boards of Nursing, Inc. (2002b, March). The NCLEX-RN delay pass rate study. Retrieved February 3, 2004, from http://www.ncsbn.org/pdfs/ RecentNCLEXResearch_Web_Testing017B02.pdf. National League for Nursing (1997). Guide for the interpretation of NLN test reports. (Publication No. 17-1932). New York, NY: Author. Nelson, R. (2002). U.S. nursing shortage a “national security concern.” The Lancet, 360, 855. Newman, M., Britt, R., & Lauchner, K. (2000). Predictive accuracy of the HESI exit exam: A follow-up study. Computers in Nursing, 18, 132-136. Nibert, A., & Young, A. (2001). Predicting NCLEX-RN success with the HESI exit exam: A third study. Computers in Nursing, 19, 172-178. Roncoli, M., Lisanti, P., & Falcone, A. (2000). Characteristics of baccalaureate graduates and NCLEX-RN performance. Journal of the New York State Nurses Association, 31, 17-19. Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics. New York: HarperCollins. Walsh, C. M., & Seldomridge, L. A. (In press). Clinical grades: Upward bound. Journal of Nursing Education. Waterhouse, J. K., & Beeman, P. B. (2003). Predicting NCLEX-RN success: Can it be simplified? Nursing Education Perspectives, 24, 35-3.
The current shortage of nurses and declining national pass rate on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) has heightened educators’ interest in identifying students at risk for failure. A retrospective descriptive study was conducted at a rural, public baccalaureate nursing program to determine variables that best predict NCLEX-RN success and failure. Data collected from 1998 through 2002 (N ⴝ 186) included entry as native or transfer student, preadmission grade point average (GPA), GPA after completing one semester of nursing courses, cumulative GPA at graduation, grades earned in prerequisite and core nursing courses, test averages in beginning and advanced medical/surgical nursing courses, and performance on the National League for Nursing Comprehensive Achievement Test for Baccalaureate Students (NLNCATBS). Logistic regression analysis revealed that a combination of test average in advanced medical/ surgical nursing and percentile score on the NLNCATBS predicted 94.7 percent of NCLEX-RN passes and 33.3 percent of failures. The combination of NLNCATBS score and pathophysiology grade predicted 93.3 percent of NCLEX-RN passes and 50 percent of failures. Although success could be accurately predicted across all models, predicting failure was far more difficult. (Index words: NCLEX-RN predictors; Nursing licensure; Baccalaureate education) J Prof Nurs 20:361-368, 2004. © 2004 Elsevier Inc. All rights reserved.
T
HE CURRENT nursing shortage, acknowledged as one of significant magnitude, is projected to worsen, reaching a 20 percent deficit by the year 2020 (Nelson, 2002). This crisis, coupled with the climate of health care reform, has placed unprecedented pressure on nursing programs to increase the supply of qualified graduates. Entry of competent nurses into the work*Associate Professor of Nursing, Salisbury University Department of Nursing, Salisbury, MD. †Assistant Professor of Nursing, Salisbury University Department of Nursing, Salisbury, MD. Address correspondence and reprint requests to Dr. Seldomridge: Salisbury University Department of Nursing, 1101 Camden Ave., Salisbury, MD 21801. E-mail: [email protected] 8755-7223/$30.00 © 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.profnurs.2004.08.005
force is facilitated by first-time success on the National Council Licensure Examination for Registered Nurses (NCLEX-RN). NCLEX-RN pass rates remain an important criterion for measuring effectiveness of a nursing program (Arathuzik & Aber, 1998). Acceptable NCLEX-RN pass rates are necessary to receive continued state board of nursing approval, maintain accreditation, attract qualified students, and uphold a program’s reputation and obligation to protect the public from unsafe care (Beeson & Kissling, 2001). Recent data depict a steady decline in NCLEX-RN pass rates for first-time candidates (Maryland Board of Nursing, 2002; National Council of State Boards of Nursing, 2002a), from an 87.7 percent pass rate in 1997 to an 82.2 percent pass rate in 2002. Examining the relationships among admissions criteria, performance in nursing courses, and NCLEX-RN results assists programs in strengthening admissions and progression policies so that students selected to begin nursing courses have the best possibility of completing the typically rigorous curriculum. Once enrolled in nursing courses, early identification of at-risk students is useful so they may receive interventions to improve likelihood of initial NCLEX-RN success. This is essential in an environment where limited clinical placements, qualified faculty, and financial resources can greatly restrict enrollment capacity (Byrd, Garza, & Neiswiadomy, 1999). Initial NCLEX-RN success obviously helps students avoid the consequences of failure, such as the negative impact on self-esteem and financial setbacks when career plans are delayed. Predicting NCLEX-RN success is vital for educators and administrators because potential applicants use the passing rate to evaluate and compare programs. Furthermore, nursing programs that exhibit difficulty with maintaining a satisfactory pass rate may also face the possibility of probationary status with the state board of nursing (Endres, 1997). The purposes of this retrospective study were to identify the best models for predicting NCLEX-RN success and failure at three points in the nursing curriculum: preadmission, after the completion of 1 year of nursing courses, and immediately prior to gradua-
Journal of Professional Nursing, Vol 20, No 6 (November–December), 2004: pp 361-368
361
362
SELDOMRIDGE AND DIBARTOLO
tion. Additionally, this study aims to answer an elusive question—to what extent, if any, can NCLEX-RN success and failure be accurately predicted?
Review of the Literature
Since the NCLEX-RN was instituted as the nationally approved licensing examination for registered nurses in 1982, a multitude of research has been conducted to identify factors associated with the greatest likelihood of success. Studies have explored a variety of academic and nonacademic predictors. Academic predictors have included high school rank, Scholastic Aptitude Test (SAT) scores, American College Test (ACT) scores, entering GPA, freshman and prenursing GPA, nursing GPA, GPA upon graduation, achievement in specific nursing courses, science course grades, Mosby AssessTest scores, National League for Nursing Achievement Test scores, and Watson-Glaser Critical Thinking Appraisal scores. Nonacademic predictors of success on NCLEX-RN have ranged from age, sex, ethnicity, primary language, and test anxiety to selfesteem and self-efficacy. Although several studies were reported after the initial state board exam was revised to become the NCLEX-RN in 1982, few predictive studies of NCLEX-RN success in baccalaureate students have been conducted since the examination was modified in 1988 to the pass/fail format. Even fewer studies have been conducted since the format was changed to computer adaptive testing (CAT) in April of 1994. Beeson and Kissling (2001) found that students who passed the NCLEX-RN had fewer grades of C or below in nursing courses, scored higher on the Mosby AssessTest and had significantly higher average GPAs than those who failed. Beeman and Waterhouse (2001) identified seven significant predictor variables, with the total number of C⫹ or lower grades earned in nursing didactic courses as the best predictor, followed by grades in several individual nursing courses. Waterhouse and Beeman (2003) tested the accuracy of a simple risk appraisal instrument in predicting NCLEX-RN success; nearly 61 percent of failures were correctly classified; 72 percent were accurately classified overall. In a recent study, final course grade for a didactic, senior-level medical/surgical nursing course and cumulative program GPA were the only program variables consistently associated with NCLEX-RN success (Daley, Kirkpatrick, Frazier, Chung, & Moser, 2003). Related studies have recently investigated the Health Education Systems, Inc. (HESI) Exit Examina-
tion as a predictor of NCLEX-RN success. The HESI Exit Examination is a computerized comprehensive examination that uses a proprietary mathematical model to offer immediate feedback to students in the form of a probability score. The exam has been used as both a measure of preparedness to take the NCLEX-RN and a benchmark for remediation (Morrison, Free, & Newman, 2002). Studies have found HESI exam scores to be highly predictive of NCLEX-RN success (Lauchner, Newman, & Britt, 1999; Newman, Britt, & Lauchner, 2000; Nibert & Young, 2001) with results in the 96 to 99 percent range. It is evident from the literature that while the number and variety of variables that may affect NCLEX-RN success has steadily increased, the accuracy of predictions of NCLEX-RN success has declined overall since 1988 when studies were generally able to correctly classify up to 86 percent of students (Waterhouse & Beeman, 2003). Since that time, fewer studies have been published; this may reflect the difficulty of predicting success based exclusively on pass/ fail scores that usually fall into the “pass” category. Furthermore, the adoption of the computerized testing format in 1994 has complicated the search for the best set of predictors by requiring that students have some expertise in computerized test-taking (Beeman & Kissling, 2001). The implementation of a modified test plan and a more rigorous passing standard in 1998 by the National Council of State Boards of Nursing has also made comparisons among cohorts difficult. The following research questions were developed to guide this study: ●
●
●
● ●
Which prerequisites to the nursing major are the best predictors of success/failure on the NCLEX-RN? Which variables occurring after completion of junior-year nursing courses best predict success/failure on the NCLEX-RN? Which variables occurring between junior-year nursing courses and graduation best predict success/failure on the NCLEX-RN? Which overall combination of variables best predicts success/failure on the NCLEX-RN? Can success or failure on the NCLEX-RN be accurately predicted? Methodology
The sample consisted of graduates of the traditional baccalaureate nursing program at a rural, mid-Atlantic
363
NCLEX-RN PREDICTORS
public institution from 1998 through 2002. To minimize the potential effects of periodic changes in the NCLEX-RN, the sample was limited to students taking the computerized version according to the modified test plan and passing standards instituted in April 1998. After approval from the University Committee on Human Volunteers, data were collected from student records and official reports of NCLEX-RN results. One graduate was excluded from the sample due to missing NCLEX-RN results. The final sample consisted of 186 students, 174 women and 12 men, evenly divided between native students (beginning their college careers at this university, n ⫽ 95 or 51 percent) and transfer students (n ⫽ 91 or 49 percent). Of the 186 subjects, 150 (80.6 percent) passed NCLEX-RN on the first attempt, and 36 (19.3 percent) did not. The dependent variable was the NCLEX-RN outcome, noted as pass or fail. A total of 13 independent variables were selected as predictors of NCLEX-RN success or failure based on a thorough review of the literature. The independent variables, divided into three groups for analysis by time period, included preadmission indicators used to make decisions about starting the nursing curriculum, indicators available after completing junior-year nursing courses, and indicators available prior to graduation. Independent variables of relevance to the admissions process were grades in specific prerequisites, including Anatomy and Physiology I, Pathophysiology, and Chemistry (4 credits each) and Statistics (3 credits); overall pattern of performance in these prerequisites measured by the number of grades of C or lower (0-4); and cumulative GPA prior to beginning nursing courses. GPAs and course grades were based on a scale of 0 to 4 (0 ⫽ F, 4 ⫽ A). SAT scores were excluded from this study because they were not available for transfer students who comprised half the sample. It is important to note that pathophysiology is the only prerequisite course in which a minimum grade of C is required. Independent variables at the conclusion of the junior year included cumulative GPA after completion of the first semester of junior nursing courses (selected because of an observed decline in academic performance in this particular semester), number of Cs in junior nursing courses (0-9), and test averages (out of 100 percent) in two medical/surgical didactic nursing courses (Adult Health I & II, 3 credits each). Test averages, not examined in any previous research as a predictor variable, were selected as precise measures of performance in a testing setting. Test averages from the medical/surgical didactic courses were used because of
their history of discriminating between passing and failing students. Independent variables selected for study in the senior year prior to graduation included number of Cs in all nursing courses (0-17) and percentile score on the National League for Nursing Comprehensive Achievement Test for Baccalaureate Students (NLNCATBS)—a paper and pencil test taken 2 weeks before graduation that compares individual achievement to a national sampling of students completing baccalaureate nursing programs. DATA ANALYSIS
All data were entered into SPSSX version 11.0 (SPSS, Chicago, IL) for analysis. Descriptive statistics were used to characterize the sample and to identify possible predictors of NCLEX-RN outcome. Pearsonproduct moment correlation coefficients were computed for NCLEX-RN success with continuous variables. The two-sample t test was used as a screening procedure to identify possible predictors of NCLEX-RN success. Group differences were also examined using the nonparametric Mann-Whitney test because some variables were not normally distributed. Since the results of this comparison were of the same magnitude of significance as the two-sample t test, only the latter are reported here. Logistic regression, a multivariate technique, was chosen to combine continuous and categorical predictor variables to predict outcome on a categorical variable (Hosmer & Lemeshow, 1989) such as the NCLEX-RN.
Results
Student characteristics and their association with NCLEX-RN performance are summarized in Tables 1 and 2. Percentile score on the NLNCATBS showed the highest correlation with NCLEX-RN success, (r ⫽ .452, P ⫽ .000), followed by grade in Pathophysiology (r ⫽ .377, P ⫽ .000), test average in the advanced medical/surgical course (r ⫽ .307, P ⫽ .000), and test average in the introductory adult medical/surgical course (r ⫽ .303, P ⫽ .000). Patterns of low grades in prerequisites (r ⫽ ⫺.245, P ⫽ .002) and nursing courses (r ⫽ ⫺.342, P ⫽ .000) were negatively correlated with NCLEX-RN success. Those successful on NCLEX-RN had significantly higher test averages in both medical/surgical nursing courses, had higher GPAs (upon entry to nursing courses, at the end of 1 semester of nursing courses, and on program comple-
364 TABLE
SELDOMRIDGE AND DIBARTOLO
tistical significance. Students who passed the NCLEX-RN were correctly classified 100 percent of the time based on their grade in pathophysiology. However, students who failed the NCLEX-RN were correctly classified only 2.8 percent of the time. The overall accuracy in prediction was 81.2 percent. For each letter grade increase in Pathophysiology, the odds of passing the NCLEX-RN improved by nearly 5 times. The second model, using data available after completion of junior-year nursing courses, included test averages from two medical/surgical didactic courses (Adult Health I & II), number of Cs in junior-year nursing courses (maximum ⫽ 9) and overall GPA after 1 semester of nursing courses. Test averages from both medical/surgical didactic courses predicted NCLEX-RN pass 98.7 percent of the time but correctly predicted NCLEX-RN failure only 5.6 percent of the time. The overall prediction accuracy was 80.6 percent. For every 1-point increase in each test average, the odds of passing NCLEX-RN improved by one time. Those passing NCLEX-RN had higher mean test averages in both courses compared to those who failed NCLEX-RN. Students who failed NCLEX-RN had twice the number of Cs in the junior-year nursing courses as students who passed NCLEX-RN. Of the students with five or more Cs (n ⫽ 11) in junior-year nursing courses, 50 percent passed NCLEX-RN, while those without any Cs (n ⫽ 45) had a pass rate of 100 percent. The third model examined predictors available on completion of the senior year, including test averages in both medical/surgical courses, number of Cs in all nursing courses (maximum ⫽ 17), and percentile score on the standardized NLNCATBS. Score on the NLNCATBS, which entered the model on Step 1, correctly predicted 94.7 percent of NCLEX-RN passes and 25 percent of NCLEX-RN failures, with an overall pre-
1. Descriptive Statistics and Correlations with Passing the NCLEX-RN (n ⫽ 186) Variable
Mean
Grade-Anatomy/physiology 2.87 GradePathophysiology 2.98 Grade-Chemistry I 2.88 Grade-Statistics 3.07 Number of Cs or lower in pre-requisites 1.16 Test average-Adult Health I 79.01 Test average-Adult Health II 76.56 GPA-preadmission 3.07 GPA-after one semester of nursing 3.04 Number of Cs in junior nursing courses 2.2 Number of Cs or lower in all nursing courses 4.0 GPA-Final 3.11 Percentile score on NLNCATBS 48.4
Standard Deviation
Correlation with NCLEX-RN
.77
.131
.74 .81 .81
.377** .009 .126
1.13
⫺.222**
5.83
.303**
5.13 .41
.307** .195**
.43
.309**
1.61
⫺.306**
3.30 .34
⫺.342** .258**
25.58
.452**
**p ⬍ .01, two-tailed.
tion), and performed at a higher level on the NLNCATBS. Logistic regression analyses were performed to determine the most parsimonious set of predictors for each of three groupings of variables and when all variables were entered into the model. These results are reported in Table 3. Variables were selected for inclusion using the stepwise method with a significance level of .15 for entry into or removal from the model. The first model tested preadmission variables: grade in Pathophysiology, preadmission GPA, and number of Cs or lower in Anatomy & Physiology, Statistics, Chemistry I, and Pathophysiology. Performance in Pathophysiology was the only variable to achieve staTABLE
2. Bivariate Comparisons Between NCLEX-RN Outcome and Continuous Variables NCLEX-RN Outcome
Two Sample t Test Results
Variable
Passed (n ⫽ 150) Mean (SD)
Failed (n ⫽ 36) Mean (SD)
t(df ⫽ 184) Value
P
Preadmission GPA Grade Pathophysiology Number of Cs or lower in prerequisites Test average Adult Health I Test average Adult Health II GPA after one semester of nursing courses Number of Cs-junior nursing courses Number of Cs-all nursing courses Final GPA Percentile score NLNCATBS
3.11 (.42) 3.12 (.68) 1.03 (1.07) 79.87 (5.77) 77.34 (5.12) 3.09 (.42) 1.81 (1.59) 3.43 (3.09) 3.15 (.35) 54.06 (24.09)
2.91 (.32) 2.42 (.73) 1.67 (1.24) 75.41 (4.62) 73.35 (3.80) 2.79 (.37) 3.06 (1.26) 6.23 (3.16) 2.93 (.24) 24.86 (16.74)
⫺2.70 (184) ⫺5.52 (184) 3.09 (184) ⫺4.32 (184) ⫺4.38 (184) ⫺3.97 (184) 4.37 (184) 4.95 (184) ⫺3.62 (184) ⫺6.88 (184)
.008 .000 .002 .000 .000 .000 .000 .000 .000 .000
365
NCLEX-RN PREDICTORS
TABLE
3. Stepwise Logistic Regression Results: Odds Ratios and Confidence Intervals
Regression Models and Variables
Preadmission Grade in Pathophysiology Constant End of junior year Test average advanced med/surg Test average intro med/surg Constant End of final semester NLNCATBS score Test average advanced med/surg Constant All time periods NLNCATBS score Grade in Pathophysiology Constant
Odds Ratio
95% Confidence Intervals
4.47 .066
(2.37, 8.45)**
1.12 1.09 .000
(1.00, 1.26)** (.99, 1.21)
1.057
(1.032, 1.08)**
1.117 .000
(1.01, 1.24)*
1.057 3.15 .022
(1.032, 1.08)** (1.56, 6.36)**
NOTE. Advanced med/surg ⫽ Adult Health II nursing course; intro med/surg ⫽ Adult Health I nursing course. Odds ratios reflect one-unit change in the independent variable *p ⱕ .05, **p ⱕ .001.
diction accuracy of 81.2 percent. Test average for Adult Health II entered on Step 2, correctly predicting 94 percent of NCLEX-RN passes and 33.3 percent of NCLEX-RN failures, with an overall accuracy of 82.3 percent. Score on the NLNCATBS contributed most to the prediction (P ⫽ .000). For every 10-point increase in percentile score, the odds of passing NCLEX-RN increased by nearly 11 times. Upon program completion, students without any grades of C in nursing courses (n ⫽ 32) had an NCLEX-RN pass rate of 99.3 percent. Neither of the two students with the highest number of C grades (13 of a possible 17) passed the licensure examination on the first attempt. The final model for NCLEX-RN success used variables from all three time periods. At the end of Step 1, score on the NLNCATBS correctly predicted success on the NCLEX-RN 94.7 percent of the time and correctly predicted failure 25 percent of the time. Grade in Pathophysiology entered the model at Step 2, improving the prediction for NCLEX-RN failures to 50 percent; prediction of success dropped slightly to 93.3 percent. Overall success in predicting NCLEX-RN performance was 84.9 percent. No other variables entered the model. Discussion
The purposes of this study were to identify the best models for predicting NCLEX-RN outcome based on
data from several time points (on preadmission, after completion of junior-level nursing courses, and immediately prior to graduation) and to determine the extent to which NCLEX-RN success or failure can be accurately predicted. The results of this investigation suggest that while several variables were very accurate in predicting success (grade in Pathophysiology, test averages in two medical/surgical nursing didactic courses, and score on a National League for Nursing achievement test), they were much less accurate in predicting failure. The most accurate combination was the score on the NLNCATBS and Pathophysiology grade; this predicted failures 50 percent of the time. These findings are consistent with other recent studies (Beeman & Waterhouse, 2001; Beeson & Kissling, 2001; Byrd et al., 1999; Daley et al, 2003) that also reported a substantial disparity in accuracy of predicting success versus failure. The pass/fail nature of NCLEX-RN limits the use of more sophisticated statistical procedures, and the disproportionately low numbers who fail NCLEX-RN compared to those who pass also adversely impacts the degree to which failure can be predicted. Unless schools admit an exceedingly high number of unqualified applicants and/or provide such low-quality education that more than 20 percent of students fail the licensing exam (Hanks, 1999), predicting NCLEX-RN failure will continue to be a challenge. At this and many other universities, students are selected during their sophomore year to enter upper division nursing courses. Identifying predictors of success at this point aids in admission decisions, particularly during times of a burgeoning applicant pool for a limited number of spaces. In the preadmission phase, grade in Pathophysiology was the only significant predictor variable, confirming the need to require a minimum grade of C to be considered. Although GPA in prerequisite courses was not a statistically significant predictor of NCLEX-RN success, there is support for requiring a GPA of at least 2.5 of 4.0. As noted in this and other studies (Beeson & Kissling, 2001; Daley et al., 2003; Endres, 1997; Roncoli, Lisanti, & Falcone, 2000), students who passed NCLEX-RN had significantly higher GPAs than those who failed. Because pattern of performance in science courses appears to be important in NCLEX-RN success, it might be valuable to use GPA in science courses rather than overall GPA as one of the admission criteria. When and where students take science courses may reveal important information about their background and abilities. Often students who perceive themselves as weak in science take them during summer or winter sessions when they
366
have no other academic commitments or enroll at community colleges where such courses are believed to be less rigorous. After completion of the junior nursing courses, test averages in the introductory and advanced medical/ surgical courses best predicted NCLEX-RN success (98.7 percent accuracy). Although test averages only predicted failure 5.6 percent of the time, they were more powerful predictors of NCLEX-RN success than the number of Cs in nursing courses. Moreover, students who failed NCLEX-RN had significantly lower test averages and nearly double the number of Cs in nursing courses compared with those who passed NCLEX-RN. Daley et al. (2003) also found that those who passed NCLEX-RN had significantly higher didactic medical/surgical course grades than those who failed. To progress in the nursing curriculum at Salisbury University, students must achieve an average of 70 percent in didactic courses and a minimum grade of C in clinical courses; furthermore, they may only repeat one nursing course. Very few students receive grades of C or lower in clinical nursing courses, largely due to the more subjective nature of performance in practice settings (Walsh & Seldomridge, 2004). The combination of these factors reduces the number of students with weaker grades who complete the nursing curriculum and take NCLEX-RN and may explain why test averages in the medical/surgical courses were superior to the number of C grades in predicting NCLEX-RN success in this study. Regarding senior level indicators, performance on NLNCATBS taken 2 weeks before graduation was the best predictor of both NCLEX-RN success (94.7 percent) and failure (25 percent). When combined with test average from the advanced medical/surgical course, accuracy of prediction improved to 94 percent for those who passed NCLEX-RN and to 33.3 percent for those who failed. However, it should be noted that the NLNCATBS is not based on the NCLEX-RN test plan and measures overall achievement at the end of the program (National League for Nursing, 1997). Additionally, its usefulness as a predictor is limited because the results are not available until several weeks after graduation. When entering all study variables into the model, the NLNCATBS and grade in Pathophysiology predicted 93.3 percent of the NCLEX-RN successes and 50 percent of the failures. The results of this investigation suggest that success on NCLEX-RN can be predicted with a high degree of accuracy but that failure cannot. While the notion of identifying potential NCLEX-RN failures with more certainty is desirable for implementation
SELDOMRIDGE AND DIBARTOLO
of remedial strategies, specific predictors of failure are much less convincing due to the methodological problems with analysis of this relatively small group. In the meantime, results of this study suggest that faculty be alert to students with Cs in Pathophysiology and a pattern of marginal test performance in medical/surgical didactic courses in the junior year so that remedial interventions can be implemented. By examining additional data, such as test averages from all nursing didactic courses, which were not available for this study, it may be possible to improve the accuracy of predicting NCLEX-RN failure. However, even if prediction of failure were improved, there are no clear data to indicate that categorizing students at risk for NCLEX-RN failure motivates them (Hanks, 1999). In fact, singling out students may frighten them unnecessarily or become a self-fulfilling prophecy. Therefore, it may be more prudent to provide an intervention for all students as early in the curriculum as possible, which continues until graduation. However, more research is needed to elucidate whether the use of remedial strategies should be directed only to the students identified as at risk or used with all students and which strategies are the most effective at which points in the curriculum. Currently at the authors’ institution, beginning with the first semester of nursing courses, students subscribe to a computer software package for tutoring and testing. It is hoped that requiring periodic use of this software will assist students with mastery of course content, improve their comfort with the computerized testing format, and ultimately provide them with detailed information about their readiness for NCLEX-RN. Admission criteria have also been revised based on the findings of this study. Applicants are grouped into pools based first on their entry status (native or transfer) and then on the strength of their academic performance in approximately 60 credits of prerequisites. Data such as number of credits and grades in courses taken at 4-year versus community colleges, and patterns of withdrawals or repeating science courses are taken into consideration when ranking candidates. Native students with the best overall credentials are offered admission first, followed by transfer students, until all spaces are filled. Admission is usually denied if students have more than one D or F in a science course. A minimum grade of C in Pathophysiology continues to be required. The effect of these new standards will not be known until summer 2005 when two cohorts admitted under them will have
367
NCLEX-RN PREDICTORS
taken the licensing exam, producing a sample large enough for analysis. While it may be wise for programs to revise admission criteria, the realities of the nursing shortage must also be considered. Supply and demand in the health care industry have always influenced admission and progression policies. When there is a surplus of job opportunities, many programs experience an influx of applications and can afford to be more selective. Programs, especially those receiving public funding, are under pressure to increase enrollments to meet the demand for qualified nurses.A careful balance must be maintained to insure selection of applicants who can be successful in the curriculum and on the licensing examination. Admission and progression policies should be reviewed regularly and revised as needed based on new research findings. Certain limitations must be acknowledged when interpreting the findings of this study. First, the subjects were all enrolled in the traditional undergraduate baccalaureate program at one mid-Atlantic public, comprehensive university so the results may not be generalizable to other programs. Second, subjects took the NLNCATBS, a paper and pencil standardized test designed to measure overall student knowledge at the end of the curriculum (NLN, 1997). It is just one source of data about student performance and does not reflect the emphasis of the NCLEX-RN test plan. Third, potential problems with multicollinearity could occur because several predictor variables were highly correlated (Tabachnick & Fidell, 1996).
Conclusion
Predicting success and failure on the NCLEX-RN will continue to be an area of inquiry despite some conflicting results and difficulties in data comparisons across curricula and populations. Failing the licensure examination adversely affects students, educational programs, and society and is particularly troubling given the current nursing shortage. Early identification of factors that restrict or support academic achievement is a priority for nurse educators (Jeffreys, 1998). Increased diversity among learners creates special challenges for nursing programs, and as “student populations become more diverse, predictors might change or previously unidentified variables may become more important in predicting completion of a nursing program and NCLEX-RN success” (Beeson & Kissling, 2001). While there is no one formula to be applied
across curricula, it is apparent that predicting success in the post-1998 pass/fail era of NCLEX-RN is much more accurate than predicting failure. The challenge of identifying students at risk for failure using logistic regression will continue because it is “sensitive to the relative sizes of the two component groups and will always favor classification into the larger group” (Hosmer & Lemeshow, 1989). Comparison across studies and cohorts is also complicated by periodic revisions to the NCLEX-RN test plan, question format, and passing standard. While one could easily assume that the more data available about student performance, the greater the accuracy of prediction, even with data through graduation the accuracy of predicting NCLEX-RN failure in this study was 50 percent— about as scientific as the toss of a coin. Although it remains difficult, if not impossible, to arrive at a comprehensive model to predict NCLEX-RN failure because of the complex interaction of demographic, academic, and psychosocial variables that cannot possibly be addressed in any one study, replication of this research by other baccalaureate programs would enhance generalizability of the models proposed thus far. Variables not addressed in this study that may have accounted for variability in results include readiness to take the exam, experience with computer testing, emotional distress, fatigue/illness, family responsibilities, work obligations, age, ethnic background, and primary language. Postgraduation issues of interest are length of time between graduation and taking the NCLEX-RN, estimated hours of study time, use of specific study strategies, and attitude toward taking the examination. These factors are undoubtedly relevant to some degree, with elapsed time between graduation and taking the NCLEX-RN of particular importance in light of a recent study by the National Council of State Boards of Nursing (2002b), which found the more time between graduation and initial NCLEX-RN testing, the greater likelihood of failure. Research to identify additional preadmission academic factors, including credit load each semester and number of course withdrawals or repeats, as well as nonacademic factors such as hours per week of employment, employment in health-related fields, and nature of family responsibilities, is warranted. Examining test averages from all required didactic nursing courses would provide a pattern of test performance that might improve prediction of failure and assist students in actively identifying the need for remediation as opposed to being singled out by faculty. Faculty knowledge about the NCLEX-RN test plan, test items, passing
368
SELDOMRIDGE AND DIBARTOLO
standards, and how this is communicated to students has yet to be explored. Evaluation of the type and timing of interventions to improve NCLEX-RN perfor-
mance is needed to discover which are most effective or whether targeted interventions should be implemented at all.
References Arathuzik, D., & Aber, C. (1998). Factors associated with National Council Licensure Examination-Registered Nurse success. Journal of Professional Nursing, 14, 119-126. Beeman, P. B., & Waterhouse, J. K. (2001). NCLEX-RN performance: Predicting success on the computerized examination. Journal of Professional Nursing, 17, 158-165. Beeson, S. A., & Kissling, G. (2001). Predicting success for baccalaureate graduates on the NCLEX-RN. Journal of Professional Nursing, 17, 121-127. Byrd, G., Garza, C., & Nieswiadomy, R. (1999). Predictors of successful completion of a baccalaureate nursing program. Nurse Educator, 24, 33-37. Daley, L. K., Kirkpatrick, B. L., Frazier, S. K., Chung, M. L., & Moser, D. K. (2003). Predictors of NCLEX-RN success in a baccalaureate nursing program as a foundation for remediation. Journal of Nursing Education, 42, 390-398. Endres, D. (1997). A comparison of predictors of success on NCLEX-RN for African American, foreign-born, and white baccalaureate graduates. Journal of Nursing Education, 36, 365-371. Hanks, C. (1999). How well does a test predict an outcome? Computers in Nursing, 17, 241-243. Hosmer, D. W., & Lemeshow, S. (1989). Applied logistic regression. New York: Wiley. Jeffreys, M. R. (1998). Predicting nontraditional student retention and academic achievement. Nurse Educator, 23, 42-48. Lauchner, K., Newman, M., & Britt, R. (1999). Predicting licensure success with a computerized comprehensive nursing exam: The HESI Exit Exam. Computers in Nursing, 17, 120-125. Maryland Board of Nursing. (2002). NCLEX-RN Firsttime candidate performance, fiscal year 2002. Retrieved February 2, 2004, from http://www.mbon.org/main.php?v⫽ norm&p⫽0&c⫽education/nclex_rn_stats_fy02.html.
Morrison, S., Free, K. W., & Newman, M. (2002). Do progression and remediation policies improve NLCEX-RN pass rates? Nurse Educator, 27, 94-96. National Council of State Boards of Nursing, Inc. (2002a). Number of candidates taking NCLEX-RN examination and percent passing, 2002, by type of candidate. Retrieved February 3, 2004, from http://www.ncsbn.org/research _stats/nclex.asp. National Council of State Boards of Nursing, Inc. (2002b, March). The NCLEX-RN delay pass rate study. Retrieved February 3, 2004, from http://www.ncsbn.org/pdfs/ RecentNCLEXResearch_Web_Testing017B02.pdf. National League for Nursing (1997). Guide for the interpretation of NLN test reports. (Publication No. 17-1932). New York, NY: Author. Nelson, R. (2002). U.S. nursing shortage a “national security concern.” The Lancet, 360, 855. Newman, M., Britt, R., & Lauchner, K. (2000). Predictive accuracy of the HESI exit exam: A follow-up study. Computers in Nursing, 18, 132-136. Nibert, A., & Young, A. (2001). Predicting NCLEX-RN success with the HESI exit exam: A third study. Computers in Nursing, 19, 172-178. Roncoli, M., Lisanti, P., & Falcone, A. (2000). Characteristics of baccalaureate graduates and NCLEX-RN performance. Journal of the New York State Nurses Association, 31, 17-19. Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics. New York: HarperCollins. Walsh, C. M., & Seldomridge, L. A. (In press). Clinical grades: Upward bound. Journal of Nursing Education. Waterhouse, J. K., & Beeman, P. B. (2003). Predicting NCLEX-RN success: Can it be simplified? Nursing Education Perspectives, 24, 35-3.