- Email: [email protected]
An investigation of predictors of NCLEX-RN outcomes among nursing content standardized tests
Nurse Education Today 33 (2013) 1523–1528
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
An investigation of predictors of NCLEX-RN outcomes among nursing content standardized tests Yei-Jin Yeom ⁎ Department of Baccalaureate Nursing, Indiana State University, NU 328, 749 Chestnut Street, Terre Haute, IN 47809, United States
a r t i c l e
i n f o
Article history: Accepted 8 April 2013 Keywords: NCLEX-RN Success Failure Outcome Predictors Standardized tests
s u m m a r y In order to meet increased demands for qualified registered nurses and prevent negative effects from graduates' failure on the National Council Licensure Examination-Registered Nurse, it is important to promote students' success in the exam. The purpose of this study was to investigate effective predictors of NCLEX-RN outcomes on the first attempt among nursing content standardized tests (adult medical–surgical, fundamentals for nursing, pharmacology, maternal–newborn, nursing care of children, mental health, community health, and leadership and management) conducted throughout the nursing program. NCLEX-RN outcomes and individual adjusted scores on the standardized tests of 151 graduates from the traditional baccalaureate nursing program of a public university located in the Midwest from May 2010 to December 2011 were analyzed by a t-test and logistic regression. The participants included 118 graduates who passed and 33 graduates who failed the NCLEX-RN on the first attempt. Significant statistical differences were found between the two groups with NCLEX-RN success and failure in the individual adjusted scores on all of the standardized tests except the fundamental for nursing (p = .62) and nursing care of children (p = .759) standardized tests. In addition, logistic regression indicated that the overall regression models were significant in predicting both NCLEX-RN success and failure. Adult medical–surgical, pharmacology, and community health standardized tests were central to the prediction of both NCLEX-RN success and failure; however, a much lower percentage of NCLEX-RN failure than success was classified. It can be concluded that the adult medical–surgical, pharmacology, and community health standardized tests were effective in predicting NCLEX-RN success and not effective in predicting NCLEX-RN failure on the first attempt. The NCLEX-RN success predictors can be utilized to identify students at risk and provide early remediation. After early remediation is implemented, the comprehensive standardized tests may be used as a mid-point indicator of the remediation's effectiveness before taking the NCLEX-RN. © 2013 Elsevier Ltd. All rights reserved.
Introduction Nursing shortage is a critical issue in the United States (U.S.). The shortage is expected to get even worse since the demands for registered nurses (RN) are outstripping the supply. According to the American Association of Colleges of Nursing (AACN, 2012), there will be more than 581,500 new positions for RNs through 2018, demands for RNs in acute care will increase by 36% through 2020, and a lack of more than 260,000 RNs will affect the U.S. health care system by 2025. Even though demands for RNs have dramatically increased, the enrollment of baccalaureate nursing students increased only 5.7% in 2010. This increase in enrollment is not enough to meet the increased demands for RNs. Furthermore, due to the increase in the average age of RNs, it is expected that there will be an additional loss of a large number of RNs due to retirement over the next 20 years. While at the same time the demands for RNs will increase as baby boomers age and need health
⁎ Tel.: +1 812 230 6120; fax: +1 812 237 8022. E-mail address: [email protected]. 0260-6917/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.nedt.2013.04.004
care services during that time (AACN, 2011). One of the strategies to resolve the problem of the nursing shortage is the expansion of nursing program enrollments. However, enrollments remain limited due to inadequate numbers of nursing faculty (AACN, 2011). Despite this limitation, nursing programs are still accountable to mitigate the nursing shortage. In order to meet the nursing demands and lessen the nursing shortage, enhancing the National Council Licensure Examination-Registered Nurse (NCLEX-RN) pass rate is critical since the successful completion of this exam produces more RNs. Graduates are ensured to have competencies and can be in their nursing careers as entry-level RNs when they pass NCLEX-RN, which measures competencies required to provide quality nursing care and ensures the public's safety (National Council of State Boards of Nursing [NCSBN], 2012). Graduates' failures on the NCLEX-RN negatively affect stakeholders such as the graduates, health care organizations, and nursing programs. Graduates who fail the NCLEX-RN experience feelings of embarrassment, anxiety, loss of selfesteem, guilt, and social stigma, and those feelings may negatively affect the graduates' further NCLEX-RN attempts (Roa et al., 2011; Griffiths et al., 2004). Also, NCLEX-RN failure causes financial loss to the graduates.
1524
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
The graduates expect to be compensated for tuition and other school related expenses through work as RNs after graduation. However, if they fail the NCLEX-RN, they cannot work as an RN for a minimum of 45 days while they wait to be eligible for the next attempt. They also spend additional money for remediation and fees to retake the NCLEX-RN. Moreover, health care organizations expect that graduates will pass the NCLEX-RN when they hire these graduates. When the newly hired graduates fail the NCLEX-RN, health care organizations must fill positions vacated by graduates who have failed with temporary nursing staff or they must increase the overtime pay of existing employees. This causes not only higher costs for the organizations, but also a lack of qualified nurses within organizations and increases the staff–patient ratio (Roa et al., 2011; Greenpan et al., 2009). Without a doubt, nursing programs are affected by their graduates' NCLEX-RN failures. The NCLEX-RN pass rates on the first-attempt are a major factor used in evaluating the nursing programs' effectiveness and can influence the accreditation of nursing programs (Norton et al., 2006). Nursing programs do not meet the national benchmark standards, they are at risk of being placed on probation or losing their accreditation. Also, the nursing programs, colleges, and universities with low NCLEX-RN pass rates may face financial risks. Roa et al. (2011) and Chang et al. (2001) indicated that each nursing program's NCLEX-RN pass rates are public information, and prospective students may evaluate the pass rates to decide to which programs they will apply and enroll in to meet their needs. The nursing programs, colleges, and universities with lower pass rates may experience decreased enrollments which negatively affects their operational budgets through decreased tuition. Nursing programs are ethically responsible to their graduates who were qualified and met academic standards for graduation yet failed the NCLEX-RN (Roa et al., 2011; Catalano, 2009). Nursing programs are “challenged to facilitate knowledge development, ensure competence of their new graduates, and demonstrate organizational and curricular effectiveness through identification and remediation of students at high-risk of failure among first-time test candidates” (Norton et al., 2006, p. 322). With effective NCLEX-RN predictors, students' outcomes on the NCLEX-RN can be better predicted, and educators can better support students at risk through developing and providing early remediation resulting in improved NCLEX-RN pass rates. Such predictive measures can result in decreasing the nursing shortage and its negative effects. Currently, many nursing programs use standardized assessments purchased from outside companies to predict their students' first attempt outcome on the NCLEX-RN. Nursing educators need to distinguish the more powerful and effective subjects tested.
Literature Review In order to meet the demand of increased entry level RN's competence, the NCLEX-RN is reviewed by the NCLEX Examination Committee every three years. The minimum score for passing on the NCLEX-RN has been raised by the NCBSN. Because nursing programs have been challenged by this, educators focus upon finding strong predictors of NCLEX-RN outcome and providing interventions to ensure that their students pass the NCLEX-RN (Sifford and McDaniel, 2007). The American College Testing (ACT) cumulative score, performance in pharmacology, adult medical–surgical nursing, and community health nursing, and cumulative grade point average (GPA) were found to be predictors of NCLEX-RN success on the first-attempt (Vandenhouten, 2008). In addition, it was found that academic predictors (composite, math, and science scores on the ACT and college GPA) strongly indicated success on the NCLEX-RN while nonacademic predictors (gender, ethnicity, and marital status) did not (Humphreys, 2008). However, older age on admission (Vandenhouten, 2008) and age at NCLEX-RN sittings (Humphreys, 2008) were founded as predictors of NCLEX-RN success.
Most of these variables are not predictive of the NCLEX-RN failure and only a few variables have been identified as a predictor of the NCLEX-RN failure. Seldomridge and Dibartolo (2004) found that the combination of the composite score on the National League for Nursing Comprehensive Achievement Test for Baccalaureate Students (NLNCATBS) and grade in pathophysiology was the best model of predicting NCLEX-RN outcome on the first attempt. However, only 50% of NCLEX-RN failure was predicted by the model, and each of these variables did not predict NCLEX-RN failure accurately alone (2.8% by the grade in pathophysiology and 25% by the score on the NLNCATBS). Also, none of the mean scores in Medical–Surgical Nursing course, the number of failing grades in nursing courses, and the cumulative GPA in pre-nursing and nursing courses were accurate in predicting NCLEX-RN failure. The Seldomridge and Dibartolo's study results were congruent with the study of Vandenhouten (2008) which indicated that all of the admission variables, performance in nursing courses except pharmacology, and cumulative GPA did not predict NCLEX-RN failure. Only lower performance in pharmacology courses indicated a more likely failure on the NCLEX-RN (Vandenhouten, 2008). Upon introducing standardized tests in nursing education, many nursing programs have used standardized tests as predictors to promote their students' success on the NCLEX-RN. Uyehara et al. (2007) indicated that there were “significant correlations between NCLEX-RN success and the Mosby Assess Test, the National League for Nursing (NLN) Adult Health Comprehensive Test, the NLN Maternal Newborn Comprehensive Test, and the NLN Pediatric Nursing Comprehensive Test” (p. 34). In addition, Harding (2010) reviewed 16 studies to identify the accuracy of standardized comprehensive assessment products developed by commercial vendors on predicting NCLEX-RN success. In most of the16 studies, the Health Education Systems, Inc (HESI) Exit Exam was found as a predictor of NCLEX-RN with 96.4% to 98.3% accuracy in predicting NCELX-RN success. The first attempt score on the HESI Exit Exam had a strong predictability on NCLEX-RN success. However, the subsequent scores' predictability decreased as students took the exam multiple times after failure on the exam. One study indicated that the HESI Exit Exam predicted only 34 of the 1248 NCLEX-RN failures. Vandenhouten (2008) analyzed NCLEX-RN outcomes and scores on the Assessment Technologies Institute (ATI) content exams and the ATI RN Comprehensive Predictor 3.0 of 296 graduates from a baccalaureate nursing program by using logistic regression in order to study predictors of NCLEX-RN outcomes. This study showed all of the ATI content exams (Fundamentals, Maternal Newborn, Medical Surgical, Leadership, Community Health, and Pharmacology) significantly predicted NCLEX-RN success. The ATI RN Comprehensive Predictor was also effective in predicting success on the NCLEX-RN. The test classified 100% of NCLEX-RN success. However, those were not effective in predicting NCLEX-RN failure. The ATI content exams classified only 7.7% to 30% of failure on the NCLEX-RN, and any NCLEX-RN failure was not classified by the ATI RN Comprehensive Predictor, although low scores in the exam were correlated with NCLEX-RN failure. Many researchers have tried to find strong predictors of NCLEX-RN outcomes. Several variables have been studied to identify their effectiveness as a predictor. Early studies regarding NCLEX-RN predictors focused on admission criteria and performances in cognate and nursing courses. As standardized tests have been made available by commercial vendors, the standardized tests' effectiveness as NCLEX-RN predictors has been studied by many researchers. Comprehensive standardized tests, which are usually administered at the end of the nursing programs, have been the most studied, and there was a controversy about whether standardized tests were effective in predicting failure on the NCLEX-RN. There is a rising need to study nursing content standardized tests, which are utilized throughout the nursing program, in order to detect students at risk and provide early remediation.
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
Purpose of the Study The purpose of this study was to investigate effective predictors of NCLEX-RN outcomes on the first attempt among nursing content standardized tests conducted throughout the nursing program: adult medical surgical, fundamentals for nursing, pharmacology, maternal– newborn, nursing care of children, mental health, community health, and leadership and management. The following research questions guided the investigation of this study: 1. Are there significant differences in the individual adjusted scores of the standardized tests (adult medical–surgical, fundamentals, pharmacology, maternal–newborn, nursing care of children, mental health, community health, and leadership and management) between the two groups of students with NCLEX-RN success and failure on their first attempt? 2. Can the NCLEX-RN outcome (success or failure) on the first attempt be correctly predicted from the standardized tests? 3. If the NCLEX-RN outcome on the first-attempt can be predicted correctly, which variables are central to the prediction of the NCLEX-RN outcome on the first-attempt? 4. How many of the NCLEX-RN successes on the first-attempt are classified correctly? How many of the NCLEX-RN failures on the first-attempt are classified correctly? Methods A convenience sample of all eligible graduates identified by the inclusion and exclusion criteria was utilized in this study in order to have the largest possible number of participants. The inclusion criteria of the sample included all graduates from the traditional baccalaureate nursing program of a public university located in the Midwest from May 2010 to December 2011 who took the NCLEX-RN revised in April 2010 by the time of data collection. The ATI Content Mastery Series was utilized in the traditional baccalaureate nursing program, which was a setting used to collect data. If any students took any standardized tests more than once, only individual adjusted scores achieved on the first attempt were collected in order to maintain the same conditions and collect data from the same tests since test items on the first attempt and second attempt were not identical. Also, any other individual adjusted scores achieved while retaking nursing courses were excluded from this study. The individual adjusted score is a score calculated by the number of questions answered correctly divided by the number of questions on the test with an adjustment for the differences in the difficulty of the form taken (ATI, 2007). There were 167 graduates from the program from May 2010 to December 2011. Among the 167 graduates, 11 graduates' NCLEX-RN outcomes were not available; therefore, the number of eligible graduates was reduced to 156. It was assumed that the graduates had not taken the NCLEX-RN by the time of data collection. Among the 156 graduates, 5 graduates' individual adjusted scores on one or more the standardized tests were missing. Therefore, a total of 151 participants, which were comprised of 118 graduates who passed the NCLEX-RN on the first-attempt and 33 graduates who failed the NCLEX-RN on the first attempt, were eligible after applying the inclusion and exclusion sample criteria. Since both numbers of students succeeding and failing on the NCLEX-RN are greater than 30, it is not vulnerable to the normality of sample distribution. A random number between 1 and 200 was assigned to each participant for confidentiality. The NCLEX-RN outcomes and individual adjusted scores on the nursing content standardized tests were obtained. The participants took fundamentals for nursing and mental health standardized tests during their sophomore year, maternal newborn, nursing care of children, and pharmacology standardized tests during their junior year, and adult medical–surgical, community health, and leadership and management standardized tests during their senior year.
1525
Since this study used a quantitative research approach, all nonquantitative data, except the participants' demographic data, was converted into quantitative data and analyzed by using the Statistical Package for the Social Sciences (SPSS) 20.0. The participants' NCLEX RN outcomes on the first attempt took the value of one (1) with success and value of zero (0) with failure as dichotomous variables for the statistical analysis for NCLEX-RN success. Conversely, NCLEX-RN success took the value of zero (0), and NCLEX-RN failure took the value of one (1) for statistical analysis for NCLEX-RN failure. Descriptive statistics were conducted to summarize the participants' demographic data. A t-test of the statistical methods was conducted in order to determine if there were significant differences in the adjusted individual scores of the standardized tests between the two groups of participants with NCLEX-RN success and failure on the first attempt. The individual adjusted scores on the standardized tests of the two groups were compared separately by the t-test and an alpha level of .05 was used as the level of significance for data analysis. Although this analysis method could not be used to conclude if the tests predict NCLEX-RN outcomes, it helped to provide ideas of possible predictors of NCLEX-RN outcomes among the standardized tests. The logistic regression, which “tests the ability of a model or group of variables to predict group membership as defined by some categorical dependent variables” (Mertler and Vannatta, 2010, p. 304), was used to determine whether the NCLEX-RN outcomes could be correctly predicted from the standardized tests conducted throughout the nursing program. If the NCLEX-RN outcomes on the first attempt could be predicted correctly then the standardized tests central to the prediction of the NCLEX-RN failure and how many of the NCLEX-RN failures were classified correctly were determined by logistic regression. An alpha level of .05 was used as the level of significance for data analysis using the logistic regression. Results The participants were comprised of 78.1% graduates who passed the NCLEX-RN on the first-attempt and 21.9% graduates who failed the NCLEX-RN on the first attempt. Their demographic information was not analyzed in this study; however, the participants' mean age, gender, and ethnicity self-reported to the university are presented in Table 1 only for descriptive purposes. Females and Caucasians were the majority of the participants. The mean age of the NCLEX-RN success group was slightly older than the NCLEX-RN failure group. There were significant statistical differences between the two groups with NCLEX-RN success and failure in the individual adjusted scores on almost all of the standardized tests. There were no significant statistical differences between the two groups only in individual adjusted scores on the fundamental (p = .62) and nursing care of children (p = .759) standardized tests. Table 2 shows the two groups' means and standard deviation for each standardized test (adult medical–surgical, fundamentals, pharmacology, maternal–newborn, nursing care of children, mental health, community health, and leadership and management), t statistic, and p value (2-tailed). The participants with NCLEX-RN success had higher mean scores than those with NCLEX-RN failure on all of the standardized tests. However, the p values for fundamentals and nursing care of children were greater than .05, and these were the only results indicating that there were no statistically significant differences in the individual adjusted scores between the participants who succeeded and failed on the NCLEX-RN on the first attempt. The results of logistic regression indicated that the overall model was statistically reliable in distinguishing those who succeed on the NCLEX-RN on the first attempt (−2 Log likelihood = 111.713, X 2(2) = 46.854, and p = .000). The model correctly classified 93.2% (110 out of 118) of the NCLEX-RN success cases. Regression coefficients are presented in Table 3. Wald statistics indicated that the adult medical–surgical, pharmacology, and community health standardized
1526
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
Table 1 Demographic information. Mean age
NCLEX-RN success NCLEX-RN failure The participants
Gender
27.02 26.78 26.97
Ethnicity
Male
Female
African American
Caucasian
Asian/Pacific Islander
Hispanic
9 3 12
109 30 139
3 3 6
29 112 141
2 1 3
1 0 1
Note: N = 151.
Table 2 The standardized tests scores of the groups of NCLEX-RN success and failure. Mean
Adult Medical–Surgical Fundamentals for Nursing Pharmacology Maternal–Newborn Nursing Care of Children Mental Health Community Health Leadership & Management
Standard deviation
Success
Failure
Success
Failure
61.40 74.05 64.79 76.21 70.62 72.82 78.45 72.14
52.79 70.70 56.02 71.87 70.04 67.73 74.24 68.27
8.03 8.90 9.27 7.18 8.70 8.41 6.21 5.86
6.18 9.51 7.93 7.30 12.09 7.49 5.01 6.19
t-value
Sig. (2-tailed)
−5.697 −1.880 −4.944 −3.060 −0.308 −3.143 −3.582 −3.311
.000 .062 .000 .003 .759 .002 .000 .001
Note: N = 151.
tests significantly predicted NCLEX-RN success on the first attempt. The participants achieving higher scores on these standardized tests were more likely to succeed on the NCLEX-RN on the first attempt. Although the adult medical–surgical standardized test has the highest odds ratio, there are no significant differences between odds ratios of adult medical–surgical, pharmacology, and community health standardized tests. The odds ratios for the adult medical–surgical, pharmacology, and community health standardized tests revealed little increase in the likelihood of NCLEX-RN success when the predictors increase by 1. Also, the results of logistic regression indicated that the overall model was statistically significant in distinguishing those who failed the NCLEX-RN on the first attempt (− 2 Log likelihood = 111.713, X 2(2) = 46.854, and p = .000). However, the model correctly classified only 33.3% (11 out of 33) of the NCLEX-RN failure cases. Regression coefficients for predicting NCLEX-RN failure are presented in Table 4. Wald statistics indicated that the adult medical–surgical, pharmacology, and community health standardized tests significantly predicted NCLEX-RN failure. However, the odds ratios for the adult medical–surgical, pharmacology, and community health standardized tests indicated little change in the likelihood of NCLEX-RN failure.
There were significant statistical differences between the two groups with NCLEX-RN success and failure in the individual adjusted scores of the adult medical–surgical, pharmacology, maternal–newborn, mental health, community health, and leadership and management standardized tests. The NCLEX-RN success group's mean scores in those
standardized tests were higher than the NCLEX-RN failure group. These findings can be linked to the finding of Uyehara et al. (2007)'s study which indicated that there was correlation between the NLN Adult Health Comprehensive Test and the NLN Maternal Newborn Comprehensive tests and NCLEX-RN success. However, while Uyehara et al. (2007) found that there was a correlation between the NLN Pediatric Nursing Comprehensive Test and the performance in the fundamental course and NCLEX-RN success, this study's findings indicated that there were no significant statistical differences between the NCLEX-RN success and failure groups in the individual adjusted scores on the nursing care of children and fundamentals standardized tests. Although these results provided ideas about the correlations between scores on the standardized tests and outcomes on the NCLEXRN, it does not mean that those standardized tests can predict the NCLEX-RN outcomes. The results of the logistic regression indicated whether the standardized tests could be used as a predictor of the NCLEX-RN outcomes. The logistic regression models showed that adult-medical surgical, pharmacology, and community health standardized tests were effective in predicting NCLEX-RN success. These tests were accurate to classify 93.2% of the NCLEX-RN cases. This finding is consistent with Vandenhouten's (2008) study indicating that medical–surgical, community health, and pharmacology standardized tests significantly predicted NCLEX-RN success. However, Vandenhouten's (2008) study also included leadership and maternal–newborn standardized tests as NCLEX-RN success predictors while this study's finding indicated that those two tests did not predict NCLEX-RN success. Moreover, logistic regression in this study demonstrated adult medical–surgical, pharmacology, and community health standardized tests
Table 3 Regression coefficients for predicting NCLEX-RN success.
Table 4 Regression coefficients for predicting NCLEX-RN failure.
Discussion
Adult Medical–Surgical Fundamentals Pharmacology Maternal–Newborn Nursing Care of Children Mental Health Community Health Leadership and Management
B
Wald
df
P
Odds ratio
.115 −.010 .084 .041 −.011 −.015 .096 .050
6.550 .127 6.554 1.471 .218 .145 4.899 1.261
1 1 1 1 1 1 1 1
.010 .721 .010 .225 .641 .703 .027 .261
1.122 .990 1.087 1.042 .989 .985 1.101 1.052
Adult Medical–Surgical Fundamentals for Nursing Pharmacology Maternal–Newborn Nursing Care of Children Mental Health Community Health Leadership and Management
B
Wald
df
p
Odds ratio
−.115 .010 −.084 −.041 .011 .015 −.096 −.050
6.550 .127 6.554 1.471 .218 .145 4.899 1.261
1 1 1 1 1 1 1 1
.010 .721 .010 .225 .641 .703 .027 .261
.891 1.010 .920 .960 1.011 1.015 .908 .951
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
were able to predict NCLEX-RN failure; however, only 33.3% of NCLEXRN failure cases could be classified by the generated model. It can be concluded that the standardized tests were not accurate in predicting NCLEX-RN failure. This finding is consistent with Seldomridge and Dibartolo's (2004) and Vandenhouten's (2008) studies which concluded that any nursing content standardized tests were not accurate in predicting students who were likely to fail the NCLEX-RN. Limitations One of the limitations of this study was the unequal sizes of groups of students who succeeded and failed on the NCLEX-RN on the first attempt even though this limitation has persisted in most studies regarding NCLEX-RN predictors since the majority of candidates pass NCLEX-RN as shown in the national NCLEX-RN pass rate. Another limitation is that all of this study's data were collected in one traditional baccalaureate nursing program of a public university located in the Midwest. A diversity of different curriculums, student support programs, remediation programs, student populations, and other aspects, which are unique to each nursing program, is not captured in this study. Also, only the ATI Content Series was used as variables of the standardized tests in this study although various products of standardized tests offered by vendors are available besides the ATI Content Series. Each test of the ATI Content Series had different question numbers. Although individual adjusted scores on each standardized test were used for this study, the different numbers of the questions could limit comparisons between effectiveness as an NCLEX-RN predictor between tests. In addition, individual differences were not captured in this study. Each participant's NCLEX-RN outcome could be affected by different motivations, maturations, life situations, emotional reactions to NCLEX-RN, and other aspects. Although students did not achieve benchmark scores on the standardized tests, the nursing program might provide remediation to these individuals, and this might be effective in helping the students to succeed on the NCLEX-RN. Also, students who achieved low scores on the standardized tests might be concerned about their ability to be successful, and this factor might motivate them to prepare more extensively for the NCLEX-RN. The other consideration is that students might feel comfortable with NCLEX-RN after achieving benchmark scores on the standardized tests, and this might reduce their intension of preparation for the NCLEX-RN and result in failure on the NCLEX-RN. These factors can limit the ability to generalize this study's findings to other nursing programs and standardized tests products and have to be considered when interpreting this study's results.
1527
remediation can be provided. After early remediation is implemented, the comprehensive standardized test can be used as a mid-point indicator of the remediation's effectiveness before taking the NCLEX-RN. This study also raises a question about the predictors of NCLEX-RN outcomes. Why are adult medical–surgical, pharmacology, and community health standardized tests effective in predicting the NCLEX-RN success while the other subject tests are not? It can be assumed that adult medical–surgical and pharmacology standardized tests can predict the NCLEX-RN success because the NCLEX-RN has relatively many questions related to those areas. Also, it can be assumed that community health standardized tests can predict the NCLEX-RN success because the subject requires integrating nursing knowledge and skills for health promotion and public health. However, more studies are needed to find evidence that can support this assumption or answer the question. In order to have more accurate results of regression models predicting and classifying NCLEX-RN failure, it is recommended to include a large number of participants who fail on the NCLEX-RN if possible. This will result in having more accurate results of regression models predicting and classifying NCLEX-RN failure. In addition, since there are different kinds of standardized tests, which are offered by various commercial vendors, more studies including various products are needed to investigate strong NCLEX-RN predictors and compare the effectiveness of each nursing content standardized tests. Therefore, it is possible to have more powerful evidence showing if the findings of this study are repeated in other studies having variables of different products of the standardized tests. Moreover, the NCLEXRN is revised every three years, and the next revision will be in April 2013. After the revised version of NCLEX-RN is conducted, commercial vendors will accordingly revise their products of standardized tests based on the revised NCLEX-RN. There is a need for having longitudinal and repeated similar studies in more diverse populations and in many different nursing programs.
Acknowledgment The author would like to thank Dr. Marcia Miller, Associate Professor of the Department of Baccalaureate Nursing, Indiana State University for her support, encouragement, and efforts throughout this research and for reading and editing the manuscript. Funding Sigma Theta Tau, Lambda Sigma Chapter provided a scholarship for this research. The funding source was not involved in the study design.
Conclusions Since the nursing shortage is a critical issue in the U.S., more RNs are needed to meet the increased demands for nursing care. Nursing students have to pass the NCLEX-RN to become RNs. The NCLEX-RN is one of the important evaluation factors of the quality of education provided by nursing programs. Graduates' NCLEX-RN failure causes many negative effects on graduates, nursing programs, consumers, and society. In order to improve an NCLEX-RN pass rate and prevent negative effects of the NCLEX-RN failure, supporting students who are at risk of failure on the NCLEX-RN is indispensable. Since identification of the students at risk should be preceded in order to provide remediation to them, many studies have investigated effective NCLEX-RN predictors; however, common variables of the studies are limited to admission criteria, comprehensive standardized tests, and performances in courses with controversies over the results of the studies. The result of this study indicated that adult medical–surgical, pharmacology, and community health nursing were powerful NCLEX-RN success predictors. Through using these predictors, students at risk can be identified before taking comprehensive standardized tests usually conducted in the last semester of nursing programs, and early
References American Association of Colleges of Nursing, 2011. Nursing shortage fact sheet. Retrieved from http://www.aacn.nche.edu/media-relations/NrsgShortageFS.pdf. American Association of Colleges of Nursing, 2012. Nursing fact sheet. Retrieved from http://www.aacn.nche.edu/media-relations/fact-sheets/nursing-fact-sheet. Assessment Technologies Institute, 2007. RN Content Mastery Series 2007 national standard setting study: Executive summary. ATI, Stilwell, KS. Catalano, J., 2009. Nursing Now: Today's Issues, Tomorrow's Trends. F.A. Davis Company, Philadelphia, PA. Chang, C., Price, S., Pfoutz, S., 2001. Economics and Nursing: Critical Professional Issues. F.A. Davis Company, Philadelphia, PA. Greenpan, V., Springer, P., Ray, K., 2009. A tri-nodal model for NCLEX-RN success. Nurse Educator 34 (3), 101–102. Griffiths, M., Papastrat, K., Czekanski, K., Hagen, K., 2004. The lived experience of NCLEX failure. Nurse Educator 43 (7), 322–325. Harding, M., 2010. Predictability associated with exit examinations: literature review. Journal of Nursing Education 49 (9), 493–497. Humphreys, J.A., 2008. Academic and nonacademic predictors of future success on the NCLEX-RN licensure examination for nurses ((Doctoral dissertation). Retrieved from ProQuest. (3315143)). Mertler, C.A., Vannatta, R.A., 2010. Advanced and Multivariate Statistical Methods. Pyrczak Publishing, Glendale, CA. National Council of State Boards of Nursing, 2012. NCLEX examinations. Retrieved from https://www.ncsbn.org/nclex.htm.
1528
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
Norton, C.K., Rele, M.V., Cox, C.W., Farley, J., Lachat, M., Tucker, M., 2006. Ensuring NCLEX-RN success for first-time test takers. Journal of Professional Nursing 22 (5), 322–326. Roa, M., Shipman, D., Hooten, J., Carter, M., 2011. The Cost of NCLEX-RN Failure. Nurse Education Today 31 (4), 373–377. Seldomridge, L.A., Dibartolo, M.C., 2004. Can success and failure be predicted for baccalaureate graduates on the computerized NCLEX-RN. Journal of Professional Nursing 20 (6), 361–368.
Sifford, S., McDaniel, D.M., 2007. Results of a remediation program for students at risk for failure on the NCLEX exam. Nursing Education Perspectives 28 (1), 34–36. Uyehara, J., Magnussen, L., Itano, J., Zhang, S., 2007. Facilitating program and NCLEX-RN success in a generic BSN program. Nursing Forum 42 (1), 31–38. Vandenhouten, C.L., 2008. Predictors of success and failure on the NCLEX-RN for baccalaureate graduates ((Doctoral dissertation). Retrieved from ProQuest. (3306523)).
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
An investigation of predictors of NCLEX-RN outcomes among nursing content standardized tests Yei-Jin Yeom ⁎ Department of Baccalaureate Nursing, Indiana State University, NU 328, 749 Chestnut Street, Terre Haute, IN 47809, United States
a r t i c l e
i n f o
Article history: Accepted 8 April 2013 Keywords: NCLEX-RN Success Failure Outcome Predictors Standardized tests
s u m m a r y In order to meet increased demands for qualified registered nurses and prevent negative effects from graduates' failure on the National Council Licensure Examination-Registered Nurse, it is important to promote students' success in the exam. The purpose of this study was to investigate effective predictors of NCLEX-RN outcomes on the first attempt among nursing content standardized tests (adult medical–surgical, fundamentals for nursing, pharmacology, maternal–newborn, nursing care of children, mental health, community health, and leadership and management) conducted throughout the nursing program. NCLEX-RN outcomes and individual adjusted scores on the standardized tests of 151 graduates from the traditional baccalaureate nursing program of a public university located in the Midwest from May 2010 to December 2011 were analyzed by a t-test and logistic regression. The participants included 118 graduates who passed and 33 graduates who failed the NCLEX-RN on the first attempt. Significant statistical differences were found between the two groups with NCLEX-RN success and failure in the individual adjusted scores on all of the standardized tests except the fundamental for nursing (p = .62) and nursing care of children (p = .759) standardized tests. In addition, logistic regression indicated that the overall regression models were significant in predicting both NCLEX-RN success and failure. Adult medical–surgical, pharmacology, and community health standardized tests were central to the prediction of both NCLEX-RN success and failure; however, a much lower percentage of NCLEX-RN failure than success was classified. It can be concluded that the adult medical–surgical, pharmacology, and community health standardized tests were effective in predicting NCLEX-RN success and not effective in predicting NCLEX-RN failure on the first attempt. The NCLEX-RN success predictors can be utilized to identify students at risk and provide early remediation. After early remediation is implemented, the comprehensive standardized tests may be used as a mid-point indicator of the remediation's effectiveness before taking the NCLEX-RN. © 2013 Elsevier Ltd. All rights reserved.
Introduction Nursing shortage is a critical issue in the United States (U.S.). The shortage is expected to get even worse since the demands for registered nurses (RN) are outstripping the supply. According to the American Association of Colleges of Nursing (AACN, 2012), there will be more than 581,500 new positions for RNs through 2018, demands for RNs in acute care will increase by 36% through 2020, and a lack of more than 260,000 RNs will affect the U.S. health care system by 2025. Even though demands for RNs have dramatically increased, the enrollment of baccalaureate nursing students increased only 5.7% in 2010. This increase in enrollment is not enough to meet the increased demands for RNs. Furthermore, due to the increase in the average age of RNs, it is expected that there will be an additional loss of a large number of RNs due to retirement over the next 20 years. While at the same time the demands for RNs will increase as baby boomers age and need health
⁎ Tel.: +1 812 230 6120; fax: +1 812 237 8022. E-mail address: [email protected]. 0260-6917/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.nedt.2013.04.004
care services during that time (AACN, 2011). One of the strategies to resolve the problem of the nursing shortage is the expansion of nursing program enrollments. However, enrollments remain limited due to inadequate numbers of nursing faculty (AACN, 2011). Despite this limitation, nursing programs are still accountable to mitigate the nursing shortage. In order to meet the nursing demands and lessen the nursing shortage, enhancing the National Council Licensure Examination-Registered Nurse (NCLEX-RN) pass rate is critical since the successful completion of this exam produces more RNs. Graduates are ensured to have competencies and can be in their nursing careers as entry-level RNs when they pass NCLEX-RN, which measures competencies required to provide quality nursing care and ensures the public's safety (National Council of State Boards of Nursing [NCSBN], 2012). Graduates' failures on the NCLEX-RN negatively affect stakeholders such as the graduates, health care organizations, and nursing programs. Graduates who fail the NCLEX-RN experience feelings of embarrassment, anxiety, loss of selfesteem, guilt, and social stigma, and those feelings may negatively affect the graduates' further NCLEX-RN attempts (Roa et al., 2011; Griffiths et al., 2004). Also, NCLEX-RN failure causes financial loss to the graduates.
1524
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
The graduates expect to be compensated for tuition and other school related expenses through work as RNs after graduation. However, if they fail the NCLEX-RN, they cannot work as an RN for a minimum of 45 days while they wait to be eligible for the next attempt. They also spend additional money for remediation and fees to retake the NCLEX-RN. Moreover, health care organizations expect that graduates will pass the NCLEX-RN when they hire these graduates. When the newly hired graduates fail the NCLEX-RN, health care organizations must fill positions vacated by graduates who have failed with temporary nursing staff or they must increase the overtime pay of existing employees. This causes not only higher costs for the organizations, but also a lack of qualified nurses within organizations and increases the staff–patient ratio (Roa et al., 2011; Greenpan et al., 2009). Without a doubt, nursing programs are affected by their graduates' NCLEX-RN failures. The NCLEX-RN pass rates on the first-attempt are a major factor used in evaluating the nursing programs' effectiveness and can influence the accreditation of nursing programs (Norton et al., 2006). Nursing programs do not meet the national benchmark standards, they are at risk of being placed on probation or losing their accreditation. Also, the nursing programs, colleges, and universities with low NCLEX-RN pass rates may face financial risks. Roa et al. (2011) and Chang et al. (2001) indicated that each nursing program's NCLEX-RN pass rates are public information, and prospective students may evaluate the pass rates to decide to which programs they will apply and enroll in to meet their needs. The nursing programs, colleges, and universities with lower pass rates may experience decreased enrollments which negatively affects their operational budgets through decreased tuition. Nursing programs are ethically responsible to their graduates who were qualified and met academic standards for graduation yet failed the NCLEX-RN (Roa et al., 2011; Catalano, 2009). Nursing programs are “challenged to facilitate knowledge development, ensure competence of their new graduates, and demonstrate organizational and curricular effectiveness through identification and remediation of students at high-risk of failure among first-time test candidates” (Norton et al., 2006, p. 322). With effective NCLEX-RN predictors, students' outcomes on the NCLEX-RN can be better predicted, and educators can better support students at risk through developing and providing early remediation resulting in improved NCLEX-RN pass rates. Such predictive measures can result in decreasing the nursing shortage and its negative effects. Currently, many nursing programs use standardized assessments purchased from outside companies to predict their students' first attempt outcome on the NCLEX-RN. Nursing educators need to distinguish the more powerful and effective subjects tested.
Literature Review In order to meet the demand of increased entry level RN's competence, the NCLEX-RN is reviewed by the NCLEX Examination Committee every three years. The minimum score for passing on the NCLEX-RN has been raised by the NCBSN. Because nursing programs have been challenged by this, educators focus upon finding strong predictors of NCLEX-RN outcome and providing interventions to ensure that their students pass the NCLEX-RN (Sifford and McDaniel, 2007). The American College Testing (ACT) cumulative score, performance in pharmacology, adult medical–surgical nursing, and community health nursing, and cumulative grade point average (GPA) were found to be predictors of NCLEX-RN success on the first-attempt (Vandenhouten, 2008). In addition, it was found that academic predictors (composite, math, and science scores on the ACT and college GPA) strongly indicated success on the NCLEX-RN while nonacademic predictors (gender, ethnicity, and marital status) did not (Humphreys, 2008). However, older age on admission (Vandenhouten, 2008) and age at NCLEX-RN sittings (Humphreys, 2008) were founded as predictors of NCLEX-RN success.
Most of these variables are not predictive of the NCLEX-RN failure and only a few variables have been identified as a predictor of the NCLEX-RN failure. Seldomridge and Dibartolo (2004) found that the combination of the composite score on the National League for Nursing Comprehensive Achievement Test for Baccalaureate Students (NLNCATBS) and grade in pathophysiology was the best model of predicting NCLEX-RN outcome on the first attempt. However, only 50% of NCLEX-RN failure was predicted by the model, and each of these variables did not predict NCLEX-RN failure accurately alone (2.8% by the grade in pathophysiology and 25% by the score on the NLNCATBS). Also, none of the mean scores in Medical–Surgical Nursing course, the number of failing grades in nursing courses, and the cumulative GPA in pre-nursing and nursing courses were accurate in predicting NCLEX-RN failure. The Seldomridge and Dibartolo's study results were congruent with the study of Vandenhouten (2008) which indicated that all of the admission variables, performance in nursing courses except pharmacology, and cumulative GPA did not predict NCLEX-RN failure. Only lower performance in pharmacology courses indicated a more likely failure on the NCLEX-RN (Vandenhouten, 2008). Upon introducing standardized tests in nursing education, many nursing programs have used standardized tests as predictors to promote their students' success on the NCLEX-RN. Uyehara et al. (2007) indicated that there were “significant correlations between NCLEX-RN success and the Mosby Assess Test, the National League for Nursing (NLN) Adult Health Comprehensive Test, the NLN Maternal Newborn Comprehensive Test, and the NLN Pediatric Nursing Comprehensive Test” (p. 34). In addition, Harding (2010) reviewed 16 studies to identify the accuracy of standardized comprehensive assessment products developed by commercial vendors on predicting NCLEX-RN success. In most of the16 studies, the Health Education Systems, Inc (HESI) Exit Exam was found as a predictor of NCLEX-RN with 96.4% to 98.3% accuracy in predicting NCELX-RN success. The first attempt score on the HESI Exit Exam had a strong predictability on NCLEX-RN success. However, the subsequent scores' predictability decreased as students took the exam multiple times after failure on the exam. One study indicated that the HESI Exit Exam predicted only 34 of the 1248 NCLEX-RN failures. Vandenhouten (2008) analyzed NCLEX-RN outcomes and scores on the Assessment Technologies Institute (ATI) content exams and the ATI RN Comprehensive Predictor 3.0 of 296 graduates from a baccalaureate nursing program by using logistic regression in order to study predictors of NCLEX-RN outcomes. This study showed all of the ATI content exams (Fundamentals, Maternal Newborn, Medical Surgical, Leadership, Community Health, and Pharmacology) significantly predicted NCLEX-RN success. The ATI RN Comprehensive Predictor was also effective in predicting success on the NCLEX-RN. The test classified 100% of NCLEX-RN success. However, those were not effective in predicting NCLEX-RN failure. The ATI content exams classified only 7.7% to 30% of failure on the NCLEX-RN, and any NCLEX-RN failure was not classified by the ATI RN Comprehensive Predictor, although low scores in the exam were correlated with NCLEX-RN failure. Many researchers have tried to find strong predictors of NCLEX-RN outcomes. Several variables have been studied to identify their effectiveness as a predictor. Early studies regarding NCLEX-RN predictors focused on admission criteria and performances in cognate and nursing courses. As standardized tests have been made available by commercial vendors, the standardized tests' effectiveness as NCLEX-RN predictors has been studied by many researchers. Comprehensive standardized tests, which are usually administered at the end of the nursing programs, have been the most studied, and there was a controversy about whether standardized tests were effective in predicting failure on the NCLEX-RN. There is a rising need to study nursing content standardized tests, which are utilized throughout the nursing program, in order to detect students at risk and provide early remediation.
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
Purpose of the Study The purpose of this study was to investigate effective predictors of NCLEX-RN outcomes on the first attempt among nursing content standardized tests conducted throughout the nursing program: adult medical surgical, fundamentals for nursing, pharmacology, maternal– newborn, nursing care of children, mental health, community health, and leadership and management. The following research questions guided the investigation of this study: 1. Are there significant differences in the individual adjusted scores of the standardized tests (adult medical–surgical, fundamentals, pharmacology, maternal–newborn, nursing care of children, mental health, community health, and leadership and management) between the two groups of students with NCLEX-RN success and failure on their first attempt? 2. Can the NCLEX-RN outcome (success or failure) on the first attempt be correctly predicted from the standardized tests? 3. If the NCLEX-RN outcome on the first-attempt can be predicted correctly, which variables are central to the prediction of the NCLEX-RN outcome on the first-attempt? 4. How many of the NCLEX-RN successes on the first-attempt are classified correctly? How many of the NCLEX-RN failures on the first-attempt are classified correctly? Methods A convenience sample of all eligible graduates identified by the inclusion and exclusion criteria was utilized in this study in order to have the largest possible number of participants. The inclusion criteria of the sample included all graduates from the traditional baccalaureate nursing program of a public university located in the Midwest from May 2010 to December 2011 who took the NCLEX-RN revised in April 2010 by the time of data collection. The ATI Content Mastery Series was utilized in the traditional baccalaureate nursing program, which was a setting used to collect data. If any students took any standardized tests more than once, only individual adjusted scores achieved on the first attempt were collected in order to maintain the same conditions and collect data from the same tests since test items on the first attempt and second attempt were not identical. Also, any other individual adjusted scores achieved while retaking nursing courses were excluded from this study. The individual adjusted score is a score calculated by the number of questions answered correctly divided by the number of questions on the test with an adjustment for the differences in the difficulty of the form taken (ATI, 2007). There were 167 graduates from the program from May 2010 to December 2011. Among the 167 graduates, 11 graduates' NCLEX-RN outcomes were not available; therefore, the number of eligible graduates was reduced to 156. It was assumed that the graduates had not taken the NCLEX-RN by the time of data collection. Among the 156 graduates, 5 graduates' individual adjusted scores on one or more the standardized tests were missing. Therefore, a total of 151 participants, which were comprised of 118 graduates who passed the NCLEX-RN on the first-attempt and 33 graduates who failed the NCLEX-RN on the first attempt, were eligible after applying the inclusion and exclusion sample criteria. Since both numbers of students succeeding and failing on the NCLEX-RN are greater than 30, it is not vulnerable to the normality of sample distribution. A random number between 1 and 200 was assigned to each participant for confidentiality. The NCLEX-RN outcomes and individual adjusted scores on the nursing content standardized tests were obtained. The participants took fundamentals for nursing and mental health standardized tests during their sophomore year, maternal newborn, nursing care of children, and pharmacology standardized tests during their junior year, and adult medical–surgical, community health, and leadership and management standardized tests during their senior year.
1525
Since this study used a quantitative research approach, all nonquantitative data, except the participants' demographic data, was converted into quantitative data and analyzed by using the Statistical Package for the Social Sciences (SPSS) 20.0. The participants' NCLEX RN outcomes on the first attempt took the value of one (1) with success and value of zero (0) with failure as dichotomous variables for the statistical analysis for NCLEX-RN success. Conversely, NCLEX-RN success took the value of zero (0), and NCLEX-RN failure took the value of one (1) for statistical analysis for NCLEX-RN failure. Descriptive statistics were conducted to summarize the participants' demographic data. A t-test of the statistical methods was conducted in order to determine if there were significant differences in the adjusted individual scores of the standardized tests between the two groups of participants with NCLEX-RN success and failure on the first attempt. The individual adjusted scores on the standardized tests of the two groups were compared separately by the t-test and an alpha level of .05 was used as the level of significance for data analysis. Although this analysis method could not be used to conclude if the tests predict NCLEX-RN outcomes, it helped to provide ideas of possible predictors of NCLEX-RN outcomes among the standardized tests. The logistic regression, which “tests the ability of a model or group of variables to predict group membership as defined by some categorical dependent variables” (Mertler and Vannatta, 2010, p. 304), was used to determine whether the NCLEX-RN outcomes could be correctly predicted from the standardized tests conducted throughout the nursing program. If the NCLEX-RN outcomes on the first attempt could be predicted correctly then the standardized tests central to the prediction of the NCLEX-RN failure and how many of the NCLEX-RN failures were classified correctly were determined by logistic regression. An alpha level of .05 was used as the level of significance for data analysis using the logistic regression. Results The participants were comprised of 78.1% graduates who passed the NCLEX-RN on the first-attempt and 21.9% graduates who failed the NCLEX-RN on the first attempt. Their demographic information was not analyzed in this study; however, the participants' mean age, gender, and ethnicity self-reported to the university are presented in Table 1 only for descriptive purposes. Females and Caucasians were the majority of the participants. The mean age of the NCLEX-RN success group was slightly older than the NCLEX-RN failure group. There were significant statistical differences between the two groups with NCLEX-RN success and failure in the individual adjusted scores on almost all of the standardized tests. There were no significant statistical differences between the two groups only in individual adjusted scores on the fundamental (p = .62) and nursing care of children (p = .759) standardized tests. Table 2 shows the two groups' means and standard deviation for each standardized test (adult medical–surgical, fundamentals, pharmacology, maternal–newborn, nursing care of children, mental health, community health, and leadership and management), t statistic, and p value (2-tailed). The participants with NCLEX-RN success had higher mean scores than those with NCLEX-RN failure on all of the standardized tests. However, the p values for fundamentals and nursing care of children were greater than .05, and these were the only results indicating that there were no statistically significant differences in the individual adjusted scores between the participants who succeeded and failed on the NCLEX-RN on the first attempt. The results of logistic regression indicated that the overall model was statistically reliable in distinguishing those who succeed on the NCLEX-RN on the first attempt (−2 Log likelihood = 111.713, X 2(2) = 46.854, and p = .000). The model correctly classified 93.2% (110 out of 118) of the NCLEX-RN success cases. Regression coefficients are presented in Table 3. Wald statistics indicated that the adult medical–surgical, pharmacology, and community health standardized
1526
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
Table 1 Demographic information. Mean age
NCLEX-RN success NCLEX-RN failure The participants
Gender
27.02 26.78 26.97
Ethnicity
Male
Female
African American
Caucasian
Asian/Pacific Islander
Hispanic
9 3 12
109 30 139
3 3 6
29 112 141
2 1 3
1 0 1
Note: N = 151.
Table 2 The standardized tests scores of the groups of NCLEX-RN success and failure. Mean
Adult Medical–Surgical Fundamentals for Nursing Pharmacology Maternal–Newborn Nursing Care of Children Mental Health Community Health Leadership & Management
Standard deviation
Success
Failure
Success
Failure
61.40 74.05 64.79 76.21 70.62 72.82 78.45 72.14
52.79 70.70 56.02 71.87 70.04 67.73 74.24 68.27
8.03 8.90 9.27 7.18 8.70 8.41 6.21 5.86
6.18 9.51 7.93 7.30 12.09 7.49 5.01 6.19
t-value
Sig. (2-tailed)
−5.697 −1.880 −4.944 −3.060 −0.308 −3.143 −3.582 −3.311
.000 .062 .000 .003 .759 .002 .000 .001
Note: N = 151.
tests significantly predicted NCLEX-RN success on the first attempt. The participants achieving higher scores on these standardized tests were more likely to succeed on the NCLEX-RN on the first attempt. Although the adult medical–surgical standardized test has the highest odds ratio, there are no significant differences between odds ratios of adult medical–surgical, pharmacology, and community health standardized tests. The odds ratios for the adult medical–surgical, pharmacology, and community health standardized tests revealed little increase in the likelihood of NCLEX-RN success when the predictors increase by 1. Also, the results of logistic regression indicated that the overall model was statistically significant in distinguishing those who failed the NCLEX-RN on the first attempt (− 2 Log likelihood = 111.713, X 2(2) = 46.854, and p = .000). However, the model correctly classified only 33.3% (11 out of 33) of the NCLEX-RN failure cases. Regression coefficients for predicting NCLEX-RN failure are presented in Table 4. Wald statistics indicated that the adult medical–surgical, pharmacology, and community health standardized tests significantly predicted NCLEX-RN failure. However, the odds ratios for the adult medical–surgical, pharmacology, and community health standardized tests indicated little change in the likelihood of NCLEX-RN failure.
There were significant statistical differences between the two groups with NCLEX-RN success and failure in the individual adjusted scores of the adult medical–surgical, pharmacology, maternal–newborn, mental health, community health, and leadership and management standardized tests. The NCLEX-RN success group's mean scores in those
standardized tests were higher than the NCLEX-RN failure group. These findings can be linked to the finding of Uyehara et al. (2007)'s study which indicated that there was correlation between the NLN Adult Health Comprehensive Test and the NLN Maternal Newborn Comprehensive tests and NCLEX-RN success. However, while Uyehara et al. (2007) found that there was a correlation between the NLN Pediatric Nursing Comprehensive Test and the performance in the fundamental course and NCLEX-RN success, this study's findings indicated that there were no significant statistical differences between the NCLEX-RN success and failure groups in the individual adjusted scores on the nursing care of children and fundamentals standardized tests. Although these results provided ideas about the correlations between scores on the standardized tests and outcomes on the NCLEXRN, it does not mean that those standardized tests can predict the NCLEX-RN outcomes. The results of the logistic regression indicated whether the standardized tests could be used as a predictor of the NCLEX-RN outcomes. The logistic regression models showed that adult-medical surgical, pharmacology, and community health standardized tests were effective in predicting NCLEX-RN success. These tests were accurate to classify 93.2% of the NCLEX-RN cases. This finding is consistent with Vandenhouten's (2008) study indicating that medical–surgical, community health, and pharmacology standardized tests significantly predicted NCLEX-RN success. However, Vandenhouten's (2008) study also included leadership and maternal–newborn standardized tests as NCLEX-RN success predictors while this study's finding indicated that those two tests did not predict NCLEX-RN success. Moreover, logistic regression in this study demonstrated adult medical–surgical, pharmacology, and community health standardized tests
Table 3 Regression coefficients for predicting NCLEX-RN success.
Table 4 Regression coefficients for predicting NCLEX-RN failure.
Discussion
Adult Medical–Surgical Fundamentals Pharmacology Maternal–Newborn Nursing Care of Children Mental Health Community Health Leadership and Management
B
Wald
df
P
Odds ratio
.115 −.010 .084 .041 −.011 −.015 .096 .050
6.550 .127 6.554 1.471 .218 .145 4.899 1.261
1 1 1 1 1 1 1 1
.010 .721 .010 .225 .641 .703 .027 .261
1.122 .990 1.087 1.042 .989 .985 1.101 1.052
Adult Medical–Surgical Fundamentals for Nursing Pharmacology Maternal–Newborn Nursing Care of Children Mental Health Community Health Leadership and Management
B
Wald
df
p
Odds ratio
−.115 .010 −.084 −.041 .011 .015 −.096 −.050
6.550 .127 6.554 1.471 .218 .145 4.899 1.261
1 1 1 1 1 1 1 1
.010 .721 .010 .225 .641 .703 .027 .261
.891 1.010 .920 .960 1.011 1.015 .908 .951
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
were able to predict NCLEX-RN failure; however, only 33.3% of NCLEXRN failure cases could be classified by the generated model. It can be concluded that the standardized tests were not accurate in predicting NCLEX-RN failure. This finding is consistent with Seldomridge and Dibartolo's (2004) and Vandenhouten's (2008) studies which concluded that any nursing content standardized tests were not accurate in predicting students who were likely to fail the NCLEX-RN. Limitations One of the limitations of this study was the unequal sizes of groups of students who succeeded and failed on the NCLEX-RN on the first attempt even though this limitation has persisted in most studies regarding NCLEX-RN predictors since the majority of candidates pass NCLEX-RN as shown in the national NCLEX-RN pass rate. Another limitation is that all of this study's data were collected in one traditional baccalaureate nursing program of a public university located in the Midwest. A diversity of different curriculums, student support programs, remediation programs, student populations, and other aspects, which are unique to each nursing program, is not captured in this study. Also, only the ATI Content Series was used as variables of the standardized tests in this study although various products of standardized tests offered by vendors are available besides the ATI Content Series. Each test of the ATI Content Series had different question numbers. Although individual adjusted scores on each standardized test were used for this study, the different numbers of the questions could limit comparisons between effectiveness as an NCLEX-RN predictor between tests. In addition, individual differences were not captured in this study. Each participant's NCLEX-RN outcome could be affected by different motivations, maturations, life situations, emotional reactions to NCLEX-RN, and other aspects. Although students did not achieve benchmark scores on the standardized tests, the nursing program might provide remediation to these individuals, and this might be effective in helping the students to succeed on the NCLEX-RN. Also, students who achieved low scores on the standardized tests might be concerned about their ability to be successful, and this factor might motivate them to prepare more extensively for the NCLEX-RN. The other consideration is that students might feel comfortable with NCLEX-RN after achieving benchmark scores on the standardized tests, and this might reduce their intension of preparation for the NCLEX-RN and result in failure on the NCLEX-RN. These factors can limit the ability to generalize this study's findings to other nursing programs and standardized tests products and have to be considered when interpreting this study's results.
1527
remediation can be provided. After early remediation is implemented, the comprehensive standardized test can be used as a mid-point indicator of the remediation's effectiveness before taking the NCLEX-RN. This study also raises a question about the predictors of NCLEX-RN outcomes. Why are adult medical–surgical, pharmacology, and community health standardized tests effective in predicting the NCLEX-RN success while the other subject tests are not? It can be assumed that adult medical–surgical and pharmacology standardized tests can predict the NCLEX-RN success because the NCLEX-RN has relatively many questions related to those areas. Also, it can be assumed that community health standardized tests can predict the NCLEX-RN success because the subject requires integrating nursing knowledge and skills for health promotion and public health. However, more studies are needed to find evidence that can support this assumption or answer the question. In order to have more accurate results of regression models predicting and classifying NCLEX-RN failure, it is recommended to include a large number of participants who fail on the NCLEX-RN if possible. This will result in having more accurate results of regression models predicting and classifying NCLEX-RN failure. In addition, since there are different kinds of standardized tests, which are offered by various commercial vendors, more studies including various products are needed to investigate strong NCLEX-RN predictors and compare the effectiveness of each nursing content standardized tests. Therefore, it is possible to have more powerful evidence showing if the findings of this study are repeated in other studies having variables of different products of the standardized tests. Moreover, the NCLEXRN is revised every three years, and the next revision will be in April 2013. After the revised version of NCLEX-RN is conducted, commercial vendors will accordingly revise their products of standardized tests based on the revised NCLEX-RN. There is a need for having longitudinal and repeated similar studies in more diverse populations and in many different nursing programs.
Acknowledgment The author would like to thank Dr. Marcia Miller, Associate Professor of the Department of Baccalaureate Nursing, Indiana State University for her support, encouragement, and efforts throughout this research and for reading and editing the manuscript. Funding Sigma Theta Tau, Lambda Sigma Chapter provided a scholarship for this research. The funding source was not involved in the study design.
Conclusions Since the nursing shortage is a critical issue in the U.S., more RNs are needed to meet the increased demands for nursing care. Nursing students have to pass the NCLEX-RN to become RNs. The NCLEX-RN is one of the important evaluation factors of the quality of education provided by nursing programs. Graduates' NCLEX-RN failure causes many negative effects on graduates, nursing programs, consumers, and society. In order to improve an NCLEX-RN pass rate and prevent negative effects of the NCLEX-RN failure, supporting students who are at risk of failure on the NCLEX-RN is indispensable. Since identification of the students at risk should be preceded in order to provide remediation to them, many studies have investigated effective NCLEX-RN predictors; however, common variables of the studies are limited to admission criteria, comprehensive standardized tests, and performances in courses with controversies over the results of the studies. The result of this study indicated that adult medical–surgical, pharmacology, and community health nursing were powerful NCLEX-RN success predictors. Through using these predictors, students at risk can be identified before taking comprehensive standardized tests usually conducted in the last semester of nursing programs, and early
References American Association of Colleges of Nursing, 2011. Nursing shortage fact sheet. Retrieved from http://www.aacn.nche.edu/media-relations/NrsgShortageFS.pdf. American Association of Colleges of Nursing, 2012. Nursing fact sheet. Retrieved from http://www.aacn.nche.edu/media-relations/fact-sheets/nursing-fact-sheet. Assessment Technologies Institute, 2007. RN Content Mastery Series 2007 national standard setting study: Executive summary. ATI, Stilwell, KS. Catalano, J., 2009. Nursing Now: Today's Issues, Tomorrow's Trends. F.A. Davis Company, Philadelphia, PA. Chang, C., Price, S., Pfoutz, S., 2001. Economics and Nursing: Critical Professional Issues. F.A. Davis Company, Philadelphia, PA. Greenpan, V., Springer, P., Ray, K., 2009. A tri-nodal model for NCLEX-RN success. Nurse Educator 34 (3), 101–102. Griffiths, M., Papastrat, K., Czekanski, K., Hagen, K., 2004. The lived experience of NCLEX failure. Nurse Educator 43 (7), 322–325. Harding, M., 2010. Predictability associated with exit examinations: literature review. Journal of Nursing Education 49 (9), 493–497. Humphreys, J.A., 2008. Academic and nonacademic predictors of future success on the NCLEX-RN licensure examination for nurses ((Doctoral dissertation). Retrieved from ProQuest. (3315143)). Mertler, C.A., Vannatta, R.A., 2010. Advanced and Multivariate Statistical Methods. Pyrczak Publishing, Glendale, CA. National Council of State Boards of Nursing, 2012. NCLEX examinations. Retrieved from https://www.ncsbn.org/nclex.htm.
1528
Y.-J. Yeom / Nurse Education Today 33 (2013) 1523–1528
Norton, C.K., Rele, M.V., Cox, C.W., Farley, J., Lachat, M., Tucker, M., 2006. Ensuring NCLEX-RN success for first-time test takers. Journal of Professional Nursing 22 (5), 322–326. Roa, M., Shipman, D., Hooten, J., Carter, M., 2011. The Cost of NCLEX-RN Failure. Nurse Education Today 31 (4), 373–377. Seldomridge, L.A., Dibartolo, M.C., 2004. Can success and failure be predicted for baccalaureate graduates on the computerized NCLEX-RN. Journal of Professional Nursing 20 (6), 361–368.
Sifford, S., McDaniel, D.M., 2007. Results of a remediation program for students at risk for failure on the NCLEX exam. Nursing Education Perspectives 28 (1), 34–36. Uyehara, J., Magnussen, L., Itano, J., Zhang, S., 2007. Facilitating program and NCLEX-RN success in a generic BSN program. Nursing Forum 42 (1), 31–38. Vandenhouten, C.L., 2008. Predictors of success and failure on the NCLEX-RN for baccalaureate graduates ((Doctoral dissertation). Retrieved from ProQuest. (3306523)).