Factors that predict performance on the national certification examination for nurse-midwives

Judith T. Fullerton,CNM,phi, FACNM,and Richard!Sev&no, MS

A national certification examination for entry-into-practice of nursemidwifeywaslnandatedbytheprofessional association of nursemidwi~theAnIe&an~Of Nurse-W (ACNM). in 1971. -rhiiexaminattwtassesses~ competencies (knowledge). The skills and abilities of nurse-midwifery pratticemdbyttre@dllCati program a&d must be demon-colrespDndence IMI,

C?W.

RD.

F&C?W,

Lane.%UlfX?g$XCA92126

IO-T. 7717 hlyXl

F&Ypaint

stmtecl prior to admission to rhe certification examination. The examination is developed and administered by the ACNM Cert%ation Council, Inc. (ACC). a private tesling corpomiion acting on behalf of the American College of Nurse. . l%dwwes The ACC sets eligibility reqtaimxmts for the certification examination. These eligibility requirements include graduation from an educationprogmmthathaslxenaccredited or is pen&g acaeditation, bytheACNMCM.icmdieditition.TheI)ivisionofAcaeS.iitatkIn

accredits nurse-rnidw3ery education programs that admit RNs from broad variety of en&y-inbmnMg pathways. The llm-W&

a

CdtionppEiOffkp2pEdWfor

nurse-mkkriferypacticeon~ pgd&-$==wpmsams~ praaensi0nal-L basklzldiate,andga&atehvek (1). ‘The ACNM has enanmgd innovationinthede5ignofprogmnsinordertoi-to nurse-~ edueatbn in pursuitofthegoEdof10,aKtmidvkRs by the year ml1 G!).

ate Record Ezarninalion )RarsofP=P=-YP-e-

AapTgrambaseMei~ on the Core Competencies &r p Fkwrthn u9m SbndpndsoJtheAmeriorrnC~ c$ Nurse-W (1993). and the @&filKSOftheAC~~Of

ActudWm (1); however, lmny~hacademk~te-

thereare

ifgiL=-= wiihthem Mewrthetsa theprogWn&ectu?*augraduatesof these programs are eligible candidates for the national osrh6catiot-1 exWtinaUon in nurse-midwifery. Successful candidates may, in turn. forward evidence of this credential to their state boards, as a sole. or equivalent. route to nurse-midwifery licenslJn2 (31. State jurisdlcUons that accept na W catiftcation as a basis for licensure rely on the ACC process as an unbiased assessment measure. Therearealsoseverallegalandprofessionai standards that dictate the requirement that cer&ation examinations be reviewed to determine their potential for bias for members of any ethnic or minority group (4, 5). Demographic factors of concern

T. Fullerton receiwzd from Wayne State Uniwtsity. Judith

her J3SN

her MS

and cen!$c& in nurse-midwifery brn Columbia, and her PhD in he&h edducationl~ adminidmdon jiwm

Temple University. Fhilodelphia. Pennsyhnia Dr. FuNerton IS the Test Consdbmt and the Chair of the Research Comrttitzee Ce@icatbn Counci!. the Ametin Co/&e NtUSdidtiU~.

jar the ACNM She is a Fellow of

oj

Richard Seuerino nxctwd his BS in M mathematics and his MS in smstia~sanDiegos&are Uhxmity, San Diego, Cahjomia. Mr. Setwino is a teearch statistician ot the Queen’s Uedicd Center, Hof~&l~. kkwaii, and member of ihe ACNM Crzrt$cation Council Research Committee.

20

FEVEWOF’DiEtFlERAfllRf lhnerous studies have been conducted to predict the performance of students on the resistered nurse entry-into-practice licensure examination (6-8). Factors of interest have included Scholastic Aptitude Test scores, the level of educAional preparation of registered nurse studenk feg, amciate versus baccalaureate), grade point averages (GPA) obtained in preclinical science and nursing core courses, and faculty ratings of clinical pizrhnar~ce. Resdk indicate that standardized aptitude measures (such as the Scholastic Aptitude Test) and gmde point averages obtained in nursing clinical courses were the best predictors of performance on the nursing licensing examination. Similar investigations have been reported in the medical (9. 10) and allied health professional (11-13) literature. These studies generally include an analysis of preadmission, demographic, and academic variables. Results are generally consistent with the findings from the studies of nursing studenk. Scores achieved on standardized aptitude measures and clinical science GPAs help to predict certification examination performance. Studies particularly relevant to thii investigation are those that focus on prediction of the performance of nursing clinical specialists or numpractitioner studenk on the specialty certification eXamiMbOfts (14. 15). These studies have Muded Gradu-

scotts.

EEhpre&trxaArw; nb sfudenk (14) ti C&3dtlWtht2-WAobeLred WhikenroBedi?l~~~countedfor2496oftheto&ilv&anos expkdl&(27%)intheoveraUcertiBcauonexaminationscore(mlduple-anaqsk)vMaMiy sis of variance methods indicated that RNs with bachelor’s or master’s degrees, and candidates enrolled in gmluate. rather than cedhate, pmgmns of nurseane&esia education did achieve higher mean scores on the ewmination. METHODS Design

AND FRCXXDUFE

and Sample

This re&ospecBve study analyzed the performance of aI nurse-midwifery candidates who were examined for the first time be!ween January 1988 and April 1994. The study sample was composed only of candidates who took any one of the hve forms of the criterion-referenced, modified essay forms of the examination that were first instituted in 1988. Instrument

and Measures

The independent variables in this study were age. ethnicity. marital status, U.S. or foreign education as a registered nurse, U.S. or foreign education as a midwife/nurse-midwife, academic level of registered nurse education, years of nursing pmctice prior to midwifery/nurse-midwifery education, academic level of nmsemidwifery education program, and highest educationaI degree obtained The dependent variable in this study was the overall certification examination haw) score ICES). The subscores achieved in each of the major content areas (ie, antepartum, intrapat-turn. postpartum. newborn. farndY planning/gynecdogy. and ProfQ

sbndlls5ue5trpprend~as rk!pded--dlhe mElkau&h-ofmh snnfzofmese~andtk~ -cdthe~whkh werenwerk&?n&dcoktndependaothQr-iflathavebeen usedRstlr&r~dthatwere ofitwe3tbtheACC,~sueh -as-alassesmKRtmeasUreS,fheWAin nurse-mlhrriiery

core

courses,

IWedwatedasaregkwwtnuseh: rIits?z FWdipbmaorWasaregtsaered-

-sPwmd

=0-Yand

theowaEGPATheACCcor&ddreqctestingthesedatahdtmticdprqpams However.tttedlva3ityofacadelnkplugtamb?veLs inpr-dmiskdCkt0urldeMMme sion requirements In ackiilhn the

m Fast etdwakd as a midwif&urse-midwife Fmmby unttedstates Missing ~~~~wwm kid

in:

certificate

Or[f2llhtiOlldth?cuniarlum~ries

widely among progmms, and the ACC could not be certain that the lzouEegradeswolltdreflectsimilar content, for example, grades achieved in tz&itional (integrated) or modular-focused core courses. If a

TABLE1 Demogmphk-erlstics of the Total Study Sample Vcuiable

n

Marital st3tus Never married F%wiody manied cluTently malTled Misstrig Edlntcity E (non-Hispanic)

214 125 605 102 861 : 13 I 4 2

1 wgrt

Staxtardd&a%n Mtnimwn = 24 X&mum : 62 Medlan = 35 9ottlPerceThk?=45

22.7 13.2 64.1

= 6.2

degree obtained:+

Associate degree BaccAureate Post-baccalaureate MastQis Pc5t-master*s Doctorate Missing Years of nursing pm* before nurse-midwifery education: Mean = 9.4 Standarddevwion = 6.2 Minimum=1 -WI=39 Median=8 9Od-1Percena - 18 ‘All value5 exclude missing data thAding

Hispanic Asian/pacific Istander IndianAl&an MiddleEatem East& be&

Pemm*

!TLtzze M-w l+i$estm~mic

85.9 7.3 3.1 1.3 0.7 iit: 0.9

ma&r’s

aquiml

during

nurse-midwhy

variable resp:xw rate wds received from progmms, and in light of the difficulty predkted in determining equivalence of the data to be requestedfr0mprogiams.itwoukibe likely that a complete data sel would not be zwtiiable for s&r&cant numbers of candidates. Therefore, the shdy variables included only data at hand, ie, those items for whkh data are requested on the application for examination. Even these data are IWted. however.Ged2roftheczlndkh

edwat~~~~.

ically requested on the application form.andtherefore.nodatawere available. In addition, car&dates may decline to provide certain personalctat23(ethntcityandmaritalstatus).

F’rocedure

Stxperoentofthecandidateshad

Data analysis procedures included stepwise multiple r-n. using the oved CE as the dependent variable. The certification examinationscoreachievedbyeachcandidate was bzdorrnecl to a standard scale (a e score) so that data from the five forms of the examination

could

be pookd for the analysis. The demogaphk, preadmission, and academic factors were used as independent variables. The same independent variables weTe also used in a logistic regmsion that used the candidate’s pass/fail status as the dependent variable. The level of signifiante was set at .05. RESULTS Demographic

Charaderisks

The demographic statistics for the population are lied in Table 1. The majority of candidates were 45 years of age or younger, although, there were 23 czudidate.s in the 50 to 59 year age range and two candidates over the age of 60. The majority of candidates were married (64.1%) and of white, non-Hispanic ethnic@ (85.9% of candidates who dklosed this information). Preadmission and academic characteristicsare indicated in Table 2. All paths to registered nurse education wre reflected; the largest number of applkants (50.5%) received their RN in a baccalaureate program. Eight percent of the candidates had received their nursing education in foreign progmms. The mean number

recekred their midwifery OT nursernidwhy education in foreign pro grams The majority of candidates had completed their U.S. nursemidwifery educaBon in master’s programs (53.6%); however, there were also many who were graduates of

precettification (5.4%) and certificate-bask (40.6%,) progmms. The master’s degree was als4.3the

highest academic degree obtained by the majority (53.1%) of candidates. The degree was obtained either prior to or at the time of nurse-midwifery studk.s. A total of 666 (63.5%) candidates had moved beyond their initial RN edudonal tevel. Eighty-six indhiduak 18.2% of the sample) had the diploma as their highest educational degree, and 85 individuals (8.1 W) remained at the associate degree level There were 1.046 candidat& ill ihe total sample. The scores for 74 candidates were pending at the time of the analysis. Of the 972 scores available, 907 (93.3%) met or exceeded the passing standard, and 65 (6.7%) did not meet the passing

AWmodelMderrebpatP a~s&pw&emanner.Theregressbnanalyses~perkm#d~ only those candidates for whom compkte dmmgrapltk data umtz adhle f/V = 778). All tndq>endent

varlabfes were considered for entry. Five variables contributed significantiytotheregresSonmodelThese variables were ethnkity. age, years of nursing practice prior to nursemidwifery education, highest educational degree obtained, and academic level of nurse-midwifery education. Table 3 shows the order in which the variables were entered into the mod4 the aggregate R2 (the pro-

portionofthevarkmce inthedependent variable that can be attributed to the variance in the independent variableIs] at each step), and the change in the R2 due to the addition of each variable to the model The R2 for the final model is 0.162, indicating that 16.2% of the variation in CES is ex-

pidined by the variance in the five independent variables in the model The categorical variable of ethnicity was collapsed to five levels. This variable alone contributed 11.5% of the variation (P < .OOOl) in the certification examination scores of the

TABLE 3

Regmssion Summary Table for Certification Examination Scores* Variable (hl o&r oj Enfryl

RZt

Ettmkiiy Age

Change

in R2t

Y Value

0.115 0.131

0.016

<.a301 .ooo2

Nurse-midwifery educational progmmhl Years of nursing practice ~&t;urse-midwifery

0.146

0.015

*cm7

Highest education degree

0.154 0.162

0.007 O.OCB

:E

‘The hrst variable. ethnicity. pdicted 115% of the wsianco in cettificationexarrtinahon suxe. The next four variables. in c&r. explained an addtbonal 1.68. 158. 0.7%. and 0 8% of the variance. +Fwres

for a total 0: 16.2% rounded IO dree decid

places

uknL4hik!iHlaeoornpiapdavith blrpcks, the difference was hi, it was 1.007 higher compared titi~ Asians and l-igherwhenawnpamdwithaElotl?eT ehkgroupscombinedHispania were jilredicted to scme 0.570 thanbtadg0.515highe.rthan~ Paiflc lslmdq and 0.490 than all other ethnic groups white (on average). fThesedifferencesarenotabso!ute number5 rather, theyarediffer~

1.062 when 0.982

higher higher except

behueen~prediCbedZSCores,On

tbeav~,whenaucltherfa~ are co& to be equal to each other. Poave values dect better prahmance compared with the referetxegroup.Avalueofl.OisequivalenttoafuU&ndarddeviation+) Age had a slight negative influence on the CES. For every increasing year of age, the predicted CB score deueasd by 0.029. This variable explained an additional 1.61% (P = .0002) in the variation of scores. The nurse-midwifery program Ifmel was a third sign&ant contibutortott4enmddDoctwal,ma5ter’s, and certificate level ondkiates each had a positi increase in the predicted CES when they were cornpared with precerhF:ation candidates (1.321 for do&ml, 0.360 for master’s, and 0.383 for certificate level candidates). Doctoml candtdates(n = 4)akohadaposftiveina-ease of O.%l wbell compared with master’s czudidates and an increase of 0.937 cor~pared with certificate -tes. f-buevw, when master’s ad certi$c.at~ candidates were comwed to each other, the advantage favored the certificate candidates.

Fmafly, years of nursing practice accounted for 0.74% variance in the examination score (P = .0099). For eachirrcnMngyearofpmctice,the predicted CE.S increased by 0.018. W!-rentherern&ing variakare ford into the model+ an additional 0.77% of tmiance can be explained, bringing the total to 17.0%. but thts increase is not stalHrzJIy sign&ant Moreover, some of these four variabfesafeFikelycon&tedwitheach other (such as interrbational- or u.s.based mkhuifery and registered nursing education); therefore, some of the additional variance expkhecl by these factors, that were not statistically significant, may also not be ptacticafly significant in the prediction model. The interaction (ie. the concurrent influence) of the five variables that were significant in the regression model was also tested. This testing was hited to firstder intemcWx (as opposed to the potential interacGonofallfactors,someofwhiihad several levels. working simultaneously) because of the absence of data for every possibfe combination of factor and fevel and because little practid usefulness could be envistoned for the information. Akhough entering lhese first-order interactions into the modei woufd increase the amount of variance in CES that COUldbeexphined(theR2),d,X-

AkJ&tiC-RgW#ionK#tMW6asnj3utedtoc!dfy~~pcedjeted topassortofaila5afurMionofthe WlTteindependent\riP*.Afnmwaldstepwisemethodwas~h ~e-t!!.wl~!R= of nursing practice We the or&~ variables that entered thic Iogkticregression model. Nurse-mfdu&ry education level was not sQnikmt using this approach, nor wils fhe highest It?vel of educalion Th4?sensitivityoftimodd(ttle abititytoconMIy&sstfytho6ewbo did, in fact, pass the examination) was9945%.chltheoth~the specificity of this mode.! (the abi?ity to correctly classify those who did noi pass the exarn&iation) was 21.28% Using these. results, the predicHve vahepositiw (theiikdhodtian aptiOri~OfpassingVJOUtd be correct) is computed to be 95.16%. The predicb vahe negative (the fikelihood that a Wing status could be actmat* p&ictedI ws 71.43%. DfSClBSKIN Nineidpedentva3iabkurereteviewedforinfluencecmnationalcertificatkmexzimination-Wresearch took two different appmches tottlel??viwofcandidatedah

The6rstmdmainbmsdadysis

mgmphjcc4adabkRvekwlethnidly,yearsfJfMnsingFtkepriorionurse-MeducaSJhhishest-mobtahed,andacademkLeveloftlle m-1 -had a !jwMdy s@tfkant Influenctghowever,the5ewereofltttIe practkal significance. The total amount of vafiance in candldabe scaa?sthatwas~byY five factors tn the regrfssion model was only 16.2%. The remaining (approMI 83.8% k WLexpbined by demogmphk variables Asecondfocusofanalyskwasto explore whether a ciassification dmne could be developed to predict the likelhood of any individual candidate to pass the examination. This focus is particularly appropriate to uitdn-referenced examinations. in which the important discrimination behveen candidates is not the magnitude of the saxe. but mther, whether each candidate has achieved a predetermined score level. The very high predidive value positive indicates that the three factor5 could be used to predict accurately a candidate’s liketlhood of passing the examination. However, the mediocre predictive value negative indicated that these same three factors are not wry reliable indic;ators of not passing ti exam. Therefore, again, these vatibles do not yield very much predictive value. However, the use of the logistic-regression model for classification was limited in this instance because of the large imbalance in sample sizes of the passing and notpassing groups. Such a difference in sample size will always favor classification into the larger group. decreasing the specificity and the predictive value negative (16). Some of the unexplained variance in scores can be attributed to the SOUK~Sof error that are inherent in every examination (random error). Some of this variance might arise from variables that were not included in this research because they were available (not gathered from edu-

24

~).Someofitm.ight afkefmmtbtestingprcressitse~. TheAcctakes~steppoesiMeb minimbtheerrorduetofactors uiWniiscontrd.tiqirdsWgonexir-ltmw~~cotremeivhish effkknb for the rater/scorer5. fogicalth2fo~,toatMbuteagreat

It is

aYnotmtoftheu~~ce to the actual differences in knowledge demonstrated by the examination candidates, which is the objective of competency-based testins Neither of the models reported here are inclusive of all variables that have been reported in similar studies and, therefore, have only limited ap plicabiity for use in academic admissions purposes. For example, administrators of educational programs may wish to consider preadmission grade point averages, in addition to these demographic variables. as a likely indicator of eventual graduation from the program and success on the national cerl&ation examination. Because of this limited application of the prediitiw model, and because the predictive value was moce5t at best, the co&cients for the regression model have not been reported in Table 3. As previously stated, although five variables made a statistically significant contribution to the amount of variance that could be explained, the total amount of variance that could be predicted on the basis of these factors was not practically significant Nevertheless, these factors deserve some comment There is discussion in education and policy arenas concerning whether nurse-midwifery education should occur only at the graduate level. In this study candidates with the baccalaureate degree as their highest educational degree and candidates who had enrolled in certificate programs of nursemidwifery study were predicted to perform slightly better, on avenge, than candidates with less or with more educai5on and candidates en-

Thewisa?sumuchdjQTyiRh cemtngthe~of~~ lkmprlorboelldmmt fnv educatlol-l.DaBfromthis~$w cated that ezh additional year of nursing practice contibuted a slight positive influence to the predi&d score. Netedwk. the same cmsenmtive intqretation of data must prevail. The amount of variance that was predicted by this factor was statisticafly but not practically significant In addition. each additional year of ageiedtoapredictionofaslightdecrease in CE.S, acting as a bafancing or buffering factor. The effect of missing data on the-se conclusions also deserves comment. Candidates may decline to provide selected information. Ethical and kgal principles define certain rights to privacy. If each of these candidates who declined to provide ethnic datd were of nonwhite ethnic@ and if each had failed the examination, then the differences in CES for nonwhite candidates would have had sign&ant practical as well as stali&cal importance. However, one-half of the 44 candidate (4.2% of the sarnplel who dectined to pmvtde ethnic data achieved a passing score on the exam. The ACC has recently proposed several srrategies to identify gtaduate:candidate ratios on a number of issues and will continue to IX vigilant for any evidence of adverse impact SUMMARY The ACNM Certification Council is responsible for determining that the national certification examination in nurse-midwifery is not biased lie, in the language of the law. does not have an “adverse impact” on) te ward any protected group. Educa-

=:p”-cala&%ez,C Imsaerlsoi.~~,srrbs GsBindum EaalaawQPrtla awdtard&tddA1BBI~ emmWrkmpezbrraanoesdGemli38ea kd lm3$s-a 11.-sAw~~ppedid!amesontbe~Boad~ Iiygi~ERmiMgon~dDengl l-lygem 1989m73-6. 5.EqualEm&ymentOppwhtnity GrrmrtsicnlGvilservke-Il. Department of Labor. et al. ~~anptoyee-pncedwes. Federal 315.

Re$skr

Uniform

1. Manual of kential Washington (DC). Amerkan Nurse-Midwives. 1993.

Documents CoIkge

2. Natimal Commission on NurseMidwifery Education. Educating nursemidwives: a strategy for achieving

of

sialal1993;14%43.

1978$3:3829&

13.

6. Froman RD. Owen S. P&i&g performance cm the National council censureExamina~.WestJf’Ju.rsRes 1989;11:334-46.

RWEREWES

12. FidierFRAnapp8catSondpiactld~lmassersingthel-et&dvalk6lyd~lims.EtehaionandtheHealth~

Li-

Bra&y Fq Conner admissionand-e oflhescoresontheNationdFkg&at&n

WD.

se-

-for-irrdAmDietAsSGS 1993;93:9I23.

7. Lengachet CA, Keller R. Academic predictors of success OR the NCLEX-RN examhMon for assodate degree nursing students. J Nurs Educ 194o;m1s9.

14. Zaglanicmy KL Factors which ptediclpafonnanceonthe~certifldimexamtnatlonformIrseanestt&

8. Foti 1, DeYouncl S. Predktiq cesontheNationalCouncillicensure Examination-Registered Nurse, &erpieceofthepuzde.JProfNurs 1991;7:9%104.

suc-

15. Raymond MR The relationship between educational preparation and performame on nursing wtif3cafion examinatiw. J Nurs Educ 1982127:&9.

9. Norcini Langdon LO,

LJ. RL.

JJ, Day Kimball

SC, Grass.0 HR. Popp

An-

tists.JoulllaloftheA~~ of Nurse Aneskt%

1992$&M.

16. Haner DW, Lemshow kgisticregre&on.NewYorkJotmWiky & sons. 1989.

S. Appkl