Protecting the Public Through the National Practitioner Data Bank and Nursys® Compliance: An Exploratory Analysis

Protecting the Public Through the National Practitioner Data Bank and Nursys Compliance: An Exploratory Analysis ®

David Benton, RGN, PhD, FFNF, FRCN, FAAN, and Nur Rajwany, BS, MS Introduction: The National Practitioner Data Bank and Nursys ® are important mechanisms for protecting the public against

practitioners who pose a risk. Aims: To assess the level of performance associated with uploading final board of nursing (BON) actions to Nursys and the National Practitioner Data Bank to determine whether the BON governance model or usage of the National Council of State Boards of Nursing’s (NCSBN’s) agent service has an effect on performance against the 30-day federal reporting standard and the NCSBN–American National Standards Institute 15-day reporting standard. Method: A secondary analysis of data relating to completed BON actions submitted to the Nursys database was conducted for all entries held by NCSBN for 2013 to 2015. Results: A substantial number of BONs are not compliant with the 30-day federal reporting standard. BONs with an independent governance structure performed better than those with an umbrella arrangement (Mann Whitney U test of data for years 2013, 2014, and 2015, z = −2.726, −2.180, and −2.263, with p = .006, .029, .024, respectively). Conclusion: The use of routine collected data shows potential for measuring the effect of improvement strategies in achieving the 30-day federal standard and the 15-day NCSBN–American National Standards Institute standard. Umbrella BONs are generally less effective than independent BONs in meeting the 30-day federal standard and thus place the public at increased risk. Some BONs have room for performance improvement.

Keywords: Board determinations, independent and umbrella boards, National Practitioner Data Bank, Nursys®, regulatory body performance, statistical process control

T

he National Practitioner Data Bank (NPDB) and Nursys® are important mechanisms for protecting the public against practitioners who pose a risk. Created by U.S. Congress, the NPDB is charged with gathering information on practitioners who pose a risk to the public and is a key component of quality improvement of health care (Health Care Quality Improvement Act of 1986). As a data bank, it provides an important portal for information for both aggregated and practitionerspecific details to authorized employers, state agencies, and payors (Health Care Quality Improvement Act of 1986). The system charges users a small fee to access the data, and although the fee is modest at the individual unit-cost level, the aggregate total can be considerable for large employers with many practitioners. Information on the use of the system is readily available on the NPDB website, and the U.S. Department of Health and Human Services (2015) has developed and published a guide for nurses on the data bank, which includes the types of information that the data bank contains as well as a set of responses to frequently asked questions. Nursys provides a similar service to the boards of nursing (BONs) and the public free of charge; it includes infor46

Journal of Nursing Regulation

mation on all practitioners, not just those who have had an action against their license. However, like any data sources, the NPDB and Nursys are effective only if they contain up-to-date, readily accessible, complete information on nurses who have been judged by the court or a BON to pose a risk. A U.S. Department of Health and Human Service rule (2013) requires that after a BON completes its investigation, follows due process, and reaches a final determination, its judgment should be coded, and the required information should be uploaded to the NPDB within 30 calendar days. Via the American National Standards Institute (ANSI), NCSBN has a more challenging standard of 15 calendar days, and via operational policy, the Nurse Licensure Compact administrators have a standard of 10 working days. However, the use of working days as the basis of measurement when data comes from multiple sources is flawed because variations in local holidays introduce variations in measurement from one state to the next. Meeting the NPDB and Nursys standards is important for public protection, particularly when considering nurses who have more than one active license, because NPDB and Nursys provide

a means of alerting other BONs and enabling them to take steps to initiate their own investigation. Similarly, if the nurse moves and uses a second licence in another state to gain employment, the NPDB and Nursys can provide important information to enable employers to make informed decisions about the appropriateness of the nurse for employment in their organizations. Hence, timely, accurate, and accessible information is a critical component in protecting the public.

Data

Using Nursys to Support NPDB Compliance

Analysis

To support BONs in fulfilling their NPDB reporting requirements, NCSBN uses its Nursys system to upload the necessary data. Nursys is a database product developed by the NCSBN in collaboration with state BONs to provide a comprehensive information system that facilitates the exchange of data among BONs and provides key information to the public and employers (Poe, 2008). At no cost, Nursys provides a means of collating information on a nurse from different BONs and alerting the nurse or the nurse’s employer to changes in the nurse’s license status via the Nursys eNotify system. All information on an individual is aggregated and viewed through a single query. In other words, the information about a nurse with multiple state licences is pulled from the various sources and displayed on a single webpage. The origins and ownership of the data remain with the source BON, which is clearly indicated on the resulting output; thus, the approach enhances transparency, efficiency, and public safety. In addition, via Nursys, NCSBN automatically uploads reported BON actions directly to the NPDB for BONs that wish to do so. Thus, Nursys provides not only a useful service but also a means of providing BONs with data to monitor their own performance against the NPDB’s 30-day and other relevant standards.

Aims This study was intended to assess the level of performance associated with uploading final BON actions to Nursys and their subsequent upload to the NPDB. In addition, data are analyzed to determine whether the BON’s governance model—umbrella or independent—or its use of the NCSBN’s agent (Nursys automatic upload service) has any effect on performance against the 30-day federal reporting standard and the NCSBN–ANSI 15-day standard.

Method A secondary analysis of data related to completed BON actions submitted to Nursys and NPDB was conducted in relation to all entries held by the NCSBN for 2013 to 2015.

Volume 7/Issue 4 January 2017

Data for 2013 to 2015, the most recent complete years of data, were extracted from the Nursys database on a state-by-state basis. Data provided the number of BON actions in the selected year and the number of days from the finalization of the action until the data were uploaded to the Nursys database. In addition, reconciled data between the NPDB and Nursys were used to identify the percentage of actions not uploaded by the 15- and 30-day standards.

Descriptive statistics for 2015 (mean and standard deviations) were generated to plot group-based means and 95% confidence intervals to examine the effect on performance of the BON governance model and the decision to use NCSBN’s upload service or another agent. A Mann Whitney U test was used to test the following null hypotheses: Null Hypothesis One: How data are uploaded to the NPDB (by NCSBN or another agent) has no effect on 15- and 30-day performance. Null Hypothesis Two: The governance model, umbrella or independent, has no effect on 15- and 30-day performance. The Mann Whitney U test is a nonparametric test, does not require data to be drawn from a normally distributed distribution of data, requires the data to be from independent groups, and is ordinal in data type. A two-tailed test was used, and the level of significance was set at p =< .05. In addition, trend data for year-on-year results (2013–2015) and more granular month-by-month data for a 36-month period (August 2013 to July 2016) were reviewed for anonymized BONs to examine how BONs may track their strategic and operational results. In the month-by-month data, the variation in results was reviewed by use of a statistical control chart technique. Statistical control charts come in a variety of types, and the choice of type depends on the data. Because the data being examined were attribute data, a p-chart was used to facilitate a close examination of performance with minimal mathematical complexity (Perla, Provost, & Murray, 2011). Statistical control charts, through a set of simple rules, detect shifts and trends in performance, enable identification of nonrandom patterns or runs in the data, and detect astronomical points where an unusually large deviation appears as a blatantly different data point (Mitra, 2016).

Results Data from 51 BONs were analyzed. The majority of the BONs licenced both registered and practical nurses. Of the 51 BONs, 24 had umbrella governance arrangements, and 27 had independent governance arrangements. NCSBN acted as the data processing agent (it submitted BON action information) to the NPDB for all but 12 of the BONs. www.journalofnursingregulation.com

47

FIGURE 1

% of Decisions Uploaded Within 15 Calendar Days

Percentage of BON Decisions Uploaded to the National Practitioner Data Bank Within 15 Calendar Days Using NCBSN vs Other Agentsa 90 80 70 60 50 40 30 20 10 0

NCSBN Agent Other Agents National Practitioner Data Bank Upload

Note. N = 51. BON = board of nursing. NCSBN = National Council of State Boards of Nursing. aError bars represent 95% confidence intervals.

TABLE 1

Mann Whitney U Test of Model of Governance (Umbrella or Independent) Year

p

z

Mann Whitney U

2013

.006

−2.726

195.50

2014

.029

−2.180

226.00

2015

.024

−2.263

221.50

FIGURE 2

% of Decisions Uploaded Within 15 Calendar Days

Percentage of BON Decisions Uploaded to the National Practitioner Data Bank Within 15 Calendar Days by BON Governance Structurea 90 80 70 60 50 40 30 20 10 0

Umbrella

Independent

Note. N = 51. BON = board of nursing. aError bars represent 95% confidence intervals.

In 2015, seven BONs met the NPDB standard of data entry within 30 days, and three BONs met the NCSBN–ANSI standard of 15 calendar days. For each BON, the percentage of actions that failed to meet the NPDB 30-day data entry standard ranged from 0% (the seven BONs that had all actions entered within 30 days) to 81.9% (the poorest performing BON). Regarding the 48

Journal of Nursing Regulation

NCSBN–ANSI standard, two boards of nursing met the 15-day standard and the poorest performing BON failed to uploaded 96.14% of its actions within 15 calendar days of the decision. Figure 1 illustrates that the 95% confidence intervals were narrower and that the mean percentage of decisions reported within 15 calendar days was higher when NCSBN acted as the agent for processing NPDB data; however, the results were within the margin of error. The Mann Whitney U test for both the 15and 30-day standards across all 3 years of data (2013 to 2015) showed no difference in performance relative to how data were uploaded; hence, null hypothesis one was accepted. Figure 2 shows that the 95% confidence intervals were narrower for independent BONs than for umbrella BONs and that the mean percentage of BON actions uploaded within 15 days was slightly higher for umbrella BONs than that for independent BONs, but the difference was within the margin of error. However, a significant difference (p ≤ .05) for each of the 3 years was found regarding the 30-day standard (See Table 1) Accordingly, null hypothesis two was rejected for the 30-day standard. Independent BONs performed statistically significantly better than umbrella BONs. Longitudinal data from the Nursys database for three anonymized BONs are presented in Figures 3, 4, and 5. Because the number of nurses varies considerably from one state to another, percentage upload results were used to facilitate comparison between the BONs. Examining 15-day and 30-day performance can offer potentially useful insights into current performance and assist with identifying opportunities for improvement and setting future performance targets. Although year-on-year data offer insights into overall performance, the level of detail is insufficient to provide the granularity needed for the assessment and design of quality improvement interventions. Accordingly, Figure 6 shows a BON that had an erratic month-on-month performance. During the measurement period, the BON met the 30-day standard on only four occasions, and on two of those occasions, the BON did not have any decisions to report. The BON, therefore, demonstrated considerable room for improvement. When the control limit was calculated, the mean failure to meet the standard over the 36-month period was found to be 0.47 (47%). That is, just over half (53%) of all final determinations were reported within the 30-day standard. Because the number of BON determinations varied from month to month, the upper and lower control limit for each point needed be calculated separately. However, because of the considerable variability and high control limit, the upper control limit was 1. The lower control limit varied from 0 to 0.25. On two occasions (August 2015 and July 2016), the results achieved by the BON dipped below the lower control limit, indicative of a special cause variation. Understanding the reasons for this dip may provide insights into quality improvement opportunities. The remainder of the data indicate that common cause variation was occurring; that

Volume 7/Issue 4 January 2017

100

% of Decisions Meeting the Standard

Data originally submitted as requirements to enhance public protection via Nursys and NPDB have offered an important insight into the effectiveness of different BON governance models. Examination of year-on-year results could be used as a strategic indicator of public protection performance. When a 15- or 30-day standard is not met, BONs could use trend data to set challenging yet achievable performance targets. Figure 3 shows a BON whose 30-day performance was maintained while its 15-day performance deteriorated year on year. To meet the 30-day standard, this BON needed to achieve a 15% improvement. An examination of how uploads immediately after determination is made may provide a means of improving performance (within 15 days) and could contribute to attaining the standard. In Figure 4, the BON had a year-on-year improvement over the 3 years; at the end of the study period, the BON was exceeding the 30-day standard and uploading all decisions within 15 days. This BON may have expertise to offer other BONs to help improve efficiency. Finally, Figure 5 indicates a marked deterioration in performance at the 15- and 30-day standards. This BON, therefore, needs to take urgent action to improve performance. A more detailed examination of the data may be helpful in formulating and tracking a quality improvement strategy. When examined in detail, data offer monthly operational performance information capable of identifying adhoc anomalies in performance or recurring problems. Examination of the raw data relating to Figure 6 (numbers of cases as opposed to the ratio meeting the standard) demonstrated no correlation with upload performance. Closer examination of the data (which can be viewed on the journal website at www.journalofnursingregulation.com) of decisions uploaded in fewer than 15 days, those uploaded between 15 days and 30 days, and those uploaded within 31 days or more indicates random performance and may imply insufficient resources allocated to the task or a lack of clarity as to who is responsible for it. These suggestions would require further exploration and process mapping if a definitive improvement strategy were to be developed. The effect of improvement interventions could be detected and tracked using repeated statistical process control charting. Examining the effect of the BON governance framework and the agent who uploads the data to the NPDB identifies important findings to inform public protection. Although determining a definitive causal link to BON structure per se is not possible, the findings suggest that umbrella structures may have weaknesses that need to be rectified. Moreover, this study has demonstrated that valuable insights into current performance as well as opportunities for improvement can be identified as a by-product of data gener-

One BON’s Marked Deterioration in Meeting 15-Day Standards and Consistent Underperformance in Meeting 30-Day Standards for Uploading Decisions to the National Practitioner Data Bank

80 60 40 20 0

2013 ≤30 days

2014 ≤15 days

2015

Note. BON = board of nursing.

FIGURE 4

One BON’s Year-on-Year Improvement in Meeting 15-Day and 30-Day Standards for Uploading Decisions to the National Practitioner Data Banka 100

% of Decisions Meeting the Standard

Discussion

FIGURE 3

80 60 40 20 0

2013 ≤30 days

2014 ≤15 days

2015

Note. BON = board of nursing. a The BON achieved full compliance with the 15- and 30-day standard by the

end of the study period (2015).

FIGURE 5

One BON’s Year-on-Year Deterioration in Meeting 15-Day and 30-Day Standards for Uploading Decisions to the National Practitioner Data Bank 100

% of Decisions Meeting the Standard

is, random performance within the bounds of normal variation (Duclos & Vorin, 2010).

80 60 40 20 0

2013 ≤30 days

2014 ≤15 days

2015

Note. BON = board of nursing.

www.journalofnursingregulation.com

49

FIGURE 6

p-Chart of One BON’s Performance by Month in Meeting the 30-Day Federal Standard for Uploading Decisions to the National Practitioner Data Bank Proportion not meeting standard

1.2 1.0 0.8 0.6 0.4 0.2

Jun-16

Jul-16

May-16

Apr-16

Mar-16

Feb-16

Dec-15

Jan-16

Oct-15

Nov-15

Sep-15

Aug-15

Jun-15

Jul-15

Apr-15

May-15

Mar-15

Jan-15

Upper control limit Control limit

Feb-15

Dec-14

Oct-14

Nov-14

Sep-14

Aug-14

Jul-14

Jun-14

Apr-14

May-14

Mar-14

Jan-14

Feb-14

Dec-13

Oct-13

Nov-13

Aug-13

Sep-13

0.0

Lower control limit Anonomous state 1

Note. BON = board of nursing.

ated through day-to-day work. Data mining of information that is routinely collected can highlight variations in performance. Furthermore, after baseline measures have been established, statistical control techniques can be used to monitor and assess the effect of changes introduced to improve performance (Stapenhurst, 2005; Montgomery, 2013). Complaints management and resolution is gaining more attention, although the focus of the investigations tend to vary. The Alabama BON (2016) in its white paper sought to understand the nature of complaints and to determine whether certain factors, such as being a registered nurse or a licensed practical nurse, sex, education level, and previous history of discipline, had an effect. Furthermore, work in the United Kingdom conducted by the Professional Standards Authority (2011) sought to better understand how qualitative improvements to the fitnessto-practice and adjudications process can be achieved. However, few studies consider quantitative measures; most studies use simple descriptive statistics to examine measures, such as the rates of complaint per 1,000 nurses, the mean time taken to reach a determination, or the resources deployed to carry out the conduct function (NCSBN, 2008). Studies using parametric or nonparametric statistics to provide evidence on how to improve efficiency and effectiveness of processes or to provide definitive guidance on the effect of governance arrangements or the relative effect of different diversion schemes are even rarer. The authors, therefore, conclude that studies such as this one that use routinely collected data should be encouraged and replicated if regulatory bodies are to take full advantage of the potential of such information. 50

Journal of Nursing Regulation

Indeed, we suggest that a failure to do so could be viewed as a significant missed opportunity for BONs to fulfill their duty to protect the public and to improve the efficiency and effectiveness of their processes.

Limitations The number of nurses with active licenses in the United States varies considerably by jurisdiction, just as the number of citizens varies from state to state. Thus, the number of BON actions per month can be small, and the examination of month-bymonth data may be difficult to interpret. In addition, the use of bimonthly, quarterly, or biannual data may obscure actionable trends because of the confounding effect of aggregation of data over the longer periods. Accordingly, those BONs that can use techniques such as statistical process control should be encouraged to share their findings so that others may reflect on whether their improvement strategies have relevance in other settings. Other factors such as the size of the BON or the ratio of resources to the number of complaints handled may also have an effect and may act as confounding factors. Considering the wide variation in performance of umbrella BONs and despite the statistically significant findings regarding the apparent advantage of independent BONs in meeting the 30-day standard, further investigation would be warranted. Such research could identify any confounding factors that need to be considered before a definitive conclusion on the better model of governance can be reached.

Conclusion The timely availability of information on final BON actions regarding nurses subject to investigation makes an important contribution to the protection of the public. Delays in the availability of such information can place the public at risk. The use of the NCSBN data-upload service to the NPDB repository has no statistically significant effect on performance. However, the analysis demonstrated that umbrella BONs are generally less efficient than independent BONs in meeting the 30-day federal standard and, accordingly, place the public at increased risk. On the basis of these findings, some BONs have room for performance improvement. The protection of the public is only as strong as the weakest link; therefore, compliance with, at a minimum, the 30-day federal standard by all BONs should be a priority for performance improvement efforts.

(NCSBN). Nur Rajwany, BS, MS, is Chief Information Officer of NCSBN.

References Alabama Board of Nursing. (2016). Analysis of complaints and discipline against licensed nurses. Retrieved from https://abn.alabama.gov/ UltimateEditorInclude/UserFiles/docs/research/White%20paperAnalysis%20of%20Complaints%20and%20Discipline.pdf National Practitioner Data Bank, 45 C.F.R. § 60 et seq. (2013). Retrieved from https://www.gpo.gov/fdsys/pkg/FR-2013-04-05/pdf/201307521.pdf Duclos, A., & Vorin, N. (2010). The p-control chart: A tool for quality care improvement. International Journal in Health Care, 22(5), 402– 407. Health Care Quality Improvement Act of 1986, 42 U.S.C. §§ 11101 et seq. (1986). Mitra, A. (2016). Fundamentals of quality control and improvement (4th ed). Hoboken, NJ: John Wiley & Sons, Inc. Montgomery, D. C. (2013). Introduction to statistical quality control (7th ed). Danvers, MA: John Wiley & Sons, Inc. National Council of State Boards of Nursing. (2008). Commitment to ongoing regulatory excellence (CORE): Results of FY2007 data final reportumbrella vs. independent. Chicago, IL: Author. Perla, R. J., Provost, L. P., & Murray, S. K. (2011). The run chart: A simple analytical tool for learning from variation in healthcare processes. BMJ Quality and Safety, 20(46), 46–51. Poe, L. (2008). Nursing regulation, the nurse licensure compact, and nurse administrators. Nurse Administration Quarterly, 32(4), 267– 272. Professional Standards Authority. (2011). Modern and efficient fitness to practise adjudication: CHRE’s advice for secretary of state. London, England: Author. Stapenhurst, T. (2005). Mastering statistical process control: handbook for performance improvement using cases. Burlington, MA: Elsevier. U.S. Department of Health and Human Services. (2015). NPDB 101 for nurses: A guide to the NPDB and how it affects you. Washington, DC: Author. Retrieved from https://www.npdb.hrsa.gov/resources/ factsheets/Nurses.pdf

David Benton, RGN, PhD, FFNF, FRCN, FAAN, is Chief Executive Officer of National Council of State Boards of Nursing

Volume 7/Issue 4 January 2017

www.journalofnursingregulation.com

51