Measuring Acute Pain Over Time in the Critically Ill Using the Multidimensional Objective Pain Assessment Tool (MOPAT)

Original Article Measuring Acute Pain Over Time in the Critically Ill Using the Multidimensional Objective Pain Assessment Tool (MOPAT)

From the *University of Maryland School of Nursing, Baltimore, Maryland; †Medical Intensive Care Unit, University of Maryland Medical Center, Baltimore, Maryland; ‡ Surgical Intensive Care Unit, University of Maryland Medical Center, Baltimore, Maryland; § Palliative Care, University of Maryland Medical Center, Baltimore, Maryland; jjUniversity of Maryland Baltimore Graduate School, Baltimore, Maryland; {University of Maryland Medical System, Baltimore, Maryland; #University of Maryland School of Nursing, Baltimore, Maryland; **Virginia Commonwealth University School of Nursing, Richmond, Virginia. Address correspondence to Debra L. Wiegand, PhD, RN, CCRN, CHPN, FAHA, FPCN, FAAN, University of Maryland School of Nursing, 655 West Lombard Street, Office 445C, Baltimore, MD 20201. E-mail: [email protected] Received March 21, 2017; Revised October 7, 2017; Accepted October 11, 2017. This study was funded by a joint research grant from the University of Maryland School of Nursing and the University of Maryland Medical Center Department of Nursing. It was conducted as a separate but embedded sub-study of a larger project entitled Pain Assessment in Non-communicative Palliative Care Patients funded by the National Institute of Nursing Research (5R01NR009684, Deborah B. McGuire, PI). 1524-9042/$36.00 Ó 2017 by the American Society for Pain Management Nursing https://doi.org/10.1016/ j.pmn.2017.10.013

Debra L. Wiegand, PhD, RN, CCRN, CHPN, FAHA, FPCN, FAAN,* Tracey Wilson, DNP, ACNP,† Diane Pannullo, BSN, CHPN,‡ Marguerite M. Russo, PhD, CRNP, ACHPN,§ ,jj Karen Snow Kaiser, PhD, RN,{ Karen Soeken, PhD,# and Deborah B. McGuire, PhD, RN, FAAN** ---

-

ABSTRACT:

A valid and reliable instrument is needed to assess acute pain in critically ill patients unable to self-report and who may be transitioning between critical care and other settings. To examine the reliability, validity, and clinical utility of the Multidimensional Objective Pain Assessment Tool (MOPAT) when used over time by critical care nurses to assess acute pain in non-communicative critically ill patients. Twenty-seven patients had pain assessed at two time points (T1 and T2) surrounding a painful event for up to 3 days. Twenty-one ICU nurses participated in pain assessments and completed the Clinical Utility Questionnaire. Internal consistency reliability coefficient alphas for the MOPAT were .68 at T1 and .72 at T2. Inter-rater agreement during painful procedures or turning was 68% for the behavioral dimension and 80% for the physiologic dimension. Validity was evidenced by decreases (p < .001) in the MOPAT total and behavioral and physiologic dimension scores when comparing T1 and T2. Nurses found the tool clinically useful. The MOPAT can be used in the critical care setting as a helpful tool to assess pain in non-communicative patients. The MOPAT is unique in that the instrument can be used over time and across settings. Ó 2017 by the American Society for Pain Management Nursing

Pain Management Nursing, Vol -, No - (--), 2017: pp 1-11

2

Wiegand et al.

The International Association for the Study of Pain (IASP, 2014) defines pain as an unpleasant sensory and emotional experience associated with actual or potential tissue damage. Pain is a subjective experience, best described by the person who has it. Although the gold standard of pain assessment is patient selfreport, critical care nurses are challenged by patients’ decreased levels of consciousness as a result of illness, injuries, and iatrogenic and treatment factors. Unrecognized pain leads to unrelieved pain, which has multiple detrimental effects on patients, including immune suppression and increased cardiac and respiratory demands that can ultimately affect morbidity and mortality, as well as the development of chronic pain (Barr et al., 2013; Desbiens et al., 1997). Improving pain management can lead to better clinical outcomes, but a necessary prerequisite is a reliable, valid, clinically useful tool that clinicians can use to screen for pain and evaluate outcomes of pain interventions in patients who cannot self-report. Pain in the critically ill is a significant problem because it is often underrecognized and undertreated (Hall-Lord, Larsson, & Steen, 1998; Kabes, Graves, & Norris, 2009; Klein, Dumpe, Katz, & Bena, 2010; Puntillo, 1990; Puntillo et al., 1997). Patients who have survived their critical illnesses reported experiencing moderate to severe levels of pain and distress (Bjoro & Herr, 2008; Puntillo, 1990; Rotondi et al., 2002). Accurate pain assessment is associated with improved patient outcomes and decreased critical care length of stay (Ferrell & Schaffner, 1997; Kowal, 2010; Payen, Bosson, Chanques, Mantz, & Labere, 2009). Pain in the critically ill can result from a variety of causes such as endotracheal intubation and endotracheal suctioning (Bjoro & Herr, 2008; Puntillo, 1990; Rotondi et al., 2002). Patients have also reported pain as a result of arterial and central venous catheter insertion, phlebotomy, wound care, and positioning (Bjoro & Herr, 2008; Puntillo et al., 2001, 2004). Pain is associated with physiologic responses that result in the release of catecholamines, thus increasing oxygen demands and altering tissue perfusion (Blakely & Page, 2001; Hamill-Ruth & Marohn, 1999). Untreated pain can result in delirium, anxiety, cognitive impairment, sleep disruption, social isolation, and decreased function (Hamill-Ruth & Marohn, 1999; McNicoll et al., 2003; Schuler, Njoo, Hestermann, Oster, & Hauer, 2004). Patients surviving critical care who have experienced unrelieved pain are also at risk for the development of post-traumatic stress syndrome and chronic pain syndromes (Garcia, Peres, DeCubber, & Vincent, 2003; Lamer et al., 2004; Schelling et al., 1998).

The gold standard for pain assessment is selfreport, which allows for direct measurement of one’s subjective pain (Bjoro & Herr, 2008; Ersek, Herr, Neredilek, Buck, & Black, 2010; Herr, Coyne, McCafferey, Manworren, & Merkel, 2011; IASP, 2014; Payen et al., 2009). However, not all critically ill patients are able to self-report pain that they may be experiencing because of medically induced conditions, altered levels of consciousness, anxiolytic infusions, and/or the inability to speak because they have a tracheostomy or have an endotracheal tube in place (Hamill-Ruth & Marohn, 1999; Herr et al., 2011; Pasero & McCaffery, 2005; Stannard et al., 1996; Topolovec-Vranic et al., 2010). Thus, the need for a valid and reliable scale that accurately detects the presence and severity of pain in those who cannot communicate is critical to improving pain assessment and management (Bjoro & Herr, 2008; Kabes et al., 2009; Kowal, 2010; Topolovec-Vranic et al., 2010). Because acute pain is more common in critical care and is manifested by relatively objective indicators, most attempts to measure pain in this setting focus on acute pain rather than chronic pain. Two groups of variables have been used to assess acute pain in critically ill patients who cannot selfreport: physiologic and behavioral. Physiologic variables that may indicate acute pain include changes in systolic and diastolic blood pressure, heart rate, respirations, pallor, and perspiration (Gelinas, Fortier, Viens, Fillion, & Puntillo, 2004; Prendergast, 2002; Puntillo et al., 1997). Activation of the autonomic nervous system during acute painful events is considered to be responsible for these changes, but acute nonpainful events can also cause autonomic nervous system activation, thereby confounding the use of physiologic indicators to assess pain (Melzack & Katz, 1994). Additional considerations include acute treatments common in critical care, such as beta blockade, inotropic stimulation, and fluid resuscitation, which have independent effects on physiologic indicators. The relevance of physiologic indicators for pain assessment needs to be further explored (Gelinas et al., 2004). Behavioral variables are also used to assess pain in noncommunicative patients. Studies have occurred primarily within the context of critical care units or in selected populations of elders with some degree of cognitive impairment (Decker & Perry, 2003; Feldt, 2000; Fuchs-Lacelle & Hadjistavropoulos, 2004; Gelinas, Fillon, Puntillo, Viens, Fortier, 2006; Gelinas & Johnston, 2007; Herr et al., 2011; Mateo & Krenzischek, 1992; Payen et al., 2001). Behavioral variables that have been used to assess pain in patients who are noncommunicative include

Measuring Acute Pain Over Time Using MOPAT

restlessness, muscle tension, facial grimacing, and vocalization. The lack of reliable, valid, and feasible tools is implicated in unrecognized pain, as well as in undertreatment and adverse sequelae (Bjoro & Herr, 2008; Herr et al., 2011; Kowal, 2010). Desired attributes of effective pain scales for assessment in noncommunicative critical care patients include feasibility (defined as tool brevity, ease of use, and short administration time), validity (the confirmed ability to measure pain), and reliability (Ersek et al., 2010; Gelinas, 2010; Gelinas et al., 2008). Guidelines from the American College of Critical Care Medicine suggest using valid behavioral pain scales such as the Critical Care Pain Observation Tool (CPOT) or the Behavioral Pain Scale (BPS) to assess pain in critically ill patients who cannot self-report (Barr et al., 2013). The psychometric testing of the CPOT English version was primarily performed using critically ill patients who could and could not self-report (Gelinas & Johnston, 2007). For the mixed sample, the intraclass correlation coefficient ranged from .80-.93 and the authors reported differences between groups, thus limiting generalization of the results. For the BPS, a weighted k of .71 and interrater agreement of .94 (allowing a 1-point difference between raters) was reported for intubated, sedated critically ill patients during a nociceptive procedure, but results at rest were not provided (Payen et al., 2001). Reliability and validity of these instruments varies depending on the version, population, setting, psychometric test, and other factors (McGuire, Kaiser, Haisfield-Wolfe, & Iyamu, 2016). Other behaviorally based tools have been tested in the critical care setting (Herr et al., 2011; McGuire et al., 2016), but they have not been tested outside of that setting and do not address the need for a tool that can be used for patients who transition across care settings. Thus, there is a need for pain assessment tools that have been tested longitudinally across settings, including critical care. To address this need, two of the coauthors and others conducted several preliminary studies aimed at refining a postoperative acute pain assessment tool first in the inpatient hospice setting (McGuire, Reifsynder, Soeken, Kaiser, & Yeager, 2011b) and then in a larger psychometric study conducted in both inpatient hospice (McGuire, Harrold, Kaiser, & Bortle, 2013) and acute care inpatient hospital settings (McGuire, Kaiser, Soeken, Reifsnyder, & Keay, 2011a) to determine the reliability, validity, and clinical utility of the tool, the Multidimensional Objective Pain Assessment Tool (MOPAT). In testing the MOPAT, the researchers focused on one-time use before and after a pain intervention by nurses working in

3

acute care hospitals (McGuire et al., 2011a) and repeated use by inpatient hospice nurses (McGuire et al., 2013). Reliability and validity of the MOPAT is acceptable and varies based on the version, setting, population, research design, and other factors. Specifics are available in the cited references. The acute care hospital component of the large psychometric study used a cross-sectional design, with pairs of nurses performing a one-time-only rating of patients’ pain before and after a pain-relieving intervention, and did not test the use of the MOPAT over time. Therefore, the specific aims of the study reported here were to compliment the larger study, specifically by doing the following: (1) examine the internal consistency and inter-rater reliability and validity of the MOPAT when used longitudinally by critical care nurses to assess acute pain in noncommunicative critically ill patients, and (2) appraise the clinical utility of the MOPAT when used longitudinally by critical care nurses to assess acute pain in noncommunicative critically ill patients. The development of the MOPAT has been previously described (McGuire et al., 2011b). It encompasses two dimensions of acute pain that are designed to assess Behavioral Dimension indicators (restlessness, muscle tension, facial expression, and vocalizations) and Physiologic Dimension indicators (blood pressure, heart rate, respiratory rate, and diaphoresis). The MOPAT has established evidence of reliability and validity when used cross-sectionally in the inpatient hospice and hospital settings (McGuire et al., 2011a, 2011b) and longitudinally in the inpatient hospice setting (McGuire et al., 2013). When tested in hospitalized patients who were both critically ill and not critically ill but could not self-report, percent agreement ranged between 69.4-86.6 and 74.4-81.3 for the MOPAT Behavioral and Physiologic subscales, respectively, in the acute care setting (McGuire et al., 2011a). Internal consistency of the Behavioral subscale (a ¼ .84) was higher for patients who could vocalize. The present study extends these data by examining validity, reliability, and clinical utility of the MOPAT when used over time by nurses in the critical care setting. It also addresses the need described earlier for a pain assessment tool that can be used across settings, not just in the critical care setting. The conceptual framework guiding this study consisted of a multidimensional conceptualization of the pain experience based on the Gate Control theory of pain (Melzack & Wall, 1965) and expanded in subsequent work (Ahles, Blanchard, & Ruckdeschel, 1983; McGuire, 1995) to six dimensions of pain. Four of these dimensions (sensory, affective, cognitive, and sociocultural) rely on patient subjective self-report

4

Wiegand et al.

for assessment, but two dimensions (behavioral and physiologic) can be ascertained through objective observation and are the foundation of the MOPAT, described next in more detail.

METHODS The study’s research design was adapted from the larger psychometric study, which used a descriptive instrument testing design with a cross-sectional data collection approach. The adapted design in the present study used a repeated measure approach in which patients had their pain rated by trained nurses surrounding painful and nonpainful events every 2 hours during the day shift for up to a 72-hour period. Nurse raters included coauthors (D.P., M.R., D.W., T.W.) and volunteer staff nurses from the participating critical care unit. The volunteer staff nurses consented to serve as participants, received training on use of the MOPAT, and rated patients’ pain using the MOPAT. The volunteer staff nurses also rated the clinical utility of the MOPAT. Setting and Sample The study was conducted in a 29-bed medical intensive care unit (MICU) of a large, urban, academic medical center. The MICU was selected because of its diverse population, high numbers of noncommunicative patients who had conditions causing acute pain, and high enrollment of both patients and nurses in the larger psychometric project. The study received a separate Institutional Review Board approval from the larger psychometric study. Two samples were recruited: patients and volunteer staff nurses. Patient inclusion criteria included MICU patients who (1) met the World Health Organization (WHO) definition for receiving palliative care, which included having at least one of the following: (a) a life-threatening illness or injury; (b) pain, or (c) an illness or injury that could end in death; (2) were receiving palliative care regardless of whether they were managed by a palliative care team; (3) were thought to have acute pain because of their conditions; (4) were unable to communicate their pain; and (5) had a sedation score of þ4 to 4 as measured by the Richmond Agitation-Sedation Scale (RASS). Patients were excluded from the study if they (1) had a history of dementia; (2) were younger than 18 years of age; (3) were receiving a paralytic infusion; (4) had severe visual problems, hearing loss, or were quadriplegic; or (5) had a RASS score of 5 (unarousable). Nurses were eligible to participate in the study if they worked in the MICU, worked some day shifts, were willing to participate in study training, and provided informed consent.

Sample size for the nonprobability patient group was calculated using preliminary data from the larger psychometric study and the planned reliability and validity analyses. Reliability was based on the Pearson correlation coefficient for an a of .05, power of 80%, and effect size of 0.30 (Bausell & Li, 2002). Validity was based on a two-tailed test of significance with paired t tests using an a of .05, power of 90%, and effect size of 1.0. The validity analysis required a minimum sample of 15 patients. The reliability analysis required 65 paired ratings, but because the study was designed to use three raters at each rating time, once per day for up to 3 days, it was anticipated that 15 patients would easily generate more than 65 paired ratings. The nonprobability sample of MICU staff nurses did not require a minimum number for analysis, thus any number of nurses who met the inclusion criteria and consented could participate in the study. The final sample sizes were N ¼ 27 patients and N ¼ 21 nurses. Data Collection Measures The RASS was used to screen patients for eligibility. Consisting of a 10-point scale used to rate the level of sedation or agitation on a continuum from 5 (unarousable) through þ4 (combative), the RASS has excellent inter-rater reliability (k ¼ .80) and validity (correlation with the Sedation Scale of r ¼ .78 and correlation of .78 with the Ramsey Scale) when used in MICU patients at specific time points (Sessler et al., 2002). When used longitudinally, the RASS had good to excellent construct validity compared with multiple measures in its ability to discriminate between various levels of sedation (Ely et al., 2003). General patient demographic and clinical information was recorded at baseline on a patient data form modified from the larger study. To further characterize the patient sample, the Palliative Performance Scale version 2 (PPSv2), a validated and widely used tool designed to describe patients’ functional ability, was used (Anderson et al., 2010; PPSv2, 2006). Scores range from 0% (death) to 100% (full performance). General demographic information from the volunteer nurse sample was recorded on a nurse data form, also modified from the larger study. The Clinical Utility Questionnaire (CUQ) was used to assess the nurses’ perceptions regarding the usefulness of the MOPAT. The CUQ was developed in previous research (McGuire et al., 2011b) to assess the clinical usefulness of the MOPAT. On this tool, volunteer study nurses responded using a 5-point Likert-type scale (ranging from strongly disagree to strongly agree) to nine items: time required for completion, ease of use, feasibility, understandability, guidance in assessing pain, assistance in

5

Measuring Acute Pain Over Time Using MOPAT

communicating about pain, helpfulness in determining presence of pain, helpfulness in determining need for pain intervention, and potential use of MOPAT by informal caregivers. The MOPAT used previously in the acute care hospital setting and in this study consists of two objective dimensions of pain (McGuire et al., 2011a). The Behavioral Dimension has four indicators of acute pain—restlessness, muscle tension, facial grimacing, and patient sounds—each rated on a scale of 0 (none) to 3 (severe). The Physiologic Dimension has four physiologic indicators of acute pain—blood pressure, heart rate, respiratory rate, and diaphoresis—each rated as 0 (no change) or 1 (change), from the patient’s usual values on these indicators. The MOPAT has evidence of validity and both internal consistency and inter-rater reliability (McGuire et al., 2011a, 2011b, 2013). Recruitment and Training of Volunteer Staff Nurses Two of the coinvestigators (D.P., T.W.) identified critical care nurses who wished to participate in the study as volunteer staff nurses. After completing the informed consent process, each volunteer staff nurse completed the nurse data form. Nurses received systematic training in the use of the MOPAT during which the investigators reviewed the MOPAT and discussed the study procedures including patient selection and eligibility criteria. Each volunteer staff nurse viewed a video of simulated patients in pain, used the MOPAT to determine each patient’s pain ratings, compared his or her pain ratings with consensus ratings predetermined by an interdisciplinary group of clinical pain experts, and discussed and resolved differences in ratings. Recruitment of Critically Ill Patients Potential study patients were identified by four of the coinvestigators (D.P., M.R., D.W., T.W.) after reviewing the eligibility criteria and talking with each patient’s assigned MICU nurse. In the beginning of the study, the investigators procured written informed consent from each patient’s legally authorized representative. However, this proved challenging given variable availability of the legally authorized representatives, so the research team obtained Institutional Review Board approval for a waiver of written consent because the study was minimal risk and the procedures were no different from normal nursing care. Data Collection Procedures After each patient was enrolled in the study, one of the coinvestigators (D.P., M.R., D.W., T.W.) completed the patient data form and the PPSv2 and recorded the patient’s RASS. The presence and severity of pain were

assessed by the coinvestigators (D.P., M.R., D.W., T.W.) or the volunteer staff nurses at 2-hour intervals between 7:00 a.m.-7:00 p.m. until 72 hours had elapsed, the patient was transferred from the MICU, or the patient died. The patient’s vital signs (blood pressure, heart rate, and respiratory rate) were recorded with each MOPAT pain rating. In addition, pain was assessed daily by three nurses, either two volunteer staff nurses and one coauthor or two coauthors and one volunteer staff nurse. The assessments were performed at two time points (referred to as T1 and T2) surrounding a painful event. The painful events included turning, suctioning, dressing changes, and insertion of an arterial catheter. The interval between the two ratings was typically within 10 minutes. For example, pain was rated during suctioning (labeled as ‘‘pain’’) and then again after the pain or discomfort had subsided (labeled as ‘‘no pain’’). Volunteer staff nurses completed the CUQ every month during the data collection period.

TABLE 1. Patient Demographic Data (N ¼ 27) Variable Gender Male Female Race Black/African American White More than one race Ethnicity Hispanic/Latino Not Hispanic/Latino Not reported Marital status Single Married Divorced Widowed Missing Major medical diagnosis Diseases of blood and blood-forming organs Diseases of circulatory system Diseases of genitourinary system Diseases of nervous system and sense organs Diseases of respiratory system Diseases of skin and subcutaneous tissue Endocrine, nutritional, and metabolic diseases, immunity disorders Infections and parasitic disease Neoplasms

Frequency (%) 13 (48.1) 14 (51.9) 17 (63.0) 9 (33.3) 1 (3.7) 1 (3.7) 25 (92.6) 1 (3.7) 11 (40.7) 10 (37.0) 2 (7.4) 2 (7.4) 2 (7.4) 2 (7.4) 7 (25.9) 1 (3.7) 5 (18.5) 2 (7.4) 1 (3.7) 2 (7.4) 4 (14.8) 3 (11.1)

6

Wiegand et al.

Data Analysis Data were transferred by the investigators from the paper forms into an ACCESS database that was modified from the larger study (McGuire et al., 2011a). Descriptive statistics were used to summarize patient and volunteer staff nurse demographic data and clinical utility data. Means and standard deviations were used for continuous variables. Frequencies and percentages were used for categorical variables. Internal consistency reliability was assessed for the Behavioral and Physiologic dimensions of the MOPAT and for the total MOPAT using Cronbach’s a coefficient. Inter-rater reliability was conducted for each dimension using percent agreement between nurse rater pairs. Generalizability (G) theory was used to assess sources of variability. Validity was assessed by analyzing the MOPAT’s sensitivity to changes in pain using a paired t test between ratings at the two time points surrounding a painful event. Volunteer staff nurses’ perceptions of clinical utility were examined using descriptive statistics. Clinical utility of the MOPAT was assessed by examining the extent of agreement with the items on the tool. Nurses’ narrative comments on the CUQ were also reviewed.

RESULTS Demographic Data The 27 patients had a total of 164 pain ratings. Patient age ranged from 27-84 years with a mean of 56.6 years (standard deviation [SD] ¼ 14.93) and median of 57 years. Patients were balanced on gender and about

TABLE 2. Frequency Distribution for RASS and PPSv2 Scores on Day of Rating (N ¼ 27 Patients) Variable RASS 4 3 2 1 0 1 Missing PPS level 10% 20% 30% Missing

Frequency (%) 18 (30.5) 6 (10.2) 12 (20.3) 6 (10.2) 7 (11.9) 2 (3.4) 8 (13.6) 41 (69.5) 6 (10.2) 4 (6.8) 8 (13.6)

RASS ¼ Richmond Agitation-Sedation Scale; PPSv2 ¼ Palliative Performance Scale version 2. Most were rated on more than one day; thus, there are 59 days of ratings.

two-thirds of the patients were black/African American (Table 1). Patients had a wide variety of medical diagnoses, with diseases of the circulatory system (25.9%), nervous system (18.5%), and infections and parasitic diseases (14.8%) being most common. Roughly twothirds of the patients had a RASS of 2 to 4 and three-quarters of the patients had a PPSv2 of 10% on the days that pain ratings were obtained (Table 2). The 21 nurses (Table 3) reported that they frequently or constantly cared for patients who were unable to self-report pain. The mean age of the nurses was 33.4 (SD ¼ 7.84) and the majority were female, had a bachelor’s degree, held specialty certifications, and were working in a staff nurse position. Psychometric Data Internal consistency reliability coefficient a of the total MOPAT surrounding painful events were .68 at T1 and .72 at T2 (Table 4). Coefficient a for the Behavioral Dimension were .80 at T1 and .81 at T2 and for the Physiologic Dimension were .37 at T1 and .57 at T2. Inter-rater agreement during painful procedures (‘‘pain’’) was 68.2% for the Behavioral Dimension and 79.5% for the Physiologic Dimension (Table 5). Lastly, because internal consistency reliability and measures of rater agreement consider only one source of measurement error at a time, generalizability theory was used to estimate the variance contributed by each source (seven items on the MOPAT and three raters) as well as interaction between the sources. The estimated proportion of variance for the interaction of patients and items was 26%, indicating that the rank ordering of patients differed across the items. In addition, 18% of the variance was attributed to items and 15% to patients. The G coefficients or reliabilities were .69 across the three raters and .61 across the 7 items. As noted earlier, validity was assessed using sensitivity analysis, based on the assumption that if the MOPAT was valid, total and individual dimension scores would improve if the reason for the pain was resolved. There were significant (p < .001) decreases in the total MOPAT, as well as the Behavioral and Physiologic Dimension scores among all raters when comparing T1 and T2 (Table 6). The Behavioral Dimension was more sensitive than the Physiologic when examined by rater subgroups (volunteer staff nurses or study nurses). Clinical Utility The 21 volunteer staff nurses completed the CUQ 146 times (coefficient a ¼ .85). Most responses indicated agreement or strong agreement with all nine items (data not shown). Similarly, among those who used

7

Measuring Acute Pain Over Time Using MOPAT

TABLE 3. Volunteer Staff Nurse Demographic Data (N ¼ 21) Variable Age (mean ¼ 33.4, SD ¼ 7.84) 21-29 years 30-39 years 40-49 years Missing Gender Male Female Race White Asian Black/African American More than one race Ethnicity Not Hispanic/Latino Hispanic/Latino Not reported Highest level of education Bachelor’s degree Master’s degree Doctoral degree Specialty certifications Yes No Current position Staff nurse Charge nurse NP or CNS How often do you encounter patients who are unable to self-report pain? Frequently Constantly Hours of pain management in-service training/continuing education in the last year (mean ¼ 2.1, SD ¼ 2.15) 0 1 2 4 6 8 Number of journal articles on pain assessment read during the last year (mean ¼ 2.1, SD ¼ 2.49) 0 1 2 3 4 6 10 Experience as an RN (mean ¼ 7.6, SD ¼ 7.04) <2 years 2-5 years 6-10 years 11-15 years

TABLE 3. Continued Variable

Frequency (%)

16-20 years 21-25 years Years in current position (mean ¼ 3.3, SD ¼ 2.25) <2 years 2-5 years 6-10 years 11-15 years 16-20 years

3 (14.3) 1 (4.8)

Frequency (%) 6 (28.6) 10 (47.6) 4 (19.0) 1 (4.8) 4 (19.0) 17 (81.0) 11 (52.4) 6 (28.6) 3 (14.3) 1 (4.8)

7 (33.3) 11 (52.4) 2 (9.5) 0 (0.0) 1 (4.8)

SD ¼ standard deviation; NP ¼ nurse practitioner; CNS ¼ certified nurse specialist; RN ¼ registered nurse.

17 (81.0) 3 (14.3) 1 (4.8)

the MOPAT within the previous 30 days (n ¼ 87 ratings), most respondents reported agreement or strong agreement with all items (Table 7). Nurses’ narrative comments on the MOPAT included ‘‘very useful tool,’’ ‘‘easy to use,’’ and ‘‘very beneficial to pain management of non-verbal patients.’’

12 (57.1) 9 (42.9)

DISCUSSION

18 (85.7) 1 (4.8) 2 (9.5)

The MOPAT was used successfully as a repeated measure in this study to assess pain in this critically ill population. Information regarding the psychometric properties is further discussed below.

18 (85.7) 1 (4.8) 2 (9.5)

19 (90.5) 2 (9.5)

5 (23.8) 5 (23.8) 5 (23.8) 4 (19.0) 1 (4.8) 1 (4.8)

7 (33.3) 3 (14.3) 5 (23.8) 1 (4.8) 3 (14.3) 1 (4.8) 1 (4.8) 6 (28.6) 5 (23.8) 4 (19.0) 2 (9.5) (Continued )

Reliability The internal consistency reliability for the total MOPAT was moderate in this sample and considered acceptable for early testing of an instrument. The coefficient a was stronger for the Behavioral Dimension than for the Physiologic Dimension, similar to findings in the hospitalized inpatients from the larger psychometric study (McGuire et al., 2011a). Given that clinicians and researchers alike have questioned the use of physiologic indicators as measures of acute pain, and

TABLE 4. Internal Consistency Reliability as Measured With Cronbach’s a Patients

N

Ratings

Time 1

Time 2

Total MOPAT Behavioral dimension Physiologic dimension

27 patients

164

.68 .80

.72 .81

.37

.57

MOPAT ¼ Multidimensional Objective Pain Assessment Tool.

8

Wiegand et al.

range of behaviors was limited. In contrast, the poorer level of agreement when patients were in pain may be due to the fact that the MOPAT training used both in this study and in hospitalized inpatients from the larger psychometric study could have been more comprehensive (McGuire et al., 2011a). Specifically, although the training included simulated patients for whom the MOPAT was used to rate pain (with consensus ratings provided by pain experts), not every level of every behavior was demonstrated; thus nurses had limited exposure to the levels of each behavior before using the MOPAT in the clinical setting.

TABLE 5. Inter-rater Reliability Among Volunteer Staff Nurses and Study Nurses as Measured by Percent Agreement

Restless Tense muscles Frowning/grimacing Patient verbal sounds Mean for Behavioral Dimension Blood pressure Heart rate Respiratory rate Diaphoresis Mean for Physiologic Dimension

Pain (n ¼ 104 Pairs)

No Pain (n ¼ 107 Pairs)

65.4% 55.8% 52.9% 99.0% 68.2% 72.4% 81.8% 76.8% 86.9% 79.5%

83.2% 73.8% 75.7% 98.1% 82.7% 67.3% 79.4% 78.4% 91.1% 79.0%

Validity The validity results strongly suggest that the MOPAT is a sensitive measure of acute pain in noncommunicative patients and are commensurate with findings in the initial developmental work (McGuire et al., 2011b) and the larger psychometric study (McGuire et al., 2011a, 2013). Changes in pain noted by all raters combined were significant (p < .001) for both dimensions and for the total MOPAT score (Table 6).

the potential for physiologic indicators to be influenced by other factors (e.g., hyperthermia, hypoxemia, vasopressors, etc.) (Herr et al., 2011), this finding is not surprising. The G theory results indicated that the interaction of patients and items only explained 59% of the variability, leaving 41% the result of other unknown factors. The G coefficients of .69 for raters and .61 for items are considered low to borderline, which might be expected when testing a new instrument. They are considered comparable to the Cronbach a coefficient of .72 for the total MOPAT score, because the G coefficient assessed three raters instead of paired raters. The internal consistency and G theory reliability results may reflect the relatively small sample size. Finally, inter-rater reliability agreement was reasonable for both dimensions and the Total MOPAT, but less than desired. Agreement on the Behavioral Dimension was higher when patients were not in pain, but this might be expected given that the

Clinical Utility Feedback from nurses who used the MOPAT was very positive, similar to the preliminary work and the larger study (McGuire et al., 2011a, 2011b, 2013). The critical care nurses in this study found the MOPAT helpful in assessing pain for critically ill noncommunicative patients. The instrument was quick and easy to use and no negative comments were reported. The nurses thought it would be helpful to incorporate the instrument into future clinical practice. Limitations The study was designed to collect data for 72 hours for each patient, but not all patients reached this

TABLE 6. Validity All Raters

VSN Raters

SN Raters

Dimension

Pain

No Pain

p*

Pain

No Pain

p*

Pain

No Pain

p*

Behavioral, M (SD) Physiologic, M (SD) Total score, M (SD)

2.42 (1.97) 1.30 (1.06) 3.72 (2.49)

0.80 (1.23) 0.93 (1.07) 1.74 (1.86)

<.001 <.001 <.001

2.67 (2.13) 1.27 (1.11) 3.87 (2.74)

0.94 (1.36) 0.85 (1.03) 1.76 (2.02)

<.001 .006 <.001

2.17 (1.76) 1.34 (1.01) 3.57 (2.21)

0.66 (1.08) 1.01 (1.11) 1.71 (1.69)

<.001 .02 <.001

VSN ¼ volunteer staff nurse; SN ¼ study nurse; M ¼ mean; SD ¼ standard deviation. *Paired t-test.

9

Measuring Acute Pain Over Time Using MOPAT

TABLE 7. Clinical Utility Questionnaire Among Nurses Using the MOPAT Within the Previous 30 Days (N ¼ 87 Ratings) Questions

Strongly Disagree

Disagree

Undecided

Agree

Strongly Agree

MOPAT took a reasonable amount of time to complete. MOPAT was easy to use. MOPAT was easy for me to understand. MOPAT guided me in what to look for when assessing pain. MOPAT would be feasible for regular use in my clinical setting. MOPAT assisted me in communicating to others about a patient’s pain. MOPAT was helpful in determining the presence of pain in a noncommunicative patient. MOPAT was helpful in determining whether a patient might need a pain intervention. MOPAT could be used by informal caregivers (family, friends) with some training

1 (1.1)

5 (5.7)

1 (1.1)

54 (62.1)

26 (29.9)

3 (3.4) 12 (13.8)

51 (58.6) 48 (55.2)

33 (37.9) 27 (31.0)

47 (54.0)

40 (46.0)

2 (2.3)

52 (59.8)

33 (37.9)

17 (19.5)

48 (55.2)

22 (25.3)

6 (6.9)

51 (58.6)

30 (34.5)

9 (10.3)

50 (57.5)

28 (32.2)

24 (27.6)

43 (49.4)

19 (21.8)

1 (1.1)

MOPAT ¼ Multidimensional Objective Pain Assessment Tool.

time point because some of them became communicative, were transferred from the MICU, or died. Despite these issues, most patients had data for up to 48 hours, enabling an appraisal of the MOPAT with repeated use. As described earlier, pain was assessed daily by a triad of nurses at two time points surrounding a painful event. For some of these ratings, pain was assessed during a painful event, and then it was assessed after the painful stimulus ended. For other ratings, the patient was assessed when resting comfortably, and then a few minutes later during a painful event such as a wound dressing change. A more consistent approach to the triad assessment may have revealed different findings. Finally, the sample size was small and may have influenced some of the psychometric results. Although the results suggest that the MOPAT may be reliable and valid when used repeatedly by critical care nurses in this small sample in a MICU setting, the study’s limitations may hamper wider generalization. Additional validity and reliability testing in a larger MICU sample as well as a broader array of critical care settings across time would be helpful. Future research could include using the CPOT or BPS to test for convergent validity of the

Behavioral Dimension of the MOPAT. The Physiologic Dimension may also need revision and subsequent psychometric evaluation. Nurses found the MOPAT to be clinically useful, suggesting that it could be incorporated into routine pain practice. When considered in conjunction with the crosssectional and longitudinal psychometric data from the larger study (McGuire et al., 2011a, 2013), these results add to reliability and validity evidence for the MOPAT’s use in critically ill patients who cannot communicate.

CONCLUSIONS The MOPAT is an instrument that has been tested in a variety of settings, unlike other behavioral instruments designed for patients who cannot communicate. This study expanded existing data by finding that the MOPAT appears moderately reliable, valid, and clinically useful when used by MICU nurses longitudinally to assess acute pain in the critically ill. The Behavioral Dimension of the MOPAT appears to be the most helpful component of the tool, whereas the Physiologic Dimension needs further study. One key advantage of the MOPAT is that it can be used

10

Wiegand et al.

in noncommunicative patients as they transition from other settings into the intensive care unit and from the intensive care unit out to other settings, thereby providing a consistent means of assessing pain and potentially enhancing communication among health care providers.

Acknowledgments The authors thank the MICU volunteer staff nurses for their assistance with this study and the leadership support of Kerry Sue Mueller, MSN, RN, and Carl Shanholtz, MD. They also thank Richard Shrout, MSN, RN and Mary Ellen Haisfield-Wolfe, PhD, RN, OCNÔ for assistance with implementation of the study.

REFERENCES Ahles, T. A., Blanchard, E. B., & Ruckdeschel, J. C. (1983). The multidimensional nature of cancer related pain. Pain, 17(3), 277–288. Anderson, F., Downing, G. M., Hill, J., Casorso, L., & Lerch, N. (2010). Palliative performance scale (PPS): A new tool. Journal of Palliative Care, 12(1), 5–11. Barr, J., Fraser, G. L., Puntillo, K., Ely, E. W., Gelinas, C., Dasta, J. F., Davidson, J. E., Devlin, J. W., Kress, J. P., Joffe, A. M., Coursin, D. B., Herr, D. L., Tung, A., Robinson, B. R., Fontaine, D. K., Ramsay, M. A., Riker, R. R., Sessler, C. N., Pun, B., Skrobik, Y., & Jaeschke, R. (2013). Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Critical Care Medicine, 41(1), 263–306. Bausell, R. B., & Li, Y. (2002). Power analysis for experimental research. Cambridge, UK: Cambridge University Press. Bjoro, K., & Herr, K. (2008). Assessment of pain in the nonverbal or cognitively impaired older adult. Clinics in Geriatric Medicine, 24(2), 237–262. Blakely, W., & Page, G. G. (2001). Pathophysiology of pain in critically ill patients. Critical Care Nursing Clinics of North America, 13(2), 167–179. Decker, S. A., & Perry, A. G. (2003). The development and testing of the PATCOA to assess pain in confused older adults. Pain Management Nursing, 4(2), 77–86. Desbiens, N. A., Wu, A. W., Alzola, C., Mueller-Rizner, N., Wenger, N. S., Connors A. F. Jr., Lynn, J., & Phillips, R. S. (1997). Pain during hospitalization is associated with continued pain six months later in survivors of serious illness. The American Journal of Medicine, 102(3), 269– 276. Ely, E. W., Truman, B., Shintani, A., Thomason, J. W., Wheeler, A. P., Gordon, S., Francis, J., Speroff, T., Gautam, S., Margolin, R., Sessler, C. N., Dittus, R. S., & Bernard, G. R. (2003). Monitoring sedation status over time in ICU patients: Reliability and validity of the Richmond Agitation-Sedation Scale (RASS). The Journal of the American Medical Association, 289(22), 2983–2991. Ersek, M., Herr, K., Neredilek, M. B., Buck, H. G., & Black, B. (2010). Comparing the psychometric properties of the Checklist of Nonverbal Pain Behaviors (CNPI) and the Pain Assessment in Advanced Dementia (PAIN-AD) instruments. Pain Medicine, 11(3), 395–404. Feldt, K. S. (2000). The Checklist of Nonverbal Pain Indicators (CNPI). Pain Management Nursing, 11(3), 13–21. Ferrell, B. A., & Schaffner, M. (1997). Pharmacoeconomics and medical outcomes in pain management. Seminars in Anesthesia, Perioperative Medicine and Pain, 16(2), 152–159. Fuchs-Lacelle, S., & Hadjistavropoulos, T. (2004). Development and preliminary validation of the pain assessment

Checklist for Seniors with limited ability to communicate (PACSLAC). Pain Management Nursing, 5(1), 37–49. Garcia, L. F., Peres, B. D., De Cubber, M., & Vincent, J. (2003). Long-term outcome in ICU patients: What about quality of life? Intensive Care Medicine, 29(8), 1286–1293. Gelinas, C. (2010). Nurses’ evaluations of the feasibility and the clinical utility of the critical-care pain observation tool. Pain Management Nursing, 11(2), 115–125. Gelinas, C., Fillon, L., Puntillo, K. A., Viens, C., & Fortier, M. (2006). Validation of the Critical-Care Pain Observation Tool in adult patients. American Journal of Critical Care, 15(4), 420–427. Gelinas, C., Fortier, M., Viens, C., Fillion, L., & Puntillo, K. (2004). Pain assessment and management in critically ill intubated patients: A retrospective study. American Journal of Critical Care, 13(2), 126–135. Gelinas, C., & Johnston, C. (2007). Pain assessment in the critically ill ventilated adults: Validation of the Critical-Care Pain Observation Tool and physiologic indicators. Clinical Journal of Pain, 23(6), 497–505. Gelinas, C., Loiselle, C. G., LeMay, S., Ranger, M., Bouchard, E., & McCormack, D. (2008). Theoretical, psychometric, and pragmatic issues in pain measurement. Pain Management Nursing, 9(3), 120–130. Hall-Lord, M. L., Larsson, G., & Steen, B. (1998). Pain and distress among elderly intensive care unit patients: Comparison of patients’ experiences and nurses’ assessments. Heart & Lung, 27(2), 123–132. Hamill-Ruth, R. J., & Marohn, M. L. (1999). Evaluation of pain in the critically ill patient. Critical Care Clinician, 15(1), 35–54. Herr, K., Coyne, P. J., McCaffrey, M., Manworren, R., & Merkel, S. (2011). Pain assessment in the patient unable to self-report: Position statement with clinical practice recommendations. Pain Management Nursing, 12(4), 230–250. International Association for the Study of Pain (IASP). (2014). IASP taxonomy. Retrieved from. http://www. iasp-pain.org/Taxonomy?navItemNumber¼576. Accessed August 14, 2016. Kabes, A. M., Graves, J. K., & Norris, J. (2009). Further validation of the Nonverbal Pain Scale in intensive care patients. Critical Care Nurse, 29(1), 59–66. Klein, D. G., Dumpe, M., Katz, E., & Bena, J. (2010). Pain assessment in the intensive care unit: Development and psychometric testing of the Nonverbal Pain Assessment Tool. Heart & Lung, 39(6), 521–527. Kowal, C. D. (2010). Implementing the Critical-Care Pain Observation Tool using the Iowa model. Journal of the New York State Nurses Association, 41(1), 4–10. Lamer, C., Harboun, M., Knani, L., Moreau, D., Tric, L., LeGuillou, J. L., Gasquet, I., & Moreau, T. (2004). Quality of life after complicated elective surgery

Measuring Acute Pain Over Time Using MOPAT

requiring intensive care. Intensive Care Medicine, 30(8), 1594–1601. Mateo, O. M., & Krenzischek, D. A. (1992). A pilot study to assess the relationship between behavioral manifestations and self-report of pain in postanesthesia care unit patients. Journal of Post Anesthesia Nursing, 7(1), 12–21. McGuire, D. B. (1995). The multiple dimensions of cancer pain: A framework forassessment and management. In D. B. McGuire, C. H. Yarbro, & B. R. Ferrel (Eds.), Cancer pain management, (2nd ed.) (pp. 1–17) Sudbury, Mass: Jones and Bartlett Publishers. McGuire, D. B., Harrold, J., Kaiser, K. S., & Bortle, D. (2013). Measuring pain in non-communicative patients in the inpatient hospice setting: Psychometric evaluation of theMultidimensional Objective Pain Assessment Tool (MOPAT). Journal of Pain and Symptom Management, 45, 403–404. McGuire, D. B., Kaiser, K. S., Haisfield-Wolfe, M. E., & Iyamu, F. (2016). Pain assessment in non-communicative adult palliative care patients. Nursing Clinics of North America, 51, 397–431. McGuire, D. B., Kaiser, K. S., Soeken, K., Reifsnyder, J., & Keay, T. (2011). Measuring pain in non-communicative patients in the acute care setting: Psychometric evaluation of the Multidimensional Objective Pain Assessment Tool (MOPAT). Journal of Pain and Symptom Management, 41, 299–300. McGuire, D. B., Reifsnyder, J., Soeken, K., Kaiser, K. S., & Yeager, K. A. (2011). Assessing pain in non-responsive hospice patients: Development and preliminary testing of the Multidimensional Objective Pain Assessment Tool (MOPAT). Journal of Palliative Medicine, 14(3), 287–292. McNicoll, L., Pasani, M. A., Zhang, Y., Ely, E. W., Siegel, M. D., & Inouye, S. K. (2003). Delirium in the intensive care unit: Occurrence and clinical course in older patients. Journal of American Geriatrics, 51(5), 591–598. Melzack, R., & Katz, J. (1994). Pain measurement in persons with pain. In P. D. Wall, & R. Melzack (Eds.), Textbook of pain, (3rd ed.) (pp. 337–351) Edinburgh: Churchill Livingstone. Melzack, R., & Wall, P. D. (1965). Pain mechanisms: A new theory. Science, 150, 971–979. Palliative Performance Scale (PPSv2) version 2. (2006). Medical care of the dying, (4th ed.) Victoria Hospice Society. Retrieved from. http://www.victoriahospice.org/sites/ default/files/pps_english.pdf. Accessed May 2, 2016. Pasero, C., & McCaffrey, M. (2005). No self-report means no pain-intensity rating: Assessing pain in patients who cannot provide a report. American Journal of Nursing, 105(10), 50–53. Payen, J. F., Bosson, J. L., Chanques, G., Mantz, J., & Labere, J. (2009). Pain assessment is associated with decreased duration of mechanical ventilation in the intensive care unit. Anesthesiology, 111(6), 1308–1316.

11

Payen, J. F., Bru, O., Bosson, J. L., Lagrasta, A., Novel, E., Deschaux, I., Lavagne, P., & Jacquot, C. (2001). Assessing pain in critically ill sedated patients by using a Behavioral Pain Scale. Critical Care Medicine, 29(12), 2258–2263. Prendergast, T. J. (2002). Palliative care in the intensive care unit setting. 1086–1086. In A. M. Berger, R. K. Portenoy, & D. E. Weissman (Eds.), Principles and practice of palliative care and supportive pncology. Philadelphia: Lippincott Williams & Wilkins. Puntillo, K. A. (1990). Pain experiences in intensive care unit patients. Heart & Lung, 19, 526–533. Puntillo, K. A., Miaskowski, C., Kehrie, K., Stannard, D., Gleeson, S., & Nye, P. (1997). Relationship between behavioral and physiological indicators of pain, critical care patients’ self-reports of pain, and opioid administration. Critical Care Medicine, 25(7), 1159–1166. Puntillo, K. A., Morris, A. B., Thompson, C. L., StanikHutt, J., White, C. A., & Wild, L. R. (2004). Pain behaviors observed during six common procedures: Results from Thunder Project II. Critical Care Medicine, 32(2), 421–427. Puntillo, K. A., White, C., Morris, A. B., Perdue, S. T., Stanik-Hutt, J., Thompson, C. L., & Wild, L. R. (2001). Pateints’ perceptions and responses to procedural pain: results from Thunder Project II. American Journal of Critical Care, 19(4), 238–251. Rotondi, A. J., Chelluri, L., Sirio, C., Mendelsohn, A., Schulz, R., Bele, S., Im, K., Donahoe, M., & Pinsky, M. R. (2002). Patients’ recollection of stressful experiences while receiving prolonged mechanical ventilation in an intensive care unit. Critical Care Medicine, 30(4), 746–752. Schelling, G., Stoll, C., Haller, M., Briegel, J., Manert, W., & Lenhart, A. (1998). Health-related quality of life and posttraumatic stress disorder in survivors of the acute respiratory syndrome. Critical Care Medicine, 26(4), 651–659. Schuler, M., Njoo, N., Hestermann, M., Oster, P., & Hauer, K. (2004). Acute and chronic pain in geriatrics: Clinical characteristics of pain and the influence of cognition. Pain Medicine, 5(3), 253–262. Sessler, C. N., Gosnell, M. S., Grap, M. J., Brophy, G. M., O’Neal, P. V., Keane, K. A., Tesoro, E. P., & Elswick, R. K. (2002). The Richmond Agitation-Sedation Scale: Validity and reliability in adult intensive care patients. American Journal of Respiratory Critical Care Medicine, 166(10), 1338–1344. Stannard, D., Puntillo, K., Miaskowski, C., Gleeson, S., Kehrie, K., & Nye, P. (1996). Clinical judgment and management of postoperative pain in critical care patients. American Journal of Critical Care, 5(6), 433–441. Topolovec-Vranic, J., Canzian, S., Innis, J., PollmannMudryj, M. A., McFarlan, A. W., & Baker, A. J. (2010). Patient satisfaction and documentation of pain assessments and management after implementing the Adult Nonverbal Pain Scale. American Journal of Critical Care, 19(4), 345–354.