Evaluation of outcomes in Psychiatric Consultation-Liaison Nursing practice

Evaluation of Outcomes in Psychiatric Consultation-Liaison Nursing Practice Richard Yakimo, Lenore H. Kurlowicz, and Ruth Beckmann Murray This report describes and evaluates the current status of outcome analysis in Psychiatric Consultation-Liaison Nursing (PCLN) and offers suggestions for future development. The status of outcome evaluation generally in psychiatric nursing is described with attention given to the scope of practice of PCLN and outcomes used by psychiatric consultation-liaison nurses (PCLNs) to evaluate their interventions. An evaluation framework based on Donabedian’s paradigm of structure, process, and outcome is presented and its applicability shown to PCLN. This framework is further explicated with regard to a review of published studies of PCLN within the outcome domains of cost reduction, satisfaction with services, changes in clinical status, and perceptions of work environment. Finally, recommendations for further development of outcomes in PCLN are offered, based on the strengths and limitations of the extant literature. © 2004 Elsevier Inc. All rights reserved.

P

SYCHIATRIC CONSULTATION-Liaison Nursing (PCLN) is an advanced practice nursing subspecialty issuing from the specialty of psychiatric nursing. Nurses who practice this role typically have both psychiatric/mental health and medical-surgical experience, and perform nursing assessment, diagnosis, intervention, and evaluation for psychosocial distress manifested by patients, their families, or staff at medical-surgical inpatient, outpatient, and long-term care facilities (Happell & Sharrock, 2001; Minarik & Neese, 2002; Roberts, 1997; Robinson, 1982; Tunmore & Thomas, 1992). Psychiatric disorders occur in the medicalsurgical patient population at significant rates of comorbidity, with 30% to 50% of patients with primary medical diagnosis(es) manifesting psychiatric symptoms (Arolt & Driessen, 1996; Clarke, Smith, & Herrman, 1993; Mayou & Hawton, 1986; Saravay & Lavin, 1994; Savoca, 1999). Staff in medical-surgical settings feel unprepared to deal with the distress and demoralization that frequently accompany physical illness in their patients (Angelino & Treisman, 2001), to manage patient behavior problems that interfere with giving care, to handle conflicts with other staff members, and to cope with the demands of an ever-

changing structure within health care settings (Norwood, 1998b). Psychiatric consultation-liaison nurses (PCLNs) also consult with managers at all levels of the health care system regarding organizational issues and serve as opinion leaders in advancing administrative initiatives. PCLN has evolved over the past 40 years in an attempt to offer immediate, short-term, crisis-oriented mental health intervention and education to individuals in medical-surgical settings, to bridge the gap often found between psychiatric and medical-surgical nursing care, and to facilitate clients’ transition to additional health services of both a physical and psychosocial nature (Kurlowicz, 1998; Norwood, 1998b; Roberts, 1997; Tunmore & Thomas, 1992).

From the Washington University School of Medicine, Department of Psychiatry, St. Louis, MO; the University of Pennsylvania School of Nursing, Philadelphia, PA; and Saint Louis University School of Nursing, St. Louis, MO. Address reprint requests to Richard Yakimo, PhD, APRN, BC, Washington University School of Medicine, 40 North Kingshighway, Suite 3, St. Louis, MO 63108. E-mail: [email protected] 䊚 2004 Elsevier Inc. All rights reserved. 0883-9417/04/1806-0003$30.00/0 doi:10.1016/j.apnu.2004.09.004

Archives of Psychiatric Nursing, Vol. XVIII, No. 6 (December), 2004: pp 215-227

215

216

YAKIMO, KURLOWICZ AND MURRAY

PURPOSE

The general purpose of this report is to describe and evaluate the current status of outcome analysis in PCLN and to formulate suggestions for future development. This aim will be accomplished by: (a) describing the scope of practice and outcomes used by PCLNs to evaluate their assessment and intervention procedures; (b) depicting the status of outcome evaluation in psychiatric nursing; (c) presenting an evaluation framework based on Donabedian’s paradigm of structure, process, and outcome and demonstrating its application to PCLN; (d) applying the framework to a review of published studies of PCLN interventions within the outcome domains of cost reduction, satisfaction with services, changes in patient clinical status, and perceived work environment. Finally, recommendations for further development of outcomes in PCLN will be offered, based on the review and evaluation of the extant literature. Outcome Evaluation in Psychiatric Consultation-Liaison Nursing (PCLN) The nursing profession has championed the description and classification of nursing-defined problems, interventions, and outcomes in an effort to document the work activities of nurses and effects on patient status. However, existing taxonomies are conceptually broad (Johnson, Maas, & Moorhead, 2000; McCloskey & Bulechek, 1996) and thus do not address the specific outcome criteria relevant to nursing subspecialties. In particular, the description of categories of psychosocial outcomes is at a formative stage (Johnson, Bulechek, Dochterman, Maas, & Moorhead, 2001), and psychiatric nurses do not appear knowledgeable about systematic outcome procedures. In addition, outcome evaluation in psychiatric nursing practice has typically been formulated according to vague concepts such as “clinical judgment,” or has been performed over extended periods of time ranging from months to years (Barrell, Merwin, & Poster, 1997). Scant attention has been paid to the results of the specialized interventions and brief evaluation period typical of PCLN (Kurlowicz, 1998; Shahinpour, Hollinger-Smith, & Perlia, 1995). The PCLN literature has called for a focus on outcome dimensions relating to: (a) the patient and family, such as decreased psychiatric symptom distress, treatment adherence, positive caregiver interaction, and

safety maintenance; (b) the provider, such as staff satisfaction, increased mental health knowledge, and improved interactions with patients; and (c) the institution, such as decreased length of stay, prevention of medical complications associated with psychiatric comorbidity, decline in readmissions, and increased referral to other health services (Kurlowicz, 1998). However, no published outcome measures utilizing these multiple dimensions are available. Single studies have shown PCLN intervention effective with selected problems, such as reducing demanding patients’ requests of staff nurses’ time (Mallory, Lyons, & Scherubel, & Reichelt, 1993), cutting costs by decreasing length of stay associated with untreated mental health issues (Ragaisis, 1996), increasing patient satisfaction with health care services (Newton & Wilson, 1990), and identifying and treating depression and delirium within hospitalized elderly (Kurlowicz, 2001). Although these studies focused on effective problem resolution among special subgroups of patients and decrease in perceived patient deviance (Carveth, 1995), they did not address outcomes that occur in multiple dimensions of functioning in the course of everyday PCLN practice. The documentation of PCLN interventions and associated outcomes are critical to the implementation of evidence-based nursing practice (Rosswurm & Larrabee, 1999) and to counter the decreasing reimbursement of psychiatric services and elimination of clinical specialist nursing positions because of lack of positive outcome evidence (Norwood, 1998a; 1998b). Essential components of PCLN practice need to be evaluated to discern the structure, process, and outcome factors indicative of quality care for various patient populations, the family members, their providers, and health care institutions (Donabedian, 1988, Kurlowicz, 1998). QUALITY OF CARE THEORY

Donabedian’s structure-process-outcome model of health care delivery (1966; 1980; 1982; 1988) has directed quality evaluations for nearly 4 decades. Quality of care theory states that there are three sources of evidence from which inferences can be drawn regarding quality: structure, process, and outcome. Structure refers to the physical, human, and financial resources available in the settings in which

EVALUATION OF OUTCOMES IN PSYCHIATRIC

care occurs (Donabedian, 1966; 1980). This includes: (a) material resources (facilities, equipment, level of technology, and money); (b) human resources (the number, professional affiliations, and qualifications of personnel), and (c) organizational structure (type of administrative hierarchy, methods of reimbursement, models of peer review). Structure also entails the demographic characteristics of the population served, including age, social class, and types of prevalent health problems (Donabedian, 1966; 1980). Structure thus includes the relatively enduring characteristics of a health care institution. Process describes what is actually done in the giving and receiving of care. It includes the health care provider’s activities in diagnosing and treating problems, as well as the patient’s efforts in accessing care, adhering to the treatment plan, and implementing self-care measures (Donabedian, 1966; 1988). Outcome refers to any consequence of care. More specifically, outcome refers to states or conditions of individuals and populations that are attributable to prior health care; an outcome is the expected change sought by applying health-related interventions (Donabedian, 1982; 1993). Outcome is a multifaceted concept that includes clinical, physiological, physical, psychological, psychosocial, integrative, and evaluative dimensions. Interrelationship of Structure, Process, and Outcome Donabedian (1992) cautioned that the structure-process-outcome sequence is a simplistic paradigm and that the reality of health care issues reveals complex interrelated chains of events in which one concept links to another. The boundaries placed between structure, process, and outcome may also be indistinct and arbitrary. Of greater importance than mere labeling is delineating and understanding the sources of evidence in the sequence and the causal probabilities among them. The strength of the causal probabilities determines the usefulness of any variable chosen as an indicator of quality. Donabedian (1988; 1992) further warned that outcomes may be misleading as indicators of quality because they are not direct assessments of quality; outcome statements only offer an inference about the quality of process and structure. The

217

strength of the inference depends on the strength of the causal relationships between structure, process, and outcome. In turn, the causal relationships can be influenced by factors outside of health care. This necessitates the confirmation of the causal sequence by process verification in which a large number of cases is examined to determine whether care is implemented according to standards, and whether variation in care results in associated changes in outcomes. Outcomes may reflect skill in the selection of type of care as well as its execution. However, outcomes are also integrative; they reflect all the inputs into care, including the contributions of patients to their own care. The integrative nature of outcomes unfortunately leads to concealment of what precisely what went wrong, which can be discovered only by detailed analysis of process and structure. However, the value of outcomes lies in prompting such analyses. Outcomes are further constrained by the “time window”; the more an outcome is delayed, the more difficult it is to obtain information about it and more opportunities occur for factors other than care to have influenced the outcome. Poor outcomes may also reflect damage that cannot be prevented or ameliorated because it occurred long before the care processes were instituted (Donabedian 1966; 1988; 1992). Quality of care theory emphasizes the multiple causation of outcomes as well as the concurrent strengths and weaknesses of any one measure of structure, process, or outcome. Thus, the best strategy for evaluating quality of care is to include a mix of indicators drawn from the concepts of structure, process, and outcome. This allows a multidimensional evaluation of quality because certain indicators may be more relevant to particular aspects of quality. Multiple dimensions also identify areas and causes of poor quality and suggest appropriate changes in structure and process. Concomitant agreement among several types of indicators increases confidence in the judgments about quality. However, disagreement among several types of indicators may suggest problems in measurement, an inappropriate time window between care given and outcome determination, or the need to formulate a new model of structure, process, and outcome (Donabedian, 1966; 1988; 1992).

218

YAKIMO, KURLOWICZ AND MURRAY

Application of the Quality of Care Framework to PCLN The concepts of structure, process, and outcome as evidence of quality of care can be applied to the practice of PCLN. Structural aspects may include the type and size of the clinical facility, number of PCLN positions, as well as education, certification, and level of experience of the nurse. Characteristics of the patient population, such as age, social class, cultural identity and values, and typical health problems treated at the facility, also are included within structure. Process factors may include patterns of PCLN assessment, intervention, evaluation, and follow-up. Outcome factors involve administrative concerns, such as costs and length of stay; application of education by patients, families, and staff; integration of care; improved collaboration and communication among patients, families, staff, and administrators; positive attitudes and satisfaction toward care; level of discharge disposition; satisfaction with care; improved coping skills; and maintenance of safety (Kurlowicz, 1998, 2001; Yakimo & Murray, 2002). The difficulty in using outcomes as measures of quality of care is also evident in PCLN practice. Various factors contribute to difficulty in evaluating the role, such as the multifaceted nature of the PCLN role, the lack of agreement on evaluation or performance criteria, the overlap of care processes with other professionals, and the lack of standardization of the “time window.” PCLN roles frequently shift focus to meet the changing needs of health care facilities. The effects of PCLN processes may be indirect and thus not readily visible (Kurlowicz, 1998; Lewis & Levy, 1982; Sharrock & Happell, 2000; 2001). Only a careful delineation of associated structure, processes and outcomes in PCLN practice will make the role more visible and show its impact on the quality of care (Kurlowicz, 1998, Norwood, 1998a). EVALUATION OF PCLN INTERVENTIONS

The PCLN literature has called for researchbased evidence of the effectiveness of PCLN practice to advance knowledge about the specialty, to provide for education of PCLN professionals, to improve the quality of care for various patient populations, and to show the effect of the role to administrators and payers (Kurlowicz, 1998; Lewis

& Levy, 1982, Nelson & Schilke, 1976; Norwood, 1998b, Robinson, 1982; Shahinpour, HollingerSmith, & Perlia, 1995). However, there are limited published studies that directly link PCLN interventions to measurable outcomes. The available studies will be described in this section according to the type of outcome examined. Cost Outcomes Medical-surgical patients with psychiatric comorbidities use a disproportionate amount of health care services (Katon et al., 1990; Kessler et al., 1994; Lyons, Larson, Burns, Cope, Wright, & Hammer, 1986; Ormel & Oldehinkel, 1993) and tend to stay longer within health care facilities than those without psychiatric problems (Fulop, Strain, Vita, Lyons, & Hammer, 1987; Lyons, Hammer, Larson, Petraitis, & Strain, 1986; Saravay, Steinberg, Weinschel, Pollack, & Alovis, 1991). High use of services and extended hospital stays result in increased costs for health care institutions. The first group of outcome studies concerns the effect of PCLN on decreasing cost, unnecessary service utilization, and hospital length of stay (see Table 1). Ragaisis (1996) evaluated the results of PCLN intervention on two complex patients and their families using the case study method. The interventions implemented with the patients, families, and staff included typical PCLN measures to decrease anxiety, to encourage discussion regarding treatment options and end-of-life care, and to access community resources that would facilitate timely discharge and provide appropriate aftercare. Ragaisis (1996) estimated that the outcomes of reduced hospital stay and transfer to more efficient care in these cases resulted in a hospital cost savings of over $18,000. The cost of care has also been studied in relation to the characteristics of hospitalized patients receiving low, medium, and high amounts of consultative/liaison (C/L) services from psychiatrists and PCLNs (Mallory, Lyons, Scherubel, & Reichelt, 1993). Patients receiving high intensity C/L services tended to be high consumers of health care resources as measured by nursing care hours, length of stay, contact with both the C/L physician and nurse, direct and indirect (staff-focused) consultative time, and transfer to other health care facilities, compared with other C/L patients. The number of nursing care hours actually increased from pre- to postconsultation within the high in-

EVALUATION OF OUTCOMES IN PSYCHIATRIC

219

Table 1. Published Studies of Psychiatric Consultation-Liaison Nursing (PCLN) Outcomes: Cost Reduction Authors

Participants

Outcome Measures

Mallory, Lyons, Scherubel & Reichelt (1993)

Patients receiving low, medium, and high amounts of consultative/ liason services from nurses and psychiatrists. Amount of services determined by patient need.

Ragaisis (1996)

Two complex patients and 1. their families; case 2. study method 3.

Results

Comments

1. Nursing care hours. 1. Nursing care hours 2. Length of hospital stay. increased in high intensity group. 2. Medium intensity group reported fewer nursing care hours than high intensity and was comparable to low intensity group. Low intensity group showed decrease in nursing care hours 3. Relationship found between earlier initiation of consultation referral and decreased length of stay in medium intensity group.

Talley, Davis, Suicidal and non-suicidal 1. Goicoechea, patients randomly 2. Brown, & assigned to PCLN Barber intervention and control (1990) groups. 3. 4.

(⫹) Revealed distinct patient groups for consultative resource consumption and length of stay. (⫹) Shown effectiveness of intervention on medium and low-intensity patient groups. (⫹) Suggested relationship between timing of referral to length of stay. (⫺) Absence of comparison group in study not receiving consultative services. Difficult to determine whether cost containment occurred in the high intensity group or whether results are truly because of PCLN interventions. (⫺) Relative effects of MD and RN consultations not compared. Length of stay. 1. Reduced length of stay. (⫹) Study estimated actual Initiation of efficient 2. Patients transferred monetary reductions care. from acute to hospice associated with PCLN Patient, family, and care. interventions. provider satisfaction. 3. Hospital cost savings (⫹) Suggests optimum ⬎$18,000. balance is possible 4. Satisfaction with PCLN between cost reduction interventions expressed and satisfaction. by patient, families, and (⫺) Small sample size providers. Number of sitter shifts. No differences found on (⫹) Used randomized Number of charted any of the outcome design with control nursing observations measurements between group. of patients. intervention and control (⫺) Outcome measures Number of safety and control groups. may have been incidents. inappropriate and Length of hospital stay. insensitive to PCLN interventions, eg., free availability of sitters, length of stay confounded with discharge disposition, high variability on outcome measures.

tensity group, but the investigators did not test this increase for statistical significance. Because the study did not have a comparison group, the effects of intervention on cost containment could not be addressed. After consultation, the high intensity

group continued to report significantly more nursing care hours than the medium or low intensity groups (Mallory et al., 1993). Patients receiving medium intensity C/L services showed a decrease in nursing care hours

220

between pre- and postconsultation, but this withingroup decrease was not tested for significance. After consultation, the medium-intensity group reported significantly fewer nursing care hours than the high-intensity group, but was comparable with the low-intensity group. The medium-intensity group also showed a relationship between early initiation of the consultation referral and decreased length of stay (Mallory et al., 1993). The low-intensity consultative group consisted of two subgroups: (a) patients with a primary psychiatric disorder who were referred or transferred to psychiatric care, and (b) patients referred late in their hospitalization for psychosocial assessment. The latter subgroup was usually seen solely by the C/L nurse. The low-intensity group showed a decrease in nursing care hours pre- and postconsultation, although the decrease was not tested for significance (Mallory et al., 1993). This study revealed patient groups that were distinct in terms of cost-related indicators such as resource consumption and length of stay, as well as differential response to consultative interventions. It showed the effectiveness of mental health interventions on reducing nursing care hours for patients receiving medium- and low-intensity consultative services. It also pointed out the importance of early referral to decreased length of stay among the medium-intensity group. The absence of a comparison group makes it impossible to determine whether the positive effects are because of the consultative interventions, or whether the interventions resulted in cost containment in the high-intensity group. The relative effects of physician and nurse consultations were also not analyzed. Another indicator of costs and service utilization is the use of sitters: lay staff who constantly monitor and meet the safety needs of patients who are confused, who may be harmful to themselves or others, who are prone to falls but unable to adhere to physical restrictions, and whose behavior is unpredictable and challenging to manage. Such highrisk patients are often not referred to PCLNs, because the sitter is considered adequate intervention by staff. Talley, Davis, Goicoechea, Brown, and Barber (1990) investigated the effect of PCLN consultation on patients who were assigned sitters by medical and nursing staff. The patients were divided into suicidal and nonsuicidal groups, and then randomly assigned to the intervention or no contact control group. Suicidal and nonsuicidal

YAKIMO, KURLOWICZ AND MURRAY

patients in the intervention group received a PCLN consultation individualized to their identified problems, with interventions targeted to nursing staff, patients, and sitters. No differences were found among any of the groups for number of sitter shifts, number of charted nursing observations of patients, number of safety incidents, and length of hospital stay. Talley, Davis, Goicoechea, Brown, and Barber (1990) attributed the lack of results to outcome measures that were either inappropriate or insensitive to the interventions. The decision to discontinue sitters for the suicidal patients was the responsibility of the consulting psychiatrist who tended to conservatively maintain sitters throughout the hospital stay to ensure the safety of the patient and liability of the hospital. Within this setting, sitters were available without financial constraints or justification for further use, whereas nursing staff was limited. Many of the patients within the nonsuicidal group were elderly and with multiple medical problems that were not expected to improve dramatically in the course of a hospitalization. Thus, the margin of safety provided by sitters outweighed any evidence of behavioral improvement resulting from PCLN interventions. Discharge for these elderly patients often depended on the availability of a care facility; discharge disposition confounded the measure of length of stay. In addition, the marked variability shown within groups on the outcome measures may have obscured any between group differences (Talley et al., 1990). Patient, Family, and Staff Satisfaction Several studies have shown the outcome of patient and staff satisfaction with PCLN services (see Table 2). Although the Ragaisis (1996) study previously reviewed primarily concerned costs, satisfaction by patients, families, and providers was also noted as an outcome of PCLN interventions. Wand and Happel (2001) reported high patient and provider satisfaction ratings with a pilot program of PCLN services within the emergency department. Newton and Wilson (1990) conducted a consumer satisfaction survey on a case-by-case basis with nurses who made requests to a PCLN service. Consultee ratings indicated consistent satisfaction with the services provided and judgment that the nurse’s recommendations were useful. Particular strengths of the service noted by the respondents included the ease with which the consultant could

EVALUATION OF OUTCOMES IN PSYCHIATRIC

221

Table 2. Published Studies of Psychiatric Consultation-Liaison Nursing (PCLN) Outcomes: Satisfaction With PCLN Services Outcome Measures

Results

Comments

Happell & Sharrock (2002)

Authors

Staff nurses in focus groups describing their experiences with PCLN consultation.

Participants

Satisfaction with PCLN consultation as depicted by themes emerging from focus group interaction.

Major themes: 1. Making contact 2. Helping staff 3. Implementing strategies 4. Utilizing attributes. Limitations of PCLN role: 1. Time factors limiting availability of PCLN; 2. PCLN over-stretched; 3. PCLN not being able to get others to take required action.

Newton & Wilson (1990)

Nurses who made requests to a PCLN service.

1. Consistent satisfaction with services 2. Recommendations found clear and useful 3. PCLN easily accessible 4. Least satisfied with written documentation, following of recommendations follow-up 5. Number of PCLN visits sole predictor of summed satisfaction score 6. More effective with family issues than in management of major psychiatric illness.

Sharrock & Happell (2001)

Staff who had utilized PCLN services.

Wand & Happell (2001)

Physicians and nurses from an Emergency Department (ED).

Consumer satisfaction survey consisting of items: 1. Ease of notification 2. Promptness of response 3. Written documentation 4. Verbal communication 5. Recommendation clarity 6. Recommendation practicality 7. Recommendations followed 8. Satisfaction with recommendations 9. Suggestions useful 10. Sufficient follow-up 11. Overall satisfaction 12. Likelihood of another referral. PCLN services rated on the following items for a consumer survey: 1. Response timeliness. 2. Relevance of services 3. Documentation. 4. Professional manner. 5. Improved continuity of care. 6. Answered unmet needs. Satisfaction ratings with pilot program of PCLN services.

(⫹) Concept of satisfaction elaborated by themes expressing different aspects of care processes (⫹) Complementary qualitative and quantitative studies (see Sharrock & Happell, 2001) (⫺) Small sample size (⫺) Evaluation general, does not focus on one consultation episode (⫺) Little variation in positive evaluations. (⫹) Satisfaction divided into multiple components (⫹) Range of satisfaction ratings by consultees (⫺) Evaluation summed over all consultation episodes, not specific (⫺) Confined to satisfaction of nurse consultees.

High level of satisfaction in all outcome measures. PCLN contributed to the professional role of condultees and to improved health outcomes. Staff expressed need for more PCLNs to provide adequate coverage of units throughout the day and week. High satisfaction ratings with PCLN consultation from ED staff as demonstrated by awareness of role, utilization, expressed benefits of continuing and expanding the role, and perceived contributions to improved patient care.

(⫹) Satisfaction divided into multiple components. (⫹) Participants included nonRN staff. (⫺) General evaluation of all PCLN consultations. (⫺) Little variation in outcome measures.

(⫹) Elaborated the components of staff satisfaction with PCLN services. (⫹) Included both physician and nurse satisfaction ratings. (⫹) Included both quantitative and qualitative measures. (⫹) Evaluates PCLN practice in an unstudied yet relevant clinical setting. (⫺) General evaluation. (⫺) No direct assessment of patient outcomes

Note. For studies documenting Satisfaction as a secondary outcome, see Kurlowicz (2001), Ragaisis (1996), and Tommasini (1992).

222

be accessed, the speed of response, and the clarity with which recommendations for patient care were formulated. However, the respondents were least satisfied with written documentation and the amount of follow-up. Accordingly, the total number of PCLN visits was the sole predictor of satisfaction. Higher satisfaction was also associated with consultations regarding assistance with patients’ families compared with requests for the management of major psychiatric disorders (Newton & Wilson, 1990). Happell & Sharrock (2002) conducted focus groups with nurses at a large Australian general hospital to determine their satisfaction with services offered by the psychiatric consultation-liaison nurse. Responses of the participants were remarkably positive. The four main themes to emerge from the groups were: (a) making contact; (b) helping staff; (c) implementing strategies; and (d) using attributes. Making contact involved the staff calling the psychiatric nurse when caring for a challenging or complex patient, including her as part of the ward team, and feeling assured that PCLN resources would be available when needed. Participants provided numerous examples of situations in which the nurse helped staff through providing support, advice, and education that assisted them in providing optimal care for patients with complex needs. The nurse also assisted the staff in formulating and implementing individualized strategies for improving patient care. The personal and professional attributes of the consulting nurse were considered as important as the interventions she applied. The consultant’s nonjudgmental and nonthreatening manner also facilitated the ease with which staff members approached her and considered her suggestions. The staff also felt the nurse’s outsider status was important in that she was perceived as more objective than other personnel and not allied with administration. Sharrock and Happell (2001) conducted a hospital survey to obtain overall satisfaction ratings from staff who had used PCLN services. The results of the survey indicated a high level of agreement that the consulting nurse’s response was timely, relevant to the referral problem, well documented, professional, improved the continuity of care, and answered previously unmet needs. The respondents felt that the nurse assisted them in giving more effective care by improving patient management skills and providing education and

YAKIMO, KURLOWICZ AND MURRAY

support. The major limitations in the PCLN service involved the need for more than one nurse to adequately cover the hospital units and to extend availability of the service throughout the day and week. In a study of changes in the perceived work environment of staff nurses, Tommasini (1992) also noted the request for continued PCLN consultation as an indicator for staff satisfaction. Changes in Patient Clinical Status Two published studies attempted to assess changes in patients’ clinical status after PCLN interventions (see Table 3). Priami and Plati (1997) adapted the Nurses’ Observation Scale for Inpatient Evaluation (NOSIE-30) to evaluate problems with depression, withdrawal, reality testing, anxiety, and motivation for treatment among medicalsurgical patients referred for PCLN consultation in a Greek hospital. The NOSIE-30 was completed by a staff psychiatric nurse before and after four PCLN consultations. The consultations consisted of individualized interventions addressing the problems indicated on the first administration of the NOSIE-30. Results of the study shown significant decreases in problematic behaviors and increases in coping behaviors on virtually all items of the NOSIE-30; 83% of the patients attributed their decrease in symptoms and relief of distress to PCLN interventions. Strengths of the study included direct assessment of the specific distressing symptoms that prompted PCLN consultation and independent assessment of the patients’ clinical status. Limitations included alteration of the reliability and validity of the NOSIE-30 by eliminating items and the lack of a comparison group that would strengthen the association of the patients’ improvement to PCLN interventions. Kurlowicz (2001) examined patient characteristics, changes in psychiatric symptoms, and discharge disposition of older hospitalized patients after PCLN consultation. The Geriatric Depression Scale (GDS) and Confusion Assessment Method (CAM) were administered to all patients at the first visit, and again at the time of discharge, to those patients who initially screened positive for depression or delirium. PCLN interventions for the screen-positive patients were tailored to their individual needs. Significant decreases in symptoms of depression and delirium after PCLN intervention were noted on the GDS and CAM; 12% of the patients who had screened positive for depression

EVALUATION OF OUTCOMES IN PSYCHIATRIC

223

Table 3. Published Studies of Psychiatric Consultation-Liaison Nursing (PCLN) Outcomes: Changes in Patient Psychiatric Symptoms Authors

Participants

Outcome Measures

Results

Comments

Kurlowicz (2001)

Elderly patients screening positive for symptoms of depression or delirium on hospital admission.

1. Geriatric Depression Scale (GDS) 2. Confusion Assessment Method (CAM) 3. Staff evaluation of expected patient discharge disposition. 4. Staff satisfaction (2item questionnaire).

(⫹) Used reliable and valid instruments. (⫹) Pre-post measures. (⫹) Multiple outcomes measured concurrently. (⫺) Lack of variation in perception of PCLN helpfulness. (⫺) Absence of comparison group not receiving PCLN services. Comparison group would increase confidence that changes were because of PCLN interventions.

Priami & Plati (1997)

Hospitalized medicalsurgical patients referred for PCLN consultation.

1. Nurses’ Observation Scale for Inpatient Evaluation (NOSIE-30). Items reflect depression, withdrawal, reality testing, anxiety, and motivation for treatment 2. Patient attribution of behavioral change.

1. Significant decreases in depression and delirium symptoms as measured by GDS and CAM. 2. 12% of screen positive patients showed improved discharge dispositions. 3. All staff agreed consulting nurse had been helpful and supportive in teaching skills in managing confused and depressed elderly. 4. Content analysis revealed three themes describing how PCLN helped: cognitive/perceptual (reframing, knowledge), affective (support, efficacy) and behavioral (increasing skill-building). 1. Decreases in problematic behaviors and increases in adaptive behaviors on all items of NOSIE-30. 2. Patients attributed changes to PCLN interventions.

or delirium showed improved discharge dispositions beyond that expected at the time of admission. In addition, high staff satisfaction was found with PCLN consultation. On a two-item questionnaire, all of the involved staff nurses agreed that PCLN consultation had been helpful. Staff described that the consulting nurse was most helpful in improving their understanding of the patients’ problems, feeling supported by the consulting nurse, and developing an increased capacity to handle clinical situations involving confused and depressed elderly (Kurlowicz, 2001). Changes in Perceived Work Environment Tommasini (1992) investigated changes in the perceptions of a teaching hospital specialty unit among eight registered nurses attending a 12-week support

(⫹) Pre-post measures. (⫹) Multiple outcomes. (⫹) Direct and independent assessment of patients’ clinical status. (⫹) Multiple outcome measures. (⫺) Alteration of NOSIE-30 items3 altered reliability and validity. (⫺) Lack of comparison group to strengthen association of improvement to PCLN interventions.

group facilitated by a PCLN clinician (Table 4). Participants were asked to give real versus ideal ratings of their current work setting on the Work Environment Scale (WES) before and after group intervention. Group topics and interventions focused on discrepancies between the ideal and real work environment dimensions assessed by the WES as well as disturbed communication and accountability issues raised by the participants. After group completion, the staff reported significant changes in the dimensions of clarity and control. Clarity referred to the nurses perceiving greater stability in rules and policies and feeling that work activities were wellplanned. Control referred to the expectation that rules were followed and enforced in the carrying out of work activities. An improvement in communication and decrease in staff conflict was observed within the

224

YAKIMO, KURLOWICZ AND MURRAY

Table 4. Published Studies of Psychiatric Consultation-Liaison Nursing (PCLN) Outcomes: Perceived Work Environment Authors

Tommasini (1992)

Participants

Registered nurses from a specialty unit of a teaching hospital attending a 12-week staff support group led by a PCLN.

Outcome Measures

Results

Comments

Real versus ideal ratings of current work setting using the Work Environment Scale (WES) both prior to and after group intervention.

1. Statistically significant changes found on WES measures of clarity and control pre- and postintervention. 2. Qualitative observations showed a decrease in staff conflict and improvement in interpersonal communication within the group meetings. 3. Participants felt transfer of improved behaviors from group to work setting was incomplete: requested continued meetings with the PCLN.

(⫹) Pre- and postintervention measures. (⫹) Use of real and ideal discrepancies in premeasures to guide intervention. (⫹) Use of both quantitative and qualitative measures. (⫹) Continued utilization of PCLN services is proxy for staff satisfaction. (⫺) No measures of nursing staff performance or patient outcomes (⫺) Lack of control group. (⫺) Small sample size.

group meetings but no changes were found on parallel dimensions on the WES for staff cohesion or supervisor support after the intervention. The group participants felt that the transfer of behaviors from the group to the work setting was incomplete and thus requested further meetings with the PCLN clinician. Strengths of Tommasini’s (1992) study included pre- and postintervention outcome measures that were congruent with topics addressed in the group, the use of real and ideal discrepancies in the preintervention measures to tailor group intervention, the inclusion of both quantitative and qualitative measures, and the inference of requests for continued consultation with the PCLN as a proxy for satisfaction. Limitations included the lack of measures of nursing staff performance or patient outcomes, the absence of a comparison group to link the changes in perception to the group intervention, and the small sample size. SUGGESTIONS FOR DEVELOPMENT OF OUTCOME ANALYSIS IN PCLN

The strengths and limitations of current PCLN outcome studies provide the guidelines for future development. The foundation of all future outcomes research lies in psychiatric nurses acknowledging the importance of measuring outcomes for the scientific and economic viability of the discipline. This involves a number of changes in cognition and practice: (a) acknowledging the necessity of proving the worth PCLN practice, (b) showing the measurability of changes within

PCLN clients, (c) defining outcomes that are of special interest to different groups such as patients, families, staff, and administrators, (d) selecting appropriate measures for documenting these changes, and (e) selecting the best methods for documenting outcomes. Proving the Worth of PCLN Practice Proving the worth of nursing practice is a relatively new idea within health care. For many years, the health care system has evaluated its practitioners in terms of structure and process measures—type of personnel and facilities offering care, types of problems shown by the population, the specific care provided, and the volume of people served. During the late 1980s, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) spearheaded the shift from structure and process to outcome measures–from measures of resources and service activities to showting health care professionals’ contribution to changing the health status of the recipient. Despite this shift to outcome orientation, many nurses are not familiar with formulating and measuring outcome indicators relevant to their daily practice (Barrell, Merwin, & Poster, 1997; Buchanan, 1994; Oermann & Huber, 1999). Measurability of Changes Within PCLN Clients Fortunately, several groups of nursing researchers have initiated the task of defining and validating the outcomes of nursing care. PCLNs would

EVALUATION OF OUTCOMES IN PSYCHIATRIC

benefit from examining available outcome taxonomies to stimulate outcome-oriented thinking and to make informed selections of those outcomes representative of their customary practices. A review of the expanding and evolving concept of outcome as described historically by nursing theorists was offered by Neale (2001) and surveys of outcome classification schemes and associated potential measurement instruments are available (Holzemer & Henry, 1999; Jennings, Staggers, & Brosch, 1999; Urden, 1999). The Nursing Outcomes Classification (NOC) from the University of Iowa (Johnson, Bulechek, Dochterman, Maas, & Moorhead, 2001) presents an extensive outcome taxonomy with its own schemes for definition and measurement. Recent work from this group has focused on discerning the patterns of linkages among the structure, process, and outcome variables of nursing diagnoses, interventions, and outcomes (Johnson, Bulechek, Dochterman, Maas, & Moorhead, 2001). Defining Outcomes According to Interest Group Perhaps the greatest motivator for measuring outcomes are the demands of administrators, payers, and consumers. Although such groups may call for evidence supporting one or two outcome categories, they do not typically ask for a variety of outcome data. In fact, the outcomes of interest may change with time, depending on the changing initiatives and goals of the organizations (Yakimo & Murray, 2002). Because outcome has been shown to be a concept best measured by multiple indicators (Donabedian, 1992), it is proactive to document multiple outcome indicators and judiciously offer the data to involved parties, even if not explicitly requested. No one understands the goals of practice and thus the nature of relevant outcomes like the clinician within a particular practice setting. The foci and method of presenting outcome evidence needs to be tailored to the interests of the intended recipient: such as costs for administrators, satisfaction with care for patients and families, and work satisfaction for staff (Jennings, Staggers, & Brosch, 1999; Kleinpell, 1997; Neale, 1999). Choosing Outcome Instruments Selecting appropriate outcome instruments may be accomplished through guidelines proposed by Stewart and Archbold (1992, 1993) who contended

225

that outcome measures must be responsive–able to detect clinically significant changes. Responsivity may be assured through several means. First, outcome measures must be closely tied conceptually to the interventions used by PCLNs (Stewart & Archbold, 1992). In other words, the outcome measure should be specific enough to reflect the intent of the intervention(s). For example, if the PCLN professional is working with a patient on anxiety over impending surgery, a state anxiety measure is a more appropriate outcome than a more global measure of health-related quality of life that is affected by many other sources than PCLN intervention. Second, the outcome variable must be amenable to change (Stewart & Archbold, 1992). For example, patients with chronic confusion are unlikely to show improved orientation after PCLN intervention compared with those with acute confusion. A thorough assessment of the problem situation will detail those aspects potentially responsive to intervention and point to those measures that will capture the intended outcomes. Third, outcome measures thus need to be tailored according to the specific problems shown by the client (Stewart & Archbold, 1992, 1993). No single outcome measure is appropriate for all clinical situations and existing instruments need to be tailored according to the demands of the particular situation in which PCLNs are intervening. Best Methods for Measuring Outcomes Because PCLNs cannot impartially determine their own outcomes, it is ideal for others to provide outcome ratings. This may be the patient, staff member, or administrator who requested the consultation or a third party who will judge change within the person or situation that is the focus of the consultation. However, PCLNs can provide the type of evidence that will maximize the outside rater’s orientation to outcome change, such as providing a clear report of the referral request and the steps taken in assessment, problem formulation, and intervention that will support the evaluation of an outcome by a third party (Anderson, 1983). Outcomes need to be measured at several time points during PCLN intervention, at least at the beginning and conclusion of consultation. The demonstration of change over time is the strongest evidence for both the efficacy of PCLN intervention and for the responsivity of the outcome instrument (Stewart & Archbold, 1992, 1993). If possi-

226

YAKIMO, KURLOWICZ AND MURRAY

ble, outcome studies should include a comparison group that does not receive the intervention so that the changes can be more reliably tied to the intervention than to the effects of other variables. CONCLUSION

The available literature shows the paucity of systematic research on the outcomes of mental health consultation and liaison. The PCLN literature has evolved through conceptual stages that reflect the professional development of the subspecialty (Robinson, 1982). The literature of the 1960s through 1980s concerned definition of the role and detailing of the care processes used by PCLNs. However, the ascendancy of managed care systems in the 1990s, rising concern of health care agencies regarding costs, and associated interest of all health professionals in evidence-based care resulted in the first systematic PCLN outcome studies. Unfortunately, these studies only looked at a few isolated variables that did not capture the multiple processes and range of outcomes that PCLNs may potentially affect (Kurlowicz, 1998; 2001). The present state of PCLN requires a comprehensive delineation of outcomes that reflects the specific goals of PCLN practice, the multiple settings and ways in which the role is enacted, and the effects of practice on patients, families, staff, administrators, and the health care system. The sources of meaningful outcomes are similar to the sources of effective interventions: deductions from relevant conceptual models, the results of intervention effectiveness studies, and the observation, interview, and validation of practicing clinicians that result in exhaustive description of the phenomenon (Johnson, Maas, & Moorhead, 2000; Morse, 2002; Morse, Penrod, & Hupcey, 2000). REFERENCES Anderson, M.L. (1983). Nursing interventions: What did you do that helped? Perspectives in Psychiatric Care, 21(1), 4-8. Angelino, A.F., & Treisman, G.J. (2001). Major depression and demoralization in cancer patients: Diagnostic and treatment considerations. Supportive Care of Cancer, 9, 355349. Arolt, V., & Driessen, M. (1996). Alcoholism and psychiatric comorbidity in general hospital inpatients. General Hospital Psychiatry, 18, 271-277. Barrell, L.J., Merwin, E.I., & Poster, E.C. (1997). Patient outcomes used by advanced practice psychiatric nurses to evaluate effectiveness of practice. Archives of Psychiatric Nursing, 11, 184-197.

Buchanan, L.M. (1994). Therapeutic nursing intervention knowledge and outcome measures for advanced practice nurses. Nursing and Health Care, 15, 190-195. Carveth, J.A. (1995). Perceived patient deviance and avoidance by nurses. Nursing Research, 44, 173-178. Clarke, D.M., Smith, G.C., & Hermann, H.E. (1993). A comparative study of screening instruments for mental disorders in the general hospital. International Journal of Psychiatry in Medicine, 23, 323-37. Donabedian, A. (1966). Evaluating the quality of medical care. Millbank Memorial Fund Quarterly, 44(Pt. 2), 166-203. Donabedian, A. (1980). The definition of quality and approaches to its assessment. Ann Arbor, MI: Health Administration Press. Donabedian, A. (1982). The criteria and standards of quality. Ann Arbor, MI: Health Administration Press. Donabedian, A. (1988). The quality of care: How can it be assessed? Journal of the American Medical Association, 260, 1743-1748. Donabedian, A. (1992). The role of outcomes in quality assessment and assurance. Quality Review Bulletin, 18, 356360. Donabedian, A. (1993). Quality in health care: Whose responsibility is it? American Journal of Medical Quality, 8(2), 32-36. Fulop, G., Strain, J., Vita, J., Lyons, J.S., & Hammer, J.S. (1987). Impact of psychiatric cormorbidity on length of hospital stay for medical/surgical patients. American Journal of Psychiatry, 144, 878-882. Happell, B., & Sharrock, J. (2001). The psychiatric consultation-liaison nurse: Towards articulating a model for practice. Journal of Psychiatric and Mental Health Nursing, 8, 411-417. Happell, B., & Sharrock, J. (2002). Evaluating the role of a psychiatric consultation-liaison nurse in the Australian general hospital. Issues in Mental Health Nursing, 23, 43-60. Holzemer, W.L., & Henry, S.B. (1999). Therapeutic outcomes sensitive to nursing. In A.S. Hinshaw, S.L. Feetham, & J.L.F. Shaver (Eds.), Handbook of clinical nursing research. Thousand Oaks, CA: Sage. Jennings, B.M., Staggers, N., & Brosch, L.R. (1999). A classification scheme for outcome indicators. Journal of Nursing Scholarship, 31, 381-388. Johnson, M., Maas, M., & Moorhead, S. (2000). Nursing outcomes classification. St. Louis: Mosby. Johnson, M., Bulechek, G., Dochterman, J.M., Maas, M., & Moorhead, S. (2001). Nursing diagnosis, outcomes, and interventions: NANDA, NIC, and NOC linkages. St. Louis: Mosby. Katon, W., Von Korff, M., Lin, E., Bush, T., Lipscomb, P., & Russo, J. (1990). Distressed high users of medical care: DSM-III-R diagnoses and treatment needs. General Hospital Psychiatry, 12, 355-362. Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., & Eshelman, S. (1994). Lifetime and 12 month prevalence of DSM-III psychiatric disorders in the United States. Archives of General Psychiatry, 51, 8-19. Kleinpell, R.M. (1997). Whose outcomes: patients, providers, or payers? Nursing Clinics of North America, 32, 513-20.

EVALUATION OF OUTCOMES IN PSYCHIATRIC

Kurlowicz, L.H. (1998). Psychiatric consultation-liaison nursing. In A.W. Burgess (Ed.), Advanced practice psychiatric nursing. Stamford, CT: Appleton & Lange. Kurlowicz, L.H. (2001). Benefits of psychiatric consultationliaison nurse interventions for older hospitalized patients and their nurses. Archives of Psychiatric Nursing, 15, 53-61. Lewis, A., & Levy, J.S. (1982). Psychiatric liaison nursing: The theory and practice. Reston, VA: Reston Publishing. Lyons, J.S., Hammer, J.S., Larson, D.B., Petraitis, J., & Burns, B.J. (1988). Treatment opportunities on a consultation/ liaison service. American Journal of Psychiatry, 145, 1415-1438. Lyons, J.S., Larson, D.B., Burns, B.J., Cope, N., Wright, S., & Hammer, J. (1988). Psychiatric co-morbidities and patients with head and spinal cord trauma: Effects on acute hospital care. General Hospital Psychiatry, 10, 292-297. Mallory, G.A., Lyons, J.S., Scherubel, J.C., & Reichelt, P.A. (1993). Nursing care hours of patients receiving varying amounts and types of consultation/liaison services. Archives of Psychiatric Nursing, 7, 353-360. Mayou, R., & Hawton, K. (1986). Psychiatric disorders in the general hospital. British Journal of Psychiatry, 149, 172-190. McCloskey, J.C., & Bulechek, G.M. (1996). Nursing interventions classification. St. Louis: Mosby. Minarik, P.A., & Neese, J.B. (2002). Essential educational content for psychiatric consultation-liaison nursing. Archives of Psychiatric Nursing, 16, 3-15. Morse, J.M. (2002). Where do interventions come from? Qualitative Health Research, 12, 435-6. Morse, J.M., Penrod, J., & Hupcey, J.E. (2000). Qualitative outcome analysis: Evaluating nursing interventions for complex clinical phenomena. Journal of Nursing Scholarship, 32, 125-130. Neale, J.E. (2001). Patient outcomes: A matter of perspective. Nursing Outlook, 49, 93-99. Nelson, J.K.N., & Schilke, D.A. (1976). The evolution of psychiatric liaison nursing. Perspectives in Psychiatric Care, 14, 60-65. Newton, L., & Wilson, K.G. (1990). Consultee satisfaction with a psychiatric consultation-liaison nursing service. Archives of Psychiatric Nursing, 4, 254-270. Norwood, S.L. (1998a). Nurses as consultants: Essential concepts and processes. Menlo Park, CA: Addison Wesley. Norwood, S.L. (1998b). Psychiatric consultation-liaison nursing: Revisiting the role. Clinical Nurse Specialist, 12, 153-156. Oermann, M.H., & Huber, D. (1999). Patient outcomes: A measure of nursing’s value. American Journal of Nursing, 99(9), 40-47. Ormel, J., & Oldehinkel, T. (1993). Recognition, management, and course of anxiety and depression in general practice. Archives of Psychiatric Nursing, 13, 248-260. Priami, M., & Plati, C. (1997). The effectiveness of mental health nursing interventions in a general hospital. Scandinavian Journal of Caring Sciences, 11, 56-62. Ragaisis, K.M. (1996). The psychiatric consultation-liaison nurse and medical family therapy. Clinical Nurse Specialist, 10, 50-56.

227

Roberts, D. (1997). Liaison mental health nursing: Origins, definitions and rospects. Journal of Advanced Nursing, 25, 101-108. Robinson, L. (1982). Psychiatric liaison nursing 1962-1982: A review and update of the literature. General Hospital Psychiatry, 4, 139-145. Rosswurm, M.A., & Larrabee, J.H. (1999). A model for change to evidence- based practice. Image: Journal of Nursing Scholarship, 31, 317-322. Saravay, S.M., & Lavin, M. (1994). Psychiatric comorbidity and length of stay in the general hospital. Psychosomatics, 35, 233-252. Saravay, S.M., Steinberg, M.D., Weinschel, B., Pollack, S., & Alovis, N. (1991). Psychological comorbidity and length of stay in the general hospital. American Journal of Psychiatry, 148, 324-329. Savoca, E. (1999). Psychiatric co-morbidity and hospital utilization in the general medical sector. Psychological Medicine, 29, 457-464. Shahinpour, N., Hollinger-Smith, L., & Perlia, M.A. (1995). The medical psychiatric consultation-liaison nurse: Meeting psychosocial needs of medical patients in the acute setting. Nursing Clinics of North America, 30, 77-85. Sharrock, J., & Happell, B. (2000). The psychiatric consultation-liaison nurse: Towards articulating a model for practice. Australian and New Zealand Journal of Mental Health Nursing, 9, 19-28. Sharrock, J., & Happell, B. (2001). The role of the psychiatric consultation-liaison nurse in the improved care of patients experiencing mental health problems receiving care within a general hospital environment. Contemporary Nurse, 11, 260-270. Stewart, B.J., & Archbold, P.G. (1992). Nursing intervention studies require outcome measures that are sensitive to change: Part one. Research in Nursing and Health, 15, 477-481. Stewart, B.J., & Archbold, P.G. (1993). Nursing intervention studies require outcomes measures that are sensitive to change: Part two. Research in Nursing and Health, 16, 77-81. Talley, S., Davis, D.S., Goicoechea, N., Brown, L., & Barber, L.L. (1990). Effect of psychiatric liaison nurse specialist consultation on the care of medical-surgical patients with sitters. Archives of Psychiatric Nursing, 4, 114-123. Tommasini, N.R. (1992). The impact of a staff support group on the work environment of a specialty unit. Archives of Psychiatric Nursing, 6, 40-47. Tunmore, R., & Thomas, B. (1992). Models of psychiatric consultation-liaison nursing. British Journal of Nursing, 1, 447-51. Urden, L.D. (1999). Outcome evaluation: An essential component for CNS practice. Clinical Nurse Specialist, 13, 39-46. Wand, T., & Happell, B. (2001). The mental health nurse: Contributing to improved outcomes for patients in the emergency department. Accident and Emergency Nursing, 9, 166-176. Yakimo, R., & Murray, R. (2002). Refining psychiatric consultation-liaison nursing outcomes through analysis of interview data. Unpublished manuscript, Saint Louis University School of Nursing.