- Email: [email protected]
A Prospective Study of the Reliability and Validity of the Live Donor Assessment Tool
Psychosomatics 2017:58:519–526
& 2017 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.
Original Research Report A Prospective Study of the Reliability and Validity of the Live Donor Assessment Tool Brian M. Iacoviello, PhD., Akhil Shenoy, M.D., Julia Hunt, LMSW., Zorica Filipovic-Jewell, M.D., Brandy Haydel, B.S., Dianne LaPointe Rudow, D.N.P.
Background: The psychosocial evaluation is an important part of the live organ donor evaluation process, yet this is not standardized across institutions. Objective: This study was designed to prospectively test the reliability and validity of a semistructured psychosocial evaluation tool that was recently developed and reported in the literature (the Live Donor Assessment Tool [LDAT]). Methods: A total of 248 live donor candidates who presented for evaluation were invited to participate in a study that involved the LDAT being scored as part of the standard psychosocial evaluation process; 222 provided informed consent. Evaluations were conducted by staff experienced with psychosocial evaluation of living donors and trained in the use of the LDAT. Furthermore, 123 donor candidates were evaluated twice, as per routine standard of care, and had 2 LDATs administered. Reliability of the LDAT was
assessed by calculating the internal consistency of the LDAT items and inter-rater reliability. Validity was assessed by comparing LDAT scores across the riskgroup categories (the traditional outcome designation of the psychosocial evaluation) and in 86 eventual donors, associations between LDAT scores, and indicators of psychosocial outcomes post-donation. Results: The LDAT was found to have good internal consistency, strong inter-rater reliability, and showed signs of validity: LDAT scores differentiated the traditional risk-group categories, and a significant association between LDAT score and treatment adherence postdonation was revealed. Conclusions: The LDAT demonstrated good reliability and validity, but future research on the LDAT and the ability to implement the LDAT across institutions is warranted. (Psychosomatics 2017; 58:519–526)
Key words: Live donors, Organ, Psychosocial, Assessment, Donation. INTRODUCTION Live organ donors are a vital source of donor organs for transplantation. Live kidney donation for patients with chronic kidney failure has been performed for over 5 decades. Kidney donation has been shown to be safe, with kidney donors exhibiting minimal risk of end-stage renal disease provided they lead a healthy lifestyle.1–3 Live liver donation has been performed in adults since 1998, most often at large centers in geographical areas with deceased donor shortages. Psychosomatics 58:5, September/October 2017
Studies of medical outcomes have shown that although liver donation portends a higher risk than Received August 30, 2016; revised March 13, 2017; accepted March 14, 2017. From the Department of Psychiatry (BMI, AS, ZFJ), Icahn School of Medicine at Mount Sinai; Recanati-Miller Transplantation Institute at Mount Sinai (AS, JH, ZFJ, BH, DLR), The Zweig Family Center for Living Donation, New York, NY. Send correspondence and reprint requests to Brian M. Iacoviello, PhD, Department of Psychiatry, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1230, New York, NY 10029; e-mail: [email protected] & 2017 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.
www.psychosomaticsjournal.org
519
Prospective study of the LDAT kidney donation, it is still safe enough to perform at experienced centers.4,5 Psychologic outcomes for liver and kidney donors have also been investigated. These studies have shown that donors often report a better quality of life compared to age-matched healthy adults, and most donors indicate that they would donate again.6–8 Studies have also demonstrated that live donors exhibit adaptive personality traits (agreeableness, conscientiousness, and lower neuroticism) to a greater degree compared than age-matched healthy adults who are equally as resilient as the general population, are significantly more resilient than primary care patients, and who score very highly on measures of Purpose in Life and Growth after donation.9 However, there are also reports of less-frequent negative outcomes associated with live organ donation, including suicides and serious psychiatric complications,10 often associated with predonation psychiatric histories. Perhaps transplant studies are biased toward favorable outcome results by the inclusion of patients who make it through the selection process. That being said, despite the current screening process, some donors report negative psychosocial outcomes, including depression, anxiety, and regret.8 Further, 9% (n ¼ 231) of kidney donors in the RELIVE study reported one or more of the following poor psychosocial outcomes: fair or poor overall donor experience, financial burden, regret or discomfort with decision to donate, or psychologic difficulties since donation.11 Many donors require counseling in the year after donation,12 with a smaller number needing pharmacotherapy for depression or anxiety. Organ donation can be a stressful experience for the donors and their family. Identifying a strong social support system is important for resilient outcomes.13 Taken together, consideration of the psychosocial risk factors, in addition to the medical risk factors, is critical for prediction of patients who will have positive vs negative donation experiences and outcomes, despite screening processes that may already bias studies toward favorable results. The psychosocial evaluation of live organ donors has traditionally relied on a clinical interview to investigate the presence or absence of predonation psychosocial stressors and underlying psychological disorders. However, the psychosocial evaluation of live donors has not yet been standardized and can vary widely across providers and institutions,14 and there are no validated measures available to determine 520
www.psychosomaticsjournal.org
psychosocial risks for donation. There are several tools used to assess organ recipients: the Stanford Integrated Psychosocial Assessment for Transplantation,15 the Psychosocial Assessment of Candidates for Transplantation,16 the Transplant Evaluation Rating Scale,17 and the Structured Interview for Renal Transplantation.18 The Stanford Integrated Psychosocial Assessment for Transplantation is the newest of these tools; its strengths include its comprehensive nature and standardization of the information collected during the psychosocial evaluation process. Although it did not predict the primary outcomes of graft failure and mortality, the Stanford Integrated Psychosocial Assessment for Transplantation was shown to have excellent inter-rater reliability and to be predictive of the transplant recipient’s rejection episodes, rehospitalization rate, psychiatric decompensation, and social support failure.19 This type of assessment tool is needed for the psychosocial assessment of live organ donors. One semistructured psychosocial assessment tool has been developed and reported in the literature: the Live Donor Assessment Tool (LDAT20). The LDAT is a tool for psychiatrists, psychologists, social workers, and other clinicians involved in the evaluation process to quantify the degree of psychosocial appropriateness vs riskiness of potential donors. Comprising 29 items scored 0–2 or 0–3 across 9 domains (motivations for donation, knowledge about donation process, relationship with the recipient, support available to the donor, donor’s feelings about donation, post-donation expectations, stability in life, psychiatric history, and alcohol and substance use), the LDAT is scored such that higher scores indicate greater psychosocial appropriateness for donation. A retrospective chart-review study using the LDAT revealed preliminary indications of very good inter-rater reliability between 2 independent raters, and validity in LDAT scores being consistent with the risk category designation resulting from the traditional psychosocial evaluation.20 The purpose of the project reported in this article was to prospectively test the reliability and validity of the LDAT as a semistructured psychosocial assessment tool in a sample of live organ donor candidates presenting for evaluation at a large, urban medical center. Reliability was assessed by investigating inter-rater reliability of the LDAT scores obtained by 2 independent raters; validity was assessed by investigating associations between the LDAT score Psychosomatics 58:5, September/October 2017
Iacoviello et al. obtained during the evaluation and the traditional evaluation outcome categories as well as psychosocial outcomes postdonation. METHODS Participants Between January 1, 2015 and October 31, 2015, 248 live donor candidates presented for evaluation; 222 provided informed consent and were included in the study protocol. All potential live donors who underwent a live donor psychosocial evaluation using the LDAT were eligible for the study. Participants must have been English-speaking live organ donor candidates who presented for the donor evaluation and saw the transplant coordinator, physician, and completed a psychosocial evaluation. Liver donors who sought to donate for the reason of acute liver failure in an emergent situation were excluded. During the psychosocial evaluation, all candidates were evaluated for psychosocial suitability by an experienced clinician according to the standard practice in the transplant program. An LDAT was completed by the clinician after each evaluation. All liver donors were evaluated by both a social worker and a psychiatrist. Kidney donors saw only a social worker unless they were determined to be at increased risk for psychosocial complications, had a history of psychiatric illness, were distantly or unknown to the recipient, or were participating in a paired kidney donation. Of the 222 candidates, 123 evaluated were administered 2 evaluations. Of the 222 candidates who were evaluated, 86 eventually donated an organ. The Table provides descriptive characteristics of the donor candidates evaluated and the eventual donors. Evaluations were conducted by licensed clinicians (psychiatrists [A.S. or Z.F.J.] and a social worker [J.H.]) employed in the transplant program. Clinicians all had several years of experience conducting psychosocial evaluations of live donor candidates and were familiar with the development and study of the LDAT. Some clinicians (A.S. and J.H.) had previously been responsible for training LDAT raters for a retrospective, chart-review study of the LDAT, and thus had excellent understanding of the LDAT and its use. Owing to limited resources, one of the clinicians involved in the development of the LDAT (A.S.) was also involved in the evaluation of donor candidates in Psychosomatics 58:5, September/October 2017
TABLE.
Characteristics of the Sample of Donor Candidates and Eventual Donors
Age (y) Sex (% female) Ethnicity (%) White African American/ Black Asian Other LDAT
Age (y) Sex (% female) Ethnicity (%) White African American/ Black Asian Other LDAT
Kidney donor candidates (n ¼ 171)
Liver donor candidates (n ¼ 51)
42.4 (12.7) 55.6
37.0 (9.3) 49.0
71.3 16.4
66.7 13.7
7.0 5.3
13.7 5.9
65.72 (7.48)
66.71 (5.52)
Kidney donors (n ¼ 65)
Liver donors (n ¼ 21)
42.4 (12.0) 60.0
36.7 (9.6) 47.6
81.5 12.3
57.1 19.0
1.5 4.6
9.5 14.3
67.2 (6.0)
68.5 (3.5)
Note: Values shown indicate mean (SD) or %.
the current study. Other clinicians involved in the development of the LDAT were excluded from evaluating donors in the current study. Procedures All live donor candidates who presented for evaluation at a large, urban medical center were invited to participate in the study. Those who were interested were provided information about the study purpose and procedures and were explicitly informed that study participation would not affect their evaluation process nor would it influence their clearance as a donor. They were further informed that the LDAT score was not calculated by the clinician during the psychosocial evaluation, rather it was entered into the REDCap database and scored by the research assistant. All interested participants provided written informed consent. The outcome of the psychosocial evaluation process was categorized www.psychosomaticsjournal.org
521
Prospective study of the LDAT using the subjective categories traditionally employed to describe the evaluation outcome (declined for psychosocial reasons, cleared but high risk, cleared with moderate risk, or cleared with low risk). For all study participants, an LDAT was completed, and an LDAT score was obtained after the traditional psychosocial evaluations were conducted. Possible LDAT scores ranged from 0–82, and although they were not available to the selection committee members at the time of the committee meetings, the clinicians who conducted the psychosocial evaluations and completed LDATs were involved in the discussions on decision-making. Data were entered directly into an electronic data capture system (REDCap) by raters after the evaluation, which resulted in zero missing data. For the 86 candidates who completed live organ donation, psychosocial outcomes were assessed at 2 time points: 6 weeks and 6 months post-donation. Outcome assessments were conducted by a doctorallevel nurse practitioner with many years experience working with live organ donors (D.L.R.), but who was not involved with the psychosocial evaluation or the scoring of the LDAT. The following 6 psychosocial outcomes were assessed as present/absent based on the entire live donor population: work impairment, relationship impairment, change in psychological status, nonstandard psychosocial interventions required, excessive pain, and treatment adherence. A total psychosocial outcome score was also computed as the sum of the factors above rated as present and ranged from 0–6 with greater scores indicating increased impairment post-donation.
related constructs, was measured by calculating Cronbach alpha21 across all 345 ratings, with the value interpreted according to established ranges (e.g., α o 0.5 ¼ “unacceptable”; 0.5 r α o 0.6 ¼ “ poor”; 0.6 r α o 0.7 ¼ “questionable”; 0.7 r α o 0.8 ¼ “acceptable”; 0.8 r α o 0.9 ¼ “good”; and α Z 0.9 ¼ “excellent”22). Inter-rater reliability was measured by computing Pearson correlation coefficients for the LDAT scores when 2 evaluations were completed for a donor candidate, resulting in 123 LDAT pairs compared. Values were interpreted according to Cohen (1988): r Z 0.1 ¼ small; r Z 0.3 ¼ medium; and r Z 0.5 ¼ large/strong.23 To assess the potential validity of the LDAT in standardizing evaluations and predicting evaluation outcome, analysis of variance of LDAT scores was conducted across the outcome categories traditionally employed in the transplant program (declined for psychosocial reasons, cleared but high risk, cleared with moderate risk, or cleared with low risk) with Tukey’s honest significant difference follow-up tests planned to reveal significant differences between groups. When participants had 2 LDAT scores assigned during the evaluation, the mean of the 2 LDAT scores was used for the prospective analyses. To assess the associations between LDAT scores and post-donation psychosocial outcomes, t-tests of LDAT scores between groups (present vs absent) were conducted on each psychosocial outcome variable, and Pearson correlation coefficient was calculated as a measure of the association between the overall psychosocial outcome score and LDAT score. RESULTS
Data Analytic Strategy The total score on the LDAT, which could range from 0–82, is calculated as the sum of the individual item scores. Higher scores indicate more desirable predonation psychosocial characteristics, and lower scores indicate less desirable characteristics or increased risk for donation. A total of 123 candidates were evaluated independently by 2 raters and administered 2 LDATs; 99 were evaluated once with 1 resultant LDAT score. This resulted in completion of 345 LDATs. The reliability of the LDAT was assessed in 2 ways. Internal consistency of the LDAT, or the degree to which the various items tap into the same or 522
www.psychosomaticsjournal.org
LDAT scores assigned by raters ranged from 42–79. When a participant was assigned 2 LDAT scores during the evaluation, the mean was used for the subsequent prospective analyses. LDAT scores used in prospective analyses ranged from 44–78 with a mean of 65.95 (standard deviation [SD] ¼ 7.08); mean LDAT for kidney donor candidates (65.72, SD ¼ 7.48) was not significantly different from liver donor candidates (66.71, SD ¼ 5.52), t(220)¼ 0.871, p ¼ 0.385. As a measure of reliability, the internal consistency (Cronbach alpha) of LDAT items across all 345 LDAT ratings was calculated at α ¼ 0.799. Inter-rater reliability was assessed by calculating Pearson Psychosomatics 58:5, September/October 2017
Iacoviello et al. FIGURE 1.
Scatterplot of LDAT Scores Between Raters. A Total of 123 Participants Were Administered 2 LDATs in Independent Evaluations. Pearson Correlation Coefficient Was Calculated as r ¼ 0.639, p o 0.001, Indicating “Strong” Inter-rater Reliability.
correlation coefficient for the 123 participants who were administered 2 LDATs in independent evaluations: r ¼ 0.639, p o 0.001. Figure 1 provides the scatterplot of LDAT scores across the 2 raters, depicting the strong correlation between LDAT scores. As an index of validity, LDAT scores were analyzed for associations with the risk status determined during the psychosocial evaluation (declined for psychosocial reasons, cleared but high risk, cleared moderate risk, and cleared low risk). Furthermore, 10 participants were declined, with a mean LDAT ¼ 52.3 (SD ¼ 7.1); 3 participants were cleared but high risk, with a mean LDAT ¼ 55.2 (SD ¼ 9.8); 100 participants were cleared with moderate risk, with a mean LDAT ¼ 63.3 (SD ¼ 6.0); and 109 participants were cleared with low risk, with a mean LDAT ¼ 69.6 (SD ¼ 4.9). Analysis of variance of LDAT scores revealed significant differences across groups: F(3,218) ¼ 44.69, p o 0.001. Tukey’s honest significant difference follow-up test revealed that the declined and highrisk groups did not differ significantly on LDAT score (p ¼ 0.865), whereas all other group comparisons were significantly different at p o 0.05. Figure 2 shows LDAT scores across the evaluation outcome groups. Follow-up analysis was conducted to determine the best cutoff score to sensitively and specifically discriminate donors determined to be suitable for donation. A cutoff score of 59 was observed to correctly Psychosomatics 58:5, September/October 2017
categorize 181 of 209 (86.6%) in the low- or moderaterisk groups and correctly categorize 11 of 13 (84.6%) in the declined or high-risk groups. The occurrence of impairment post-donation was too infrequent, and the LDAT scores for those donors experiencing impairment were not variable enough to determine a cutoff score to discriminate those at risk for post-donation psychosocial impairment. Validity was also assessed by investigating associations of LDAT scores with post-donation psychosocial outcomes. Of the 222 evaluations, there were 86 donations completed with at least a 6-week outcome time-point. Forty-four had 6-week and 6-month outcomes. Sixty-five donated a kidney and 21 donated liver. Mean LDAT of eventual donors was 67.5 (SD ¼ 5.5); 67.2 for kidney donors (SD ¼ 6.0) and 68.5 for liver donors (SD ¼ 3.5). There was no significant LDAT difference between eventual kidney and liver donors (t¼ 1.206, p ¼ 0.233). For the purpose of the study, treatment adherence was defined as complying with the center’s requirements for live donor follow-up, which included a 6-week visit with the nurse practitioner and dietitian and a 6-month appointment with the nurse
FIGURE 2.
LDAT Scores Across Evaluation Outcome Groups. ANOVA of LDAT Scores Revealed Significant Differences Across Groups: F(3218) ¼ 44.69, p o 0.001. Tukey's HSD Follow-Up Tests Revealed Significant Differences Between All Groups at p o 0.05, Except Between the Declined and High-Risk Groups That Did Not Differ Significantly (p ¼ 0.865). ANOVA ¼ analysis of variance, HSD ¼ honest significant difference.
www.psychosomaticsjournal.org
523
Prospective study of the LDAT practitioner. For donors who lived a distance from the transplant center or had work/family conflicts, they were permitted to perform the follow-up with their local physician. Seven of 86 participants were rated as treatment nonadherent post-donation, and LDAT scores significantly differed between groups, t(84) ¼ 2.812, p ¼ 0.006. None of the other psychosocial outcomes revealed significant differences in LDAT score. Nine of 86 participants were rated as experiencing work impairment post-donation, and LDAT scores did not significantly differ between groups, t(84) ¼ 0.027, p ¼ 0.787. Three of 86 participants were rated as experiencing relationship impairment postdonation, and LDAT scores did not significantly differ between groups, t(84) ¼ 0.595, p ¼ 0.554. Sixteen of 86 participants were rated as experiencing change in psychologic status post-donation, which included mild anxiety/depression/insomnia and were classified as self-limiting, requiring psychological counseling/medication. No severe psychologic complications were seen in the group. The LDAT scores did not significantly differ between groups, t(84) ¼ 1.926, p ¼ 0.058. Seventeen of 86 participants were rated as requiring nonstandard psychosocial interventions meaning that they required additional social work or psychiatry visits, and LDAT scores did not significantly differ between groups, t(84) ¼ 1.109, p ¼ 0.271. Eight of 86 participants were rated as experiencing excessive pain post-donation (requesting more than prescribed discharge pain medication), and LDAT scores did not significantly differ between groups, t(84) ¼ 1.583, p ¼ 0.117. Overall psychosocial outcome score ranged from 0–5 in this sample, with 51 participants scored “0,” 18 participants scored “1,” 6 participants scored “2,” 8 participants scored “3,” 2 participants scored “4,” and 1 participant scored “5.” Overall psychosocial outcome score mean equaled 0.7558 (SD ¼ 1.14). Kidney donors demonstrated a mean score of 0.7385 (SD ¼ 1.13), and liver donors demonstrated a mean score of 0.8095 (SD ¼ 1.21). Kidney and liver donors were not significantly different in outcome variable score (t(84) ¼ 0.245, p ¼ 0.807). There was no evidence of a significant correlation between overall psychosocial outcome and LDAT: r¼ 0.017, p ¼ 0.880. Comparison of LDAT scores between those with no psychosocial impairment (outcome ¼ 0) vs any impairment score also revealed no significant group difference. 524
www.psychosomaticsjournal.org
In the 44 participants with 6-week and 6-month outcome scores, a paired t-test indicated that psychosocial outcomes tended to improve, from 6 weeks (mean ¼ 0.750, SD ¼ 1.14) to 6 months (mean ¼ 0.386, SD ¼ 0.722), t(43)¼ 2.110, p ¼ 0.041. DISCUSSION The purpose of this study was to prospectively investigate the reliability and validity of the Live Donor Assessment Tool (LDAT) as a semistructured psychosocial assessment in a sample of potential live organ donors. The LDAT was developed to address the psychosocial factors that are associated, in the literature and through clinical experience of experts in the live donor field, with the live organ donation process, donation and transplant recipient outcomes, and resilience. The scale provides a score that can be used to quantify the level of psychosocial risk vs appropriateness of a live donor candidate. Despite demonstrating indicators of reliability and validity in a retrospective, medical record-review study,20 the need for a reliable and valid tool for the psychosocial assessment of live organ donors that can assist in the prediction of post-donation outcomes warranted prospective study of the LDAT. Reliability was assessed by investigating internal consistency of the LDAT as well as inter-rater reliability of the LDAT scores obtained by 2 independent raters. Validity was assessed by investigating associations between LDAT scores obtained during the evaluation and the traditional evaluation outcome categories as well as psychosocial outcomes post-donation. This study found that LDAT items demonstrate “good” internal consistency (α ¼ 0.799), closely replicating the value obtained in the retrospective study of α ¼ 0.795. The items appear to measure related facets of a construct (in this case the psychosocial appropriateness of a donor candidate) but are not overly redundant as an “excellent” degree of internal consistency (α Z 0.9) might indicate. In addition, the LDAT was reliably scored across trained raters. This study found the LDAT to demonstrate “strong” inter-rater reliability in 123 cases that were independently rated by 2 evaluators (Figure 1). When completed after a traditional psychosocial evaluation, the LDAT also exhibited signs of validity. LDAT scores differentiated the outcome categories traditionally resulting from the psychosocial evaluation. Psychosomatics 58:5, September/October 2017
Iacoviello et al. Specifically, the declined and high-risk donors did not show significantly different LDAT scores but were significantly lower in LDAT score compared to the moderate- and low-risk groups, which were also significantly different from one another in LDAT score. Moreover, the LDAT scores increased from high- to moderate- to low-risk groups as would be predicted (Figure 2). Follow-up analysis revealed a cutoff score of 59 maximally discriminated between “positive evaluation outcomes” (low- or moderaterisk assignment; correctly classifying 86.6%) vs “negative evaluation outcome” (declined or assignment to the high-risk group; correctly classifying 84.6%). It is noteworthy that this cutoff score and its ability to discriminate between evaluation outcome groups is similar to that observed in the retrospective LDAT study,20 in which a score of 60 correctly classified 88% of the positive outcome group and 83% of the negative outcome group. Lastly, LDAT scores were significantly associated with 1 indicator of psychosocial outcome: treatment adherence. Donors who were not treatment adherent at 6 weeks post-donation showed significantly lower LDAT scores during the evaluation compared to donors who were treatment adherent. There was also a trend toward LDAT scores differentiating those who experienced a change in psychologic status. The other indicators (work impairment, relationship impairment, excessive pain, and the need for additional psychologic services post-donation) were not associated with LDAT scores. As in most prospective transplant studies, but perhaps especially with living donors, screening processes in place for the medical and psychosocial health of candidates likely biases away from significant post-surgical impairment. Thus, detection of signals for significant associations with post-donation impairment will be more difficult and may require larger sample sizes than in the current study. These indicators may also very well be affected by multiple factors beyond those that are measured by the LDAT. Nonetheless, the significant association of LDAT scores with treatment adherence suggests value of the LDAT not just during the psychosocial evaluation process but also in predicting a specific and important post-donation psychosocial outcome. Another interesting and important finding was that the psychosocial outcomes measured in this study tended to improve over time, from the 6-week to the 6-month time points. Coupled with the relatively small Psychosomatics 58:5, September/October 2017
number of cases that experienced negative psychosocial outcomes during follow-up, this reinforces the relatively safe nature of these procedures as regards psychosocial outcomes and suggests that even those who have difficulties immediately after surgery can be expected to improve over time. This study has some notable strengths. The LDAT was developed by a team with vast experience in the live organ donation process and psychosocial research, and in consultation with experts around the world. As a result, the LDAT covers a range of psychosocial factors related to resilience in general as well as the process of live organ donation. This investigation of the LDAT’s reliability and validity included a diverse sample in age, sex, and both liver and kidney donor candidates, increasing the generalizability of the results to the larger live organ donor population. However, there are also limitations of the study. The fact that the tool was developed and tested in the same center limits the interpretation of the findings. The LDAT items could have been conceived of and developed in a way biased toward how the evaluations are conducted at this center, artificially inflating measures of inter-rater reliability and the congruence between LDAT scores and the outcomes of the psychosocial evaluations conducted. Another limitation of the current study involves the limited occurrence of post-donation impairment, resulting in difficulty detecting a signal for associations with post-donation impairment. It is possible that, during the evaluation process, donor “weaknesses” are revealed and may begin to remediate through the evaluation process and discussion with clinicians. Together, these results, strengths, and limitations suggest that further research on the LDAT is warranted but should include multiple transplant centers beyond the one where the LDAT was developed and initially tested, as well as including large enough samples to evaluate associations with relatively rare impairment outcomes. In summary, the LDAT was developed to address the need for a standardized approach to the psychosocial evaluation for live organ donors. In this prospective study, the LDAT tool demonstrated good psychometric properties (reliability and validity) as well as utility in predicting post-donation treatment adherence. The LDAT affords transplant teams a common language to use when discussing live donor psychosocial suitability and helps to encourage a standardized, unbiased process for determining www.psychosomaticsjournal.org
525
Prospective study of the LDAT candidacy. It may also be used to facilitate communication with providers at other centers when transferring patients or performing kidney paired donation. A tool like the LDAT should be completed after a standard psychosocial interview for living donor candidacy and used as a guideline to train new evaluators. It can enable the clinicians to determine areas for intervention and to address areas of concern before donation, which could ultimately improve the donation experience. Future research on the properties of the LDAT and the ability to implement and use the
LDAT prospectively at additional transplant centers is warranted. Disclosure: The authors acknowledge partial financial support for this project in the form of an American Society of Transplantation TIRN/Wood MacMillan Charitable Fund Transplant Nursing Research Grant to DLR. The funding agency did not play a role in the study design, data collection, analysis or interpretation, writing of the report, or decision to submit for publication.
References 1. Ibrahim HN, Foley MB, Liping tan, et al: Long-term consequences of kidney donation. N Engl J Med 2009; 360 (5):459–469 2. Segev DL, Muzzaale AD, Mehta SH, et al: Perioperative mortality and long term survival following live kidney donation. J Am Med Assoc 2010; 303(10):959–966 3. Lentine KL, Schnitzler MA, Huiling X, et al: Racial variation in medical outcomes among living kidney donors. N Engl J Med 2010; 363(8):724–732 4. Olthoff KM, Merion RM, Ghobrial RM, A2ALL Study Group. Outcomes of 385 adult-to-adult living donor liver transplant recipients: a report from the A2ALL Consortium. Ann Surg 2005; 242(3):314–323 5. Abecassis MM, Fisher RA, Olthoff KM, et al: Complications of living donor hepatic lobectomy—a comprehensive report. Am J Transplant 2012; 12(5):1208–1217 6. LaPointe Rudow D, Charlton M, Sanchez C, et al: Kidney and liver living donors: a comparison of experiences. Prog Transplant 2005; 15(2):185–191 7. Kim-Schluger L, Florman SS, Schiano T, et al: Quality of life after lobectomy for adult liver transplantation. Transplantation 2002; 73(10):1593–1597 8. Clemens KK, Thiessen-Philbrook H, Parikh CR, et al: Psychosocial health of living kidney donors: a systematic review. Am J Transplant 2006; 6(12):2965–2977 9. LaPointe Rudow D, Iacoviello BM, Charney DS: Resilience and personality traits among live liver and kidney donors. Prog Transplant 2014; 24(1):82–90 10. Trotter JF, Hill-Callahan M, Gillespie BW, et al: Deaths due to psychiatric complications in right hepatic lobe donors for adult to adult living donor liver transplantation. Hepatology 2005; 42(suppl 1):451A 11. Jacobs CL, Gross CR, Messersmith EE, et al: Emotional and financial experiences of kidney donors over the past 50 years: the RELIVE study. Clin J Am Soc Nephrol 2015 Dec 7; 10(12):2221–2231 12. Kita Y, Fukunishi I, Harihara M, et al: Psychiatric disorders in living related liver transplantation. Transplant Proc 2001; 33(1-2):1350–1352
526
www.psychosomaticsjournal.org
13. Iacoviello BM, Charney DS: Psychosocial facets of resilience: implications for preventing post-trauma psychopathology, treating trauma survivors and enhancing community resilience. Eur J Psychotraumatol 2014; 5:23970 Available from: https://doi.org/10.3402/ejpt.v5.23970 14. Simpson MA, Pomfret EA: Searching for the optimal living liver donor psychosocial evaluation. Am J Transplant 2012; 12(1):7–8 15. Maldonado JR, Dubois HC, David EE, et al: The Stanford Integrated Psychosocial Assessment for Transplantation (SIPAT): a new tool for the psychosocial evaluation of pre-transplant candidates. Psychosomatics 2012; 53(2): 123–132 16. Olbrisch M, Levenson J, Hamer R: The PACT: a rating scale for the study of clinical decision making in psychosocial screening of organ transplant candidates. Clin Transplant 1989; 3(3):164–169 17. Twillman RK, Manetto C, Wellisch DK, Wolcott DL: The Transplant Evaluation Rating Scale: a revision of the psychosocial levels system for evaluation organ transplant candidates. Psychosomatics 1993; 34(2):144–153 18. Mori DL, Gallagher P, Milne J: The Structured Interview For Renal Transplantation—SIRT. Psychosomatics 2000; 41(5):393–405 19. Maldonado Sher Y, Lolak S, Swendsen H, et al: The Stanford Integrated Psychosocial Assessment for Transplantation: a prospective study of medical and psychosocial outcomes. Psychosom Med 2015; 77(9):1018–1030 20. Iacoviello BM, Shenoy AS, Braoude J, et al: The Live Donor Assessment Tool: a psychosocial assessment tool for live organ donors. Psychosomatics 2015; 56(3):254–261 21. Cronbach LJ: Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16(3):297–334 22. George D, Mallery P: [11.0 update]. SPSS for Windows Step by Step: A Simple Guide and Reference. 4th ed. Boston: Allyn & Bacon; 2003 23. Cohen, Jacob: Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Mahwarh, NJ: Lawrence Erlbaum Associates; 1988
Psychosomatics 58:5, September/October 2017
& 2017 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.
Original Research Report A Prospective Study of the Reliability and Validity of the Live Donor Assessment Tool Brian M. Iacoviello, PhD., Akhil Shenoy, M.D., Julia Hunt, LMSW., Zorica Filipovic-Jewell, M.D., Brandy Haydel, B.S., Dianne LaPointe Rudow, D.N.P.
Background: The psychosocial evaluation is an important part of the live organ donor evaluation process, yet this is not standardized across institutions. Objective: This study was designed to prospectively test the reliability and validity of a semistructured psychosocial evaluation tool that was recently developed and reported in the literature (the Live Donor Assessment Tool [LDAT]). Methods: A total of 248 live donor candidates who presented for evaluation were invited to participate in a study that involved the LDAT being scored as part of the standard psychosocial evaluation process; 222 provided informed consent. Evaluations were conducted by staff experienced with psychosocial evaluation of living donors and trained in the use of the LDAT. Furthermore, 123 donor candidates were evaluated twice, as per routine standard of care, and had 2 LDATs administered. Reliability of the LDAT was
assessed by calculating the internal consistency of the LDAT items and inter-rater reliability. Validity was assessed by comparing LDAT scores across the riskgroup categories (the traditional outcome designation of the psychosocial evaluation) and in 86 eventual donors, associations between LDAT scores, and indicators of psychosocial outcomes post-donation. Results: The LDAT was found to have good internal consistency, strong inter-rater reliability, and showed signs of validity: LDAT scores differentiated the traditional risk-group categories, and a significant association between LDAT score and treatment adherence postdonation was revealed. Conclusions: The LDAT demonstrated good reliability and validity, but future research on the LDAT and the ability to implement the LDAT across institutions is warranted. (Psychosomatics 2017; 58:519–526)
Key words: Live donors, Organ, Psychosocial, Assessment, Donation. INTRODUCTION Live organ donors are a vital source of donor organs for transplantation. Live kidney donation for patients with chronic kidney failure has been performed for over 5 decades. Kidney donation has been shown to be safe, with kidney donors exhibiting minimal risk of end-stage renal disease provided they lead a healthy lifestyle.1–3 Live liver donation has been performed in adults since 1998, most often at large centers in geographical areas with deceased donor shortages. Psychosomatics 58:5, September/October 2017
Studies of medical outcomes have shown that although liver donation portends a higher risk than Received August 30, 2016; revised March 13, 2017; accepted March 14, 2017. From the Department of Psychiatry (BMI, AS, ZFJ), Icahn School of Medicine at Mount Sinai; Recanati-Miller Transplantation Institute at Mount Sinai (AS, JH, ZFJ, BH, DLR), The Zweig Family Center for Living Donation, New York, NY. Send correspondence and reprint requests to Brian M. Iacoviello, PhD, Department of Psychiatry, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1230, New York, NY 10029; e-mail: [email protected] & 2017 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.
www.psychosomaticsjournal.org
519
Prospective study of the LDAT kidney donation, it is still safe enough to perform at experienced centers.4,5 Psychologic outcomes for liver and kidney donors have also been investigated. These studies have shown that donors often report a better quality of life compared to age-matched healthy adults, and most donors indicate that they would donate again.6–8 Studies have also demonstrated that live donors exhibit adaptive personality traits (agreeableness, conscientiousness, and lower neuroticism) to a greater degree compared than age-matched healthy adults who are equally as resilient as the general population, are significantly more resilient than primary care patients, and who score very highly on measures of Purpose in Life and Growth after donation.9 However, there are also reports of less-frequent negative outcomes associated with live organ donation, including suicides and serious psychiatric complications,10 often associated with predonation psychiatric histories. Perhaps transplant studies are biased toward favorable outcome results by the inclusion of patients who make it through the selection process. That being said, despite the current screening process, some donors report negative psychosocial outcomes, including depression, anxiety, and regret.8 Further, 9% (n ¼ 231) of kidney donors in the RELIVE study reported one or more of the following poor psychosocial outcomes: fair or poor overall donor experience, financial burden, regret or discomfort with decision to donate, or psychologic difficulties since donation.11 Many donors require counseling in the year after donation,12 with a smaller number needing pharmacotherapy for depression or anxiety. Organ donation can be a stressful experience for the donors and their family. Identifying a strong social support system is important for resilient outcomes.13 Taken together, consideration of the psychosocial risk factors, in addition to the medical risk factors, is critical for prediction of patients who will have positive vs negative donation experiences and outcomes, despite screening processes that may already bias studies toward favorable results. The psychosocial evaluation of live organ donors has traditionally relied on a clinical interview to investigate the presence or absence of predonation psychosocial stressors and underlying psychological disorders. However, the psychosocial evaluation of live donors has not yet been standardized and can vary widely across providers and institutions,14 and there are no validated measures available to determine 520
www.psychosomaticsjournal.org
psychosocial risks for donation. There are several tools used to assess organ recipients: the Stanford Integrated Psychosocial Assessment for Transplantation,15 the Psychosocial Assessment of Candidates for Transplantation,16 the Transplant Evaluation Rating Scale,17 and the Structured Interview for Renal Transplantation.18 The Stanford Integrated Psychosocial Assessment for Transplantation is the newest of these tools; its strengths include its comprehensive nature and standardization of the information collected during the psychosocial evaluation process. Although it did not predict the primary outcomes of graft failure and mortality, the Stanford Integrated Psychosocial Assessment for Transplantation was shown to have excellent inter-rater reliability and to be predictive of the transplant recipient’s rejection episodes, rehospitalization rate, psychiatric decompensation, and social support failure.19 This type of assessment tool is needed for the psychosocial assessment of live organ donors. One semistructured psychosocial assessment tool has been developed and reported in the literature: the Live Donor Assessment Tool (LDAT20). The LDAT is a tool for psychiatrists, psychologists, social workers, and other clinicians involved in the evaluation process to quantify the degree of psychosocial appropriateness vs riskiness of potential donors. Comprising 29 items scored 0–2 or 0–3 across 9 domains (motivations for donation, knowledge about donation process, relationship with the recipient, support available to the donor, donor’s feelings about donation, post-donation expectations, stability in life, psychiatric history, and alcohol and substance use), the LDAT is scored such that higher scores indicate greater psychosocial appropriateness for donation. A retrospective chart-review study using the LDAT revealed preliminary indications of very good inter-rater reliability between 2 independent raters, and validity in LDAT scores being consistent with the risk category designation resulting from the traditional psychosocial evaluation.20 The purpose of the project reported in this article was to prospectively test the reliability and validity of the LDAT as a semistructured psychosocial assessment tool in a sample of live organ donor candidates presenting for evaluation at a large, urban medical center. Reliability was assessed by investigating inter-rater reliability of the LDAT scores obtained by 2 independent raters; validity was assessed by investigating associations between the LDAT score Psychosomatics 58:5, September/October 2017
Iacoviello et al. obtained during the evaluation and the traditional evaluation outcome categories as well as psychosocial outcomes postdonation. METHODS Participants Between January 1, 2015 and October 31, 2015, 248 live donor candidates presented for evaluation; 222 provided informed consent and were included in the study protocol. All potential live donors who underwent a live donor psychosocial evaluation using the LDAT were eligible for the study. Participants must have been English-speaking live organ donor candidates who presented for the donor evaluation and saw the transplant coordinator, physician, and completed a psychosocial evaluation. Liver donors who sought to donate for the reason of acute liver failure in an emergent situation were excluded. During the psychosocial evaluation, all candidates were evaluated for psychosocial suitability by an experienced clinician according to the standard practice in the transplant program. An LDAT was completed by the clinician after each evaluation. All liver donors were evaluated by both a social worker and a psychiatrist. Kidney donors saw only a social worker unless they were determined to be at increased risk for psychosocial complications, had a history of psychiatric illness, were distantly or unknown to the recipient, or were participating in a paired kidney donation. Of the 222 candidates, 123 evaluated were administered 2 evaluations. Of the 222 candidates who were evaluated, 86 eventually donated an organ. The Table provides descriptive characteristics of the donor candidates evaluated and the eventual donors. Evaluations were conducted by licensed clinicians (psychiatrists [A.S. or Z.F.J.] and a social worker [J.H.]) employed in the transplant program. Clinicians all had several years of experience conducting psychosocial evaluations of live donor candidates and were familiar with the development and study of the LDAT. Some clinicians (A.S. and J.H.) had previously been responsible for training LDAT raters for a retrospective, chart-review study of the LDAT, and thus had excellent understanding of the LDAT and its use. Owing to limited resources, one of the clinicians involved in the development of the LDAT (A.S.) was also involved in the evaluation of donor candidates in Psychosomatics 58:5, September/October 2017
TABLE.
Characteristics of the Sample of Donor Candidates and Eventual Donors
Age (y) Sex (% female) Ethnicity (%) White African American/ Black Asian Other LDAT
Age (y) Sex (% female) Ethnicity (%) White African American/ Black Asian Other LDAT
Kidney donor candidates (n ¼ 171)
Liver donor candidates (n ¼ 51)
42.4 (12.7) 55.6
37.0 (9.3) 49.0
71.3 16.4
66.7 13.7
7.0 5.3
13.7 5.9
65.72 (7.48)
66.71 (5.52)
Kidney donors (n ¼ 65)
Liver donors (n ¼ 21)
42.4 (12.0) 60.0
36.7 (9.6) 47.6
81.5 12.3
57.1 19.0
1.5 4.6
9.5 14.3
67.2 (6.0)
68.5 (3.5)
Note: Values shown indicate mean (SD) or %.
the current study. Other clinicians involved in the development of the LDAT were excluded from evaluating donors in the current study. Procedures All live donor candidates who presented for evaluation at a large, urban medical center were invited to participate in the study. Those who were interested were provided information about the study purpose and procedures and were explicitly informed that study participation would not affect their evaluation process nor would it influence their clearance as a donor. They were further informed that the LDAT score was not calculated by the clinician during the psychosocial evaluation, rather it was entered into the REDCap database and scored by the research assistant. All interested participants provided written informed consent. The outcome of the psychosocial evaluation process was categorized www.psychosomaticsjournal.org
521
Prospective study of the LDAT using the subjective categories traditionally employed to describe the evaluation outcome (declined for psychosocial reasons, cleared but high risk, cleared with moderate risk, or cleared with low risk). For all study participants, an LDAT was completed, and an LDAT score was obtained after the traditional psychosocial evaluations were conducted. Possible LDAT scores ranged from 0–82, and although they were not available to the selection committee members at the time of the committee meetings, the clinicians who conducted the psychosocial evaluations and completed LDATs were involved in the discussions on decision-making. Data were entered directly into an electronic data capture system (REDCap) by raters after the evaluation, which resulted in zero missing data. For the 86 candidates who completed live organ donation, psychosocial outcomes were assessed at 2 time points: 6 weeks and 6 months post-donation. Outcome assessments were conducted by a doctorallevel nurse practitioner with many years experience working with live organ donors (D.L.R.), but who was not involved with the psychosocial evaluation or the scoring of the LDAT. The following 6 psychosocial outcomes were assessed as present/absent based on the entire live donor population: work impairment, relationship impairment, change in psychological status, nonstandard psychosocial interventions required, excessive pain, and treatment adherence. A total psychosocial outcome score was also computed as the sum of the factors above rated as present and ranged from 0–6 with greater scores indicating increased impairment post-donation.
related constructs, was measured by calculating Cronbach alpha21 across all 345 ratings, with the value interpreted according to established ranges (e.g., α o 0.5 ¼ “unacceptable”; 0.5 r α o 0.6 ¼ “ poor”; 0.6 r α o 0.7 ¼ “questionable”; 0.7 r α o 0.8 ¼ “acceptable”; 0.8 r α o 0.9 ¼ “good”; and α Z 0.9 ¼ “excellent”22). Inter-rater reliability was measured by computing Pearson correlation coefficients for the LDAT scores when 2 evaluations were completed for a donor candidate, resulting in 123 LDAT pairs compared. Values were interpreted according to Cohen (1988): r Z 0.1 ¼ small; r Z 0.3 ¼ medium; and r Z 0.5 ¼ large/strong.23 To assess the potential validity of the LDAT in standardizing evaluations and predicting evaluation outcome, analysis of variance of LDAT scores was conducted across the outcome categories traditionally employed in the transplant program (declined for psychosocial reasons, cleared but high risk, cleared with moderate risk, or cleared with low risk) with Tukey’s honest significant difference follow-up tests planned to reveal significant differences between groups. When participants had 2 LDAT scores assigned during the evaluation, the mean of the 2 LDAT scores was used for the prospective analyses. To assess the associations between LDAT scores and post-donation psychosocial outcomes, t-tests of LDAT scores between groups (present vs absent) were conducted on each psychosocial outcome variable, and Pearson correlation coefficient was calculated as a measure of the association between the overall psychosocial outcome score and LDAT score. RESULTS
Data Analytic Strategy The total score on the LDAT, which could range from 0–82, is calculated as the sum of the individual item scores. Higher scores indicate more desirable predonation psychosocial characteristics, and lower scores indicate less desirable characteristics or increased risk for donation. A total of 123 candidates were evaluated independently by 2 raters and administered 2 LDATs; 99 were evaluated once with 1 resultant LDAT score. This resulted in completion of 345 LDATs. The reliability of the LDAT was assessed in 2 ways. Internal consistency of the LDAT, or the degree to which the various items tap into the same or 522
www.psychosomaticsjournal.org
LDAT scores assigned by raters ranged from 42–79. When a participant was assigned 2 LDAT scores during the evaluation, the mean was used for the subsequent prospective analyses. LDAT scores used in prospective analyses ranged from 44–78 with a mean of 65.95 (standard deviation [SD] ¼ 7.08); mean LDAT for kidney donor candidates (65.72, SD ¼ 7.48) was not significantly different from liver donor candidates (66.71, SD ¼ 5.52), t(220)¼ 0.871, p ¼ 0.385. As a measure of reliability, the internal consistency (Cronbach alpha) of LDAT items across all 345 LDAT ratings was calculated at α ¼ 0.799. Inter-rater reliability was assessed by calculating Pearson Psychosomatics 58:5, September/October 2017
Iacoviello et al. FIGURE 1.
Scatterplot of LDAT Scores Between Raters. A Total of 123 Participants Were Administered 2 LDATs in Independent Evaluations. Pearson Correlation Coefficient Was Calculated as r ¼ 0.639, p o 0.001, Indicating “Strong” Inter-rater Reliability.
correlation coefficient for the 123 participants who were administered 2 LDATs in independent evaluations: r ¼ 0.639, p o 0.001. Figure 1 provides the scatterplot of LDAT scores across the 2 raters, depicting the strong correlation between LDAT scores. As an index of validity, LDAT scores were analyzed for associations with the risk status determined during the psychosocial evaluation (declined for psychosocial reasons, cleared but high risk, cleared moderate risk, and cleared low risk). Furthermore, 10 participants were declined, with a mean LDAT ¼ 52.3 (SD ¼ 7.1); 3 participants were cleared but high risk, with a mean LDAT ¼ 55.2 (SD ¼ 9.8); 100 participants were cleared with moderate risk, with a mean LDAT ¼ 63.3 (SD ¼ 6.0); and 109 participants were cleared with low risk, with a mean LDAT ¼ 69.6 (SD ¼ 4.9). Analysis of variance of LDAT scores revealed significant differences across groups: F(3,218) ¼ 44.69, p o 0.001. Tukey’s honest significant difference follow-up test revealed that the declined and highrisk groups did not differ significantly on LDAT score (p ¼ 0.865), whereas all other group comparisons were significantly different at p o 0.05. Figure 2 shows LDAT scores across the evaluation outcome groups. Follow-up analysis was conducted to determine the best cutoff score to sensitively and specifically discriminate donors determined to be suitable for donation. A cutoff score of 59 was observed to correctly Psychosomatics 58:5, September/October 2017
categorize 181 of 209 (86.6%) in the low- or moderaterisk groups and correctly categorize 11 of 13 (84.6%) in the declined or high-risk groups. The occurrence of impairment post-donation was too infrequent, and the LDAT scores for those donors experiencing impairment were not variable enough to determine a cutoff score to discriminate those at risk for post-donation psychosocial impairment. Validity was also assessed by investigating associations of LDAT scores with post-donation psychosocial outcomes. Of the 222 evaluations, there were 86 donations completed with at least a 6-week outcome time-point. Forty-four had 6-week and 6-month outcomes. Sixty-five donated a kidney and 21 donated liver. Mean LDAT of eventual donors was 67.5 (SD ¼ 5.5); 67.2 for kidney donors (SD ¼ 6.0) and 68.5 for liver donors (SD ¼ 3.5). There was no significant LDAT difference between eventual kidney and liver donors (t¼ 1.206, p ¼ 0.233). For the purpose of the study, treatment adherence was defined as complying with the center’s requirements for live donor follow-up, which included a 6-week visit with the nurse practitioner and dietitian and a 6-month appointment with the nurse
FIGURE 2.
LDAT Scores Across Evaluation Outcome Groups. ANOVA of LDAT Scores Revealed Significant Differences Across Groups: F(3218) ¼ 44.69, p o 0.001. Tukey's HSD Follow-Up Tests Revealed Significant Differences Between All Groups at p o 0.05, Except Between the Declined and High-Risk Groups That Did Not Differ Significantly (p ¼ 0.865). ANOVA ¼ analysis of variance, HSD ¼ honest significant difference.
www.psychosomaticsjournal.org
523
Prospective study of the LDAT practitioner. For donors who lived a distance from the transplant center or had work/family conflicts, they were permitted to perform the follow-up with their local physician. Seven of 86 participants were rated as treatment nonadherent post-donation, and LDAT scores significantly differed between groups, t(84) ¼ 2.812, p ¼ 0.006. None of the other psychosocial outcomes revealed significant differences in LDAT score. Nine of 86 participants were rated as experiencing work impairment post-donation, and LDAT scores did not significantly differ between groups, t(84) ¼ 0.027, p ¼ 0.787. Three of 86 participants were rated as experiencing relationship impairment postdonation, and LDAT scores did not significantly differ between groups, t(84) ¼ 0.595, p ¼ 0.554. Sixteen of 86 participants were rated as experiencing change in psychologic status post-donation, which included mild anxiety/depression/insomnia and were classified as self-limiting, requiring psychological counseling/medication. No severe psychologic complications were seen in the group. The LDAT scores did not significantly differ between groups, t(84) ¼ 1.926, p ¼ 0.058. Seventeen of 86 participants were rated as requiring nonstandard psychosocial interventions meaning that they required additional social work or psychiatry visits, and LDAT scores did not significantly differ between groups, t(84) ¼ 1.109, p ¼ 0.271. Eight of 86 participants were rated as experiencing excessive pain post-donation (requesting more than prescribed discharge pain medication), and LDAT scores did not significantly differ between groups, t(84) ¼ 1.583, p ¼ 0.117. Overall psychosocial outcome score ranged from 0–5 in this sample, with 51 participants scored “0,” 18 participants scored “1,” 6 participants scored “2,” 8 participants scored “3,” 2 participants scored “4,” and 1 participant scored “5.” Overall psychosocial outcome score mean equaled 0.7558 (SD ¼ 1.14). Kidney donors demonstrated a mean score of 0.7385 (SD ¼ 1.13), and liver donors demonstrated a mean score of 0.8095 (SD ¼ 1.21). Kidney and liver donors were not significantly different in outcome variable score (t(84) ¼ 0.245, p ¼ 0.807). There was no evidence of a significant correlation between overall psychosocial outcome and LDAT: r¼ 0.017, p ¼ 0.880. Comparison of LDAT scores between those with no psychosocial impairment (outcome ¼ 0) vs any impairment score also revealed no significant group difference. 524
www.psychosomaticsjournal.org
In the 44 participants with 6-week and 6-month outcome scores, a paired t-test indicated that psychosocial outcomes tended to improve, from 6 weeks (mean ¼ 0.750, SD ¼ 1.14) to 6 months (mean ¼ 0.386, SD ¼ 0.722), t(43)¼ 2.110, p ¼ 0.041. DISCUSSION The purpose of this study was to prospectively investigate the reliability and validity of the Live Donor Assessment Tool (LDAT) as a semistructured psychosocial assessment in a sample of potential live organ donors. The LDAT was developed to address the psychosocial factors that are associated, in the literature and through clinical experience of experts in the live donor field, with the live organ donation process, donation and transplant recipient outcomes, and resilience. The scale provides a score that can be used to quantify the level of psychosocial risk vs appropriateness of a live donor candidate. Despite demonstrating indicators of reliability and validity in a retrospective, medical record-review study,20 the need for a reliable and valid tool for the psychosocial assessment of live organ donors that can assist in the prediction of post-donation outcomes warranted prospective study of the LDAT. Reliability was assessed by investigating internal consistency of the LDAT as well as inter-rater reliability of the LDAT scores obtained by 2 independent raters. Validity was assessed by investigating associations between LDAT scores obtained during the evaluation and the traditional evaluation outcome categories as well as psychosocial outcomes post-donation. This study found that LDAT items demonstrate “good” internal consistency (α ¼ 0.799), closely replicating the value obtained in the retrospective study of α ¼ 0.795. The items appear to measure related facets of a construct (in this case the psychosocial appropriateness of a donor candidate) but are not overly redundant as an “excellent” degree of internal consistency (α Z 0.9) might indicate. In addition, the LDAT was reliably scored across trained raters. This study found the LDAT to demonstrate “strong” inter-rater reliability in 123 cases that were independently rated by 2 evaluators (Figure 1). When completed after a traditional psychosocial evaluation, the LDAT also exhibited signs of validity. LDAT scores differentiated the outcome categories traditionally resulting from the psychosocial evaluation. Psychosomatics 58:5, September/October 2017
Iacoviello et al. Specifically, the declined and high-risk donors did not show significantly different LDAT scores but were significantly lower in LDAT score compared to the moderate- and low-risk groups, which were also significantly different from one another in LDAT score. Moreover, the LDAT scores increased from high- to moderate- to low-risk groups as would be predicted (Figure 2). Follow-up analysis revealed a cutoff score of 59 maximally discriminated between “positive evaluation outcomes” (low- or moderaterisk assignment; correctly classifying 86.6%) vs “negative evaluation outcome” (declined or assignment to the high-risk group; correctly classifying 84.6%). It is noteworthy that this cutoff score and its ability to discriminate between evaluation outcome groups is similar to that observed in the retrospective LDAT study,20 in which a score of 60 correctly classified 88% of the positive outcome group and 83% of the negative outcome group. Lastly, LDAT scores were significantly associated with 1 indicator of psychosocial outcome: treatment adherence. Donors who were not treatment adherent at 6 weeks post-donation showed significantly lower LDAT scores during the evaluation compared to donors who were treatment adherent. There was also a trend toward LDAT scores differentiating those who experienced a change in psychologic status. The other indicators (work impairment, relationship impairment, excessive pain, and the need for additional psychologic services post-donation) were not associated with LDAT scores. As in most prospective transplant studies, but perhaps especially with living donors, screening processes in place for the medical and psychosocial health of candidates likely biases away from significant post-surgical impairment. Thus, detection of signals for significant associations with post-donation impairment will be more difficult and may require larger sample sizes than in the current study. These indicators may also very well be affected by multiple factors beyond those that are measured by the LDAT. Nonetheless, the significant association of LDAT scores with treatment adherence suggests value of the LDAT not just during the psychosocial evaluation process but also in predicting a specific and important post-donation psychosocial outcome. Another interesting and important finding was that the psychosocial outcomes measured in this study tended to improve over time, from the 6-week to the 6-month time points. Coupled with the relatively small Psychosomatics 58:5, September/October 2017
number of cases that experienced negative psychosocial outcomes during follow-up, this reinforces the relatively safe nature of these procedures as regards psychosocial outcomes and suggests that even those who have difficulties immediately after surgery can be expected to improve over time. This study has some notable strengths. The LDAT was developed by a team with vast experience in the live organ donation process and psychosocial research, and in consultation with experts around the world. As a result, the LDAT covers a range of psychosocial factors related to resilience in general as well as the process of live organ donation. This investigation of the LDAT’s reliability and validity included a diverse sample in age, sex, and both liver and kidney donor candidates, increasing the generalizability of the results to the larger live organ donor population. However, there are also limitations of the study. The fact that the tool was developed and tested in the same center limits the interpretation of the findings. The LDAT items could have been conceived of and developed in a way biased toward how the evaluations are conducted at this center, artificially inflating measures of inter-rater reliability and the congruence between LDAT scores and the outcomes of the psychosocial evaluations conducted. Another limitation of the current study involves the limited occurrence of post-donation impairment, resulting in difficulty detecting a signal for associations with post-donation impairment. It is possible that, during the evaluation process, donor “weaknesses” are revealed and may begin to remediate through the evaluation process and discussion with clinicians. Together, these results, strengths, and limitations suggest that further research on the LDAT is warranted but should include multiple transplant centers beyond the one where the LDAT was developed and initially tested, as well as including large enough samples to evaluate associations with relatively rare impairment outcomes. In summary, the LDAT was developed to address the need for a standardized approach to the psychosocial evaluation for live organ donors. In this prospective study, the LDAT tool demonstrated good psychometric properties (reliability and validity) as well as utility in predicting post-donation treatment adherence. The LDAT affords transplant teams a common language to use when discussing live donor psychosocial suitability and helps to encourage a standardized, unbiased process for determining www.psychosomaticsjournal.org
525
Prospective study of the LDAT candidacy. It may also be used to facilitate communication with providers at other centers when transferring patients or performing kidney paired donation. A tool like the LDAT should be completed after a standard psychosocial interview for living donor candidacy and used as a guideline to train new evaluators. It can enable the clinicians to determine areas for intervention and to address areas of concern before donation, which could ultimately improve the donation experience. Future research on the properties of the LDAT and the ability to implement and use the
LDAT prospectively at additional transplant centers is warranted. Disclosure: The authors acknowledge partial financial support for this project in the form of an American Society of Transplantation TIRN/Wood MacMillan Charitable Fund Transplant Nursing Research Grant to DLR. The funding agency did not play a role in the study design, data collection, analysis or interpretation, writing of the report, or decision to submit for publication.
References 1. Ibrahim HN, Foley MB, Liping tan, et al: Long-term consequences of kidney donation. N Engl J Med 2009; 360 (5):459–469 2. Segev DL, Muzzaale AD, Mehta SH, et al: Perioperative mortality and long term survival following live kidney donation. J Am Med Assoc 2010; 303(10):959–966 3. Lentine KL, Schnitzler MA, Huiling X, et al: Racial variation in medical outcomes among living kidney donors. N Engl J Med 2010; 363(8):724–732 4. Olthoff KM, Merion RM, Ghobrial RM, A2ALL Study Group. Outcomes of 385 adult-to-adult living donor liver transplant recipients: a report from the A2ALL Consortium. Ann Surg 2005; 242(3):314–323 5. Abecassis MM, Fisher RA, Olthoff KM, et al: Complications of living donor hepatic lobectomy—a comprehensive report. Am J Transplant 2012; 12(5):1208–1217 6. LaPointe Rudow D, Charlton M, Sanchez C, et al: Kidney and liver living donors: a comparison of experiences. Prog Transplant 2005; 15(2):185–191 7. Kim-Schluger L, Florman SS, Schiano T, et al: Quality of life after lobectomy for adult liver transplantation. Transplantation 2002; 73(10):1593–1597 8. Clemens KK, Thiessen-Philbrook H, Parikh CR, et al: Psychosocial health of living kidney donors: a systematic review. Am J Transplant 2006; 6(12):2965–2977 9. LaPointe Rudow D, Iacoviello BM, Charney DS: Resilience and personality traits among live liver and kidney donors. Prog Transplant 2014; 24(1):82–90 10. Trotter JF, Hill-Callahan M, Gillespie BW, et al: Deaths due to psychiatric complications in right hepatic lobe donors for adult to adult living donor liver transplantation. Hepatology 2005; 42(suppl 1):451A 11. Jacobs CL, Gross CR, Messersmith EE, et al: Emotional and financial experiences of kidney donors over the past 50 years: the RELIVE study. Clin J Am Soc Nephrol 2015 Dec 7; 10(12):2221–2231 12. Kita Y, Fukunishi I, Harihara M, et al: Psychiatric disorders in living related liver transplantation. Transplant Proc 2001; 33(1-2):1350–1352
526
www.psychosomaticsjournal.org
13. Iacoviello BM, Charney DS: Psychosocial facets of resilience: implications for preventing post-trauma psychopathology, treating trauma survivors and enhancing community resilience. Eur J Psychotraumatol 2014; 5:23970 Available from: https://doi.org/10.3402/ejpt.v5.23970 14. Simpson MA, Pomfret EA: Searching for the optimal living liver donor psychosocial evaluation. Am J Transplant 2012; 12(1):7–8 15. Maldonado JR, Dubois HC, David EE, et al: The Stanford Integrated Psychosocial Assessment for Transplantation (SIPAT): a new tool for the psychosocial evaluation of pre-transplant candidates. Psychosomatics 2012; 53(2): 123–132 16. Olbrisch M, Levenson J, Hamer R: The PACT: a rating scale for the study of clinical decision making in psychosocial screening of organ transplant candidates. Clin Transplant 1989; 3(3):164–169 17. Twillman RK, Manetto C, Wellisch DK, Wolcott DL: The Transplant Evaluation Rating Scale: a revision of the psychosocial levels system for evaluation organ transplant candidates. Psychosomatics 1993; 34(2):144–153 18. Mori DL, Gallagher P, Milne J: The Structured Interview For Renal Transplantation—SIRT. Psychosomatics 2000; 41(5):393–405 19. Maldonado Sher Y, Lolak S, Swendsen H, et al: The Stanford Integrated Psychosocial Assessment for Transplantation: a prospective study of medical and psychosocial outcomes. Psychosom Med 2015; 77(9):1018–1030 20. Iacoviello BM, Shenoy AS, Braoude J, et al: The Live Donor Assessment Tool: a psychosocial assessment tool for live organ donors. Psychosomatics 2015; 56(3):254–261 21. Cronbach LJ: Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16(3):297–334 22. George D, Mallery P: [11.0 update]. SPSS for Windows Step by Step: A Simple Guide and Reference. 4th ed. Boston: Allyn & Bacon; 2003 23. Cohen, Jacob: Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Mahwarh, NJ: Lawrence Erlbaum Associates; 1988
Psychosomatics 58:5, September/October 2017