Self-Management and Transition Readiness Assessment: Concurrent, Predictive and Discriminant Validation of the STARx Questionnaire

Journal of Pediatric Nursing (2015) 30, 668–676

Self-Management and Transition Readiness Assessment: Concurrent, Predictive and Discriminant Validation of the STARx Questionnaire1 Sarah E. Cohen MS a , Stephen R. Hooper PhD a , Karina Javalkar BS b , Cara Haberman MD c,d , Nicole Fenton PhD a , Hsiao Lai MD e , John D. Mahan MD f,g , Susan Massengill MD h , Maureen Kelly PNP i , Guillermo Cantú MD, PhD j , Mara Medeiros MD, PhD k , Alexandra Phillips l , Gregory Sawicki MD m , David Wood MD, MPH n , Meredith Johnson MPH a , Mary H. Benton BS l , Maria Ferris MD, MPH, PhD a,⁎ a

The University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, NC The University of North Carolina at Chapel Hill School of Public Health, 135 Dauer Dr, Chapel Hill, NC c Wake Forest University, 1834 Wake Forest Rd, Winston-Salem, NC d Victory Junction Camp, 4500 Adam's Way, Randleman, NC e The Brody School of Medicine at East Carolina University, 600 Moye Blvd, Greenville, NC f Nationwide Children's Hospital, 700 Children's Dr, Columbus, OH g The Ohio State University, Columbus, OH h Levine Children's Hospital at Carolinas Medical Center, 1000 Blythe Blvd, Charlotte, NC i The University of North Carolina at Chapel Hill School of Nursing, Carrington Hall, CB#7460, Chapel Hill, NC j Escuela de Medicina, Universidad Panamericana, Puerto Vallarta, Mexico k Hospital Infantil de México Federico Gómez, Calle Dr. Márquez 162, Cuahtemoc, Doctores, Ciudad de México, D.F., Mexico l The University of North Carolina at Chapel Hill, Chapel Hill, NC m Boston Children's Hospital, 300 Longwood Ave, Boston, MA n East Tennessee State University James H. Quillen College of Medicine, 178 Maple Avenue, Mountain Home, TN b

Received 20 March 2015; revised 16 May 2015; accepted 18 May 2015

Key words: Self-management; Transition; Chronic conditions; STARx validity; Self-report transition measure

Introduction The STARx Questionnaire was designed with patient and provider input, to measure selfmanagement and transition skills in adolescents and young adults (AYA) with chronic health conditions. With proven reliability and an empirically-based factor structure, the self-report STARx Questionnaire requires further validation to demonstrate its clinical and research utility. In this study we examine the concurrent, predictive, and discriminant validity of the STARx Questionnaire. Methods: To examine concurrent validity, the STARx Questionnaire was compared to two other published transition readiness tools. Predictive validity was examined using linear regressions between

⁎ Corresponding author: Maria Ferris, MD, MPH, PhD. E-mail address: [email protected] 1 This work received support in part by: The Center for Education Research and Treatment through a grant from the USA Health Services Research Administration, The UNC Kidney Center, KB Reynolds Charitable Trust, Victory Junction Camp, Carolinas Medical Center, Nationwide Children's Hospital, Hospital Infantil Federico Gómez (México City) and The Renal Research Institute. URL: http://www.med.unc.edu/transition (M. Ferris). http://dx.doi.org/10.1016/j.pedn.2015.05.006 0882-5963/© 2015 Elsevier Inc. All rights reserved.

Self-Management and Transition Readiness Assessment

669

the STARx Total Score and literacy, medication adherence, quality of life, and health services use. Discriminant validity was examined by comparing the performance of three chronic illness conditions on the STARx Total Score and associated subscales. Results: The STARx Questionnaire and its subscales positively correlated with the scores for both transition readiness tools reflecting strong concurrent validity. The STARx Questionnaire also correlated positively with the literacy, self-efficacy, and adherence measures indicating strong predictive validity; however, it did not correlate with either quality of life or health care utilization. The performance of AYA across three different clinical conditions was not significant, indicating the clinical utility of this HCT tool for a variety of chronic health conditions. Conclusion: The strong validity of the STARx Questionnaire, in tandem with its strong reliability, indicated adequate psychometric properties for this generic self-report measure. These strong psychometric properties should contribute to the STARx being a viable measure of health care transition for both research and clinical purposes. © 2015 Elsevier Inc. All rights reserved.

AS CHILDREN WITH chronic health conditions progress into adulthood, they experience many transitions, especially in regards to their health care. It is important for adolescents and young adults (AYA) to not only understand their condition, but also learn to self-manage their disease and become independent. A poor transition to adult care, where the AYA does not learn how to properly manage their disease can lead to poor health outcomes, which has been documented across many chronic health conditions (Ferris & Mahan, 2009; Gurvitz & Saidi, 2014; Lotstein et al., 2013). In order to achieve better outcomes, there is need for assessment of the skills for a successful health care transition (HCT). A systematic approach to assess HCT and self-management skills using tools with strong psychometric properties is warranted. Currently, there are few tools to assess these constructs. The Successful Transition to Adulthood with Therapeutics = Rx (STARx) Questionnaire is a self-report measure to assess self-management and transition readiness skills in AYA with a variety of chronic health conditions (Ferris et al., 2015). We have described the item development, reliability, temporal stability and the six factor-based subdomains of this tool (medication management, provider communication, engagement during appointments, disease knowledge, adult health responsibilities and resource utilization) elsewhere (Ferris et al., 2015), with the current version of the STARx Questionnaire demonstrating good reliability. With proven reliability and an empirically-based factor structure, the STARx Questionnaire requires further validation to demonstrate its clinical and research utility. This paper examines the validity of the STARx Questionnaire, and includes examination of concurrent, predictive, and discriminant types of validity. We measured how the STARx Questionnaire correlated with other measures of transition readiness and self-management, namely the provider-verified UNC TRxANSITION Scale (Ferris et al., 2012) and the self-administered Transition Readiness Assessment Questionnaire (TRAQ) (Sawicki et al., 2009; Wood et al., 2014). It was hypothesized that the higher scores on the STARx Questionnaire would correlate with higher scores on the UNC TRxANSITION Scale and TRAQ. We also examined how the STARx Questionnaire would correlate with potential

transition-related predictors or outcomes such as health literacy, self-efficacy, medication adherence, health care utilization, and quality of life. It was hypothesized that higher scores on the STARx Questionnaire would be significantly correlated with higher scores in measures of self-efficacy, health literacy, quality of life, and medication adherence. Conversely, we hypothesized that higher scores on the STARx Questionnaire would correlate with lower levels of health care utilization. Lastly, we assessed the STARx performance of three groups of AYA with chronic conditions in an effort to demonstrate its applicability to a wide variety of AYA with chronic conditions.

Methods Data were collected from AYA with chronic health conditions who were seen at six large health systems representing the north-east, south-east, and mid-west regions of the United States, or who came from several states in the USA to attend a community-based therapeutic camp in the southeast region of the country. Studies were IRB-approved through the individual institutions, with one host institution, which coordinated the overall IRB. In the hospital setting, the survey data were collected online using the survey engine Qualtrics™ or via paper-and-pencil formats. Consents/Assents were obtained from all participants prior to completion of the survey.

Measures Participants at all sites completed a demographics questionnaire which included questions about their age, race/ethnicity, sex, insurance status, current age, diagnosis, age at diagnosis of their chronic condition, and their highest grade in school. The STARx Questionnaire This self-report survey is comprised of 18 questions that are answered on a Likert scale. The STARx Questionnaire provides a total score and six factor-based scores: medication management, provider communication, engagement during appointments, disease knowledge, adult health responsibilities, and resource utilization. Scores range on each item from 0 (never, very hard, or nothing) to 4 (always, very easy, a lot), with an additional answer choice of “I Do Not Take Medicines Right Now,” which is rated as a 5. The total raw score can range from 0 to 90, with higher scores reflecting more intact

670 health care transition skills. The alpha coefficient for the overall scale is 0.80, which indicates good internal consistency of the measure (Ferris et al., 2015). The scale also demonstrates good temporal stability and a stable factor structure (Ferris et al., 2015). This measure was completed through an online survey (Qualtrics™) or by paper-and-pencil depending on the site. The UNC TRxANSITION Scale™ (Ferris et al., 2012) This 33-question scale is administered by a trained health professional or research assistant who gives patients scores for their answers across 10 domains: type of illness, Rx (medications), adherence, nutrition, self-management, issues of reproduction, trade/ school, insurance, ongoing support, new provider knowledge. The trained personnel asked each question of the patient, and verify/rate the patient's answer (based on standardized answer guides) as follows: 1 (complete knowledge/self-management), 0.5 (partial knowledge/selfmanagement), or 0 (no knowledge/self-management). The scores for each item in a sub-domain are then averaged. Participants can obtain between a 0 and 1 for each of the 10 sub-domains, and up to a total score (the sum of all sub-domain scores) of 10. The inter-rater reliability of the scale has a weighted kappa statistic of 0.71, and the measure also has good internal consistency, with all of the items on the scale correlating with their subscale in a moderate to strong fashion (r N 0.42) (Ferris et al., 2012). The total score of the TRxANSITION scale increases with age, which indicates good test–retest reliability (Ferris et al., 2012). Participants at one of the large hospital sites completed this scale. Transition Readiness Assessment Questionnaire (TRAQ) (Sawicki et al., 2009; Wood et al., 2014) This is a 20-question self-report scale measuring transition readiness in five domains: (1) managing medications; (2) appointment keeping; (3) tracking health issues; (4) talking with providers; and (5) managing daily activities. Each item is scored from 1 (No, I do not know how) to 5 (Yes, I always do this when I need to). Scores are calculated by averaging the scores across the items in the questionnaire (or subscale). The alpha coefficient for the overall scale is 0.93, demonstrating good internal consistency (Wood et al., 2014). Participants at three large hospital sites completed this scale through the web-based survey engine Qualtrics™. Rapid Estimate of Adult Literacy in Medicine-Teen (REALM-Teen; Davis et al., 2006) This is an English-language word-recognition test developed to measure health literacy in adolescents ages 10–19. Participants at one of the southeastern sites completed this scale. Participants read out loud the list of 66 words, listed in ascending order of difficulty, to a trained research assistant who scored the scale based on the participant's ability to recognize and pronounce each word. Scores on the scale range from 0 to 66 based on the number of correct words. All test words were commonly used health terms. For participants older than 19 years, the Rapid Estimate of Adult Literacy in Medicine (REALM) was used for testing. This measure is scored the same as the REALM-Teen, but has more advanced health terms. The REALM and REALM-Teen demonstrate good

S.E. Cohen et al. concurrent validity with other health literacy measures, as well as test–retest reliability (Davis et al., 1993; Davis et al., 2006). Newest Vital Sign (Weiss et al., 2005) Participants at one of the southeastern sites completed this test of health literacy and numeracy. This scale contains six questions about a nutrition fact label and is administered by a trained research assistant. Questions are asked, and answers are provided verbally. Each question is scored as a “yes” or “no.” Scores on the newest vital sign range from 0 to 6, which higher scores indicate better health literacy. The measure displays both good criterion validity and internal consistency (Weiss et al., 2005) and has also been validated in adolescents (Driessnack, Chung, Perkhounkova, & Hein, 2014). Self-Efficacy Scale (Iannotti et al., 2006) This is a brief measure of diabetes self-efficacy comprised of nine questions. Items pertaining to diabetes were modified in order to be used for all patients in the current sample. Questions are asked on a Likert scale, and the scale asks, “How sure are you that you can do the following things almost all of the time?” The Likert scale ranges from 1 (not sure at all) to 10 (completely sure). The scores on each item are summed to create a total score. Participants from the camp site completed this scale through the web-based survey engine Qualtrics™. Morisky Medication Adherence Scale (Morisky, Green, & Levine, 1986) This self-report measure of medication adherence was administered to participants at one of the southeastern hospital sites and the camp. The brief version of this scale contains four questions that are scored as “yes” = 1 or “no” = 0. The total score can range from 0 to 4. The Morisky Medication Adherence Scale demonstrates good reliability, with an alpha coefficient of 0.61, as well as good concurrent and predictive validity (Morisky et al., 1986). Participants completed this scale on-line through Qualtrics™. PedsQL (Varni, Seid, & Rode, 1999) Participants at one of the southeastern hospital sites completed this 23-item scale measuring quality of life across four domains: physical, emotional, social, and school. The measure is a five-point Likert Scale, with answers ranging from 0 (never) to 4 (almost always). The scores are then transformed to 0 to 100 as follows: 0 = 100, 1 = 75, 2 = 50, 3 = 25, and 4 = 0. The total score is computed by taking an average of scores on each item in the scale. Higher scores on the scale indicate better quality of life. Internal consistency reliability is high, as 92% of correlations meet the .40 standard (Varni, Seid, & Rode, 1999). The PedsQL discriminates between patients that are on and off treatment, thus demonstrating clinical validity (Varni et al., 1999). Participants completed this scale on-line through Qualtrics™. Health Care Utilization Participants at three of the hospital sites in the southeast provided specific information about their health

Self-Management and Transition Readiness Assessment care utilization for the 12 months prior to completion of the survey. Three questions on health care utilization were asked: (1) frequency of emergency department visits; (2) frequency of hospitalizations; and (3) lengths of hospitalizations. Participants completed these questions through Qualtrics™.

Data Analysis Prior to individual analyses, a correlation was run between demographic factors (age, age of diagnosis, sex, race, type of insurance, diagnosis) and each of the six STARx Questionnaire variables. Those demographic variables that were significantly correlated with the STARx Questionnaire variables were controlled for in the rest of the analyses. If no factors were found to be correlated, then a partial correlation, controlling for chronological age, was used for analysis due to the known relationship of the STARx Questionnaire with age. If any additional demographic variables were found to be significantly correlated, then a linear regression was used. Concurrent Validity The STARx Questionnaire was compared to both the UNC TRxANSITION Scale™ (Ferris et al., 2012) and Transition Readiness Assessment Questionnaire (TRAQ) (Sawicki et al., 2009; Wood et al., 2014) to evaluate concurrent validity. A partial correlation, controlling for age and age of diagnosis, was run between the STARx Questionnaire and UNC TRxANSITION Scale™. A bivariate correlation was run between the STARx Questionnaire and TRAQ. Predictive Validity The relationship between the STARx Questionnaire and various outcomes including literacy, self-efficacy, medication adherence, health care utilization, and quality of life, was examined using the STARx Questionnaire results as both an outcome and a predictor. For literacy, linear regressions were run using the REALM and Newest Vital Sign as predictors, and the STARx Total Score and the six subscales as outcomes. A similar analysis was used to determine the relationship between self-efficacy and the STARx Questionnaire Total Score and subscales, with the STARx Total Score and subscales as outcomes and self-efficacy as a predictor. For medication adherence, a linear regression was run with the STARx Questionnaire total score and subscales as predictors, and medication adherence as an outcome. A similar linear regression was used to determine the relationship of the STARx Questionnaire and health care utilization wherein the STARx Total Score and subscales were used as predictors, and the health care utilization variables were used as outcomes. A linear regression was used to determine the relationship between quality of life and transition readiness, with the STARx Questionnaire Total Score and subscales as predictors and quality of life as an outcome. Discriminant Validity The STARx Questionnaire was administered to 351 AYA with one of 18 different chronic health conditions (e.g., lupus,

671 heart disease, diabetes, etc.). A series of ANCOVAs, adjusting for chronological age, were conducted to determine the ability of the STARx Questionnaire to distinguish among the three clinical groups with the largest number of cases: kidney disease (n = 100), gastrointestinal disease (n = 70), and lung disease (n = 29). In addition to adjusting for chronological age, several other key variables were examined as potential covariates in the analyses: race, sex, socioeconomic status as defined by type of insurance, age of diagnosis, and highest grade of school completed. For any significant ANCOVA, follow-up pair-wise comparisons were examined to determine which groups were different from the others across the STARx variables.

Results The participant characteristics are described in Table 1. The sample size, locations of administration, diagnoses, and age range is described for each tool administered.

Concurrent Validity UNC TRxANSITION Scale™ (Ferris et al., 2012) For this measure, data were collected from 142 participants. Age and age at diagnosis were correlated with the TRxANSITION Scale™ Total Score. Therefore, these two factors were controlled for in subsequent analyses. The STARx Questionnaire Total Score correlated with every subscale of the TRxANSITION Scale™ except for the Type of Diagnosis, Issues of Reproduction, and Ongoing Support subscales. Higher self-reported transition readiness and self-management correlated with higher provider-assessed transition readiness. The TRxANSITION Scale™ Total Score significantly correlated with every score on the STARx Questionnaire, with the total scores for each scale having a moderate correlation coefficient (r = .365, p = .000). As can be seen in Table 2, many of individual subscales from each measure significantly correlated with each other in the expected direction and pattern. TRAQ (Sawicki et al., 2009; Wood et al., 2014) For this measure, data were collected from 125 participants from two of the southeastern children's hospitals and one from the northeast. Demographic data were not collected from all the participants that only completed the TRAQ and STARx, so zero-order correlations were run between the TRAQ scores and STARx scores. Every subscale of the TRAQ significantly correlated with every subscale of the STARx Questionnaire in the expected direction and pattern. Additionally, the STARx Total Score significantly correlated with every subscale of the TRAQ. All correlations were such that higher transition readiness in the STARx Questionnaire was associated with higher transition readiness in the TRAQ. As can be seen in Table 3, the majority of the correlation coefficients were moderate to large in magnitude.

672 Table 1

S.E. Cohen et al. STARx Questionnaire Validity, participant characteristics and locations for each tool administered

Tool

Validity type

Sample USA region size

UNC TRxANSITION Scale™ Concurrent validity 142 (Ferris et al., 2012)

TRAQ (Sawicki et al., 2009)

Concurrent validity 125

REALM and REALM-Teen (Davis et al., 2006)

Predictive validity

79

Newest Vital Sign (Weiss et al., 2005)

Predictive validity

79

Self-Efficacy Scale (Iannotti et al., 2006)

Predictive validity

158

Morisky Medication Adherence Scale (Morisky et al., 1986)

Predictive validity

252

PedsQL (Varni, Seid, & Rode, 1999)

Predictive validity

57

Health Care Utilization

Predictive validity

222

Medical conditions

South-east site

Chronic kidney disease = 49; systemic lupus = 12; Crohn's = 53; hypertension = 2; kidney transplant = 20 and dialysis = 6 North- and south-east sites Cystic fibrosis = 90; chronic kidney disease = 23 and IBD = 12 South-east site Kidney disease = 43; SLE = 10; Crohn's = 1; kidney transplant = 18 and dialysis = 7 South-east site Kidney disease = 43; SLE = 10; Crohn's = 1; kidney transplant = 18 and dialysis = 7 Victory Junction Camp Cancers = 10; physical disabilities = 10; (youth from across lung = 5; bleeding disorders = 3; the USA) skin diseases = 3; craniofacial anomaly = 5; CP = 28; GI = 13; diabetes = 21; down syndrome = 7; neurological = 6; kidney = 10; heart = 15; sickle cell = 12; spina bifida = 5 and other genetic conditions = 5 South-east site and Victory Bleeding = 3; cancers = 10; CP = 18; Junction Camp (youth craniofacial = 2; diabetes = 20; from across the USA) down syndrome = 4; GI = 64; heart = 13; HIV = 1; kidney = 70; lung = 5; lupus = 10; neurological diseases = 6; other genetic = 3; physical disability = 5; sickle cell disease = 11; skin disease = 2 and spina bifida = 4 South-east site Kidney disease = 28; SLE = 12; Crohn's = 1; heart transplant = 1; kidney transplant = 12 and dialysis = 4 South-east site and Victory Bleeding = 3; cancers = 10; CP = 28; Junction Camp (youth craniofacial = 5; diabetes = 21; from across the USA) down syndrome = 7; GI = 38; heart = 15; kidney = 46; lung = 5; lupus = 7; neurological disorder = 6; other genetic = 5; physical disability = 10; sickle cell = 11; skin disease = 3; spina bifida = 5

Predictive Validity Health Literacy For these measures, data were collected from 79 participants at one site from the southeast. Insurance, the socioeconomic proxy, was correlated with the REALM, REALM-Teen and Newest Vital Sign, so this variable was included in the regressions along with chronological age and STARx Questionnaire scores. Patients with higher health literacy scores had higher transition readiness and self-management. Higher REALM-Teen scores significantly correlated with the STARx Questionnaire Total Score (β = .280, p = .013, R2 = .185) and higher medication management (β = .263, p = .024,

Age range 12 to 22

12–24 12 to 22

12 to 22

8 to 16

8 to 22

12 to 22

9 to 21

R2 = .125). Higher scores on the Newest Vital Sign correlated with higher STARx Total Score (β = .237, p = .030, R2 =.185), higher medication management (β = .307, p = .006, R2 = .152), and higher resource utilization (β = .235, p =.038, R2 = .122). Self-Efficacy For self-efficacy, data were collected from 158 participants from the camp site. No demographic variables correlated with the self-efficacy scale, so only age was adjusted in the analysis. The STARx Questionnaire variables were highly correlated with self-efficacy, such that more self-efficacy correlated with more self-reported transition readiness and self-management.

Self-Management and Transition Readiness Assessment Table 2

673

Correlations between the STARx Questionnaire and the UNC TRxANSITION Scale™ (N = 142)

Control variables age & age of diagnosis

Adult health Resource STARx Medication Provider Engagement Disease knowledge responsibilities utilization total management communication during appointments

T

.153 .072 .185 ⁎ .028 .290 ⁎⁎ .001 .205 ⁎ .015 .316 ⁎⁎ .000 .161 .058 .299 ⁎⁎ .000 .166 ⁎ .050 .097 .254 .175 ⁎ .039 .365 ⁎⁎ .000

Correlation Significance (2-tailed) R Correlation Significance (2-tailed) A Correlation Significance (2-tailed) N Correlation Significance (2-tailed) S Correlation Significance (2-tailed) I Correlation Significance (2-tailed) T Correlation Significance (2-tailed) I Correlation Significance (2-tailed) O Correlation Significance (2-tailed) N Correlation Significance (2-tailed) TRxANSITION Correlation total Significance (2-tailed)

.037 .666 .172 ⁎ .042 .365 ⁎⁎ .000 .139 .103 .285 ⁎⁎ .001 .101 .237 .147 .083 .134 .116 .026 .762 .115 .175 .264 ⁎⁎ .002

.141 .096 .082 .334 .177 ⁎ .036 .120 .156 .179 ⁎ .034 .071 .404 .080 .349 .094 .271 .059 .486 .143 .091 .204 ⁎ .016

.144 .089 .123 .149 .114 .180 .160 .058 .303 ⁎ .000 .119 .162 .260 ⁎⁎ .002 .150 .077 .126 .139 .252 ⁎⁎ .003 .317 ⁎⁎ .000

.078 .360 .200 ⁎ .018 .212 ⁎ .012 .267 ⁎⁎ .001 .124 .144 .042 .618 .288 ⁎⁎ .001 .012 .891 .068 .424 − .027 .748 .220 ⁎ .009

.180 ⁎ .033 .079 .356 .191 ⁎ .024 − .020 .817 .128 .132 .179 ⁎ .034 .186 ⁎ .028 .134 .114 .026 .757 .138 .104 .223 ⁎⁎ .008

.036 .670 .064 .450 .044 .604 .115 .178 .164 .053 .111 .190 .217 ⁎⁎ .010 .102 .233 .070 .413 .047 .578 .177 ⁎ .036

⁎⁎ Correlation is significant at the 0.01 level (2-tailed). ⁎ Correlation is significant at the 0.05 level (2-tailed).

Every subscale of the STARx Questionnaire and Total Score, except for the Medication Management Subscale, correlated with the Self-Efficacy Scale (Table 4). Patients that had higher self-efficacy had higher self-reported transition readiness (β = .471, p = .000, R2 = .244), communicated more with their providers (β = .539, p = .000, R2 = .291), engaged more during their medical appointments (β = .432, p = .000, R2 = .195), had more knowledge about their disease (β =.451, p = .000, R2 = .245), engaged in more adult health responsibilities (β = .362, p = .000, R2 = .131), and had better resource utilization (β = .193, p = .013, R2 = .085). Medication Adherence For medication adherence, data were collected from 252 participants. No demographic variables correlated with the Morisky Medication Adherence Scale, so only chronological age was used as a covariate in the analysis. The STARx Questionnaire Total Score significantly correlated with the Morisky Medication Adherence Scale, such that higher transition readiness and self-management correlated with higher medication adherence (β = .301, p = .000, R2 = .070). Higher STARx Medication Management correlated with higher self-reported medication adherence (β = .499, p = .000, R2 = .224). More knowledge about one's disease (STARx Disease Knowledge Subscale) significantly correlated

with higher medication adherence (β = .216, p = .001, R2 = .044). Health Care Utilization For these variables, data were collected from 222 participants the three southeastern sites. Age of diagnosis was significantly correlated to health care utilization, so this variable, along with chronological age, was used as a covariate in subsequent analyses. No significant correlations were found between the STARx Total Score and health care utilization. The only significant correlation was found between STARx Adult Health Responsibilities and ED visits. We found that patients who engaged in more adult health responsibilities had more ED visits (β = .176, p = .014, R2 = .028), although this relationship accounted for less than 3% of the variance. A subset of 69 patients from one site in the southeast, consisting of patients with chronic kidney disease (CKD) and inflammatory bowel disease (IBD), also completed the health care utilization measures. Insurance, the socioeconomic proxy, was significantly correlated to health care utilization, so this was used as a covariate, along with chronological age, in subsequent analyses. Higher scores on the STARx Disease Knowledge Subscale were significantly associated with fewer number of hospitalizations (β = − .453, p = .000, R2 = .262) and number of inpatient days in the past year (β = − .432, p = .000, R2 = .187).

674

S.E. Cohen et al.

Table 3

Correlations between the STARx Questionnaire and TRAQ (N = 125) Adult health Resource STARx Medication Provider Engagement Disease knowledge responsibilities utilization total management communication during appointments

TRAQ managing medications TRAQ appointment keeping TRAQ tracking health issues TRAQ talking with providers TRAQ managing daily activities TRAQ total

Pearson correlation Sig. (2-tailed) Pearson correlation Sig. (2-tailed) Pearson correlation Sig. (2-tailed) Pearson correlation Sig. (2-tailed) Pearson correlation Sig. (2-tailed) Pearson correlation Sig. (2-tailed)

.670 ⁎⁎ .000 .691 ⁎⁎ .000 .625 ⁎⁎ .000 .583 ⁎⁎ .000 .656 ⁎⁎ .000 .775 ⁎⁎ .000

.355 ⁎⁎ .000 .327 ⁎⁎ .000 .325 ⁎⁎ .000 .340 ⁎⁎ .000 .276 ⁎⁎ .002 .383 ⁎⁎ .000

.534 ⁎⁎ .000 .487 ⁎⁎ .000 .353 ⁎⁎ .000 .508 ⁎⁎ .000 .548 ⁎⁎ .000 .556 ⁎⁎ .000

.630 ⁎⁎ .000 .714 ⁎⁎ .000 .626 ⁎⁎ .000 .394 ⁎⁎ .000 .608 ⁎⁎ .000 .755 ⁎⁎ .000

.524 ⁎⁎ .000 .429 ⁎⁎ .000 .423 ⁎⁎ .000 .557 ⁎⁎ .000 .542 ⁎⁎ .000 .550 ⁎⁎ .000

.346 ⁎⁎ .000 .431 ⁎⁎ .000 .318 ⁎⁎ .000 .355 ⁎⁎ .000 .269 ⁎⁎ .002 .426 ⁎⁎ .000

.320 ⁎⁎ .000 .397 ⁎⁎ .000 .449 ⁎⁎ .000 .254 ⁎⁎ .004 .393 ⁎⁎ .000 .452 ⁎⁎ .000

⁎⁎ Correlation is significant at the 0.01 level (2-tailed).

Quality of Life For this measure, data were collected from 57 participants at one southeastern site. No demographic variables correlated with the STARx, so only age was controlled for in the analyses. There were no significant correlations between the STARx Questionnaire and the PedsQL.

Discriminant Validity A series of ANCOVAs, controlling for chronological age and other key covariates, were conducted to determine the ability of the STARx Questionnaire to distinguish among the three largest clinical groups in our sample: kidney disease (n = 100), gastrointestinal disease (n = 70), and lung disease (n = 29). Results indicated no significant differences among groups for the STARx Total Score or any of the six subscales after adjusting for key variables in the group comparisons.

Table 4

Discussion The primary purpose of this study was to provide initial validity information for the STARx Questionnaire. Examining a number of different types of validity, the STARx Questionnaire demonstrated good concurrent and predictive validity, while findings pertaining to its lack of discriminant validity supported its generic nature and use across a number of different chronic health conditions.

Concurrent Validity For concurrent validity, the measure was moderately (UNC TR xANSITION Scale™) to strongly (TRAQ) correlated with two other established transition readiness and self-management measures. This is particularly noteworthy for the relationship with the TRAQ where both are self-report tools that directly address transition skills, with

Relationship between the STARx Questionnaire and clinical outcomes

Outcome or characteristic

STARx total

β .471 ⁎⁎ Sig. .000 R2 .244 Medication adherence (N = 252) β .301 ⁎⁎ Sig. .000 R2 .070 Number of hospitalizations β .034 past year (N = 69) Sig. .798 R2 .071 Length of hospitalizations β − .059 past year (N = 69) Sig. .666 R2 .015 Self-efficacy (N = 158)

Medication Provider Engagement Disease Adult health Resource management communication during knowledge responsibilities utilization appointments .109 .173 .017 .499 ⁎⁎ .000 .224 .116 .337 .083 .012 .924 .013

⁎⁎ Correlation is significant at the 0.01 level (2-tailed). ⁎ Correlation is significant at the 0.05 level (2-tailed).

.539 ⁎⁎ .000 .291 .105 .119 .012 .131 .293 .086 − .011 .930 .013

.432 ⁎⁎ .000 .195 .125 .090 .014 .117 .403 .080 .091 .528 .019

.451 ⁎⁎ .000 .245 .216 ⁎⁎ .001 .044 − .453 ⁎⁎ .000 .262 − .432 ⁎⁎ .000 .187

.362 ⁎⁎ .000 .131 .067 .320 .006 − .093 .445 .079 − .055 .659 .016

.193 ⁎ .013 .085 .055 .424 .005 .074 .545 .076 .025 .840 .013

Self-Management and Transition Readiness Assessment the overall relationship accounting for about 60% of the variance. The relationship with the clinician-verified tool (UNC TRxANSITION Scale™), only accounted for about 13% of the variance, possibly suggesting that the perceptions of AYA may not directly align with clinician ratings of patient's skill set and/or degree of disease knowledge/self-management. This will be an important consideration when using any self-report tool. The individual subscales of the STARx Questionnaire also showed good concurrent validity with the associated subscales from the UNC TRxANSITION™ scale and TRAQ. For example, the Medication Management Subscale of the STARx Questionnaire significantly correlated with the Medication, Adherence, and Self-Management subscales of the UNC TRxANSITION Scale™, as well as the Managing Medications Subscale of TRAQ. The STARx Provider Communication Subscale had a large correlation with the TRAQ Subscale of Talking with Providers. The STARx Engagement During Appointment Subscale had a moderate correlation with the Self-Management Subscale on the UNC TRxANSITION scale ™, which includes an item about asking the doctor or nurse questions. The STARx Engagement Subscale also strongly correlated with the TRAQ subscales of Appointment Keeping, Tracking Health Issues, and Managing Daily Activities. The STARx Disease Knowledge Subscale correlated the most highly with the Nutrition and Trade/School subscales of the UNC TRxANSITION Scale™. This latter association suggests a keen relationship between patients having a good understanding of their condition and the need to follow their diets better and thinking about their futures.

Predictive Validity For predictive validity, the STARx Questionnaire was significantly related to a number of outcomes related to the transition readiness process. It was hypothesized that higher scores on the STARx Questionnaire would significantly correlate with higher scores in measures of self-efficacy, literacy, quality of life, and medication adherence, and lower levels of health care utilization. Our hypotheses were supported for health literacy, self-efficacy, and medication management, as these outcomes were significantly correlated with the STARx Total Score. Many of the STARx subscales were also correlated with these outcomes (Table 4). Our hypotheses were not supported for quality of life or health care utilization. However, it is important to note that the medication management subscale was significantly correlated with fewer and shorter length of hospitalizations in the past year. The STARx Questionnaire Total Score correlated with measures of health literacy and numeracy, indicating that the health literacy of patients with chronic health conditions likely influences their readiness for transition and self-management. Furthermore, self-efficacy was highly correlated with STARx Questionnaire scores. Thus, the more self-efficacy a patient has, the more prepared they are to transition to adult health care. Another factor that is crucial to health outcomes is medication adherence. Patients with higher self-reported transition readiness had higher medication adherence. This

675 should be expected, given that one of the factors of the STARx Questionnaire is directly related to medical management, but the relationship with the STARx Questionnaire factor of disease knowledge also reinforces the importance of understanding some basic reasons why it is important to be adherent to a medication regimen. These identified factors can help guide clinicians and providers conducting risk assessment and interventions with their patients. Our results also suggest that transition readiness plays a significant role in other psychosocial and medical outcomes for adolescents and young adults with chronic health conditions.

Discriminant Validity Finally, the STARx Questionnaire was designed as a generic tool for use across a wide range of chronic health conditions. Our examination of the discriminant validity of this tool revealed no significant differences among three groups of patients, thus suggesting the use of the STARx Questionnaire across a wide range of patients. Additionally, from a conceptual perspective, these findings may indicate the genetic nature of transition concepts in general, and that transition readiness is related more to individual skills and knowledge than to specific disease factors. At present, and in line with the goals for developing this tool, the STARx Questionnaire does not appear to be disease-specific in nature, and can be used to measure self-management and transition readiness in AYA with a variety of chronic health conditions. Findings from this study showed the STARx Questionnaire to be a valid self-report tool that can be used to assess transition readiness and self-management skills in adolescents and young adults with a variety of health conditions. In tandem with the strong reliability of the STARx Questionnaire (Ferris et al., 2015), these findings indicate that the psychometric properties of this tool are strong, and that the STARx Questionnaire should be a much welcomed tool in the relatively new area of health care transition. The addition of this tool to the assessment armamentarium should provide both clinicians and researchers the opportunity to obtain a reliable and valid assessment of the transition process for their AYA patients with chronic illness, and hopefully provide more objective guidance for intervention planning for their patients throughout the transition process. Additional evidence regarding the clinical and research utility of this tool awaits.

Acknowledgments We would like to thank our adolescents and young adults with chronic conditions and their parents for guiding this work and generous participation in our longitudinal cohort study. We also would like to thank our team of volunteers for their dedication including: Sofia Ocegueda, Anabel Gutiérrez Almaraz; Kristen Wolbert, RN; Jim O’Neill; Mark Moultrie; Robert Imperial; Keith Gerarden; Marcia Dias, RN, Lynn McCoy, RN, David Tauer, RN; Caroline Jennette, MSW; Bradley Manton, MSW, Megan Fox, Elizabeth Prata, FNP, Ana María Hernández BS, Claudia Rojas and the team of scientists and providers from England, Mexico and the USA:

676 Drs. Sue Tolleson-Rinehart, Zachary Smith, Mary Beth Champion, Ali Calicoglu, Nina Jain, Leonard Stein, Tom Belhorn, Peter Sims, Victoria Pham, Alan Watson, Janet McDonough, Rupa Redding-Lallinger, Zion Ko, Ana Catalina Alvarez, Keisha Gibson, Debbie Gipson, Karin True, Randy Detwiler and William Primack.

References Davis, T. C., Long, S. W., Jackson, R. H., Mayeaux, E. J., George, R. B., Murphy, P. W., et al. (1993). Rapid estimate of adult literacy in medicine: A shortened screening instrument. Family Medicine, 25, 391–395. Davis, T. C., Wolf, M. S., Arnold, C. L., Byrd, R. S., Long, S. W., Springer, T., et al. (2006). Development and validation of the Rapid Estimate of Adolescent Literacy in Medicine (REALM-Teen): A tool to screen adolescents for below-grade reading in health care settings. Pediatrics, 118, e1707–e1714. Driessnack, M., Chung, S., Perkhounkova, E., & Hein, M. (2014). Using the “newest vital sign” to assess health literacy in children. Journal of Pediatric Health Care, 28, 165–171. Ferris, M., Cohen, S., Haberman, C., Javalkar, K., Massengill, S., Mahan, J. D., et al. (2015). Self-management and transition readiness assessment: Development, reliability, and factor structure of the STARx Questionnaire. Journal of Pediatric Nursing, 30, 691–699 (in this issue). Ferris, M. E., Harward, D. H., Bickford, K., Layton, J. B., Ferris, M. T., Hogan, S. L., et al. (2012). A clinical tool to measure the components of health-care transition from pediatric care to adult care: The UNC TRxANSITION Scale. Renal Failure, 34, 744–753.

S.E. Cohen et al. Ferris, M. E., & Mahan, J. D. (2009). Pediatric chronic kidney disease and the process of health care transition. Seminars in nephrology. vol. 29, No. 4. (pp. 435–444). WB Saunders. Gurvitz, M., & Saidi, A. (2014). Transition in congenital heart disease: It takes a village. Heart, 100, 1075–1076. Iannotti, R. J., Schneider, S., Nansel, T. R., Haynie, D. L., Plotnick, L. P., Clark, L. M., et al. (2006). Self-efficacy, outcome expectations, and diabetes self-management in adolescents with type 1 diabetes. Journal of Developmental & Behavioral Pediatrics, 27, 98–105. Lotstein, D. S., Seid, M., Klingensmith, G., Case, D., Lawrence, J. M., Pihoker, C., et al. (2013). Transition from pediatric to adult care for youth diagnosed with type 1 diabetes in adolescence. Pediatrics, 131, e1062–e1070. Morisky, D. E., Green, L. W., & Levine, D. M. (1986). Concurrent and predictive validity of a self-reported measure of medication adherence. Medical Care, 24, 67–74. Sawicki, G. S., Lukens-Bull, K., Yin, X., Demars, N., Huang, I. C., Livingood, W., et al. (2009). Measuring the transition readiness of youth with special healthcare needs: Validation of the TRAQ—Transition Readiness Assessment Questionnaire. Journal of Pediatric Psychology, 2, 160–171. Varni, J. W., Seid, M., & Rode, C. A. (1999). The PedsQL: Measurement model for the pediatric quality of life inventory. Medical Care, 37, 126–139. Weiss, B. D., Mays, M. Z., Martz, W., Castro, K. M., DeWalt, D. A., Pignone, M. P., et al. (2005). Quick assessment of literacy in primary care: The newest vital sign. The Annals of Family Medicine, 3, 514–522. Wood, D. L., Sawicki, G. S., Miller, M. D., Smotherman, C., Lukens-Bull, K., Livingood, W. C., et al. (2014). The Transition Readiness Assessment Questionnaire (TRAQ): Its factor structure, reliability, and validity. Academic Pediatrics, 14, 415–422.