- Email: [email protected]
Literacy assessment in a cardiovascular nutrition education setting
Patient Education and Counseling 31 (1997) 139–150
Literacy assessment in a cardiovascular nutrition education setting a,
b
a
b
Thomas R. TenHave *, Barbara Van Horn , Shiriki Kumanyika , Eunice Askov , Yvonne Matthews a , Lucile L. Adams-Campbell c a
b
Center for Biostatistics and Epidemiology, Pennsylvania State University College of Medicine, Hershey PA 17033, USA Institute for the Study of Adult Literacy, College of Education, Pennsylvania State University, University Park, PA, USA c Department of Medicine, Howard University, Washington, DC, USA Received 16 July 1996; revised 7 January 1997; accepted 7 January 1997
Abstract We assessed functional literacy of hypercholesterolemic or hypertensive African Americans (n 5 339) prior to their participation in a nutrition education program. A word pronunciation and recognition test using 20 common cardiovascular or nutrition terms was first developed based on correlations with standardized reading achievement test scores, then administered to program participants. Nearly half (48%) had word recognition scores equivalent to a T 8th grade reading level. Lower scores were associated with less education, lower income, unemployment, heavier work activity if employed, less healthy diets, history of heart disease or diabetes, and higher depression scores (all P 0.01); several of these associations were independent of education. The educational materials were geared to a 5th to 8th grade reading level. However, when both audiotaped and printed instruction were provided, individuals with reading scores T 8th grade preferentially used the tapes. This brief and relatively unobtrusive literacy assessment may help to identify persons who can benefit most from audiovisual approaches to cardiovascular nutrition education. 1997 Elsevier Science Ireland Ltd. Keywords: Literacy; African Americans; Cardiovascular disease; Patient education; Nutrition
1. Introduction According to the 1992 National Adult Literacy Survey (NALS), a significant percentage of American adults are unable to perform literacy tasks above the most basic levels [1]. Over 50%
*Corresponding author. Tel: 1 1 717 5317178; fax: 1 1 717 5779; e-mail: [email protected].
of US adults perform at the lowest levels of literacy. Those performing at the lowest two quintiles of literacy levels are more often from minority populations and report lower per capita income. Of particular concern for health professionals, lack of reading skills may limit the ability of persons in some high risk populations to access and use critical information needed for self-care or risk reduction [2–6]. In this regard low literacy may be one of the key mediating
0738-3991 / 97 / $17.00 1997 Elsevier Science Ireland Ltd. All rights reserved PII S0738-3991( 97 )01003-3
140
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
variables predisposing socially disadvantaged individuals to high rates of many types of illnesses and disproportionately high death rates. Literacy-related problems in accessing health care relate in part to the form in which health care information is provided [2,4]. Studies of the estimated reading levels of existing health information and education materials [7,8] conclude, for example, that the majority of health information and education materials are too difficult for many in the general public to understand. The need to modify health information materials and approaches to be effective with persons with limited literacy skills and to offer alternative formats for information delivery is now recognized widely [5,6,9–11]. The potential value of assessing literacy in clinical settings in order to identify individuals who have problems reading or understanding health information is also recognized, since such individuals cannot be readily identified through routine questioning about educational background or even about reading ability as such [4]. Just as adults engaged in job-related literacy instruction preferred job-related functional literacy assessments compared to traditional standardized reading tests, so has the concept of relevant testing instruments been extended to the health field [12,13], giving rise to such instruments as the Rapid Estimate of Adult Literacy in Medicine (REALM) [14] and, more recently, the Test of Functional Health Literacy in Adults (TOFHLA) [15]. We report on the development and use of an easy-to-administer literacy screening instrument that applies this concept of relevant testing instruments to the context of outpatient nutrition counseling for cardiovascular risk reduction. The instrument was created to characterize urban African-American adults on reading skills necessary for learning nutrition and health content as they enrolled in a 1-year program to motivate and help them learn to reduce their dietary fat, cholesterol, and sodium. We also determined the relationship of reading levels ascertained in this way to the sociodemographic and health profiles of these program participants.
2. Methods
2.1. Background CARDES (acronym for CARdiovascular Dietary Education System) offers cardiovascular nutrition education strategies appropriate for adults with diverse reading levels, including those with reading skills at least at the 5th to 8th grade level. CARDES was designed specifically to reach African Americans with high blood cholesterol, high blood pressure, or both of these conditions because of the lack of cardiovascular nutritional educational materials that incorporate the unique culture of African Americans. The CARDES educational program includes a motivational video about dietary change issues for an African American woman and her extended family; food picture cards encoded with symbols indicating whether the food shown is low, medium, or high in sodium, fat, and cholesterol; an accompanying workbook that includes replicas of the food cards; and 12 audiotaped nutritional counseling vignettes with accompanying worksheets. A total of 339 African American men and women aged 40–70 years were recruited from the Washington DC community, primarily through finger-prick cholesterol screenings in local supermarkets, and enrolled in a randomized clinical trial to compare the effectiveness of two formats for using the CARDES materials in the management of high blood pressure and high cholesterol. The two formats were: (1) Instruction Group — receiving printed materials, motivational audiotape cassettes, and group counseling sessions, and (2) Self-help Group — receiving printed materials only. A literacy screening instrument was developed for this trial because the objectives included ultimately analyzing program effectiveness according to participants’ ability to read and understand CVD nutrition information. Standardized reading tests commonly used in Adult Basic Education (ABE) Programs (e.g., Tests of Adult Basic Education (TABE) [16], Tests of Applied Literacy Skills (TALS) [17], Adult Basic Learn-
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
ing Examination (ABLE) [18] were considered inappropriate for administration in a clinical setting as well as potentially displeasing to participants. The trial protocol, which included the use of a videotaped explanation for obtaining informed consent, was approved by the committees for the protection of human subjects at participating institutions.
2.2. Instrument development 2.2.1. Objectives and design We modeled the CARDES instrument on existing word recognition assessments such as the Slosson Oral Reading Test (SORT) [19] and the REALM [14] that provide a global assessment of reading ability. In such assessments, the respondent is shown a word and asked to pronounce it. The ability to pronounce words correctly is strongly correlated to overall reading facility [20]. Not only does word recognition skill correlate with reading comprehension ability in adults, it is actually an independent predictor. That is, word recognition skill predicts reading comprehension ability in adults even after variance due to listening comprehension ability has been partialled out. While it is quite possible for an adult to have poor reading comprehension ability despite adequate word-decoding skills — probably due to deficient general listening comprehension skills — it is highly unlikely that excellent reading comprehension will be observed in the face of deficient word recognition skills. Words on the list used in CARDES were intended to be relevant to the audience as well as to program content. Using focus groups, we identified 200 terms commonly found in CVD health and nutrition information, emphasizing food and diet-related words familiar to African Americans, to serve as the pool of items for development of the word recognition test. To compare word recognition with scores on a norm-referenced, standardized reading test, 200 words, or subsets of these words, were administered to persons for whom reading subtest scores on the TABE were also available. The analysis was performed on a dichotomized version of this
141
outcome measure: immediate recognition versus skipping or misreading the word. TABE Scores were categorized into three grade groups: 5th, 5–8, 8. (See table in Results section.)
2.2.2. Subjects Instrument norming focused on AfricanAmerican adults. Word recognition and TABE scores were collected from 62 African-American volunteers (44 in Washington, DC and 18 in Harrisburg, PA) recruited from ABE programs by advertisement and instructors’ recommendations. Most (n 5 58) were female. Most (n 5 46) were less than 35 years of age, but seven were older than 65. About half (n 5 29) were high school graduates. Some subjects had taken the Tests of Adult Basic Education (TABE) previously and their scores were considered still current by their instructors. For others, the TABE was administered (either to small groups of two to four subjects or on an individual basis), after which a tra ined administrator met with each subject to administer the 200 word recognition test. TABE scores equated to a range from grades 2.5 to 12.9: 13 subjects had reading levels less than 5th grade; 19 had reading levels between the 5th and 8th grade. The Harrisburg subjects had a somewhat lower average TABE score (716.2, SE 6.45) than the Washington DC subjects (733.1, SE 9.5), although the difference was not statistically significant (P 0.1). 2.2.3. Word recognition test In a private one-to-one meeting, each subject was presented with 200 cards placed in three stacks. Cards were white index cards on which one word printed in black ink in a 48-point serif font was centered. Cards were placed in random order, mixing long and short, simple and complex words. The subject was asked to read and pronounce the word while the interviewer evaluated the subject’s comprehension of the word by how well he or she pronounced it. Three possible responses were recorded for each word: 1. Immediate recognition (within 4 s) 2. Skipping the word
142
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
3. Misreading the word The analysis was performed on a dichotomized version of this outcome measure: immediate recognition (1) (correct pronunciation) versus skipping or misreading the word (0).
2.2.4. Prior knowledge In the initial stage of instrument development, each subject was asked several questions about CVD awareness and about their own or a family member’s prior exposure to CVD nutrition counseling. Preliminary analyses showed no relationship between prior knowledge and word recognition, and this aspect was not further pursued. 2.2.5. List development A sequence of three stages of analyses reduced the 200-word list to the desired type of brief instrument by successively eliminating words that did not contribute uniquely or sufficiently to the association of word recognition scores and TABE scores. The Stage 1 analyses were done with data obtained from the 44 subjects in Washington DC, using the entire 200-word list. One hundred words read correctly by all subjects in the sample were uninformative, and thus were eliminated. No words were read incorrectly by all subjects in the sample. The investigation at this point was performed as a sequence of two analyses: (1) a refinement of the 100 informative words to 67 words based upon the strength of association among the words (internal association); and (2) an analysis of the association of the resulting 67 words with the overall TABE score (external association), which produced a final list of 12 words. For the internal association track, all words for which their item-total score correlation (a measure of the strength of association between an individual word and remaining words) was less than 0.3 or more than 0.7 were eliminated. This range was employed to select those words that provided a reasonable balance between unique information (low item-total correlation) and overlapping information (high itemtotal correlation). For the external association track, a step-wise linear regression was then implemented on the 67 words to obtain a list of
12 words. This analysis consisted of a sequence of multiple regressions of the overall TABE score (dependent variable) on the individual word scores (separate independent variables), entered into the regression model one at a time in an order determined by the strength of the association between the TABE score and the word score. Only those word scores for which the statistical significance level of the corresponding adjusted correlation with the TABE score was less than 10% were included in the model. Any word that had been previously entered into the model was removed when the significance level of its adjusted correlation with the TABE score rose above 10% with the inclusion of additional words in the model. The Stage 2 investigation involved adminstration of a 71 word list to a new sample of 18 ABE subjects in Pennsylvania, validation of the 12 word list from Stage 1 on this sample, and development of a new word list. The 71 word list consisted of the 67 word list in Stage 1 augmented by four additional words that correlated well with the TABE in Stage 1. The validation of the 12 word list from Stage 1 entailed a multiple regression of the TABE scores for these 18 subjects on the sum of the scores of the 12 words from the Stage 1 list was performed. A crossvalidation R-square statistic was computed to assess the predictive power of the sum of these 12 word scores in this regression. This statistic avoids the pitfalls of the R-square statistic that involve assessing the predictive validity of a regression on the same sample from which it is derived [21]. Computation of the cross-validation R-square entailed a separate regression for each individual where that person’s data was omitted from the regression. The results of these iterative regressions were then pooled to yield a crossvalidation R-square. Because the cross-validation R-square (0.23) was not satisfactory, the forward step-wise regression analysis of Stage 1 was repeated using the 71 words on the combined Washington, DC and Harrisburg samples. The resulting Stage 2 list consisted of 13 words, of which 4 overlapped with those 12 words in the Stage 1 list. These four words were coronary, recipes, hydrogenated, and occurs.
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
The cross-validation R-square was 0.27 for the regression of the TABE score on the sum of the 13 word scores in the Stage 2 list. Because of relatively low R-squares for each list and because we believed that the CARDES clinical trial protocol could accommodate several more words than the 12–13 words in each list, we combined the Stage 1 and 2 lists to form a final list of 21 words. We excluded one word, ‘poultry’ from this list of 21 words to obtain the final list of 20 words, because of a misunderstanding between the administrators of the test for Washington and Harrisburg samples and the sample participants over the pronunciation of the word. This misunderstanding led to the misperception on the part of the test administrators that the participants did not understand the word when they pronounced it in a way not recognized by the test administrators. We reviewed the remaining 20 words in the final list to ensure that this issue did not pertain to them. The Stage 3 analysis entailed the assessment of the final list of 20 words. An ordinal logistic regression model, for which the TABE three grade variable was the response variable and the total word score was the covariate, was used to derive thresholds for the total word score corresponding to the three grade categories based upon the TABE scores. Agreement between the TABE score and word recognition score was evaluated with Spearman rank order correlation and, when categorized, with the kappa statistic. Kappa represents the ratio of the observed percentage of agreement not due to chance to the maximum percentage of agreement not due to chance. Weighted kappas were calculated employing different weighting schemes to take into account the ordinal nature of the categorized variables [22].
2.3. Literacy assessment of program participants The final, 20-item list was administered as part of the baseline interview of 339 men and women who enrolled in the CARDES randomized trial. As in the developmental phase, the test was administered with a series of index cards, one at a time and each with one word, for the respon-
143
Table 1 Word list for the CARDES Literacy Screening Instrument, in ascending order of difficulty* 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8
Affect Guidelines Advice Substitute Recipes Hocks Broiled; Physician Acute; Monosodium Occurs Polyunsaturated Marinate Coronary, Hemorrhage, Hereditary Hydrogenated Aneurysms Paralysis Arteriosclerosis
*Difficulty ranking is based on percent of validation study participants who correctly pronounced the words, e.g., 98% correctly pronounced ‘Affect’ but only 19% correctly pronounced arteriosclerosis. Words on the same row had the same percent correct on the validation study.
dent to pronounce. The words were arranged in ascending order of difficulty (Table 1). Since administrators of the CARDES literacy instrument were not reading experts, individuals responsible for screening CARDES participants were trained via guidelines and an audiotape modeling acceptable pronunciation of the words. The American Heritage Dictionary (1991) was used as the standard for word pronunciation. Interviewers were cautioned to mark words incorrect if the subject added or deleted endings or letters since deviations from standard pronunciation would have altered the assumed correlation between the individual’s score on the CARDES word list and his or her TABE scores. We investigated the association of the literacy scores with several variables assessed on the same occasion as the literacy assessment (i.e., the initial interview conducted as participants were randomized into the clinical trial). These variables included socio-demographic characteristics such as age, education, income level and employment status for which one would expect an association with literacy skills as well as selected behavioral and health status characteristics. The general objective of these analyses was to de-
144
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
termine whether knowing the literacy score would contribute useful information about participants’ characteristics upon entry into the study over and above that contributed by the more conventional sociodemographic indicators. Independence of associations of variables with these CVD nutrition literacy scores from associations with years of education was of particular interest, since literacy scores would not be of interest if simply a proxy for years of school. The analyses of these associations treated a given baseline variable as the response and literacy as the independent variable of interest with age and sex included as covariates to address potential confounding. Parallel analyses were performed with and without education ( T 12th or 12) as a covariate to identify the literacy contribution to these associations that was statistically independent of education. Logistic regression was performed for binary and ordinal baseline variables and linear regression was implemented for continuous responses. A questionnaire was administered to participants who completed a follow-up visit approximately 4 months after randomization to ascertain frequency of use of the printed nutrition guide and the audiotaped instruction. The relative use of these two types of materials was analyzed separately for each literacy and grade group, among participants who received both printed materials and the audioseries (and portable cassette player), i.e. those in the ‘instruction’ arm of the CARDES trial, with the expectation that those with lower literacy scores might use the audiotapes preferentially.
scores were unrelated to prior knowledge and experience and because the CARDES program was not designed to improve literacy.
3. Results
3.1. Word recognition test characteristics For the 62 subjects in the instrument development phase, the regression of the TABE total score on the literacy assessment score (0–20 words pronounced correctly) yielded an R 2 of 81%. An R 2 of 42% (cross validation R 2 5 41%) was obtained from a regression of the total TABE score on the sum of the 20 word scores in the list. The ratio of the number of words to the sample size (20 / 62) is large enough to yield a relatively high R 2 by chance, but the overall data as described here suggest that the word list generated from the whole sample would correlate highly with the TABE in the general population from which the sample was selected. A high internal consistency was indicated by a Cronbach’s of 0.87. The rank-order (r) correlation between the literacy screening instrument score and the TABE of 0.73. The thresholds for the total word score that corresponded to the three grade categories based upon the TABE were defined as follows:
2.4. Repeatability
8 5th grade (TABE 702): total word score 0–9 8 5th to 8th grade (702 T TABE T 751): 10 T total word score T 16 8 8th grade (TABE 751): total word score 17–20
As a test of the repeatability of the scores obtained with the literacy assessment instrument, we re-administered it to every second CARDES participant who completed the final 12-month follow-up visit, until approximately 50 repeat assessments had been made. Exposure to the CARDES program during this 12-month period was assumed to have no effect on literacy skills. This assumption seemed reasonable based on our preliminary finding that word recognition
Cross-tabulating grade classifications based upon the total word score and TABE yielded a percent agreement of 68% and a kappa (standard error) of 0.51 (0.09). Weighted kappas ranged between 0.49 and 0.70 under the different weighting schemes for ordinal data. Data from the 12-month repeat administration of the literacy screening instrument were available for 59 CARDES participants. These 59 participants were not signficantly different from
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
the remainder of those randomized on sex, age, marital status, years of school completed, medical history (heart attack, heart attack hospitalization, diabetes, or all three), depression score, their original literacy score, or the study arm (instructional or self-help) to which they were randomized. The mean (SD) change in number correct between the first and second administrations was 0.5 (2.3) and the two scores were highly correlated (Pearson correlation) and concordant [23] (r 5 0.88 for both coefficients). Good agreement was also observed based on the categorical reading level scores (weighted kappa 5 0.47–0.82). Seven of nine persons who initially scored less than 5th grade (0–9 words correct) and 26 of 27 who initially scored above 8th grade (17–20 words correct) scored in the same category on repeat administration. Of the 23 people who scored in the intermediate category (5th to 8th grade, or 10–16 words correct), four scored in the lower category and five in the higher category on repeat administration.
3.2. CARDES participant characteristics Table 2 shows the demographic chacteristics of the men and women randomized in CARDES. All except two of the 339 participants were African American or black (i.e. of African descent but not born in America), primarily of US Southern background, and most were female. Men and women with diverse educational and income levels were included. The literacy screening instrument was easy to administer, requiring approximately 5 min. All randomized participants completed the task and, based on feedback from staff, found it acceptable. Approximately half (48%) had literacy assessment scores at or below 8th grade. The distribution of literacy scores is shown in Fig. 1. The medical history and behavioral characteristics selected for examination with respect to literacy levels are also shown in Table 2. Approximately 5% of participants reported a prior history of a heart attack, 11% having been hospitalized for a heart condition, or 15% with a history of diabetes. The majority were sedentary
145
Table 2 Selected characteristics of CARDES I sample at enrollment (n 5 339) Variable
Percent of sample
African American Male / female Age 40–54 years 55–70 years Currently married or living with partner Employed Education 8 years 8–11 years 12 years 12 years Income (adjusted for household size) US$10 000 per year US$10 000–19 999 per year US$20 000–29 999 per year US$30 000 1 per year Regional background Southern United States Other Occupational category (n 5 332) Administrative and managerial Professionals / teachers / school personnel Technicians / clinicians Labour, maintenance, factory worker Service occupations, safety, security Hypertension (by history) Serum total cholesterol 200 mg / day Medical history Heart attack Hospitalization for heart condition Diabetes Activity at leisure light or inactive Activity at work light or inactive (n 5 196) Rate your Plate Score (heart healthy diet) 20–33 (least) 34–47 (somewhat) 48–60 (very)
99 26 / 74 41 59 33 51 8 20 32 38 38 27 20 15 87 13 12 40 8 21 19 50 86 6 12 14 79 74 9 55 36
and, from the Rate Your Plate scores, had relatively high fat or high salt diets.
3.3. Association of literacy scores with sociodemographic and health variables Literacy scores were not strongly associated with age (P 5 0.5) among the 40- to 70-year-old participants in CARDES. Those with the lowest literacy scores were somewhat more likely to be
146
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
Fig. 1. Distribution of scores on Literacy Screening Instrument among participants enrolling in CARDES clinical trial (n 5 339).
men and to have less than an 8th grade education (Table 3). Overall, however, although the association between education and literacy scores is strong, the overlap is not complete. For example, 8% of those scoring in the low (0–9 correct responses, scored as reading at less than 5th grade level) and 29% of those scoring in the middle categories (10–16 words, scored as reading at the 5th to 8th grade level), had more than a 12th grade education. The Spearman’s rank order correlation of years of education and literacy scores was 0.55 (P 0.0001). In spite of the weak associations of literacy scores with age and sex, these were included as adjustment variables (with age as a continuous variable) for clarity of interpretation when examining associations with the other variables shown on Table 3, with and without an additional adjustment for education. Both income and employment were significantly associated with literacy scores when adjusting only for age and sex, but not with additional adjustment for education. Also shown in Table 3, the proportion of respondents reporting a history of heart disease or diabetes was inversely related to literacy scores, independent of age and sex. The association with heart disease or having any of the three
conditions remained after adjustment for education. Depression scores were based on the Beck Depression Inventory Short Form [24], and were restricted to the range between 0 and 20 because persons scoring above 20 were considered ineligible for CARDES. Depression scores of those who were enrolled were strongly inversely associated with literacy scores (more responses indicative of depression at lower literacy levels) even after adjustment for education. The age-sex adjusted association between physical activity and literacy differed for work and leisure. The proportion reporting a low level of physical activity at work (analyzed only for the subsample of employed participants) decreased as literacy scores increased (Table 3). This association was diminished, but not removed by adjustment for education. There was a weaker, non-significant trend in the opposite direction for leisure time activity (not shown). Similar logistic regression results were obtained for a two-level classification of literacy scores (0–16 versus 17–20 words, or T 8th versus 8th grade reading level), although the associations were less often statistically significant than those described above, which are based on three categories. This was probably because, as is evident in Table 3, for certain variables the 5th
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
147
Table 3 Participant characteristics associated with scores on CARDES CVD Nutrition Literacy Assessment Variable
Literacy Score (number of words pronounced correctly)
P-value**
Percentages within literacy score categories
% Male Education 8th grade 8–12th grade 12th grade Annual income adjusted household size US$10 000 / year US$10 000–19 999 / year US$20 000–29 999 / year US$30 000 1 / year Employment Activity at work light / inactive (n 5 196) Rate Your Plate Score Least heart healthy (20–34) Somewhat heart healthy (35–47) Heart healthy (48–60) Medical history Heart attack Hospitalized for heart condition Diabetes Any of the above Depression score (mean (SD))
0–9
10–16
17–20
P-value*
38
20
26
0.056 0.000
18 74 8
1 70 29
0 33 67
64 18 10 8 36 32
42 32 17 9 48 52
26 27 24 22 57 63
16 56 28
10 60 31
6 52 42
14 24 20 42 4.58 (4.65)
4 12 20 29 3.50 (3.47)
3 7 10 18 2.56 (2.90)
0.000
0.2
0.002 0.002 0.008
0.2 0.05 0.436
0.012 0.003 0.053 0.001 0.0001
0.04 0.03 0.2 0.03 0.019
*Adjusted for age (continuous) and sex where applicable. **Adjusted for age (continuous), sex and education ( T 12 or 12 years).
to 8th grade group is more like the 5th grade group, whereas for other variables it is more like the 8th grade group.
3.4. Use of oral versus written materials Results of the comparison of reported frequency of using the printed and audiotaped materials in the CARDES educational package are shown in Table 4. This was a within-person analysis, restricted to those who received both printed and audiotaped information and who completed the 4-month follow-up interview. Persons with literacy scores T 8th grade were approximately twice as likely to report a greater relative frequency of using the audiotapes (P 5 0.03).
4. Discussion This brief instrument, or similar instruments developed for specific populations using this approach, can be used to classify patients into categories indicative of the ease with which they will understand printed information. The use of CVD-related vocabulary renders our instrument particularly relevant to clinical settings serving men and women with hypertension, elevated cholesterol, or heart disease, and, since prior knowledge did not seem to be a major influence on the final instrument, in many other circumstances in which this terminology is relevant. The apparent preferential use of the audiotaped versus the printed CARDES materials among those with reading scores at or below 8th grade supports, indirectly, the ability of this instrument to
148
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
Table 4 Reported use of CARDES Nutrition Guide and Audioseries four months after enrollment Literacy score (reading level)
Used Nutrition Guide more than Audioseries Used Nutrition Guide and Audioseries equally Used Audioseries more than Nutrition Guide
0–16 correct ( T 8th grade, n 5 48)
17–20 correct ( 8th grade, n 5 50)
19% 27 54%
28%* 44% 28%
}Subset of participants who were given both types of materials and who completed a four month follow up interview. *P 5 0.02 for difference between literacy score groups.
identify those less likely to use printed information even when it is designed to have a relatively low literacy demand. To the extent that this sample is representative of a population of urban African Americans seeking dietary counseling for the management of high blood pressure or high blood cholesterol, the 48% who scored at or below the 8th grade reading level is noteworthy; it suggests a large population in need of special educational approaches. The extent of the problem may even be underestimated in our sample. Adults with limited educational backgrounds tend not to participate in studies involving instruction [25]. Our advertisting for participants in ‘easy to read’ nutrition information probably selected for a higher distribution of literacy than applies in the community from which our participants were recruited. An inverse association of literacy skills with age is sometimes reported [1,4], although we did not observe this, possibly because the age range of our sample was restricted. We did observe a sex difference in literacy skills; the men in our sample were more likely to have low literacy scores. Others have reported that women have lower literacy than men [1]. Whether a sex difference is observed and of what nature may be population or sample dependent. Our data mirrored the positive correlation between literacy scores and level of education that has been reported in other research [1]. Correlates of lower literacy scores on this CVD nutrition-oriented instrument that were independent of education were moderate or heavy work activity, having a medical history of heart disease or diabetes, and a more depressed
psychosocial profile compared to persons with higher scores. The link between literacy and work activity, even when some adjustment has been made to control for educational level, may indicate a clustering of less educated persons with low literacy skills in jobs requiring manual labor. The associations with medical history and psychosocial status may reflect causes or consequences of health-related low literacy skills or both. It is also possible that these associations reflect a selection bias such that persons with low literacy skills who also had prior heart disease or psychosocial problems were more likely to enroll in our study than those with no such problems. The similarity of these findings with other reports in which low literacy has been associated with poor physical and psychosocial health [4,26] suggests that these findings may be meaningful. Limitations of this study include the relative small size and lack of diversity in the sample used to develop the word recognition test. The statistical analyses suggested that the instrument performs relatively well, within the limits reported, as a proxy for more formal reading assessments such as the TABE. In addition, the instrument generated a good distribution of scores in the more broadly-based CARDES population, was repeatable, and correlated with years of education at a level that is consistent with the literature [1]. Nevertheless, because word recognition may differ with age or with other population characteristics, validity testing in a larger and more diverse population would be desired. In particular, experience is needed in interpretationg this type of test in adults who are not native speakers of English or who speak dialects of English.
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
149
Table 5 Practice implications 8 Persons with lower literacy skills are at higher than average risk of health problems and are less likely than persons with higher literacy skills to use printed health information 8 Assessment of literacy skills facilitates identification of problems in reading health information that cannot be inferred from years of school completed 8 Brief word recognition tests can be used in clinical settings to assess functional literacy skills that are directly relevant to patient education 8 Audiotaped oral instructions may be well-received by patients who have problems reading printed information
5. Implications and conclusions There is increasing awareness of the link between literacy and health and the consequent need to identify patients who have problems reading standard health-related information (Table 5). Even easy-to-use literacy appropriate instructional materials may be less utilized by persons with lower literacy skills. Such individuals may rely on oral information sources such as radio or television [6] to supplement or substitute for advice they receive during face-to-face encounters with health personnel. The marked socioeconomic differentials in CVD mortality [27,28] and the apparent association of lower literacy levels with poorer CVD risk profiles underscore the need to consider literacy as one mediator of these differentials that can be addressed through screening and tailored programming. Specifically, identification of persons with problems reading printed information and perhaps a preference for oral instruction can lead to: (1) an increasing appreciation for the need to have alternative instructional formats available and (2) offering these formats to those who can most benefit from them. As the awareness of literacy issues and their importance increases, effective patient education approaches will increasingly include direct assessment of patients’ relevant literacy skills. Although literacy skills are associated with formal education, years of school completed is not an informative proxy measure for reading problems. Some persons who have completed high school have difficulty reading and some people with very limited formal education ready very well. Standardized reading tests designed to
diagnose specific problems or test reading comprehension are neither appropriate nor necessary in clinical setting where the objective is primarily to match patients to educational approaches that they will find acceptable and understandable. Word recognition tests as described here, that can provide reliable proxy indicators of reading problems, can be used effectively for this purpose and are relatively unobtrusive when based on content relevant to the clinical issues being addressed.
Acknowledgments This research was supported by grant number R01 HL46778 from the National Heart, Lung, and Blood Institute, National Institutes of Health. The authors are grateful to staff and students in Adult Basic Education programs who participated in the development of the literacy assessment instrument and to the many others involved in implementing the CARDES Program. Some of these results were presented at the 1993 and 1995 annual meetings of the American Public Health Association.
References [1] Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, DC: U.S. Government Printing Office, 1993. [2] Doak L, Doak C. Patient comprehension profiles. Recent findings and strategies. Patient Counseling and Health Education 1980; 2: 101–106.
150
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
[3] Hussey LC, Gilliland K. Compliance, low literacy, and locus of control. Nurs Clin North Am 1989; 24: 605–611. [4] Williams MV, Parker RM, Baker DW, Parikh NS, Pitkin K, Coates WC, Nurss JR. Inadequate functional health literacy among patients at two public hospitals. JAMA 1995; 274: 1677–1682. [5] Doak CC, Doak LG, Root JH. Teaching Patients with Low-Literacy Skills, 2nd Ed. Philadelphia, Pa: JB Lippincott; 1995. [6] Weiss BD, Reed RL, Kligman EW. Literacy skills and communication methods of low-income older persons. Patient Education and Counseling 1995; 25: 109–119. [7] Glanz K, Rudd J. Readability and content analysis of printed cholesterol education materials. Patient Education and Counseling 1990; 16: 109–117. [8] Mallett L, Spruill WJ. Readability evaluation of nine patient drug education sources. Am Pharm 1988; NS28: 33–39. [9] Petterson R, Visuals for information: Research and practice. Educational Technology Publication. Englewood Cliffs, NJ 07632. Chapter 3, Literacy; 1989. [10] AMC Cancer Research Center. Beyond the Brochure: Alternative Approaches to Effective Health Communication. Denver, CO 1993. [11] Gagliano ME. A literature review on the efficacy of video in patient education. J Med Ed 1988; 63: 785–792. [12] Murphy PW, Davis TC, Long SW, Jackson RH, Decker BC. Rapid estimate of adult literacy in medicine (REALM). A quick reading test for patients. J Reading 1993; 37: 124–130. [13] Nurss JR, Baken DW, Davis TC, Parker RM, Williams MV. Difficulties in Functional Health Literacy Screening in Spanish-Speaking Adults. J Reading 1995; 38: 632– 637. [14] Davis TC, Long SW, Jackson RI, Mayeau EJ, Crouch MA. Rapid estimate of adult literacy in medicine. A shortened screening instrument. Fam Med 1993; 25: 391–395. [15] Parker R, Baker D, Williams M, Nurss J. The Test of
[16] [17] [18]
[19] [20]
[21]
[22]
[23]
[24] [25]
[26]
[27]
[28]
Functional Health Literacy in Adults (TOFHLA): a new instrument for measuring patients’ literacy skills. J Gen Intern Med 1995; 10: 537–545. Tests of Adult Basic Education (TABE) Monterey, CA: CTB / McGraw-Hill, 1987. Tests of Applied Literacy Skills (TALS), New York: Simon and Schuster Workplace Resources, NY, 1991. Adult Basic Learning Examination (ABLE), San Antonio, TX: The Psychological Corp., Harcourt Brace and Co, 1986. Slosson RJL. Slosson Oral Reading Test – Revised. East Aurora, NY: Slosson Educational Publishing 1990. Stanovich KE. Word Recognition: Changing Perspectives. In: Barr R, Kamil ML, Mosenthal P, Pearson PD, editors. Handbook of Reading Research, vol. II. New York: Longman, 1991: 418–452. Efron B. The jackknife, bootstrap, and other resampling plans. Society for Industrial and Applied Mathematics: Philadelphia, 1982. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. Chinchilli VM, Martel JK, Kumanyika MS, Lloyd T. A weighted concordance coefficient for repeated measures designs. Biometrics 1996; 52: 341–353. Beck AT. Depression: Clinical, Experimental and Theoretical Aspects. London, 1969. Darkenwald GG, Merriam SB. Adult Education: Foundations of Practice. New York: Harper and Row, 1982. Weiss BD, Hart G, Pust RE. The relationship between literacy and health. J Health Care Poor Underserved 1991; 1: 351–363. Burt VL, Whelton P, Roccella EJ, et al. Prevalence of Hypertension in The U.S. Adult Population. Hypertension 1995; 25: 305–313. Saunders E. Tailoring Treatment to Minority Patients. Am J Med 1990; 88: 3B–21S.
Literacy assessment in a cardiovascular nutrition education setting a,
b
a
b
Thomas R. TenHave *, Barbara Van Horn , Shiriki Kumanyika , Eunice Askov , Yvonne Matthews a , Lucile L. Adams-Campbell c a
b
Center for Biostatistics and Epidemiology, Pennsylvania State University College of Medicine, Hershey PA 17033, USA Institute for the Study of Adult Literacy, College of Education, Pennsylvania State University, University Park, PA, USA c Department of Medicine, Howard University, Washington, DC, USA Received 16 July 1996; revised 7 January 1997; accepted 7 January 1997
Abstract We assessed functional literacy of hypercholesterolemic or hypertensive African Americans (n 5 339) prior to their participation in a nutrition education program. A word pronunciation and recognition test using 20 common cardiovascular or nutrition terms was first developed based on correlations with standardized reading achievement test scores, then administered to program participants. Nearly half (48%) had word recognition scores equivalent to a T 8th grade reading level. Lower scores were associated with less education, lower income, unemployment, heavier work activity if employed, less healthy diets, history of heart disease or diabetes, and higher depression scores (all P 0.01); several of these associations were independent of education. The educational materials were geared to a 5th to 8th grade reading level. However, when both audiotaped and printed instruction were provided, individuals with reading scores T 8th grade preferentially used the tapes. This brief and relatively unobtrusive literacy assessment may help to identify persons who can benefit most from audiovisual approaches to cardiovascular nutrition education. 1997 Elsevier Science Ireland Ltd. Keywords: Literacy; African Americans; Cardiovascular disease; Patient education; Nutrition
1. Introduction According to the 1992 National Adult Literacy Survey (NALS), a significant percentage of American adults are unable to perform literacy tasks above the most basic levels [1]. Over 50%
*Corresponding author. Tel: 1 1 717 5317178; fax: 1 1 717 5779; e-mail: [email protected].
of US adults perform at the lowest levels of literacy. Those performing at the lowest two quintiles of literacy levels are more often from minority populations and report lower per capita income. Of particular concern for health professionals, lack of reading skills may limit the ability of persons in some high risk populations to access and use critical information needed for self-care or risk reduction [2–6]. In this regard low literacy may be one of the key mediating
0738-3991 / 97 / $17.00 1997 Elsevier Science Ireland Ltd. All rights reserved PII S0738-3991( 97 )01003-3
140
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
variables predisposing socially disadvantaged individuals to high rates of many types of illnesses and disproportionately high death rates. Literacy-related problems in accessing health care relate in part to the form in which health care information is provided [2,4]. Studies of the estimated reading levels of existing health information and education materials [7,8] conclude, for example, that the majority of health information and education materials are too difficult for many in the general public to understand. The need to modify health information materials and approaches to be effective with persons with limited literacy skills and to offer alternative formats for information delivery is now recognized widely [5,6,9–11]. The potential value of assessing literacy in clinical settings in order to identify individuals who have problems reading or understanding health information is also recognized, since such individuals cannot be readily identified through routine questioning about educational background or even about reading ability as such [4]. Just as adults engaged in job-related literacy instruction preferred job-related functional literacy assessments compared to traditional standardized reading tests, so has the concept of relevant testing instruments been extended to the health field [12,13], giving rise to such instruments as the Rapid Estimate of Adult Literacy in Medicine (REALM) [14] and, more recently, the Test of Functional Health Literacy in Adults (TOFHLA) [15]. We report on the development and use of an easy-to-administer literacy screening instrument that applies this concept of relevant testing instruments to the context of outpatient nutrition counseling for cardiovascular risk reduction. The instrument was created to characterize urban African-American adults on reading skills necessary for learning nutrition and health content as they enrolled in a 1-year program to motivate and help them learn to reduce their dietary fat, cholesterol, and sodium. We also determined the relationship of reading levels ascertained in this way to the sociodemographic and health profiles of these program participants.
2. Methods
2.1. Background CARDES (acronym for CARdiovascular Dietary Education System) offers cardiovascular nutrition education strategies appropriate for adults with diverse reading levels, including those with reading skills at least at the 5th to 8th grade level. CARDES was designed specifically to reach African Americans with high blood cholesterol, high blood pressure, or both of these conditions because of the lack of cardiovascular nutritional educational materials that incorporate the unique culture of African Americans. The CARDES educational program includes a motivational video about dietary change issues for an African American woman and her extended family; food picture cards encoded with symbols indicating whether the food shown is low, medium, or high in sodium, fat, and cholesterol; an accompanying workbook that includes replicas of the food cards; and 12 audiotaped nutritional counseling vignettes with accompanying worksheets. A total of 339 African American men and women aged 40–70 years were recruited from the Washington DC community, primarily through finger-prick cholesterol screenings in local supermarkets, and enrolled in a randomized clinical trial to compare the effectiveness of two formats for using the CARDES materials in the management of high blood pressure and high cholesterol. The two formats were: (1) Instruction Group — receiving printed materials, motivational audiotape cassettes, and group counseling sessions, and (2) Self-help Group — receiving printed materials only. A literacy screening instrument was developed for this trial because the objectives included ultimately analyzing program effectiveness according to participants’ ability to read and understand CVD nutrition information. Standardized reading tests commonly used in Adult Basic Education (ABE) Programs (e.g., Tests of Adult Basic Education (TABE) [16], Tests of Applied Literacy Skills (TALS) [17], Adult Basic Learn-
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
ing Examination (ABLE) [18] were considered inappropriate for administration in a clinical setting as well as potentially displeasing to participants. The trial protocol, which included the use of a videotaped explanation for obtaining informed consent, was approved by the committees for the protection of human subjects at participating institutions.
2.2. Instrument development 2.2.1. Objectives and design We modeled the CARDES instrument on existing word recognition assessments such as the Slosson Oral Reading Test (SORT) [19] and the REALM [14] that provide a global assessment of reading ability. In such assessments, the respondent is shown a word and asked to pronounce it. The ability to pronounce words correctly is strongly correlated to overall reading facility [20]. Not only does word recognition skill correlate with reading comprehension ability in adults, it is actually an independent predictor. That is, word recognition skill predicts reading comprehension ability in adults even after variance due to listening comprehension ability has been partialled out. While it is quite possible for an adult to have poor reading comprehension ability despite adequate word-decoding skills — probably due to deficient general listening comprehension skills — it is highly unlikely that excellent reading comprehension will be observed in the face of deficient word recognition skills. Words on the list used in CARDES were intended to be relevant to the audience as well as to program content. Using focus groups, we identified 200 terms commonly found in CVD health and nutrition information, emphasizing food and diet-related words familiar to African Americans, to serve as the pool of items for development of the word recognition test. To compare word recognition with scores on a norm-referenced, standardized reading test, 200 words, or subsets of these words, were administered to persons for whom reading subtest scores on the TABE were also available. The analysis was performed on a dichotomized version of this
141
outcome measure: immediate recognition versus skipping or misreading the word. TABE Scores were categorized into three grade groups: 5th, 5–8, 8. (See table in Results section.)
2.2.2. Subjects Instrument norming focused on AfricanAmerican adults. Word recognition and TABE scores were collected from 62 African-American volunteers (44 in Washington, DC and 18 in Harrisburg, PA) recruited from ABE programs by advertisement and instructors’ recommendations. Most (n 5 58) were female. Most (n 5 46) were less than 35 years of age, but seven were older than 65. About half (n 5 29) were high school graduates. Some subjects had taken the Tests of Adult Basic Education (TABE) previously and their scores were considered still current by their instructors. For others, the TABE was administered (either to small groups of two to four subjects or on an individual basis), after which a tra ined administrator met with each subject to administer the 200 word recognition test. TABE scores equated to a range from grades 2.5 to 12.9: 13 subjects had reading levels less than 5th grade; 19 had reading levels between the 5th and 8th grade. The Harrisburg subjects had a somewhat lower average TABE score (716.2, SE 6.45) than the Washington DC subjects (733.1, SE 9.5), although the difference was not statistically significant (P 0.1). 2.2.3. Word recognition test In a private one-to-one meeting, each subject was presented with 200 cards placed in three stacks. Cards were white index cards on which one word printed in black ink in a 48-point serif font was centered. Cards were placed in random order, mixing long and short, simple and complex words. The subject was asked to read and pronounce the word while the interviewer evaluated the subject’s comprehension of the word by how well he or she pronounced it. Three possible responses were recorded for each word: 1. Immediate recognition (within 4 s) 2. Skipping the word
142
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
3. Misreading the word The analysis was performed on a dichotomized version of this outcome measure: immediate recognition (1) (correct pronunciation) versus skipping or misreading the word (0).
2.2.4. Prior knowledge In the initial stage of instrument development, each subject was asked several questions about CVD awareness and about their own or a family member’s prior exposure to CVD nutrition counseling. Preliminary analyses showed no relationship between prior knowledge and word recognition, and this aspect was not further pursued. 2.2.5. List development A sequence of three stages of analyses reduced the 200-word list to the desired type of brief instrument by successively eliminating words that did not contribute uniquely or sufficiently to the association of word recognition scores and TABE scores. The Stage 1 analyses were done with data obtained from the 44 subjects in Washington DC, using the entire 200-word list. One hundred words read correctly by all subjects in the sample were uninformative, and thus were eliminated. No words were read incorrectly by all subjects in the sample. The investigation at this point was performed as a sequence of two analyses: (1) a refinement of the 100 informative words to 67 words based upon the strength of association among the words (internal association); and (2) an analysis of the association of the resulting 67 words with the overall TABE score (external association), which produced a final list of 12 words. For the internal association track, all words for which their item-total score correlation (a measure of the strength of association between an individual word and remaining words) was less than 0.3 or more than 0.7 were eliminated. This range was employed to select those words that provided a reasonable balance between unique information (low item-total correlation) and overlapping information (high itemtotal correlation). For the external association track, a step-wise linear regression was then implemented on the 67 words to obtain a list of
12 words. This analysis consisted of a sequence of multiple regressions of the overall TABE score (dependent variable) on the individual word scores (separate independent variables), entered into the regression model one at a time in an order determined by the strength of the association between the TABE score and the word score. Only those word scores for which the statistical significance level of the corresponding adjusted correlation with the TABE score was less than 10% were included in the model. Any word that had been previously entered into the model was removed when the significance level of its adjusted correlation with the TABE score rose above 10% with the inclusion of additional words in the model. The Stage 2 investigation involved adminstration of a 71 word list to a new sample of 18 ABE subjects in Pennsylvania, validation of the 12 word list from Stage 1 on this sample, and development of a new word list. The 71 word list consisted of the 67 word list in Stage 1 augmented by four additional words that correlated well with the TABE in Stage 1. The validation of the 12 word list from Stage 1 entailed a multiple regression of the TABE scores for these 18 subjects on the sum of the scores of the 12 words from the Stage 1 list was performed. A crossvalidation R-square statistic was computed to assess the predictive power of the sum of these 12 word scores in this regression. This statistic avoids the pitfalls of the R-square statistic that involve assessing the predictive validity of a regression on the same sample from which it is derived [21]. Computation of the cross-validation R-square entailed a separate regression for each individual where that person’s data was omitted from the regression. The results of these iterative regressions were then pooled to yield a crossvalidation R-square. Because the cross-validation R-square (0.23) was not satisfactory, the forward step-wise regression analysis of Stage 1 was repeated using the 71 words on the combined Washington, DC and Harrisburg samples. The resulting Stage 2 list consisted of 13 words, of which 4 overlapped with those 12 words in the Stage 1 list. These four words were coronary, recipes, hydrogenated, and occurs.
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
The cross-validation R-square was 0.27 for the regression of the TABE score on the sum of the 13 word scores in the Stage 2 list. Because of relatively low R-squares for each list and because we believed that the CARDES clinical trial protocol could accommodate several more words than the 12–13 words in each list, we combined the Stage 1 and 2 lists to form a final list of 21 words. We excluded one word, ‘poultry’ from this list of 21 words to obtain the final list of 20 words, because of a misunderstanding between the administrators of the test for Washington and Harrisburg samples and the sample participants over the pronunciation of the word. This misunderstanding led to the misperception on the part of the test administrators that the participants did not understand the word when they pronounced it in a way not recognized by the test administrators. We reviewed the remaining 20 words in the final list to ensure that this issue did not pertain to them. The Stage 3 analysis entailed the assessment of the final list of 20 words. An ordinal logistic regression model, for which the TABE three grade variable was the response variable and the total word score was the covariate, was used to derive thresholds for the total word score corresponding to the three grade categories based upon the TABE scores. Agreement between the TABE score and word recognition score was evaluated with Spearman rank order correlation and, when categorized, with the kappa statistic. Kappa represents the ratio of the observed percentage of agreement not due to chance to the maximum percentage of agreement not due to chance. Weighted kappas were calculated employing different weighting schemes to take into account the ordinal nature of the categorized variables [22].
2.3. Literacy assessment of program participants The final, 20-item list was administered as part of the baseline interview of 339 men and women who enrolled in the CARDES randomized trial. As in the developmental phase, the test was administered with a series of index cards, one at a time and each with one word, for the respon-
143
Table 1 Word list for the CARDES Literacy Screening Instrument, in ascending order of difficulty* 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8
Affect Guidelines Advice Substitute Recipes Hocks Broiled; Physician Acute; Monosodium Occurs Polyunsaturated Marinate Coronary, Hemorrhage, Hereditary Hydrogenated Aneurysms Paralysis Arteriosclerosis
*Difficulty ranking is based on percent of validation study participants who correctly pronounced the words, e.g., 98% correctly pronounced ‘Affect’ but only 19% correctly pronounced arteriosclerosis. Words on the same row had the same percent correct on the validation study.
dent to pronounce. The words were arranged in ascending order of difficulty (Table 1). Since administrators of the CARDES literacy instrument were not reading experts, individuals responsible for screening CARDES participants were trained via guidelines and an audiotape modeling acceptable pronunciation of the words. The American Heritage Dictionary (1991) was used as the standard for word pronunciation. Interviewers were cautioned to mark words incorrect if the subject added or deleted endings or letters since deviations from standard pronunciation would have altered the assumed correlation between the individual’s score on the CARDES word list and his or her TABE scores. We investigated the association of the literacy scores with several variables assessed on the same occasion as the literacy assessment (i.e., the initial interview conducted as participants were randomized into the clinical trial). These variables included socio-demographic characteristics such as age, education, income level and employment status for which one would expect an association with literacy skills as well as selected behavioral and health status characteristics. The general objective of these analyses was to de-
144
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
termine whether knowing the literacy score would contribute useful information about participants’ characteristics upon entry into the study over and above that contributed by the more conventional sociodemographic indicators. Independence of associations of variables with these CVD nutrition literacy scores from associations with years of education was of particular interest, since literacy scores would not be of interest if simply a proxy for years of school. The analyses of these associations treated a given baseline variable as the response and literacy as the independent variable of interest with age and sex included as covariates to address potential confounding. Parallel analyses were performed with and without education ( T 12th or 12) as a covariate to identify the literacy contribution to these associations that was statistically independent of education. Logistic regression was performed for binary and ordinal baseline variables and linear regression was implemented for continuous responses. A questionnaire was administered to participants who completed a follow-up visit approximately 4 months after randomization to ascertain frequency of use of the printed nutrition guide and the audiotaped instruction. The relative use of these two types of materials was analyzed separately for each literacy and grade group, among participants who received both printed materials and the audioseries (and portable cassette player), i.e. those in the ‘instruction’ arm of the CARDES trial, with the expectation that those with lower literacy scores might use the audiotapes preferentially.
scores were unrelated to prior knowledge and experience and because the CARDES program was not designed to improve literacy.
3. Results
3.1. Word recognition test characteristics For the 62 subjects in the instrument development phase, the regression of the TABE total score on the literacy assessment score (0–20 words pronounced correctly) yielded an R 2 of 81%. An R 2 of 42% (cross validation R 2 5 41%) was obtained from a regression of the total TABE score on the sum of the 20 word scores in the list. The ratio of the number of words to the sample size (20 / 62) is large enough to yield a relatively high R 2 by chance, but the overall data as described here suggest that the word list generated from the whole sample would correlate highly with the TABE in the general population from which the sample was selected. A high internal consistency was indicated by a Cronbach’s of 0.87. The rank-order (r) correlation between the literacy screening instrument score and the TABE of 0.73. The thresholds for the total word score that corresponded to the three grade categories based upon the TABE were defined as follows:
2.4. Repeatability
8 5th grade (TABE 702): total word score 0–9 8 5th to 8th grade (702 T TABE T 751): 10 T total word score T 16 8 8th grade (TABE 751): total word score 17–20
As a test of the repeatability of the scores obtained with the literacy assessment instrument, we re-administered it to every second CARDES participant who completed the final 12-month follow-up visit, until approximately 50 repeat assessments had been made. Exposure to the CARDES program during this 12-month period was assumed to have no effect on literacy skills. This assumption seemed reasonable based on our preliminary finding that word recognition
Cross-tabulating grade classifications based upon the total word score and TABE yielded a percent agreement of 68% and a kappa (standard error) of 0.51 (0.09). Weighted kappas ranged between 0.49 and 0.70 under the different weighting schemes for ordinal data. Data from the 12-month repeat administration of the literacy screening instrument were available for 59 CARDES participants. These 59 participants were not signficantly different from
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
the remainder of those randomized on sex, age, marital status, years of school completed, medical history (heart attack, heart attack hospitalization, diabetes, or all three), depression score, their original literacy score, or the study arm (instructional or self-help) to which they were randomized. The mean (SD) change in number correct between the first and second administrations was 0.5 (2.3) and the two scores were highly correlated (Pearson correlation) and concordant [23] (r 5 0.88 for both coefficients). Good agreement was also observed based on the categorical reading level scores (weighted kappa 5 0.47–0.82). Seven of nine persons who initially scored less than 5th grade (0–9 words correct) and 26 of 27 who initially scored above 8th grade (17–20 words correct) scored in the same category on repeat administration. Of the 23 people who scored in the intermediate category (5th to 8th grade, or 10–16 words correct), four scored in the lower category and five in the higher category on repeat administration.
3.2. CARDES participant characteristics Table 2 shows the demographic chacteristics of the men and women randomized in CARDES. All except two of the 339 participants were African American or black (i.e. of African descent but not born in America), primarily of US Southern background, and most were female. Men and women with diverse educational and income levels were included. The literacy screening instrument was easy to administer, requiring approximately 5 min. All randomized participants completed the task and, based on feedback from staff, found it acceptable. Approximately half (48%) had literacy assessment scores at or below 8th grade. The distribution of literacy scores is shown in Fig. 1. The medical history and behavioral characteristics selected for examination with respect to literacy levels are also shown in Table 2. Approximately 5% of participants reported a prior history of a heart attack, 11% having been hospitalized for a heart condition, or 15% with a history of diabetes. The majority were sedentary
145
Table 2 Selected characteristics of CARDES I sample at enrollment (n 5 339) Variable
Percent of sample
African American Male / female Age 40–54 years 55–70 years Currently married or living with partner Employed Education 8 years 8–11 years 12 years 12 years Income (adjusted for household size) US$10 000 per year US$10 000–19 999 per year US$20 000–29 999 per year US$30 000 1 per year Regional background Southern United States Other Occupational category (n 5 332) Administrative and managerial Professionals / teachers / school personnel Technicians / clinicians Labour, maintenance, factory worker Service occupations, safety, security Hypertension (by history) Serum total cholesterol 200 mg / day Medical history Heart attack Hospitalization for heart condition Diabetes Activity at leisure light or inactive Activity at work light or inactive (n 5 196) Rate your Plate Score (heart healthy diet) 20–33 (least) 34–47 (somewhat) 48–60 (very)
99 26 / 74 41 59 33 51 8 20 32 38 38 27 20 15 87 13 12 40 8 21 19 50 86 6 12 14 79 74 9 55 36
and, from the Rate Your Plate scores, had relatively high fat or high salt diets.
3.3. Association of literacy scores with sociodemographic and health variables Literacy scores were not strongly associated with age (P 5 0.5) among the 40- to 70-year-old participants in CARDES. Those with the lowest literacy scores were somewhat more likely to be
146
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
Fig. 1. Distribution of scores on Literacy Screening Instrument among participants enrolling in CARDES clinical trial (n 5 339).
men and to have less than an 8th grade education (Table 3). Overall, however, although the association between education and literacy scores is strong, the overlap is not complete. For example, 8% of those scoring in the low (0–9 correct responses, scored as reading at less than 5th grade level) and 29% of those scoring in the middle categories (10–16 words, scored as reading at the 5th to 8th grade level), had more than a 12th grade education. The Spearman’s rank order correlation of years of education and literacy scores was 0.55 (P 0.0001). In spite of the weak associations of literacy scores with age and sex, these were included as adjustment variables (with age as a continuous variable) for clarity of interpretation when examining associations with the other variables shown on Table 3, with and without an additional adjustment for education. Both income and employment were significantly associated with literacy scores when adjusting only for age and sex, but not with additional adjustment for education. Also shown in Table 3, the proportion of respondents reporting a history of heart disease or diabetes was inversely related to literacy scores, independent of age and sex. The association with heart disease or having any of the three
conditions remained after adjustment for education. Depression scores were based on the Beck Depression Inventory Short Form [24], and were restricted to the range between 0 and 20 because persons scoring above 20 were considered ineligible for CARDES. Depression scores of those who were enrolled were strongly inversely associated with literacy scores (more responses indicative of depression at lower literacy levels) even after adjustment for education. The age-sex adjusted association between physical activity and literacy differed for work and leisure. The proportion reporting a low level of physical activity at work (analyzed only for the subsample of employed participants) decreased as literacy scores increased (Table 3). This association was diminished, but not removed by adjustment for education. There was a weaker, non-significant trend in the opposite direction for leisure time activity (not shown). Similar logistic regression results were obtained for a two-level classification of literacy scores (0–16 versus 17–20 words, or T 8th versus 8th grade reading level), although the associations were less often statistically significant than those described above, which are based on three categories. This was probably because, as is evident in Table 3, for certain variables the 5th
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
147
Table 3 Participant characteristics associated with scores on CARDES CVD Nutrition Literacy Assessment Variable
Literacy Score (number of words pronounced correctly)
P-value**
Percentages within literacy score categories
% Male Education 8th grade 8–12th grade 12th grade Annual income adjusted household size US$10 000 / year US$10 000–19 999 / year US$20 000–29 999 / year US$30 000 1 / year Employment Activity at work light / inactive (n 5 196) Rate Your Plate Score Least heart healthy (20–34) Somewhat heart healthy (35–47) Heart healthy (48–60) Medical history Heart attack Hospitalized for heart condition Diabetes Any of the above Depression score (mean (SD))
0–9
10–16
17–20
P-value*
38
20
26
0.056 0.000
18 74 8
1 70 29
0 33 67
64 18 10 8 36 32
42 32 17 9 48 52
26 27 24 22 57 63
16 56 28
10 60 31
6 52 42
14 24 20 42 4.58 (4.65)
4 12 20 29 3.50 (3.47)
3 7 10 18 2.56 (2.90)
0.000
0.2
0.002 0.002 0.008
0.2 0.05 0.436
0.012 0.003 0.053 0.001 0.0001
0.04 0.03 0.2 0.03 0.019
*Adjusted for age (continuous) and sex where applicable. **Adjusted for age (continuous), sex and education ( T 12 or 12 years).
to 8th grade group is more like the 5th grade group, whereas for other variables it is more like the 8th grade group.
3.4. Use of oral versus written materials Results of the comparison of reported frequency of using the printed and audiotaped materials in the CARDES educational package are shown in Table 4. This was a within-person analysis, restricted to those who received both printed and audiotaped information and who completed the 4-month follow-up interview. Persons with literacy scores T 8th grade were approximately twice as likely to report a greater relative frequency of using the audiotapes (P 5 0.03).
4. Discussion This brief instrument, or similar instruments developed for specific populations using this approach, can be used to classify patients into categories indicative of the ease with which they will understand printed information. The use of CVD-related vocabulary renders our instrument particularly relevant to clinical settings serving men and women with hypertension, elevated cholesterol, or heart disease, and, since prior knowledge did not seem to be a major influence on the final instrument, in many other circumstances in which this terminology is relevant. The apparent preferential use of the audiotaped versus the printed CARDES materials among those with reading scores at or below 8th grade supports, indirectly, the ability of this instrument to
148
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
Table 4 Reported use of CARDES Nutrition Guide and Audioseries four months after enrollment Literacy score (reading level)
Used Nutrition Guide more than Audioseries Used Nutrition Guide and Audioseries equally Used Audioseries more than Nutrition Guide
0–16 correct ( T 8th grade, n 5 48)
17–20 correct ( 8th grade, n 5 50)
19% 27 54%
28%* 44% 28%
}Subset of participants who were given both types of materials and who completed a four month follow up interview. *P 5 0.02 for difference between literacy score groups.
identify those less likely to use printed information even when it is designed to have a relatively low literacy demand. To the extent that this sample is representative of a population of urban African Americans seeking dietary counseling for the management of high blood pressure or high blood cholesterol, the 48% who scored at or below the 8th grade reading level is noteworthy; it suggests a large population in need of special educational approaches. The extent of the problem may even be underestimated in our sample. Adults with limited educational backgrounds tend not to participate in studies involving instruction [25]. Our advertisting for participants in ‘easy to read’ nutrition information probably selected for a higher distribution of literacy than applies in the community from which our participants were recruited. An inverse association of literacy skills with age is sometimes reported [1,4], although we did not observe this, possibly because the age range of our sample was restricted. We did observe a sex difference in literacy skills; the men in our sample were more likely to have low literacy scores. Others have reported that women have lower literacy than men [1]. Whether a sex difference is observed and of what nature may be population or sample dependent. Our data mirrored the positive correlation between literacy scores and level of education that has been reported in other research [1]. Correlates of lower literacy scores on this CVD nutrition-oriented instrument that were independent of education were moderate or heavy work activity, having a medical history of heart disease or diabetes, and a more depressed
psychosocial profile compared to persons with higher scores. The link between literacy and work activity, even when some adjustment has been made to control for educational level, may indicate a clustering of less educated persons with low literacy skills in jobs requiring manual labor. The associations with medical history and psychosocial status may reflect causes or consequences of health-related low literacy skills or both. It is also possible that these associations reflect a selection bias such that persons with low literacy skills who also had prior heart disease or psychosocial problems were more likely to enroll in our study than those with no such problems. The similarity of these findings with other reports in which low literacy has been associated with poor physical and psychosocial health [4,26] suggests that these findings may be meaningful. Limitations of this study include the relative small size and lack of diversity in the sample used to develop the word recognition test. The statistical analyses suggested that the instrument performs relatively well, within the limits reported, as a proxy for more formal reading assessments such as the TABE. In addition, the instrument generated a good distribution of scores in the more broadly-based CARDES population, was repeatable, and correlated with years of education at a level that is consistent with the literature [1]. Nevertheless, because word recognition may differ with age or with other population characteristics, validity testing in a larger and more diverse population would be desired. In particular, experience is needed in interpretationg this type of test in adults who are not native speakers of English or who speak dialects of English.
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
149
Table 5 Practice implications 8 Persons with lower literacy skills are at higher than average risk of health problems and are less likely than persons with higher literacy skills to use printed health information 8 Assessment of literacy skills facilitates identification of problems in reading health information that cannot be inferred from years of school completed 8 Brief word recognition tests can be used in clinical settings to assess functional literacy skills that are directly relevant to patient education 8 Audiotaped oral instructions may be well-received by patients who have problems reading printed information
5. Implications and conclusions There is increasing awareness of the link between literacy and health and the consequent need to identify patients who have problems reading standard health-related information (Table 5). Even easy-to-use literacy appropriate instructional materials may be less utilized by persons with lower literacy skills. Such individuals may rely on oral information sources such as radio or television [6] to supplement or substitute for advice they receive during face-to-face encounters with health personnel. The marked socioeconomic differentials in CVD mortality [27,28] and the apparent association of lower literacy levels with poorer CVD risk profiles underscore the need to consider literacy as one mediator of these differentials that can be addressed through screening and tailored programming. Specifically, identification of persons with problems reading printed information and perhaps a preference for oral instruction can lead to: (1) an increasing appreciation for the need to have alternative instructional formats available and (2) offering these formats to those who can most benefit from them. As the awareness of literacy issues and their importance increases, effective patient education approaches will increasingly include direct assessment of patients’ relevant literacy skills. Although literacy skills are associated with formal education, years of school completed is not an informative proxy measure for reading problems. Some persons who have completed high school have difficulty reading and some people with very limited formal education ready very well. Standardized reading tests designed to
diagnose specific problems or test reading comprehension are neither appropriate nor necessary in clinical setting where the objective is primarily to match patients to educational approaches that they will find acceptable and understandable. Word recognition tests as described here, that can provide reliable proxy indicators of reading problems, can be used effectively for this purpose and are relatively unobtrusive when based on content relevant to the clinical issues being addressed.
Acknowledgments This research was supported by grant number R01 HL46778 from the National Heart, Lung, and Blood Institute, National Institutes of Health. The authors are grateful to staff and students in Adult Basic Education programs who participated in the development of the literacy assessment instrument and to the many others involved in implementing the CARDES Program. Some of these results were presented at the 1993 and 1995 annual meetings of the American Public Health Association.
References [1] Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, DC: U.S. Government Printing Office, 1993. [2] Doak L, Doak C. Patient comprehension profiles. Recent findings and strategies. Patient Counseling and Health Education 1980; 2: 101–106.
150
T.R. TenHave et al. / Patient Education and Counseling 31 (1997) 139 – 150
[3] Hussey LC, Gilliland K. Compliance, low literacy, and locus of control. Nurs Clin North Am 1989; 24: 605–611. [4] Williams MV, Parker RM, Baker DW, Parikh NS, Pitkin K, Coates WC, Nurss JR. Inadequate functional health literacy among patients at two public hospitals. JAMA 1995; 274: 1677–1682. [5] Doak CC, Doak LG, Root JH. Teaching Patients with Low-Literacy Skills, 2nd Ed. Philadelphia, Pa: JB Lippincott; 1995. [6] Weiss BD, Reed RL, Kligman EW. Literacy skills and communication methods of low-income older persons. Patient Education and Counseling 1995; 25: 109–119. [7] Glanz K, Rudd J. Readability and content analysis of printed cholesterol education materials. Patient Education and Counseling 1990; 16: 109–117. [8] Mallett L, Spruill WJ. Readability evaluation of nine patient drug education sources. Am Pharm 1988; NS28: 33–39. [9] Petterson R, Visuals for information: Research and practice. Educational Technology Publication. Englewood Cliffs, NJ 07632. Chapter 3, Literacy; 1989. [10] AMC Cancer Research Center. Beyond the Brochure: Alternative Approaches to Effective Health Communication. Denver, CO 1993. [11] Gagliano ME. A literature review on the efficacy of video in patient education. J Med Ed 1988; 63: 785–792. [12] Murphy PW, Davis TC, Long SW, Jackson RH, Decker BC. Rapid estimate of adult literacy in medicine (REALM). A quick reading test for patients. J Reading 1993; 37: 124–130. [13] Nurss JR, Baken DW, Davis TC, Parker RM, Williams MV. Difficulties in Functional Health Literacy Screening in Spanish-Speaking Adults. J Reading 1995; 38: 632– 637. [14] Davis TC, Long SW, Jackson RI, Mayeau EJ, Crouch MA. Rapid estimate of adult literacy in medicine. A shortened screening instrument. Fam Med 1993; 25: 391–395. [15] Parker R, Baker D, Williams M, Nurss J. The Test of
[16] [17] [18]
[19] [20]
[21]
[22]
[23]
[24] [25]
[26]
[27]
[28]
Functional Health Literacy in Adults (TOFHLA): a new instrument for measuring patients’ literacy skills. J Gen Intern Med 1995; 10: 537–545. Tests of Adult Basic Education (TABE) Monterey, CA: CTB / McGraw-Hill, 1987. Tests of Applied Literacy Skills (TALS), New York: Simon and Schuster Workplace Resources, NY, 1991. Adult Basic Learning Examination (ABLE), San Antonio, TX: The Psychological Corp., Harcourt Brace and Co, 1986. Slosson RJL. Slosson Oral Reading Test – Revised. East Aurora, NY: Slosson Educational Publishing 1990. Stanovich KE. Word Recognition: Changing Perspectives. In: Barr R, Kamil ML, Mosenthal P, Pearson PD, editors. Handbook of Reading Research, vol. II. New York: Longman, 1991: 418–452. Efron B. The jackknife, bootstrap, and other resampling plans. Society for Industrial and Applied Mathematics: Philadelphia, 1982. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. Chinchilli VM, Martel JK, Kumanyika MS, Lloyd T. A weighted concordance coefficient for repeated measures designs. Biometrics 1996; 52: 341–353. Beck AT. Depression: Clinical, Experimental and Theoretical Aspects. London, 1969. Darkenwald GG, Merriam SB. Adult Education: Foundations of Practice. New York: Harper and Row, 1982. Weiss BD, Hart G, Pust RE. The relationship between literacy and health. J Health Care Poor Underserved 1991; 1: 351–363. Burt VL, Whelton P, Roccella EJ, et al. Prevalence of Hypertension in The U.S. Adult Population. Hypertension 1995; 25: 305–313. Saunders E. Tailoring Treatment to Minority Patients. Am J Med 1990; 88: 3B–21S.