Errors in interpreting quantities as procedures: The case of pharmaceutical labels

International Journal of Medical Informatics 65 (2002) 193 /211 www.elsevier.com/locate/ijmedinf

Errors in interpreting quantities as procedures: The case of pharmaceutical labels Vimla L. Patel a,*, Timothy Branch b, Jose F. Arocha c a

Departments of Medical Informatics and Psychiatry, Laboratory for Decision Making and Cognition, Columbia Presbyterian Medical Center, Columbia University, Vanderbilt Clinic-5, 622 West 168th Street, New York, NY 10032-372, USA b Centre for Medical Education, McGill University, Montreal, QC, Canada c Department of Health Studies and Gerontology, University of Waterloo, Waterloo, ON, Canada Received 24 January 2002; received in revised form 28 July 2002; accepted 8 August 2002

Abstract The purpose of this study is to investigate and characterize the errors in cognitive processes deployed in the comprehension of procedural texts found on pharmaceutical labels by subjects of different cultural and educational backgrounds. In this study, participants were asked to read and interpret three pharmaceutical labels related to children’s medications of varying complexity: oral rehydration therapy (ORT); over-the-counter cough medicine; and over-the-counter fever medicine. Results indicate that: (1) all groups of participants had considerable difficulty in interpreting the instructions; (2), cultural and educational background appeared to be only weakly related to the accuracy of dosage and administration; and (3) errors of comprehension were related to three features of the texts: situation-representational complexity, inherent quantification complexity, and conformity with intuitive models of therapy based on prior knowledge. The results are discussed in terms of the role of multiple representations (boundary objects) and theories of text comprehension to facilitate the reduction of errors. # 2002 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Medication errors; Drug labeling; Comprehension; Mental processes; Cognitive science; Over-the-counter drugs; Text design

1. Introduction Everyday non-specialists are called upon to follow instructions in a variety of application domains (e.g. completing income-tax forms,

* Corresponding author. Tel.:
/1-212-305-5643; fax:
/1212-305-3302 E-mail address: [email protected] (V.L. Patel).

configuring and using a VCR, or cooking something for the first time), where accurate comprehension is necessary for successful action. A problem arises because comprehension of instructions sometimes involves the integration of quantitative and qualitative information. This is nowhere more apparent, and critical, than in the case of pharmaceutical information, the correct use of which often demands that the user translate quanti-

1386-5056/02/$ - see front matter # 2002 Elsevier Science Ireland Ltd. All rights reserved. PII: S 1 3 8 6 - 5 0 5 6 ( 0 2 ) 0 0 0 4 5 - X

194

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

tative formulas into qualitative, and frequently complex, procedures. Such translation is bound to generate errors of interpretation, as evidenced by the frequency of medical errors involving the use of therapeutic drugs [1]. An understanding of the way in which instructions are understood and acted upon is crucial for the design and development of patient-centered information systems that are optimally adaptable to the users needs. Furthermore, as patient health information systems become more widely available [2,3], key steps in reducing medication errors associated with poor understanding and use of pharmaceutical instructions must be taken. Investigating the ways in which users of overthe-counter drugs interpret label information when administering therapy and designing information systems that can support correct interpretation are two aspects that demand attention. In this paper, we investigate this by using theories and methods from the domain of health cognition Health cognition research looks at the psychological processes underlying health care performance. It does this via an in-depth analysis of the perceptual and cognitive processes that lead to observable behavior. The focus is on understanding the knowledge structures and mental processes brought to bear during cognitive activity (e.g. problem solving, decision making). Although substantial individual variation can be found through this method, cognitive theories and methods allow us to capitalize on certain structural and processing regularities of the human information processing system, which give strength to generalizations. Health cognition is a discipline that draws on theories and methods from cognitive science. Although great technological advances have been made in the development of health information systems, the design of simple, everyday functional tasks is still at its infancy. These designs could and

should be informed by cognitive constraints that users impose on the systems with which they interact. In this paper we explore these constraints in the contexts of lay peoples’ use of pharmaceutical information given on the over-the-counter drug labels and how these constraints may inform safer systems design for lay use of health care information.

2. Background Research on the interpretation of instructions on pharmaceutical labels is surprisingly limited. Whereas most of the reported research focuses on the perceptual aspects [4] of pharmaceutical instructional texts (e.g. font size and text legibility), we have very little information on the cognitive aspects involved in the interpretation of pharmaceutical instructions, especially those involving quantitative information. Interpretation of information on pharmaceutical instructions depends on the user’s ability to relate formal expressions in the text (e.g. dosages in ml) to an appropriate referent situation (e.g. amount of drug to be used and the conditions under which it is to be administered) [5 7]. Furthermore, correct interpretation of a text demands that the referent situation that is identified by the user be congruent with the situation intended by the author of the text. A problem is that texts are not static objects with fixed meanings, but instead they allow for a variety of interpretations [8]. For most texts, because of ambiguity or unfamiliarity, there is great latitude in interpretation and actual congruence is rarely attainable. In the case of emergency procedures or medication therapies, failure to apprehend the situation that the text presupposes may make it impossible to take the appropriate action. For example, pharmaceutical labels may refer to unfamiliar units of measurement and substances, leaving /

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

the user with little knowledge to rely upon when interpreting these texts. When faced with such difficult or unfamiliar information, users are forced to rely on their prior beliefs [9] or on their knowledge of the domain [10], which may be biased or mistaken. However, there should be very little room for error when interpreting pharmaceutical instructions for the purpose of treatment, where incorrect interpretation has potentially fatal consequences [11]. This is particularly important since tasks that call for the translation of knowledge into an appropriate course of concrete action, such as the interpretation of pharmaceutical labels, are the most vulnerable to human error [12]. There are three representations that must be considered when designing and evaluating the use of pharmaceutical labels [13]: (1) the representation of the situation as understood by the pharmaceutical manufacturer/designer; (2) the external representation; i.e. the one present in the instructional text; and (3) the representation of the situation as derived by the user. The representation of the situation as understood by the designer has an anchoring effect on the instructional text, where the anchoring situation is an ideal representation of the action of the drug which often involves the interaction between quantitative variables, such as body weight and serum-concentration levels. This representation is highly abstract and is often described using quantified, continuous mathematical models. Conversely, the representation of the situation as derived by the user is essentially procedural, applying discrete doses of medication to people with discrete bodies. For example, when interpreting pharmaceutical labels, quantitative expressions contained in the text are used to determine dosages (e.g. give 2 ml of the syrup per 1 kg of body weight), which then must be integrated into the administration of the medication (e.g. give x number of teaspoons).

195

Thus, the interpretation task requires the translation of a quantitative model into a procedural one, where the external representation contains the information about the translation parameters, as it acts as a bridge between the designer and the user. The interpretation of instructions is further complicated by the fact that the user brings an intuitive, a priori model of the therapeutic situation to the task [14]. This model is based, in part, on past experiences with medication and treatments, subjective evaluations of previous responses to drugs, and an understanding of the pathophysiological and pharmacological processes involved. For example, research has found that users rely on intuition and prior knowledge when formulating procedures for administering medication, particularly with complex information [15 17]. Therefore, the user’s task can be thought of as an integration of selective quantitative data from the instructional text into an intuitive model of the situation. The resulting application situation that is derived can be regarded as a composite model of the therapeutic situation, where the actual therapeutic situation consists of quantitative, procedural, and a priori representations. Thus, comprehension is measured in terms of how well the composite models approximate the ideal models of the therapeutic situation. The present study examines the roles of these three types of representations as related to text comprehension and errors in interpretation. It is essential for the design of instructional information to provide different types of information to people with divergent viewpoints while being able to retain their integrity across time, space, and local contingencies. Since different types of information mean various things depending on context, users are faced with the task of reconciling these multiple meanings if they are to correctly comprehend the text. This can be aided /

196

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

through the active co-operation and communication among all of the intended audiences in order to promote mutual learning between communities [18]. Although instructions that support multiple interpretations may increase the difficulty of the written information for some users, they also allow for certain benefits. For example, these texts are used to establish a common understanding among pharmacists, physicians and patients about the action of a particular medication. 2.1. Pharmaceutical procedures as boundary objects To understand how representations are able to afford multiple interpretations across domains while maintaining coherence, these texts may be construed as ‘boundary objects [19]’. These are artifacts, documents, or texts that by supporting multiple representations allow for shared understanding among different communities (e.g. physicians, nurses, patients). Boundary objects are said to be ‘plastic’ in that they adapt to the local needs and the constraints of those employing them, yet ‘robust’ enough to maintain an identity across sites [20]. This allows them to be recognizable in different social settings, where they may be translated according to the user’s needs and serve to act as interfaces that facilitate the flow of information between multiple communities. Their boundary nature is reflected by the fact that they are simultaneously concrete and abstract, specific and general, conventionalized and customized. For example, the same set of engineering drawings may be used in different departments of an airplane manufacturing company, including the shop floor, engineering design studio, accounting department, inventory management office, and structural engineering bureau [21,22]. The same information that is contained in the drawings is used and

interpreted differently in multiple domains to meet a variety of goals. Boundary objects are both concrete and abstract in the sense that the same concrete information is abstracted and interpreted differently by a variety of people. They are both conventional and customized in that they are presented in a conventional format that is understood across domains, which is then reinterpreted and customized to meet specific goals within each domain of use. Similarly, instead of having a fixed meaning, pharmaceutical labels construed as boundary objects are able to convey different types of information to different people in different communities, such as pharmacists, physicians and patients. To create shared understanding pharmaceutical instructions, conceived of as a boundary object, should serve a variety of goals, where a representation appropriate to the end-user should be complemented by aspects of the representations appropriate to the designer of the instructions and to the administrator of the drug. In this way, a composite model, which is sensitive to the informational requirements across multiple domains, is developed that has aspects of all people involved in its production and use (e.g. designer, physician, patient). However, the same ambiguity can be disadvantageous in that it can also promote interpretations that are very different, with little point of contact among its users. This may be the case of many pharmaceutical labels.

3. The study purpose and rationale One problem with procedural texts is that they are often designed such that multiple representations are not readily facilitated (e.g. by failure to provide a context to which different people can relate to). One reason may be that labels are designed by specialists

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

without concern for who the end-users will be or without any input from them. For example, mothers have been found to experience difficulty in performing the seemingly simple operations associated with preparing solutions from packaged mixtures for an oral rehydration therapy (ORT) in the treatment of diarrheal dehydration [17]. The effective use of ORT requires some rudimentary knowledge of the physical and biological causes of diarrheal diseases, the capacity to make inferences from this knowledge, and skill in preparing and administering medicines. Thus, in situations where the user lacks such knowledge, pharmaceutical labels fail to support multiple representations that would allow for the correct interpretation. If pharmaceutical labels lack certain information necessary for multiple representations among all of the communities involved, their interpretation may lead to error. A second problem that arises in the interpretation of pharmaceutical labels is in the application of information contained in these texts to the appropriate real-world situation to which they refer. A lack of specific knowledge related to these referent situations will result in problems of interpretation [23]. For example, when university biology students were compared to practicing scientists in their ability to interpret graphs, the lack of knowledge of specific examples in particular natural settings that the scientists appropriated through familiarity with their domain was the origin of many of the problems the students experienced [24]. To administer a medication successfully, the user must have sufficient interpretive resources to identify the referent situation intended by the pharmaceutical label. This involves both personal experiences in, and familiarity with, the intended domain, where breakdowns in understanding may be related to a lack of experience [25]. This is cognitively

197

expressed as a failure to account for anomalies in the user’s understanding of the text [26]. However, when administering a medication for the first time, users do not have such experience readily available. Consequently, the representation of the procedure that the user develops may not be the most effective representation for the proper execution of the procedure. This problem is further complicated since what is left out of these texts is often what is critical for understanding the reasons for the procedure. It should also be noted that in such cases irrelevant and potentially misleading information is frequently retained as part of the user’s understanding of the text [27]. Aside from the inherent difficulties in understanding written instructions, there is the added problem that the beliefs that users of pharmaceutical products bring to bear in interpreting label information are extremely varied and originate from different sources. These beliefs, in the form of prior knowledge, constrain how the instructions will be interpreted [16]. Some of these beliefs originate in mass-media information, whereas others may originate in traditional beliefs or belief systems that have been either systematically codified (as in the case of Ayurveda [14] medicine) or unsystematically passed from one generation to another (as in the case of the ‘grandmother’s remedies’ and lay health beliefs in many Western and non-Western societies [17]). The purpose of this study is to explore and characterize the errors in cognitive processes deployed in the comprehension of procedural texts found on pharmaceutical labels by subjects of different cultural backgrounds. Our goal is therefore to characterize problems in their interpretation and understanding rather than to compare the groups involved. Specifically, two issues will be addressed: (a) the difficulty of interpreting procedural texts will

198

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

be assessed in order to explore aspects of the therapeutic situation (i.e. the text) that may make them difficult to understand; and (b) the role that prior cultural beliefs about health and disease causality and treatment play in the interpretation of pharmaceutical labels. The study presents an in-depth look at different labels (which can be seen as three case studies) that were found on three actual pharmaceutical products.

4. Method This study consisted of three groups of participants who were asked to read and interpret pharmaceutical labels for treatments of varying complexity. The participants were volunteers, mostly mothers with small children, and included people of diverse ethnic and cultural backgrounds. Their educational backgrounds ranged from no formal education to graduate level education.

medications for children: cough mixture and antipyretic drops. An effort was made to select over-the-counter medications that were the most widely used for children in Canada at the time the studies were conducted. For this, the first author consulted with pharmacists regarding the most effective medications for children with coughs and fever. Again, as in the first label, no modifications were made to the labels on the medication for our study. Participants for the second label involved a sample of 25 English, seven East Indian, and 16 Greek parents, all living in Canada, with a total of 48 participants. Most of the participants in the sample were well educated, where the majority had undergraduate and graduate degrees. The third label used involved a sample of eight English, seven East Indian, and 16 Greek parents, all living in Canada, with a total of 31 participants. The education level in this sample varied from participants with little or no formal education to those with high school and graduate degrees.

4.1. Materials and participants 4.2. Procedure The first label used was on the package of an ORT solution widely used in Kenya, Ethiopia, and other parts of Eastern Africa. The label used for the study was selected among the ones recommended for sale in ‘dukas’ or convenience stores common in rural Kenya. In order to preserve the ecological validity of the research results, the labels were not modified in any way for this investigation. The study involved a sample of eight rural and urban Kenyan mothers who were living in Africa, and five urban Canadians, with a total of 13 participants. Most of the participants in the sample had little formal education, although a few had high schoollevel training and graduate degrees. The second and third labels were also taken from commercially available over-the-counter

Each participant was interviewed individually, with the help of research assistants who had familiarity with the individual cultures. Specifically, responses to questions that focused on the causes of the corresponding childhood disease (diarrhea, respiratory illness such as coughs and colds, and fever) and the appropriate treatment modalities were elicited. The participants were then asked to read and interpret manufacturer-supplied written instructions for the target medication. The subjects were asked to calculate the amount of medication to be given to the youngest child in their family. The participants were asked to think aloud as they read the instructions. All interviews were taperecorded and transcribed for analysis.

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

4.2.1. Label 1: Instructions from oral rehydration therapy In the first label, the participants were asked to read and interpret the instructions for an ORT used in the treatment of dehydration in children with diarrheal diseases. The data in Africa was collected by the first author. Implementing the treatment correctly required the preparation of sterile media and the administration of a constant dosage of medication at irregular intervals. Fig. 1 gives

199

the idealized course of treatment for the ORT, and, from the point of view of medication levels, this consists of three stages of therapy: rapid rehydration to replace lost fluid volume and serum electrolyte, high fluid-level maintenance to compensate for fluid and electrolyte loss during the period when the disease is still active, and lowered fluid-level replacement as the disease moderates. This information is provided on the label as indicated at the bottom of Fig. 1. This course of treatment

Fig. 1. Idealized dosage of medication and pharmaceutical instructions for ORT. The solution replaces body water and body salts lost during diarrhea. How to use this solution (for children up to 5 years old). Fill tumbler with water up to mark (300 ml). Add all powder from sachet. Stir. Give two or three tumblers during the first 4 /6 h. Give two or three more tumblers over the next 18 /24 h. Give two more tumblers in the following 24 h. Do not give more than six tumblers in 24 h. Important: Always use as instructed unless otherwise directed by your doctor. Give slowly to prevent vomiting during treatment. Use clean spoon to give the solution to small babies. If baby is thirsty between drinks of the solution give plain boiled and cooled water. Begin normal feeding as soon as possible. Warning: See a doctor whenever diarrhea is severe or it is has not stopped in 2 days.

200

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

presents a complex usage picture, which consists of two components: preparation of the solution and administration. Accuracy was coded in terms of four steps, where the first step represents the correct preparation of the solution and the following three steps represent the correct administration for each of the three stages of therapy. Thus, accuracy scores were generated for each participant based on the number of steps they correctly completed. 4.2.2. Label 2: Instructions from over-thecounter cough syrup for children In the second label, the participants were asked to read and interpret the instructions for the cough medicine for their young children. The therapy involved a relatively simple procedure but required a complex calculation to determine the proper dosage based on the weight of the child. The idealized course of treatment for the cough syrup, from the point of view of total daily dosage as determined by body weight, is represented in Fig. 2. Specifically, the total daily amounts are achieved by periodic doses of medication, where each dose is a fraction of the total daily dosage. The text provided with the cough medication is given at the bottom of Fig. 2. Fig. 3 represents the calculations necessary in determining the appropriate dosage of cough syrup and the correct calculation of dosage for a 22-pound child. Specifically, the formula for determining the volume, in teaspoons, per single dose of medication can be reduced to an expression based on the weight of the child, in kilograms, divided by the number of times per day that one chooses to administer the medication and by 15 mg to convert the dosage into teaspoons. The participants’ responses were assessed for accuracy, where the calculated amount of medicine was coded as an under-dose, the correct dose, a slight overdose, or an extreme overdose.

4.2.3. Label 3: Instructions from over-thecounter antipyretic drops for children In the third label, the participants were asked to read and interpret the instructions for fever medication. The implementation of the therapy involved a relatively simple procedure, where there was essentially no calculation of dosage, but required determining the appropriate medication plan for the child. As given in Fig. 4, the implementation of this medicine involves a monotonic relationship between the age of the children and the maximum single dose of medicine to be administered, as seen in the text. A dropper calibrated in milliliters was provided to aid in the calculation of the dosage, where the participants were required to give the medicine four-to-five times daily. The participants’ responses were assessed for accuracy, where the overall amount of medicine based on the number of times it was given daily was coded as either an under-dose, the correct dose, a slight overdose, or an extreme overdose. Consult a physician if the underlying condition requires use for more than 5 days. It is hazardous to exceed recommended dose unless advised by a physician. In order to assess any differences in performance between the participating groups, data from all the pharmaceutical labels were subjected to statistical analyses. Specifically, for the data from ORT, T test was used in order to determine if performance differed significantly as a function of cultural background (Kenyan or Canadian) and/or level of formal education (graduate degree or all other levels). For the data from the ‘over-the-counter’ cough syrup and antipyretic medication, analysis of variance (ANOVA) was used to determine if performance differed significantly as a function of cultural background (Canadian, Greek or East Indian) and/or the level of formal education of the participants (graduate

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

201

Fig. 2. Idealized total daily dosage of medication and pharmaceutical instructions for over-the-counter cough syrup. Each teaspoonful (5 ml) contains 15 mg of dextromethorphan hydrobromide U.S.P., in a palatable yellow, lemon flavored syrup. Dosage adults: 1 or 2 teaspoonfuls three or four times daily. Dosage children: 1 mg per kg of body weight daily in three or four divided doses.

degree, undergraduate degree, high school or all other levels).

5. Results The results from the use of the each of the three labels are documented separately and then some attempts are made to tie them together in discussion of results 5.1. Procedural complexity in the instructions for ORT The mean accuracy of performance index was calculated as a function of the participants’ level of education (graduate degree vs. all other levels) and cultural background

(Canadian vs. Kenyan). Specifically, in determining the score for each participant, a score of one represented the correct preparation of the solution but an incorrect administration for all three stages of therapy, while a score of four represented the correct preparation of the solution and the correct administration for all three stages of therapy. The results show that all participants read and interpreted the preparation instructions for the ORT correctly, regardless of cultural background and level of formal education. However, only three of the eight (37.5%) Kenyan mothers were able to correctly administer the treatment for even the first stage of therapy (M 1.12, SD 0.35). On the other hand, the Canadian mothers were more accurate in their administration of the treatment (M 2.8, SD  1.30). Thus, the /

/

/

/

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

202

Fig. 3. Calculation of a single dose of cough syrup for a 22-pound child based on the instructions.

Canadian participants showed significantly higher accuracy than the Kenyan participants, [T(1, 11) 3.52, P B 0.00]. Furthermore, only those with graduate degrees were able to correctly interpret the administration instructions for all three stages of the therapy based on the pharmaceutical label (M  4.0, SD 0.00), while those with lower levels of education did not (M 1.36, SD 0.67). When compared on the basis of their level of formal education, those participants with graduate degrees performed significantly better than all others, [T(1, 11) 5.33, P B 0.00], showing /

/

/

/

/

/

/

/

that it needs a qualified academician to accurately follow the instructions. The majority of participants (76.9%) were unable to correctly administer the ORT therapy, where performance appeared to be weakly related to the cultural background and level of formal education of the users. An analysis of the transcripts revealed that some participants had difficulty in determining the pacing required to administer the ORT therapy accurately. This is illustrated in the transcript of one of the Canadian mothers with a high school education, who was unable

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

203

Fig. 4. Idealized dosage of medication and pharmaceutical instructions for over-the-counter antipyretic drops. Each 1 ml dose contains: 80 mg acetaminophen. Indications: For fast and effective relief of children’s fever and pain. Dosage: Administer single dose orally according to age as listed, four to five times daily, for maximum of 5 days.

Age (years)

Maximum single dose

Under 2 years 2 /3 4 /5 6 /8 9 /10 11 /12

As directed 2.0 ml (160 3.0 ml (240 4.0 ml (320 5.0 ml (400 6.0 ml (480

by Physician mg) mg) mg) mg) mg)

204

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

to determine the appropriate dosage plan described in the text: Give them 300 ml of water in a glass, and you empty the packet in the glass and stir it. And you give them 4 6, no, 2 4 glasses in the first 4 6 h, and then 4 6 in the next 18 24 h, and then 2 4 in the 24 h after that. /

/

/

/

/

/

Thus, the non-uniformity in administration procedures made it difficult to follow the instructions in a consistent way. 5.2. Quantitative complexity in the instructions for cough syrup The results show that, overall, the majority of the participants (56.25%) were unable to calculate appropriate dosages for their children, where the statistical analyses revealed no significant differences in performance based on cultural background or level of formal education. Specifically, among the 25 English participants, five (20%) calculated amounts of medication representing an extreme overdose, where the calculated dosage fell in the upper range for heavy adults; six (24%) calculated amounts of medication representing a simple overdose, where the calculated dosage fell in the range for older children or lighter adults; and one (4%) derived an amount of medication representing an under-dose. Only 13 (52%) of the English participants were able to calculate the appropriate dose of medication. Similar results were obtained from the East Indian and the Greek parents. Among the seven East Indian participants, two (28.6%) calculated doses representing an extreme overdose, two (28.6%) with a simple overdose, and one (14.3%) representing an under-dose of the medication. Of the 16 Greek parents, one (6.25%) participant’s calculation represented an extreme overdose, three (18.75%) simple overdoses, and two (12.5%) with an under-dose of medication.

A qualitative analysis of the responses suggests that the principal obstacle in correctly interpreting the instructions appears to be the overall complexity of conversions of quantities. For all participants, the calculation of an individual dose of cough syrup in teaspoons required converting the child’s weight in pounds to weight in kilograms, weight in kilograms to total daily dosage in milligrams, total daily dosage in milligrams to total daily dosage in teaspoons, and total daily dosage in teaspoons to individual doses in teaspoons. The conversions necessary in the calculation of a single dose of cough syrup are best exemplified in the following excerpt, where an English mother with a graduatelevel education (MA) determines the dosage for her child: 27 kg, 27 mg? 27 mg daily. Each teaspoon is 15 mg so you want two teaspoons a day divided by four. Hold it, two teaspoons a day divided by three or four doses. So you are talking either one-half to two-thirds a teaspoon, depending on whether it is three or four times a day. She was able to accurately perform all of the necessary conversions in determining the dosage, which ends in 1/2 or 2/3 teaspoon 3 4 times a day. Even if one can calculate, it is impossible to measure such a dose for accurate administration. A more typical response can be seen in the following excerpt, where a Greek mother with a high school education attempts to determine the correct dosage of cough syrup for a child that weighs 10 kg: /

Each teaspoon contains 15 mg? Okay, for the children you have to write the body of the baby, 10 kg. You have to give it like that. Ten kilograms, I give him only two teaspoons for the day. Yeah, I give him only two teaspoons, two times a day, because you see here 15 mg is one teaspoon. So, one teaspoon is five ml

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

contains 15 mg? Yeah, that’s it. I give him only two, two teaspoons. She was unable to correctly perform a number of the conversions necessary in determining the correct dosage. Indeed, the resulting dosage is three times as much medication as the correct dosage for a child that weighs 10 kg. The problems that the participants experienced in interpreting the instructions may be broken down further into specific problems with several of the conversions necessary in the calculation of dosage. The most consistent problem involved the translation from weight in pounds to weight in kilograms. After first determining the dosage using the child’s weight in pounds and yielding an incorrect calculation, one parent made the following comment: I was thinking pounds not kilograms. I do not know how to convert pounds by kilograms, I do not know. I would have to find out how to calculate it. They should give this in pounds, instead of just in kilograms. Without the information necessary to convert pounds into kilograms she was unable to determine an accurate dosage of medication. Furthermore, she went on to mention that the pharmaceutical label did not provide any alternative interpretations for those who lacked the information the text presupposes, stating: The instructions are not too clear and they do not give you any choices. You only have one way of doing it, one way of measuring it. What needs to be added is they should put more alternatives. They should put, like, more ways of doing it, measuring it. So if you do not understand that way. You understand it but you just do not have the measurement. But they should give you a different way.

205

Even among those who calculated appropriate dosages, 10 (55.6%) gave the dose in terms of weight, i.e. milligrams, instead of a volumetric unit. This would clearly provide another opportunity for error in the administration of medication in these cases since they would have to use a volumetric vessel, such as a teaspoon, to dispense the medication. For example, in the following excerpt an English mother with an undergraduate degree (BA) was able to correctly calculate the appropriate dosage for her child in milligrams, but was unable to convert this amount into an equivalent measurement in teaspoons: One milligram per kilogram of body weight. So 13, 13 daily, 13 mg it would be. One milligram per kilogram? 13 kg let’s say and one times 13 is 13 daily in three or four divided doses. So it would be four doses, I guess, 3 mg each? How many milligrams in a teaspoon? Oh, gosh, I said 3 mg. Each teaspoonful there is five to a teaspoon. Oh, I do not understand. Well, it would not be a full teaspoon it would be more like half a teaspoon, four times a day. Although she determined the correct amount of medication in milligrams, a total daily dosage equal to approximately four-fifths of a teaspoon, the resulting dosage in teaspoons represented more than twice this amount. In addition, errors consistently occurred when converting the total daily dosage of cough syrup into individual doses. A typical response that was given by one person when he was asked to calculate a single dose of cough syrup for a child who weighed 30 pounds was: Well, it is 1 mg per kg so, well 30 pounds is about 15 kg, maybe 14 kg. So if it is that I would give him 15 mg, I guess three or four times a day. He was able to correctly determine the total daily dosage of cough syrup to be given to a

206

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

child who weighs 30 pounds, but he failed to convert this dosage into fractional amounts to be given in single doses. Such a therapy plan would therefore result in the administration of three to four times the recommended daily dosage of cough syrup. When probed about the difficulty of the calculations, he responded that ‘the dosage is very easy to understand’.

5.3. Intuitive complexity in the instructions for antipyretic medication The results show that the participants had no difficulty in calculating an accurate singledosage amount of medication for the child. They simply had to find the child’s age in the instruction table, which was provided, to determine the corresponding amount of medication. Nonetheless, many of the participants (67.6%) still planned therapy schedules that resulted in the administration of inaccurate amounts of medication. Specifically, the participants had a tendency to under-medicate the child, unlike the cough-syrup label, where the participants tended to overmedicate their children. Also, the statistical analyses revealed no significant differences in performance related to cultural background or level of formal education. Analysis of the data show that the problems for the participants arise when dealing with the application level of therapy, rather than with the quantification level. In this case, the text instructs the reader to administer the medication four-to-five times daily. However, the participants consistently said that this struck them as ‘too much’. Among the 31 participants, three (9.7%) would give the medication only once daily, two (6.4%) would give the medication only two-to-three times daily, six (19.3%) said they would give the medication three times daily, six (19.3%) said they would give the medication three-to-four times daily, and three

(9.7%) said they would give the medication only ‘when needed’. Therefore, the resulting total daily dosage is less than the dosage recommended. Only one (3.2%) overdose was noted, where the mother said she would give her child five-to-six daily doses of medicine. The results of the interviews confirm that there were no difficulties with the quantification and the usage picture, but that there were problems at the application level. The interaction of instructional data with intuitions about the application of antipyretic medicine and its effect on fever is demonstrated in subjects’ transcripts. For example, one of the Greek parents who had a college diploma stated: I would give him 3 ml using the pharmaceutical measuring spoon I have. I only give fever medicine when it is necessary. I do not believe in giving a lot of medicine to the children. I am really cautious when it comes to that, I only treat the fever when it needs it. The major error appears be related to the number of times administration is recommended in the instructions, which was perceived as being too high, where the participants’ intuition was to give the antipyretic drops less frequently than recommended on the pharmaceutical label. As seen in the excerpts, the result of this problem is a consistent tendency to under-medicate the children. The extent of this problem is best demonstrated by one of the English participants with a graduate degree, who would not administer the medication as often as directed even after being told by a physician, saying: The doctor told us how much to give her, but I would not give it to her five times a day. The maximum four and probably we might give it to her twice in the daytime and once before she went to bed. I would not give it to her unless I

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

thought she needed it. I have never given it five times a day. Many of the parents said they would give the antipyretic drops based on when they felt the child ‘needed it’. This would necessarily involve prior intuitions since knowledge about the symptoms, causality and treatment of the common fever would be required to determine this. Although in the case of antipyretic drugs this intuition would correspond to most current advice1, there may be cases where information failing to support users’ understanding may pose danger to health. Therefore, it appears that instructions which are incongruent with the user’s intuitive representation of the application situation tend not to be followed, where people generally fall back on prior intuitions when uncertain about specific procedures.

6. Discussion The results of this study indicate that all groups of participants had considerable problems in the interpretation of pharmaceutical labels. This problem appears to be universal and independent of culture or the level of education. Our analysis of the transcripts suggests that task difficulty and the errors in the interpretation of pharmaceutical instructions result from a failure of the texts to act as boundary objects. Specifically, we argue that the failure is due to three distinct features of these texts: (1) situation-representational complexity; (2) inherent quantification complexity; and (3) conformity with intuitive models of therapy. 1 We thank an anonymous reviewer who pointed out the non-standard nature of the label on the antipyretic drug used in this study. In this case, the medication approach used by parents were more in line with pediatric use of the drug than the instructions provided in the label.

207

We suggest, first, that situation-representational complexity led to errors in the interpretation of the instructions for the ORT, because the participants had great difficulty in establishing an appropriate procedural framework based on the usage picture with nonuniform procedures found in the text. Second, inherent quantification complexity led to errors in the interpretation of the instructions for the over-the-counter cough syrup in cases where the participants were unable to calculate the appropriate dosages based on the complex quantification-interpretation formulas found on the label. Third, a lack of conformity with intuitive models of therapy (and therefore a failure of the text to support multiple representations) led to errors in the interpretation of the instructions for the overthe-counter fever medicine, where the prior knowledge that the participants brought to the task collided with the identification of an appropriate therapeutic situation given on the instructions. Based on these results, a classification of the difficulties in interpreting pharmaceutical information can be related to three features of the therapeutic situation presupposed in the label information: (a) the uniformity of the application procedure; (b) the complexity of the quantified variables; and (c) the congruency with intuitive models of therapy. Uniformity of application may be violated when there are irregular intervals or varying amounts of medication between doses. Complexity of quantified variables can take the form of inherently difficult conversions, such as converting milligrams to milliliters, or too many calculations. Congruency with intuition may be violated when procedures or their instruments must be applied in non-standard ways, such as when the recommended frequency of administration exceeds the user’s intuitive representation of the application situation. Furthermore, the results of this study suggest

208

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

that individual biases, prior knowledge and intuition play a strong role in interpreting textual instructions for medication. Finally, the ability to accurately integrate quantified variables into a procedural mental model appeared to vary weakly with the cultural background and level of formal education among the participants in this study. It is only at higher levels of formal education that the representations of the participants more closely matched those of the designers of the instructions. Similar differences have been reported between lay people and professional biologists [28] and between students and practicing scientists [29].

6.1. Implications for information design of medication instructions If procedures, such as medication information, are to be used accurately and efficiently across multiple domains and communities, by pharmacists, physicians and patients, it is essential that the informational requirements of each domain be satisfied. These requirements are not met by ‘static’ instructions, such as those found on the labels investigated. Given that it is at the most critical stage, namely the point of administration, where the greatest gulf between the representational practices of users and designers exist, a focus on the design of information systems based on boundary objects offers new opportunities for bridging these gaps, as some research on information system design suggests [30,31]. A solution to the problem of difficult texts in the specific case of pharmaceutical instructions must encompass the following: making changes in instructions to eliminate complexity and extraneous information; paying attention to the subjective biases and intuitive models that people bring to instruction interpretation tasks; and providing a context /

for interpretation that brings out the aspects of the underlying model of the designer that are useful for the user’s interpretation, but also decreases the cognitive load involved in processing such instructions. These changes would likely provide coherence to the text by filling in the users’ knowledge gap while decreasing the likelihood of making incorrect inferences [32]. Given that errors in the administration of medications are commonplace in hospitals, nursing homes, and other health-care settings, improvements in the design of pharmaceutical labels are critical [12]. In addition, such improvements would allow the users of pharmaceutical products to learn from their usage with these labels and transfer what they have learned to other similar situations [32,33]. As we pointed out above, the ability to integrate quantities into a plan for therapy does not appear to be a function of either education or social acculturation. Pharmaceutical labels that are written without clarifying the role of appropriate quantities in clear patterns of usage and without use of standard measures and procedures will be subject to widespread misinterpretation. By including a context for interpretation, a procedural text may provide a basis by which the text supports more fully multiple interpretations [19]. Written text information lacks such context, making it difficult to understand the underlying procedural situation [32]. By designing information systems that allow for multiple views on the same information* where procedural, quantitative, and a priori representations are built as boundary objects* the user would be provided with a more integrated and coherent information that is sensitive to the informational requirements of each of the intended audiences. Situation-representational complexity, as demonstrated in the interpretation of the /

/

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

instructions for ORT, arises when information fails to provide adequate context in the establishment of an appropriate procedural framework by the user. This limitation must be addressed if the same boundary object is to be used differentially across multiple domains. One way to achieve this is through the designing of accompanying examples and diagrams [33 36], which have been demonstrated to be effective in communicating information about the sequence of procedures necessary to prepare and administer medications [16,36]. In addition, the elaboration of instructions to include explanations of various steps can significantly improve performance and promote more accurate use of products [15]. However, this is unlikely to happen in traditional printed material given the limited nature of such media. For this to happen, possibilities exist in complex instructional information systems that provide alternative models for representation [33], a point that has also been made in other contexts [34,37]. The inherent complexity of the quantified variables found in pharmaceutical labels, as demonstrated in the interpretation of the instructions for the cough syrup, may leads to errors in interpretation when the information fails to provide adequate representation of the translation of these variables. If the variables are to be accurately translated across multiple domains, the information required in each domain must be present in the text. By including such information in the text representation, such as the formula for translating pounds into kilograms in the case of the cough syrup, the text will more readily facilitate multiple representations by providing crucial information needed by each user. The problems and difficulties described above result from a failure of design in that the instructions do not act as boundary objects, which lead to representational mis/

209

match between the designer, the text information, and the user. A design decision to focus on specific shared knowledge may serve to support different understandings while facilitating the construction of an appropriate model. Information that supports understanding from the various perspectives involved, from the designers to the users of such texts, should help overcome biases and errors in interpretation. One method that has been developed, which has been applied to the analysis of clinical practice guidelines, can aid in this process by uncovering the underlying semantics of representation [32], and we need to explore this and other such methods to support multiple representations.

Acknowledgements The preparation of this manuscript was supported from a grant from the Agency for Healthcare Research and Quality HS11806 ‘Mining complex clinical data for patient safety research’. The studies reported in this paper were conducted at the Center for Medical Education at McGill University in Montreal, and were supported in part by an award from the Social Sciences and Humanities Research Council of Canada (239-07) to Dr Patel. The late Thomas O. Eisemon inspired these studies and we dedicate this paper to his memory. We graciously acknowledge all of the participants who willingly volunteered their time.We wish to acknowledge David Evans for providing critical comments and discussions on the earlier versions of this paper. We thank Ann Cruess, Laura Vidalis, Vanessa Allen, and Julie Kaufman for their contributions with the collection and analysis of data at various stages of this investigation.

210

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211

References [1] D.W. Bates, Frequency, consequences and prevention of adverse drug events, J. Qual. Clin. Pract. 19 (1999) 13 /17. [2] B. Richards, A.W. Colman, R.A. Hollingsworth, The current and future role of the internet in patient education, Int. J. Med. Inform. 50 (1998) 279 /285. [3] R. Bellazzi, S. Montani, A. Riva, M. Stefanelli, Web-based telemedicine systems for home-care: technical issues and experiences, Comput. Methods Programs Biomed. 64 (2001) 175 /187. [4] M.S. Wogalter, R.J. Sojourner, Research on pharmaceutical labeling: an information processing approach, in: D.C. Park, R.W. Morrell, K. Shifren (Eds.), Processing of Medical Information in Aging Patients: Cognitive and Human Factors Perspectives, Lawrence Erlbaum Associates, Mahwah, NJ, 1999, pp. 291 /310. [5] W. Kintsch, Comprehension: A Paradigm for Cognition, Cambridge University Press, New York, NY, 1998, p. 461. [6] M.J. Nathan, W. Kintsch, E. Young, A theory of algebraword-problem comprehension and its implications for the design of learning environments, Cogn. Instr. 9 (1992) 329 /389. [7] W. Kintsch, J.G. Greeno, Understanding and solving work arithmetic problems, Psychol. Rev. 92 (1985) 109 / 129. [8] E. Gerson, S.L. Star, Analyzing due process in the workplace, ACM Trans. Office Inf. Syst. 4 (1986) 257 / 270. [9] J. Otero, W. Kintsch, Failures to detect contradictions in a text: what readers believe versus what they read, Psychol. Sci. 3 (1992) 229 /235. [10] D.S. McNamara, E. Kintsch, N.B. Songer, W. Kintsch, Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text, Cogn. Instr. 14 (1996) 1 /43. [11] L.T. Kohn, J.M. Corrigan, M.S. Donaldson (Eds.), To Err Is Human, National Academy Press, Washington, DC, 1999. [12] H. Van Cott, Human errors: their causes and reduction, in: M.S. Bogner (Ed.), Human Error in Medicine, Lawrence Erlbaum Associates, Hillsdale, NJ, 1994, pp. 53 /66. [13] D.A. Evans, V.L. Patel (Eds.), Cognitive Science in Medicine: Biomedical Modeling, MIT Press, Cambridge, MA, 1989. [14] M. Sivaramakrishnan, V.L. Patel, Reasoning about childhood nutritional deficiencies by mothers in rural India: a cognitive analysis, Soc. Sci. Med. 37 (1993) 937 /952. [15] T.O. Eisemon, T. Ratzlaff, V.L. Patel, Reading instructions for using commercial medicines: when do literacy and numeracy become functional?, in: D.A. Wagner, L.D. Puchner, (Eds.), World literacy in the year 2000: The Annals of the American Academy of Political and Social Science, 1992, pp. 76 /90.

[16] V.L. Patel, T.O. Eisemon, J.F. Arocha, Comprehending instructions for using pharmaceutical products in rural Kenya, Instr. Sci. 19 (1990) 71 /84. [17] T.O. Eisemon, V.L. Patel, S. Ole Sena, Uses of formal and informal knowledge in the comprehension of instructions for oral rehydration therapy in Kenya, Soc. Sci. Med. 25 (1987) 1225 /1234. [18] J. Fujimura, Crafting science: standardized packages, boundary objects, and ‘translation’, in: A. Pickering (Ed.), Science as Practice and Culture, University of Chicago Press, Chicago, 1992, pp. 168 /211. [19] S.L. Star, The structure of ill-structured solutions: boundary objects and heterogeneous distributed problem solving, in: L. Gasser, M.N. Huhns (Eds.), Distributed Artificial Intelligence, Morgan Kaufmann, San Mateo, CA, 1989, pp. 37 /54. [20] S.L. Star, J.R. Griesemer, Institutional ecology, ‘translations’ and boundary objects: amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907 /1939, Soc. Stud. Sci. 19 (1989) 387 /420. [21] K. Henderson, Flexible sketches and inflexible databases: visual communication, conscription devices, and boundary objects in design engineering, Sci. Tech. Hum. Val. 16 (1991) 448 /473. [22] K. Henderson, The role of material objects in the design process: a comparison of two design cultures and how they contend with automation, Sci. Tech. Hum. Val. (1998) 23. [23] W.-M. Roth, M.G. Bowen, Complexities of graphical representations during ecology lectures: an analysis rooted in semiotics and hermeneutic phenomenology, Learn. Instr. 9 (1999) 235 /255. [24] M.G. Bowen, W.-M. Roth, M.K. McGinn, Interpretations of graphs by university biology students and practicing scientists: toward a social practice view of scientific representation practices, J. Res. Sci. Teach. 36 (1999) 1020 /1043. [25] M. Kyng, Making representations work, Commun. ACM 38 (1995) 46 /55. [26] V.L. Patel, D.R. Kaufman, J.F. Arocha, Conceptual change in the biomedical and health sciences domain, in: R. Glaser (Ed.), Advances in Instructional Psychology: Educational Design and Cognitive Science, Lawrence Erlbaum Associates, Mahwah, NJ, 2000, pp. 329 /392. [27] V.L. Patel, J.F. Arocha, D.R. Kaufman, Diagnostic reasoning and expertise, Psychol. Learn. Motiv. 31 (1994) 137 /252. [28] G.C. Bowker, S.L. Star, Sorting Things Out: Classification and Its Consequences, MIT Press, Cambridge, MA, 1999. [29] W.-M. Roth, M.K. McGinn, Inscriptions: toward a theory of representing as social practice, Rev. Educ. Res. 68 (1998) 35 /59. [30] S.L. Star, K. Ruhleder, Steps towards an ecology of infrastructure: design and access for large information spaces, Inf. Sys. Rev. 7 (1996) 111 /134. [31] S. Gasson, A social action model of information systems development, ACM SIGMIS 30 (1999) 82 /97.

V.L. Patel et al. / International Journal of Medical Informatics 65 (2002) 193 /211 [32] V.L. Patel, J.F. Arocha, M. Diermeier, R.A. Greenes, E.H. Shortliffe, Methods of cognitive analysis to support the design and evaluation of biomedical systems: the case of clinical practice guidelines, J. Biomed. Inform. 34 (2001) 52 /66. [33] W. Kintsch, Text comprehension, memory, and learning, Am. Psychol. 49 (1994) 294 /303. [34] P.J. Feltovich, R.L. Coulson, R.J. Spiro, B.K. DawsonSaunders, Knowledge application and transfer for complex tasks in ill-structured domains: implications for instruction and testing in biomedicine, in: D. Evans, V.L. Patel (Eds.), Advanced Models of Cognition for Medical Training and Practice, Springer-Verlag, New York, NY, 1991, pp. 113 / 165.

211

[35] D.S. McNamara, W. Kintsch, Learning from texts: effects of prior knowledge and text coherence, Discour. Proc. 22 (1996) 247 /288. [36] F. Schmalhofer, The acquisition of knowledge from text and example situations: an extension to the constructionintegration model, in: C.A. Weaver, S. Mannes, C.R. Fletcher (Eds.), Discourse Comprehension: Essays in Honor of Walter Kintsch, Lawrence Erlbaum Associates, Hillsdale, NJ, 1995, pp. 257 /283. [37] R.J. Spiro, P.J. Feltovich, R.L. Coulson, D.K. Anderson, Multiple analogies for complex concepts: antidotes for analogy-induced misconceptions in advanced knowledge acquisition, in: S. Vosniadou, R. Ortony (Eds.), Similarity and Analogical Reasoning, Cambridge University Press, Cambridge, MA, 1989, pp. 498 /531.