The potential of computers in patient education

PTiENT

EdUCATiON

ANd Counseling Patient Education and Counseling 22 (1993) 27-34

The potential of computers in patient education Celette Sugg Skinner*“, Juliette C. Siegfriedb, Michelle C. Keglerc, Victor J. Strecherc “Department of Medicine, School of Medicine. Indiana University Medical Center, Indianapolis, Indiana, USA ‘Health Communications Programs, Prospect Associaies, Rockville, Maryland, USA ‘Department of Health Behavior and Health Education, School of Public Health, University of North Carolina al Chapel Hill, Chapel Hill, North Carolina, USA

(Received 6 September 1991; accepted 18 June 1993)

Abstract Typical computer programs for patient education are didactic and fail to tailor information to an individual’s specific needs. New technology greatly enhances the potential of computers in patient education. Computer-assisted instruction programs can now elicit information from users before leading them through problem-solving exercises. New authoring systems enable health professionals to develop their own programs. The capacity to elicit and report back information about factors that influence patients’ health behaviors give the newest computer programs one of the strengths of faceto-face patient counseling: the ability to tailor an educational message for an individual patient. These programs are not intended to replace but .rather to enhance personal interaction between providers and patients. This article describes the advantages of using computers for individualizing patient education and assessing trends across groups of patients. Innovative programs and features to look for in programs and equipment selection are also described.

Key words: Computers; communication

Computer-assisted

instruction;

1. Introduction Over the last decade, medical offices across the country have streamlined their procedures with the aid of computers. Billing procedures, appointment logs and patient mailing lists that used to be * Corresponding author, Regenstrief Institute for Health Care (RGS), Indiana University Medical Center, 1001 West Tenth Street, Indianapolis, Indiana 46202-2859, USA.

Tailoring;

Informatics;

Patient

education;

Persuasive

laboriously managed by hand are now neatly and efficiently handled electronically. In addition, health professionals have begun to employ computer technology for patient education and counseling. This paper discusses the evolution of computer programs for patient education, describes some currently available programs, outlines recent innovations and future directions for expanded use of computers and suggests guidelines for program selection.

0738-3991/93/$06.00 0 1993 Elsevier Scientific Publishers Ireland Ltd. All rights reserved. SSDI 0738-3991(93)00573-Q

28

1.1. Computer programs for patient education

The first computerized patient education materials were unimaginative tutorials that asked patients to read information and then take tests. These programs did little more than reproduce standard written materials on a computer screen. There are now a number of innovative computer programs available on health-related topics. Generally labeled computer-assisted instruction (CAI), these programs are usually preventive in nature, aimed at the well rather than the sick. For example, several interactive programs have been developed for preventing alcohol abuse [l-5], teaching adolescents about the consequences of sexual activity [6], promoting self-care in disease control [7,8] and educating patients’ families [9]. CA1 programs typically have several components, including compilation and review of personal data, quizzes and other educational activities. Some also include an interactive component with practical problem-solving or role-playing exercises [4,6]. For example, one program called Diabetes in Self-Control (DISC) has helped teens achieve better scores than their counterparts receiving traditional written materials in outcomes such as knowledge about their disease, management practices such as glucose monitoring and daily blood glucose levels [lo]. Health professionals who wish to design customized CA1 health education programs for patients rather than use existing programs can use microcomputer authoring systems. Authoring systems allow educators who do not have programming expertise to create moderately complex educational materials by mediating between the user and a high-level programming language. Patient educators can choose to create several types of lesson: drills, tutorials, simulations or games. The type of question can be varied within a lesson and can be multiple-choice, yes-or-no, fillin-the-blank, matching or word responses. A variety of options for displaying text and graphics on the screen are also available. Record keeping is an important feature of the authoring systems, allowing health professionals to store the results of completed quizzes as baseline measurements. After an educational intervention, the same quiz can be given and the post-test compared with baseline

C. Sugg Skinner et al. /Patient

Educ. Couns. 22 (1993) 27-34

scores. Authoring systems allow the expertise of the health professional to be combined with the power of a microcomputer. Authorware [l 11, Toolbook [12] and Icon Author [13] are examples of authoring systems; each is available for IBM (or compatible) and Apple computers. 1.2. Recent innovations and future directions The potential for using computers in patient education goes beyond the programs described above. Computers can not only be used to create interactive learning situations for patients, they can also generate individualized reports for both patients and health professionals based on the information patients enter into the computer. By eliciting and reporting back information about factors that influence patients’ health behaviors, computer programs can achieve one of the strengths of face-to-face patient counseling: the ability to tailor the educational message for the individual patient. Health professionals can proceed more intelligently in patient counseling when they have background information about beliefs, practices and other factors that influence health behaviors than when they either assume that all patients are similar or try to infer what is important for each patient [ 14,151. Even individuals who need to make similar behavior changes are likely to differ on factors that influence their health behaviors, and counseling to motivate one set of individuals toward certain health behaviors may be irrelevant for, or actually discourage, appropriate behavior among others. For example, one woman may not feel at risk for developing breast cancer and thus may not perceive any need for mammography screening. This woman should be counseled differently from someone else who is avoiding mammography for fear of finding cancer. Counseling that encourages mammography for the second woman by allaying a fear of finding breast cancer might discourage mammography for the first woman by even further reducing her perceived susceptibility to the disease. Computer technology can be used to elicit information from patients and then tailor patient education, but few programs have exploited this potential. Even computerized health risk apprai-

C. Sugg Skinner et al. /Patient

Educ. Couns. 22 (1993) 27-34

sals (HRAs), which collect data on individuals’ risk factors and then deliver written information back to the. individuals, have not reached their potential to tailor information to individual circumstances [ 16- 181. The HRA information typically contains only a risk status report, with simple standard recommendations to change behavior, and these reports have not been very effective in promoting behavior change [ 19-211. The reports might be more effective if they contained tailored information about individual patients’ behavior-influencing factors. Computers can facilitate tailored patient education as follows. After a patient completes a computerized questionnaire, the computer can generate a tailored, individualized report to guide health professionals in patient counseling. For instance, the report can tell the health professional that Ms. Smith has finally quit smoking but is concerned about the weight she is now gaining, or report that Mr. Jones has considered having a cholesterol test but thinks he is too young to be concerned about his cholesterol level. These computer-generated reports of patient information may greatly help busy professionals who find themselves with heavy patient loads. There is seldom adequate time during an office visit to sufficiently elicit the breadth of individual information necessary to tailor counseling on a variety of preventive health behaviors and then to actually engage in the counseling. Indeed, lack of time and lack of compensation for counseling patients about preventive health practices have been found to deter these activities among health professionals [22-251. A study currently under way at the Indiana University Medical Center will assess the effectiveness of tailored medical chart prompts relaying information collected from patients via computer during office visit waiting time. These easy-to-use computer questionnaires may replace some of the time-consuming question-and-answer sessions through which prevention-oriented patient information is elicited, thereby allowing health professionals to focus on issues already identified by patients as important, rather than discussing a litany of potentially irrelevant issues. In addition to generating reports to guide health professionals in patient counseling, computers can

29

create tailored written or video materials for patients. These materials, supplementing face-toface patient counseling, can address individual differences instead of delivering the same message for all recipients. Individually tailored print materials are created via data management capabilities of microcomputers and page make-up (‘desktop publishing’) techniques that facilitate the assembly of different combinations of text for individual recipients. The Partners in Prevention Project at the University of North Carolina School of Public Health has used this technology to create individualized printed recommendations on preventive health behavior change tailored to patients’ baseline survey responses [26]. The project is guided by two models of health behavior change. Messages to patients and physicians are tailored on several variables from the health belief model [27] that have received considerable support in predicting preventive health behavior [28,29]. These components of the health belief model, including perceived benefits and barriers to behavior change and personal susceptibility to disease, are considered across stages of behavior change adapted from a self-change model developed by Prochaska and DiClemente [30-321. Message recipients are addressed differently depending on whether they have already considered the recommended behavior changes and, if so, to what degree they have considered the changes. The tailored messages identify and group individuals into audience segments and then supply appropriate messages for each behavior-influencing factor considered. A library of texts and graphic files is created to specifically address varying beliefs, stages of behavior change, risk factors and barriers impeding behavior change. When assembling a letter for a particular patient, the computer reads the patient’s baseline interview data and, at multiple junctures in the letter layout, inserts one of several different message texts. The program allows for more than 391 000 different tailored letter versions. Every patient’s letter is different from every other patient’s letter, unless people give the exact same set of responses in the baseline interview. The effectiveness of these computer-tailored messages in comparison with traditional, standardized materials delivering the same message for all

30

recipients has been evaluated in several samples of family practice patients. Results from these evaluation studies indicate that tailored printed recommendations for cancer screening procedures [33] and for dietary behavior changes [34] were better remembered and more thoroughly read than standardized comparison messages. Further, comwere more puter-tailored recommendations effective than standardized recommendations for eliciting dietary changes (reduction of fats) in a heterogeneous patient population [34] and for screening increasing mammography among minority and low-income women [33]. Computer-tailored messages can also be delivered via video disks. The Foundation for Informed Medical Decision Making has developed five touch-screen, interactive video disk programs to increase patient participation in treatment decisions for non-emergency situations [35-381. By outlining treatment options, including benefits, risks and statistics regarding patients with similar medical histories to the user, the disks may allow physicians to spend less time outlining options and more time counseling patients in their personal decisions. Another interactive, computer-based multimedia program is currently being developed at the University of North Carolina’s Health Communications Research Laboratory. A randomized trial involving indigent patients at high risk for cardiovascular disease will test the effectiveness of this technology for nutrition education and dietary modifications. The multimedia module, entitled Health Talk, is modeled on traditional television talk shows, such as the Oprah Winfrey Show and Donahue. What makes Health Talk unique, however, is that each viewer sees a personalized version of the show. The talk show host guides each viewer through conversations that allow the computer to identify and focus on the viewer’s individual dietary risk areas. Whereas traditional talk shows are mediated by the host, Health Talk viewers decide which audience members will ask questions and which expert panelists will answer them. Using information collected from the viewer, including specific dietary practices, readiness for dietary change and support from the social environment, the program will deliver an in-

C. Sugg Skinner et al. /Patient Educ. Couns. 22 (1993) 27-34

dividually tailored goal-setting program through the talk show format. Users will also receive tailored, easy-to-read printed materials. A final example of computer tailoring comes from Noells and his colleagues at the Oregon Research Institute (391, who have developed a touchscreen, multimedia program for smoking cessation for use in medical offices during primary care visits. Patients see only a television screen and a printer; a Macintosh computer and disk drive are hidden underneath. After a personal introduction to the program by a physician, patients view a television skit that makes fun of the way smokers are often treated by non-smokers. The program then elicits patient information through the use of simple questions that can be answered by touching one of several responses on the screen. The personal information is used to tailor the remainder of the program, which presents a role model, similar to the patient, who has overcome obstacles to quitting smoking. The information and role models presented in the Oregon program were developed after extensive research into values and interests of the client population. Focus groups and personal interviews with low-literacy smokers were used to create a computer program based on one of the most fundamental tenets of patient education: health behavior change is more likely to occur when patients realize their personal susceptibility to a health hazard and believe they have the ability to avoid the hazard through behavior change [40,41]. This program is an example of how important educational principles can be combined with current computing technology. In a pilot test with 27 family practice patients, the system was rated as easy to use, and 70% of the users set quit-smoking dates [42]. Using computers for patient education has the advantage, at the micro level, of tailoring relevant information to individual patients and their health professionals. However, there are also macro-level advantages that stem from the computer’s ability to link data-bases. For example, physicians at the Harvard Community Health Plan outpatient clinic can retrieve patient records via their microcomputers. At the touch of a button, medications can be ordered that will be ready within moments at the

C. Sugg Skinner et al. /Patient Educ. Cows. 22 (1993) 27-34

center’s pharmacy - but only if the computer record detects no allergies or other contraindications for the prescription. One hundred terminals have been linked to the system and installed in private homes, where patients can interact with the center via microcomputer. If the patient’s symptoms are serious, the computer will arrange for a physician to call or will contact a staff member to schedule an appointment. If the patient has symptoms that can be treated at home, such as those due to a common cold, the computer provides instruction in self-care. Proponents estimate that these offerings could reduce the number of unnecessary office visits by 25 to 44% [43]. Computers can also form networks among health professionals, patients and their families. In describing ComputerLink, a network through which care-givers of people with Alzheimer’s disease can communicate with one another and with health professionals, Brennan [44] writes that ‘these technologies bring to a caregiver too busy or unable to leave home many of the benefits of support groups and other service interventions, relevant information, peer advice, and professional counsel.’ Another potential for computers in patient education is the compilation of data collected from individual patients, allowing for assessing trends across groups of patients. Looking at trends may enable health professionals to identify innovative strategies for promoting behavior change. If, for example, the data show that a large proportion of women in a certain medical practice or health clinic have not had mammograms because of the cost, health professionals may recruit a local community group to fund a screening clinic or provide coupons for low-cost mammograms. Or perhaps the data show that many women have had an initial mammogram but do not know that they need repeat screenings. This could indicate the need for a tracking and reminder system. Group data collected via computer can be used to guide the selection of patient seminar topics or, for evaluation purposes, to indicate changes in health knowledge, attitudes or behaviors before and after interventions. Computer-collected patient education data could be linked to computerized medical records.

31

Comprehensive computerized systems capable of documenting entire medical records already exist [45,46]. One of the most prominent is the Regenstrief Medical Record System, in operation at the Indiana University Medical Center [47]. In addition to computerized information storage, the Regenstrief system is one of several examples of computerized physician prompting for preventive health recommendations [48-501. Such systems, which automatically indicate which preventive health measures are indicated for specific patients, could also be programmed to display specific information about behavior-influencing factors for use in patient counseling. Combining data on behavior-influencing factors with medical records data would facilitate the incorporation of such factors in quality assurance programs, which typically involve setting standards for quality care, reviewing care patterns, designing and implementing strategies for improving quality and measuring performance against the standards [51]. In their review of published studies on computer-aided quality assurance, Haynes and Walker [52] conclude that computerized records and reports can influence health professionals’ behavior, but that more attention must be paid to the link between the process of care and patient outcomes. The inclusion of behavior-influencing factors in patients’ computerized records could strengthen the ability of such programs to improve patient outcomes. Despite the fact that the majority of physicians’ offices now have computers for purposes such as billing [53], health professionals are not taking advantage of computer technology for patient education. Lack of time is often cited as the main reason for not conducting patient education. Yet, although computers could greatly increase efficiency of patient education efforts, they are not commonly seen as a time-saving solution. Christine Bolwell, editor of a newsletter and author of a directory of educational software [54], has described the brief rise of a company that attempted to produce software for medical offices and ultimately failed because of lack of physician interest. Many health professionals, Bolwell explains, just do not see the advantages of using computers in their oftices for patient education

C. Sugg Skinner et al. /Patient Educ. Couns. 22 (1993) 27-34

32

(551. ‘They’re terrified of the technology think keyboards are for secretaries.’

... they

1.3. Guidelines for program selection Health professionals considering implementing computer programs into their patient education practices or upgrading current systems should keep the following criteria in mind. (1) The programs should be easy to use. Many patients are afraid of ‘breaking’ the computer. Instructions should be simple; consider a touchscreen, mouse or joystick instead of a keyboard to provide easy responses to computer-generated questions. (2) The program should provide printed feedback to the user. The feedback should include personalized strategies for change, rather than just textbook-style information. A patient counseling session with the health professional should follow computer use, so that questions can be answered and a plan developed for dealing with the information provided by the computer. (3) The program should be developed not only by computer experts but also by reputable health professionals who are acquainted with health education and behavior change principles. There is more to health education than simply writing a good computer program. Tested, accurate programs are essential for successful computerized patient education. Most of the systems available at present are stand-alone programs that run on IBM (and compatible) or Apple computers. Many programs require a graphics card, color monitor, a certain amount of memory or a specific operating system. A printer may also be necessary. Staffing needs will vary depending on whether an organization buys a program or develops its own. It is also important that a health professional be available to the patients to answer questions both during and after a computer session. 2. Summary In the past, typical computer programs in patient education have been didactic and have failed to tailor information to individuals’ needs. Such

programs are not necessarily any more effective than traditional written materials, and they do not make use of the powerful capabilities of small computers. This is beginning to change. Programs are currently available that elicit information from users, then lead the users through problem-solving exercises. Authoring systems that enable health professionals to develop their own programs are also available. Programs that elicit information on factors that influence health behavior are being developed and evaluated. The potential for using computers in patient education is vast and varied. Whereas only the most sophisticated and well-financed clinics may become the earliest adopters of very complex interactive systems, smaller offices with more limited resources need not be left out. The greater affordability of small computers and their ‘userfriendly’ nature ensure that health professionals with minimal computer experience will be able to include some degree of computer-assisted instruction into their practices. The newest programs available help patients to confront important health issues in a private, user-paced environment. The programs are not intended to replace but rather to enhance personal interaction between providers and patients; information collected by the computer can be used to stimulate relevant, useful counseling sessions that address patient needs. 3. References Meier S: Alcohol education through computer-assisted instruction. .I Couns Dev 1984; 66: 141-144. Reynolds N: The Alcohol IQ Network [computer program], Cornell University Health Services, Ithaca (NY), 1988. Thomas B: Drinking and Not Drinking [computer program], Kinko’s Academic Courseware Exchange, Santa Barbara (Calif.), 1987. If You Drink: a Guide to Alcohol Education [computer program]. Multi-Health systems, Willowdale (Canada), 1986. Meier S, Scott T and Simpson JP: Use of computerassisted instruction in the prevention of alcohol abuse. J Drug Educ 1989; 19 (3): 245-256. Alemi F, Cherry F and Meffert G: Rehearsing decisions may help teenagers: an evaluation of a simulation game. Comput Biol Med 1989; 19 (4): 283-290. Horan P, Yarborough M, Besigel G and Carlson D:

C. Sugg Skinner et al. /Patient Educ. Couns. 22 (1993) 27-34 Computer-assisted self-control of diabetes by adolescents. Diabetes Educ 1989; I6 (3): 205-211. 8 Marerro DG. Computers and the assessment of diabetes control. In: CE Mogenson and E Stand1 (eds.): Concepts for the Ideal Diabetes Clinic (Diabetes Forum Series, vol. IV), de Gruyter, New York, 1993. Personal communication, October 9 Bolwell, Christine. 1990. 10 Mazzuca SA, Moorman NH, Wheeler ML, Norton JA, Fineberg NF, Vinicor F, Cohen FJ and Clark CM: The Diabetes Education Study: a controlled trial of the effect of diabetes patient education. Diabetes Care 1986; 9: l-10. (8500 Normandale Lake Boulevard, Min11 Authorware neapolis, MN 55437). I2 Whiteside M and Whiteside A: Microcomputer authoring systems; valuable tools for health educators. Health Educ 1987/88; 4-6. (20 Trafalgar Square, Nashua, NH 13 AimTech Corporation 03063), 1973. of behavioral science to I4 Bartlett EE: The contributions patient education practice: a review. In: Ruffini, JL (ed): Advances in Medical Social Science , vol. II. New York: Gordon and Breach Science Publishers, 1984. G and Hahn P: Evaluation of a 15 DeVellis B, Hochbaum problem-solving intervention for patients with arthritis. Patient Education and Counseling 1988; 11: 29-42. I6 Kirscht JP: Process and measurement issues in health risk appraisal. Am J Public Health 1989; 79: 1598-l 599. I7 Becker MH and Janz NK. On the effectiveness and utility of health hazard/health risk appraisal in clinical and nonclinical settings: behavioral science perspectives on health hazard/health risk appraisal. Health Services Research, 1987; 22 (4): 537-551. 18 De Friese GH and Fielding JE. Health risk appraisal in the 1990s: opportunities, challenges and expectations. Annual Review of Public Health 1990; I I: 401-418. JB: Accuracy of I9 Avis NE, Smith KW and McKinlay perceptions of heart attack risk: what influences perceptions and can they be changed? Am J Public Health 1989; 79: 1608-1612. 20 Schoenbach VJ, Wagner EH and Beery WL. Health risk appraisal: review of evidence for effectiveness. Health Services Research 1987; 22 (4): 553-580. VJ, Graham RM. 21 Wagner EH, Beery WL, Schoenbach An assessment of health hazard/health risk appraisal. Am J Public Health, 1982; 72 (4): 347-352. 22 Romm FJ, Fletcher SW and Hulka BS: The periodic health examination: comparison of recommendations and internists’ performance. South Med J 1981; 74: 265-271. 23 Woo B, Cook EF. Weisberg M and Goldman L: Screening procedures in the asymptomatic adult: comparisons of physicians’ recommendations. patients’ desires, published guidelines, and actual practice. J Am Med Assoc 1985; 254: 1480-1484. 24 McPhee SJ, Richard RJ and Solkowitz SN: Performance of cancer screening in university general internal medicine

33

25 26

27

28 29

30

31

32

33

34

35 36

37

38

39 40 41

42 43 44

practices: compliance with the 1980 American Cancer Society guidelines. J Gen Intern Med 1986; 1: 275-281. Goldman L: Changing physicians’ behavior: the pot and the kettle. N Engl J Med 1990; 24: 1524-1525. Kreuter MW, Strecher VJ, Kegler MC, Harris R, Farrell DA, and Skinner CS. Facilitating preventive health care activities in a primary care setting (presentation, American Public Health Association, November, 1992). Hochbaum GM: Public Participation in Medical Screening Programs: a Sociopsychological Study (Public Health Service publication no. 572). Washington: US Government Printing Office, 1958. Janz NK and Becker ME: The Health Belief Model: a decade later. Health Educ Q 1984; 11 (1). Kirscht JP and Rosenstock IM: Patients’ problems in following recommendations of health experts. In: G Stone (ed.): Health Psychology. Jossey-Bass, 1979. Prochaska JO and DiClemente CC. Stages and processes of self-change of smoking: toward an integrative model of change. J Consult Clin Psycho] 1983; 51: 390-395. Prochaska JO and DiClemente CC: Measuring process of change: applications to the cessation of smoking. J Consult Clin Psycho1 1988; 56: 520-528. Rakowski W, Jube CE, Marcus BH, Prochaska JO, Velicer WF, and Abrams DB: Assessing elements of women’s decisions about mammography. Health Psychology, 1992; II (2): 111-118. Skinner CS, Strecher VJ, Hospers H. Physician recommendations for mammography: do tailored messages make a difference? Am J Public Health (in press). Campbell MK, DeVellis BM, Strecher VJ, Ammerman A, De Vellis RF and Sandler RS. The impact of message tailoring on dietary behavior change for disease prevention in primary care settings. Am J Pub1 Health. In press. Foundation for Informed Medical Decision-Making (PO Box C17, Hanover, New Hampshire 03755). Clements B. Interactive videos: new system walks patients through treatment decisions, lessening liability, saving time - and maybe money. American Medical News, 1992; 35 (43): 17-19. Winslow R. Videos, Questionnaires aim to expand role of patients in treatment decisions. Wall Street Journal, 25 February 1992, B3. Holzman D. Interactive video promotes patient-doctor partnership. Business and Health, Mid-March 1992; 42-47. Noells J. Personal communication, November, 1990. Becker ME. The Health Belief Model and personal health behavior. Health Educ Mon 1974; 2 (4). Strecher VJ, DeVellis BM, Becker MH and Rosenstock IM: The role of self-eflicacy in achieving health behavior change. Health Educ Q 1986; 13: 73-91. Noells J. Unpublished data, December, 1992. Faltermayer E. Let’s really cure the health system. Fortune, 23 March 1992; 46-58. Brennan PF. Computer networks promote caregiving collaboration: the ComuuterLink nroiect. In Proceedines of

C. Sugg Skinner et al. / Patienr Educ. Couns. 22 (1993) 27-34

34

45 46

41

48

the Sixteenth Annual Symposium on Computer Applications in Medical Care, Frisse ME ed. American Medical Informatic Association, 1993; 156- 160. Saba VK: Nursing and Computers: an Anthology. New York: Spring-Verlag, 1989. Lussier YA, Maksud M, Desruisseaux B, Yale P, StArneault R, Pure MD A computerized patient record software for direct data entry by physicians using a keyboard-free pen-based portable computer. In: Proceedings of the Sixteenth Annual Symposium on Computer Applications in Medical Care, Frisse ME ed. American Medical Informatic Association, 1993; 261-264. McDonald CJ, Tierney WM, Overhage JM, Martin DK and Wilson GA. The Regenstrief Medical Record System: twenty years of experience in hospital, clinics, and neighborhood health centers. MD Computing, 1992; 9 (4): 206-217. Turner BJ, Day SC and Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders? Journal of General Internal Medicine 1989; 4: 403-409.

49

50

51

52

53 54

McPhee SJ, Bird JA, Fordham D, Rodnick Je and Osborn EH. Promoting cancer prevention activities by primary care physicians: results of a randomized, controlled trial. JAMA 1991; 266 (4): 538-544. Ornstein SM, Garr DR, Jenkins RG, Rust PF and Arnon A. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services. J Fam Pratt 32 (1): 82-90. Goldfield N, Nash D eds: Providing Quality Care: the Challenge to Clinicians. Philadelphia: American College of Physicians, 1989. Haynes RB, Walker CJ: Computer-aided quality assurance: a critical appraisal. Arch Intern Med, 1987; 147: 1297-1301. Schmittling GT. Computer use by family physicians in the United States. J Fam Pratt 1989; 29: 198-200. Rankin SH, Stallings KD: Patient Education. Lippincott, Philadelphia, 1990: 209-10.