- Email: [email protected]
Increasing influenza immunization among high-risk patients: Education or financial incentive?
Increasing Influenza Immunization among High-Risk Patients: Education or Financial Incentive? William P. Moran, MD, MS, Karen Nelson, MD, James L. Wofford, MD, MS, Ramon Velez, MD, MSc, L. Douglas Case, PhD, Winston-Salem, North Carolina and Boston, Massachusetts
PURPOSE: To determine whether an educational brochure or a lottery-type incentive increases influenza immunization rates. PATIENTS AND METHODS: ln a prospective, Singleblind factorial design randomized trial at an urban community health center, all high-risk patients (n = 797) seen in the preceding 18 months were randomly assigned to one of four groups: a control group; a group mailed a large print, illustrated educational brochure emphasizing factors important to patients in making a decision about influenza immunization; a group mailed a lottery-type incentive announcing that all patients receiving influenza immunization would be eligible for grocery gift certificates; and a group mailed both educational brochure and incentive. Immunization was free, available without an appointment, and recorded by a computerized tracking system. RESULTS: The group mailed the brochure was more likely to be immunized than control (odds ratio [OR] = 2.29, 95% confidence interval [Cl] 1.45 to 3.61), as was the group mailed the incentive (OR = 1.68, 95% Cl 1.05 to 2.68), but there was no difference between the group mailed both interventions and the control group. The effectiveness of the brochure was more striking for individuals who had not accepted immunization in the prior year (OR = 4.21, 95% Cl 2.48 to 7.14), suggesting a true educational effect rather than simply a reminder. CONCLUSION: In this community health center setting, an illustrated educational brochure increased influenza immunization among high-
From the Departments of Medicine and Public Health Sciences, Bow man Gray School of Medicine of Wake Forest University, WinstonSalem, North Caroltna; and the Department of Medicine, Boston City Hospital and Dorchester House Multi-Service Center; Boston, Massachusetts. Supported In part by National Research Service Award, National lnsti tute on Aging (5T32 AG00182). Dr. Moran IS a Robert Wood Johnson Generalist Faculty Scholar. Requests for reprints should be addressed to William P. Moran, MD, MS. Section on General Internal Medicine and Gerontology, Department of Medlclne, Bowman Gray School of Medicine, Medical Center Boulevard, Wlnston-Salem, North Caroltna 27157.1051. Manuscript submttted September 12, 1995 and accepted in revised form September 23, 1996.
612
01996 by Excerpta All rights reserved.
Medica,
Inc.
risk patients, a lottery-type incentive was much less effective, and both together was not effective. 0 1996 by Excerpta Medica, Inc. Am J Med. 1996;101:612-620.
T
here is now overwhelming evidence that a large proportion of the morbidity and mortality resulting from influenza is preventable.‘-” In the United States, epidemics of influenza may cause 20,000 to 40,000 deaths and result in as many as 1.72,OOOhospitalizations. ‘J Annual influenza immunization has been recommended for individuals who are most likely to die or develop serious complications from influenza. These high-risk individuals are persons aged 65 years or older, and persons under 65 with chronic cardiopulmonary disease, metablolic disease (including diabetes mellitus), hemoglobinopathies, immunosuppression, or renal dysfunction.’ Influenza immunization has been shown to reduce mortality, rates of hospitalization, and the severit;y of clinical illness from influenza when the prevalent strain matches that in the vaccine.‘-“,’ Assuming a 70% vaccine efficacy and 70% immunization rate of individuals at high risk, influenza immunization could prevent almost 10,000 deaths and 80,000 hospitalizations annually.‘,” Despite the proven efficacy of influenza immunization, the national rate for influenza immunization of high-risk individuals is estimated to be 40%, although the rate of influenza immunization may be over 50% for individuals age 65 and older.Y*10 Physician knowledge and awareness seems not to be a critical barrier, since the majority of physicians are convinced that influenza is a serious disease, believe influenza immunization is safe and effective, and accept the national recommendations for immunization. It should not be surprising that approximately 70% to 80% of all influenza immunizations are given in physicians’ offices, and high-risk p,atients with physician appointments during the immunization season are likely to be immunized.” The Public Health Service has established the “Healthy People 2000” goal of immunizing 60% to 80% of high-risk individuals each year.” Primary care practices are most likely to achieve this goal if several systematic strategies are implemented.” The practice could establish a “standing order” to immunize all high-risk individuals without requiring an 0002-9343/96/$15.00 PII SOOO2-9:343(96)00327-O
INFLUENZA IMMUNIZATION
FOR HIGH-RISK PATIENTS/MORAN
ET AL
The Flu Shot is FREE and available NOW at the Dorchester House! No appointment needed.
You decide! ’ l The Flu is not just a common cold. l The Flu is a serious illness. l It is very contaaious and you may give it to your friends and family. Figure
1.
a F/u ‘Shot! The modern Flu Shot does got cause the flu. l Your doctor at the Dorchester House wants you to get the flu shot this fall and every year.
is
Get l
*The Flu Shot is s fe and effective Y”,u can get the Fiu Shot even if you are allergic to other medicines. It is safe to get even if you are taking medicines.
The illustrated educational brochure based on the decision model of Carter et al.22~23
individual physician authorization.‘” Clinicians could also be prompted either with paper-based systems or computerized reminders.14 The practice could reassign tasks of staff members to facilitate influenza immunization during the autumn months.‘” Immunization goals could be established for the practice and feedback provided to the practice staff on performance may also effectively improve immunization rate.‘“-l8 Obviously such strategies are effective for patients who have a scheduled appointment in the practice or clinic, but high-risk patients may not have appointments during the months when vaccine is available. 19,20 Therefore, to maximize influenza immunization, outreach strategies to prompt patients to seek influenza immunization are needed. Alerting high-risk patients with mailed reminders for influenza immunization is recommended in efforts to increase immunization among high-risk individuals.“,“’ However, in a prior study at the community health center where the current study was conducted, letter reminders appeared to have no impact on the rate of inlluenza immunization.” To enhance the impact of outreach efforts, an educational component can be added to mailed reminders. In an effort to develop effective educational materials, Carter et alD constructed and tested a decision model to determine which factors were considered important to older patients when deciding to seek or decline influenza immunization. The model suggested that the avoidance of the discomfort and complications of influenza, the desire not to infect others, and a physician’s recommendation favored patient acceptance of influenza immunization. Side effects and complications of the vaccine itself were the major factors against accepting immunization. A
brochure addressing these concerns led to a 15%increase in the influenza immunization rate of older, predominantly male patients who would have been predicted by the model to refuse immunization.“” Thus, interventions that specifically target patients’ attitudes about the side effects and complications of the influenza vaccine may improve compliance with immunization. Such interventions could have a sustained impact on immunization, since patients who accept influenza immunization are also more likely to accept immunization in subsequent years.‘4 Offering a financial incentive to high-risk patients is an alternative to education in efforts to persuade patients to consider accepting influenza irnmunization. Financial incentives to patients have not previously been used to promote influenza irnmunization. Food and other non-cash incentives have been employed to improve compliance with follow-up for a number of health care efforts,“‘12” and cash incentives effectively increased adherence to first followup appointment for tuberculosis therapy among homeless individuals.“’ Rather than a direct financial incentive, however, in this study we offersed an incentive in the form of a modest lottery-type prize to induce high-risk individuals to consider infhienza immunization. Lottery-type prize incentives have been most commonly employed in smoking cessation trials, both in community trials and ln targeted trials in individuals with smoking-related disease.““-“’ Many of the high-risk patients at this community health center participate in the Massachusetts state lottery so a lottery prize, in the form of a grocery gift certificate, was thought by center staff to be an acceptable but less costly alternative to a direct cash incentive. To test individual and joint effects of
December
1996
The American
Journal
of MedicineL
Volume
101
613
INFLUENZA IMMUNIZATION
FOR HIGH-RISK PATIENTS/MORAN
If you get could win
a Flu Shot
ET AL
at the
Dorchester
House,
you
House
wants
tuwo waysl
First, you win because your doctor at the Dorchester you to get the flu shot this fall to prevent the flu.
Second, everyone who gets a flu shot at the Dorchester House has a chance to win one of three gift certificates for $50 worth of groceries.
The Flu Shot is FREE and available Dorchester House. No appointment
NOW
at the
is needed.
Drawing for the gift certificates will be held December 20. 1991. You do not have to be present to win. You must be 18 years or older and have a medical reason for the Flu Shot (everyone age 65 or older. or less than 65 with heart, lung, blood, kidney or liver problems, immune deficiency or diabetes). If you are pregnant, please check with your doctor befoe getting the Flu Shot.
Figure 2. The lottery-type incentive announcement.
education and incentive in increasing influenza immunization, we mailed an illustrated educational brochure and a lottery-type incentive announcement, alone or in combination, to high-risk patients.
formation about where and when the immunization may be obtained. The lottery-type incentive (Figure 2) announced that patients could “win two ways” in accepting influenza immunization: (1) immunization is recomMETHODS mended by the doctors at the center to reduce the This study was conducted during the 1991-1992 chances of getting influenza, and (2) if patients are influenza season on the adult medicine service of an immunized, they are eligible to win one of three grourban community health center, staffed by five full cery store gift certificates. The announcement limor part-time physicians board certified in internal ited eligibility to patients with high-risk indications medicine or family medicine, four resident physi- for the immunization. High-risk patients were randomly allocated to one cians, and one physician assistant. The service proof four groups. Patients in the control group received vides for more than 9,000 ambulatory care visits an“usual care” and were not mailed an intervention. nually by poor to lower income patients. All high-risk The three intervention groups were mailed either the ambulatory patients seen within the preceding 18 educational brochure, the lottery-type i:ncentive, or months and entered into the computerized tracking both the educational brochure and the incentive. The system were included in the study. Patients were interventions were sent by first class mail, the eduidentified by a search of the tracking database using cational brochure initially, and 1 week later the lotage and ICD-9 CM codes corresponding to high-risk tery-type incentive announcements. Twelve interconditions.4 A three-panel educational brochure (Figure 1) ventions were returned by the post office as was designed to address factors shown to influence undeliverable, eight of which were readldressed and remailed, and all were included in the analysis. Since patients’ decision to accept influenza immunization (the “flu shot”) and was based on the model devel- immunization is recommended at least 4 weeks prior oped by Carter et al.15,1B A statement and the line to potential outbreaks of influenza in late January drawings in the brochure indicated that influenza and early February, only immunizations adminis(the “flu”) is a serious disease with potentially severe tered from the beginning of October to the last week consequences, and the caption present.s the choice of December were considered for analysis. The drawing for the lottery-type incentive prizes was facing the patient: “Where will you be this winter?” “You decide . . .” is intended to empower the indi- held during the last week of December. Health care vidual, and “Get a flu shot!” presents the necessary providers and staff at the center were blind to the action clearly. The lower panels emphasize impor- patients’ group assignment. Influenza immunization was given at the health tant decision factors, and two statements addressing the immunization fears of the individuals appear on center free of charge during scheduled provider visthe lower second panel. The third panel contains in- its, at a “health fair” in late October, or on a “walk614
December
1996
The American
Journal
of Medicine@
Volume
101
TABLE
I Baseline Comparison
of Covariates by Group Treatment Brochure n (%)
Control n (%) Total Age Mean age/SD* Median age Age range Age 65 or older Gender Female Risk factors Alcoholism Cardiac disease Diabetes mellitus lmmunodeficiency Liver disease Pulmonary disease Renal disease Number of risk factors (excluding age) None One Two Three Four Prior immunization history when known (n = 328) Immunized prior year/total
Group (n = 797) incentive n (%)
202 (100)
198 (100)
198 (100)
Both n (%) 199 (100)000-000
66/16.6 70 18-99 138 (68)
65/l 5.9 69 19-92 139 (70)
65/l 7.2 69 19-92 133 (67)
68/15.2 71 21-95 153 (77)
134 (66)
128 (65)
129 (65)
133 (671
17 (8) 51 (25) 25 (12) 4 (2) 6 (3) 72 (36) l(l)
12 (6) 53 (27) 28 (14) 4 (2) 1 (1) 69 (35) l(l)
22 54 24 5 2 60
(11) (27) (12) (3) (1) (30) 2 (1)
(5) (18) (9) (2) 2 (1) 62 (31) -
73 (36) 91 (49) 30 (14) 7 (3) 1 (1)
79 78 33 8
74 (37) 90 (45) 24 (12) 9 (5) l(l)
101 (50) 69 (35) 26 (13) 3 (11 -
37/97 (38)
25/79 (32)
P value 0.143
0.965
(40) (39) (171 (4) -
38/82 (46)
10 36 17 3
0.104 0.118 0.368 0.914 0.153 0.593 0.567 0.147
27/70 (39)
0.297
’ Standard deviation.
in” basis. A nurse routinely obtained an informed consent for influenza immunization after explaining the indications, contraindications, and potential side effects to influenza immunization. Influenza immunization was counted only if given at the health center. The data sources for immunization data were the clinical encounter (billing) form and the informed consent. After completion of the study in early January, a single follow-up postcard survey of 100 randomly selected study patients, not documented as immunized at the center, was performed to estimate the proportion of patients who were immunized at other health care sites. Chi-square and Wilcoxon rank sum tests were used to assesswhich patient characteristics individually were predictive of obtaining influenza immunization (Table I). Logistic regression was then used to assesswhich variables were significant in a multivariate analysis via likelihood ratio statistics. The outcome variable was influenza immunization and variables used in the logistic model included treatment group, age (in years), gender, alcohol abuse, cardiac disease, diabetes, pulmonary function, and “risk,” where “risk” was defined as the arithmetical sum of all individual diagnosis risk factors and excluding age alone as a risk factor. Each separate risk factor included in the model was coded in such a
way that absence of the factor was the reference group. The number of risk factors was indicated in the model as 0 = no risk factors, 1 = one risk factor, and 2 = two or more risk factors. Two dummy variables were used to code risk in such a way that absence of any risk factor was the reference group. Patients with immunodeficiency, liver disease, and renal disease were excluded from the logistic regression analysis because there were so few patients with these diagnoses. Both forward and backward stepwise algorithms were used to select a #subsetof variables predictive of obtaining influenza immunization. All two-way interactions between the patient covariates and treatment group were assessed initially. Data on prior immunization were incomplete in the practice information system, so analyses both included and excluded prior year influenza. immunization. Data on prior immunization status was available for 328 patients in the study. Confidence intervals were obtained by exponentiating beta i 1.96 + SE, where beta is the estimate of a factor effect and SE is its standard error. Neither the Homer-Lemeshow nor the CC Brown goodness-of-fit statistic was significant in any model, indicating good fit for the model. Immunization data were entered by keyboard entry to the clinical tracking program (Med/Track, ver-
December
1996
The
American
Journal
of Medicinea
Volume
101
615
INFLUENZA IMMUNIZATION
TABLE
FOR HIGH-RISK PATIENTS/MORAN
ET AL
II Univariate Analysis of Patients Accepting
Total
Influenza Immunization
by Group
Treatment Group (n = 797) Control Brochure n (%) n (%) 41/202 (20) 71/l 98 (36)
57/198(29)
52/l 99 (261
35/l 38 (25)
57/139 (41)
40/l 33 (30)
40/l 53 (26)
29/134(22)
45/128 (35)
39/l 29 (301
32/l 33 (24)
l/17 13/51 9/25 l/4 O/6 14/72 O/l
3/12 21/53 11/28 l/4 O/l 30/69 O/l
(25) (40) (39) (25) (0) (43) (0)
25/79 28/78 16/33 2/8
(32) (36) (48) (25)
Incentive n (%)
Both n (%)
Age
65 or older Gender Female Risk factors Alcoholism Cardiac disease Diabetes mellitus Immune deficiency Liver disease Pulmonary disease Renal disease Number of risk factors (excluding age) None One Two Three Four Prior immunization history when known (n = 328) Immunized in ‘91 Not Immunized in ‘91 The number
of patients
immunized,
total number
of risk and percent
(16) (25) (36) (25) (01
(19) (0)
(19) 18/91 (20)
14/73 7/30
2/7 O/l 16/37 4/60
(23) (9)
(0)
-
(43) (7)
15/25 20/54
lmmunlzed
are presented
(60) (37)
stratified
by factor
4/22
(18)
19/54 (351 11/24 (45)
l/10 (101 9/36
(25)
5/17 (291
l/5 2/2 19/60 2/2
(20) (100) (32) (100)
l/3 o/2 21/62 -
20/74 22/90 lo/24 4/9
(27) (24) (42) (44)
22/101 (22) 23/69 (33) 7/26 (27) o/3 (0)
l/l
(100)
18/38 7/44
(47) (16)
(33) (0) (34)
12/27 (441 8/43
(19)
and group.
sion 2.0, Clinical Software Inc., Hingham, Massachu- n = 400, and incentive n = 397 versus (control n = setts). EpiInfo (version 5.0, Centers for Disease 400) within the factorial design requires that the efControl, Atlanta, Georgia) was used for the initial fect of each intervention not be influenced by the data analysis and subsequent statistical analysis was presence of the other intervention factor. The broperformed using BMDP (version 1990, BMDP Statis- chure incentive first order interaction term was statical Software, Los Angeles, California). tistically significant in the logistic regres;sion models (P = 0.0023) indicating a signiIicant subadditive inRESULTS teraction for the two interventions. Therefore, all A total of 816 patients were identified by searching subsequent analyses were performed as four group the tracking database. Sixteen randomized patients comparisons. were not high risk by age or medical diagnosis, a codImmunization rates by group stratifield by covariing error resulted in a single patient being assignedto ate are presented in Table II. The immunization two groups (n = 2), and one nonexistent medical rec- rate for the educational brochure group was 16% ord number was identified. After excluding these data, higher than control whereas the incentive was 9% 797 patients remained for analysis. Influenza immuni- higher, and the group mailed both exceeded the zation data for the prior year was available for 328 control by only 6%. As shown in Table III, the edstudy patients. The mean age of the study patients was ucational brochure more than doubled the likeli66 years old with an age range of 18 to 99 years. There hood of influenza immunization (odds ratio [OR] = were 524 women (66%) and 273 men (34%). Baseline 2.29, 95% confidence interval [CI] 1.45 to 3.61), whereas the incentive had less of an effect on imdata for the four groups (Table I) showed no stat&itally significant differences among groups by age or munization (OR = 1.68, 95% CI 1.05 to 2.68). Imgender, and no differences in the type or number of munization for the group mailed both interventions disease-relatedhigh-risk indications for patients. There was not significantly different from control. For the was neither a significant difference among the groups subset of individuals for whom prior immunization in the proportion of individuals aged 65 years or status was known, the impact of the educational greater nor the proportion of patients for whom prior brochure was even more significant (OR = 3.95,95% CI 1.92 to S.lO), but the groups mailed incentive or immunization history was known. The main effects comparison between intervenboth interventions were not significantly different tions and control (brochure n = 397 versus control from the control group. 616
December
1996
The American
Journal
of Medicine@
Volume
101
INFLUENZA IMMUNIZATION TABLE
FOR HIGH-RISK PATIENTS/MORAN
ET AL
Ill Factor Significantly Associated with Influenza Immunizations; and those Study Patients with Known Immunization All Patients (n = 797) Odds ratio
Educational Incentive Educational Immunization
brochure
95% Cl 1.45-3.61 1.05-2.68
All Study Patients History Patients with prior immunization known (n = 328) Odds ratio 95% Cl
p value
0.0002
1.59 1.56 4.21 -
0.731-3.34 2.48-7.14 -
0.2460
0.88-2.27 -
0.0004 0.0308 0.1527 -
Age
1.02
1.01-1.03
0.0003
Pulmonary disease Diabetes mellitus Alcoholism
1.80 1.73 -
1.25-2.58 1.09-2.74
0.0015 0.0215
-
-
-
0.395
’ 95% Cl IS derived
from the Wald
test, while the Pvalue
IS derived
from the likelihood
Several other factors were associated with influenza immunization independent of mailed intervention. Inhuenza immunization in the prior year was strongly associated with immunization in the current year (OR = 4.21), and the likelihood of immunization increased with increasing age by about 20%every 10 years. Patients with pulmonary disease (OR = 1.80) and diabetes mellitus (OR = 1.73) were also more likely to be immunized. Patients with a diagnosis of alcohol abuse were somewhat less likely to be immunized, but the difference was significant only in the group where prior immunization was known. Costs for graphic development, printing, prizes, postage, and clerical time, and excluding the cost of the tracking hardware, software, and study personnel, totaled $215.45, or $0.54 per educational brochure, and $298.74, or $0.75 per lottery-type incentive. The total cost for the educational brochure group was $106.92, and with 31 additional immunizations (after adjusting for sample size difference with the control group), the cost effectiveness of the educational brochure was $106.92 for 31, or $3.45 per additional immunization. Similarly, the total cost of the incentive group was $148.50, and with 17 immunizations more than control (after adjustment for sample size difference), the cost effectiveness was $148.50 for 17, or $8.74 per additional immunization. Employing both educational brochure and incentive was not effective, and therefore was not cost effective. A very low response rate (25%) for the follow-up postcard survey is problematic, and although it must be viewed with caution, is presented to illustrate the potential for misclassification bias. Of 100 postcards surveys mailed to patients not documented as immunized at the center, 25 postcards were returned by patients. Four of 25 patients responding reported being immunized at the health center (although consent forms could not be located) and 10 of 25 pa-
ratlo.
p value
1.92-8.10 0.769-3.28
2.29 1.68 1.41 -
brochure and incentive in prior year
history
3.95
Both are presented
0.2113 0.0001 -
0.151-1.03*
0.0405* -
for completeness.
tients reported having obtained immunization other health care sites.
at
DISCUSSION In this study, we examined two mailed interventions to increase influenza immunization: an educational brochure emphasizing factors identified as positively associated with obtaining influenza immunization and addressing patient concerns about the vaccine, and a lottery-type incentive with a prize of free groceries. The odds of immunization for patients in the group mailed the educational brochure were almost twice as great as those of colntrol patients, while the odds of immunization for those mailed the lottery incentive were approximately one and one half times as great. Surprisingly, patients in the group mailed both interventions were less likely to be immunized than those in the single intervention groups, and the proportion immunized this combination group was not significantly different from that in the contro1 group. In randomized trials attempting to incre.ase influenza immunization by prompting individuals to seek immunization, the effects of simple maileld patient reminders, those not designed with a patient educational component, have been inconsiste:nt. While two studies’G,31with high control group irnmunization rates showed no effect of reminders, two large tfials32,“:3with low control rates demonstr
December
1996
The
American
Journal
of Medicine”
Volume
101
617
INFLUENZA IMMUNIZATION
FOR HIGH-RISK PATIENTS/MORAN
rate where prompts and reminders become ineffective.31 The increased infhienza immunization rate achieved by mailing the educational brochure in this study is consistent with the studies of Carter et a1.22,“3Their decision model was developed and tested at a veterans hospital with predominantly (96%) male patients, none of whom had previously been immunized. The modeling process identifies factors that play a critical role in the patient’s decision to accept or reject infhrenza immunization such as personal physician recommendation, patient’s desire to avoid infecting others, and the perceived side effects and complications of the immunization itself. Furthermore, the model seeks to predict patient acceptance or rejection of immunization based on these factors, and interventions were tested on patients who would have been predicted by the model to refuse immunization. Our study targeted a more heterogenous patient population (greater than 60% female), many of whom had accepted immunization in the past year, in the distinctly different setting of a community health center. Thus, it was possible that factors unimportant in the Carter decision model (transportation, access to care, and so forth) could have been more important to the patients in our study. Furthermore, the decision model would have predicted that a significant proportion of our patients would seek immunization without prompting or educational outreach. Nonetheless, the increase of 15% over the control immunization rate of 20% in this study was very similar to incremental increase in immunization noted in the prior study.23 It is possible that the brochure had the effect of “recruiting” more patients for immunization, but we would have expected such an effect from simple mailed reminders in our previous study.” Finally, patients who had not accepted immunization in the prior year were more than three times more likely to accept immunization in the current year when mailed the educational brochure. Thus, despite differences in study population, geographic location, health care practice, and delivery system, this study confirms that a mailed educational brochure based on a patient decision model increased the rate of influenza immunization. The lottery-type incentive intervention led to a more modest increase in influenza immunization rate, which was not as clinically significant. The assumption underlying use of the incentive was that a noneducational means to encourage immunization might result in a short-term change in attitude and behavior regarding immunization. Since prior immunization is strongly associated with acceptance of immunization in subsequent years, the lottery incentive offered the potential to change the long-term at618
December
1996
The American
Journal
of Medicine”
Volume
ET AL
titude and behavior of high-risk individuals around the issue of influenza immunization. Incentives may vary from direct cash payment to groceries,37 clothing, books,26v27 or lottery tickets3’ In the current study, the modest monetary value incentive or the type of the prize (a lottery drawing for three $50 certificates for groceries) in all likelihood influenced the impact of the incentive. Thle monetary amount of the incentive was deliberately chosen as an intervention that could generalize to other health care settings if successful. Furthermore, the amount and type of the incentive were modest because of ethical concerns: the coercive influence of incentives to accept preventive care must be balanced against the risk (and perceived risks) of influenza immunization. Although the safety of the immunization is well demonstrated and the health risk of influenza far exceeds the risks associated with influenza immunization, many patients believe there may be significant health risks from influenza immunization itself. Thus, a modest non-cash lottery incentive was chosen by the center staff and investigators to minimize the “coercive” potential of the incentive among the patients of the health center, many of whom are poor. Interestingly, several patients expressed suspicion about the concept of a lottery as am incentive for preventive health care (KN, personal observation) . This type of modest financial incentive seems most effective when a discreet behavior change is targeted, but incentives have not been uniformly successful in clinical preventive studies. Incentives had a limited effect in efforts to increase the use of bicycle helmets and operating lights, and were ineffective in efforts to increase automobile seat restraint use.38-40 Two studies used ince:ntives with limited success in worksite mammography programs,41~42 yet several studies have successfully employed modest incentives to increase prenatal clinic attendance.37s43 Incentives may be an effective method of initiating behavioral changes in health problems that have been more difficult to influence such as changing the attitudes of patients toward smoking cessation, weight control, and diet changes.44-46 0:n the other hand, in efforts to achieve compliance with directly observed antituberculosis therapy by health departments, incentives may be as effective and less expensive than coercive interventions such as court ordered institutionalization.2” Incentives have also been effective in populations who are traditionally difficult to reach, such as homeless individuals requiring tuberculosis treatmentz7 and injection drug users at risk for HIV infection,47 but apparently not for adolescents seen at a sexually transmitted disease clinic.48 101
INFLUENZA IMMUNIZATION
The results of the combined brochure and incentive group were unexpected although not surprising in light of the reservations and suspicion expressed by some patients about the incentive. However, the effect does suggest that education and incentive influence patient decisions in different ways, and in this trial, an adverse impact of the lottery incentive on patient decisions overcame any educational effect of the brochure. Such subadditive effects have been seen in other studies targeting change in behavior using education, feedback, and incentives to influence behavior,4g and suggests that educational and financial interventions may interact in other health care decisions. The strategy of using a factorial design was chosen to maximize the power to detect the effects of two distinct interventions on influenza immunization without significantly increasing the size of the study population or cost of the trial.“’ This design also allows measurement of an effect of one intervention on the other (ie, interaction), an occurrence that may not be measurable in any other way.51 Unfortunately, the occurrence of an interaction between the interventions resulted in a dramatic decrease in the power of the study to detect an effect, and interaction reduced the study from a two-intervention, single-outcome factorial experiment to a four group trial. Despite the loss of statistical power, the educational brochure resulted in a significant increase in immunization rate, compelling evidence that the brochure is effective. The overall influenza immunization rate was low in this study, but this rate is probably an underestimate for several reasons. The computerized tracking database is not always updated when a patient moves from the area, leaves the practice, or dies. The survey results indicated that a significant proportion of high-risk individuals seek immunizations at health care facilities other than their primary care site, as we have found in another study.52 These influenza immunizations cannot currently be captured by the primary care information systems, making practicebased estimates of influenza immunization rates suspect to significant bias. Finally, several survey respondents reported immunization at the health center for which there was no documentation, and this could reflect oversight by staff members. Successful influenza immunization programs require a primary care centered systematic populationbased approach to maximize immunization of highrisk individuals using clinician and patient prompts, education, and performance feedback.16-‘” As information systems become more common in supporting the coordination of primary care and targeting specific at risk patients for preventive care and screening interventions, it is important to test various outDecember
FOR HIGH-RISK PATIENTS/MORAN
ET AL
reach strategies to assure effectiveness and cost effectiveness. In this study, the incentive had a limited clinical effect at a marginal cost effectiveness of more than $8 per additional immunization. On the other hand, the marginal cost effectiveness of the educational brochure was less than $4 per additional immunization over those in the control group. Larger incentives might increase immunization rates, but it would be difficult to surpass the cost effectiveness of the educational intervention. Furthermore, educational interventions are ethically less likely to be coercive and seem intuitively preferable to financial incentives. McDonald53 has suggested the most cost-effective use of mailed educational materials would result if interventions are used later in the immunization season, after many patients who would accept without prompting have received immunization. The remaining patients would be those individuals who would not otherwise receive immunization or those who have refused in the past, both of which may respond to an educational prompt. This strategy would minimize the cost of mailed educational prompts and effectively contribute to an overall practice strategy to maximize influenza immunization of high-risk individuals.
ACKNOWLEDGEMENTS The authors wish to thank the providers and staff of the Dorchester House MultiService for their cooperation during the study, Carl Spence, Clinical Software Systems Inc., and Del Davis for technical assistance.
REFERENCES 1. Govaert ME, Thjis CTMCN, Masurel N, et al. The efficacy of influenza crnatlon In elderly individuals: a randomized double-blind placebocontrolled JAMA. 1994;272:1661-1665. 2. Mullooly JP, Bennett MD, Hornbrook MC, et al. Influenza vaccination grams for elderly persons: cost-effectiveness in a health maintenance zation. Ann intern Med. 1994;121:947-952. 3. Nichol KL, Margolis KL, Wourena J, Von Sternberg T. The efiicacy
vactrial. proorgani-
and cost-
effectiveness of vaccination against influenza among elderly persons living In the community. NEJM. 1994;331:778-784. 4. Centers for Disease Control. Immunization Practices Advisory Committee (ACIP): prevention and control of Influenza. MMWR. 1995;44(RR-3):1-22. 5. Williams WW, Hickson MA, Kane MA, et al. Immunization policres and vaccine coverage tn adults: the rusk for missed opportunities. Ann Intern Med. 1988; 108:616-625. 6. US Preventive Services Task Force. Gurde to Clinical An Assessment of the Effectiveness of 169 Interventrons.
Preventrve Baltimore:
Services: Williams
and Wilkins: 1989:363-368. 7. Douglas RG. Prophylaxis and treatment of influenza. NEJM. 1990;322:443450. 8. Gardner P, Schafer W. Immunization of adults. NEJM. 1993;32;3:1252-1258. 9. Fedson DS. Adult immunization: summary of the National Aclvisory Committee report. JAMA. 1994;272:1133-1137. 10. Centers for Disease Control and Prevention. Influenza and pneumococcal vaccination coverage levels among persons aged ~65 years--United States, 1973-1993. MMWR. 1995;44:506-515. 11. US Department of Health and Human Services, Public IHealth Service. Healthy People 2000: Nabonal Health Promotion and Disease Prevention ObJectives.
1996
Boston:
The
Jones
and Bartlett;
American
Journal
1992:511-528. of Medicine”
Volume
101
619
12. Nichol KL. Long-term success enza vaccrnation: an instrtution-wide 600.
wrth the National Health Objectrve for rnflumodel. J Gen intern Med. 1992;7:595-
13. Margolls KL, Lofgren RP, Korn JE. Organizational strategies influenza vaccine delivery: a standing order in a general medicine intern Med. 1988;148:2205-2207. 14.
Yano EM, Fink A, Harsh SH, Robbins
AS, Rubenstein
to Improve clinic. Arch
LV. Helping
practices
primary care goals. Arch Intern Med. 1995;155:1146-1156. 15. Herman CJ, Speroff T, Cebul RD. Improving complrance with immunrzation in the older adult: results of a randomized trial. J Am Geriatr Sot.
reach
1994;42:1154-1159, 16. Buffington A target-based
J, Bell KM, LaForce model for Increasing
FM, and the Genesee Hospital Medrcal Staff. influenza immunizations in private practice.
J Gen Intern Med. 1991;6:204-209. 17. Kouides RW, Lewis B, Bennett program for Influenza immunization
NM, et al. A performance-based in the elderly. Am JPrevMed.
255. 18. Bennett NM, Lewis B, Doniger AS, et al. A coordinated, program rn Monroe County, New York, to increase influenza in the elderly.
19. Moran
Incentive
community-wide immunization rates
WP, Nelson K, Wofford J, Velez R. Computer-generated reminders immunization: a clinical trial. J Gen intern Med. 1992;7:535-537.
for influenza
20. Gillick M, Ditzion B. Influenza vaccination: are we doing better Arch Intern Med. 1991;151:1742-1744. 21. Mullooly JP. increasing influenza vaccination among highrisk
than we think? elderly:
a ran-
domized controlled trial of a mail cue In an HMO setting. Am J Pub Health. 1987;77:626-627. 22. Carter WB, Beach LR, lnui TS, et al. Developing and testing a decrsion model for predicting 1986;20:888-932.
influenza
vaccination
complrance.
Health
Serv
Res.
23. Carter WB, Beach LR, lnui TS. The flu shot study: usrng multr-attribute theory to desrgn a vaccination intervention. Org Behav Hum Decis
utility Proc.
1986;38:378-391. 24.
Buchner
DM, Carter
WB, lnui TS. The relahonship
compliance with influenza 1985;23:771-779. 25. Morisky DE, Malotte improve
adherence
immunization: CK, Choi
of attitude
a prospective
P, et al. A patient
rates with antrtuberculosis
changes
study.
Med
education
drug regimens.
to Care.
program Health
to
Educ
Q.
1990;17:253-267. 26.
Pozsik
C, Krnney
antituberculosis
J, Breeden
therapy:
South
D. Approaches Carolina
to Improving
and New York,
adherence
1986-1991.
to
MMWR.
1993;42:74-81. 27. Prlote L, Tulsky JP, Zolopa AR, et al. Tuberculosis control in the homeless: a trial to Improve adherence to referral. Arch Intern Med. 1996;156:161-165. 28. Elder JP, McGraw SA, Rodrigues A, et al. Evaluation of two communitywide smoking cessation 29. Elder JP, Campbell
contests, Prev Med. 1987;16:221-234. NR, Mielchen SD, et al. Implementation
of a community-sponsored
smoking
cessation
contest.
and evaluation
Am J Pub
Health.
1991;5:200-207. 30. Crowley TJ, MacDonald MJ, Zerbe GO, Petty TL. Reinforcing monoxrde reductrons In chronic obstructive pulmonary disease. Depend. 1991;29:47-62.
620
December
1996
The American
Journal
breath carbon Drug Alcohol
of Medicinea
Buchner
DM, Larson
EB, White RF. Influenza
vaccination
elderly:
Impact Fam Phys.
of a com1994;40:
1116-1124. 39. Ferguson B, Blampied NM. Unenlightened: an unsuccessful attempt to promote the use of cycle lights at night. Accid Anal Prev. 1991;23:561-571. 40. Foss RD. Evaluation of a community-wide incentive program to promote safety restraint use. Am J Pub Health. 1989;79:304-306. 41. Turnbull D, lrwig L. Ineffective recrurtment strategres for screening mammography: letterbox drops and invitations for frrends. Aust J Pub Health. 1992;16:79-81. 42. Mayer JA, Jones JA, Eckhardt LE, et al. Evaluation of a worksite mammography program. Am J Prev Med. 1993;9:244-249. 43. Carr C. A four-week observation of materntty care In Finland. J Obstet Gynecol Neonatal Nurs. 1989;18:100-104. 44. Chapman S, Smith W, Mowbray G, et al. Quit and win smoking cessation con tests: how should effectiveness be evaluated? Prev Med. 1993;22:423-432. 45. Lasater TM, Sennett LL, Lefebvre RC, et al. Community-based approach to weight loss: the Pawtucket “weigh-In”. Addrct Behav. 1991;16:175-181. 46. Forster JL, Jeffery RW, Schmid TL, Kramer FM. Preventing weight gain in adults: a pound of prevention. Health Psychol. 1988;7(6):515-525. 47. Deren S, Stephens R, Davis WR, et al. The impact of providing incentives for attendance at AIDS prevention sessions. Pub Health Rep. 1994;109:548554. 48. Chacko MR, Cromer BA, Philltps SA, Glasser D. Farlure of a lottery incentrve to Increase compliance with return visit for test-of-cure culture for Neisseria gonorrhoeae. Sex Transm Dis. 1987;14:75-78. 49. Stock LZ, Milan MA. Improving dietary practices of elderly individuals: the power of prompting feedback, and socral reinforcement. J Appl Behav Anal.
1993;26:379-387. 50.
Bunng
JE, Hennekrns
CH. Cost
and
efficiency
In clinical
trials:
the U.S.
physicians health study. Stat Med. 1990;9:29-33. 51. Daniel WW. Biostatrstics: A Foundatron for Analysis m the Health Scrences. 4th ed. New York: John Wiley and Sons; 987:307. 52. Moran WP, Wofford JL, Velez R. Assessment of influenza immunrzation of community elderly: illustrating the need for community-level health information. Carolina Health Sew Rev. 1995;3:21-29. 53. McDonald JC. Commentary on “Reminders in preventive c,are.” ACPJ Club. Ann Intern Med.
Volume
in community
34. Belcher DW. Implementing preventive services: success and failure in an outpatient trial. Arch Intern Med. 1990;150:2533-2541. 35. Leirer VO, Morrow DG, Parrente G, Doksum T. lncreasirg Influenza vaccination adherence through voice mail. JAm Genatr Sot. 1989;37:1147-1150. 36. Brimberry R. Vaccination of high-risk patients for influenza: a comparison of telephone and mail reminder methods. J Fam Pratt. 1988;26:397-400. 37. Elder JP, Salgado R. The well baby lottery: motivational procedures for increasing attendance at maternal and child health clinics. int J Health Serv. 1988;18:165-171. 38. Rourke LL. Bicycle helmet use among schoolchildren. munity education program and a cycling fatality. Can
1993;9:250-
Med. 1994;154:1741-1745.
Arch Intern
31.
a controlled trial of postcard reminders. J Am Geriatr Sot. 1987;35:755-760. 32. Rosser WW, McDowell I, Newell C. Use of remtnders for preventive procedures in family medicine. Can Med Assoc J. 1991;145:807-814. 33. McDowell I, Newell C, Rosser W. Comparison of three methods of recalling patients for Influenza vaccination. Can Med Assoc J. 1986;135:991-997.
101
1992;116(suppl2k63.
PURPOSE: To determine whether an educational brochure or a lottery-type incentive increases influenza immunization rates. PATIENTS AND METHODS: ln a prospective, Singleblind factorial design randomized trial at an urban community health center, all high-risk patients (n = 797) seen in the preceding 18 months were randomly assigned to one of four groups: a control group; a group mailed a large print, illustrated educational brochure emphasizing factors important to patients in making a decision about influenza immunization; a group mailed a lottery-type incentive announcing that all patients receiving influenza immunization would be eligible for grocery gift certificates; and a group mailed both educational brochure and incentive. Immunization was free, available without an appointment, and recorded by a computerized tracking system. RESULTS: The group mailed the brochure was more likely to be immunized than control (odds ratio [OR] = 2.29, 95% confidence interval [Cl] 1.45 to 3.61), as was the group mailed the incentive (OR = 1.68, 95% Cl 1.05 to 2.68), but there was no difference between the group mailed both interventions and the control group. The effectiveness of the brochure was more striking for individuals who had not accepted immunization in the prior year (OR = 4.21, 95% Cl 2.48 to 7.14), suggesting a true educational effect rather than simply a reminder. CONCLUSION: In this community health center setting, an illustrated educational brochure increased influenza immunization among high-
From the Departments of Medicine and Public Health Sciences, Bow man Gray School of Medicine of Wake Forest University, WinstonSalem, North Caroltna; and the Department of Medicine, Boston City Hospital and Dorchester House Multi-Service Center; Boston, Massachusetts. Supported In part by National Research Service Award, National lnsti tute on Aging (5T32 AG00182). Dr. Moran IS a Robert Wood Johnson Generalist Faculty Scholar. Requests for reprints should be addressed to William P. Moran, MD, MS. Section on General Internal Medicine and Gerontology, Department of Medlclne, Bowman Gray School of Medicine, Medical Center Boulevard, Wlnston-Salem, North Caroltna 27157.1051. Manuscript submttted September 12, 1995 and accepted in revised form September 23, 1996.
612
01996 by Excerpta All rights reserved.
Medica,
Inc.
risk patients, a lottery-type incentive was much less effective, and both together was not effective. 0 1996 by Excerpta Medica, Inc. Am J Med. 1996;101:612-620.
T
here is now overwhelming evidence that a large proportion of the morbidity and mortality resulting from influenza is preventable.‘-” In the United States, epidemics of influenza may cause 20,000 to 40,000 deaths and result in as many as 1.72,OOOhospitalizations. ‘J Annual influenza immunization has been recommended for individuals who are most likely to die or develop serious complications from influenza. These high-risk individuals are persons aged 65 years or older, and persons under 65 with chronic cardiopulmonary disease, metablolic disease (including diabetes mellitus), hemoglobinopathies, immunosuppression, or renal dysfunction.’ Influenza immunization has been shown to reduce mortality, rates of hospitalization, and the severit;y of clinical illness from influenza when the prevalent strain matches that in the vaccine.‘-“,’ Assuming a 70% vaccine efficacy and 70% immunization rate of individuals at high risk, influenza immunization could prevent almost 10,000 deaths and 80,000 hospitalizations annually.‘,” Despite the proven efficacy of influenza immunization, the national rate for influenza immunization of high-risk individuals is estimated to be 40%, although the rate of influenza immunization may be over 50% for individuals age 65 and older.Y*10 Physician knowledge and awareness seems not to be a critical barrier, since the majority of physicians are convinced that influenza is a serious disease, believe influenza immunization is safe and effective, and accept the national recommendations for immunization. It should not be surprising that approximately 70% to 80% of all influenza immunizations are given in physicians’ offices, and high-risk p,atients with physician appointments during the immunization season are likely to be immunized.” The Public Health Service has established the “Healthy People 2000” goal of immunizing 60% to 80% of high-risk individuals each year.” Primary care practices are most likely to achieve this goal if several systematic strategies are implemented.” The practice could establish a “standing order” to immunize all high-risk individuals without requiring an 0002-9343/96/$15.00 PII SOOO2-9:343(96)00327-O
INFLUENZA IMMUNIZATION
FOR HIGH-RISK PATIENTS/MORAN
ET AL
The Flu Shot is FREE and available NOW at the Dorchester House! No appointment needed.
You decide! ’ l The Flu is not just a common cold. l The Flu is a serious illness. l It is very contaaious and you may give it to your friends and family. Figure
1.
a F/u ‘Shot! The modern Flu Shot does got cause the flu. l Your doctor at the Dorchester House wants you to get the flu shot this fall and every year.
is
Get l
*The Flu Shot is s fe and effective Y”,u can get the Fiu Shot even if you are allergic to other medicines. It is safe to get even if you are taking medicines.
The illustrated educational brochure based on the decision model of Carter et al.22~23
individual physician authorization.‘” Clinicians could also be prompted either with paper-based systems or computerized reminders.14 The practice could reassign tasks of staff members to facilitate influenza immunization during the autumn months.‘” Immunization goals could be established for the practice and feedback provided to the practice staff on performance may also effectively improve immunization rate.‘“-l8 Obviously such strategies are effective for patients who have a scheduled appointment in the practice or clinic, but high-risk patients may not have appointments during the months when vaccine is available. 19,20 Therefore, to maximize influenza immunization, outreach strategies to prompt patients to seek influenza immunization are needed. Alerting high-risk patients with mailed reminders for influenza immunization is recommended in efforts to increase immunization among high-risk individuals.“,“’ However, in a prior study at the community health center where the current study was conducted, letter reminders appeared to have no impact on the rate of inlluenza immunization.” To enhance the impact of outreach efforts, an educational component can be added to mailed reminders. In an effort to develop effective educational materials, Carter et alD constructed and tested a decision model to determine which factors were considered important to older patients when deciding to seek or decline influenza immunization. The model suggested that the avoidance of the discomfort and complications of influenza, the desire not to infect others, and a physician’s recommendation favored patient acceptance of influenza immunization. Side effects and complications of the vaccine itself were the major factors against accepting immunization. A
brochure addressing these concerns led to a 15%increase in the influenza immunization rate of older, predominantly male patients who would have been predicted by the model to refuse immunization.“” Thus, interventions that specifically target patients’ attitudes about the side effects and complications of the influenza vaccine may improve compliance with immunization. Such interventions could have a sustained impact on immunization, since patients who accept influenza immunization are also more likely to accept immunization in subsequent years.‘4 Offering a financial incentive to high-risk patients is an alternative to education in efforts to persuade patients to consider accepting influenza irnmunization. Financial incentives to patients have not previously been used to promote influenza irnmunization. Food and other non-cash incentives have been employed to improve compliance with follow-up for a number of health care efforts,“‘12” and cash incentives effectively increased adherence to first followup appointment for tuberculosis therapy among homeless individuals.“’ Rather than a direct financial incentive, however, in this study we offersed an incentive in the form of a modest lottery-type prize to induce high-risk individuals to consider infhienza immunization. Lottery-type prize incentives have been most commonly employed in smoking cessation trials, both in community trials and ln targeted trials in individuals with smoking-related disease.““-“’ Many of the high-risk patients at this community health center participate in the Massachusetts state lottery so a lottery prize, in the form of a grocery gift certificate, was thought by center staff to be an acceptable but less costly alternative to a direct cash incentive. To test individual and joint effects of
December
1996
The American
Journal
of MedicineL
Volume
101
613
INFLUENZA IMMUNIZATION
FOR HIGH-RISK PATIENTS/MORAN
If you get could win
a Flu Shot
ET AL
at the
Dorchester
House,
you
House
wants
tuwo waysl
First, you win because your doctor at the Dorchester you to get the flu shot this fall to prevent the flu.
Second, everyone who gets a flu shot at the Dorchester House has a chance to win one of three gift certificates for $50 worth of groceries.
The Flu Shot is FREE and available Dorchester House. No appointment
NOW
at the
is needed.
Drawing for the gift certificates will be held December 20. 1991. You do not have to be present to win. You must be 18 years or older and have a medical reason for the Flu Shot (everyone age 65 or older. or less than 65 with heart, lung, blood, kidney or liver problems, immune deficiency or diabetes). If you are pregnant, please check with your doctor befoe getting the Flu Shot.
Figure 2. The lottery-type incentive announcement.
education and incentive in increasing influenza immunization, we mailed an illustrated educational brochure and a lottery-type incentive announcement, alone or in combination, to high-risk patients.
formation about where and when the immunization may be obtained. The lottery-type incentive (Figure 2) announced that patients could “win two ways” in accepting influenza immunization: (1) immunization is recomMETHODS mended by the doctors at the center to reduce the This study was conducted during the 1991-1992 chances of getting influenza, and (2) if patients are influenza season on the adult medicine service of an immunized, they are eligible to win one of three grourban community health center, staffed by five full cery store gift certificates. The announcement limor part-time physicians board certified in internal ited eligibility to patients with high-risk indications medicine or family medicine, four resident physi- for the immunization. High-risk patients were randomly allocated to one cians, and one physician assistant. The service proof four groups. Patients in the control group received vides for more than 9,000 ambulatory care visits an“usual care” and were not mailed an intervention. nually by poor to lower income patients. All high-risk The three intervention groups were mailed either the ambulatory patients seen within the preceding 18 educational brochure, the lottery-type i:ncentive, or months and entered into the computerized tracking both the educational brochure and the incentive. The system were included in the study. Patients were interventions were sent by first class mail, the eduidentified by a search of the tracking database using cational brochure initially, and 1 week later the lotage and ICD-9 CM codes corresponding to high-risk tery-type incentive announcements. Twelve interconditions.4 A three-panel educational brochure (Figure 1) ventions were returned by the post office as was designed to address factors shown to influence undeliverable, eight of which were readldressed and remailed, and all were included in the analysis. Since patients’ decision to accept influenza immunization (the “flu shot”) and was based on the model devel- immunization is recommended at least 4 weeks prior oped by Carter et al.15,1B A statement and the line to potential outbreaks of influenza in late January drawings in the brochure indicated that influenza and early February, only immunizations adminis(the “flu”) is a serious disease with potentially severe tered from the beginning of October to the last week consequences, and the caption present.s the choice of December were considered for analysis. The drawing for the lottery-type incentive prizes was facing the patient: “Where will you be this winter?” “You decide . . .” is intended to empower the indi- held during the last week of December. Health care vidual, and “Get a flu shot!” presents the necessary providers and staff at the center were blind to the action clearly. The lower panels emphasize impor- patients’ group assignment. Influenza immunization was given at the health tant decision factors, and two statements addressing the immunization fears of the individuals appear on center free of charge during scheduled provider visthe lower second panel. The third panel contains in- its, at a “health fair” in late October, or on a “walk614
December
1996
The American
Journal
of Medicine@
Volume
101
TABLE
I Baseline Comparison
of Covariates by Group Treatment Brochure n (%)
Control n (%) Total Age Mean age/SD* Median age Age range Age 65 or older Gender Female Risk factors Alcoholism Cardiac disease Diabetes mellitus lmmunodeficiency Liver disease Pulmonary disease Renal disease Number of risk factors (excluding age) None One Two Three Four Prior immunization history when known (n = 328) Immunized prior year/total
Group (n = 797) incentive n (%)
202 (100)
198 (100)
198 (100)
Both n (%) 199 (100)000-000
66/16.6 70 18-99 138 (68)
65/l 5.9 69 19-92 139 (70)
65/l 7.2 69 19-92 133 (67)
68/15.2 71 21-95 153 (77)
134 (66)
128 (65)
129 (65)
133 (671
17 (8) 51 (25) 25 (12) 4 (2) 6 (3) 72 (36) l(l)
12 (6) 53 (27) 28 (14) 4 (2) 1 (1) 69 (35) l(l)
22 54 24 5 2 60
(11) (27) (12) (3) (1) (30) 2 (1)
(5) (18) (9) (2) 2 (1) 62 (31) -
73 (36) 91 (49) 30 (14) 7 (3) 1 (1)
79 78 33 8
74 (37) 90 (45) 24 (12) 9 (5) l(l)
101 (50) 69 (35) 26 (13) 3 (11 -
37/97 (38)
25/79 (32)
P value 0.143
0.965
(40) (39) (171 (4) -
38/82 (46)
10 36 17 3
0.104 0.118 0.368 0.914 0.153 0.593 0.567 0.147
27/70 (39)
0.297
’ Standard deviation.
in” basis. A nurse routinely obtained an informed consent for influenza immunization after explaining the indications, contraindications, and potential side effects to influenza immunization. Influenza immunization was counted only if given at the health center. The data sources for immunization data were the clinical encounter (billing) form and the informed consent. After completion of the study in early January, a single follow-up postcard survey of 100 randomly selected study patients, not documented as immunized at the center, was performed to estimate the proportion of patients who were immunized at other health care sites. Chi-square and Wilcoxon rank sum tests were used to assesswhich patient characteristics individually were predictive of obtaining influenza immunization (Table I). Logistic regression was then used to assesswhich variables were significant in a multivariate analysis via likelihood ratio statistics. The outcome variable was influenza immunization and variables used in the logistic model included treatment group, age (in years), gender, alcohol abuse, cardiac disease, diabetes, pulmonary function, and “risk,” where “risk” was defined as the arithmetical sum of all individual diagnosis risk factors and excluding age alone as a risk factor. Each separate risk factor included in the model was coded in such a
way that absence of the factor was the reference group. The number of risk factors was indicated in the model as 0 = no risk factors, 1 = one risk factor, and 2 = two or more risk factors. Two dummy variables were used to code risk in such a way that absence of any risk factor was the reference group. Patients with immunodeficiency, liver disease, and renal disease were excluded from the logistic regression analysis because there were so few patients with these diagnoses. Both forward and backward stepwise algorithms were used to select a #subsetof variables predictive of obtaining influenza immunization. All two-way interactions between the patient covariates and treatment group were assessed initially. Data on prior immunization were incomplete in the practice information system, so analyses both included and excluded prior year influenza. immunization. Data on prior immunization status was available for 328 patients in the study. Confidence intervals were obtained by exponentiating beta i 1.96 + SE, where beta is the estimate of a factor effect and SE is its standard error. Neither the Homer-Lemeshow nor the CC Brown goodness-of-fit statistic was significant in any model, indicating good fit for the model. Immunization data were entered by keyboard entry to the clinical tracking program (Med/Track, ver-
December
1996
The
American
Journal
of Medicinea
Volume
101
615
INFLUENZA IMMUNIZATION
TABLE
FOR HIGH-RISK PATIENTS/MORAN
ET AL
II Univariate Analysis of Patients Accepting
Total
Influenza Immunization
by Group
Treatment Group (n = 797) Control Brochure n (%) n (%) 41/202 (20) 71/l 98 (36)
57/198(29)
52/l 99 (261
35/l 38 (25)
57/139 (41)
40/l 33 (30)
40/l 53 (26)
29/134(22)
45/128 (35)
39/l 29 (301
32/l 33 (24)
l/17 13/51 9/25 l/4 O/6 14/72 O/l
3/12 21/53 11/28 l/4 O/l 30/69 O/l
(25) (40) (39) (25) (0) (43) (0)
25/79 28/78 16/33 2/8
(32) (36) (48) (25)
Incentive n (%)
Both n (%)
Age
65 or older Gender Female Risk factors Alcoholism Cardiac disease Diabetes mellitus Immune deficiency Liver disease Pulmonary disease Renal disease Number of risk factors (excluding age) None One Two Three Four Prior immunization history when known (n = 328) Immunized in ‘91 Not Immunized in ‘91 The number
of patients
immunized,
total number
of risk and percent
(16) (25) (36) (25) (01
(19) (0)
(19) 18/91 (20)
14/73 7/30
2/7 O/l 16/37 4/60
(23) (9)
(0)
-
(43) (7)
15/25 20/54
lmmunlzed
are presented
(60) (37)
stratified
by factor
4/22
(18)
19/54 (351 11/24 (45)
l/10 (101 9/36
(25)
5/17 (291
l/5 2/2 19/60 2/2
(20) (100) (32) (100)
l/3 o/2 21/62 -
20/74 22/90 lo/24 4/9
(27) (24) (42) (44)
22/101 (22) 23/69 (33) 7/26 (27) o/3 (0)
l/l
(100)
18/38 7/44
(47) (16)
(33) (0) (34)
12/27 (441 8/43
(19)
and group.
sion 2.0, Clinical Software Inc., Hingham, Massachu- n = 400, and incentive n = 397 versus (control n = setts). EpiInfo (version 5.0, Centers for Disease 400) within the factorial design requires that the efControl, Atlanta, Georgia) was used for the initial fect of each intervention not be influenced by the data analysis and subsequent statistical analysis was presence of the other intervention factor. The broperformed using BMDP (version 1990, BMDP Statis- chure incentive first order interaction term was statical Software, Los Angeles, California). tistically significant in the logistic regres;sion models (P = 0.0023) indicating a signiIicant subadditive inRESULTS teraction for the two interventions. Therefore, all A total of 816 patients were identified by searching subsequent analyses were performed as four group the tracking database. Sixteen randomized patients comparisons. were not high risk by age or medical diagnosis, a codImmunization rates by group stratifield by covariing error resulted in a single patient being assignedto ate are presented in Table II. The immunization two groups (n = 2), and one nonexistent medical rec- rate for the educational brochure group was 16% ord number was identified. After excluding these data, higher than control whereas the incentive was 9% 797 patients remained for analysis. Influenza immuni- higher, and the group mailed both exceeded the zation data for the prior year was available for 328 control by only 6%. As shown in Table III, the edstudy patients. The mean age of the study patients was ucational brochure more than doubled the likeli66 years old with an age range of 18 to 99 years. There hood of influenza immunization (odds ratio [OR] = were 524 women (66%) and 273 men (34%). Baseline 2.29, 95% confidence interval [CI] 1.45 to 3.61), whereas the incentive had less of an effect on imdata for the four groups (Table I) showed no stat&itally significant differences among groups by age or munization (OR = 1.68, 95% CI 1.05 to 2.68). Imgender, and no differences in the type or number of munization for the group mailed both interventions disease-relatedhigh-risk indications for patients. There was not significantly different from control. For the was neither a significant difference among the groups subset of individuals for whom prior immunization in the proportion of individuals aged 65 years or status was known, the impact of the educational greater nor the proportion of patients for whom prior brochure was even more significant (OR = 3.95,95% CI 1.92 to S.lO), but the groups mailed incentive or immunization history was known. The main effects comparison between intervenboth interventions were not significantly different tions and control (brochure n = 397 versus control from the control group. 616
December
1996
The American
Journal
of Medicine@
Volume
101
INFLUENZA IMMUNIZATION TABLE
FOR HIGH-RISK PATIENTS/MORAN
ET AL
Ill Factor Significantly Associated with Influenza Immunizations; and those Study Patients with Known Immunization All Patients (n = 797) Odds ratio
Educational Incentive Educational Immunization
brochure
95% Cl 1.45-3.61 1.05-2.68
All Study Patients History Patients with prior immunization known (n = 328) Odds ratio 95% Cl
p value
0.0002
1.59 1.56 4.21 -
0.731-3.34 2.48-7.14 -
0.2460
0.88-2.27 -
0.0004 0.0308 0.1527 -
Age
1.02
1.01-1.03
0.0003
Pulmonary disease Diabetes mellitus Alcoholism
1.80 1.73 -
1.25-2.58 1.09-2.74
0.0015 0.0215
-
-
-
0.395
’ 95% Cl IS derived
from the Wald
test, while the Pvalue
IS derived
from the likelihood
Several other factors were associated with influenza immunization independent of mailed intervention. Inhuenza immunization in the prior year was strongly associated with immunization in the current year (OR = 4.21), and the likelihood of immunization increased with increasing age by about 20%every 10 years. Patients with pulmonary disease (OR = 1.80) and diabetes mellitus (OR = 1.73) were also more likely to be immunized. Patients with a diagnosis of alcohol abuse were somewhat less likely to be immunized, but the difference was significant only in the group where prior immunization was known. Costs for graphic development, printing, prizes, postage, and clerical time, and excluding the cost of the tracking hardware, software, and study personnel, totaled $215.45, or $0.54 per educational brochure, and $298.74, or $0.75 per lottery-type incentive. The total cost for the educational brochure group was $106.92, and with 31 additional immunizations (after adjusting for sample size difference with the control group), the cost effectiveness of the educational brochure was $106.92 for 31, or $3.45 per additional immunization. Similarly, the total cost of the incentive group was $148.50, and with 17 immunizations more than control (after adjustment for sample size difference), the cost effectiveness was $148.50 for 17, or $8.74 per additional immunization. Employing both educational brochure and incentive was not effective, and therefore was not cost effective. A very low response rate (25%) for the follow-up postcard survey is problematic, and although it must be viewed with caution, is presented to illustrate the potential for misclassification bias. Of 100 postcards surveys mailed to patients not documented as immunized at the center, 25 postcards were returned by patients. Four of 25 patients responding reported being immunized at the health center (although consent forms could not be located) and 10 of 25 pa-
ratlo.
p value
1.92-8.10 0.769-3.28
2.29 1.68 1.41 -
brochure and incentive in prior year
history
3.95
Both are presented
0.2113 0.0001 -
0.151-1.03*
0.0405* -
for completeness.
tients reported having obtained immunization other health care sites.
at
DISCUSSION In this study, we examined two mailed interventions to increase influenza immunization: an educational brochure emphasizing factors identified as positively associated with obtaining influenza immunization and addressing patient concerns about the vaccine, and a lottery-type incentive with a prize of free groceries. The odds of immunization for patients in the group mailed the educational brochure were almost twice as great as those of colntrol patients, while the odds of immunization for those mailed the lottery incentive were approximately one and one half times as great. Surprisingly, patients in the group mailed both interventions were less likely to be immunized than those in the single intervention groups, and the proportion immunized this combination group was not significantly different from that in the contro1 group. In randomized trials attempting to incre.ase influenza immunization by prompting individuals to seek immunization, the effects of simple maileld patient reminders, those not designed with a patient educational component, have been inconsiste:nt. While two studies’G,31with high control group irnmunization rates showed no effect of reminders, two large tfials32,“:3with low control rates demonstr
December
1996
The
American
Journal
of Medicine”
Volume
101
617
INFLUENZA IMMUNIZATION
FOR HIGH-RISK PATIENTS/MORAN
rate where prompts and reminders become ineffective.31 The increased infhienza immunization rate achieved by mailing the educational brochure in this study is consistent with the studies of Carter et a1.22,“3Their decision model was developed and tested at a veterans hospital with predominantly (96%) male patients, none of whom had previously been immunized. The modeling process identifies factors that play a critical role in the patient’s decision to accept or reject infhrenza immunization such as personal physician recommendation, patient’s desire to avoid infecting others, and the perceived side effects and complications of the immunization itself. Furthermore, the model seeks to predict patient acceptance or rejection of immunization based on these factors, and interventions were tested on patients who would have been predicted by the model to refuse immunization. Our study targeted a more heterogenous patient population (greater than 60% female), many of whom had accepted immunization in the past year, in the distinctly different setting of a community health center. Thus, it was possible that factors unimportant in the Carter decision model (transportation, access to care, and so forth) could have been more important to the patients in our study. Furthermore, the decision model would have predicted that a significant proportion of our patients would seek immunization without prompting or educational outreach. Nonetheless, the increase of 15% over the control immunization rate of 20% in this study was very similar to incremental increase in immunization noted in the prior study.23 It is possible that the brochure had the effect of “recruiting” more patients for immunization, but we would have expected such an effect from simple mailed reminders in our previous study.” Finally, patients who had not accepted immunization in the prior year were more than three times more likely to accept immunization in the current year when mailed the educational brochure. Thus, despite differences in study population, geographic location, health care practice, and delivery system, this study confirms that a mailed educational brochure based on a patient decision model increased the rate of influenza immunization. The lottery-type incentive intervention led to a more modest increase in influenza immunization rate, which was not as clinically significant. The assumption underlying use of the incentive was that a noneducational means to encourage immunization might result in a short-term change in attitude and behavior regarding immunization. Since prior immunization is strongly associated with acceptance of immunization in subsequent years, the lottery incentive offered the potential to change the long-term at618
December
1996
The American
Journal
of Medicine”
Volume
ET AL
titude and behavior of high-risk individuals around the issue of influenza immunization. Incentives may vary from direct cash payment to groceries,37 clothing, books,26v27 or lottery tickets3’ In the current study, the modest monetary value incentive or the type of the prize (a lottery drawing for three $50 certificates for groceries) in all likelihood influenced the impact of the incentive. Thle monetary amount of the incentive was deliberately chosen as an intervention that could generalize to other health care settings if successful. Furthermore, the amount and type of the incentive were modest because of ethical concerns: the coercive influence of incentives to accept preventive care must be balanced against the risk (and perceived risks) of influenza immunization. Although the safety of the immunization is well demonstrated and the health risk of influenza far exceeds the risks associated with influenza immunization, many patients believe there may be significant health risks from influenza immunization itself. Thus, a modest non-cash lottery incentive was chosen by the center staff and investigators to minimize the “coercive” potential of the incentive among the patients of the health center, many of whom are poor. Interestingly, several patients expressed suspicion about the concept of a lottery as am incentive for preventive health care (KN, personal observation) . This type of modest financial incentive seems most effective when a discreet behavior change is targeted, but incentives have not been uniformly successful in clinical preventive studies. Incentives had a limited effect in efforts to increase the use of bicycle helmets and operating lights, and were ineffective in efforts to increase automobile seat restraint use.38-40 Two studies used ince:ntives with limited success in worksite mammography programs,41~42 yet several studies have successfully employed modest incentives to increase prenatal clinic attendance.37s43 Incentives may be an effective method of initiating behavioral changes in health problems that have been more difficult to influence such as changing the attitudes of patients toward smoking cessation, weight control, and diet changes.44-46 0:n the other hand, in efforts to achieve compliance with directly observed antituberculosis therapy by health departments, incentives may be as effective and less expensive than coercive interventions such as court ordered institutionalization.2” Incentives have also been effective in populations who are traditionally difficult to reach, such as homeless individuals requiring tuberculosis treatmentz7 and injection drug users at risk for HIV infection,47 but apparently not for adolescents seen at a sexually transmitted disease clinic.48 101
INFLUENZA IMMUNIZATION
The results of the combined brochure and incentive group were unexpected although not surprising in light of the reservations and suspicion expressed by some patients about the incentive. However, the effect does suggest that education and incentive influence patient decisions in different ways, and in this trial, an adverse impact of the lottery incentive on patient decisions overcame any educational effect of the brochure. Such subadditive effects have been seen in other studies targeting change in behavior using education, feedback, and incentives to influence behavior,4g and suggests that educational and financial interventions may interact in other health care decisions. The strategy of using a factorial design was chosen to maximize the power to detect the effects of two distinct interventions on influenza immunization without significantly increasing the size of the study population or cost of the trial.“’ This design also allows measurement of an effect of one intervention on the other (ie, interaction), an occurrence that may not be measurable in any other way.51 Unfortunately, the occurrence of an interaction between the interventions resulted in a dramatic decrease in the power of the study to detect an effect, and interaction reduced the study from a two-intervention, single-outcome factorial experiment to a four group trial. Despite the loss of statistical power, the educational brochure resulted in a significant increase in immunization rate, compelling evidence that the brochure is effective. The overall influenza immunization rate was low in this study, but this rate is probably an underestimate for several reasons. The computerized tracking database is not always updated when a patient moves from the area, leaves the practice, or dies. The survey results indicated that a significant proportion of high-risk individuals seek immunizations at health care facilities other than their primary care site, as we have found in another study.52 These influenza immunizations cannot currently be captured by the primary care information systems, making practicebased estimates of influenza immunization rates suspect to significant bias. Finally, several survey respondents reported immunization at the health center for which there was no documentation, and this could reflect oversight by staff members. Successful influenza immunization programs require a primary care centered systematic populationbased approach to maximize immunization of highrisk individuals using clinician and patient prompts, education, and performance feedback.16-‘” As information systems become more common in supporting the coordination of primary care and targeting specific at risk patients for preventive care and screening interventions, it is important to test various outDecember
FOR HIGH-RISK PATIENTS/MORAN
ET AL
reach strategies to assure effectiveness and cost effectiveness. In this study, the incentive had a limited clinical effect at a marginal cost effectiveness of more than $8 per additional immunization. On the other hand, the marginal cost effectiveness of the educational brochure was less than $4 per additional immunization over those in the control group. Larger incentives might increase immunization rates, but it would be difficult to surpass the cost effectiveness of the educational intervention. Furthermore, educational interventions are ethically less likely to be coercive and seem intuitively preferable to financial incentives. McDonald53 has suggested the most cost-effective use of mailed educational materials would result if interventions are used later in the immunization season, after many patients who would accept without prompting have received immunization. The remaining patients would be those individuals who would not otherwise receive immunization or those who have refused in the past, both of which may respond to an educational prompt. This strategy would minimize the cost of mailed educational prompts and effectively contribute to an overall practice strategy to maximize influenza immunization of high-risk individuals.
ACKNOWLEDGEMENTS The authors wish to thank the providers and staff of the Dorchester House MultiService for their cooperation during the study, Carl Spence, Clinical Software Systems Inc., and Del Davis for technical assistance.
REFERENCES 1. Govaert ME, Thjis CTMCN, Masurel N, et al. The efficacy of influenza crnatlon In elderly individuals: a randomized double-blind placebocontrolled JAMA. 1994;272:1661-1665. 2. Mullooly JP, Bennett MD, Hornbrook MC, et al. Influenza vaccination grams for elderly persons: cost-effectiveness in a health maintenance zation. Ann intern Med. 1994;121:947-952. 3. Nichol KL, Margolis KL, Wourena J, Von Sternberg T. The efiicacy
vactrial. proorgani-
and cost-
effectiveness of vaccination against influenza among elderly persons living In the community. NEJM. 1994;331:778-784. 4. Centers for Disease Control. Immunization Practices Advisory Committee (ACIP): prevention and control of Influenza. MMWR. 1995;44(RR-3):1-22. 5. Williams WW, Hickson MA, Kane MA, et al. Immunization policres and vaccine coverage tn adults: the rusk for missed opportunities. Ann Intern Med. 1988; 108:616-625. 6. US Preventive Services Task Force. Gurde to Clinical An Assessment of the Effectiveness of 169 Interventrons.
Preventrve Baltimore:
Services: Williams
and Wilkins: 1989:363-368. 7. Douglas RG. Prophylaxis and treatment of influenza. NEJM. 1990;322:443450. 8. Gardner P, Schafer W. Immunization of adults. NEJM. 1993;32;3:1252-1258. 9. Fedson DS. Adult immunization: summary of the National Aclvisory Committee report. JAMA. 1994;272:1133-1137. 10. Centers for Disease Control and Prevention. Influenza and pneumococcal vaccination coverage levels among persons aged ~65 years--United States, 1973-1993. MMWR. 1995;44:506-515. 11. US Department of Health and Human Services, Public IHealth Service. Healthy People 2000: Nabonal Health Promotion and Disease Prevention ObJectives.
1996
Boston:
The
Jones
and Bartlett;
American
Journal
1992:511-528. of Medicine”
Volume
101
619
12. Nichol KL. Long-term success enza vaccrnation: an instrtution-wide 600.
wrth the National Health Objectrve for rnflumodel. J Gen intern Med. 1992;7:595-
13. Margolls KL, Lofgren RP, Korn JE. Organizational strategies influenza vaccine delivery: a standing order in a general medicine intern Med. 1988;148:2205-2207. 14.
Yano EM, Fink A, Harsh SH, Robbins
AS, Rubenstein
to Improve clinic. Arch
LV. Helping
practices
primary care goals. Arch Intern Med. 1995;155:1146-1156. 15. Herman CJ, Speroff T, Cebul RD. Improving complrance with immunrzation in the older adult: results of a randomized trial. J Am Geriatr Sot.
reach
1994;42:1154-1159, 16. Buffington A target-based
J, Bell KM, LaForce model for Increasing
FM, and the Genesee Hospital Medrcal Staff. influenza immunizations in private practice.
J Gen Intern Med. 1991;6:204-209. 17. Kouides RW, Lewis B, Bennett program for Influenza immunization
NM, et al. A performance-based in the elderly. Am JPrevMed.
255. 18. Bennett NM, Lewis B, Doniger AS, et al. A coordinated, program rn Monroe County, New York, to increase influenza in the elderly.
19. Moran
Incentive
community-wide immunization rates
WP, Nelson K, Wofford J, Velez R. Computer-generated reminders immunization: a clinical trial. J Gen intern Med. 1992;7:535-537.
for influenza
20. Gillick M, Ditzion B. Influenza vaccination: are we doing better Arch Intern Med. 1991;151:1742-1744. 21. Mullooly JP. increasing influenza vaccination among highrisk
than we think? elderly:
a ran-
domized controlled trial of a mail cue In an HMO setting. Am J Pub Health. 1987;77:626-627. 22. Carter WB, Beach LR, lnui TS, et al. Developing and testing a decrsion model for predicting 1986;20:888-932.
influenza
vaccination
complrance.
Health
Serv
Res.
23. Carter WB, Beach LR, lnui TS. The flu shot study: usrng multr-attribute theory to desrgn a vaccination intervention. Org Behav Hum Decis
utility Proc.
1986;38:378-391. 24.
Buchner
DM, Carter
WB, lnui TS. The relahonship
compliance with influenza 1985;23:771-779. 25. Morisky DE, Malotte improve
adherence
immunization: CK, Choi
of attitude
a prospective
P, et al. A patient
rates with antrtuberculosis
changes
study.
Med
education
drug regimens.
to Care.
program Health
to
Educ
Q.
1990;17:253-267. 26.
Pozsik
C, Krnney
antituberculosis
J, Breeden
therapy:
South
D. Approaches Carolina
to Improving
and New York,
adherence
1986-1991.
to
MMWR.
1993;42:74-81. 27. Prlote L, Tulsky JP, Zolopa AR, et al. Tuberculosis control in the homeless: a trial to Improve adherence to referral. Arch Intern Med. 1996;156:161-165. 28. Elder JP, McGraw SA, Rodrigues A, et al. Evaluation of two communitywide smoking cessation 29. Elder JP, Campbell
contests, Prev Med. 1987;16:221-234. NR, Mielchen SD, et al. Implementation
of a community-sponsored
smoking
cessation
contest.
and evaluation
Am J Pub
Health.
1991;5:200-207. 30. Crowley TJ, MacDonald MJ, Zerbe GO, Petty TL. Reinforcing monoxrde reductrons In chronic obstructive pulmonary disease. Depend. 1991;29:47-62.
620
December
1996
The American
Journal
breath carbon Drug Alcohol
of Medicinea
Buchner
DM, Larson
EB, White RF. Influenza
vaccination
elderly:
Impact Fam Phys.
of a com1994;40:
1116-1124. 39. Ferguson B, Blampied NM. Unenlightened: an unsuccessful attempt to promote the use of cycle lights at night. Accid Anal Prev. 1991;23:561-571. 40. Foss RD. Evaluation of a community-wide incentive program to promote safety restraint use. Am J Pub Health. 1989;79:304-306. 41. Turnbull D, lrwig L. Ineffective recrurtment strategres for screening mammography: letterbox drops and invitations for frrends. Aust J Pub Health. 1992;16:79-81. 42. Mayer JA, Jones JA, Eckhardt LE, et al. Evaluation of a worksite mammography program. Am J Prev Med. 1993;9:244-249. 43. Carr C. A four-week observation of materntty care In Finland. J Obstet Gynecol Neonatal Nurs. 1989;18:100-104. 44. Chapman S, Smith W, Mowbray G, et al. Quit and win smoking cessation con tests: how should effectiveness be evaluated? Prev Med. 1993;22:423-432. 45. Lasater TM, Sennett LL, Lefebvre RC, et al. Community-based approach to weight loss: the Pawtucket “weigh-In”. Addrct Behav. 1991;16:175-181. 46. Forster JL, Jeffery RW, Schmid TL, Kramer FM. Preventing weight gain in adults: a pound of prevention. Health Psychol. 1988;7(6):515-525. 47. Deren S, Stephens R, Davis WR, et al. The impact of providing incentives for attendance at AIDS prevention sessions. Pub Health Rep. 1994;109:548554. 48. Chacko MR, Cromer BA, Philltps SA, Glasser D. Farlure of a lottery incentrve to Increase compliance with return visit for test-of-cure culture for Neisseria gonorrhoeae. Sex Transm Dis. 1987;14:75-78. 49. Stock LZ, Milan MA. Improving dietary practices of elderly individuals: the power of prompting feedback, and socral reinforcement. J Appl Behav Anal.
1993;26:379-387. 50.
Bunng
JE, Hennekrns
CH. Cost
and
efficiency
In clinical
trials:
the U.S.
physicians health study. Stat Med. 1990;9:29-33. 51. Daniel WW. Biostatrstics: A Foundatron for Analysis m the Health Scrences. 4th ed. New York: John Wiley and Sons; 987:307. 52. Moran WP, Wofford JL, Velez R. Assessment of influenza immunrzation of community elderly: illustrating the need for community-level health information. Carolina Health Sew Rev. 1995;3:21-29. 53. McDonald JC. Commentary on “Reminders in preventive c,are.” ACPJ Club. Ann Intern Med.
Volume
in community
34. Belcher DW. Implementing preventive services: success and failure in an outpatient trial. Arch Intern Med. 1990;150:2533-2541. 35. Leirer VO, Morrow DG, Parrente G, Doksum T. lncreasirg Influenza vaccination adherence through voice mail. JAm Genatr Sot. 1989;37:1147-1150. 36. Brimberry R. Vaccination of high-risk patients for influenza: a comparison of telephone and mail reminder methods. J Fam Pratt. 1988;26:397-400. 37. Elder JP, Salgado R. The well baby lottery: motivational procedures for increasing attendance at maternal and child health clinics. int J Health Serv. 1988;18:165-171. 38. Rourke LL. Bicycle helmet use among schoolchildren. munity education program and a cycling fatality. Can
1993;9:250-
Med. 1994;154:1741-1745.
Arch Intern
31.
a controlled trial of postcard reminders. J Am Geriatr Sot. 1987;35:755-760. 32. Rosser WW, McDowell I, Newell C. Use of remtnders for preventive procedures in family medicine. Can Med Assoc J. 1991;145:807-814. 33. McDowell I, Newell C, Rosser W. Comparison of three methods of recalling patients for Influenza vaccination. Can Med Assoc J. 1986;135:991-997.
101
1992;116(suppl2k63.