Consumer decision making for using comprehensive medication review services

SCIENCE AND PRACTICE Journal of the American Pharmacists Association xxx (2018) 1e10

Contents lists available at ScienceDirect

Journal of the American Pharmacists Association journal homepage: www.japha.org

RESEARCH

Consumer decision making for using comprehensive medication review services Yiran Zhang, William R. Doucette* a r t i c l e i n f o

a b s t r a c t

Article history: Received 25 April 2018 Accepted 7 November 2018

Objectives: To identify main factors associated with older adults’ decision making for using a Medicare Part D comprehensive medication review (CMR) service. Design: Cross-sectional self-administered mailed survey. Setting and participants: The survey was conducted from December 2016 to February 2017. Sampled subjects were 1) at least 65 years of age, 2) taking at least 1 prescription medication, and 3) a Medicare Part D beneficiary living in Florida (n ¼ 310), Washington (n ¼ 310), Wisconsin (n ¼ 310), or Pennsylvania (n ¼ 310) or active members of an Iowa senior registry (n ¼ 460). Main outcome measures: Responses to survey items assessing factors in the domains of internal need, external influences, perceived risks of using CMRs, and alternatives comparison that may affect older adults’ decision to use CMRs. Results: The overall completed response rate was 24% (n ¼ 381). About 28% of respondents (n ¼ 105) reported being a CMR recipient. Recommendations from a pharmacist (P < 0.0001) or a physician (P ¼ 0.0350), pharmacist’s communication in previous encounters (P ¼ 0.0007), perceived susceptibility to medication-related problems (P < 0.0001), and positive outcome expectancy (P ¼ 0.0147) were positively associated with consumers’ decision to participate in CMRs, whereas perceived functional risk (P < 0.0001), access to general counseling in previous experiences (P ¼ 0.0145), and family or friends’ influence (P ¼ 0.0065) were negatively associated factors. Conclusion: CMR uptake remains low after being available for years. Recommendations from health professionals and understanding of service benefits were identified as main factors affecting consumers’ decision making for participating in CMRs. Policy makers could consider 1) seeking collaboration with community pharmacists and physicians and 2) addressing key components and benefits of CMRs in older adults as new promotion strategies. © 2018 American Pharmacists Association®. Published by Elsevier Inc. All rights reserved.

Elderly patients have a high risk of experiencing medication-related problems (MRPs), particularly in problems related to safety and nonadherence. This risk typically results from the combined effects of their use of multiple medications and the recession of biologic functions owing to aging.

Disclosure: The authors declare no conflicts of interest or financial interests in any product or service mentioned in this article, including grants, employment, gifts, stock, holdings, or honoraria. Funding: This research was sponsored by the Deborah K. Veale Professorship in Healthcare Policy. Previous presentation of the work: This work has been presented as a podium presentation at American Pharmacist Association (APhA) Annual Meeting, Nashville TN, March 16e19, 2018. * Correspondence: William R. Doucette, PhD, College of Pharmacy, University of Iowa, 115 S. Grand Ave., S518 PHAR, Iowa City, IA 52242. E-mail address: [email protected] (W.R. Doucette).

Undoubtedly, such problems negatively affect older adults’ health and quality of life often lead to a financial burden that comes from additional use of health care resources.1,2 Fortunately, many MRPs are preventable.3-5 One approach for older adults to reduce the risk of MRPs is to participate in medication therapy management (MTM) services. Recognizing the importance of medication therapy reviews, the federal government required the implementation of MTM services among eligible Medicare Part D beneficiaries in 2006.6 The provision of an annual comprehensive medication review (CMR) is considered to be a fundamental service in MTM programs.6,7 A CMR is an interaction between a patient and a health care provider, primarily a pharmacist, to identify MRPs and provide solutions accordingly.7,8 Unlike general medication counseling, a CMR provides a comprehensive assessment and education for all of the medications a patient

https://doi.org/10.1016/j.japh.2018.11.003 1544-3191/© 2018 American Pharmacists Association®. Published by Elsevier Inc. All rights reserved.

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

Key Points Background:  Given the underuse of comprehensive medication reviews (CMRs) among older adults in the United States, current policies from the government to strategically promote the service have been focused on Medicare Part D plan sponsors but not on patients or consumers.  There was insufficient evidence in current literature to comprehensively assess factors affecting older adults' uptake of CMRs from a patient or consumer perspective.  Owing to the lack of a guiding theory, limited potential facilitators or barriers were explored separately and descriptively in the literature, without hypothesis testing or theoretical interpretation. Findings:  This study adapted consumer behavior theories to comprehensively explore factors affecting older adults’ decision making for participating in Medicare Part D CMRs.  Recommendations from health professionals and understanding of service benefits were identified as main factors affecting consumers’ decision making for using CMRs.  Seeking collaboration from community pharmacists and physicians and addressing key components and benefits of CMRs in older adults may be considered as strategies for promoting CMRs to Part D beneficiaries.

is taking, including not only prescriptions but also over-thecounter (OTC) medications, herbal therapies, and dietary supplements. To further promote CMRs among Medicare beneficiaries, the Centers for Medicare and Medicaid Services (CMS) enacted a series of policies to support CMR availability. As of 2018, all MTM-eligible beneficiaries enrolled in Medicare Part D plans are offered an annual free CMR with a result reported in a standardized format.8-10 The qualifications for being an MTM-eligible beneficiary vary across different plan sponsors. However, according to the general requirement set by CMS, all Medicare Part D beneficiaries who meet all 3 of the following criteria should automatically be enrolled into the MTM program and thus be eligible for the free annual CMR: 1) have more than 1 chronic health condition, 2) take several different medications (minimum threshold is any number from 2 to 8) for those health conditions, and 3) use medications that have a combined cost more than a threshold value for the year, which was $3507 for the year of 2016. Source: https://www.cms.gov/ Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovCon tra/Downloads/Memo-Contract-Year-2016-Medication-TherapyManagement-MTM-Program-Submission-v-040715.pdf. In addition, since 2016 CMR completion rate has become a quality metric in the Star rating used to rank and reward

2

high-performing plans.11 Meanwhile, accumulated empirical evidence supports pharmacist-provided medication review services as beneficial to improving the use of medication therapies.12-15 Despite the increased promotion and potential benefits of using CMRs, however, statistics from CMS annual quality metric reports showed that CMR completion rates among MTM-eligible beneficiaries have remained low. Data for the CMS 2017 Star rating measures showed that the median of CMR completion rate was only 37% among all Medicare Part D plans (Medicare Advantage Prescription Plans and Prescription Drug Plans) and even less (22%) for the stand-alone Part D plans (Prescription Drug Plans).16 Even though it is the elderly patients ultimately making the decision, there are insufficient U.S. studies examining factors affecting the use of CMRs based on perceptions of patients or consumers.17,18 Also, existing research regarding the use of CMR or general MTM services has managed to identify only a limited number of facilitators or barriers separately and descriptively, owing to a lack of theory support.17-24 The identified facilitators included gaining medication-related information and reducing medication-related concerns; whereas the barriers were lack of awareness, lack of perceived need, time, privacy concern, and cost.17-24 However, without a theory’s guidance, understanding of the coherence and meaning of different factors can be limited. As policy makers keep pursuing effective approaches to promote CMRs, a more comprehensive theory-guided exploration is critically needed to better understand patient decision making for using CMRs from the consumers’ perspective. Objective We sought to identify main factors associated with older adults’ decision making for using a CMR service, guided by a conceptual framework derived from consumer behavior theories. Methods An anonymous self-administered survey was mailed to a sample of older adults from December 2016 to February 2017. Two sampling frames, one local and one nonlocal, were used with the anticipation that the survey response rate among those who had received a CMR would be acceptable and that the sample would be geographically dispersed enough for more variation. The study sample was randomly selected from each sampling frame: 1) older adults included in an Iowa senior registry (n ¼ 460); and 2) older adult (65 years of age and older) residents of the states of Florida (n ¼ 310), Washington (n ¼ 310), Wisconsin (n ¼ 310), and Pennsylvania (n ¼ 310) covered in a commercial mailing list provided by Redi-Data, a division of Redi-Direct Marketing. Redi-Data’s database covers more than 200 million consumers nationwide with daily updates, data quality verification, and some demographic characteristics available on request. For the second sampling frame, 2 objective criteria were applied to select the representative states in every U.S. Census region. First, the state needed to be in the top 5 for its region based on the percentage of population 65 years of age and older. It was anticipated that a greater density of older adults might translate to a greater presence of CMR

SCIENCE AND PRACTICE Consumer decisions for using CMRs

Need Recognition

Alternative Comparison

Internal Need

Provider Credibility

Previous Experiences

Patient Characteristics

Convenience

CMR Participation

Motives

Perceived Risks of Using CMRs Psychological Risk

External Influences*

Functional Risk

Social Influence

Social Risk

Figure 1. Conceptual framework for survey development. *“Past marketing stimuli” was another construct originally proposed in the framework under the domain of external influences, but it was excluded from survey development based on findings from a qualitative study we conducted prior to the survey. Abbreviation used: CMR, comprehensive medication review.

services in the state. Second, the state must have a relatively high performance of CMRs nationwide (i.e., first or second tier of a 4-tier CMR performance scale as presented by a national MTM administrator). If multiple states satisfied both criteria in a region, expert opinion was used to determine which state was more likely to respond to our survey. For instance, WI was chosen over OH and MI in the region of Midwest. The final sampling frame of FL, WA, WI, and PA were constructed according to the latest data available when the study was designed.25,26 A disproportionate stratified sampling was performed to ensure sufficient samples from each state and to avoid conclusions based on dominant states because of low response rates from others. To be an eligible survey respondent, survey participants had to be 1) at least 65 years old, 2) taking at least 1 prescription medication, and 3) a Medicare Part D beneficiary in 2016. Inclusion criterion 1 was ensured during sampling phase, and criteria 2 and 3 were confirmed through screening items in the survey questionnaire. Survey items were developed with the guidance of a conceptual framework derived from consumer behavior theories, including multistage consumer decision making,27-30 need recognition,31,32 and consumer prepurchase in a service context.30,33 A consumer decision-making model suggested involvement of multiple activities (i.e., need or problem recognition, information search, and alternatives evaluation) before the final purchase was made. Of those, the key activity, “need recognition,” has been developed into a comprehensive model by Bruner and Pomazal.31 The model further was simplified by Punj and Srinivasan32 and thus became

measurable in empirical research. Meanwhile, risk identification has been determined to be important during information search in a service purchase context, owing to perceived uncertainty of service.30,33 The model assumes that older adults are rational consumers in their decision-making process and go through a series of decision-making aspects before they choose to use CMRs. They have to recognize an internal need, gain information externally, particularly identify risks of using the service, compare to an alternative service, and then make their final decision for participating in CMRs or not. As such, a conceptual framework adapted from the above models was used to guide the understanding of older adults’ decision making for participating in CMRs (Figure 1). According to the framework, the survey was designed for measuring a number of factors in the domain of internal need, external influences, perceived risks of using CMRs, and alternatives comparison (Appendix 1). A review of the literature suggested that there were established instruments that have been used to measure past experience,34,35 motives,36-38 and perceived risks.39 Therefore, existing instruments were adapted to the study context with appropriate changes. The remaining measurements were generated initially based on theories and informed by representative quotes from a qualitative study conducted before the survey. That study included semistructured personal interviews conducted among a convenience sample of 13 older adult residents in Iowa, exploring the use of the same conceptual framework in understanding older adults’ uptake of CMRs. All 4 domains in the framework appeared to be necessary for understanding older adults’ 3

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

decision making for using CMRs. However, it was found that general marketing stimuli to the older population were quite infrequent. As a result, the construct of “past marketing stimuli” under the domain of external influences was excluded from the survey development. Instead, an item assessing older adults’ preferred promotion strategies for CMRs in the future was included. The dependent variable was a dichotomous variable measured by a single question asking whether the survey participant had a CMR in 2016 or before, with a brief definition of CMRs having been provided before this question was asked. The CMR definition included in the survey was simplified from the official definition of CMRs,7,8,40 with an emphasis on its key elements, including the interactive review process and its comprehensiveness of the reviewed medications. This definition was also included in the prior qualitative study to help check whether the language used to describe CMRs was valid and understandable by older adults. It was further revised for ease of reading through review by a panel of experts (5 experts in survey development and a convenience sample of 5 older adults). When spelling out CMR in the survey, the modified term “complete medicine review” was used instead of the official term “comprehensive medication review,” as a result of suggestions from the same expert panel. All survey items were refined through review by this panel of experts and a pilot study (n ¼ 140). The final version of the survey questionnaire is presented in Appendix 2. The score of the Flesch-Kincaid reading ease test of the questionnaire was 59.2, suggesting it should be understood by people with an education level of junior high school or higher when selfadministered. The final version of the mail survey was a 4-page (front/ back), 51-item questionnaire, which was printed on one 11  17einch sheet and then folded into a booklet. Besides the questionnaire, each survey package also included a personalized cover letter and a preaddressed return envelope. The survey package was mailed to each subject in the study sample. Each subject was contacted twice: a survey package was sent first, followed by a reminder postcard 2 weeks later. In addition, to increase the response rate in the nonlocal sample, strategies suggested in the Dillman method were used in the mailings41: 1) a 1-dollar bill was included as an incentive in their first contact; and 2) a third contact (i.e., the same survey package but no incentives) was made to nonrespondents in about 2 weeks after the reminder postcard. Item internal consistency was assessed with the use of Cronbach alpha.42,43 Generally, a Cronbach alpha of 0.90 or above shows excellent reliability, 0.70 to 0.90 shows high reliability, 0.50 to 0.70 shows moderate reliability, and 0.50 or below shows low reliability.43 Variables with low reliability were unacceptable, and measures with moderate reliability were assessed with the use of expert opinion to decide whether certain items should be removed from further analysis or not. Nonresponse bias was evaluated through comparing several demographic variables between respondents and nonrespondents.44 The compared characteristics included age, gender, race, and self-reported household income, which were collected and provided by Redi-Data in addition to consumers’ mailing addresses. Note that per the senior registry user agreement, it was not feasible to conduct the same comparison among the registry sample because of the lack of a tracking system to identify nonrespondents. Only

4

eligible respondents who had a completed survey response were included for data analysis. Descriptive analysis and multiple logistic regression modeling were conducted with the use of SAS version 9.4. In the regression modeling, a P value of 0.05 was used as the threshold for significance. All study activities were approved by the University of Iowa Institutional Review Board. Results A visualized process of survey responses collection is displayed in Figure 2. Nonresponse bias was of concern in the commercial mailing list sample because its usable response rate was 18%. However, there were no significant differences identified between respondents and nonrespondents regarding age, gender, race, and self-reported household income (all P > 0.05). One exception was the state of residence (P ¼ 0.0018): residents of WA and WI seemed more likely to respond to our survey than those in the FL and PA. Compared with the nonlocal sample, the usable response rate of the local senior registry sample was higher (53% vs. 18%), but data were not available to assess their nonresponse bias. The overall completed survey response rate was 24% (n ¼ 381). There were slightly more female respondents (55.91%), with an overall mean age of 76.82 ± 6.59 years. The mean and median of the total number of currently taken prescription medications (both regularly and as needed) were 4.52 ± 2.90 and 4, respectively. The majority of the respondents thought that their health status was very good (40.68%) or good (37.80%) compared with others of their age. There were 105 respondents (27.56%) reporting that they had a CMR in 2016 or before. Table 1 presents the respondents’ characteristics of the survey sample. After internal consistency assessment, 2 items, measuring the variables “physician influence” and “pharmacist influence,” were removed from the descriptive and modeling analyses. Therefore, most Cronbach alphas ranged from 0.80 to 0.90, indicating good reliability for the measures. Note that all items, from 2 variables with moderate reliability (“perceived seriousness of having MRPs” and “psychologic risk”) were kept for further analysis based on expert opinion: 1) Each item theoretically measured 1 reasonably unique aspect of the conceptual variable; and 2) sacrificing this unique aspect of measuring the variable (i.e., removing it) did not gain much in the increase of Cronbach alpha. Table 2 presents the descriptive analysis results of the variables measured by multiple items. In addition, of all 381 respondents, 52 (13.65%) previously received a CMR recommendation from a physician, and 60 (15.75%) received one from a pharmacist. When asked about perceived effectiveness of promotion strategies for CMRs in the future, respondents favored recommendation from a physician the most (89.50%), followed by a pharmacist’s recommendation (67.19%), flyers available in the pharmacy (30.71%), information delivered through print media (27.03%), and information delivered through broadcast media (20.47%). The logistic regression modeling results are presented in Table 3. It was determined that a pharmacist or physician recommendation, pharmacist’s communication in previous pharmacy encounters, perceived susceptibility to MRPs, and positive outcome expectancy were positively associated with Medicare Part D beneficiaries’ decision making for using CMRs

SCIENCE AND PRACTICE Consumer decisions for using CMRs

Survey to Be Mailed (n = 1700)

Sampling Frame 1: Senior Registry (n = 460)

Sampling Frame 2: Commercial Mailing List (n = 1240) Undeliverable (n = 2)

Survey Mailed (n = 460)

Survey Mailed (n = 1238)

Survey Returned (n = 292)

Survey Returned (n = 386)

Deceased (n = 4)

Deceased (n = 7)

Ineligible* (n = 51)

Ineligible† (n = 48) No Participation (n = 119)

No Participation (n = 23) Usable Survey (n = 212)

Usable Survey (n = 214)

Incomplete (n = 45) Completed Usable Survey (n = 381) Figure 2. Survey data collection visualization. *Ineligible in the Senior Registry sample included respondents without a Part D plan (neither prescription drug plan nor Medicare Advantage plan that offers prescription drug coverage; n ¼ 34), respondents without any medication coverage (n ¼ 3), respondents currently taking no prescription medication (n ¼ 13), and respondents who could not take the survey because they were not currently living at home (n ¼ 1). yIneligible in the Commercial Mailing List sample included respondents without a Part D plan (neither prescription drug plan nor Medicare Advantage plan that offers prescription drug coverage; n ¼ 29), respondents without any medication coverage (n ¼ 4), respondents currently taking no prescription medication (n ¼ 14), and respondents who could not take the survey owing to mental illness (n ¼ 1).

at a significance level of 0.05 when holding other variables fixed. Meanwhile, perceived functional risk, access to general counseling in previous experiences, and influence of family/ friends were negatively associated with Medicare Part D beneficiaries’ decision making for using CMRs at a significance level of 0.05.

was a risk factor for having MRPs.4,48-51 Because U.S. older adults had a higher prescription use compared with our sample, they may perceive themselves to be more susceptible to MRPs and may have higher benefit expectancy from using CMRs. This implies that the effects of internal need identified in our study might have a larger degree of impact among U.S. older adults.

Discussion CMR participation rate Survey respondent characteristics The characteristics of our analytical sample were roughly aligned with the demographic characteristics of the U.S. population 65 years of age and older regarding gender and age in the 2015 American Community Survey conducted by the U.S. Census Bureau (n ¼ 47,732,480).45 The health-related characteristics of our analytical sample were slightly different from those of the national population of older adults. Although the medians of self-reported health status were the same (median good), our respondents used fewer prescriptions than U.S. older adults in general (median 4 vs. 6).46,47 Literature suggested that the number of prescriptions

Overall, more than one-fourth of our study respondents (27.56%) reported that they had a CMR in 2016 or before, based on the survey terminology and definition used. Note that the number of reported CMR recipients from the survey (n ¼ 105) was likely to be a total number of CMR recipients of multiple years up to 2016. If that was the case, CMR uptake among our survey respondents was still low after CMR being available for several years. Not every Medicare Part D beneficiary is eligible to receive a free annual CMR. They have to have a certain number of chronic health conditions, be taking a certain number of medications, and reach a cost threshold. All these criteria vary by different plans as well. Because we did not 5

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

Table 1 Survey respondents’ characteristics (n ¼ 381) Characteristics Location FL resident PA resident WA resident WI resident IA Senior Registry CMR Recipient Nonrecipient Type of prescription drug plan PDP MA-PD Age Gender Female Male No. of prescription medications No. of medical conditions 0 1e2 3e5 >5 No. of physicians 0 1 2e3 >3 Health status (self-reported) Excellent Very good Good Fair Poor

Statistics

Value

n, %

38 43 60 56 184

n, %

105 (27.56%) 276 (72.44%)

n, %

292 (76.64%) 89 (23.36%) 76.82 ± 6.59 76 (67e96)

Mean ± SD Median (range) n, % Mean ± SD Median (range)

(9.97%) (11.29%) (15.75%) (14.70%) (48.29%)

213 (55.91%) 168 (44.09%) 4.52 ± 2.90 4 (1e20)

n, %

9 184 166 22

(2.36%) (48.29%) (43.57%) (5.77%)

n, %

2 125 192 62

(0.52%) (32.81%) (50.39%) (16.27%)

n, %

33 155 144 41 8

(8.66%) (40.68%) (37.80%) (10.76%) (2.10%)

Abbreviations used: CMR, comprehensive medication review; PDP, prescription drug plan; MA-PD, Medicare Advantage plan that offers prescription drug coverage.

collect detailed plan information from our study sample, our self-reported CMR recipients may or may not have to pay for their CMRs. Yet, the cost of service might affect their perceived value of CMRs. Thus, the self-reported CMR rate from the survey likely was affected by several factors as described subsequently.

Positive factors associated with CMR participation This study modeled the uptake of CMRs with the use of factors in the domain of internal need, external influences, perceived risks of using CMRs, and alternatives comparison. After controlling for other variables, the most notable factor that positively affected older adults’ decision making for using CMRs was having a pharmacist recommendation, in the domain of external influences. This finding can be supported by the literature. First, a pharmacist recommendation can make an older adult aware of CMRs or understand more about CMRs. Previous studies identified that a lack of awareness or understanding is a key barrier for older adults to use MTM services.17,20,22 Having the pharmacist introduce CMRs to older adults overcomes such a barrier. Second, a pharmacist

6

recommendation can serve as an outreach approach toward older adults. One recent study found that having outreach personnel offer/recommend a CMR yielded the highest CMR completion rate compared with other approaches that lacked such a key component.52 It was likely that the outreach personnel offer/recommendation increased patients’ familiarity and potential expectation of CMRs. When such a recommendation came from a professional, particularly an expert on medications, the effect could be larger. Such an impact was also supported by the results that 67% respondents perceived “pharmacist recommendation” as the best future promotion strategy to inform older adults about CMRs. A physician recommendation to have a CMR also positively affected older adults’ uptake of CMRs. Previous studies in Australia found a similar influence from physicians on patients’ willingness to use Australia’s Home Medicines Review (HMR).37,53,54 Note that the effect of a physician’s influence on the uptake of CMR versus HMR might be different, because HMR is required to be initiated by a physician whereas CMR is not.55 Yet, some findings from U.S. studies also support the importance of having a physician’s involvement in MTM.18,19,56 A physician was often considered to be a top “authority” among the older population. A physician recommendation is possibly seen as an endorsement, and thereby older adults might be more likely to consider that having a CMR is beneficial. Another notable positively associated factor was provider communication in previous pharmacy encounters, in the domain of internal need. To be more specific, it referred to previous communication experience with a pharmacist. The better the communication felt to the older adults, the more likely it was that they would use a CMR. It is possible that previous good communication experiences help build up patients’ trust of pharmacists in providing medication management services. Previous studies identified that when people did not trust pharmacists’ expanded role in pharmaceutical care, they did not appreciate or were unwilling to use those services.19,57 Conversely, when older adults appreciate pharmacists’ expanded role, which could be established through skillful communication by pharmacists in usual interactions, they would be more likely to use CMRs later. The next 2 significant positive factors were perceived susceptibility of MRPs and positive outcome expectancy. Both factors represented the domain of internal need. Because older adults could perceive themselves as being more susceptible to MRPs, they might be more likely to decide to have a CMR. Such susceptibility could result from taking a number of medications, taking high-risk medications, or having multiple prescribers. It is also possible that they have had MRPs previously and could be concerned about having them again in the future. Regardless, the perceived susceptibility could reflect patients’ concerns for safety while using medications. This mirrored findings from previous studies finding that having concerns about medicines or experiencing adverse effects influenced people’s participation in a medication review service.37,54 The interpretation of positive outcome expectancy’s effect on CMR uptake appeared to involve older adults’ concern of medication safety as well. Often, using medications could lead to actual experience or worries about medication interactions or adverse effects among older adults.

SCIENCE AND PRACTICE Consumer decisions for using CMRs

Table 2 Descriptive analysis on multi-item factors associated with older adults' use of CMRs (n ¼ 381) Variable Previous experience Access to care Provider communication Motives Perceived seriousness of medication-related problems Perceived susceptibility of medication-related problems Positive outcome expectancy Self-efficacy in medication understanding and use Social influence Physician influence Pharmacist influence Family/friends influence Perceived risks of using CMRs Psychological risk Functional risk Social risk

No. of survey items

Cronbach alpha

Mean ± SD

Median

3 5

0.84 0.89

3.55 ± 0.70 3.46 ± 0.66

4.0 3.6

3 3 4 3

0.51 0.90 0.89 0.73

2.84 2.07 2.07 3.32

± ± ± ±

0.58 0.82 0.82 0.53

3.0 2.0 3.0 3.3

2 2 1

0.86 0.86 n/a

3.51 ± 0.56 3.02 ± 0.73 2.75 ± 0.80

3.5 3.0 3.0

2 2 3

0.67 0.89 0.80

1.79 ± 0.65 2.45 ± 0.87 1.90 ± 0.65

2.0 2.5 2.0

Measurement scale for all listed variables: 1 ¼ strongly disagree; 2 ¼ disagree; 3 ¼ agree; 4 ¼ strongly agree. Abbreviation used: CMR, comprehensive medication review.

The need for protecting themselves from such potential harms would potentially motivate older adults to seek a professional solution, such as a medication review. If older adults perceive CMRs as a service that can meet their need to relieve their

medication safety concerns, they might have higher expectancy of CMR benefits and may be more likely to use CMRs. Such an explanation paralleled an Australian study in which positive outcome expectancies were found to influence

Table 3 Multiple logistic regression analysis on factors associated with older adults' use of CMRs (n ¼ 381) Variable Intercept Access to care Provider communication Perceived seriousness of medication-related problems Perceived susceptibility of medication-related problems Positive outcome expectancy Self-efficacy in medication understanding and use Physician influence Pharmacist influence Family/friends influence Physician recommendation Pharmacist recommendation Other recommendation Psychological risk Functional risk Social risk Credibility differencea Convenience differenceb Age Genderc Number of prescriptions Number of medical conditions Number of physicians Self-reported health status Type of prescription coveraged State of Floridae State of Pennsylvaniae State of Washingtone State of Wisconsine

Parameter estimate

Standard error

Chi-square

P value

12.74 2.92 6.49 0.33 3.90 2.26 0.90 1.39 1.07 2.38 3.29 6.61 2.88 0.60 4.69 1.84 0.24 0.36 0.04 0.05 0.16 0.21 0.50 0.19 0.97 0.96 0.23 0.98 0.11

7.55 1.19 1.90 0.89 0.97 0.93 0.78 0.91 0.83 0.88 1.56 1.64 1.50 0.90 1.16 0.92 0.23 0.23 0.07 0.47 0.18 1.12 0.73 0.57 0.58 1.01 1.07 0.81 0.91

2.85 5.98 11.63 0.14 16.21 5.98 1.35 2.35 1.65 7.40 4.44 16.15 3.68 0.45 16.32 3.99 1.11 2.46 0.38 0.01 0.79 0.03 0.47 0.11 2.85 0.92 0.05 1.43 0.02

0.0916 0.0145* 0.0007* 0.7097 <0.0001* 0.0147* 0.2460 0.1249 0.1984 0.0065* 0.0350* <0.0001* 0.0551 0.5018 <0.0001* 0.0498* 0.2928 0.1165 0.5360 0.9142 0.3750 0.8543 0.4915 0.7356 0.0916 0.3383 0.8299 0.2312 0.9019

a Credibility difference was calculated as the rating for physicianrating for pharmacist. Each rating scale ranges from 1 to 10, where higher number indicates higher level of credibility. b Convenience difference was calculated as the rating for contact physicianrating for contact pharmacist. Each rating scale ranges from 1 to 10, where higher number indicates being more convenient. c The reference group is male. d The reference group is MA-PD (Medicare Advantage plans that include prescription drug benefits). e The reference group is the geographic area covered by the Iowa Senior Registry. * Statistically significant at a significance level of 0.05.

7

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

patients’ willingness to use Australia HMR.55 Another possible positive outcome expectation of using CMR might come from the financial aspect: older adults might expect to save money through conducting a CMR. This perspective was supported by a previous study exploring patients’ perceived value of MTM.24 Negative factors associated with CMR participation The remaining 3 main factors identified in our model were all negatively associated with older adults’ uptake of CMRs. The first factor was functional risk, which represented the domain of perceived risks of using a service. This factor supported previous studies that identified a lack of need and time as barriers for older adults to use pharmaceutical care services.18,21-23 In contrast to positive CMR benefits expectancy increasing the probability of using it, if Medicare Part D beneficiaries did not perceive any value from using the service, especially because they have to spend time (i.e., cost) to do it, they might not think it is worthwhile to use CMRs. Possibly, the less value that they think CMRs can provide to them, the higher the functional risk that they perceive, and they would be less likely to use CMRs. The second factor was access to general counseling in previous experiences, which represented the domain of internal need. It is possible that when older adults have easy access to general counseling during their pharmacy visit, they might receive enough medication information such that there would be no additional need to use a CMR that would normally require a separate appointment. Thus, the easier the access to general counseling, the less likely older adults would be to decide to use a CMR. The last factor is influence from family or friends. Our study suggested that older adults are less likely to use a CMR if they are more likely to be influenced by family or friends in their decision making. There were 2 possible explanations for this finding. First, people who are more likely to be influenced by family or friends might not necessarily get a recommendation from them to make use of CMRs. In our study sample, only 11 people (2.89%) previously received a recommendation from family or friends. Second, people who are more likely to be influenced by family and friends could possibly have received a negative comment from them. Therefore, they might not want to use CMRs based on the negative statements. Policy implications Our study findings suggest a couple of policy implications. To engage older adults in using CMRs, it seems to be necessary to seek collaboration from pharmacists and physicians. With health professionals actively reaching out to them, older adults should be more open and willing to know details about CMRs. To achieve a better collaboration result, community pharmacists, as being more accessible to older population, could be used to help with a broader range of outreach to older adults. Also, community-based patients often are familiar with their community pharmacists, as a result of previous pharmacy encounters. When a patient has already established a relationship with the pharmacist, especially if his or her previous communication went well, the patient likely would take the pharmacist’s recommendation seriously. Meanwhile, more information related to CMR availability and related benefits 8

should be delivered to physicians along with a request to make CMR referrals. A thorough understanding of CMR could help physicians to better support the promotion of CMRs; knowing that CMR is available in a pharmacy, physicians could direct their patients to use it. Consequently, both awareness and understanding of CMRs could be improved among the older adult population. Also, during the promotion activities, it is important to address the key components and benefits of CMRs, such as providing personalized medication knowledge, reviewing appropriateness of medication comprehensively, updating a medication list, or even controlling medication costs. Meanwhile, it is important to make sure that the benefit descriptions are used in ways that older adults can easily understand. A misunderstanding or poor understanding could possibly increase the perceived functional risk from using CMRs, which was identified as a negative factor associated with CMR uptake among the elderly group. Describing benefits of CMRs from a patient perspective would help to better link the benefits with their potential demand.

Limitations As with any study, there were some limitations to this one. The representativeness of our study sample may be limited owing to a relatively low response rate, particularly in the commercial mailing list. Using more contacts, including a prenotification and a third full survey package, in a future study might yield more responses. Meanwhile, even though the local senior registry had a response rate over 50%, concerns of the nonresponse bias existed and were hard to assess owing to the lack of information. Respondents in the local registry sample might be healthier, more educated, and more proactive in decision making for using their health care product or services. As such, they might be more confident in their understanding or use of medications and less likely to use other additional medication counseling services (e.g., CMRs) compared with the general U.S. older population. A second limitation came from the fact that items or measures generated for the survey were not cross-validated with another independent study sample. Original items were removed or kept based on a reliability test and expert opinion. Such a process can be data driven or too subjective. Measures and the model in this study need future research to cross-validate and refine survey items. Meanwhile, survey measures with moderate reliability that were kept in our study need future study to be revalidated, preferably with the use of larger study samples to increase the data variance or adding more items to increase the internal consistency. Third, exploratory factor analysis could be run for each measure in addition to the reliability analysis, but the exploration of additional latent constructs is beyond our study focus. Our theory-based constructs and variables included in the questionnaire were found to be important and adequate in the personal interview study prior to the survey. Our main interest before data modeling was to determine the measures’ reliability, which we found to be acceptable. However, future research could use factor analysis to test the structural validity and refine multiitem measures if needed. Furthermore, the generalizability of the study findings is limited. There was limited demographic, social, and economic information available to make a

SCIENCE AND PRACTICE Consumer decisions for using CMRs

comprehensive and valid comparison between our study sample and the general U.S. older population. It appeared that our study respondents were slightly healthier than the general U.S. older population. Therefore, our study findings may not apply to areas where population characteristics differ considerably from those of our study respondents. Finally, all information was self-reported in the survey, including the measure for receipt of a CMR. We did include a brief definition of CMRs in the questionnaire to help participants accurately verify their receipt of CMRs. However, the terminology and definition of CMR were modified to ease respondent understanding and increase the readability for older adults. Future studies might work closely with Part D plans to obtain CMR completion records from MTM claims data to identify CMR recipients and nonrecipients. Conclusion CMR uptake among older adults was still low after several years of being offered to eligible Part D beneficiaries. Main factors positively associated with older adults’ decision making for using CMRs include pharmacist recommendation, physician recommendation, provider communication in previous pharmacy encounters, perceived susceptibility of having medication-related problems, and positive outcome expectancy; whereas main factors negatively associated with beneficiaries’ decisions about CMR use include perceived functional risk from using CMRs, access to general medication counseling in previous experiences, and being likely to be influenced by family and friends while making a decision. Policy makers should not entirely focus on promoting CMRs through Part D plan sponsors but seek collaborations from health professionals, particularly community pharmacists and physicians. Meanwhile, addressing key components and benefits of CMRs in an understandable way to older adults could help to strengthen a link between benefit expectation and demand for CMRs. Acknowledgments The authors thank Dhananjay Nayakankuppam (Department of Marketing, Henry B. Tippie College of Business, University of Iowa) for his guidance and support in reviewing theories of consumer behavior. They also thank Matthew J. Witry and Julie M. Urmie (Department of Pharmacy Practice and Science, College of Pharmacy, University of Iowa) and Elizabeth A. Chrischilles (Department of Epidemiology, College of Public Health, University of Iowa) for their comments, which greatly improved this research project. In addition, the authors thank all of the study subjects for participating in this research project. References 1. Budnitz DS, Shehab N, Kegler SR, Richards CL. Medication use leading to emergency department visits for adverse drug events in older adults. Ann Intern Med. 2007;147(11):755e765. 2. Centers for Disease Control and Prevention. Adults and older adult adverse drug events. Available at: http://www.cdc.gov/MedicationSafety/ Adult_AdverseDrugEvents.html. Accessed April 22, 2017. 3. Beijer HJ, de Blaey CJ. Hospitalisations caused by adverse drug reactions (ADR): a meta-analysis of observational studies. Pharm World Sci. 2002;24(2):46e54.

4. Field TS, Gurwitz JH, Harrold LR, et al. Risk factors for adverse drug events among older adults in the ambulatory setting. J Am Geriatr Soc. 2004;52(8):1349e1354. 5. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138(3):161e167. 6. United States Government. Medicare prescription drug, improvement, and modernization act of 2003. Available at: http://www.gpo.gov/fdsys/ pkg/PLAW-108publ173/pdf/PLAW-108publ173.pdf. Accessed April 22, 2017. 7. American Pharmacists Association, National Association of Chain Drug Stores Foundation. Medication therapy management in pharmacy practice: core elements of an MTM service model (version 2.0). J Am Pharm Assoc (2003). 2003;48(3):341e353. 8. Centers for Medicare and Medicaid Services. CY 2019 Medication Therapy Management Program Guidance and Submission Instructions. Available at: https://www.cms.gov/Medicare/Prescription-Drug-Coverage/Prescription DrugCovContra/Downloads/Memo-Contract-Year-2019-Medication-Therapy-Management-MTM-Program-Submission-v-040618.pdf. Accessed October 31, 2018. 9. Centers for Medicare and Medicaid Services. Issuance of the 2010 call letter. Available at: http://www.cms.gov/Medicare/Prescription-DrugCoverage/PrescriptionDrugCovContra/downloads/2010callletter.pdf. Accessed April 22, 2017. 10. Centers for Medicare and Medicaid Services. Announcement of calendar year (CY) 2013 Medicare Advantage capitation rates and Medicare Advantage and Part D payment policies and final call letter. Available at: http://www.cms. gov/Medicare/Health-Plans/HealthPlansGenInfo/Downloads/2013-Call-Let ter.pdf. Accessed April 22, 2017. 11. Centers for Medicare and Medicaid Services. Announcement of calendar year (CY) 2016 Medicare Advantage capitation rates and Medicare Advantage and Part D payment policies and final call letter. Available at: http://www.cms.gov/Medicare/Health-Plans/MedicareAdvtgSpecRateStats /Downloads/Announcement2016.pdf. Accessed April 22, 2017. 12. Barnett MJ, Frank J, Wehring H, et al. Analysis of pharmacist-provided medication therapy management (MTM) services in community pharmacies over 7 years. J Manag Care Pharm. 2009;15(1):18e31. 13. Graabaek T, Kjeldsen LJ. Medication reviews by clinical pharmacists at hospitals lead to improved patient outcomes: a systematic review. Basic Clin Pharmacol Toxicol. 2013;112(6):359e373. 14. Isetts BJ, Schondelmeyer SW, Artz MB, et al. Clinical and economic outcomes of medication therapy management services: the Minnesota experience. J Am Pharm Assoc (2003). 2008;48(2):203e211. 15. Ramalho de Oliveira D, Brummel AR, Miller DB. Medication therapy management: 10 years of experience in a large integrated health care system. J Manag Care Pharm. 2010;16(3):185e195. 16. Centers for Medicare and Medicaid Services. Part C and D performance data. Available at: http://www.cms.gov/Medicare/Prescription-DrugCoverage/PrescriptionDrugCovGenIn/PerformanceData.html. Accessed October 28, 2017. 17. Kuhn CH, Casper KA, Green TR. Assessing Ohio grocery store patrons’ perceptions of a comprehensive medication review. J Am Pharm Assoc (2003). 2009;49(6):787e791. 18. Doucette WR, Zhang Y, Chrischilles EA, et al. Factors affecting Medicare Part D beneficiaries’ decision to receive comprehensive medication reviews. J Am Pharm Assoc (2003). 2013;53(5):482e487. 19. Garcia GM, Snyder ME, McGrath SH, Smith RB, McGivney MS. Generating demand for pharmacist-provided medication therapy management: identifying patient-preferred marketing strategies. J Am Pharm Assoc (2003). 2009;49(5):611e616. 20. Truong HA, Layson-Wolf C, de Bittner MR, Owen JA, Haupt S. Perceptions of patients on Medicare Part D medication therapy management services. J Am Pharm Assoc (2003). 2009;49(3):392e398. 21. Lauffenburger JC, Vu MB, Burkhart JI, Weinberger M, Roth MT. Design of a medication therapy management program for Medicare beneficiaries: qualitative findings from patients and physicians. Am J Geriatr Pharmacother. 2012;10(2):129e138. 22. Law AV, Okamoto MP, Brock K. Perceptions of Medicare Part D enrollees about pharmacists and their role as providers of medication therapy management. J Am Pharm Assoc (2003). 2008;48(5):648e653. 23. Huet AL, Frail CK, Lake LM, Snyder ME. Impact of passive and active promotional strategies on patient acceptance of medication therapy management services. J Am Pharm Assoc (2003). 2015;55(2): 178e181. 24. Schultz H, Westberg SM, Ramalho de Oliveira D, Brummel AR. Patientperceived value of medication therapy management (MTM) services: a series of focus groups. Inov Pharm. 2012;3(4). Article 96. 25. OutcomesMTM. 2016 MTM trends report. Available at: http://www. outcomesmtm.com/documents/2016MTMTrendsReport.pdf. Accessed February 21, 2016.

9

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

26. United States Census Bureau. Persons 65 years and over, percent. Available at: http://www.census.gov/quickfacts/table/AGE775214/12,42 ,00,55,53. Accessed February 21, 2016. 27. Assael H. In: Assael H, ed. Consumer Behavior and Marketing Action. 5th ed. Cincinnati, OH: South-Western College Publishing; 1995: 71e108. 28. Dewey J. How We Think. Boston, MA: D.C. Heath and Co.; 1910. 29. Engel JF, Kollat RD, Blackwell RD. Consumer Behavior. New York, NY: Holt Rinehart and Winston; 1968. 30. Fisk RP. Toward a consumption/evaluation process model for services. Mark Serv. 1981:191e195. 31. Bruner GC, Pomazal RJ. Problem recognition: the crucial first stage of the consumer decision process. J Consum Mark. 1988;5(1):53e63. 32. Punj G, Srinivasan N. Influence of problem recognition on search and other decision process variables: a framework for analysis. Adv Consum Res. 1992;19(1):491e497. 33. Tsiotsou RH, Wirtz J. In: Wells VK, Forall G, eds. Handbook of Developments in Consumer Behaviour. Cheltenham, UK: Edward Elgar; 2012: 147e201. 34. Dyer N, Sorra JS, Smith SA, Cleary PD, Hays RD. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) clinician and group adult visit survey. Med Care. 2012;50(Suppl):S28eS34. 35. Agency for Healthcare Research and Quality. CAHPS clinician and group 12-month survey 2.0. Available at: https://www.ahrq.gov/cahps/surveysguidance/cg/12-month/index.html. Accessed March 1, 2016. 36. Capik C, Gozum S. Development and validation of health beliefs model scale for prostate cancer screenings (HBM-PCS): evidence from exploratory and confirmatory factor analyses. Eur J Oncol Nurs. 2011;15(5): 478e485. 37. Carter SR, Chen TF, White L. Home medicines reviews: a quantitative study of the views of recipients and eligible non-recipients. Int J Pharm Pract. 2012;20(4):209e217. 38. Cameron KA, Ross EL, Clayman ML, et al. Measuring patients’ self-efficacy in understanding and using prescription medication. Patient Educ Couns. 2010;80(3):372e376. 39. Dholakia UM. A motivational process model of product involvement and consumer risk perception. Eur J Mark. 2001;35(11/12):1340e1362. 40. American Pharmacists Association, National Association of Chain Drug Stores Foundation. Medication therapy management in community pharmacy practice: core elements of an MTM service (version 1.0). J Am Pharm Assoc (2003). 2005;45(5):573e579. 41. Dillman DA, Smyth JD, Christian LM. Internet, Phone, Mail, and MixedMode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley and Sons; 2014. 42. Field A. Discovering Statistics Using SPSS. Thousand Oaks, CA: Sage Publications; 2009. 43. Hinton PR, Brownlow C, McMurray I, Cozens B. SPSS Explained. New York, NY: Routledge; 2004. 44. Lindner JR, Murphy TH, Briers GE. Handling nonresponse in social science research. J Agric Educ. 2001;42(4):43e53.

10

45. United States Census Bureau. Population 65 years and over in the United States: 2015 American community survey 1-year estimates. Available at: https://factfinder.census.gov/faces/tableservices/jsf/pages/productview. xhtml?pid¼ACS_15_1YR_S0103&prodType¼table. Accessed March 11, 2017. 46. Agency for Healthcare Research and Quality. 2014 medical expenditure panel survey. Available at: https://meps.ahrq.gov/mepsweb/data_stats/ MEPSnetHC/datasource. Accessed March 11, 2017. 47. Centers for Disease Control and Prevention. National Health and Nutrition Examination Survey 2013e2014 questionnaire data. Available at: https://wwwn.cdc.gov/nchs/nhanes/search/datapage.aspx?Component¼ Questionnaire&CycleBeginYear¼2013. Accessed March 11, 2017. 48. Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003;289(9):1107e1116. re P, Group I. Risk factors for adverse 49. Trivalle C, Burlaud A, Ducimetie drug events in hospitalized elderly patients: a geriatric score. Eur Geriatr Med. 2011;2(5):284e289. 50. Steinman MA, Landefeld CS, Rosenthal GE, et al. Polypharmacy and prescribing quality in older people. J Am Geriatr Soc. 2006;54(10): 1516e1523. 51. Moen J, Bohm A, Tillenius T, et al. “I don’t know how many of these [medicines] are necessary”da focus group study among elderly users of multiple medicines. Patient Educ Couns. 2009;74(2):135e141. 52. Miller DE, Roane TE, Salo JA, Hardin HC. Evaluation of comprehensive medication review completion rates using 3 patient outreach models. J Manag Care Spec Pharm. 2016;22(7):796e800. 53. Baldock M, Kaufman J, Lydeamore A, Roodenburg S, Tomlinson H. Home medication reviews: what do people in the Australian Riverland think of them? Rural Remote Health. 2006;6:546. 54. Carter SR, Moles RJ, White L, Chen TF. Consumers’ willingness to use a medication management service: the effect of medication-related worry and the social influence of the general practitioner. Res Social Adm Pharm. 2013;9(4):431e445. 55. Carter SR, Moles R, White L, Chen TF. Patients’ willingness to use a pharmacist-provided medication management service: the influence of outcome expectancies and communication efficacy. Res Social Adm Pharm. 2012;8(6):487e498. 56. Witry MJ, Chang EH, Mormann MM, Doucette WR, Newland BA. Older adult perceptions of a self-reported medication risk questionaire: a focus group study. Inov Pharm. 2011;2(3):50. 57. Smith M, Cannon-Breland ML, Spiggle S. Consumer, physician, and payer perspectives on primary care medication management services with a shared resource pharmacists network. Res Social Adm Pharm. 2014;10(3): 539e553. Yiran Zhang, PhD, Post-doctoral Fellow in Department of Pharmacotherapy and Outcomes Science, School of Pharmacy, Virginia Commonwealth University, Richmond, VA William R. Doucette, RPh, PhD, Professor in Division of Health Service Research, College of Pharmacy, University of Iowa, Iowa City, IA

SCIENCE AND PRACTICE Consumer decisions for using CMRs

Appendix Appendix 1 Mapping guide between conceptual framework and survey questionnaire items Item Label Elig_1 Elig_2 A1 A2 A3 A4 A5 A6 A7 A8 A9 B1 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 D1 D2_1 D2_2 D2_3 D2_4 D2_5 D3 E1 E2 E3 E4 E5 E6 E7 F1 F2 G1 G2 G3 G4 G5

Variable

Construct

Domain

Type of prescription coverage (screening variable) No. of current prescription medications (screening variable) Access to care

e e Previous experiences

e e Internal need

e Motives

e Internal need

Social influence

External influences

e Social influence

e External influences

e e Psychological risk

e e Perceived risks of using CMRs

Provider communication

Overall satisfaction Had a CMR or not (dependent variable) Perceived seriousness of medication-related problems

Perceived susceptibility of medication-related problems

Positive outcome expectation

Self-efficacy in medication understanding and use

Physician influence

Pharmacist influence

Family/friend influence Awareness of CMRs Physician recommendation Pharmacist recommendation Family/friends recommendation Other recommendation No recommendation Perceived effective promotion strategies Psychological risk Functional risk

Functional risk

Social risk

Social risk

Provider credibility Convenience Age Gender No. of current medical conditions No. of current physicians Self-reported health status

Provider credibility Convenience Patient characteristics

Alternatives comparison Internal need

10.e1

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

Appendix 2. Survey questionnaire.

10.e2

SCIENCE AND PRACTICE Consumer decisions for using CMRs

Appendix 2. (continued).

10.e3

SCIENCE AND PRACTICE Y. Zhang, W.R. Doucette / Journal of the American Pharmacists Association xxx (2018) 1e10

Appendix 2. (continued).

10.e4

SCIENCE AND PRACTICE Consumer decisions for using CMRs

Appendix 2. (continued).

10.e5