- Email: [email protected]
A randomized controlled trial of the impact of a family planning mHealth service on knowledge and use of contraception
Contraception xx (2016) xxx – xxx
Original research article
A randomized controlled trial of the impact of a family planning mHealth service on knowledge and use of contraception Douglas Johnson a,⁎, Randall Juras b , Pamela Riley c , Minki Chatterji c , Phoebe Sloane c , Soon Kyu Choi d , Ben Johns c a
2407 15th St NW, Apartment 506, Washington, DC, United States b 55 Wheeler St., Cambridge, MA 02138, United States c 4550 Montgomery Avenue, Bethesda, MD 20814, United States d The Williams Institute, University of California Los Angeles School of Law, 337 Charles E Young Drive East, Los Angeles, CA 90095, United States Received 18 January 2016; revised 9 July 2016; accepted 11 July 2016
Abstract Objectives: mHealth, or the use of mobile phones for health, is a promising but largely untested method for increasing family planning knowledge in developing countries. This study estimates the effect of m4RH, an mHealth service in Kenya that provides family planning information via text message, on consumers' knowledge and use of contraception. Study design: We randomly assigned new consumers of the m4RH service to receive either full access or limited access to m4RH. We collected data on outcomes by sending questions directly to consumers via text message. Results: Response rates to the text message surveys ranged from 51.8% to 13.5%. Despite relatively low response rates, response rates were very similar across the full-access and limited-access groups. We find that full access to m4RH increased consumers' scores on a test of contraceptive knowledge by 14% (95% confidence interval: 9.9%–18.2%) compared to a control group with limited access to m4RH. m4RH did not increase consumers' use of contraception, likelihood of discussing family planning with their partners, or likelihood of visiting a clinic to discuss family planning. Conclusion: Text messages may increase family planning knowledge but do not, by themselves, lead to behavior change. Implications: Text messages can be an effective method of increasing family planning knowledge but may be insufficient on their own to cause behavior change. © 2016 Elsevier Inc. All rights reserved. Keywords: mHealth; Family planning; Behavior change communication; Kenya
1. Introduction Lack of information on family planning methods continues to be a major barrier to increased use of contraception in developing countries. According to a recent analysis of Demographic and Health Survey data, the two most commonly cited reasons for nonuse of contraception worldwide, after infrequent sexual activity, were concern ⁎ Corresponding author. E-mail addresses: [email protected] (D. Johnson), [email protected] (R. Juras), [email protected] (P. Riley), [email protected] (M. Chatterji), [email protected] (P. Sloane), [email protected] (S.K. Choi), [email protected] (B. Johns). http://dx.doi.org/10.1016/j.contraception.2016.07.009 0010-7824/© 2016 Elsevier Inc. All rights reserved.
over side effects or health risks and opposition to contraception by the woman, her partner or others close to her [1]. In-depth studies of these causes have shown that both of these factors — concerns over side effects and opposition to contraception — are often driven by misconceptions [2,3]. For example, in urban Kenya, the area that is the focus of the intervention evaluated in this study, a recent survey found that a majority of men and women believe that contraception may cause birth defects and harm to the uterus and 39% believe that contraception can cause infertility and cancer [4]. mHealth, or the use of mobile phones for health, is a promising method for increasing family planning knowledge and promoting positive attitudes toward contraceptive use. Access to mobile phones in the developing world is high and rising [5]; disseminating information over mobile phones is
2
D. Johnson et al. / Contraception xx (2016) xxx–xxx
inexpensive; and, unlike mass media, mobile phones allow for two-way interaction. In this study, we estimate the impact of m4RH, an mHealth family planning service in Kenya developed and implemented by FHI360, a nonprofit human development organization. m4RH is a free text-message-based platform that provides information on the benefits, disadvantages and side effects of nine family planning methods as well as a searchable database of clinics that offer family planning counseling and services. m4RH consumers may also sign up to receive “role model” stories about a person facing a difficult sexual or reproductive health issue and how they resolved the issue. A previous exploratory study of m4RH consumers provided suggestive evidence that the service was increasing use of contraception [6]. We conducted a randomized controlled trial to estimate the impact of m4RH on knowledge of contraception, use of contraception and other behavioral outcomes. During the study period, we randomly assigned all new m4RH consumers to receive either full access to the m4RH service (full-access group) or access only to the m4RH clinic database plus general health information messages (limited-access group). This study contributes to the literature on the effectiveness of text messages in increasing health-related knowledge and promoting behavior change in developing countries. Previous studies have shown text messages to be effective at increasing attendance at medical appointments [7–13] and adhering to ART treatment [14,15]. Few other published studies examine the effect of text messages on reproductive health knowledge and behavior change. Castaño et al. find that daily text messages increase continuation of oral contraceptive pills [16]. On the other hand, Jamison et al. [17] find that a text message service in Uganda that provided automated responses to sexual health questions had no effect on contraceptive knowledge and led to decreased use of condoms during sexual intercourse.
2. Methods 2.1. Recruitment Ethical approval was obtained from the Kenya Medical Research Institute ethics review committee and Abt Associates' institutional review board. As a “pull” service that only provides information upon request, FHI360 had to advertise m4RH to make potential consumers aware that it was available. To generate sufficient numbers of m4RH consumers for the study, FHI360 promoted m4RH through ads in newspapers, television and radio as well as at clinics providing family planning services and at various Ministry of Health events. These marketing activities focused primarily on urban areas. 2.2. Full-access and limited-access groups For the duration of the study period (September 2013–May 2014), we assigned all new consumers who accessed the m4RH
service to either a full-access group or a limited-access group. We assigned new consumers to each group on a rolling basis — that is, if the most recent new consumer was assigned to the full-access group, we assigned the current new consumer to the limited-access group. We consider this assignment rule effectively random for two reasons. First, m4RH had an extremely high number of consumers. Second, due to differences in network speed and coverage throughout Kenya, there was large variation in SMS delivery times. We did not seek consent from m4RH consumers prior to initial randomization as the risk to the limited-access group was low. We excluded all existing m4RH consumers from the study and continued to provide these consumers full access to the m4RH service. We provided members of the full-access group access to all of m4RH's features including information on the benefits, disadvantages and side effects of nine family planning methods, a searchable database of clinics offering family planning services and serial “role model” stories about a person facing a difficult family planning issue. We provided members of the limited-access group access to the clinic locator along with general motivational messages on a variety of health topics but did not provide access to any other m4RH content. We designed the motivational messages, examples of which are included in the appendix, to keep the consumers engaged with the m4RH service but not to directly affect any of the outcome measures focused on in this study. Members of the limited-access group were provided access to all m4RH content after data collection was complete; i.e., a period of 3 months. Unlike many other mHealth services, m4RH is a “pull” rather than a “push” service. Therefore, m4RH consumers were only sent content that they explicitly requested (Fig. 1). 2.3. Data We collected data on outcomes and covariates via text message. Due to the novelty of conducting a survey via text message, we performed several tests to determine the most effective combination of timing and incentives to encourage response and performed extensive field-testing of questions [18]. We divided survey questions into three waves as we anticipated low response rates to questions asked toward the end of the survey if all questions were included in a single survey. We sent the survey waves to each participant on a rolling basis based on the timing of their initial interaction with m4RH. We sent the first wave of survey questions to each participant approximately 24 h after the participant first accessed the m4RH system, the second wave approximately 6 days later, and the third wave approximately 3 months later. Each survey wave included an initial invitation explaining that consumers were free not to participate in the survey and that they could continue using m4RH if they chose not to respond. To prevent consumers below the age of 18 from participating in the study, we asked respondents their age at the start of each survey wave and discontinued the survey if the respondent was under 18.
D. Johnson et al. / Contraception xx (2016) xxx–xxx
3
Fig. 1. Random assignment diagram.
The first survey wave included four questions on the participant's age, use of contraception (two questions) and gender. The first question on use of contraception asked, “Do you or your sexual partner use contraception?” Respondents who replied “yes” were sent the follow-up question, “What type of contraception do you or your partner use?” The second survey wave included a question on the participant's age, a question on the participant's education and several questions designed to test the participant's knowledge of family planning. We carefully chose the knowledge questions in the second survey wave to ensure that they covered a broad range of family planning topics representative of the basic family planning information provided by m4RH. The third survey wave included questions on the participant's age, use of contraception (the same two questions as in the first wave), whether the participant had recently discussed family planning with his or her sexual partner, whether the participant had recently visited a clinic to discuss family planning, the participant's marital status and the participant's religion. Our primary outcome is the number of family planning knowledge questions (out of five) that the study participants answered correctly. Secondary outcomes are whether the participant (or the participant's partner) uses contraception, whether the participant recently discussed family planning with his or her partner and whether the participant recently visited a clinic to discuss family planning. Fig. 2 presents a modified CONSORT diagram [19] showing sources of participant attrition, as well as the sample remaining at each step of the enrollment and data-collection process. In addition to the data from these text message surveys, we had access to a log of all text messages sent or received by m4RH. This log included the full text message content,
the mobile number that the text message was sent to or received from, and the date and time the text message was sent or received. We used these data to verify that study participants were not looking up answers to survey questions by checking whether participants requested m4RH content between the time the survey question was sent and the participant submitted the survey answer. 2.4. Statistical analysis We perform two-sided t tests for equality of means to assess balance between full-access and limited-access participants on all variables except for education. For education, we test for equality by using a Pearson's χ 2 test. To estimate impact, we must confront several analytical challenges. First, overlap in response between survey waves is low — that is, there is a low correlation between answering at least one question on one survey wave and answering at least one question on another survey wave. Thus, if we only included data from study participants who answered at least one question in all survey waves, our sample would be greatly reduced. To deal with this challenge, we split our dataset into two analytic samples: study participants who answered at least one of the knowledge questions on survey wave two and study participants who answered at least one question on survey wave three. When estimating the impact of m4RH on overall knowledge of family planning, we use the first analytic sample. When estimating the impact of m4RH on behavioral outcomes, we use the second analytic sample. Due to these different analytic samples, findings from these two sets of analyses are not strictly comparable as the populations for which the results hold differ.
4
D. Johnson et al. / Contraception xx (2016) xxx–xxx
Fig. 2. Modified consort diagram.
A second analytic challenge is the high proportion of missing data within survey waves due to the fact that many study participants answered some but not all questions in a survey wave. This second challenge is particularly problematic for estimating the impact of m4RH on the overall knowledge score. Again, if we restricted analysis to only those study participants who answered all knowledge questions, our sample would be severely reduced as many study participants answered some but not all knowledge questions. To deal with this second challenge, we use multiple imputation to impute missing covariates and outcome variables in each analytic sample. Multiple imputation may also correct for imbalances in response rates such as those observed in the second set of survey questions if the assumptions behind the imputation model hold [20]. To impute data, we use a multivariate normal
model with a ridge prior and generate 20 imputations. Multivariate normal models have been shown to perform well in imputation even in cases where the true data-generating process is not multivariate normal [21]. We use separate imputation models for each analytical sample and within analytical samples for full-access and limited-access groups. Our imputation model for the analytic sample of study participants who answered at least one question on survey wave two includes all questions from that survey wave as well as age and gender from the first survey wave as these variables had low rates of missingness. Similarly, our imputation model for the analytic sample of study participants who answered at least one question from survey wave three includes all survey questions from that survey wave and age and gender from the first survey wave.
D. Johnson et al. / Contraception xx (2016) xxx–xxx
5
Table 1 Response rates by survey wave and treatment status
Survey wave 1 Answered at least one question Answered all questions Survey wave 2 Answered at least one question Answered at least one knowledge-related question Answered all questions Survey wave 3 Answered at least one question Answered all questions
Full access
Limited access
Difference
p value
51.8% 35.3%
51.4% 34.5%
0.4 ppt 0.8 ppt
.71 .31
39.1% 37.3% 21.5%
41.9% 40.1% 24.0%
−2.8 ppt −2.8 ppt −2.5 ppt
20.9% 13.5%
21.3% 13.8%
−0.4 ppt −0.3 fuppt
b.001 b.001 b.001 .57 .61
ppt: percentage points.
After imputing missing data, we estimate the impact of m4RH on each outcome variable first by performing two-sided t tests for equality of means and second by regressing the outcome variable on a dummy for full access and several covariates. Regression analysis may help correct for any remaining differences between the full-access and limited-access groups to the extent that these differences are linearly related to the covariates in our model [ibid]. We use the same covariates in each regression as we used for the original imputation model. For both the two-sided t tests and the regression analysis, we repeat each analysis for each of the 20 imputed datasets and combine the results using the standard multiple imputation formula. In addition to estimating the impact of m4RH on the overall knowledge score and the three behavioral outcomes, we also estimate the impact of m4RH on the probability of answering each individual knowledge question correctly. This analysis was not prespecified in the initial design report and thus should be viewed as exploratory. For this analysis, we do not use multiple imputation as missing data for one knowledge question do not prevent us from analyzing data
for a different knowledge question. To estimate the impact of m4RH on each individual knowledge question, we perform two-sided t tests for equality of means and regression analysis (using the same covariates as described above for the overall knowledge score). We prespecified all analytic methods employed in the study (with the exception of the exploratory analysis of the impact on individual knowledge questions) in a design report that was registered and made public prior to data collection through the 3ie Registry for Development Impact Evaluations (Study ID RIDIE-STUDY-ID-5294abf913209). Stata version 12 was used for all analyses.
3. Results Response rates to the surveys varied by survey wave (Table 1). For the first wave of text message questions, 51.6% of study participants answered at least one question and 34.9% answered all questions. For the third set of text message questions, only 21.1% of study participants
Table 2 Background characteristics by treatment status
Survey wave 1 Mean age Use contraception Female Survey wave 2 Mean age Highest education completed: primary education or less Highest education completed: secondary education Highest education completed: higher education Survey wave 3 Mean age Married Christian
Full access
Limited access
Difference
p value
24.5 (5.2) 69.6% (46.0%) 67.2% (47.0%)
24.6 (5.3) 71.4% (45.2%) 65.0% (47.7%)
−0.1 −1.8 ppt 2.2 ppt
.77 .82 .03
25.2 (7.4) 9.5% (29.3%) 40.8% (49.2%) 49.7% (50.0%)
24.6 (5.6) 9.2% (28.9%) 42.3% (49.4%) 48.5% (50.0%)
0.6 0.3 ppt −1.5 ppt 1.2 ppt
b.001 .69
24.9 (5.6) 50.4% (50.0%) 96.6% (18.3%)
25.0 (5.6) 48.7% (50.0%) 95.4% (21.0%)
−0.1 1.7 ppt 1.2 ppt
.49 .43 .16
Note: Significance level is determined by a two-sided test for equality of means of each variable (except for education, for which a Pearson's χ 2 test was used). Standard deviations in parenthesis.
6
D. Johnson et al. / Contraception xx (2016) xxx–xxx
Table 3 Impact of m4RH on overall knowledge score (n=5164). Outcome
Full access
Limited access
Difference
p value
Total # correct knowledge questions (out of 5)
2.19 (2.13–2.25)
1.92 (1.87–1.98)
0.27 (0.19–0.35)
b.001
Note: Means reported present missing answers filled in using multiple imputation (95% confidence intervals in parentheses). The fraction of missing information for total knowledge score is 0.26. Results reflect responses to the second survey.
answered at least one question and 16.3% answered all questions. Response rates for the two groups were similar for the first and third surveys. For the second survey, response rates were between 2 and 3 percentage points higher in the limited-access group than the full-access group. In addition, for the second set of survey questions, there is a statistically significant difference in the mean age of full-access and limited-access respondents. We discussed the strategies we use to deal with this imbalance in the section on statistical analysis. Table 2 presents the demographic characteristics of survey respondents. Table 3 displays unadjusted average knowledge scores, after multiple imputation, for the full-access and limited-access groups. We find that having full access to m4RH increased consumers' knowledge of family planning. On average, we find that full-access group members correctly answered 2.19 questions out of 5, while limited-access group members correctly answered 1.92 questions (pb.001). The estimated impact of 0.27 more correct knowledge question thus represents a 14% improvement in knowledge score compared with the limited-access group. When using regression to adjust for covariates, the results show a 0.26 difference (13% improvement in the full-access group compared to the limited-access group) and remains statistically significant (pb.001). In exploratory analyses of the nonimputed data from the five individual knowledge questions that form the composite knowledge score, we find that m4RH caused an increase in the proportion of correct responses for four of the five specific knowledge questions (Table 4). The only knowledge question for which we did not find a significant impact is the
question about the lactational amenorrhea method. Results do not change substantively when missing answers are imputed or when using regression-adjusted results (results not shown). We find only small and statistically insignificant (pN.1) impacts for all three behavioral outcomes: whether the consumer recently discussed family planning with his or her partner, whether the consumer visited a clinic to discuss family planning in the past month, and whether the consumer or the consumer's partner uses contraception (Table 5).
4. Discussion Response rates to the text message surveys were low, and thus p values from this study should be interpreted with caution. In particular, it is unclear whether these results are representative of the entire population of m4RH consumers. Despite relatively low response rates, response rates were very similar across the full-access and limited-access groups. Our sample of m4RH consumers is, on average, well educated (90.7% of respondents have a secondary education or higher), overwhelmingly Christian (96% are Christian compared to 82.5% of all Kenyans [22]) and more likely to use contraception than the general population of Kenya (71.4% of respondents to the first question indicated that they or their partner use contraception compared to 53.2% of married women of reproductive age and 60.9% of sexually active unmarried women of reproductive age in Kenya [23]). About 34% of consumers are male, and the average age was roughly 25 years.
Table 4 Impact of m4RH on individual knowledge questions (n=5164). Question
When is a woman most likely to get pregnant: just after period, halfway between periods, or just before period? Can women avoid pregnancy for 6 months after birth if period has not returned and breast feed baby: yes or no? About how long does coil last before it needs to be replaced: 1 year, 2 years, or more than 2 years? How many days after sex is EC pill effective: 1 day, 5 days, or 10 days? About how long do implants last before they need to be replaced: 6 months, 1 year, or more than 1 year?
n
Percentage answering correctly
Difference (in pct points)
p value
31.5% (29.7%–33.2%)
6.0 (3.4%–8.6%)
b.001
57.2% (54.9%–59.5%)
55.4% (53.2%–57.6%)
1.8 (−0.1%–5.0%)
.26
3612
45.2% (42.8%–47.6%)
38.2% (36.0%–40.3%)
7.0 (3.8%–10.2%)
b.001
3488
29.4% (27.2%–31.6%)
21.9% (20.0%–23.8%)
7.5 (4.6%–10.4%)
b.001
3329
50.3% (47.8%–52.8%)
45.0% (42.6%–47.3%)
5.4 (2.0%–8.8%)
.002
Full access
Limited access
5155
37.4% (35.6%–39.4%)
3751
Note: Means reported reflect unadjusted, nonimputed means; note results in Table 3 reflect imputed answers (95% confidence intervals in parentheses). Due to rounding, reported difference (T-C differences) may differ from differences between reported unadjusted means for the full-access and limited-access groups.
D. Johnson et al. / Contraception xx (2016) xxx–xxx
7
Table 5 Impact of m4RH on behavioral outcomes n
Full access
Limited access
Difference (in pct points) p value
Discussed family planning with partner in past month 2266 73.6% (71.0%–76.1%) 71.2% (69.1%–74.8%) 1.6% (−2.2%–5.4%) Visited clinic to discuss family planning with nurse or doctor 2197 42.1% (39.0%–45.2%) 38.8% (35.7%–41.9%) 3.2% (−1.0%–7.5%) Use contraception at endline 2452 79.7% (77.4%–81.9%) 79.6% (77.4%–81.7%) 0.1% (−3.0%–3.3%)
.40 .13 .94
Note: Means reported present missing answers filled in using multiple imputation (95% confidence intervals in parentheses). The fraction of missing information for ‘Discussed family planning with partner in past month’ is 0.37; for ‘Visited clinic to discuss family planning with nurse or doctor,’ it is 0.32; for ‘Use contraception at endline,’ it is 0.17.
We find that access to m4RH increased consumers' scores on a test of family planning by approximately 14%. This increase in family planning knowledge, while modest, appears to be of a reasonable size given the low dose of the intervention based on evidence from other low-touch interventions designed to increase family planning knowledge. For example, Valente et al. [24] find that a radio drama in The Gambia led to a 31% increase (from 4.2 to 5.5) in the number of modern family planning methods adults in the targeted area could name. Similarly, Kane et al. [25] find that a multimedia campaign in Bamako, Mali, led to a 4% increase (from 90% to 94%) in the proportion of women who could name at least one modern contraceptive method. We find that m4RH did not increase the likelihood of the consumer or the consumer's partner using contraception, the likelihood of the consumer having recently discussed family planning with his or her partner or the likelihood of the consumer having visited a clinic to discuss family planning in the past month. Our results have several implications. First, our results demonstrate that mHealth can be a cost-effective tool for increasing family planning knowledge. As the cost of each text message was only $0.045, the overall cost of delivering the program was relatively inexpensive. When compared to mass media campaigns of similar reach, text messages provide a more targeted, customizable channel to influence individual knowledge at lower cost. Second, our lack of a detectable impact on any of the behavioral outcomes suggests that text messages alone are inadequate to increase use of contraception. It is likely that the m4RH services — information, “role model” stories and clinic locator lists — were simply not sufficient to change a behavior that is governed by complex personal, cultural and systemic factors. These results are consistent with previous studies which have shown that increases in family planning knowledge alone often do not lead to changes in behavior [26]. Third, our findings suggest that programs implementing mHealth interventions should pay careful attention to the demographic profile of their consumers. With limited options for data collection from consumers, mHealth implementers often know very little about consumers of their service. Our findings show that the profile of consumers may be different from what implementers might anticipate. It is unclear whether the skewed demographic profile of m4RH consumers was due mainly to the choice of marketing, which
focused primarily on urban areas, or the fact that basic English literacy and a mobile phone are prerequisites to using the service. (According to the Communications Authority of Kenya, mobile phone penetration was above 80% at the time of this study, but this figure assumes no mobile phone users have more than one SIM card [27].) Further research is necessary to identify factors that can increase mHealth reach to underserved populations with a higher unmet need for family planning. Our study has several limitations. First, limited-access messages may have unintentionally affected outcomes among the limited-access group. We have no data to either support or refute this possibility but find it unlikely based on the general nature of the limited-access messages and our field-testing of these messages. Second, we rely on self-reported information for data on use of contraception and other behavioral outcomes. Third, the generalizability of our results is limited by both the low overall response rate to many of the survey questions and the degree to which the sample of study participants differs from the general Kenyan population. Role of the funding source The funder had no role in the study design, in the data collection or analysis of data, or in the authors' interpretation of the results but reviewed them and provided comments. The authors had final authority over the manuscript submitted for publication. Acknowledgments Douglas Johnson was employed by Abt Associates when this study was designed and carried out. This study was funded by the United States Agency for International Development through the Strengthening Health Outcomes through the Private Sector (SHOPS) project, managed by Abt Associates (Cooperative Agreement GPO-A-00-09-00007-00). We thank FHI360 and Marsden Solomon, Loice Magaria, Alice Olawo, Heather Vadhat and Kelly L'Engle in particular for allowing us to evaluate this program and for guidance and feedback on the design and implementation of the study. We thank Marie Stopes Kenya, PSI, Family Health Options of Kenya, FHI360 AHPIA Plus and the Jhpiego-led Tupange project for assistance in promoting m4RH during the study
8
D. Johnson et al. / Contraception xx (2016) xxx–xxx
period. We thank TexttoChange and Marcus Wagenaar in particular for designing the excellent platform on which m4RH was run and the text message surveys were conducted. Appendix A. Sample limited-access group messages Messages sent to the limited-access group were designed to foster engagement but not to directly impact any of the outcomes of interest. Two sample limited-access group messages are provided below: Family planning protects the health of mothers and babies. All couples have the right to choose the number of children they want. Do you ever feel pressured to have sex? Be clear about what you want and talk about your reasons. Practice negotiation with a trusted friend. References [1] Sedgh G, Hussain R. Reasons for contraceptive nonuse among women having unmet need for contraception in developing countries. Stud Fam Plann 2014;45:151–69. [2] Diamond-Smith N, Campbell M, Madan S. Misinformation and fear of side-effects of family planning. Cult Health Sex 2012;14(4):421–33. [3] Bongaarts J, Bruce J. The causes of unmet need for contraception and the social content of services. Stud Fam Plann 1995;57-75. [4] MLE, Tupange, KNBS. Report of the baseline household survey for the Kenya urban reproductive health initiative (Tupange); 2011. [5] ITU. The world in 2014: ICT facts and figures. Accessed on December 22nd 2015 at https://www.itu.int/en/ITU-D/Statistics/ Documents/facts/ICTFactsFigures2014-e.pdf. [6] L'Engle K, Vahdat H, Ndakidemi E, Lasway C, Zan T. Evaluating feasibility, reach and potential impact of a text message family planning information service in Tanzania. Contraception 2013;87:251–6. [7] Kaewkungwal J, Singhasivanon P, Khamsiriwatchara A, Sawang S, Meankaew P, Wechsart A. Application of smart phone in ‘better border healthcare program’: a module for mother and child care. BMC Med Inform Decis Mak 2010;10:69. [8] Leong KC, Chen WS, Leong KW, Mastura I, Mimi O, Sheikh MA, et al. The use of text messaging to improve attendance in primary care: a randomized controlled trial. Fam Pract 2006;23:699–705. [9] Liew SM, Tong SF, Lee VK, Ng CJ, Leong KC, Teng CL. Text messaging reminders to reduce non-attendance in chronic disease follow-up: a clinical trial. Gen Pract 2009;59(569):916–20. [10] Prasad S, Anand R. Use of mobile telephone short message service as a reminder: the effect on patient attendance. Int Dent J 2012;62:21–6.
[11] Chen ZW, Fang LZ, Chen LY, Dai HL. Comparison of an SMS text messaging and phone reminder to improve attendance at a health promotion center: a randomized controlled trial. J Zhejiang Univ Sci B 2008;9(1):34–8. [12] Da Costa TM, Salomão PL, Martha AS, Pisa IT, Sigulem D. The impact of short message service text messages sent as appointment reminders to patients' cell phones at outpatient clinics in Sao Paulo, Brazil. Med Inform 2010;79(1):65–70. [13] Odeny TA, Bailey RC, Bukusi EA, Simoni JM, Tapia KA, Yuhas K, et al. Text messaging to improve attendance at post-operative clinic visits after adult male circumcision for HIV prevention: a randomized controlled trial. PLoS One 2012;7(9):e43832. [14] Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010;376(9755):1838–45. [15] Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, De Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011;25(6):825. [16] Castaño PM, Bynum JY, Andrés R, Lara M, Westhoff C. Effect of daily text messages on oral contraceptive continuation: a randomized controlled trial. Obstet Gynecol 2012;119(1):14–20. [17] Jamison JC, Karlan D, Raffler P. Mixed method evaluation of a passive mHealth sexual information texting service in Uganda. Natl Bur Econ Res 2013. [18] Reference excluded to preserve anonymity of authors. [19] Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63(8):e1–37. [20] Little RJ, Rubin D. Statistical analysis with missing data. John Wiley & Sons; 2014. [21] Allison PD. Missing data. Sage publications; 2001. [22] Central Intelligence Agency. The world factbook. Accessed on December 22nd 2015 at https://www.cia.gov/library/publications/theworld-factbook/geos/ke.html. [23] National Bureau of Statistics-Kenya and ICF International. 2014 Kenya demographic and health survey key findings 2014. KNBS and ICF International; 2015. [24] Valente TW, Kim YM, Lettenmaier C, Glass W, Dibba Y. Radio promotion of family planning in the Gambia. Int Fam Plan Perspect 1994;96-100. [25] Kane TT, Gueye M, Speizer I, Pacque-Margolis S, Baron D. The impact of a family planning multimedia campaign in Bamako, Mali. Stud Fam Plann 1998;309-323. [26] Moos MK, Bartholomew N, Lohr K. Counseling in the clinical setting to prevent unintended pregnancy: an evidence-based research agenda. Contraception 2003;67(2):115–32. [27] Communications Authority of Kenya. Kenya's mobile penetration hits 80 per cent. Accessed from: http://www.ca.go.ke/index.php/what-wedo/94-news/285-kenya-s-mobile-penetration-hits-80-per-cent.
Original research article
A randomized controlled trial of the impact of a family planning mHealth service on knowledge and use of contraception Douglas Johnson a,⁎, Randall Juras b , Pamela Riley c , Minki Chatterji c , Phoebe Sloane c , Soon Kyu Choi d , Ben Johns c a
2407 15th St NW, Apartment 506, Washington, DC, United States b 55 Wheeler St., Cambridge, MA 02138, United States c 4550 Montgomery Avenue, Bethesda, MD 20814, United States d The Williams Institute, University of California Los Angeles School of Law, 337 Charles E Young Drive East, Los Angeles, CA 90095, United States Received 18 January 2016; revised 9 July 2016; accepted 11 July 2016
Abstract Objectives: mHealth, or the use of mobile phones for health, is a promising but largely untested method for increasing family planning knowledge in developing countries. This study estimates the effect of m4RH, an mHealth service in Kenya that provides family planning information via text message, on consumers' knowledge and use of contraception. Study design: We randomly assigned new consumers of the m4RH service to receive either full access or limited access to m4RH. We collected data on outcomes by sending questions directly to consumers via text message. Results: Response rates to the text message surveys ranged from 51.8% to 13.5%. Despite relatively low response rates, response rates were very similar across the full-access and limited-access groups. We find that full access to m4RH increased consumers' scores on a test of contraceptive knowledge by 14% (95% confidence interval: 9.9%–18.2%) compared to a control group with limited access to m4RH. m4RH did not increase consumers' use of contraception, likelihood of discussing family planning with their partners, or likelihood of visiting a clinic to discuss family planning. Conclusion: Text messages may increase family planning knowledge but do not, by themselves, lead to behavior change. Implications: Text messages can be an effective method of increasing family planning knowledge but may be insufficient on their own to cause behavior change. © 2016 Elsevier Inc. All rights reserved. Keywords: mHealth; Family planning; Behavior change communication; Kenya
1. Introduction Lack of information on family planning methods continues to be a major barrier to increased use of contraception in developing countries. According to a recent analysis of Demographic and Health Survey data, the two most commonly cited reasons for nonuse of contraception worldwide, after infrequent sexual activity, were concern ⁎ Corresponding author. E-mail addresses: [email protected] (D. Johnson), [email protected] (R. Juras), [email protected] (P. Riley), [email protected] (M. Chatterji), [email protected] (P. Sloane), [email protected] (S.K. Choi), [email protected] (B. Johns). http://dx.doi.org/10.1016/j.contraception.2016.07.009 0010-7824/© 2016 Elsevier Inc. All rights reserved.
over side effects or health risks and opposition to contraception by the woman, her partner or others close to her [1]. In-depth studies of these causes have shown that both of these factors — concerns over side effects and opposition to contraception — are often driven by misconceptions [2,3]. For example, in urban Kenya, the area that is the focus of the intervention evaluated in this study, a recent survey found that a majority of men and women believe that contraception may cause birth defects and harm to the uterus and 39% believe that contraception can cause infertility and cancer [4]. mHealth, or the use of mobile phones for health, is a promising method for increasing family planning knowledge and promoting positive attitudes toward contraceptive use. Access to mobile phones in the developing world is high and rising [5]; disseminating information over mobile phones is
2
D. Johnson et al. / Contraception xx (2016) xxx–xxx
inexpensive; and, unlike mass media, mobile phones allow for two-way interaction. In this study, we estimate the impact of m4RH, an mHealth family planning service in Kenya developed and implemented by FHI360, a nonprofit human development organization. m4RH is a free text-message-based platform that provides information on the benefits, disadvantages and side effects of nine family planning methods as well as a searchable database of clinics that offer family planning counseling and services. m4RH consumers may also sign up to receive “role model” stories about a person facing a difficult sexual or reproductive health issue and how they resolved the issue. A previous exploratory study of m4RH consumers provided suggestive evidence that the service was increasing use of contraception [6]. We conducted a randomized controlled trial to estimate the impact of m4RH on knowledge of contraception, use of contraception and other behavioral outcomes. During the study period, we randomly assigned all new m4RH consumers to receive either full access to the m4RH service (full-access group) or access only to the m4RH clinic database plus general health information messages (limited-access group). This study contributes to the literature on the effectiveness of text messages in increasing health-related knowledge and promoting behavior change in developing countries. Previous studies have shown text messages to be effective at increasing attendance at medical appointments [7–13] and adhering to ART treatment [14,15]. Few other published studies examine the effect of text messages on reproductive health knowledge and behavior change. Castaño et al. find that daily text messages increase continuation of oral contraceptive pills [16]. On the other hand, Jamison et al. [17] find that a text message service in Uganda that provided automated responses to sexual health questions had no effect on contraceptive knowledge and led to decreased use of condoms during sexual intercourse.
2. Methods 2.1. Recruitment Ethical approval was obtained from the Kenya Medical Research Institute ethics review committee and Abt Associates' institutional review board. As a “pull” service that only provides information upon request, FHI360 had to advertise m4RH to make potential consumers aware that it was available. To generate sufficient numbers of m4RH consumers for the study, FHI360 promoted m4RH through ads in newspapers, television and radio as well as at clinics providing family planning services and at various Ministry of Health events. These marketing activities focused primarily on urban areas. 2.2. Full-access and limited-access groups For the duration of the study period (September 2013–May 2014), we assigned all new consumers who accessed the m4RH
service to either a full-access group or a limited-access group. We assigned new consumers to each group on a rolling basis — that is, if the most recent new consumer was assigned to the full-access group, we assigned the current new consumer to the limited-access group. We consider this assignment rule effectively random for two reasons. First, m4RH had an extremely high number of consumers. Second, due to differences in network speed and coverage throughout Kenya, there was large variation in SMS delivery times. We did not seek consent from m4RH consumers prior to initial randomization as the risk to the limited-access group was low. We excluded all existing m4RH consumers from the study and continued to provide these consumers full access to the m4RH service. We provided members of the full-access group access to all of m4RH's features including information on the benefits, disadvantages and side effects of nine family planning methods, a searchable database of clinics offering family planning services and serial “role model” stories about a person facing a difficult family planning issue. We provided members of the limited-access group access to the clinic locator along with general motivational messages on a variety of health topics but did not provide access to any other m4RH content. We designed the motivational messages, examples of which are included in the appendix, to keep the consumers engaged with the m4RH service but not to directly affect any of the outcome measures focused on in this study. Members of the limited-access group were provided access to all m4RH content after data collection was complete; i.e., a period of 3 months. Unlike many other mHealth services, m4RH is a “pull” rather than a “push” service. Therefore, m4RH consumers were only sent content that they explicitly requested (Fig. 1). 2.3. Data We collected data on outcomes and covariates via text message. Due to the novelty of conducting a survey via text message, we performed several tests to determine the most effective combination of timing and incentives to encourage response and performed extensive field-testing of questions [18]. We divided survey questions into three waves as we anticipated low response rates to questions asked toward the end of the survey if all questions were included in a single survey. We sent the survey waves to each participant on a rolling basis based on the timing of their initial interaction with m4RH. We sent the first wave of survey questions to each participant approximately 24 h after the participant first accessed the m4RH system, the second wave approximately 6 days later, and the third wave approximately 3 months later. Each survey wave included an initial invitation explaining that consumers were free not to participate in the survey and that they could continue using m4RH if they chose not to respond. To prevent consumers below the age of 18 from participating in the study, we asked respondents their age at the start of each survey wave and discontinued the survey if the respondent was under 18.
D. Johnson et al. / Contraception xx (2016) xxx–xxx
3
Fig. 1. Random assignment diagram.
The first survey wave included four questions on the participant's age, use of contraception (two questions) and gender. The first question on use of contraception asked, “Do you or your sexual partner use contraception?” Respondents who replied “yes” were sent the follow-up question, “What type of contraception do you or your partner use?” The second survey wave included a question on the participant's age, a question on the participant's education and several questions designed to test the participant's knowledge of family planning. We carefully chose the knowledge questions in the second survey wave to ensure that they covered a broad range of family planning topics representative of the basic family planning information provided by m4RH. The third survey wave included questions on the participant's age, use of contraception (the same two questions as in the first wave), whether the participant had recently discussed family planning with his or her sexual partner, whether the participant had recently visited a clinic to discuss family planning, the participant's marital status and the participant's religion. Our primary outcome is the number of family planning knowledge questions (out of five) that the study participants answered correctly. Secondary outcomes are whether the participant (or the participant's partner) uses contraception, whether the participant recently discussed family planning with his or her partner and whether the participant recently visited a clinic to discuss family planning. Fig. 2 presents a modified CONSORT diagram [19] showing sources of participant attrition, as well as the sample remaining at each step of the enrollment and data-collection process. In addition to the data from these text message surveys, we had access to a log of all text messages sent or received by m4RH. This log included the full text message content,
the mobile number that the text message was sent to or received from, and the date and time the text message was sent or received. We used these data to verify that study participants were not looking up answers to survey questions by checking whether participants requested m4RH content between the time the survey question was sent and the participant submitted the survey answer. 2.4. Statistical analysis We perform two-sided t tests for equality of means to assess balance between full-access and limited-access participants on all variables except for education. For education, we test for equality by using a Pearson's χ 2 test. To estimate impact, we must confront several analytical challenges. First, overlap in response between survey waves is low — that is, there is a low correlation between answering at least one question on one survey wave and answering at least one question on another survey wave. Thus, if we only included data from study participants who answered at least one question in all survey waves, our sample would be greatly reduced. To deal with this challenge, we split our dataset into two analytic samples: study participants who answered at least one of the knowledge questions on survey wave two and study participants who answered at least one question on survey wave three. When estimating the impact of m4RH on overall knowledge of family planning, we use the first analytic sample. When estimating the impact of m4RH on behavioral outcomes, we use the second analytic sample. Due to these different analytic samples, findings from these two sets of analyses are not strictly comparable as the populations for which the results hold differ.
4
D. Johnson et al. / Contraception xx (2016) xxx–xxx
Fig. 2. Modified consort diagram.
A second analytic challenge is the high proportion of missing data within survey waves due to the fact that many study participants answered some but not all questions in a survey wave. This second challenge is particularly problematic for estimating the impact of m4RH on the overall knowledge score. Again, if we restricted analysis to only those study participants who answered all knowledge questions, our sample would be severely reduced as many study participants answered some but not all knowledge questions. To deal with this second challenge, we use multiple imputation to impute missing covariates and outcome variables in each analytic sample. Multiple imputation may also correct for imbalances in response rates such as those observed in the second set of survey questions if the assumptions behind the imputation model hold [20]. To impute data, we use a multivariate normal
model with a ridge prior and generate 20 imputations. Multivariate normal models have been shown to perform well in imputation even in cases where the true data-generating process is not multivariate normal [21]. We use separate imputation models for each analytical sample and within analytical samples for full-access and limited-access groups. Our imputation model for the analytic sample of study participants who answered at least one question on survey wave two includes all questions from that survey wave as well as age and gender from the first survey wave as these variables had low rates of missingness. Similarly, our imputation model for the analytic sample of study participants who answered at least one question from survey wave three includes all survey questions from that survey wave and age and gender from the first survey wave.
D. Johnson et al. / Contraception xx (2016) xxx–xxx
5
Table 1 Response rates by survey wave and treatment status
Survey wave 1 Answered at least one question Answered all questions Survey wave 2 Answered at least one question Answered at least one knowledge-related question Answered all questions Survey wave 3 Answered at least one question Answered all questions
Full access
Limited access
Difference
p value
51.8% 35.3%
51.4% 34.5%
0.4 ppt 0.8 ppt
.71 .31
39.1% 37.3% 21.5%
41.9% 40.1% 24.0%
−2.8 ppt −2.8 ppt −2.5 ppt
20.9% 13.5%
21.3% 13.8%
−0.4 ppt −0.3 fuppt
b.001 b.001 b.001 .57 .61
ppt: percentage points.
After imputing missing data, we estimate the impact of m4RH on each outcome variable first by performing two-sided t tests for equality of means and second by regressing the outcome variable on a dummy for full access and several covariates. Regression analysis may help correct for any remaining differences between the full-access and limited-access groups to the extent that these differences are linearly related to the covariates in our model [ibid]. We use the same covariates in each regression as we used for the original imputation model. For both the two-sided t tests and the regression analysis, we repeat each analysis for each of the 20 imputed datasets and combine the results using the standard multiple imputation formula. In addition to estimating the impact of m4RH on the overall knowledge score and the three behavioral outcomes, we also estimate the impact of m4RH on the probability of answering each individual knowledge question correctly. This analysis was not prespecified in the initial design report and thus should be viewed as exploratory. For this analysis, we do not use multiple imputation as missing data for one knowledge question do not prevent us from analyzing data
for a different knowledge question. To estimate the impact of m4RH on each individual knowledge question, we perform two-sided t tests for equality of means and regression analysis (using the same covariates as described above for the overall knowledge score). We prespecified all analytic methods employed in the study (with the exception of the exploratory analysis of the impact on individual knowledge questions) in a design report that was registered and made public prior to data collection through the 3ie Registry for Development Impact Evaluations (Study ID RIDIE-STUDY-ID-5294abf913209). Stata version 12 was used for all analyses.
3. Results Response rates to the surveys varied by survey wave (Table 1). For the first wave of text message questions, 51.6% of study participants answered at least one question and 34.9% answered all questions. For the third set of text message questions, only 21.1% of study participants
Table 2 Background characteristics by treatment status
Survey wave 1 Mean age Use contraception Female Survey wave 2 Mean age Highest education completed: primary education or less Highest education completed: secondary education Highest education completed: higher education Survey wave 3 Mean age Married Christian
Full access
Limited access
Difference
p value
24.5 (5.2) 69.6% (46.0%) 67.2% (47.0%)
24.6 (5.3) 71.4% (45.2%) 65.0% (47.7%)
−0.1 −1.8 ppt 2.2 ppt
.77 .82 .03
25.2 (7.4) 9.5% (29.3%) 40.8% (49.2%) 49.7% (50.0%)
24.6 (5.6) 9.2% (28.9%) 42.3% (49.4%) 48.5% (50.0%)
0.6 0.3 ppt −1.5 ppt 1.2 ppt
b.001 .69
24.9 (5.6) 50.4% (50.0%) 96.6% (18.3%)
25.0 (5.6) 48.7% (50.0%) 95.4% (21.0%)
−0.1 1.7 ppt 1.2 ppt
.49 .43 .16
Note: Significance level is determined by a two-sided test for equality of means of each variable (except for education, for which a Pearson's χ 2 test was used). Standard deviations in parenthesis.
6
D. Johnson et al. / Contraception xx (2016) xxx–xxx
Table 3 Impact of m4RH on overall knowledge score (n=5164). Outcome
Full access
Limited access
Difference
p value
Total # correct knowledge questions (out of 5)
2.19 (2.13–2.25)
1.92 (1.87–1.98)
0.27 (0.19–0.35)
b.001
Note: Means reported present missing answers filled in using multiple imputation (95% confidence intervals in parentheses). The fraction of missing information for total knowledge score is 0.26. Results reflect responses to the second survey.
answered at least one question and 16.3% answered all questions. Response rates for the two groups were similar for the first and third surveys. For the second survey, response rates were between 2 and 3 percentage points higher in the limited-access group than the full-access group. In addition, for the second set of survey questions, there is a statistically significant difference in the mean age of full-access and limited-access respondents. We discussed the strategies we use to deal with this imbalance in the section on statistical analysis. Table 2 presents the demographic characteristics of survey respondents. Table 3 displays unadjusted average knowledge scores, after multiple imputation, for the full-access and limited-access groups. We find that having full access to m4RH increased consumers' knowledge of family planning. On average, we find that full-access group members correctly answered 2.19 questions out of 5, while limited-access group members correctly answered 1.92 questions (pb.001). The estimated impact of 0.27 more correct knowledge question thus represents a 14% improvement in knowledge score compared with the limited-access group. When using regression to adjust for covariates, the results show a 0.26 difference (13% improvement in the full-access group compared to the limited-access group) and remains statistically significant (pb.001). In exploratory analyses of the nonimputed data from the five individual knowledge questions that form the composite knowledge score, we find that m4RH caused an increase in the proportion of correct responses for four of the five specific knowledge questions (Table 4). The only knowledge question for which we did not find a significant impact is the
question about the lactational amenorrhea method. Results do not change substantively when missing answers are imputed or when using regression-adjusted results (results not shown). We find only small and statistically insignificant (pN.1) impacts for all three behavioral outcomes: whether the consumer recently discussed family planning with his or her partner, whether the consumer visited a clinic to discuss family planning in the past month, and whether the consumer or the consumer's partner uses contraception (Table 5).
4. Discussion Response rates to the text message surveys were low, and thus p values from this study should be interpreted with caution. In particular, it is unclear whether these results are representative of the entire population of m4RH consumers. Despite relatively low response rates, response rates were very similar across the full-access and limited-access groups. Our sample of m4RH consumers is, on average, well educated (90.7% of respondents have a secondary education or higher), overwhelmingly Christian (96% are Christian compared to 82.5% of all Kenyans [22]) and more likely to use contraception than the general population of Kenya (71.4% of respondents to the first question indicated that they or their partner use contraception compared to 53.2% of married women of reproductive age and 60.9% of sexually active unmarried women of reproductive age in Kenya [23]). About 34% of consumers are male, and the average age was roughly 25 years.
Table 4 Impact of m4RH on individual knowledge questions (n=5164). Question
When is a woman most likely to get pregnant: just after period, halfway between periods, or just before period? Can women avoid pregnancy for 6 months after birth if period has not returned and breast feed baby: yes or no? About how long does coil last before it needs to be replaced: 1 year, 2 years, or more than 2 years? How many days after sex is EC pill effective: 1 day, 5 days, or 10 days? About how long do implants last before they need to be replaced: 6 months, 1 year, or more than 1 year?
n
Percentage answering correctly
Difference (in pct points)
p value
31.5% (29.7%–33.2%)
6.0 (3.4%–8.6%)
b.001
57.2% (54.9%–59.5%)
55.4% (53.2%–57.6%)
1.8 (−0.1%–5.0%)
.26
3612
45.2% (42.8%–47.6%)
38.2% (36.0%–40.3%)
7.0 (3.8%–10.2%)
b.001
3488
29.4% (27.2%–31.6%)
21.9% (20.0%–23.8%)
7.5 (4.6%–10.4%)
b.001
3329
50.3% (47.8%–52.8%)
45.0% (42.6%–47.3%)
5.4 (2.0%–8.8%)
.002
Full access
Limited access
5155
37.4% (35.6%–39.4%)
3751
Note: Means reported reflect unadjusted, nonimputed means; note results in Table 3 reflect imputed answers (95% confidence intervals in parentheses). Due to rounding, reported difference (T-C differences) may differ from differences between reported unadjusted means for the full-access and limited-access groups.
D. Johnson et al. / Contraception xx (2016) xxx–xxx
7
Table 5 Impact of m4RH on behavioral outcomes n
Full access
Limited access
Difference (in pct points) p value
Discussed family planning with partner in past month 2266 73.6% (71.0%–76.1%) 71.2% (69.1%–74.8%) 1.6% (−2.2%–5.4%) Visited clinic to discuss family planning with nurse or doctor 2197 42.1% (39.0%–45.2%) 38.8% (35.7%–41.9%) 3.2% (−1.0%–7.5%) Use contraception at endline 2452 79.7% (77.4%–81.9%) 79.6% (77.4%–81.7%) 0.1% (−3.0%–3.3%)
.40 .13 .94
Note: Means reported present missing answers filled in using multiple imputation (95% confidence intervals in parentheses). The fraction of missing information for ‘Discussed family planning with partner in past month’ is 0.37; for ‘Visited clinic to discuss family planning with nurse or doctor,’ it is 0.32; for ‘Use contraception at endline,’ it is 0.17.
We find that access to m4RH increased consumers' scores on a test of family planning by approximately 14%. This increase in family planning knowledge, while modest, appears to be of a reasonable size given the low dose of the intervention based on evidence from other low-touch interventions designed to increase family planning knowledge. For example, Valente et al. [24] find that a radio drama in The Gambia led to a 31% increase (from 4.2 to 5.5) in the number of modern family planning methods adults in the targeted area could name. Similarly, Kane et al. [25] find that a multimedia campaign in Bamako, Mali, led to a 4% increase (from 90% to 94%) in the proportion of women who could name at least one modern contraceptive method. We find that m4RH did not increase the likelihood of the consumer or the consumer's partner using contraception, the likelihood of the consumer having recently discussed family planning with his or her partner or the likelihood of the consumer having visited a clinic to discuss family planning in the past month. Our results have several implications. First, our results demonstrate that mHealth can be a cost-effective tool for increasing family planning knowledge. As the cost of each text message was only $0.045, the overall cost of delivering the program was relatively inexpensive. When compared to mass media campaigns of similar reach, text messages provide a more targeted, customizable channel to influence individual knowledge at lower cost. Second, our lack of a detectable impact on any of the behavioral outcomes suggests that text messages alone are inadequate to increase use of contraception. It is likely that the m4RH services — information, “role model” stories and clinic locator lists — were simply not sufficient to change a behavior that is governed by complex personal, cultural and systemic factors. These results are consistent with previous studies which have shown that increases in family planning knowledge alone often do not lead to changes in behavior [26]. Third, our findings suggest that programs implementing mHealth interventions should pay careful attention to the demographic profile of their consumers. With limited options for data collection from consumers, mHealth implementers often know very little about consumers of their service. Our findings show that the profile of consumers may be different from what implementers might anticipate. It is unclear whether the skewed demographic profile of m4RH consumers was due mainly to the choice of marketing, which
focused primarily on urban areas, or the fact that basic English literacy and a mobile phone are prerequisites to using the service. (According to the Communications Authority of Kenya, mobile phone penetration was above 80% at the time of this study, but this figure assumes no mobile phone users have more than one SIM card [27].) Further research is necessary to identify factors that can increase mHealth reach to underserved populations with a higher unmet need for family planning. Our study has several limitations. First, limited-access messages may have unintentionally affected outcomes among the limited-access group. We have no data to either support or refute this possibility but find it unlikely based on the general nature of the limited-access messages and our field-testing of these messages. Second, we rely on self-reported information for data on use of contraception and other behavioral outcomes. Third, the generalizability of our results is limited by both the low overall response rate to many of the survey questions and the degree to which the sample of study participants differs from the general Kenyan population. Role of the funding source The funder had no role in the study design, in the data collection or analysis of data, or in the authors' interpretation of the results but reviewed them and provided comments. The authors had final authority over the manuscript submitted for publication. Acknowledgments Douglas Johnson was employed by Abt Associates when this study was designed and carried out. This study was funded by the United States Agency for International Development through the Strengthening Health Outcomes through the Private Sector (SHOPS) project, managed by Abt Associates (Cooperative Agreement GPO-A-00-09-00007-00). We thank FHI360 and Marsden Solomon, Loice Magaria, Alice Olawo, Heather Vadhat and Kelly L'Engle in particular for allowing us to evaluate this program and for guidance and feedback on the design and implementation of the study. We thank Marie Stopes Kenya, PSI, Family Health Options of Kenya, FHI360 AHPIA Plus and the Jhpiego-led Tupange project for assistance in promoting m4RH during the study
8
D. Johnson et al. / Contraception xx (2016) xxx–xxx
period. We thank TexttoChange and Marcus Wagenaar in particular for designing the excellent platform on which m4RH was run and the text message surveys were conducted. Appendix A. Sample limited-access group messages Messages sent to the limited-access group were designed to foster engagement but not to directly impact any of the outcomes of interest. Two sample limited-access group messages are provided below: Family planning protects the health of mothers and babies. All couples have the right to choose the number of children they want. Do you ever feel pressured to have sex? Be clear about what you want and talk about your reasons. Practice negotiation with a trusted friend. References [1] Sedgh G, Hussain R. Reasons for contraceptive nonuse among women having unmet need for contraception in developing countries. Stud Fam Plann 2014;45:151–69. [2] Diamond-Smith N, Campbell M, Madan S. Misinformation and fear of side-effects of family planning. Cult Health Sex 2012;14(4):421–33. [3] Bongaarts J, Bruce J. The causes of unmet need for contraception and the social content of services. Stud Fam Plann 1995;57-75. [4] MLE, Tupange, KNBS. Report of the baseline household survey for the Kenya urban reproductive health initiative (Tupange); 2011. [5] ITU. The world in 2014: ICT facts and figures. Accessed on December 22nd 2015 at https://www.itu.int/en/ITU-D/Statistics/ Documents/facts/ICTFactsFigures2014-e.pdf. [6] L'Engle K, Vahdat H, Ndakidemi E, Lasway C, Zan T. Evaluating feasibility, reach and potential impact of a text message family planning information service in Tanzania. Contraception 2013;87:251–6. [7] Kaewkungwal J, Singhasivanon P, Khamsiriwatchara A, Sawang S, Meankaew P, Wechsart A. Application of smart phone in ‘better border healthcare program’: a module for mother and child care. BMC Med Inform Decis Mak 2010;10:69. [8] Leong KC, Chen WS, Leong KW, Mastura I, Mimi O, Sheikh MA, et al. The use of text messaging to improve attendance in primary care: a randomized controlled trial. Fam Pract 2006;23:699–705. [9] Liew SM, Tong SF, Lee VK, Ng CJ, Leong KC, Teng CL. Text messaging reminders to reduce non-attendance in chronic disease follow-up: a clinical trial. Gen Pract 2009;59(569):916–20. [10] Prasad S, Anand R. Use of mobile telephone short message service as a reminder: the effect on patient attendance. Int Dent J 2012;62:21–6.
[11] Chen ZW, Fang LZ, Chen LY, Dai HL. Comparison of an SMS text messaging and phone reminder to improve attendance at a health promotion center: a randomized controlled trial. J Zhejiang Univ Sci B 2008;9(1):34–8. [12] Da Costa TM, Salomão PL, Martha AS, Pisa IT, Sigulem D. The impact of short message service text messages sent as appointment reminders to patients' cell phones at outpatient clinics in Sao Paulo, Brazil. Med Inform 2010;79(1):65–70. [13] Odeny TA, Bailey RC, Bukusi EA, Simoni JM, Tapia KA, Yuhas K, et al. Text messaging to improve attendance at post-operative clinic visits after adult male circumcision for HIV prevention: a randomized controlled trial. PLoS One 2012;7(9):e43832. [14] Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010;376(9755):1838–45. [15] Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, De Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011;25(6):825. [16] Castaño PM, Bynum JY, Andrés R, Lara M, Westhoff C. Effect of daily text messages on oral contraceptive continuation: a randomized controlled trial. Obstet Gynecol 2012;119(1):14–20. [17] Jamison JC, Karlan D, Raffler P. Mixed method evaluation of a passive mHealth sexual information texting service in Uganda. Natl Bur Econ Res 2013. [18] Reference excluded to preserve anonymity of authors. [19] Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63(8):e1–37. [20] Little RJ, Rubin D. Statistical analysis with missing data. John Wiley & Sons; 2014. [21] Allison PD. Missing data. Sage publications; 2001. [22] Central Intelligence Agency. The world factbook. Accessed on December 22nd 2015 at https://www.cia.gov/library/publications/theworld-factbook/geos/ke.html. [23] National Bureau of Statistics-Kenya and ICF International. 2014 Kenya demographic and health survey key findings 2014. KNBS and ICF International; 2015. [24] Valente TW, Kim YM, Lettenmaier C, Glass W, Dibba Y. Radio promotion of family planning in the Gambia. Int Fam Plan Perspect 1994;96-100. [25] Kane TT, Gueye M, Speizer I, Pacque-Margolis S, Baron D. The impact of a family planning multimedia campaign in Bamako, Mali. Stud Fam Plann 1998;309-323. [26] Moos MK, Bartholomew N, Lohr K. Counseling in the clinical setting to prevent unintended pregnancy: an evidence-based research agenda. Contraception 2003;67(2):115–32. [27] Communications Authority of Kenya. Kenya's mobile penetration hits 80 per cent. Accessed from: http://www.ca.go.ke/index.php/what-wedo/94-news/285-kenya-s-mobile-penetration-hits-80-per-cent.