- Email: [email protected]
Development and validation of an instrument to assess institutionalization of health promotion in faith-based organizations
Evaluation and Program Planning 79 (2020) 101781
Contents lists available at ScienceDirect
Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan
Development and validation of an instrument to assess institutionalization of health promotion in faith-based organizations
T
Randi M. Williamsa,*,1, Jing Zhangb,2, Nathaniel Woodarda,3, Jimmie Sladec,4, Sherie Lou Zara Santosa,5, Cheryl L. Knotta,6 a
University of Maryland, School of Public Health, Department of Behavioral and Community Health, College Park, MD, United States University of Maryland, School of Public Health, Department of Epidemiology and Biostatistics, United States c Community Ministry of Prince George’s County, Upper Marlboro, MD, United States b
A R T I C LE I N FO
A B S T R A C T
Keywords: Adaptation Institutionalization Implementation Sustainability Faith-based organizations Community-based organizations Health promotion African Americans
Institutionalization of health promotion interventions occurs when the organization makes changes to support the program as a component of its routine operations. To date there has not been a way to systematically measure institutionalization of health promotion interventions outside of healthcare settings. The purpose of the present study was to develop and evaluate the initial psychometric properties of an instrument to assess institutionalization (i.e., integration) of health activities into faith-based organizations (i.e., churches). This process was informed by previous institutionalization models led by a team of experts and a community-based advisory panel. We recruited African American church leaders (N = 91) to complete a 22-item instrument. An exploratory factor analysis revealed four factors: 1) Organizational Structures (e.g., existing health ministry, health team), 2) Organizational Processes (e.g., records on health activities; instituted health policy), 3) Organizational Resources (e.g., health promotion budget; space for health activities), and 4) Organizational Communication (e.g., health content in church bulletins, discussion of health within sermons) that explained 62.3 % of the variance. The measure, the Faith-Based Organization Health Integration Inventory (FBO-HII), had excellent internal consistency reliability (α = .89) including the subscales (α = .90, .82, .81, and .87). This measure has promising initial psychometric properties for assessing institutionalization of health promotion interventions in faith-based settings.
1. Introduction Within the context of community-based health promotion it is important to consider how to sustain evidence-based programs beyond sporadic, term-limited, funding-dependent mechanisms. This is particularly true in community settings where the primary focus may be something other than health promotion and where resources may be limited. One strategy that may lead to sustainability is institutionalization, or the extent to which evidence-based interventions are integrated into their host settings through policies and practice
(Chambers, Glasgow, & Stange, 2013; Estabrooks et al., 2011; Pluye, Potvin, & Denis, 2004; Rabin & Brownson, 2017). Along the implementation continuum, adaptation and dynamic sustainability of interventions in organizations is receiving attention (Chambers et al., 2013). Intervention adaptation to better fit host organizational settings is increasingly embraced (Chambers et al., 2013; Chen, Reid, Parker, & Pillemer, 2013; Escoffery et al., 2018). There is also recent research on sustainability as a dynamic process (Chambers et al., 2013). Institutionalization is an important component of this continuum. However, little previous research exists on how to achieve
⁎ Corresponding author at: Georgetown University Medical Center, Lombardi Comprehensive Cancer Center, Cancer Prevention and Control Program. 3300 Whitehaven Street NW, Suite 4100, Washington, DC 20007, United States. E-mail addresses: [email protected], [email protected] (R.M. Williams), [email protected] (J. Zhang), [email protected] (N. Woodard), [email protected] (J. Slade), [email protected] (S.L.Z. Santos), [email protected] (C.L. Knott). 1 1101H Public Health Building (255), College Park, MD 20742, USA. 2 2234Q Public Health Building (255), College Park, MD 20742, USA. 3 1101G Public Health Building (255), College Park, MD 20742, USA. 4 PO Box 250, Upper Marlboro, MD 20773, USA. 5 1101E Public Health Building (255), College Park, MD 20742, USA. 6 1234W Public Health Building (255), College Park, MD 20742, USA.
https://doi.org/10.1016/j.evalprogplan.2020.101781 Received 7 July 2019; Received in revised form 19 January 2020; Accepted 20 January 2020 Available online 21 January 2020 0149-7189/ © 2020 Elsevier Ltd. All rights reserved.
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
churches’ routine operations and organizational structure. This work was done in consideration of the unique aspects of implementing a health promotion intervention in a faith-based setting where health promotion is not the primary function, specifically churches.
institutionalization or how to assess the extent to which it occurs for evidence-based health promotion interventions, particularly in community settings such as faith-based organizations. Faith-based organizations are defined as those that have religious components in their values, mission, and decision-making, such as churches. Understanding this dimension has implications for implementation, adaptation, and dynamic sustainability of interventions in organizations (Chambers et al., 2013). Public health researchers and practitioners have a long history and continue to partner with churches as a way to reach people with health promotion activities, particularly in medically underserved or minority communities (Campbell et al., 2007; DeHaven, Hunter, Wilder, Walton, & Berry, 2004; Flórez et al., 2019; Hou & Cao, 2018; Lancaster, Carter‐Edwards, Grilo, Shen, & Schoenthaler, 2014). However, with the availability of many evidence-based interventions suitable for faithbased organizations (Mercer et al., 2014; National Cancer Institute, 2019) there is a need to move beyond the initial efficacy of health promoting interventions to how to institutionalize research-tested health programs in faith-based organizations. In both faith-based organizations and other community settings that have incorporated health promotion activities, organizational level factors serve as both facilitators and barriers to program implementation and sustainability (Estabrooks et al., 2011; Johnson, Hays, Center, & Daley, 2004; Tagai et al., 2018; Weisbrod, Pirie, & Bracht, 1992; Yin, 1979). Organizational stability, funding sources (Estabrooks et al., 2011), leadership endorsement (Williams et al., 2018), organizational champions (Johnson et al., 2004), trained team members (Yin, 1979), time, resources, and programmtic interest and fit (Weisbrod et al., 1992) can impact the success or failure of planned programs. Initial tools have been developed to measure organizational factors within faith-based organizations implementing health promotion activities (Allen et al., 2015; Tagai et al., 2018), but little is known about how to capture the extent to which health programming is institutionalized or sustained in these settings. Determining the best ways to measure integration of health promotion interventions beyond its initial implementation could provide insight on organizational factors that may influence institutionalization and ultimately the long-term sustainability of evidence-based health programs. The few previously validated instruments to capture institutionalization of health promotion activities have been developed for healthcare settings (Goodman & Dean, 1982; Buller & McEvoy, 1989; Goodman, McLeroy, Steckler, & Hoyle, 1993). Goodman and Dean (1982) developed a measure of institutionalization that included five factors, Buller and McEvoy (1989) created a 3-item scale to measure this concept, and Yin (1979) used the ‘passages’ and ‘cycles’ to create a summative score of institutionalization. Goodman et al. (1993) subsequently created the Level of Institutionalization (LoIN) scale and validated it with a sample of N = 322 administrators representing various organizations including public schools, county health departments, and non-profit agencies. While the analysis confirmed an eight factor model and illustrated associations with the number of years the program had been in existence as well as perceptions of program permanency (Goodman et al., 1993), it was recommended that further development and validation were needed to facilitate widespread use (Scheirer, 1993). Other studies captured institutionalization of health promotion programs within the healthcare system and in workplaces (Patterson et al., 1998; Sorensen et al., 1998; Story et al., 2017). The organizational characteristics and structures of faith-based organizations where health promotion is not the primary mission are unique, making previous instruments unsuitable for these settings. The purpose of the present study was to develop and evaluate the initial psychometric properties of an instrument, the Faith-Based Organization Health Integration Inventory (FBO-HII), to measure the institutionalization of health promotion activities in faith-based organizations. This new tool seeks to measure the extent to which a health promotion intervention is institutionalized, or integrated into the
2. Method 2.1. Instrument development Select members of the HEAL 2.0 study team formed a working group to review existing instruments that assess institutionalization, in hopes of identifying one that could be adapted to measure this concept within a church setting. The working group was comprised of content experts including the Principal Investigator (an implementation scientist), a community partner who serves as a leader within an African American church, the study’s Project Manager, and a study team consultant with expertise in sustainability. While we reviewed a number of existing instruments, our community partners confirmed the limited fit of the items for use in a church setting (e.g., references to managers/supervisors and paid staff; extensive written organizational protocols, policies, and plans; supervisory and organizational structures). This lack of fit precluded the adaptation of an existing instrument and spoke to the need for a measure of institutionalization designed specifically for use in a faith-based setting. Inspired by the Level of Institutionalization (LoIN) scale (Goodman et al., 1993), the working group reviewed each subscale domain and corresponding items for any that may be able to be adapted for use in the church setting. No items were taken directly from the LoIN, and while most were not able to be adapted, some did inform new item development for the current instrument. Based upon the group’s itemby-item discussion, some items were deemed inappropriate for the setting (e.g., “proportion of staff having a written job description”; “extent to which evaluation reports are produced”) while others were deemed relevant for the church setting but needed to be reworded to fit (e.g., “number of times program formally evaluated” became “do you do any evaluation of the quality of your health activities”; “number of years locally adapted strategies followed” became “when you do health activities at church, are they adapted to fit the specific needs of your members”). Once an initial set of items was refined, the team presented them to the study’s advisory panel made up of community partners and church leaders who provided input on the wording and appropriateness of the items. This review resulted in negligible modifications to the items, but did suggest that a more appropriate term for use in these communities would be “integration” rather than “institutionalization”, due to a negative connotation with the latter term. Therefore, the term integration is used to reference the measure hereafter to reflect the current context. The integration measure includes items assessing both the presence and levels of integration of health activities into the organization. For the presence of health promotion items, a yes/no response format is used, for example, “Does your church have a health team, meaning more than one person who works together on health activities for the church?” Items assessing frequency of health promotion activities used a Likert-type response format, for example, “How often does your church hold organizational meetings specifically about health-related issue?” Response options included once a week, once a month, a few times a year, once a year, or never. Higher scores on the integration measure are indicative of a greater presence of integration of health activities. The full instrument and scoring information can be found in Table 1. 2.2. Data sources and collection This work was done in the context of Project HEAL 2.0 (Health through Early Awareness and Learning), a cluster randomized controlled trial that is evaluating a new approach to institutionalizing an evidence2
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
RStudio 3.6.1. Data collected in-person were entered directly into the dataset, and surveys administered online were imported into the dataset file. We first checked the missing percentages for all items, and those items with more than 50 % of the observations missing were deleted. The missing percentages of the remaining items ranged from 0.00 % (e.g., “Are the health activities your church provides done as part of the expectations of a regional or national religious organization?”) to 21.7 % (e.g., “Does your church provide a portion of the budget to support health activities?”). To address modest item missingness, we conducted imputation through logistic regression for binary data, predictive mean matching for numerical data, and proportional odds modeling for ordinal data (Little, 1988; Morris, White, & Royston, 2014; Van Buuren & Groothuis-Oudshoorn, 2011; White, Daniel, & Royston, 2010). We then computed a heterogeneous correlation matrix using the complete data (Drasgow, 1986; Holgado-Tello, ChacónMoscoso, Barbero-García, & Vila-Abad, 2010). We then conducted psychometric analyses to evaluate the integration measure and to identify potential subscales. To examine the associations between items, pearson correlations were calculated between numeric variables, polyserial correlations were calculated between numeric and ordinal variables, and polychoric correlations were calculated between ordinal variables. An exploratory factor analysis and reliability analysis were then conducted based on the heterogeneous correlation matrix using Generalized least squares extraction and Oblimin rotation. The inclusion criteria used to guide the retention of factors included: 1) all item loadings were above 0.30; 2) there were few item cross-loadings; 3) no factors had fewer than three items (Osborne, Costello, & Kellow, 2008); 4) all factors had an eigenvalue of ≥1; and 5) all the factors above (i.e. to the left of) the inflection point (i.e. the point where the curve starts to levels off) in the scree plot were retained, and any factor below (i.e. to the right of) the inflection point was eliminated. Finally, reliability was assessed through internal consistency, measured by coefficient alpha for the overall scale and subscales.
Table 1 Organization, Leader Characteristics, and Descriptive statistics for FBO-HII Measure (N = 91). Survey Item
N (%) or Mean (SD), Median
Church leader role Pastor Priest Deacon Lay minister Non-pastoral role Other Pastor employment outside church Yes No Denomination Baptist African Methodist Episcopal Zion African Methodist Episcopal Episcopal Pentecostal Church of God in Christ United Non-denominational Other Don’t Know Community setting Rural Small town Metropolitan Suburban Inner city Church owns building Yes No Size Estimated number of adult members Number of full time staff Number of part time staff
N (%) 49 (57.6) 6 (7.1) 5 (5.9) 7 (8.2) 6 (7.1) 12 (14.1) N (%) 54 (62.8) 32 (37.2) N (%) 29 (31.9) 1 (1.1) 11 (12.1) 2 (3.3) 7 (7.7) 2 (2.2) 2 (2.2) 25 (27.5) 10 (11.0) 1 (1.1) N (%) 7 (7.7) 5 (5.6) 11 (12.4) 30 (33.7) 36 (40.4) N (%) 19 (21.6) 69 (78.4) Mean (SD), Median 649.13 (2110.08), 150.00 4.76 (14.15), 2.00 4.41 (10.34), 2.00
Note: All variables are missing less than 10 % of data.
3. Results
based cancer education intervention in African American churches in Maryland, comparing it with a standard (non-institutionalized) approach (Knott et al., 2020). The primary outcome in the trial will be institutionalization as measured by the current instrument, administered at baseline and 12- and 24-month follow-ups. To evaluate the psychometric properties of the newly developed integration measure, a cross-sectional survey was conducted among N = 91 church leaders (N = 18 from the Project HEAL 2.0 trial church leaders who completed the baseline survey in-person and an ancillary sample of additional leaders (N = 73) recruited to complete the instrument online) in order to provide a sufficient number of observations. These additional individuals were recruited from a list of churches identified by team members and those who responded to advertisements posted on Facebook and a local faith-based magazine. Individuals who completed the survey were also asked to provide contact information for other leaders who may be interested in completing the survey. The eligibility criteria for the survey included: 1) self-identified leader (e.g., Pastor, Deacon, Health Ministry Leader) of a predominantly African American church and 2) able and willing to complete the questionnaire. Demographics and church characteristics were collected and analyzed including church leader employment, education, church denomination, and profile of congregation members. The survey was administered in-person and online using Qualtrics (2019). Participants received a $25 American Express gift card for their participation. This study was approved by the University of Maryland, College Park Institutional Review Board.
3.1. Participant and organization characteristics Prior to conducting the factor analysis, we conducted descriptive statistics to evaluate the distribution of all variables (Table 1). More than half (57.6 %) of the respondents indicated their role in the church as pastor, one third (31.9 %) were leaders of Baptist churches, and 40.4 % of the churches were in an inner city setting. The median size of the churches was 150 adult members. 3.2. Scale structure In the initial exploratory factor analysis, examination of the inclusion criteria involving eigenvalues, item loadings, factors, and the scree plot suggested retaining four factors. This model was refined by eliminating one item (“Is there one particular person at your church who is in charge of health activities?”) because this item had cross loadings on all factors with low loadings. Osborne et al. (2008) suggests dropping problematic items (ones that are low-loading, crossloading, or freestanding) and rerunning the analysis (Osborne et al., 2008). Table 2 presents the final factor analysis that produced an interpretable and clearer 4-factor solution. The proportion of variance explained by each factor was 17.9 %, 17.6 %, 13.9 %, and 12.9 %, for a total of 62.3 %. The 4-factor model included: 1) Organizational Structures (4 items) which assessed having a health ministry or having a team of people working together on health activities for the church, 2) Organizational Processes (6 items) which included keeping records on health activities or having an instituted health policy (e.g., about tobacco use on site, health meals served at church functions), 3) Organizational Resources (8 items) which measured concepts such as space provided for health
2.3. Data analysis Data analysis was conducted using SPSS 23.0 (IBM Corporation) and 3
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
Table 2 Pattern Coefficients for the Retain Factors from the Exploratory Factor Analysis (N = 91). Item
Component (% of variance explained) Structures (12.9 %)
Does your church have a health ministry? *Are you interested in establishing a health ministry? Does your church have a health team, meaning more than one person who works together on health activities for the church? Are the health activities your church provides done as part of the expectations of a regional or national religious organization? Approximately, how many volunteers work on health activities as part of their volunteer activities at the church? How many dedicated staff work on health activities as part of their role at the church? Does your church have a health policy (for example, about tobacco use on site or healthy meals served at church functions)? Does your church mission statement mention health? Does your church keep any records of its health activities? How often, if ever, does your church hold a health retreat, where a group of members set aside a longer period of time to talk about health? *Does your church conduct health activities? Does the church provide training for the people conducting your health activities? Do you do any evaluation of the quality of your health activities? When you do health activities at church, do you adapt them to fit your members’ needs better? Does your church have its own health clinic? Is space provided for health activities at your church? Do you do fundraising to support health activities at your church? Does your church provide a portion of the budget to support health activities? How often does your church hold organizational meetings specifically about healthrelated issues? How often, if ever, do you talk about health in your sermons? How often, if ever, do you include health content in church newsletters or bulletins? How often, if ever, do you include health content in church social media activity (e.g., web site; Facebook, etc.)?
Processes (17.6 %)
.702 1.084 .564
.497
.713
.305
Resources (17.9 %)
Communication (13.9 %)
.611 .376
.528 .675 .566 .793 .684 .755
.337
.413 .361
1.010 .419 .385 .555
.512 −.316
.349
.696 .532 .453 .689 .373
−.558 −.321 .633 .442 .814 .964
* Those who responded ‘yes’ to ‘Does your church have a health ministry?’ screened out of this item.
With regard to internal reliability, the FBO-HII had strong reliability as evidenced by a standardized Cronbach alpha value of 0.89. The internal consistency of the four factors was also acceptable (Organizational Resources α = 0.90; Organizational Processes α = 0.82; Organizational Communication α = 0.81; Organizational Structures α = 0.87).
the capacity of faith-based organizations to implement health promotion initiatives (Tagai, Scheirer, Santos, Haider, Bowie, et al., 2018). This new tool can be used in conjunction with the FBO-CI to assess organizational characteristics of African American faith-based organizations. For example, during the planning phase, the FBO-CI can determine whether churches have the capacity to implement health promotion programs and the FBO-HII can subsequently be used to capture integration of these activities in churches, as an outcome measure to evaluate the sustainability of evidence-based interventions. Future studies may adapt the tool to characterize levels of institutionalization in other types of faith-based or community-based organizations (e.g., community centers, non-profit organizations that offer social support services, salons/barbershops), which differ considerably from existing measures designed for healthcare systems.
4. Discussion
4.1. Limitations
The objective of this study was to develop a new instrument to measure integration of health promotion activities in faith-based organizations and to administer this tool to evaluate its initial psychometric properties. This new measure was able to be completed by FBO leaders, had an interpretable factor structure, and did not show signs of floor or ceiling effects in the study sample. The FBO-HII had good internal consistency reliability. The present analysis was not able to assess predictive validity of the FBO-HII, but this measure is currently being administered at baseline and follow-up in the ongoing trial, where our team will use it to evaluate the level of integrating evidence-based health promotion in African American churches pre- and post-intervention. The initial psychometrics suggest the integration measure has strong reliability and initial evidence of factorial validity. The FBO-HII extends work in this area where the Faith-Based Organization Capacity Inventory (FBO-CI) was developed for assessing
The current study is not without limitations. Cognitive interviews were not conducted to pilot test the items. However, they were presented to the advisory panel which was made up of church leaders and other community partners who provided input on the wording and appropriateness of the questions. This study included a sample of church leaders (N = 91), but future work should aim for a larger sample of church leaders. Another limitation could be the reliance upon self-report, which may have led to socially desirable responses. While this study used multiple recruitment methods, one-third of participants were leaders of Baptist churches. The type and frequency of health promotion activities in the Baptist denomination could differ from other faith-based organizations, although Baptist is a prominent denomination of African American churches (Sahgal & Smith, 2009). Additionally, this sample did not include other types of faith-based organization denominations (e.g., mosques, synagogues, Catholic churches).
activities at the church or a portion of the church’s budget dedicated to health promotion, and 4) Organizational Communication (4 items) including having health content in the church bulletin or talking about health in church sermons. Table 3 presents the FBO-HII items and scoring. 3.3. Internal reliability
4
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
Table 3 Faith-Based Organization Health Integration Inventory (FBO-HII) Items and Scoring. Component Organizational Structures
Organizational Processes
Organizational Resources
Organizational Communication
Item
Survey Question number a
Does your church have a health ministry? *Are you interested in establishing a health ministry? a Does your church have a health team, meaning more than one person who works together on health activities for the church? a Are the health activities your church provides done as part of the expectations of a regional or national religious organization? b Approximately, how many volunteers work on health activities as part of their volunteer activities at the church? c How many dedicated staff work on health activities as part of their role at the church? c Does your church have a health policy (for example, about tobacco use on site or healthy meals served at church functions)? a Does your church mission statement mention health? a Does your church keep any records of its health activities? a How often, if ever, does your church hold a health retreat, where a group of members set aside a longer period of time to talk about health? c *Does your church conduct health activities? a Does the church provide training for the people conducting your health activities? a Do you do any evaluation of the quality of your health activities? a When you do health activities at church, do you adapt them to fit your members’ needs better? a Does your church have its own health clinic? a Is space provided for health activities at your church? d Do you do fundraising to support health activities at your church? a Does your church provide a portion of the budget to support health activities? a How often does your church hold organizational meetings specifically about health-related issues? e How often, if ever, do you talk about health in your sermons? e How often, if ever, do you include health content in church newsletters or bulletins? e How often, if ever, do you include health content in church social media activity (e.g., web site; Facebook, etc.)? e
1 1b 2 3 4 5 8 9 10 16 1a 6 11 12 17 18 19 20 7 13 14 15
Item was presented with response options ‘yes’ (coded as ‘1′) and ‘no’ (coded as ‘0′). Item was presented with response options ‘yes’ (coded as ‘1′), ‘no’ (coded as ‘0′), and ‘some are, some are not’ (coded as ‘2′). c Item was presented with space for a written in response and was coded in line with the numeric response provided. d Item was presented with response options ‘space dedicated to health activities’ (coded as ‘2′), ‘space shared with other health activities’ (coded as ‘1′), and ‘no space for health activities (coded as ‘0′). e Item was presented with response options ‘once a week’ (coded as ‘4′), ‘once a month’ (coded as ‘3′), ‘a few times a year’ (coded as ‘2′), ‘once a year’ (coded as ‘1′), and ‘never’ (coded as ‘0′). * Those who responded ‘yes’ to ‘Does your church have a health ministry?’ screened out of this item. a
b
Our findings cannot be extended beyond the current sample, and future research is needed to use this tool in other faith-based organization denominations to determine its usefulness. Despite these limitations, we believe the development of this tool is an important first step in capturing adaptation and institutionalization of health promotion interventions in community-based organizations. However, further work is needed to assess the usefulness of this measure in other settings.
5. Conclusions As the field of public health moves toward greater emphasis on the dissemination and implementation of evidence-based interventions, additional study is needed to determine how to fully integrate health promotion programs into their host settings. Churches vary in their ability to initiate and sustain health promotion initiatives and tools are needed to assess these activities in order to appropriately assist faithbased organizations interested in health promotion. The FBO-HII demonstrated its ability to capture levels of institutionalization in faithbased organizations.
4.2. Lessons learned The process of developing the FBO-HII measure confirmed the particular challenges when considering health promotion in settings outside of the healthcare system, including competing priorities and limited resources. When reviewing existing indicators of institutionalization for consideration in the measure, we learned that concepts that may be taken for granted in a healthcare setting, such as health-related policy, is not something that the churches would naturally codify. We learned that the term ‘institutionalization’, though central to this work, carries a negative connotation. Therefore, we adopted the more benign term of ‘integration’. We learned the difficulty of obtaining data from church leaders who are exceptionally busy and may not be accustomed to answering detailed questions about their organization. There can be sensitivity in asking about things like resource prioritization in these organizations. Further, there are serious limitations to statistical power when the unit of analysis is the organization and the leader can only provide only one observation. Though there may be benefits of obtaining multiple observations per organization, often times the pastor may be the only person who is knowledgeable enough to suitable answer organizational questions.
Ethics approval and consent to participate Research involving human participants: This work was approved by the Institutional Review Board. Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committees and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent: A waiver of written informed consent was approved by the Institutional Review Board. This article does not contain any studies with animals performed by any of the authors. Funding This project was supported by a grant from the American Cancer Society (RSG1602201CPPB); trial registration “Integration of Cancer Health Activities Into African American Churches”: NCT03178383 5
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
(registered 06/05/17). The funding agency played no role in the study design; collection, analysis or interpretation of data; writing the report; and decision to submit for publication.
prevention innovations: A sustainability planning model. Evaluation and Program Planning, 27(2), 135–149. https://doi.org/10.1016/j.evalprogplan.2004.01.002. Knott, C. L., Bowie, J., Mullins, C. D., Santos, S. L. Z., Slade, J., Rosenberg, E., ... Williams, R. M. (2020). An approach to adapting a community-based cancer control intervention to organizational context (In Press) Health Promotion Practice. Lancaster, K. J., Carter‐Edwards, L., Grilo, S., Shen, C., & Schoenthaler, A. M. (2014). Obesity interventions in African American faith‐based organizations: A systematic review. Obesity Reviews, 15, 159–176. Little, R. J. A. (1988). Missing-data adjustments in large surveys (with discussion). Journal of Business Economics and Statistics, 6, 287–301. Mercer, S. L., Banks, S. M., Verma, P., Fisher, J. S., Corso, L. C., & Carlson, V. (2014). Guiding the way to public health improvement: exploring the connections between The Community Guide’s evidence-based interventions and health department accreditation standards. Journal of Public Health Management and Practice, 20(1), 104–110. Morris, T. P., White, I. R., & Royston, P. (2014). Tuning multiple imputation by predictive mean matching and local residual draws. BMC Medical Research Methodology, 14(1), 75. National Cancer Institute (2019). Research-tested intervention programs. Retrieved June 6, 2019, fromhttps://rtips.cancer.gov/rtips/index.do. Osborne, J. W., Costello, A. B., & Kellow, J. T. (2008). Best practices in exploratory factor analysis. Best practices in quantitative methods, 86–99. Patterson, R. E., Kristal, A. R., Biener, L., Varnes, J., Feng, Z., Glanz, K., et al. (1998). Durability and diffusion of the nutrition intervention in the Working Well Trial: For the working well research Group. Preventive Medicine, 27(5), 668–673. Pluye, P., Potvin, L., & Denis, J. L. (2004). Making public health programs last: Conceptualizing sustainability. Evaluation and Program Planning, 27(2), 121–133. Qualtrics (2019). Qualtrics software. Utah: Provo. Rabin, B. A., & Brownson, R. C. (2017). Terminology for dissemination and implementation research. Dissemination and implementation research in health: Translating science to practice, 2, 19–45. Sahgal, N., & Smith, G. (2009). A religious portrait of african-americans. Retrieved from Pew Research Center website:https://www.pewforum.org/2009/01/30/a-religiousportrait-of-african-americans/. Scheirer, M. A. (1993). Are the level of institutionalization scales ready for" prime time"? A commentary on" development of level of institutionalization (LoIn) scales for health promotion programs. Health Education Quarterly, 20(2), 179–182. Sorensen, G., Thompson, B., Basen-Engquist, K., Abrams, D., Kuniyuki, A., DiClemente, C., & Biener, L. (1998). Durability, dissemination, and institutionalization of worksite tobacco control programs: Results from the Working Well Trial. International Journal of Behavioral Medicine, 5(4), 335–351. Story, W. T., LeBan, K., Altobelli, L. C., Gebrian, B., Hossain, J., Lewis, J., ... Shanklin, D. (2017). Institutionalizing community-focused maternal, newborn, and child health strategies to strengthen health systems: A new framework for the Sustainable Development Goal era. Globalization and Health, 13(1), 37. Tagai, E. K., Scheirer, M. A., Santos, S. L. Z., Haider, M., Bowie, J., Slade, J., ... Holt, C. L. (2018). Assessing capacity of faith-based organizations for health promotion activities. Health Promotion Practice, 19(5), 714–723. Van Buuren, S., & Groothuis-Oudshoorn, K. (2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. https://www. jstatsoft.org/v45/i03/. Weisbrod, R. R., Pirie, P. L., & Bracht, N. F. (1992). Impact of a community health promotion program on existing organizations: The Minnesota Heart Health Program. Social Science & Medicine, 34(6), 639–648. https://doi.org/10.1016/0277-9536(92) 90191-R. White, I. R., Daniel, R., & Royston, P. (2010). Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Computational Statistics & Data Analysis, 54, 2267–2275. Williams, R. M., Tagai, E. K., Santos, S. L. Z., Slade, J. L., Carter, R. L., & Holt, C. L. (2018). The role of leadership support in a church-based Cancer education implementation study. Journal of Religion and Health, 57(1), 146–156. Yin, R. K. (1979). Changing urban bureaucracies: How new practices become routinized. Lexington, Mass: Lexington Books.
Financial disclosure No financial disclosures were reported by the authors of this paper. The team has no actual or potential conflicts of interest to report. CRediT authorship contribution statement Randi M. Williams: Conceptualization, Data curation, Formal analysis, Writing - original draft, Writing - review & editing. Jing Zhang: Formal analysis, Writing - review & editing. Nathaniel Woodard: Visualization, Writing - review & editing. Jimmie Slade: Project administration, Writing - review & editing. Sherie Lou Zara Santos: Project administration, Writing - review & editing. Cheryl L. Knott: Supervision, Writing - review & editing. Declaration of Competing Interest The authors declare that they have no competing interests. Acknowledgments The team would like to acknowledge content expert, Mary Ann Scheirer (posthumous), as well as members of the Community Advisory Panel for their contributions to the development of the institutionalization measure. We are also grateful for our community partners, Barbarajean Shaneman-Robinson, Rhonda Boozer-Yeary, and Ralph Williams for their recruitment support. References Allen, J. D., Tom, L. S., Leyva, B., Rustan, S., Ospino, H., Negron, R., et al. (2015). Recruiting and surveying Catholic parishes for cancer control initiatives: Lessons learned from the CRUZA Implementation Study. Health Promotion Practice, 16(5), 667–676. Buller, P. F., & McEvoy, G. M. (1989). Determinants of the institutionalization of planned organizational change. Group & Organization Studies, 14(1), 33–50. Campbell, M. K., Hudson, M. A., Resnicow, K., Blakeney, N., Paxton, A., & Baskin, M. (2007). Church-based health promotion interventions: Evidence and lessons learned. Annual Review of Public Health, 28, 213–234. Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. Chen, E. K., Reid, M. C., Parker, S. J., & Pillemer, K. (2013). Tailoring evidence-based interventions for new populations: A method for program adaptation through community engagement. Evaluation & the Health Professions, 36(1), 73–92. DeHaven, M. J., Hunter, I. B., Wilder, L., Walton, J. W., & Berry, J. (2004). Health programs in faith-based organizations: are they effective? American Journal of Public Health, 94(6), 1030–1036. Drasgow, F. (1986). Encyclopedia of statistical sciences. Escoffery, C., Lebow-Skelley, E., Udelson, H., Böing, E. A., Wood, R., Fernandez, M. E., & Mullen, P. D. (2018). A scoping study of frameworks for adapting public health evidence-based interventions. Translational Behavioral Medicine, 9(1), 1–10. Estabrooks, P. A., Smith-Ray, R. L., Dzewaltowski, D. A., Dowdy, D., Lattimore, D., Rheaume, C., ... Wilcox, S. (2011). Sustainability of evidence-based community-based physical activity programs for older adults: Lessons from active for life. Translational Behavioral Medicine, 1(2), 208–215. Flórez, K. R., Payán, D. D., Palar, K., Williams, M. V., Katic, B., & Derose, K. P. (2019). Church-based interventions to address obesity among African Americans and Latinos in the United States: A systematic review. Nutrition Reviews. Goodman, P. S., & Dean, J. W. (1982). Creating long-term organizational change. In P. S. Goodman (Ed.). Change in organizations (pp. 226–279). San Francisco, CA: JosseyBass. Goodman, R. M., McLeroy, K. R., Steckler, A., & Hoyle, R. H. (1993). Development of level of institutionalization (LoIn) scales for health promotion programs. Health Education Quarterly, 20(2), 161–178. Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, I., & Vila-Abad, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity, 44(1), 153. Hou, S. I., & Cao, X. (2018). A systematic review of promising strategies of faith-based cancer education and lifestyle interventions among racial/ethnic minority groups. Journal of Cancer Education, 33(6), 1161–1175. Johnson, K., Hays, C., Center, H., & Daley, C. (2004). Building capacity and sustainable
Randi M. Williams, PhD, MPH, is a postdoctoral fellow at Georgetown University Medical Center’s Lombardi Comprehensive Cancer Center. Broadly, her interests include cancer control and prevention, informed decision-making, and addressing health disparities through community-based participatory research. She recently completed her dissertation research project which explored the decision-making processes as well as informational and support needs of African American smokers for lung cancer screening. Randi obtained her PhD in behavioral and community health from the School of Public Health, University of Maryland, College Park and worked in the CHAMP lab during her time in the program. Jing Zhang, PhD, is an assistant professor at the Department of Epidemiology and Biostatistics of the School of Public Health at the University of Maryland. She received a PhD in Biostatistics from the University of Minnesota in 2014. Dr. Zhang conducts research in Bayesian hierarchical methods, missing data analysis, meta-analysis, network meta-analysis, diagnostic tests and clinical trials. Her work has appeared in Clinical Trials, Statistics in Medicine, Statistical Methods in Medical Research, Research Synthesis Methods, and Journal of Statistical Software. Dr. Zhang serves as an associate editor for the Journal of Biopharmaceutical Statistics. Nathaniel Woodard, MPH, is a PhD student in the Department of Behavioral and Community Health. After completing his Bachelor of Arts degree in public health and
6
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
Bachelor of Arts degree in Psychology from The George Washington University while following a Pre-med path and minoring in both Biology and Sociocultural Anthropology. She served as the Program Manager for Project HEAL (Health through Early Awareness and Learning), a National Cancer Institute-funded project that identified the best way to implement evidence-based strategies for increasing cancer screening behaviors.
psychology at the University of Rochester, Nate began in the CHAMP Lab as a summer research intern. After completing his internship, he then transitioned into a graduate research assistant position with the lab. Previously, Nate has spent time working as a peer health educator, exercise and nutrition consultant, and a health promotions specialist. His research interests surround the effects of social interaction on health behaviors and motivations.
Cheryl L. Knott, PhD, is a Professor in the Department of Behavioral and Community Health and Co-leads the Population Science Program in the University of Maryland Greenebaum Cancer Center. She is founding Director of the Community Health Awareness, Messages, and Prevention research lab. Her dissemination/implementation research program looks at organizational factors that influence implementation outcomes, and she is currently evaluating a strategy to integrate or institutionalize evidencebased cancer education in African American churches. Dr. Knott's social epidemiology research program examines religious involvement and health among African Americans, in a longitudinal national sample of African Americans.
Jimmie Slade, MA, served his country for decades as a US Army officer and now continues to serve his community as Executive Director, Community Ministry of Prince George's County, a 38 year old Faith-based non-profit organization. He earned a BS degree from NC A&T State University, MA from Boston University, and is a graduate of the US Army War College. Sherie Lou Zara Santos, MPH, is a Certified Health Education Specialist who earned her dual MPH at the Loma Linda University in Loma Linda, California with concentrations in both Health Promotion Education and Maternal and Child Health. She received her
7
Contents lists available at ScienceDirect
Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan
Development and validation of an instrument to assess institutionalization of health promotion in faith-based organizations
T
Randi M. Williamsa,*,1, Jing Zhangb,2, Nathaniel Woodarda,3, Jimmie Sladec,4, Sherie Lou Zara Santosa,5, Cheryl L. Knotta,6 a
University of Maryland, School of Public Health, Department of Behavioral and Community Health, College Park, MD, United States University of Maryland, School of Public Health, Department of Epidemiology and Biostatistics, United States c Community Ministry of Prince George’s County, Upper Marlboro, MD, United States b
A R T I C LE I N FO
A B S T R A C T
Keywords: Adaptation Institutionalization Implementation Sustainability Faith-based organizations Community-based organizations Health promotion African Americans
Institutionalization of health promotion interventions occurs when the organization makes changes to support the program as a component of its routine operations. To date there has not been a way to systematically measure institutionalization of health promotion interventions outside of healthcare settings. The purpose of the present study was to develop and evaluate the initial psychometric properties of an instrument to assess institutionalization (i.e., integration) of health activities into faith-based organizations (i.e., churches). This process was informed by previous institutionalization models led by a team of experts and a community-based advisory panel. We recruited African American church leaders (N = 91) to complete a 22-item instrument. An exploratory factor analysis revealed four factors: 1) Organizational Structures (e.g., existing health ministry, health team), 2) Organizational Processes (e.g., records on health activities; instituted health policy), 3) Organizational Resources (e.g., health promotion budget; space for health activities), and 4) Organizational Communication (e.g., health content in church bulletins, discussion of health within sermons) that explained 62.3 % of the variance. The measure, the Faith-Based Organization Health Integration Inventory (FBO-HII), had excellent internal consistency reliability (α = .89) including the subscales (α = .90, .82, .81, and .87). This measure has promising initial psychometric properties for assessing institutionalization of health promotion interventions in faith-based settings.
1. Introduction Within the context of community-based health promotion it is important to consider how to sustain evidence-based programs beyond sporadic, term-limited, funding-dependent mechanisms. This is particularly true in community settings where the primary focus may be something other than health promotion and where resources may be limited. One strategy that may lead to sustainability is institutionalization, or the extent to which evidence-based interventions are integrated into their host settings through policies and practice
(Chambers, Glasgow, & Stange, 2013; Estabrooks et al., 2011; Pluye, Potvin, & Denis, 2004; Rabin & Brownson, 2017). Along the implementation continuum, adaptation and dynamic sustainability of interventions in organizations is receiving attention (Chambers et al., 2013). Intervention adaptation to better fit host organizational settings is increasingly embraced (Chambers et al., 2013; Chen, Reid, Parker, & Pillemer, 2013; Escoffery et al., 2018). There is also recent research on sustainability as a dynamic process (Chambers et al., 2013). Institutionalization is an important component of this continuum. However, little previous research exists on how to achieve
⁎ Corresponding author at: Georgetown University Medical Center, Lombardi Comprehensive Cancer Center, Cancer Prevention and Control Program. 3300 Whitehaven Street NW, Suite 4100, Washington, DC 20007, United States. E-mail addresses: [email protected], [email protected] (R.M. Williams), [email protected] (J. Zhang), [email protected] (N. Woodard), [email protected] (J. Slade), [email protected] (S.L.Z. Santos), [email protected] (C.L. Knott). 1 1101H Public Health Building (255), College Park, MD 20742, USA. 2 2234Q Public Health Building (255), College Park, MD 20742, USA. 3 1101G Public Health Building (255), College Park, MD 20742, USA. 4 PO Box 250, Upper Marlboro, MD 20773, USA. 5 1101E Public Health Building (255), College Park, MD 20742, USA. 6 1234W Public Health Building (255), College Park, MD 20742, USA.
https://doi.org/10.1016/j.evalprogplan.2020.101781 Received 7 July 2019; Received in revised form 19 January 2020; Accepted 20 January 2020 Available online 21 January 2020 0149-7189/ © 2020 Elsevier Ltd. All rights reserved.
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
churches’ routine operations and organizational structure. This work was done in consideration of the unique aspects of implementing a health promotion intervention in a faith-based setting where health promotion is not the primary function, specifically churches.
institutionalization or how to assess the extent to which it occurs for evidence-based health promotion interventions, particularly in community settings such as faith-based organizations. Faith-based organizations are defined as those that have religious components in their values, mission, and decision-making, such as churches. Understanding this dimension has implications for implementation, adaptation, and dynamic sustainability of interventions in organizations (Chambers et al., 2013). Public health researchers and practitioners have a long history and continue to partner with churches as a way to reach people with health promotion activities, particularly in medically underserved or minority communities (Campbell et al., 2007; DeHaven, Hunter, Wilder, Walton, & Berry, 2004; Flórez et al., 2019; Hou & Cao, 2018; Lancaster, Carter‐Edwards, Grilo, Shen, & Schoenthaler, 2014). However, with the availability of many evidence-based interventions suitable for faithbased organizations (Mercer et al., 2014; National Cancer Institute, 2019) there is a need to move beyond the initial efficacy of health promoting interventions to how to institutionalize research-tested health programs in faith-based organizations. In both faith-based organizations and other community settings that have incorporated health promotion activities, organizational level factors serve as both facilitators and barriers to program implementation and sustainability (Estabrooks et al., 2011; Johnson, Hays, Center, & Daley, 2004; Tagai et al., 2018; Weisbrod, Pirie, & Bracht, 1992; Yin, 1979). Organizational stability, funding sources (Estabrooks et al., 2011), leadership endorsement (Williams et al., 2018), organizational champions (Johnson et al., 2004), trained team members (Yin, 1979), time, resources, and programmtic interest and fit (Weisbrod et al., 1992) can impact the success or failure of planned programs. Initial tools have been developed to measure organizational factors within faith-based organizations implementing health promotion activities (Allen et al., 2015; Tagai et al., 2018), but little is known about how to capture the extent to which health programming is institutionalized or sustained in these settings. Determining the best ways to measure integration of health promotion interventions beyond its initial implementation could provide insight on organizational factors that may influence institutionalization and ultimately the long-term sustainability of evidence-based health programs. The few previously validated instruments to capture institutionalization of health promotion activities have been developed for healthcare settings (Goodman & Dean, 1982; Buller & McEvoy, 1989; Goodman, McLeroy, Steckler, & Hoyle, 1993). Goodman and Dean (1982) developed a measure of institutionalization that included five factors, Buller and McEvoy (1989) created a 3-item scale to measure this concept, and Yin (1979) used the ‘passages’ and ‘cycles’ to create a summative score of institutionalization. Goodman et al. (1993) subsequently created the Level of Institutionalization (LoIN) scale and validated it with a sample of N = 322 administrators representing various organizations including public schools, county health departments, and non-profit agencies. While the analysis confirmed an eight factor model and illustrated associations with the number of years the program had been in existence as well as perceptions of program permanency (Goodman et al., 1993), it was recommended that further development and validation were needed to facilitate widespread use (Scheirer, 1993). Other studies captured institutionalization of health promotion programs within the healthcare system and in workplaces (Patterson et al., 1998; Sorensen et al., 1998; Story et al., 2017). The organizational characteristics and structures of faith-based organizations where health promotion is not the primary mission are unique, making previous instruments unsuitable for these settings. The purpose of the present study was to develop and evaluate the initial psychometric properties of an instrument, the Faith-Based Organization Health Integration Inventory (FBO-HII), to measure the institutionalization of health promotion activities in faith-based organizations. This new tool seeks to measure the extent to which a health promotion intervention is institutionalized, or integrated into the
2. Method 2.1. Instrument development Select members of the HEAL 2.0 study team formed a working group to review existing instruments that assess institutionalization, in hopes of identifying one that could be adapted to measure this concept within a church setting. The working group was comprised of content experts including the Principal Investigator (an implementation scientist), a community partner who serves as a leader within an African American church, the study’s Project Manager, and a study team consultant with expertise in sustainability. While we reviewed a number of existing instruments, our community partners confirmed the limited fit of the items for use in a church setting (e.g., references to managers/supervisors and paid staff; extensive written organizational protocols, policies, and plans; supervisory and organizational structures). This lack of fit precluded the adaptation of an existing instrument and spoke to the need for a measure of institutionalization designed specifically for use in a faith-based setting. Inspired by the Level of Institutionalization (LoIN) scale (Goodman et al., 1993), the working group reviewed each subscale domain and corresponding items for any that may be able to be adapted for use in the church setting. No items were taken directly from the LoIN, and while most were not able to be adapted, some did inform new item development for the current instrument. Based upon the group’s itemby-item discussion, some items were deemed inappropriate for the setting (e.g., “proportion of staff having a written job description”; “extent to which evaluation reports are produced”) while others were deemed relevant for the church setting but needed to be reworded to fit (e.g., “number of times program formally evaluated” became “do you do any evaluation of the quality of your health activities”; “number of years locally adapted strategies followed” became “when you do health activities at church, are they adapted to fit the specific needs of your members”). Once an initial set of items was refined, the team presented them to the study’s advisory panel made up of community partners and church leaders who provided input on the wording and appropriateness of the items. This review resulted in negligible modifications to the items, but did suggest that a more appropriate term for use in these communities would be “integration” rather than “institutionalization”, due to a negative connotation with the latter term. Therefore, the term integration is used to reference the measure hereafter to reflect the current context. The integration measure includes items assessing both the presence and levels of integration of health activities into the organization. For the presence of health promotion items, a yes/no response format is used, for example, “Does your church have a health team, meaning more than one person who works together on health activities for the church?” Items assessing frequency of health promotion activities used a Likert-type response format, for example, “How often does your church hold organizational meetings specifically about health-related issue?” Response options included once a week, once a month, a few times a year, once a year, or never. Higher scores on the integration measure are indicative of a greater presence of integration of health activities. The full instrument and scoring information can be found in Table 1. 2.2. Data sources and collection This work was done in the context of Project HEAL 2.0 (Health through Early Awareness and Learning), a cluster randomized controlled trial that is evaluating a new approach to institutionalizing an evidence2
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
RStudio 3.6.1. Data collected in-person were entered directly into the dataset, and surveys administered online were imported into the dataset file. We first checked the missing percentages for all items, and those items with more than 50 % of the observations missing were deleted. The missing percentages of the remaining items ranged from 0.00 % (e.g., “Are the health activities your church provides done as part of the expectations of a regional or national religious organization?”) to 21.7 % (e.g., “Does your church provide a portion of the budget to support health activities?”). To address modest item missingness, we conducted imputation through logistic regression for binary data, predictive mean matching for numerical data, and proportional odds modeling for ordinal data (Little, 1988; Morris, White, & Royston, 2014; Van Buuren & Groothuis-Oudshoorn, 2011; White, Daniel, & Royston, 2010). We then computed a heterogeneous correlation matrix using the complete data (Drasgow, 1986; Holgado-Tello, ChacónMoscoso, Barbero-García, & Vila-Abad, 2010). We then conducted psychometric analyses to evaluate the integration measure and to identify potential subscales. To examine the associations between items, pearson correlations were calculated between numeric variables, polyserial correlations were calculated between numeric and ordinal variables, and polychoric correlations were calculated between ordinal variables. An exploratory factor analysis and reliability analysis were then conducted based on the heterogeneous correlation matrix using Generalized least squares extraction and Oblimin rotation. The inclusion criteria used to guide the retention of factors included: 1) all item loadings were above 0.30; 2) there were few item cross-loadings; 3) no factors had fewer than three items (Osborne, Costello, & Kellow, 2008); 4) all factors had an eigenvalue of ≥1; and 5) all the factors above (i.e. to the left of) the inflection point (i.e. the point where the curve starts to levels off) in the scree plot were retained, and any factor below (i.e. to the right of) the inflection point was eliminated. Finally, reliability was assessed through internal consistency, measured by coefficient alpha for the overall scale and subscales.
Table 1 Organization, Leader Characteristics, and Descriptive statistics for FBO-HII Measure (N = 91). Survey Item
N (%) or Mean (SD), Median
Church leader role Pastor Priest Deacon Lay minister Non-pastoral role Other Pastor employment outside church Yes No Denomination Baptist African Methodist Episcopal Zion African Methodist Episcopal Episcopal Pentecostal Church of God in Christ United Non-denominational Other Don’t Know Community setting Rural Small town Metropolitan Suburban Inner city Church owns building Yes No Size Estimated number of adult members Number of full time staff Number of part time staff
N (%) 49 (57.6) 6 (7.1) 5 (5.9) 7 (8.2) 6 (7.1) 12 (14.1) N (%) 54 (62.8) 32 (37.2) N (%) 29 (31.9) 1 (1.1) 11 (12.1) 2 (3.3) 7 (7.7) 2 (2.2) 2 (2.2) 25 (27.5) 10 (11.0) 1 (1.1) N (%) 7 (7.7) 5 (5.6) 11 (12.4) 30 (33.7) 36 (40.4) N (%) 19 (21.6) 69 (78.4) Mean (SD), Median 649.13 (2110.08), 150.00 4.76 (14.15), 2.00 4.41 (10.34), 2.00
Note: All variables are missing less than 10 % of data.
3. Results
based cancer education intervention in African American churches in Maryland, comparing it with a standard (non-institutionalized) approach (Knott et al., 2020). The primary outcome in the trial will be institutionalization as measured by the current instrument, administered at baseline and 12- and 24-month follow-ups. To evaluate the psychometric properties of the newly developed integration measure, a cross-sectional survey was conducted among N = 91 church leaders (N = 18 from the Project HEAL 2.0 trial church leaders who completed the baseline survey in-person and an ancillary sample of additional leaders (N = 73) recruited to complete the instrument online) in order to provide a sufficient number of observations. These additional individuals were recruited from a list of churches identified by team members and those who responded to advertisements posted on Facebook and a local faith-based magazine. Individuals who completed the survey were also asked to provide contact information for other leaders who may be interested in completing the survey. The eligibility criteria for the survey included: 1) self-identified leader (e.g., Pastor, Deacon, Health Ministry Leader) of a predominantly African American church and 2) able and willing to complete the questionnaire. Demographics and church characteristics were collected and analyzed including church leader employment, education, church denomination, and profile of congregation members. The survey was administered in-person and online using Qualtrics (2019). Participants received a $25 American Express gift card for their participation. This study was approved by the University of Maryland, College Park Institutional Review Board.
3.1. Participant and organization characteristics Prior to conducting the factor analysis, we conducted descriptive statistics to evaluate the distribution of all variables (Table 1). More than half (57.6 %) of the respondents indicated their role in the church as pastor, one third (31.9 %) were leaders of Baptist churches, and 40.4 % of the churches were in an inner city setting. The median size of the churches was 150 adult members. 3.2. Scale structure In the initial exploratory factor analysis, examination of the inclusion criteria involving eigenvalues, item loadings, factors, and the scree plot suggested retaining four factors. This model was refined by eliminating one item (“Is there one particular person at your church who is in charge of health activities?”) because this item had cross loadings on all factors with low loadings. Osborne et al. (2008) suggests dropping problematic items (ones that are low-loading, crossloading, or freestanding) and rerunning the analysis (Osborne et al., 2008). Table 2 presents the final factor analysis that produced an interpretable and clearer 4-factor solution. The proportion of variance explained by each factor was 17.9 %, 17.6 %, 13.9 %, and 12.9 %, for a total of 62.3 %. The 4-factor model included: 1) Organizational Structures (4 items) which assessed having a health ministry or having a team of people working together on health activities for the church, 2) Organizational Processes (6 items) which included keeping records on health activities or having an instituted health policy (e.g., about tobacco use on site, health meals served at church functions), 3) Organizational Resources (8 items) which measured concepts such as space provided for health
2.3. Data analysis Data analysis was conducted using SPSS 23.0 (IBM Corporation) and 3
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
Table 2 Pattern Coefficients for the Retain Factors from the Exploratory Factor Analysis (N = 91). Item
Component (% of variance explained) Structures (12.9 %)
Does your church have a health ministry? *Are you interested in establishing a health ministry? Does your church have a health team, meaning more than one person who works together on health activities for the church? Are the health activities your church provides done as part of the expectations of a regional or national religious organization? Approximately, how many volunteers work on health activities as part of their volunteer activities at the church? How many dedicated staff work on health activities as part of their role at the church? Does your church have a health policy (for example, about tobacco use on site or healthy meals served at church functions)? Does your church mission statement mention health? Does your church keep any records of its health activities? How often, if ever, does your church hold a health retreat, where a group of members set aside a longer period of time to talk about health? *Does your church conduct health activities? Does the church provide training for the people conducting your health activities? Do you do any evaluation of the quality of your health activities? When you do health activities at church, do you adapt them to fit your members’ needs better? Does your church have its own health clinic? Is space provided for health activities at your church? Do you do fundraising to support health activities at your church? Does your church provide a portion of the budget to support health activities? How often does your church hold organizational meetings specifically about healthrelated issues? How often, if ever, do you talk about health in your sermons? How often, if ever, do you include health content in church newsletters or bulletins? How often, if ever, do you include health content in church social media activity (e.g., web site; Facebook, etc.)?
Processes (17.6 %)
.702 1.084 .564
.497
.713
.305
Resources (17.9 %)
Communication (13.9 %)
.611 .376
.528 .675 .566 .793 .684 .755
.337
.413 .361
1.010 .419 .385 .555
.512 −.316
.349
.696 .532 .453 .689 .373
−.558 −.321 .633 .442 .814 .964
* Those who responded ‘yes’ to ‘Does your church have a health ministry?’ screened out of this item.
With regard to internal reliability, the FBO-HII had strong reliability as evidenced by a standardized Cronbach alpha value of 0.89. The internal consistency of the four factors was also acceptable (Organizational Resources α = 0.90; Organizational Processes α = 0.82; Organizational Communication α = 0.81; Organizational Structures α = 0.87).
the capacity of faith-based organizations to implement health promotion initiatives (Tagai, Scheirer, Santos, Haider, Bowie, et al., 2018). This new tool can be used in conjunction with the FBO-CI to assess organizational characteristics of African American faith-based organizations. For example, during the planning phase, the FBO-CI can determine whether churches have the capacity to implement health promotion programs and the FBO-HII can subsequently be used to capture integration of these activities in churches, as an outcome measure to evaluate the sustainability of evidence-based interventions. Future studies may adapt the tool to characterize levels of institutionalization in other types of faith-based or community-based organizations (e.g., community centers, non-profit organizations that offer social support services, salons/barbershops), which differ considerably from existing measures designed for healthcare systems.
4. Discussion
4.1. Limitations
The objective of this study was to develop a new instrument to measure integration of health promotion activities in faith-based organizations and to administer this tool to evaluate its initial psychometric properties. This new measure was able to be completed by FBO leaders, had an interpretable factor structure, and did not show signs of floor or ceiling effects in the study sample. The FBO-HII had good internal consistency reliability. The present analysis was not able to assess predictive validity of the FBO-HII, but this measure is currently being administered at baseline and follow-up in the ongoing trial, where our team will use it to evaluate the level of integrating evidence-based health promotion in African American churches pre- and post-intervention. The initial psychometrics suggest the integration measure has strong reliability and initial evidence of factorial validity. The FBO-HII extends work in this area where the Faith-Based Organization Capacity Inventory (FBO-CI) was developed for assessing
The current study is not without limitations. Cognitive interviews were not conducted to pilot test the items. However, they were presented to the advisory panel which was made up of church leaders and other community partners who provided input on the wording and appropriateness of the questions. This study included a sample of church leaders (N = 91), but future work should aim for a larger sample of church leaders. Another limitation could be the reliance upon self-report, which may have led to socially desirable responses. While this study used multiple recruitment methods, one-third of participants were leaders of Baptist churches. The type and frequency of health promotion activities in the Baptist denomination could differ from other faith-based organizations, although Baptist is a prominent denomination of African American churches (Sahgal & Smith, 2009). Additionally, this sample did not include other types of faith-based organization denominations (e.g., mosques, synagogues, Catholic churches).
activities at the church or a portion of the church’s budget dedicated to health promotion, and 4) Organizational Communication (4 items) including having health content in the church bulletin or talking about health in church sermons. Table 3 presents the FBO-HII items and scoring. 3.3. Internal reliability
4
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
Table 3 Faith-Based Organization Health Integration Inventory (FBO-HII) Items and Scoring. Component Organizational Structures
Organizational Processes
Organizational Resources
Organizational Communication
Item
Survey Question number a
Does your church have a health ministry? *Are you interested in establishing a health ministry? a Does your church have a health team, meaning more than one person who works together on health activities for the church? a Are the health activities your church provides done as part of the expectations of a regional or national religious organization? b Approximately, how many volunteers work on health activities as part of their volunteer activities at the church? c How many dedicated staff work on health activities as part of their role at the church? c Does your church have a health policy (for example, about tobacco use on site or healthy meals served at church functions)? a Does your church mission statement mention health? a Does your church keep any records of its health activities? a How often, if ever, does your church hold a health retreat, where a group of members set aside a longer period of time to talk about health? c *Does your church conduct health activities? a Does the church provide training for the people conducting your health activities? a Do you do any evaluation of the quality of your health activities? a When you do health activities at church, do you adapt them to fit your members’ needs better? a Does your church have its own health clinic? a Is space provided for health activities at your church? d Do you do fundraising to support health activities at your church? a Does your church provide a portion of the budget to support health activities? a How often does your church hold organizational meetings specifically about health-related issues? e How often, if ever, do you talk about health in your sermons? e How often, if ever, do you include health content in church newsletters or bulletins? e How often, if ever, do you include health content in church social media activity (e.g., web site; Facebook, etc.)? e
1 1b 2 3 4 5 8 9 10 16 1a 6 11 12 17 18 19 20 7 13 14 15
Item was presented with response options ‘yes’ (coded as ‘1′) and ‘no’ (coded as ‘0′). Item was presented with response options ‘yes’ (coded as ‘1′), ‘no’ (coded as ‘0′), and ‘some are, some are not’ (coded as ‘2′). c Item was presented with space for a written in response and was coded in line with the numeric response provided. d Item was presented with response options ‘space dedicated to health activities’ (coded as ‘2′), ‘space shared with other health activities’ (coded as ‘1′), and ‘no space for health activities (coded as ‘0′). e Item was presented with response options ‘once a week’ (coded as ‘4′), ‘once a month’ (coded as ‘3′), ‘a few times a year’ (coded as ‘2′), ‘once a year’ (coded as ‘1′), and ‘never’ (coded as ‘0′). * Those who responded ‘yes’ to ‘Does your church have a health ministry?’ screened out of this item. a
b
Our findings cannot be extended beyond the current sample, and future research is needed to use this tool in other faith-based organization denominations to determine its usefulness. Despite these limitations, we believe the development of this tool is an important first step in capturing adaptation and institutionalization of health promotion interventions in community-based organizations. However, further work is needed to assess the usefulness of this measure in other settings.
5. Conclusions As the field of public health moves toward greater emphasis on the dissemination and implementation of evidence-based interventions, additional study is needed to determine how to fully integrate health promotion programs into their host settings. Churches vary in their ability to initiate and sustain health promotion initiatives and tools are needed to assess these activities in order to appropriately assist faithbased organizations interested in health promotion. The FBO-HII demonstrated its ability to capture levels of institutionalization in faithbased organizations.
4.2. Lessons learned The process of developing the FBO-HII measure confirmed the particular challenges when considering health promotion in settings outside of the healthcare system, including competing priorities and limited resources. When reviewing existing indicators of institutionalization for consideration in the measure, we learned that concepts that may be taken for granted in a healthcare setting, such as health-related policy, is not something that the churches would naturally codify. We learned that the term ‘institutionalization’, though central to this work, carries a negative connotation. Therefore, we adopted the more benign term of ‘integration’. We learned the difficulty of obtaining data from church leaders who are exceptionally busy and may not be accustomed to answering detailed questions about their organization. There can be sensitivity in asking about things like resource prioritization in these organizations. Further, there are serious limitations to statistical power when the unit of analysis is the organization and the leader can only provide only one observation. Though there may be benefits of obtaining multiple observations per organization, often times the pastor may be the only person who is knowledgeable enough to suitable answer organizational questions.
Ethics approval and consent to participate Research involving human participants: This work was approved by the Institutional Review Board. Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committees and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent: A waiver of written informed consent was approved by the Institutional Review Board. This article does not contain any studies with animals performed by any of the authors. Funding This project was supported by a grant from the American Cancer Society (RSG1602201CPPB); trial registration “Integration of Cancer Health Activities Into African American Churches”: NCT03178383 5
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
(registered 06/05/17). The funding agency played no role in the study design; collection, analysis or interpretation of data; writing the report; and decision to submit for publication.
prevention innovations: A sustainability planning model. Evaluation and Program Planning, 27(2), 135–149. https://doi.org/10.1016/j.evalprogplan.2004.01.002. Knott, C. L., Bowie, J., Mullins, C. D., Santos, S. L. Z., Slade, J., Rosenberg, E., ... Williams, R. M. (2020). An approach to adapting a community-based cancer control intervention to organizational context (In Press) Health Promotion Practice. Lancaster, K. J., Carter‐Edwards, L., Grilo, S., Shen, C., & Schoenthaler, A. M. (2014). Obesity interventions in African American faith‐based organizations: A systematic review. Obesity Reviews, 15, 159–176. Little, R. J. A. (1988). Missing-data adjustments in large surveys (with discussion). Journal of Business Economics and Statistics, 6, 287–301. Mercer, S. L., Banks, S. M., Verma, P., Fisher, J. S., Corso, L. C., & Carlson, V. (2014). Guiding the way to public health improvement: exploring the connections between The Community Guide’s evidence-based interventions and health department accreditation standards. Journal of Public Health Management and Practice, 20(1), 104–110. Morris, T. P., White, I. R., & Royston, P. (2014). Tuning multiple imputation by predictive mean matching and local residual draws. BMC Medical Research Methodology, 14(1), 75. National Cancer Institute (2019). Research-tested intervention programs. Retrieved June 6, 2019, fromhttps://rtips.cancer.gov/rtips/index.do. Osborne, J. W., Costello, A. B., & Kellow, J. T. (2008). Best practices in exploratory factor analysis. Best practices in quantitative methods, 86–99. Patterson, R. E., Kristal, A. R., Biener, L., Varnes, J., Feng, Z., Glanz, K., et al. (1998). Durability and diffusion of the nutrition intervention in the Working Well Trial: For the working well research Group. Preventive Medicine, 27(5), 668–673. Pluye, P., Potvin, L., & Denis, J. L. (2004). Making public health programs last: Conceptualizing sustainability. Evaluation and Program Planning, 27(2), 121–133. Qualtrics (2019). Qualtrics software. Utah: Provo. Rabin, B. A., & Brownson, R. C. (2017). Terminology for dissemination and implementation research. Dissemination and implementation research in health: Translating science to practice, 2, 19–45. Sahgal, N., & Smith, G. (2009). A religious portrait of african-americans. Retrieved from Pew Research Center website:https://www.pewforum.org/2009/01/30/a-religiousportrait-of-african-americans/. Scheirer, M. A. (1993). Are the level of institutionalization scales ready for" prime time"? A commentary on" development of level of institutionalization (LoIn) scales for health promotion programs. Health Education Quarterly, 20(2), 179–182. Sorensen, G., Thompson, B., Basen-Engquist, K., Abrams, D., Kuniyuki, A., DiClemente, C., & Biener, L. (1998). Durability, dissemination, and institutionalization of worksite tobacco control programs: Results from the Working Well Trial. International Journal of Behavioral Medicine, 5(4), 335–351. Story, W. T., LeBan, K., Altobelli, L. C., Gebrian, B., Hossain, J., Lewis, J., ... Shanklin, D. (2017). Institutionalizing community-focused maternal, newborn, and child health strategies to strengthen health systems: A new framework for the Sustainable Development Goal era. Globalization and Health, 13(1), 37. Tagai, E. K., Scheirer, M. A., Santos, S. L. Z., Haider, M., Bowie, J., Slade, J., ... Holt, C. L. (2018). Assessing capacity of faith-based organizations for health promotion activities. Health Promotion Practice, 19(5), 714–723. Van Buuren, S., & Groothuis-Oudshoorn, K. (2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. https://www. jstatsoft.org/v45/i03/. Weisbrod, R. R., Pirie, P. L., & Bracht, N. F. (1992). Impact of a community health promotion program on existing organizations: The Minnesota Heart Health Program. Social Science & Medicine, 34(6), 639–648. https://doi.org/10.1016/0277-9536(92) 90191-R. White, I. R., Daniel, R., & Royston, P. (2010). Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Computational Statistics & Data Analysis, 54, 2267–2275. Williams, R. M., Tagai, E. K., Santos, S. L. Z., Slade, J. L., Carter, R. L., & Holt, C. L. (2018). The role of leadership support in a church-based Cancer education implementation study. Journal of Religion and Health, 57(1), 146–156. Yin, R. K. (1979). Changing urban bureaucracies: How new practices become routinized. Lexington, Mass: Lexington Books.
Financial disclosure No financial disclosures were reported by the authors of this paper. The team has no actual or potential conflicts of interest to report. CRediT authorship contribution statement Randi M. Williams: Conceptualization, Data curation, Formal analysis, Writing - original draft, Writing - review & editing. Jing Zhang: Formal analysis, Writing - review & editing. Nathaniel Woodard: Visualization, Writing - review & editing. Jimmie Slade: Project administration, Writing - review & editing. Sherie Lou Zara Santos: Project administration, Writing - review & editing. Cheryl L. Knott: Supervision, Writing - review & editing. Declaration of Competing Interest The authors declare that they have no competing interests. Acknowledgments The team would like to acknowledge content expert, Mary Ann Scheirer (posthumous), as well as members of the Community Advisory Panel for their contributions to the development of the institutionalization measure. We are also grateful for our community partners, Barbarajean Shaneman-Robinson, Rhonda Boozer-Yeary, and Ralph Williams for their recruitment support. References Allen, J. D., Tom, L. S., Leyva, B., Rustan, S., Ospino, H., Negron, R., et al. (2015). Recruiting and surveying Catholic parishes for cancer control initiatives: Lessons learned from the CRUZA Implementation Study. Health Promotion Practice, 16(5), 667–676. Buller, P. F., & McEvoy, G. M. (1989). Determinants of the institutionalization of planned organizational change. Group & Organization Studies, 14(1), 33–50. Campbell, M. K., Hudson, M. A., Resnicow, K., Blakeney, N., Paxton, A., & Baskin, M. (2007). Church-based health promotion interventions: Evidence and lessons learned. Annual Review of Public Health, 28, 213–234. Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. Chen, E. K., Reid, M. C., Parker, S. J., & Pillemer, K. (2013). Tailoring evidence-based interventions for new populations: A method for program adaptation through community engagement. Evaluation & the Health Professions, 36(1), 73–92. DeHaven, M. J., Hunter, I. B., Wilder, L., Walton, J. W., & Berry, J. (2004). Health programs in faith-based organizations: are they effective? American Journal of Public Health, 94(6), 1030–1036. Drasgow, F. (1986). Encyclopedia of statistical sciences. Escoffery, C., Lebow-Skelley, E., Udelson, H., Böing, E. A., Wood, R., Fernandez, M. E., & Mullen, P. D. (2018). A scoping study of frameworks for adapting public health evidence-based interventions. Translational Behavioral Medicine, 9(1), 1–10. Estabrooks, P. A., Smith-Ray, R. L., Dzewaltowski, D. A., Dowdy, D., Lattimore, D., Rheaume, C., ... Wilcox, S. (2011). Sustainability of evidence-based community-based physical activity programs for older adults: Lessons from active for life. Translational Behavioral Medicine, 1(2), 208–215. Flórez, K. R., Payán, D. D., Palar, K., Williams, M. V., Katic, B., & Derose, K. P. (2019). Church-based interventions to address obesity among African Americans and Latinos in the United States: A systematic review. Nutrition Reviews. Goodman, P. S., & Dean, J. W. (1982). Creating long-term organizational change. In P. S. Goodman (Ed.). Change in organizations (pp. 226–279). San Francisco, CA: JosseyBass. Goodman, R. M., McLeroy, K. R., Steckler, A., & Hoyle, R. H. (1993). Development of level of institutionalization (LoIn) scales for health promotion programs. Health Education Quarterly, 20(2), 161–178. Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, I., & Vila-Abad, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity, 44(1), 153. Hou, S. I., & Cao, X. (2018). A systematic review of promising strategies of faith-based cancer education and lifestyle interventions among racial/ethnic minority groups. Journal of Cancer Education, 33(6), 1161–1175. Johnson, K., Hays, C., Center, H., & Daley, C. (2004). Building capacity and sustainable
Randi M. Williams, PhD, MPH, is a postdoctoral fellow at Georgetown University Medical Center’s Lombardi Comprehensive Cancer Center. Broadly, her interests include cancer control and prevention, informed decision-making, and addressing health disparities through community-based participatory research. She recently completed her dissertation research project which explored the decision-making processes as well as informational and support needs of African American smokers for lung cancer screening. Randi obtained her PhD in behavioral and community health from the School of Public Health, University of Maryland, College Park and worked in the CHAMP lab during her time in the program. Jing Zhang, PhD, is an assistant professor at the Department of Epidemiology and Biostatistics of the School of Public Health at the University of Maryland. She received a PhD in Biostatistics from the University of Minnesota in 2014. Dr. Zhang conducts research in Bayesian hierarchical methods, missing data analysis, meta-analysis, network meta-analysis, diagnostic tests and clinical trials. Her work has appeared in Clinical Trials, Statistics in Medicine, Statistical Methods in Medical Research, Research Synthesis Methods, and Journal of Statistical Software. Dr. Zhang serves as an associate editor for the Journal of Biopharmaceutical Statistics. Nathaniel Woodard, MPH, is a PhD student in the Department of Behavioral and Community Health. After completing his Bachelor of Arts degree in public health and
6
Evaluation and Program Planning 79 (2020) 101781
R.M. Williams, et al.
Bachelor of Arts degree in Psychology from The George Washington University while following a Pre-med path and minoring in both Biology and Sociocultural Anthropology. She served as the Program Manager for Project HEAL (Health through Early Awareness and Learning), a National Cancer Institute-funded project that identified the best way to implement evidence-based strategies for increasing cancer screening behaviors.
psychology at the University of Rochester, Nate began in the CHAMP Lab as a summer research intern. After completing his internship, he then transitioned into a graduate research assistant position with the lab. Previously, Nate has spent time working as a peer health educator, exercise and nutrition consultant, and a health promotions specialist. His research interests surround the effects of social interaction on health behaviors and motivations.
Cheryl L. Knott, PhD, is a Professor in the Department of Behavioral and Community Health and Co-leads the Population Science Program in the University of Maryland Greenebaum Cancer Center. She is founding Director of the Community Health Awareness, Messages, and Prevention research lab. Her dissemination/implementation research program looks at organizational factors that influence implementation outcomes, and she is currently evaluating a strategy to integrate or institutionalize evidencebased cancer education in African American churches. Dr. Knott's social epidemiology research program examines religious involvement and health among African Americans, in a longitudinal national sample of African Americans.
Jimmie Slade, MA, served his country for decades as a US Army officer and now continues to serve his community as Executive Director, Community Ministry of Prince George's County, a 38 year old Faith-based non-profit organization. He earned a BS degree from NC A&T State University, MA from Boston University, and is a graduate of the US Army War College. Sherie Lou Zara Santos, MPH, is a Certified Health Education Specialist who earned her dual MPH at the Loma Linda University in Loma Linda, California with concentrations in both Health Promotion Education and Maternal and Child Health. She received her
7