- Email: [email protected]
Rethinking measurement of neighborhood in the context of health research
Social Science & Medicine 71 (2010) 651e656
Contents lists available at ScienceDirect
Social Science & Medicine journal homepage: www.elsevier.com/locate/socscimed
Short report
Rethinking measurement of neighborhood in the context of health research Nicole Schaefer-McDaniel a, *, James R. Dunn a, b, Nadia Minian c, Danielle Katz d a
Centre for Research on Inner City Health, St. Michael’s Hospital, Toronto, Ontario, Canada McMaster University, Hamilton, Ontario, Canada c Echo: Improving Women’s Health, Toronto, Ontario, Canada d York University, Canada b
a r t i c l e i n f o
a b s t r a c t
Article history: Available online 2 June 2010
Systematic social observations have been gaining increasing recognition in neighborhood and health research as a way of measuring neighborhood attributes hypothesized to affect residents’ well-being. Despite the growing popularity of this methodology, there has not been a critical discussion of potential shortcomings of this approach. This paper reviews some of the challenges and limitations in the systematic social observations methodology. We especially differentiate between limitations related to the methodology itself and challenges the approach presents for researchers in the field. We conclude by offering directions for future research utilizing this technique. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Neighborhood Health Observation Systematic social observation (SSO) Ethics Methods Reflection Review
Introduction The past decade has seen a growing body of research linking neighborhood characteristics to a number of residents’ outcomes including their health and well-being. Scholars particularly embrace this body of research because findings can be useful for guiding social policy, prevention, and intervention efforts to improve health conditions (Duncan & Raudenbush, 1999) and because, in some cases, neighborhood-based interventions are believed to be more cost-effective than individual or familycentered ones (Sastry, Ghosh-Dastidar, Adams, & Pebley, 2006). However, despite the growing interest in neighborhood studies, relatively little is currently known about the measurement of attributes of neighborhoods salient to health (Caughy, O’Campo, & Patterson, 2001; Weich et al., 2001). In North America, researchers typically rely on census data to measure neighborhood attributes although scholars criticize this approach mainly due to the paucity of theoretical rationales justifying this technique and due to arbitrary neighborhood boundary definitions (Coulton, Korbin, Chan, & Su, 2001; Leventhal & BrooksGunn, 2000; Rajaratnam, Burke, & O’Campo, 2006). Some also believe that census data provide an oversimplified picture of neighborhood life because these data are aggregated to block or
* Corresponding author. Tel.: þ1 416 864 6060. E-mail address: [email protected] (N. Schaefer-McDaniel). 0277-9536/$ e see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.socscimed.2010.03.060
tract levels and this aggregation ignores important variability within neighborhoods (Caughy, O’Campo, & Brodsky, 1999). Moreover, census data are only collected every ten years in the US and the UK (five years in Canada) but the inner-city neighborhoods that are usually of specific interest to researchers and policy makers have been known to change very quickly therefore raising questions about the validity of utilizing dated census information (McGuire, 1997). Further, it is important to distinguish between measuring a neighborhood’s composition and context. Census data measure residents’ characteristics such as income, poverty, and education levels that are summarized for a particular area. So in reality, census data provide a compositional estimate of a neighborhood but do not provide information on the actual neighborhood context such as its infrastructure; presence or absence of institutions, resources, and facilities; or level of cleanliness (Dunstan et al., 2005; Laraia et al., 2006; Macintyre, Ellaway, & Cummins, 2002). Researchers in the United Kingdom commonly make use of administrative data to compute an index of multiple deprivation to measure neighborhood characteristics (Noble et al., 2007). This index is calculated by weighting area scores across the following domains: income deprivation, employment deprivation, health deprivation and disability, education skills and training deprivation, barriers to housing and services, living environment deprivation, and crime (Noble et al., 2007). This index has been linked to health behaviors and outcomes as divergent as physical activity, healthy eating, cognitive functioning, mobility disability, and atherosclerosis
652
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
(Amuzu, Carson, Watt, Lawlor, & Ebrahim, 2009; Deans et al., 2009; Jones, Hilldon, & Coombes, 2009; Lang et al., 2008). In order to address shortcomings associated with census and administrative data to measure neighborhood attributes, systematic social observations (SSO) of neighborhoods by trained outside raters have become a popular alternative research strategy for researchers wishing to study neighborhood attributes (Sampson & Raudenbush, 1999, 2004; Zenk et al., 2007). In this methodology, trained researchers assess physical and social attributes of particular blocks of interest in a given neighborhood. This method differs from utilizing census and other administrative data as it places researchers directly in the studied environments. For example, researchers can observe and rate the degree of cleanliness, upkeep of buildings, and people’s interactions and behaviors. In Chicago, Sampson and Raudenbush (1999, 2004) utilized this methodology to measure social and physical disorder. The former refers to aspects of social incivilities such as vandalism, gang activity, or drug behavior while the latter is an indicator of the degree of cleanliness of buildings, sidewalks, and streets. In Baltimore, research by Caughy et al. (2001) supports that in addition to aspects of physical incivilities or disorder, SSO can also measure residents’ territorial behaviors as well as the presence of play areas for children. Dunstan et al. (2005) used neighborhood observations to measure natural elements such as the presence of trees, gardens, planted vegetation, and green spaces. Research by McGuire (1997) revealed that SSO can even assess a block’s level of social cohesion and Laraia et al. (2006) observed the availability of social spaces in Raleigh, North Carolina. As these examples illustrate, SSO aims to be an objective method and does not purport to measure subjective or individualized concepts such as sense of community or social capital. Instead, SSO uses visual cues of neighborhood structure and functioning as a way of collecting information and thus primarily supplies quantitative data. While quantitative studies examining the effects of neighborhood social capital on residents’ health have prospered in recent years (Boyce, Davies, Gallupe, & Shelley, 2008; Franzini, Caughy, Spears, & Fernandez Esquer, 2005; Kruger, Reischl, & Gee, 2007; Lofors & Sundquist, 2007; Rojas & Carlson, 2006; Veenstra et al., 2005), researchers wishing to explore the meanings that residents ascribe to concepts such as “neighborhood” and “community” should also consider making use of qualitative methods. For example, Mannarini and Fedi (2009) used qualitative interviews to describe general themes across participants’ understanding of “community” and “sense of community.” They found that participants’ definitions generally overlapped with those proposed by the research literature, namely that community is both symbolic and pragmatic, a real physical entity as well as a relational, interpersonal space. Although the SSO methodology is gaining momentum in neighborhood and health research, there has not been a critical discussion of potential shortcomings of this approach. Therefore, the purpose of this commentary is to illustrate some of the challenges and limitations in the SSO methodology. We do so by drawing on our own experiences of conducting SSO in New York City and Toronto (Schaefer-McDaniel, 2007; Parsons et al., in press) and differentiate between limitations related to the methodology itself and challenges the approach presents for researchers in the field. We conclude by offering directions for future research utilizing this technique. Methodological issues
Roosa, Jones, Tein, & Cree, 2003; Weiss, Ompad, Galea, & Vlahov, 2007). For example, some researchers define neighborhoods in terms of entire census block groups or tracts while others rely on city specifications or on residents’ definitions. Others rely on geographic informational systems (GIS), which allow researchers to define neighborhoods as a certain radius around individual households so that every house is associated with a slightly different area defined as a particular neighborhood (e.g., Larson et al., 2009). Essentially, all of these geographic boundaries constitute ‘neighborhoods’ but this has led to a growing body of neighborhood research utilizing inconsistent boundaries since neighborhood spatial definitions affect the areas to be observed. For example, researchers who equate census block groups with neighborhoods may observe all blocks in a particular block group while others may rely on residents’ subjective spatial definitions of neighborhoods, which are often much smaller areas (see Coulton et al., 2001), and consequently only conduct observations for these blocks. This inconsistency has implications for comparisons between neighborhood studies. Further complicating the picture are researchers who only observe a random selection of blocks within a particular neighborhood. For example, Caughy et al. (2001) studied 57 neighborhoods in Baltimore (a neighborhood corresponded to a census block group) but only observed between 3 and 51 blocks for each neighborhood. While they sampled their observed blocks systematically, this technique is problematic because it assumes that observed blocks are representative of the larger neighborhood when data are aggregated to the neighborhood level. However, our observations suggest that this may not be the case since we noticed much variation in terms of physical and social attributes of blocks even for those in close proximity (Schaefer-McDaniel, 2007). While inconsistent boundary definitions can pose problems to comparing research findings across settings and cities, there are many possible ways of spatially defining a neighborhood as explained above. Choice of neighborhood boundaries and geographic scale should be theoretically driven, that is they should be specific to the outcomes of interest or contexts in which observations take place. For example, census boundaries can be used when SSO measurements are being compared to other censusbased data, such as demographics, while resident definitions can be used when other subjective resident information is being surveyed, such as level of fear felt within the neighborhood. Timing of observations Another issue affecting the SSO methodology relates to the timing of observations since social neighborhood attributes are often time-varying. For example, consider a young adult walking down the block while drinking a beer. Whether or not a rater will witness this “social disorder” taking place completely depends on the rater and the young adult being on the block during the same exact few minutes it takes the young adult to walk the distance. Furthermore, observations (and neighborhood life in general) are likely to differ substantially between day and night-time in some neighborhoods. In order to gain a complete picture of neighborhood life, observations should be conducted at multiple time points during the day, evening, and night-time (see Sampson & Raudenbush, 1999, 2004). However, night-time observations also raise potential safety issues, so some researchers might find themselves having to trade off between assuring data completeness and rater safety.
Defining neighborhood boundaries There has been much discussion in the literature related to the many spatial definitions neighborhoods can take (Nicotera, 2007;
Bias towards the visible and readily identifiable Despite the fact that some SSO measures such as the one used in the Los Angeles Family and Neighborhood Survey (Peterson, Sastry,
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
& Pebley, 2007) are quite comprehensive, this methodology can only provide information on attributes that can be observed directly. SSO cannot provide a complete picture of neighborhood life since many important aspects such as the degree of social connectedness among residents and neighborhood social capital as well as other social processes are not directly observable and thus not measurable through SSO. Further, the SSO methodology is biased towards measuring attributes that can be readily identifiable. For example, a common observation item is concerned with the presence of gang activity and gang members. This means that raters must be able to readily identify gang members but we learned in our observations that this is almost an impossible task. While we were sometimes aware of clear signs of gang presence (for example, a group of young men wearing the same colored bandana), there were other instances in which we could not get a good look to make a definite decision. This is especially complicated by raters’ status as outsiders of neighborhoods (clearly identified as such with clipboards in their hands) since gang affiliation may then be purposefully hidden. It is also possible that the mere act of observing a phenomenon can alter it (i.e., a Hawthorne effect, see Stocking, 1985). For example, some raters reported that young adults who were hanging out on the blocks would leave once they acknowledged the rater’s presence. So, outsiders observing an unfamiliar neighborhood may identify or perceive attributes differently than insiders. While insiders might identify some characteristics more accurately, relying on insider report also raises challenges related to standardizing SSO. One potential option to address problems in visibility is to supplement SSO with qualitative research based on resident interviews. For example, in a study examining families who had moved from areas of high to low poverty concentration, Keels (2008) conducted in-depth qualitative interviews with mothers to gage, among other measures, families’ access to neighborhood social capital. This type of information is most likely too nuanced to be captured through SSO alone. Similarly, Antony and Nicotera (2008) used a mixed methods approach to study the relationship between neighborhood structural resources and youth experience of daily “hassles” or environmentally related irritations, such as the presence of gangs or the lack of amenities. The researchers obtained qualitative information from the participants on their general experiences of neighborhood cohesion and feelings towards the community, as well as quantitative information regarding neighborhood structural resources and hassling experiences (Antony & Nicotera, 2008). The use of qualitative methods provided insight into additional types of neighborhood resources experienced by youths outside of the structural resources measured, such as the youths’ neighbors, as well as more detailed information on how social disorder within the neighborhood becomes a source of hassling for young people. “Disorder” relative to what? Another bias in the SSO methodology relates to the point of reference of observations. For example, researchers commonly observe the presence of groups of adults or young people loitering or hanging out on the street. For outside researchers, this is commonly considered an aspect of social disorder (see Sampson & Raudenbush, 1999, 2004). However, groups of people hanging out on the street may also symbolize close-knit neighborhoods in areas where neighbors interact and socialize with one another (see Klinenberg, 2003). Alternatively, neighborhoods that appear highly ordered and controlled may suppress outward signs of disorder that are important for health outcomes or may signal that a third factor is related to disorder and health (Dunn, Schaefer-McDaniel, & Ramsay, 2009). These issues raise concerns in terms of the validity
653
of SSO if the tool is developed without attention to the cultural norms, values, and behaviors of residents and neighborhoods of interest. This bias reflects the issue of utilizing outside raters as compared to insiders of study neighborhoods as previously discussed. Challenges for researchers in the field Observer safety and comfort levels Aside from methodological challenges, the SSO technique presents challenges for researchers implementing this approach in the field. Once reliability between raters has been established, it is generally the norm that observations are conducted solely by one rater. Raters may then find themselves alone in an unfamiliar part of town due to the “outsider” status requirement for the observation. This can raise serious safety concerns, particularly for young women (see Kenyon & Hawker, 1999; Sharp & Kremer, 2006), and especially considering that the point of some of these observations is to observe illegal behaviors (e.g., drug use, gang activity, vandalism). While having raters work in pairs would substantially increase their safety, this will also greatly increase the costs of conducting observations. Although we did not experience serious incidences related to safety concerns during the course of our observations, there were instances in which some of our raters felt uncomfortable, especially on blocks that were isolated from commercial and pedestrian activity and those that were severely abandoned. Further, in thinking about raters’ comfort levels, it is also important to recognize the time of day and season. Observing neighborhoods during the summer months means that more people will be outside enjoying the warm weather and that more activity can be observed. But this also has consequences for raters. For example, our observations in New York City were conducted during July and August and the hot, humid summer days and affected raters’ morale and energy levels (Schaefer-McDaniel, 2007). Ambiguous ethics of naturalistic observations Since raters observe occurrences during ordinary everyday life and observed people will not be identified, this research is considered “minimal risk,” meaning that raters do not need the consent of the people they are observing (Tri Council Policy Statement, 1998). However, this can raise questions in situations in which raters may feel they should intervene. For example, some raters observed several instances of parents loudly verbally abusing their children. In one occasion, a rater even witnessed a mother publicly hitting her young child but none of these interactions were met with any reaction from other people on the block who clearly witnessed these exchanges. In New York City, one of our raters observed children playing in the water of a fire hydrant that had been (illegally) opened in the hot weather. One young girl ran away from the water and pulled down her pants to urinate in a plastic cup, from which she then began to drink. Much to the rater’s surprise, the many onlookers started to laugh (thus recognizing the young girl’s actions) but did not interfere with the situation. Situations such as these raise serious questions in terms of ethical obligations raters have for their objects of study in naturalistic research. The ethical concerns regarding SSO rater responsibility call for the creation of a set of agreed upon guidelines. These guidelines should weigh rater role and safety with social responsibility, and make clear raters’ legal responsibilities, such as reporting suspected child abuse. The creation of ethical guidelines is no small task and will require initiative and input from major theorists and
654
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
researchers both within and outside the field of neighborhood and health research. Recommendations for neighborhood observations Future directions for neighborhood observations Given these limitations and challenges of the SSO methodology for measuring characteristics of the neighborhood context, we provide research directions for scholars utilizing this approach in the future to improve the methodology and minimize potential biases. Specifically, we recommend that 1) more attention needs to be paid to theories linking neighborhood attributes to health outcomes, 2) formative research needs to take place to develop SSO tools and the methodology itself for use in health research, and 3) more dialogue between researchers is warranted to openly discuss challenges surrounding the SSO methodology and raters’ experiences in the field. Apply and develop theories to examine neighborhoodehealth relations Neighborhood research, and the SSO methodology in particular, has its foundation in the sociology of crime. Over the years, some researchers examining neighborhood effects have replaced crime outcomes with a health measure rather than applying or developing theoretical frameworks to link neighborhood attributes to residents’ health and well-being. Generally, such studies have not been as useful for policy makers or intervention efforts. For instance, Cohen et al. (2000) summarized observations of New Orleans neighborhoods into a ‘broken windows index’ which consisted of measures on housing quality, presence of abandoned cars, graffiti, and trash. They then linked this index to gonorrhea rates showing that areas measuring high on their broken windows index were also areas in which residents had higher rates of gonorrhea. While this link is interesting, it does not help policy makers untangle whether physical deterioration is an indicator of or a risk factor for gonorrhea. Similarly, Weich et al. (2001) linked neighborhood observations to adult depression in London, England and found that depression was more prevalent in areas where residential properties had deck access, there were newer properties, fewer properties with private gardens, shared recreational space, and many patches of graffiti present. Policy makers might then wonder whether they should regulate buildings with deck access and public recreational spaces in the hopes of ameliorating residents’ mental health. These examples also highlight the fact that researchers have the capacity to conduct their research in a way that is potentially quite harmful (and unfairly so). Studies such as the ones described can misrepresent and stigmatize a community as no actionable message may come from the research. Aside from being conscientious about the potentially harmful effect research could have, we also advocate that scholars apply and develop existing theories linking neighborhood attributes to health outcomes rather than examining uni-directional relationships between neighborhood conditions and health outcomes (see Ellen & Turner, 1997; Macintyre et al., 2002; Saegert & Evans, 2003; Sampson, Morenoff, & Gannon-Rowley, 2002; Wandersman & Nation, 1998). For example, Wandersman and Nation propose three models in which mediating variables link neighborhood context to public health outcomes: 1) the structural characteristics model states that the level of social organization in the neighborhood such as social control and common values among residents along with psychological stress mediate the relationship between structural neighborhood characteristics (i.e., socioeconomic status [SES], racial composition) and residents’ mental health; 2) the environmental
stress model postulates that environmental stressors such as noise or crowding affects residents’ health indirectly through unsuccessful coping strategies; and 3) the neighborhood disorder model suggests that neighborhood characteristics, specifically social and physical disorder, directly relate to residents’ mental health so that residents in neighborhoods characterized by the presence of gangs, violence, abandoned buildings, and litter, have poorer mental health. Wandersman and Nation further suggest that the neighborhood disorder model may also work through residents’ fear of crime and victimization (i.e., neighborhood disorder is linked to fear of crime, which is linked to poor mental health, see also Skogan, 1990). Since these theories may not hold in all contexts, we encourage researchers to undertake formative research to introduce new theories linking neighborhood attributes to health outcomes. We particularly applaud efforts that actively involve residents of target neighborhoods. For instance, O’Campo, Salmon, and Burke (2009) conducted a series of concept mapping exercises with Toronto residents to uncover their perceptions of neighborhood attributes that relate to poor mental health outcomes. In the final activity, participants were asked to develop pathways that connect neighborhood characteristics and mental health. The researchers then translated these pathways into a theoretical framework linking neighborhood attributes to mental health outcomes. Such formative research is particularly valuable because it not only allows researchers to ground their research in theory but also gives them a chance to tailor SSO tools to specific health outcomes of interest. Conduct formative research to develop SSO tools and methodology Aside from conducting formative research to develop theories linking neighborhood attributes to health outcomes, we also emphasize the need for such research to develop SSO tools and the methodology itself. As discussed, research examining the SSO methodology needs to address the spatial neighborhood boundary issue by examining, for instance, whether different levels of aggregation of SSO data have similar or divergent effects on health outcomes. Moreover, SSO collects information on time-sensitive events so it is important for researchers to conduct multiple observations of blocks of interest. However, this also raises grounds for future empirical work to systematically examine not only how many observations need to be conducted per block to yield reliable estimates but also how factors such as time of day and seasonal variations affect ratings. Such explorations would help researchers understand place effects more concretely. We also emphasize a need for formative research to develop SSO tools. Here, we applaud the efforts of researchers who actively involve residents and expert panels to modify existing SSO tools for their needs (see Hoehner, Ramirez, Elliott, Handy, & Brownson, 2005; Pikora et al., 2002; Zenk et al., 2007). Involving insiders in SSO tool development has the additional benefit of addressing some of the insider/outsider issues we previously raised. Lastly, we encourage researchers to more closely examine psychometric properties of their SSO tools. In a previous systematic review (Schaefer-McDaniel, Caughy, O’Campo, & Gearey, 2010), we find that few researchers make use of data reduction procedures such as factor analysis to examine underlying scales and concepts that SSO tools can measure. Along similar lines, the review finds great variability in terms of reported inter-rater reliability statistics. The absence of norms regarding sound psychometric evaluations of SSO tools potentially compromises the reliability and validity of observational measures. Create a space for researchers to discuss benefits and challenges As neighborhood and health research and studies utilizing SSO are gaining increasing popularity, we recommend that researchers
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
openly discuss and reflect on their experiences with SSO tools and methods. Since surprisingly little methodological detail is provided in published studies that utilize SSO methods (Schaefer-McDaniel, Caughy, O’Campo, & Gearey, 2010), we encourage researchers to be more explicit and reflective about their neighborhood research and SSO methodologies in particular. This space can be created via conferences or special editions in academic journals dedicated to the topic and would allow researchers to openly debate and discuss their experiences, challenges, and recommendations. In addition to discussions around methodological issues, we also emphasize a need for researchers to examine raters’ experiences around safety and unforeseen ethical dilemmas as illustrated in our examples. While the latter point raises numerous questions beyond the scope of this article in terms of researchers’ responsibilities and accountability in naturalistic research, it also highlights the need for raters to have a supportive research community with which they can discuss their experiences and perceived obligations. Present-day solutions for SSO challenges While we discussed future directions that SSO research should take, we recognize that such change takes time, and that it is unreasonable to suggest that research using SSO halt until the recommended research is completed. We thus offer more immediate research directions that can be addressed in the meantime to better understand both the potential and the limitations of SSO use. The challenge regarding neighborhood measurement is that different studies will naturally use the means of creating neighborhood boundaries that best suits their particular research. We therefore do not promote a standard neighborhood definition; rather, this challenge is more a problem of comparison rather than validity (see also Schaefer-McDaniel, Caughy, O’Campo, & Geary, 2010). To shed more light on the magnitude of this issue, we recommend a systematic examination of similarities and differences in SSO and neighborhood effects on health by different observational boundaries. Until such research is conducted, we suggest that scholars be explicit about choosing specific boundary measurements and discuss whether and how their research might be applicable to other types of neighborhood definitions. Likewise, the timing of SSO ratings requires additional research, such as a systematic review examining the timing of observations, the effects of time of day (e.g., morning vs. evening), type of day (weekday vs. weekend or holiday), and season (e.g., winter vs. summer) on observation and health outcomes. In addition, a systematic analysis examining differences and similarities between inside (i.e., residents of study neighborhoods) and outside raters would greatly add to the SSO literature. Conclusion We conclude by emphasizing that neighborhood research has the potential to lead to action (for example, in the form of policy changes, initiation of new interventions). However, in order for research to lead to constructive change, it is not enough to merely discover a correlation between neighborhood characteristics and health, nor is it helpful to assume that such a correlation represents a direct causal link. Current neighborhood research benefits most strongly from studies examining the mediating variables between the qualities of an environment and the health of its citizens. Rather than observing and reporting an association between neighborhood characteristics and health outcomes without additional explanation, current research is at its most effective when researchers recognize their responsibility to use language and report results in a manner that avoids sensationalism and informs without stigmatizing certain groups or neighborhoods.
655
We hope that this commentary is the beginning of an ongoing dialogue between researchers and stakeholders engaged in neighborhood and health research to develop theory, research, and methodology (such as the use of SSO tools) in order to move the field forward. As discussed, we view that discussions and formative research around methodological issues in SSO tools are necessary to shed insight into the complex relationships between neighborhoods and residents’ health outcomes and to guide prevention and intervention efforts.
References Amuzu, A., Carson, C., Watt, H. C., Lawlor, D. A., & Ebrahim, S. (2009). Influence of area and individual lifecourse deprivation on health behaviours: findings from the British Women's Heart and Health Study. European Journal of Cardiovascular Prevention & Rehabilitation, 16(2), 169e173. Antony, E. K., & Nicotera, N. (2008). Youth perceptions of neighborhood hassles and resources: a mixed method analysis. Children and Youth Services Review, 30(11), 1246e1255. Boyce, D., Davies, D., Gallupe, O., & Shelley, D. (2008). Adolescent risk taking, neighborhood social capital, and health. Journal of Adolescent Health, 43, 246e252. Caughy, M. O., O’Campo, P., & Brodsky, A. E. (1999). Neighborhoods, families, and children: implications for policy and practice. Journal of Community Psychology, 27, 615e633. Caughy, M., O’Campo, P., & Patterson, J. (2001). A brief observational measure for urban neighbourhoods. Health and Place, 7, 225e236. Cohen, D. A., Spear, S., Scribner, R., Mason, K., Kissinger, P., & Wildgen, J. (2000). Broken windows and the risk of gonorrhea. American Journal of Public Health, 90, 230e236. Coulton, C. J., Korbin, J., Chan, T., & Su, M. (2001). Mapping residents’ perceptions of neighborhood boundaries: a methodological note. American Journal of Community Psychology, 29(2), 371e383. Deans, Kevin A., Bezlyak, Vladimir, Ford, Ian, Batty, G. David, Burns, Harry, Cavanagh, Jonathan, et al. (2009). Differences in atherosclerosis according to area level socioeconomic deprivation: cross sectional, population based study. BMJ, 339, b4710. Duncan, G. J., & Raudenbush, S. W. (1999). Assessing the effects of context in studies of child and youth development. Educational Psychologist, 34(1), 29e41. Dunn, J. R., Schaefer-McDaniel, N. J., & Ramsay, J. (2009). Neighborhood chaos and children's development: questions and contradictions. In G. W. Evans, & T. D. Wachs (Eds.), Chaos and its influence on children's development: An ecological perspective (pp. 173e189). Washington DC: American Psychological Association Press. Dunstan, F., Weaver, N., Araya, R., Bell, T., Lannon, S., Lewis, G., et al. (2005). An observation tool to assist with the assessment of urban residential environments. Journal of Environmental Psychology, 25, 293e305. Ellen, I. G., & Turner, M. A. (1997). Does neighborhood matter? Assessing recent evidence. Housing Policy Debate, 8, 833e866. Franzini, L., Caughy, M. O. B., Spears, W., & Fernandez Esquer, M. E. (2005). Neighborhood economic conditions, social processes, and self-rated health: A multilevel latent variables model. Social Science & Medicine, 61, 1135e1150. Hoehner, C. M., Ramirez, L. K. B., Elliott, M. B., Handy, S. L., & Brownson, R. C. (2005). Perceived and objective environmental measures and physical activity among urban adults. American Journal of Preventive Medicine, 28(2S2), 105e116. Jones, A., Hilldon, M., & Coombes, E. (2009). Greenspace access, use, and physical activity: understanding the effects of area deprivation. Preventive Medicine, 49, 500e505. Keels, M. (2008). Neighborhood effects examined through the lens of residential mobility programs. American Journal of Community Psychology, 42, 235e250. Kenyon, E., & Hawker, S. (1999). ‘Once would be enough’: some reflections on the issue of safety for lone researchers. International Journal of Social Research Methodology: Theory and Practice, 2(4), 313e327. Klinenberg, E. (2003). Heat wave: A social autopsy of disaster in Chicago. Chicago: University of Chicago Press. Kruger, D. J., Reischl, T. M., & Gee, G. C. (2007). Neighborhood social conditions mediate the association between physical deterioration and mental health. American Journal of Community Psychology, 40, 261e267. Lang, I. A., Llewellyn, D. J., Langa, K. M., Wallace, R. B., Huppert, F. A., & Melzer, D. (2008). Neighborhood deprivation, individual socioeconomic status, and cognitive function in older people: analyses from the English Longitudinal Study of Ageing. Journal of American Geriatric Society, 56(2), 191e198. Laraia, B. A., Messer, L., Kaufman, J. S., Dole, N., Caughy, M., O’Campo, P., et al. (2006). Direct observation of neighbourhood attributes in an urban area of the US south: characterizing the social context of pregnancy. International Journal of Health Geographics, 5(11), 1e11. Larson, K., Gilliland, J., Hess, P., Tucker, P., Irwin, J., & He, M. (2009). The influence of physical environment and sociodemographic characteristics on children’s mode of travel to and from school. American Journal of Public Health, 99, 520e526.
656
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
Leventhal, T., & Brooks-Gunn, J. (2000). The neighborhoods they live in: the effects of neighborhood residence on child and adolescent outcomes. Psychological Bulletin, 126(2), 309e337. Lofors, J., & Sundquist, K. (2007). Low-linking social capital as a predictor of mental disorders: a cohort study of 4.5 million Swedes. Social Science & Medicine, 64, 21e34. Macintyre, S., Ellaway, A., & Cummins, S. (2002). Place effects on health: how can we conceptualise, operationalise and measure them? Social Science & Medicine, 55, 25e139. Mannarini, T., & Fedi, A. (2009). Multiple senses of community: the experience and meaning of community. Journal of Community Psychology, 37(2), 211e227. McGuire, J. M. (1997). The reliability and validity of a questionnaire describing neighborhood characteristics relevant to families and young children living in urban areas. Journal of Community Psychology, 25(6), 551e566. Nicotera, N. (2007). Measuring neighborhood: a conundrum for human services researchers and practitioners. American Journal of Community Psychology, 40 (1e2), 26e51. Noble, M., Mclennan, D., Wilkinson, K., Whitworth, A., Exley, S., Barnes, H., et al. (2007). The English indices of deprivation 2007. Department of Communities and Local Government. O’Campo, P., Salmon, C., & Burke, J. G. (2009). Neighbourhoods and mental wellbeing: what are the pathways? Health & Place, 15, 56e68. Parsons, J., Singh, G., Scott, A., Nisenbaum, R., Balasubramaniam, P., Jabbar, A., et al. Standardized observation of neighborhood disorder: does it work in Canada? International Journal of Health Geographics, in press. Peterson, C. E., Sastry, N., & Pebley, A. R. (2007). The Los Angeles Family And Neighborhood survey neighborhood observations codebook. Rand corporation. Available at: http://www.rand.org/pubs/working_papers/WR240.13/. Pikora, T. J., Bull, F. C. L., Jamrozik, K., Knuiman, M., Giles-Corti, B., & Donovan, R. J. (2002). Developing a reliable audit instrument to measure the physical environment for physical activity. American Journal of Preventive Medicine, 23(3), 187e194. Rajaratnam, J. K., Burke, J. G., & O’Campo, P. (2006). Maternal and child health and neighborhood context: the selection and construction of area-level variables. Health & Place, 12, 547e556. Rojas, Y., & Carlson, P. (2006). The stratification of social capital and its consequences for self-rated health in Taganrog, Russia. Social Science & Medicine, 62, 2732e2741. Roosa, M. W., Jones, S., Tein, J., & Cree, W. (2003). Prevention science and neighborhood influences on low-income children’s development: theoretical and methodological issues. American Journal of Community Psychology, 31, 91e120. Saegert, S., & Evans, G. W. (2003). Poverty, housing niches, and health in the United States. Journal of Social Issues, 59, 569e589.
Sampson, R. J., & Raudenbush, S. W. (1999). Systemic social observation of public spaces: a new look at disorder in urban neighborhoods. American Journal of Sociology, 105(3), 603e651. Sampson, R. J., Morenoff, J. D., & Gannon-Rowley, T. (2002). Assessing neighborhood effects: social processes and new directions in research. Annual Review of Sociology, 28, 443e478. Sampson, R. J., & Raudenbush, S. W. (2004). Seeing disorder: neighborhood stigma and the social construction of “broken windows”. Social Psychology Quarterly, 67, 319e342. Sastry, N., Ghosh-Dastidar, B., Adams, J., & Pebley, A. (2006). The design of a multilevel survey of children, families and communities: the Los Angeles Family and Neighborhood survey. Social Science Research, 35(4), 803e1134. Schaefer-McDaniel, N. (2007). Children talk about their New York city neighborhoods: the role of subjective and objective neighborhood evaluations in understanding child health. Saarbrücken, Germany: VDM Verlag Dr. Müller. Schaefer-McDaniel, N. J., Caughy, M., O'Campo, P., & Gearey, W. (2010). Examining methodological details of neighborhood observations and the relationship to health: A literature review. Social Science & Medicine, 70, 277e292. Sharp, G., & Kremer, E. (2006). The safety dance: confronting harassment, intimidation, and violence in the field. Sociological Methodology, 36(1), 317e327. Skogan, W. (1990). Disorder and decline: Crime and the spiral of decay in American Neighborhoods. Berkley and Los Angeles, CA: University of California Press. Stocking, G. W. (1985). Observers observed: Essays on ethnographic fieldwork. Madison, WI: University of Wisconsin Press. Tri Council Policy Statement. (1998). Canadian institutes of health research, natural sciences and engineering research council of Canada, Social sciences and humanities research council of Canada, Tri-council policy statement: Ethical conduct for research involving humans (with 2000, 2002, 2005 amendments). Veenstra, G., Luginaah, I., Wakefield, S., Birch, S., Eyles, J., & Elliot, S. (2005). Who you know, where you live: social capital, neighbourhood and health. Social Science & Medicine, 60, 2799e2818. Wandersman, A., & Nation, M. (1998). Urban neighborhoods and mental health: psychological contributions to understanding toxicity, resilience, and interventions. American Psychologist, 53(6), 647e656. Weich, S., Burton, E., Blanchard, M., Prince, M., Sproston, K., & Erens, B. (2001). Measuring the built environment: validity of a site survey instrument for use in urban settings. Health & Place, 7, 283e292. Weiss, L., Ompad, D., Galea, S., & Vlahov, D. (2007). Defining neighborhood boundaries for urban health research. American Journal of Preventive Medicine, 22(6S), S154eS159. Zenk, S. N., Schulz, A. J., Mentz, G., House, J. S., Gravlee, C. C., Miranda, P. Y., et al. (2007). Inter-rater and testeretest reliability: methods and results for the neighborhood observational checklist. Health & Place, 13, 452e465.
Contents lists available at ScienceDirect
Social Science & Medicine journal homepage: www.elsevier.com/locate/socscimed
Short report
Rethinking measurement of neighborhood in the context of health research Nicole Schaefer-McDaniel a, *, James R. Dunn a, b, Nadia Minian c, Danielle Katz d a
Centre for Research on Inner City Health, St. Michael’s Hospital, Toronto, Ontario, Canada McMaster University, Hamilton, Ontario, Canada c Echo: Improving Women’s Health, Toronto, Ontario, Canada d York University, Canada b
a r t i c l e i n f o
a b s t r a c t
Article history: Available online 2 June 2010
Systematic social observations have been gaining increasing recognition in neighborhood and health research as a way of measuring neighborhood attributes hypothesized to affect residents’ well-being. Despite the growing popularity of this methodology, there has not been a critical discussion of potential shortcomings of this approach. This paper reviews some of the challenges and limitations in the systematic social observations methodology. We especially differentiate between limitations related to the methodology itself and challenges the approach presents for researchers in the field. We conclude by offering directions for future research utilizing this technique. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Neighborhood Health Observation Systematic social observation (SSO) Ethics Methods Reflection Review
Introduction The past decade has seen a growing body of research linking neighborhood characteristics to a number of residents’ outcomes including their health and well-being. Scholars particularly embrace this body of research because findings can be useful for guiding social policy, prevention, and intervention efforts to improve health conditions (Duncan & Raudenbush, 1999) and because, in some cases, neighborhood-based interventions are believed to be more cost-effective than individual or familycentered ones (Sastry, Ghosh-Dastidar, Adams, & Pebley, 2006). However, despite the growing interest in neighborhood studies, relatively little is currently known about the measurement of attributes of neighborhoods salient to health (Caughy, O’Campo, & Patterson, 2001; Weich et al., 2001). In North America, researchers typically rely on census data to measure neighborhood attributes although scholars criticize this approach mainly due to the paucity of theoretical rationales justifying this technique and due to arbitrary neighborhood boundary definitions (Coulton, Korbin, Chan, & Su, 2001; Leventhal & BrooksGunn, 2000; Rajaratnam, Burke, & O’Campo, 2006). Some also believe that census data provide an oversimplified picture of neighborhood life because these data are aggregated to block or
* Corresponding author. Tel.: þ1 416 864 6060. E-mail address: [email protected] (N. Schaefer-McDaniel). 0277-9536/$ e see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.socscimed.2010.03.060
tract levels and this aggregation ignores important variability within neighborhoods (Caughy, O’Campo, & Brodsky, 1999). Moreover, census data are only collected every ten years in the US and the UK (five years in Canada) but the inner-city neighborhoods that are usually of specific interest to researchers and policy makers have been known to change very quickly therefore raising questions about the validity of utilizing dated census information (McGuire, 1997). Further, it is important to distinguish between measuring a neighborhood’s composition and context. Census data measure residents’ characteristics such as income, poverty, and education levels that are summarized for a particular area. So in reality, census data provide a compositional estimate of a neighborhood but do not provide information on the actual neighborhood context such as its infrastructure; presence or absence of institutions, resources, and facilities; or level of cleanliness (Dunstan et al., 2005; Laraia et al., 2006; Macintyre, Ellaway, & Cummins, 2002). Researchers in the United Kingdom commonly make use of administrative data to compute an index of multiple deprivation to measure neighborhood characteristics (Noble et al., 2007). This index is calculated by weighting area scores across the following domains: income deprivation, employment deprivation, health deprivation and disability, education skills and training deprivation, barriers to housing and services, living environment deprivation, and crime (Noble et al., 2007). This index has been linked to health behaviors and outcomes as divergent as physical activity, healthy eating, cognitive functioning, mobility disability, and atherosclerosis
652
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
(Amuzu, Carson, Watt, Lawlor, & Ebrahim, 2009; Deans et al., 2009; Jones, Hilldon, & Coombes, 2009; Lang et al., 2008). In order to address shortcomings associated with census and administrative data to measure neighborhood attributes, systematic social observations (SSO) of neighborhoods by trained outside raters have become a popular alternative research strategy for researchers wishing to study neighborhood attributes (Sampson & Raudenbush, 1999, 2004; Zenk et al., 2007). In this methodology, trained researchers assess physical and social attributes of particular blocks of interest in a given neighborhood. This method differs from utilizing census and other administrative data as it places researchers directly in the studied environments. For example, researchers can observe and rate the degree of cleanliness, upkeep of buildings, and people’s interactions and behaviors. In Chicago, Sampson and Raudenbush (1999, 2004) utilized this methodology to measure social and physical disorder. The former refers to aspects of social incivilities such as vandalism, gang activity, or drug behavior while the latter is an indicator of the degree of cleanliness of buildings, sidewalks, and streets. In Baltimore, research by Caughy et al. (2001) supports that in addition to aspects of physical incivilities or disorder, SSO can also measure residents’ territorial behaviors as well as the presence of play areas for children. Dunstan et al. (2005) used neighborhood observations to measure natural elements such as the presence of trees, gardens, planted vegetation, and green spaces. Research by McGuire (1997) revealed that SSO can even assess a block’s level of social cohesion and Laraia et al. (2006) observed the availability of social spaces in Raleigh, North Carolina. As these examples illustrate, SSO aims to be an objective method and does not purport to measure subjective or individualized concepts such as sense of community or social capital. Instead, SSO uses visual cues of neighborhood structure and functioning as a way of collecting information and thus primarily supplies quantitative data. While quantitative studies examining the effects of neighborhood social capital on residents’ health have prospered in recent years (Boyce, Davies, Gallupe, & Shelley, 2008; Franzini, Caughy, Spears, & Fernandez Esquer, 2005; Kruger, Reischl, & Gee, 2007; Lofors & Sundquist, 2007; Rojas & Carlson, 2006; Veenstra et al., 2005), researchers wishing to explore the meanings that residents ascribe to concepts such as “neighborhood” and “community” should also consider making use of qualitative methods. For example, Mannarini and Fedi (2009) used qualitative interviews to describe general themes across participants’ understanding of “community” and “sense of community.” They found that participants’ definitions generally overlapped with those proposed by the research literature, namely that community is both symbolic and pragmatic, a real physical entity as well as a relational, interpersonal space. Although the SSO methodology is gaining momentum in neighborhood and health research, there has not been a critical discussion of potential shortcomings of this approach. Therefore, the purpose of this commentary is to illustrate some of the challenges and limitations in the SSO methodology. We do so by drawing on our own experiences of conducting SSO in New York City and Toronto (Schaefer-McDaniel, 2007; Parsons et al., in press) and differentiate between limitations related to the methodology itself and challenges the approach presents for researchers in the field. We conclude by offering directions for future research utilizing this technique. Methodological issues
Roosa, Jones, Tein, & Cree, 2003; Weiss, Ompad, Galea, & Vlahov, 2007). For example, some researchers define neighborhoods in terms of entire census block groups or tracts while others rely on city specifications or on residents’ definitions. Others rely on geographic informational systems (GIS), which allow researchers to define neighborhoods as a certain radius around individual households so that every house is associated with a slightly different area defined as a particular neighborhood (e.g., Larson et al., 2009). Essentially, all of these geographic boundaries constitute ‘neighborhoods’ but this has led to a growing body of neighborhood research utilizing inconsistent boundaries since neighborhood spatial definitions affect the areas to be observed. For example, researchers who equate census block groups with neighborhoods may observe all blocks in a particular block group while others may rely on residents’ subjective spatial definitions of neighborhoods, which are often much smaller areas (see Coulton et al., 2001), and consequently only conduct observations for these blocks. This inconsistency has implications for comparisons between neighborhood studies. Further complicating the picture are researchers who only observe a random selection of blocks within a particular neighborhood. For example, Caughy et al. (2001) studied 57 neighborhoods in Baltimore (a neighborhood corresponded to a census block group) but only observed between 3 and 51 blocks for each neighborhood. While they sampled their observed blocks systematically, this technique is problematic because it assumes that observed blocks are representative of the larger neighborhood when data are aggregated to the neighborhood level. However, our observations suggest that this may not be the case since we noticed much variation in terms of physical and social attributes of blocks even for those in close proximity (Schaefer-McDaniel, 2007). While inconsistent boundary definitions can pose problems to comparing research findings across settings and cities, there are many possible ways of spatially defining a neighborhood as explained above. Choice of neighborhood boundaries and geographic scale should be theoretically driven, that is they should be specific to the outcomes of interest or contexts in which observations take place. For example, census boundaries can be used when SSO measurements are being compared to other censusbased data, such as demographics, while resident definitions can be used when other subjective resident information is being surveyed, such as level of fear felt within the neighborhood. Timing of observations Another issue affecting the SSO methodology relates to the timing of observations since social neighborhood attributes are often time-varying. For example, consider a young adult walking down the block while drinking a beer. Whether or not a rater will witness this “social disorder” taking place completely depends on the rater and the young adult being on the block during the same exact few minutes it takes the young adult to walk the distance. Furthermore, observations (and neighborhood life in general) are likely to differ substantially between day and night-time in some neighborhoods. In order to gain a complete picture of neighborhood life, observations should be conducted at multiple time points during the day, evening, and night-time (see Sampson & Raudenbush, 1999, 2004). However, night-time observations also raise potential safety issues, so some researchers might find themselves having to trade off between assuring data completeness and rater safety.
Defining neighborhood boundaries There has been much discussion in the literature related to the many spatial definitions neighborhoods can take (Nicotera, 2007;
Bias towards the visible and readily identifiable Despite the fact that some SSO measures such as the one used in the Los Angeles Family and Neighborhood Survey (Peterson, Sastry,
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
& Pebley, 2007) are quite comprehensive, this methodology can only provide information on attributes that can be observed directly. SSO cannot provide a complete picture of neighborhood life since many important aspects such as the degree of social connectedness among residents and neighborhood social capital as well as other social processes are not directly observable and thus not measurable through SSO. Further, the SSO methodology is biased towards measuring attributes that can be readily identifiable. For example, a common observation item is concerned with the presence of gang activity and gang members. This means that raters must be able to readily identify gang members but we learned in our observations that this is almost an impossible task. While we were sometimes aware of clear signs of gang presence (for example, a group of young men wearing the same colored bandana), there were other instances in which we could not get a good look to make a definite decision. This is especially complicated by raters’ status as outsiders of neighborhoods (clearly identified as such with clipboards in their hands) since gang affiliation may then be purposefully hidden. It is also possible that the mere act of observing a phenomenon can alter it (i.e., a Hawthorne effect, see Stocking, 1985). For example, some raters reported that young adults who were hanging out on the blocks would leave once they acknowledged the rater’s presence. So, outsiders observing an unfamiliar neighborhood may identify or perceive attributes differently than insiders. While insiders might identify some characteristics more accurately, relying on insider report also raises challenges related to standardizing SSO. One potential option to address problems in visibility is to supplement SSO with qualitative research based on resident interviews. For example, in a study examining families who had moved from areas of high to low poverty concentration, Keels (2008) conducted in-depth qualitative interviews with mothers to gage, among other measures, families’ access to neighborhood social capital. This type of information is most likely too nuanced to be captured through SSO alone. Similarly, Antony and Nicotera (2008) used a mixed methods approach to study the relationship between neighborhood structural resources and youth experience of daily “hassles” or environmentally related irritations, such as the presence of gangs or the lack of amenities. The researchers obtained qualitative information from the participants on their general experiences of neighborhood cohesion and feelings towards the community, as well as quantitative information regarding neighborhood structural resources and hassling experiences (Antony & Nicotera, 2008). The use of qualitative methods provided insight into additional types of neighborhood resources experienced by youths outside of the structural resources measured, such as the youths’ neighbors, as well as more detailed information on how social disorder within the neighborhood becomes a source of hassling for young people. “Disorder” relative to what? Another bias in the SSO methodology relates to the point of reference of observations. For example, researchers commonly observe the presence of groups of adults or young people loitering or hanging out on the street. For outside researchers, this is commonly considered an aspect of social disorder (see Sampson & Raudenbush, 1999, 2004). However, groups of people hanging out on the street may also symbolize close-knit neighborhoods in areas where neighbors interact and socialize with one another (see Klinenberg, 2003). Alternatively, neighborhoods that appear highly ordered and controlled may suppress outward signs of disorder that are important for health outcomes or may signal that a third factor is related to disorder and health (Dunn, Schaefer-McDaniel, & Ramsay, 2009). These issues raise concerns in terms of the validity
653
of SSO if the tool is developed without attention to the cultural norms, values, and behaviors of residents and neighborhoods of interest. This bias reflects the issue of utilizing outside raters as compared to insiders of study neighborhoods as previously discussed. Challenges for researchers in the field Observer safety and comfort levels Aside from methodological challenges, the SSO technique presents challenges for researchers implementing this approach in the field. Once reliability between raters has been established, it is generally the norm that observations are conducted solely by one rater. Raters may then find themselves alone in an unfamiliar part of town due to the “outsider” status requirement for the observation. This can raise serious safety concerns, particularly for young women (see Kenyon & Hawker, 1999; Sharp & Kremer, 2006), and especially considering that the point of some of these observations is to observe illegal behaviors (e.g., drug use, gang activity, vandalism). While having raters work in pairs would substantially increase their safety, this will also greatly increase the costs of conducting observations. Although we did not experience serious incidences related to safety concerns during the course of our observations, there were instances in which some of our raters felt uncomfortable, especially on blocks that were isolated from commercial and pedestrian activity and those that were severely abandoned. Further, in thinking about raters’ comfort levels, it is also important to recognize the time of day and season. Observing neighborhoods during the summer months means that more people will be outside enjoying the warm weather and that more activity can be observed. But this also has consequences for raters. For example, our observations in New York City were conducted during July and August and the hot, humid summer days and affected raters’ morale and energy levels (Schaefer-McDaniel, 2007). Ambiguous ethics of naturalistic observations Since raters observe occurrences during ordinary everyday life and observed people will not be identified, this research is considered “minimal risk,” meaning that raters do not need the consent of the people they are observing (Tri Council Policy Statement, 1998). However, this can raise questions in situations in which raters may feel they should intervene. For example, some raters observed several instances of parents loudly verbally abusing their children. In one occasion, a rater even witnessed a mother publicly hitting her young child but none of these interactions were met with any reaction from other people on the block who clearly witnessed these exchanges. In New York City, one of our raters observed children playing in the water of a fire hydrant that had been (illegally) opened in the hot weather. One young girl ran away from the water and pulled down her pants to urinate in a plastic cup, from which she then began to drink. Much to the rater’s surprise, the many onlookers started to laugh (thus recognizing the young girl’s actions) but did not interfere with the situation. Situations such as these raise serious questions in terms of ethical obligations raters have for their objects of study in naturalistic research. The ethical concerns regarding SSO rater responsibility call for the creation of a set of agreed upon guidelines. These guidelines should weigh rater role and safety with social responsibility, and make clear raters’ legal responsibilities, such as reporting suspected child abuse. The creation of ethical guidelines is no small task and will require initiative and input from major theorists and
654
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
researchers both within and outside the field of neighborhood and health research. Recommendations for neighborhood observations Future directions for neighborhood observations Given these limitations and challenges of the SSO methodology for measuring characteristics of the neighborhood context, we provide research directions for scholars utilizing this approach in the future to improve the methodology and minimize potential biases. Specifically, we recommend that 1) more attention needs to be paid to theories linking neighborhood attributes to health outcomes, 2) formative research needs to take place to develop SSO tools and the methodology itself for use in health research, and 3) more dialogue between researchers is warranted to openly discuss challenges surrounding the SSO methodology and raters’ experiences in the field. Apply and develop theories to examine neighborhoodehealth relations Neighborhood research, and the SSO methodology in particular, has its foundation in the sociology of crime. Over the years, some researchers examining neighborhood effects have replaced crime outcomes with a health measure rather than applying or developing theoretical frameworks to link neighborhood attributes to residents’ health and well-being. Generally, such studies have not been as useful for policy makers or intervention efforts. For instance, Cohen et al. (2000) summarized observations of New Orleans neighborhoods into a ‘broken windows index’ which consisted of measures on housing quality, presence of abandoned cars, graffiti, and trash. They then linked this index to gonorrhea rates showing that areas measuring high on their broken windows index were also areas in which residents had higher rates of gonorrhea. While this link is interesting, it does not help policy makers untangle whether physical deterioration is an indicator of or a risk factor for gonorrhea. Similarly, Weich et al. (2001) linked neighborhood observations to adult depression in London, England and found that depression was more prevalent in areas where residential properties had deck access, there were newer properties, fewer properties with private gardens, shared recreational space, and many patches of graffiti present. Policy makers might then wonder whether they should regulate buildings with deck access and public recreational spaces in the hopes of ameliorating residents’ mental health. These examples also highlight the fact that researchers have the capacity to conduct their research in a way that is potentially quite harmful (and unfairly so). Studies such as the ones described can misrepresent and stigmatize a community as no actionable message may come from the research. Aside from being conscientious about the potentially harmful effect research could have, we also advocate that scholars apply and develop existing theories linking neighborhood attributes to health outcomes rather than examining uni-directional relationships between neighborhood conditions and health outcomes (see Ellen & Turner, 1997; Macintyre et al., 2002; Saegert & Evans, 2003; Sampson, Morenoff, & Gannon-Rowley, 2002; Wandersman & Nation, 1998). For example, Wandersman and Nation propose three models in which mediating variables link neighborhood context to public health outcomes: 1) the structural characteristics model states that the level of social organization in the neighborhood such as social control and common values among residents along with psychological stress mediate the relationship between structural neighborhood characteristics (i.e., socioeconomic status [SES], racial composition) and residents’ mental health; 2) the environmental
stress model postulates that environmental stressors such as noise or crowding affects residents’ health indirectly through unsuccessful coping strategies; and 3) the neighborhood disorder model suggests that neighborhood characteristics, specifically social and physical disorder, directly relate to residents’ mental health so that residents in neighborhoods characterized by the presence of gangs, violence, abandoned buildings, and litter, have poorer mental health. Wandersman and Nation further suggest that the neighborhood disorder model may also work through residents’ fear of crime and victimization (i.e., neighborhood disorder is linked to fear of crime, which is linked to poor mental health, see also Skogan, 1990). Since these theories may not hold in all contexts, we encourage researchers to undertake formative research to introduce new theories linking neighborhood attributes to health outcomes. We particularly applaud efforts that actively involve residents of target neighborhoods. For instance, O’Campo, Salmon, and Burke (2009) conducted a series of concept mapping exercises with Toronto residents to uncover their perceptions of neighborhood attributes that relate to poor mental health outcomes. In the final activity, participants were asked to develop pathways that connect neighborhood characteristics and mental health. The researchers then translated these pathways into a theoretical framework linking neighborhood attributes to mental health outcomes. Such formative research is particularly valuable because it not only allows researchers to ground their research in theory but also gives them a chance to tailor SSO tools to specific health outcomes of interest. Conduct formative research to develop SSO tools and methodology Aside from conducting formative research to develop theories linking neighborhood attributes to health outcomes, we also emphasize the need for such research to develop SSO tools and the methodology itself. As discussed, research examining the SSO methodology needs to address the spatial neighborhood boundary issue by examining, for instance, whether different levels of aggregation of SSO data have similar or divergent effects on health outcomes. Moreover, SSO collects information on time-sensitive events so it is important for researchers to conduct multiple observations of blocks of interest. However, this also raises grounds for future empirical work to systematically examine not only how many observations need to be conducted per block to yield reliable estimates but also how factors such as time of day and seasonal variations affect ratings. Such explorations would help researchers understand place effects more concretely. We also emphasize a need for formative research to develop SSO tools. Here, we applaud the efforts of researchers who actively involve residents and expert panels to modify existing SSO tools for their needs (see Hoehner, Ramirez, Elliott, Handy, & Brownson, 2005; Pikora et al., 2002; Zenk et al., 2007). Involving insiders in SSO tool development has the additional benefit of addressing some of the insider/outsider issues we previously raised. Lastly, we encourage researchers to more closely examine psychometric properties of their SSO tools. In a previous systematic review (Schaefer-McDaniel, Caughy, O’Campo, & Gearey, 2010), we find that few researchers make use of data reduction procedures such as factor analysis to examine underlying scales and concepts that SSO tools can measure. Along similar lines, the review finds great variability in terms of reported inter-rater reliability statistics. The absence of norms regarding sound psychometric evaluations of SSO tools potentially compromises the reliability and validity of observational measures. Create a space for researchers to discuss benefits and challenges As neighborhood and health research and studies utilizing SSO are gaining increasing popularity, we recommend that researchers
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
openly discuss and reflect on their experiences with SSO tools and methods. Since surprisingly little methodological detail is provided in published studies that utilize SSO methods (Schaefer-McDaniel, Caughy, O’Campo, & Gearey, 2010), we encourage researchers to be more explicit and reflective about their neighborhood research and SSO methodologies in particular. This space can be created via conferences or special editions in academic journals dedicated to the topic and would allow researchers to openly debate and discuss their experiences, challenges, and recommendations. In addition to discussions around methodological issues, we also emphasize a need for researchers to examine raters’ experiences around safety and unforeseen ethical dilemmas as illustrated in our examples. While the latter point raises numerous questions beyond the scope of this article in terms of researchers’ responsibilities and accountability in naturalistic research, it also highlights the need for raters to have a supportive research community with which they can discuss their experiences and perceived obligations. Present-day solutions for SSO challenges While we discussed future directions that SSO research should take, we recognize that such change takes time, and that it is unreasonable to suggest that research using SSO halt until the recommended research is completed. We thus offer more immediate research directions that can be addressed in the meantime to better understand both the potential and the limitations of SSO use. The challenge regarding neighborhood measurement is that different studies will naturally use the means of creating neighborhood boundaries that best suits their particular research. We therefore do not promote a standard neighborhood definition; rather, this challenge is more a problem of comparison rather than validity (see also Schaefer-McDaniel, Caughy, O’Campo, & Geary, 2010). To shed more light on the magnitude of this issue, we recommend a systematic examination of similarities and differences in SSO and neighborhood effects on health by different observational boundaries. Until such research is conducted, we suggest that scholars be explicit about choosing specific boundary measurements and discuss whether and how their research might be applicable to other types of neighborhood definitions. Likewise, the timing of SSO ratings requires additional research, such as a systematic review examining the timing of observations, the effects of time of day (e.g., morning vs. evening), type of day (weekday vs. weekend or holiday), and season (e.g., winter vs. summer) on observation and health outcomes. In addition, a systematic analysis examining differences and similarities between inside (i.e., residents of study neighborhoods) and outside raters would greatly add to the SSO literature. Conclusion We conclude by emphasizing that neighborhood research has the potential to lead to action (for example, in the form of policy changes, initiation of new interventions). However, in order for research to lead to constructive change, it is not enough to merely discover a correlation between neighborhood characteristics and health, nor is it helpful to assume that such a correlation represents a direct causal link. Current neighborhood research benefits most strongly from studies examining the mediating variables between the qualities of an environment and the health of its citizens. Rather than observing and reporting an association between neighborhood characteristics and health outcomes without additional explanation, current research is at its most effective when researchers recognize their responsibility to use language and report results in a manner that avoids sensationalism and informs without stigmatizing certain groups or neighborhoods.
655
We hope that this commentary is the beginning of an ongoing dialogue between researchers and stakeholders engaged in neighborhood and health research to develop theory, research, and methodology (such as the use of SSO tools) in order to move the field forward. As discussed, we view that discussions and formative research around methodological issues in SSO tools are necessary to shed insight into the complex relationships between neighborhoods and residents’ health outcomes and to guide prevention and intervention efforts.
References Amuzu, A., Carson, C., Watt, H. C., Lawlor, D. A., & Ebrahim, S. (2009). Influence of area and individual lifecourse deprivation on health behaviours: findings from the British Women's Heart and Health Study. European Journal of Cardiovascular Prevention & Rehabilitation, 16(2), 169e173. Antony, E. K., & Nicotera, N. (2008). Youth perceptions of neighborhood hassles and resources: a mixed method analysis. Children and Youth Services Review, 30(11), 1246e1255. Boyce, D., Davies, D., Gallupe, O., & Shelley, D. (2008). Adolescent risk taking, neighborhood social capital, and health. Journal of Adolescent Health, 43, 246e252. Caughy, M. O., O’Campo, P., & Brodsky, A. E. (1999). Neighborhoods, families, and children: implications for policy and practice. Journal of Community Psychology, 27, 615e633. Caughy, M., O’Campo, P., & Patterson, J. (2001). A brief observational measure for urban neighbourhoods. Health and Place, 7, 225e236. Cohen, D. A., Spear, S., Scribner, R., Mason, K., Kissinger, P., & Wildgen, J. (2000). Broken windows and the risk of gonorrhea. American Journal of Public Health, 90, 230e236. Coulton, C. J., Korbin, J., Chan, T., & Su, M. (2001). Mapping residents’ perceptions of neighborhood boundaries: a methodological note. American Journal of Community Psychology, 29(2), 371e383. Deans, Kevin A., Bezlyak, Vladimir, Ford, Ian, Batty, G. David, Burns, Harry, Cavanagh, Jonathan, et al. (2009). Differences in atherosclerosis according to area level socioeconomic deprivation: cross sectional, population based study. BMJ, 339, b4710. Duncan, G. J., & Raudenbush, S. W. (1999). Assessing the effects of context in studies of child and youth development. Educational Psychologist, 34(1), 29e41. Dunn, J. R., Schaefer-McDaniel, N. J., & Ramsay, J. (2009). Neighborhood chaos and children's development: questions and contradictions. In G. W. Evans, & T. D. Wachs (Eds.), Chaos and its influence on children's development: An ecological perspective (pp. 173e189). Washington DC: American Psychological Association Press. Dunstan, F., Weaver, N., Araya, R., Bell, T., Lannon, S., Lewis, G., et al. (2005). An observation tool to assist with the assessment of urban residential environments. Journal of Environmental Psychology, 25, 293e305. Ellen, I. G., & Turner, M. A. (1997). Does neighborhood matter? Assessing recent evidence. Housing Policy Debate, 8, 833e866. Franzini, L., Caughy, M. O. B., Spears, W., & Fernandez Esquer, M. E. (2005). Neighborhood economic conditions, social processes, and self-rated health: A multilevel latent variables model. Social Science & Medicine, 61, 1135e1150. Hoehner, C. M., Ramirez, L. K. B., Elliott, M. B., Handy, S. L., & Brownson, R. C. (2005). Perceived and objective environmental measures and physical activity among urban adults. American Journal of Preventive Medicine, 28(2S2), 105e116. Jones, A., Hilldon, M., & Coombes, E. (2009). Greenspace access, use, and physical activity: understanding the effects of area deprivation. Preventive Medicine, 49, 500e505. Keels, M. (2008). Neighborhood effects examined through the lens of residential mobility programs. American Journal of Community Psychology, 42, 235e250. Kenyon, E., & Hawker, S. (1999). ‘Once would be enough’: some reflections on the issue of safety for lone researchers. International Journal of Social Research Methodology: Theory and Practice, 2(4), 313e327. Klinenberg, E. (2003). Heat wave: A social autopsy of disaster in Chicago. Chicago: University of Chicago Press. Kruger, D. J., Reischl, T. M., & Gee, G. C. (2007). Neighborhood social conditions mediate the association between physical deterioration and mental health. American Journal of Community Psychology, 40, 261e267. Lang, I. A., Llewellyn, D. J., Langa, K. M., Wallace, R. B., Huppert, F. A., & Melzer, D. (2008). Neighborhood deprivation, individual socioeconomic status, and cognitive function in older people: analyses from the English Longitudinal Study of Ageing. Journal of American Geriatric Society, 56(2), 191e198. Laraia, B. A., Messer, L., Kaufman, J. S., Dole, N., Caughy, M., O’Campo, P., et al. (2006). Direct observation of neighbourhood attributes in an urban area of the US south: characterizing the social context of pregnancy. International Journal of Health Geographics, 5(11), 1e11. Larson, K., Gilliland, J., Hess, P., Tucker, P., Irwin, J., & He, M. (2009). The influence of physical environment and sociodemographic characteristics on children’s mode of travel to and from school. American Journal of Public Health, 99, 520e526.
656
N. Schaefer-McDaniel et al. / Social Science & Medicine 71 (2010) 651e656
Leventhal, T., & Brooks-Gunn, J. (2000). The neighborhoods they live in: the effects of neighborhood residence on child and adolescent outcomes. Psychological Bulletin, 126(2), 309e337. Lofors, J., & Sundquist, K. (2007). Low-linking social capital as a predictor of mental disorders: a cohort study of 4.5 million Swedes. Social Science & Medicine, 64, 21e34. Macintyre, S., Ellaway, A., & Cummins, S. (2002). Place effects on health: how can we conceptualise, operationalise and measure them? Social Science & Medicine, 55, 25e139. Mannarini, T., & Fedi, A. (2009). Multiple senses of community: the experience and meaning of community. Journal of Community Psychology, 37(2), 211e227. McGuire, J. M. (1997). The reliability and validity of a questionnaire describing neighborhood characteristics relevant to families and young children living in urban areas. Journal of Community Psychology, 25(6), 551e566. Nicotera, N. (2007). Measuring neighborhood: a conundrum for human services researchers and practitioners. American Journal of Community Psychology, 40 (1e2), 26e51. Noble, M., Mclennan, D., Wilkinson, K., Whitworth, A., Exley, S., Barnes, H., et al. (2007). The English indices of deprivation 2007. Department of Communities and Local Government. O’Campo, P., Salmon, C., & Burke, J. G. (2009). Neighbourhoods and mental wellbeing: what are the pathways? Health & Place, 15, 56e68. Parsons, J., Singh, G., Scott, A., Nisenbaum, R., Balasubramaniam, P., Jabbar, A., et al. Standardized observation of neighborhood disorder: does it work in Canada? International Journal of Health Geographics, in press. Peterson, C. E., Sastry, N., & Pebley, A. R. (2007). The Los Angeles Family And Neighborhood survey neighborhood observations codebook. Rand corporation. Available at: http://www.rand.org/pubs/working_papers/WR240.13/. Pikora, T. J., Bull, F. C. L., Jamrozik, K., Knuiman, M., Giles-Corti, B., & Donovan, R. J. (2002). Developing a reliable audit instrument to measure the physical environment for physical activity. American Journal of Preventive Medicine, 23(3), 187e194. Rajaratnam, J. K., Burke, J. G., & O’Campo, P. (2006). Maternal and child health and neighborhood context: the selection and construction of area-level variables. Health & Place, 12, 547e556. Rojas, Y., & Carlson, P. (2006). The stratification of social capital and its consequences for self-rated health in Taganrog, Russia. Social Science & Medicine, 62, 2732e2741. Roosa, M. W., Jones, S., Tein, J., & Cree, W. (2003). Prevention science and neighborhood influences on low-income children’s development: theoretical and methodological issues. American Journal of Community Psychology, 31, 91e120. Saegert, S., & Evans, G. W. (2003). Poverty, housing niches, and health in the United States. Journal of Social Issues, 59, 569e589.
Sampson, R. J., & Raudenbush, S. W. (1999). Systemic social observation of public spaces: a new look at disorder in urban neighborhoods. American Journal of Sociology, 105(3), 603e651. Sampson, R. J., Morenoff, J. D., & Gannon-Rowley, T. (2002). Assessing neighborhood effects: social processes and new directions in research. Annual Review of Sociology, 28, 443e478. Sampson, R. J., & Raudenbush, S. W. (2004). Seeing disorder: neighborhood stigma and the social construction of “broken windows”. Social Psychology Quarterly, 67, 319e342. Sastry, N., Ghosh-Dastidar, B., Adams, J., & Pebley, A. (2006). The design of a multilevel survey of children, families and communities: the Los Angeles Family and Neighborhood survey. Social Science Research, 35(4), 803e1134. Schaefer-McDaniel, N. (2007). Children talk about their New York city neighborhoods: the role of subjective and objective neighborhood evaluations in understanding child health. Saarbrücken, Germany: VDM Verlag Dr. Müller. Schaefer-McDaniel, N. J., Caughy, M., O'Campo, P., & Gearey, W. (2010). Examining methodological details of neighborhood observations and the relationship to health: A literature review. Social Science & Medicine, 70, 277e292. Sharp, G., & Kremer, E. (2006). The safety dance: confronting harassment, intimidation, and violence in the field. Sociological Methodology, 36(1), 317e327. Skogan, W. (1990). Disorder and decline: Crime and the spiral of decay in American Neighborhoods. Berkley and Los Angeles, CA: University of California Press. Stocking, G. W. (1985). Observers observed: Essays on ethnographic fieldwork. Madison, WI: University of Wisconsin Press. Tri Council Policy Statement. (1998). Canadian institutes of health research, natural sciences and engineering research council of Canada, Social sciences and humanities research council of Canada, Tri-council policy statement: Ethical conduct for research involving humans (with 2000, 2002, 2005 amendments). Veenstra, G., Luginaah, I., Wakefield, S., Birch, S., Eyles, J., & Elliot, S. (2005). Who you know, where you live: social capital, neighbourhood and health. Social Science & Medicine, 60, 2799e2818. Wandersman, A., & Nation, M. (1998). Urban neighborhoods and mental health: psychological contributions to understanding toxicity, resilience, and interventions. American Psychologist, 53(6), 647e656. Weich, S., Burton, E., Blanchard, M., Prince, M., Sproston, K., & Erens, B. (2001). Measuring the built environment: validity of a site survey instrument for use in urban settings. Health & Place, 7, 283e292. Weiss, L., Ompad, D., Galea, S., & Vlahov, D. (2007). Defining neighborhood boundaries for urban health research. American Journal of Preventive Medicine, 22(6S), S154eS159. Zenk, S. N., Schulz, A. J., Mentz, G., House, J. S., Gravlee, C. C., Miranda, P. Y., et al. (2007). Inter-rater and testeretest reliability: methods and results for the neighborhood observational checklist. Health & Place, 13, 452e465.