Reflections on citizen–state child welfare partnerships: Listening to citizen review panel volunteers and agency liaisons

Children and Youth Services Review 33 (2011) 612–621

Contents lists available at ScienceDirect

Children and Youth Services Review j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / c h i l d yo u t h

Reflections on citizen–state child welfare partnerships: Listening to citizen review panel volunteers and agency liaisons Valerie Bryan a,⁎, Crystal Collins-Camargo b, Blake Jones c a b c

University of South Alabama, Department of Sociology, Anthropology and Social Work, 34 Humanities Bldg., Mobile, AL 36688, USA University of Louisville Kent School of Social Work, Louisville, KY, USA University of Kentucky College of Social Work Training Resource Center, Lexington, KY, USA

a r t i c l e

i n f o

Article history: Received 24 June 2010 Received in revised form 29 October 2010 Accepted 2 November 2010 Available online 6 November 2010 Keywords: Citizen review panels Child welfare systems Grounded theory Citizen participation Child abuse

a b s t r a c t Objective: Previous research pertaining to the citizen review panel (CRP) initiative indicates that discrepancies exist between panel member and state agency liaison perceptions of CRP effectiveness in fulfilling the CAPTA CRP mandate. This study explores the impressions of both CRP members and liaisons involving barriers to effective CRP–state child welfare partnerships and recommendations to improve the relationship, provided through narrative survey responses from CRP liaisons from 30 states and D.C. and panel members from 32 states and D.C. Thematic categories which emerged from analyses of these responses are discussed, and a conceptual model and substantive-level theory of the CRP–child protective services (CPS) relationship developed from the results are presented. Results: The complex nature of CRP–CPS relationships emerged as the central theme among CRP members' and liaisons' responses. Liaisons and CRP members identified a need for CRP members to become more educated globally on child welfare despite the relatively high level of education and experience of much of the sample. Difficulties encountered by panel members in the effort to partner surfaced in themes of mistrust and skepticism about the worth of citizen participation in child protection. Negative agency attitudes, lack of role clarity, and lack of commitment to authentic partnership were cited as negative influences upon the relationship by both liaisons and CRP members. The most important preferred outcome of the partnership was that citizen volunteers would become more knowledgeable partners who could meaningfully contribute to child welfare efforts. A second desired outcome important to CRP members was to form a shared vision with CPS. Conclusions: Findings imply that when key components of mutual respect, legitimacy, CPS knowledge, shared vision, authenticity, citizen engagement, honest communication, and a serious, deliberative process are in any way impeded, the partnership is vulnerable to breakdown, and intended outcomes may be compromised. Implications: Important relational difficulties that have formed barriers between volunteers and CPS agencies have been identified, and results suggest specific issues to target for improvement. These findings can prompt a more informed discourse about the challenges and opportunities presented when attempting to engage citizens in child welfare practice and policy-making, and can lead to new research paths. Suggestions for such efforts are offered. © 2010 Elsevier Ltd. All rights reserved.

1. Introduction The concept of citizen engagement in public agencies has been studied from a variety of perspectives. It has been generally agreed upon that citizen engagement, or participation, is helpful to public bureaucracies, but only if a process is intentionally set in place whereby citizens are given the information they need to make informed judgments about the agencies they are evaluating (see, for example, King, Felty, & Susel, 1998; Osborne & Gaebler, 1992; Schacter, 1995). A large body of literature has formed examining

⁎ Corresponding author. Tel.: + 1 251 414 8041; fax: + 1 251 460 7925. E-mail address: [email protected] (V. Bryan). 0190-7409/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.childyouth.2010.11.003

the idea of citizen participation in the creation of public policy. Police review boards, environmental citizen advisory boards, and other citizen groups have been evaluated as to their impact on public policy in these areas (Barton, 1970; Creighton, 1999; Koontz, 1999; Lynn & Kartez, 1995). Prior to the 1970s, the field of public administration especially was criticized for producing a system that was hierarchical and unresponsive to the demands of citizen groups (Denhardt & Denhardt, 2000; King et al., 1998; Timney, 1998). Because of this criticism, directives arose for public administrators to make citizens an integral part of the policy process, often in the form of a federal mandate (Stivers, 1990). This call to action has historically been viewed as a response to the social upheaval of the 1960s as well as a result of recognition by policy makers that citizen input can be useful.

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

Scholars have also identified benefits of increased citizen participation in the public sector, including a sense of citizen ownership of the governmental processes which affect them, a heightened sense of community among citizens who volunteer, and mitigation of negative citizen perceptions of public administrators (Box, 1998; Creighton, 1999; Denhardt & Denhardt, 2000; Putnam, 1993; Schacter, 1995; Stivers, 1990; Verba, Schlozman, & Brady, 1995). Participation has been seen as a way to “level the playing field” between the government and its citizens, as well as a means to hold public agencies accountable in the formation and implementation of public policy (Box, 1998; Denhardt & Denhardt, 2000; Osborne & Gaebler, 1992; Schacter, 1995; Timney, 1998). Especially in the public administration literature, several important components of successful citizen–state partnerships emerge: fairness of the process, two-way information flow (which leads to trustbuilding), competence of the participants, and adequate resources to support citizen involvement. 1.1. Citizen review panels The notion that citizen review panels (CRPs) for child protective services (CPS) could improve child welfare systems and practices was formed from this context of citizen engagement and system accountability. Debates continue about whether or not external child welfare review systems are needed at all (Blome & Steib, 2007; Jordan & Franklin, 1994). The argument against such citizen participation is that it is no more than a “feel good” exercise carried out by citizens, when what is actually needed is a complete overhaul of the child welfare system in the United States (Waldfogel, 1998). However, considerable discussion surrounding the issue eventually led to a consensus that CRPs could serve as a useful catalyst for improvement and accountability within state welfare agencies (see Jones, 2004a for a detailed discussion). The Child Abuse Prevention and Treatment Act (CAPTA) is arguably the most sweeping piece of child welfare legislation ever enacted. From its original passage in 1974, CAPTA encouraged states to implement a consistent way by which to identify and address child maltreatment, created the National Center on Child Abuse and Neglect (NCCAN), and shaped uniform definitions of child abuse and neglect (Waldfogel, 1998). In 1996, CAPTA was amended to establish CRPs through Public Law 104-235. Among other provisions, Public Law 104-235 called for the implementation of at least three citizen review panels in each state by July 1999 (Administration for Children and Families, 1998). According to the law, CRPs were to be comprised of a representative sample of the community, meet at least once every 3 months, and submit an annual report to the federal government outlining their activities and recommendations (Administration for Children and Families). There was also a directive that CPS agencies be cooperative in providing needed information and technical assistance to the panels (Kot, Bruner, & Scott, 1998). In summary, the legislation provided the panels with a broad mandate: 1. To insure that the state was in compliance with the state CAPTA plan. 2. To assure that the state was coordinating with the Title IV-E foster care and adoption programs. 3. To assess the CPS agency in its compliance with the review of child fatalities. 4. To evaluate any other piece of the CPS system which the panel deemed important. CAPTA was reauthorized in 2003, and Congress added additional tasks for the panels and state agencies, including a directive for the panels to create a means for public comment, for panels to study the practices as of the state agency in addition to policy and procedures, and for state agencies to respond in writing to the recommendations of the panels within 6 months.

613

A few studies have been completed with regard to CRPs. The first multi-state study of citizen review panels in the Midwest and South studied the relationship of communication, group cohesion, and paid staff support to the panels to CRP members' perceived effectiveness to impact child welfare outcomes (Jones, 2004a,b). Impediments to effectiveness were also examined, and the results indicated that lack of trust, time constraints, unclear roles, and ineffectual communication contributed to panel members' decreased perceptions of effectiveness. However, findings from the study were limited to only CRP members' perceptions of their own effectiveness and to the variables of interest in the study, as no other stakeholders were surveyed. Additionally, this study was completed less than 5 years after states were required to have their panels in place, which may have been too soon to evaluate perceived effectiveness of the relatively new CRP initiative. Other studies have examined CRP panel members' attitudes toward CPS personnel and the inherent tension between their oversight role and the oft-stated need to serve as advocates for the agency (Bryan, Jones, Allen, & Collins-Camargo, 2007; Jones, Litzelfelner, & Ford, 2003). Citizen review panel members have consistently reported that they feel “shut out” of the policy-making process, consulted only after policies are implemented. Another common complaint is that panel members are given little feedback regarding their recommendations, even though this is a specific directive of the CAPTA CRP amendment. The feedback that is received is often construed as merely paying “lip service” to the recommendations of the panels without thoughtful consideration of how their recommendations could be implemented (Bryan et al., 2007). This sense of disconnection from authentic dialogue and low investment in the process has also been identified by child welfare agency personnel. Research has revealed disagreement within child protective services agencies on the fundamental question about whether or not citizen review is even important or desired in the policy making process (Jones et al., 2003; Jordan & Franklin, 1994). Child welfare administrators in a one-state study have reported concerns about the recommendations of the citizen review panels, complaining that they lack well-grounded reasoning and specificity. One supervisor noted that, “What may make sense to them is totally impossible for the agency to do” (Bryan et al., 2007, p. 1294). However, these researchers also note that, despite voicing a struggle with communication between the child welfare agency and citizens, both agreed that citizens can provide useful support and advocacy for the agency. Citizens can be viewed as “professional advisors” within their area of expertise and this knowledge has been identified as helpful in preventing the agency from becoming myopic and short-sighted. As a companion piece to the current study, Bryan, Jones and Lawson (2010) completed a national study of citizen review panels in which they surveyed CRP members in 32 states and the District of Columbia (D.C.) and program liaisons in 30 states and D.C. From their findings, a model was developed which explained 53% of the variance in CRP members' perceptions of their effectiveness in making a positive impact upon child welfare practices and policies. Significant variables in the model included information flow between the panels, group cohesion and self-governance. These findings support previous research on the importance of developing a means, such as training, by which panels can receive needed information (i.e., child safety data, statistics on adoption) in order to complete their tasks (Jones, 2004a,b; Jones & Royse, 2008). However, the researchers found little relation between the panel members' self-ratings and those of their CPS agency liaisons, raising additional questions about critical differences in perceptions. Given that these liaisons play a critical role in the relationship between the CRPs and the public agency, and are typically the conduit for information shared with the panels, further exploration of this apparent disconnect seemed important. Citizen review panels were conceptualized by Congress as a way for citizens to impact public policy with regard to child protective

614

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

services. The emergent research in this area has shown that this directive has proven to be challenging both from a logistical as well as a conceptual standpoint. Some of these challenges may be developmental in nature, and suggest that in order to be successful, attention needs to be paid to the sanction afforded the program and the communication and understanding among parties. It is clear that more research is needed to examine the use of citizen volunteers in child welfare systems and its effects on case planning and permanency for children in care, as well as the broader outcomes sought by the child welfare system overall. Hundreds of volunteers are currently devoting time to this endeavor, agency liaisons are working to bridge the relationship, and agency administrators and frontline staff are responsible for responding to CRP recommendations and potentially implementing systems change as result. While the importance of volunteer participation in child welfare is a matter subject to debate, much can be done to improve the collaboration between citizens and the child welfare system as it currently exists. The present study attempts to deepen and illuminate the discourse around this form of citizen participation and the citizen– state partnership, through careful analyses and exploration of these partners' reflections on the policy experiment that is the citizen review panel mandate.

2. Study purpose Prior studies suggest that constructs such as group cohesion, information flow and perceived group self-governance strongly influence CRP members' perceptions of their panels' effectiveness in making a positive impact upon child welfare systems and service delivery (Bryan et al., 2007; Jones, 2004b). However, prior findings also indicated that panel group dynamics and perceived effectiveness were only weakly related to agency liaisons' ratings of panel effectiveness (Bryan et al., 2010). Liaisons coordinate CRP activities and fill an important broker role between panels and state child welfare agencies. Narrative responses to questions posed to both CRP members and liaisons about obstacles to citizen–state child welfare partnerships and recommendations to improve the partnership were examined in an attempt to bridge the identified gap between perceived effectiveness ratings by members and liaisons. Results from the 2010 study suggested that these respondent groups viewed the relationship differently, and the guiding focus of the current study was to improve understanding about why this was the case. It was hoped that more nuanced, sophisticated and specific knowledge about the CRP–CPS relationship would emerge from the analysis.

3. Methods and procedures Approval to proceed with the study was obtained from the authors' respective university Institutional Review Boards (IRBs) in fall of 2008. Upon receipt of IRB approval, online surveys for both CRP members and liaisons were constructed to be distributed via email. CRP liaisons were asked to both participate as survey respondents and to assist with recruitment of CRP member study participants. Through one author's national citizen review panel listserv, CRP liaisons in all states and the District of Columbia (D.C.) were contacted to recruit participants. Liaisons were sent invitations to participate which contained all information necessary for them to consent to complete the surveys, and also contained a hyperlink to the online survey. Liaisons were also asked to forward to all of their states' CRP members invitations with informed consent information and hyperlinks to the online CRP member survey. Altogether, liaisons in 36 states and D.C. agreed to participate in the study and to recruit their states' CRP members to also participate. Survey responses from both members and liaisons were collected from October through November 2008.

3.1. Sample data collection Survey responses were automatically downloaded upon survey completion to a password-protected secure database and retrieved for analysis in December 2008, when the survey was closed. Due to the purposive sampling procedure implemented and the use of the national listserv as a recruitment tool, an exact response rate is not calculable. However, liaisons in the participating states and D.C. identified a total of 3748 CRP volunteers who were linked to the CRP national listserv and received an emailed invitation. More than 37 liaisons also participated as survey respondents because several states had assigned the duties of CRP coordination to more than one individual. CRP liaisons (N = 39) from 30 states and D.C. and panel members (N = 421) from 32 states and D.C. responded to openended online survey questions, providing answers to the following: What are your top three suggestions for how citizens can work more effectively with the child protective services system to ensure better outcomes for families and children? and What are the top three obstacles that prevent citizens and the child protective services system from working together? Survey respondents were free to offer any length response that they wished, so responses varied from oneword statements for each identified barrier and for each recommendation (i.e., three-word answers to each question), to full paragraphs of narrative responses. Participants also had the option to not provide an answer. 3.2. Analytic process The primary analytic approach taken modeled a classic constant comparative approach as outlined by Glaser (1964-1965); however, additional steps taken reflect methods proposed and discussed by other grounded theorists (Charmaz, 2006; Morrow & Smith, 1995; Strauss & Corbin, 1990). Three researchers first conducted open coding on all responses from liaisons and CRP members separately as barriers to partnership and suggestions to improve the citizen–state partnership. This step involved identifying patterns, similar words and phrases, among the narrative responses and creating broadly-defined codes under which similarly-defined words and phrases could be categorized. Secondary coding with these identified themes to aggregate and clarify thematic categories and subcategories was then performed (Creswell, 1998; LeCompte & Schensul, 1999). Analyses were conducted without benefit of qualitative analysis software, but survey responses were exported into word processing software and analyzed by hand by three researchers. Copies of the original textual survey output were maintained for reference throughout the analyses. Then, by constant comparison of cases to each other, focused codes were developed, which allowed the most powerful and meaningful themes to emerge (Charmaz, 2006). An axial coding procedure was performed in order to systematically relate focused codes to each other in a logical way, to illustrate meaningful relationships between and among themes important to the understanding of CRP–CPS partnerships as formed by the mandate (Strauss & Corbin, 1998). This critical step involved carefully reviewing the most saturated themes and their subcategories, and comparing to original source text to establish each code's context. The results of this process were diagrammed through a framework developed by Strauss and Corbin (1990; see also Morrow & Smith, 1995), for clarity and specification of the constructed model. Last, an emergent substantive-level theory derived from these analyses was formed, to foster a more comprehensive and deeper explication of the nature of CRP–child welfare partnerships, and the conditions, context, and strategies that help or hinder their combined efforts to improve child welfare outcomes. Within the grounded theory tradition, this level of theory is generated directly from close observation of a specific event or condition, and provides an explicit description of what was learned. Findings are intended to facilitate

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

more sophisticated understanding and knowledge gains about a particular issue, in this case, the CRP–CPS relationship, and are not expected to be generalizable. 4. Participant description Before presenting analytic results, a description of those who participated in the study and offered their reflections on the CRP–CPS relationship is warranted, in order to further interpret the research findings. Survey participants were asked questions about their background and experience, and their responses described in brief below.

615

Table 2 Agency liaison and panel characteristics (N = 39). Variable Panel type Started from scratch Used existing board Relationship to CRP State employee of child welfare agency Not a state employee

n

%

18 21

46.2 53.8

24 15

61.5 38.5

Variable

n

Mean (SD)

Median

Months Agency Liaison

39

37.9 (39.5)

24

4.1. CRP liaisons CRP liaisons generally fulfill the role of agency–panel coordinator, tasked with the responsibility of assisting panels in implementing their evaluation of child welfare agencies. Key activities involved in this effort include providing logistical support, assistance with fulfillment of CAPTA requirements, facilitation of strategic planning, and assistance with marketing and generating public awareness of CRPs. They also serve as the key communication conduit between CRPs and CPS. As noted previously, 39 liaisons from 30 states and D.C. completed one or both of the open-ended survey questions focused upon in this study (see Table 1). The majority of liaisons participating in the study were state agency employees, though a significant number were not. They averaged over 3 years of experience serving as panel–agency liaisons for their CRPs, while the median length of service was 2 years. More often than not, they worked with panels assembled specifically for the purpose of fulfilling the CRP mandate, but almost half of respondents reported that the panel they coordinated had been originally created for another child welfare-related purpose prior to the 1996 mandate (e.g., child fatality review or foster care review boards). Quantitative survey data from the first phase of this study revealed no significant differences in liaisons' ratings of CRP effectiveness based upon panel type or upon state employee status. Table 2 provides a summary description of liaison and panel characteristics. 4.2. CRP members

Table 1 Agency liaison survey respondents' states (N = 39). AL ID MD NH OR

5. Results The open coding process of barriers to CRP–CPS partnerships and recommendations to improve them resulted in the generation of 108 codes, or themes. The most prominent obstacles to citizen–state partnership included a broad lack of citizen knowledge about child welfare, lack of agency transparency, poor communication, lack of time, and lack of funding/other resources to support CRPs. Of these themes, liaisons and panel members were most in agreement about CRP member knowledge, transparency, time and resource obstacles. Other obstacles commonly identified by liaisons included perceptions of a token partnership between CRPs and agencies, uncooperative/disengaged agency leadership, and too many other worker demands, such as burdensomely large caseloads and competing priorities. Recommendations to improve citizen–state partnerships included improving the quality of a wide array of CRP efforts, educating CRP members across a range of child welfare topics, communicating consistently, increasing CRP community involvement, and changing a variety of agency practices. Other recommendations more often made by liaisons Table 3 CRP member survey respondents' states (N = 421).

Panel members from 32 states and D.C. (N = 421) provided narrative responses to one or both of the open-ended questions regarding barriers to CRP–CPS partnerships and recommendations to improve the partnership (see Table 3). These citizen volunteers were predominately female and averaged about 50 years old. They were also highly educated, most typically possessing 18 years of education post-kindergarten, or equivalent to a master's degree. They mainly reported work experience in social services professions, but many also worked in the legal or medical fields, or were already retired. Of those who selected “other,” many identified careers in education and law enforcement. Others listed positions which could be construed as social service-related but were not identified as such by the respondent, such as family services agency director and victim advocate. Quantitative survey results from the first phase of this study identified no significant differences on measures of perceived panel effectiveness by occupation. CRP members averaged over 3.5 years of experience serving on their panels, but experience levels varied

AK GA ME NE OH WY

widely as shown by the standard deviation in Table 4, which offers a summary of all of these panel member characteristics.

CA IL MI NJ TN

CT IN MN NM TX

D.C. IA MS NY WA

FL LA MO NC WI

AK HI LA MO NC WA

AL ID ME NE ND WI

CA IL MD NH OH WY

CT IN MI NJ OR

D.C. IA MN NM TN

GA KS MS NY TX

Table 4 CRP member descriptive characteristics (N = 421). Variable Gender Female Male Profession Social services Legal Medical Homemaker Retired Other Missing

n

%

340 81

80.8 19.2

189 50 35 7 39 98 3

44.9 11.9 8.3 1.7 9.3 23.3 .7

Variable

n

Mean (SD)

Median

Mode

Missing

Age (years) Education (years)a Months CRP member

406 416 409

50.5 (11.13) 17.5 (2.35) 43.0 (37.8)

51 18 34

58 18 24

15 5 12

a

Years of education excluding kindergarten.

616

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

included improved commitment to the CRP–state partnership, recruitment of diverse CRP membership, and legislative advocacy.

Table 6 CRP member (N = 403) and liaison (N = 39) suggestions to improve the partnership. Categories of combined suggestions

5.1. Focused coding A secondary coding procedure termed focused coding by Charmaz (2006) reduced the number of open codes by collapsing related themes into central categories, while also dimensionalizing these key categories by creating subcategories from those that were deemed derivative of, or a type of, a larger and more prominent theme. During this analytic process, efforts were also undertaken to identify the most heavily saturated themes of both response sets from liaisons and CRP members combined. The results of these procedures are presented in Tables 5 and 6: 5.2. Axial coding Final analytic steps involved linking the most saturated, focused codes to each other in meaningful relationships to attempt formation of a conceptual framework of CRP–CPS partnerships. This process involved inductive and deductive reasoning about the most densely developed categories and their properties, and considering how they fit together to form a picture of these complex relationships. Following the general guidelines provided by Strauss and Corbin (1990; see also Morrow & Smith, 1995), efforts were made to identify

Table 5 Obstacles to citizen–CPS partnerships identified by liaisons (N = 35) and CRP Members (N = 407). Categories of combined identified obstacles

Subcategories of combined suggestions

CRP activities and strategies

Subcategories of combined identified obstacles

Lack of knowledge/understanding regarding system/CPS General lack of knowledge Training needs/access for reviewers People are too busy Time limitations for both citizens and professionals General lack of transparency/unwillingness to change Attitude toward Resistant to outside views/dismissive of citizen input oversight/lack of Lack of agency response to CRP recommendations transparency/agency Lack of response to requests for information nonresponsive or One of many oversight bodies uncooperative Lack of resources Funding Resources for services/system Funding to support specific CRP expenses Extensive geographical distances affects cost of travel Staff Resources to address the CRP recommendations Role confusion/ Role/mission/expectations of CRP unclear clarification Lack of understanding of each others' roles (agency and CRP) General need for role clarification Lack of authority Accessibility/logistics/ Fitting participants' schedules scheduling Location/distance/transportation Too many demands on Provide feedback agency/workers Follow through/follow-up action Written agency reports and responses Listen to citizens CRP characteristics/ Follow through functioning Member characteristics Review process/forms CRP leadership Negotiation skills General lack of collaboration Lack of commitment to/appreciation of real Agency not committed to/appreciative of value of partnership partnership/token partnership Agency management CPS agency management attitude/leadership Lack of qualified leadership Lack of child welfare knowledge and understanding Time

Note: Categories of obstacles to citizen–child welfare partnerships identified by 10 or more CRP members AND four or more CRP liaisons.

CRP member self-education strategies Data collection recommendations Recommendation preparation Advocacy activities CW supportive activities Communication strategies with CW agency Develop child welfare understanding, Education on child welfare facts and knowledge, and member preparation issues Education on child welfare policies and laws Education on CW practices/services and resources Education on CW agency and system Communication Improved communication between agency and panel Open/honest communication Consistent/ongoing communication with feedback Community involvement/volunteerism Specific board/committee/program involvement Local community involvement Foster care Promote public awareness of agency/ Educate community system Positive publicity Encourage community connections Diversification and recruitment of Desirable membership characteristics members Diversity/representative of community Recruitment issues Consumer (client) recruitment/ membership Advocacy Policy/legislative advocacy System advocacy For services Child advocacy Funding/resources CW system funding Advocacy for funding CRP funding/stipends General resources Provide CRPs data and information Information flow from CPS Access to documents and data Focus of panel goals and objectives Clear/realistic goals and objectives Improved focus of panel efforts Develop relationship/trust with agency Mutual respect Promote commitment to partnership Cooperative approach Team concept Note: Categories identified by at least 10 or more CRP members AND four or more liaisons in response to a request for suggestions to improve the partnership between citizens and CPS agencies; “CW” = child welfare.

a central phenomenon of CRP–CPS relationships, causal conditions that led to the development of the central phenomenon, contextual conditions that affect the relationship, intervening conditions that influence the formation of strategies to improve the relationship, strategic actions to improve the relationship, and the consequences or outcomes from these strategies. Establishing the relationship of these themes to each other in a theoretical framework was facilitated by constant reference to the focused code within its original source text. The results of this axial coding process are summarized below, with relevant and typical comments from CRP members and liaisons offered for illustrative purposes. It should be noted that there was much more convergence than divergence noted on these key themes between the two respondent groups, and clear differences are noted within the narrative and the logic diagram. Therefore, the apparent disconnect suggested by the companion study may be less prominent than was suggested by the quantitative data. 5.2.1. The central phenomenon of relationship complexity Because respondents were directed to reflect specifically about the nature of the citizen–state partnership, it is not surprising that the

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

core phenomenon arising from the analyses is marked by complicated relations and ambivalent feelings about citizen involvement in child welfare. All aspects of the relationship as identified by causal and contextual properties reference this complexity, and in some instances, indicate outright confusion about the partnership and the purpose of the mandate. The nature of the CRP mandate may invite confusion and ambivalent feelings about citizen involvement, because if one reviews the mandate's charge, the purpose of CRPs is primarily evaluative in nature, necessitating acceptance of external critique. As one liaison noted, It requires a significant shift in CPS culture for the CPS system to embrace outside scrutiny. “Where there's a will, there's a way,” but leadership needs to lead by example and demonstrate that it truly values stakeholder input. (agency liaison) CRP members' comments further reveal the delicate and challenging parameters of this situation: CPS is guarded about giving citizens too much perceived power, competing interests among citizens. (CRP member) All the information panels receive is filtered through [redacted; child welfare agency name] upper management who have a vested interest in presenting [redacted] in the best light possible which can skew the information panels receive. (CRP member) Citizens don't know what has happened to their input, things continue on as is, and they become frustrated. (CRP member) Difficulties in navigating the contours of this apparently tenuous partnership surfaced in both CRP members' and liaisons' reflections as negative feelings, suspicion, and skepticism about the utility of citizen participation in child protection. [One barrier is] an attitude that child welfare agency folks have nothing good to offer, that they require oversight and correction (in a gotcha way!). (CRP member)

617

A “bunker” mentality of the legal community, including attorneys for CPS, that do not think transparency and reflection have any place in cps work. (agency liaison) Citizens are not empowered/accepted by the state or child welfare agency as a meaningful voice at the table. (CRP member) Department may not have clear vision of what we are charged with doing, i.e. local personnel wants us to redo the interview room. (CRP member) Lack of a clear understanding of the role of each and how they can complement one another for growth. (agency liaison) Lack of commitment/trust to the idea of a partnership between lay people and professionals. (agency liaison) CPS agencies or their staff do not provide meaningful involvement or information for panel members. (agency liaison) System's difficulty with/reluctance to true partnership with citizens. (CRP member) CRP members noted additional concerns regarding a lack of system transparency and trust, differing priorities, agency territorialism (“turfism”), and confidentiality policies which inhibit information transfer to CRPs. Panel members also discussed how the often poorly portrayed child welfare system may influence volunteers to possess a negative predisposition toward the agency they are charged to evaluate. Additionally, they noted difficulties with recruiting and engaging citizen volunteers in the mission of CRPs. [Redacted; child welfare agency name] needs to be more transparent about how they do business and educate the community about their rules and regulations. (CRP member) [One barrier is] the “us” and “them” philosophy, lack of trust by citizens and lack of respect from CPS workers, and lack of access. (CRP member) Secrecy cloaked in the excuse of preserving confidentiality. (CRP member)

Treat me as a partner rather than as an enemy. (CRP member) Confrontational styles and emotions sometimes get in the way of promoting true systems change. (CRP member) Citizens are frequently seen as time consuming and non-productive. (CRP member) [One barrier is] pointing fingers, placing blame instead of looking toward the common goal of better outcomes for children. And the lack of understanding that we all have that common goal but different perspectives on how to get to those improved outcomes, and different limitations. (agency liaison) 5.2.2. Causal conditions complicating the partnership Several specific conditions were identified by participants as influencing the development of this complex CRP–CPS relationship, including negative attitudes of CPS workers and CPS management, lack of role clarity, and lack of commitment to an authentic citizen–state partnership based on more than simply the statutorily mandated function. A sampling of comments reflecting these themes follows: [One barrier is] failure to be forthcoming with information that could cast a negative cloud around the system. You can only fix things and systems especially, if they are transparent. (CRP member)

There needs to be an intention to involve citizens and definite mechanisms for their involvement. (CRP member)

5.2.3. Contextual conditions affecting CRP–CPS relationships Contextual issues affecting the CRP–CPS relationship included the level of CRP member competence and diversity, the quality of panel leadership, panel funding as well as agency funding to address panel recommendations, and the extent to which communication between the agency and panel was perceived as open and bidirectional. Concerns about members' preparation and knowledge base within child welfare were the most prominent and frequent responses offered of all comments pertaining to barriers and suggestions to improve the CRP–CPS partnership. Citizens feel lost when all these agencies start talking about things and citizens have no idea what they're talking about. (CRP member) [One barrier is the] difficulty of explaining complex child welfare systems to panel members in sometimes short periods of time. (agency liaison) Of these contextual issues, communication was also a central theme identified primarily by CRP members.

618

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

Failure on the part of both to sit down and communicate about problems and successes. The child protective services workers do not meet with the CRP committee on a regular basis. (CRP member)

liaisons and CRP members identified a need for CRP members to become more educated globally on child welfare, in areas of policy, best practices, and child maltreatment knowledge.

Have a true dialogue with the state with the confidence that the state respects the opinions and requests of the panel. (CRP member)

Interact with caseworkers to better understand what actually happened within the investigation. Paper or electronic case does not really give you the complete picture. (CRP member)

Liaisons expressed concerns involving the need for panels to serve as independent third parties, though this issue was not raised to any meaningful extent by the members themselves. The problems with a lack of independence may be clearer to these liaisons working to serve as a bridge between the two entities and observing the ability of the panels to effectively promote change than to the members themselves who do not have the “insider” view. [We] need to have an outside person chair these panels completely, taking the responsibility off the local agencies. This can happen, but it is not happening and therefore little is being done. (agency liaison) CRP should be run outside of the child welfare agency. (agency liaison) [We need] affiliation with the School of Social Work. (agency liaison) Members identified issues related to citizen apathy, honesty and truthfulness of CPS agencies, and adversarial relationships with CPS. They also noted a need to have agency liaisons dedicated to panel service as a main responsibility. 5.2.4. Intervening conditions influencing development of strategies Other conditions impacting the formation of working strategies to effectively partner with CPS included the amount of time available to devote to collaboration, few resources dedicated to support CRP activities, scheduling problems for both volunteers and overburdened agency workers, and the level of perceived trust with agencies. Panel members also acknowledged that the slow process of change in the bureaucratic context of CPS agencies and fear of the “system” negatively influenced the development of successful joint panel–CPS endeavors, and that worker turnover impeded the growth of lasting, trusting relationships. They additionally noted that biases against CPS system clients themselves may unduly influence CRP members' perspectives and philosophies pertaining to the best ways to initiate systemic changes. I'd say the biggest obstacle I see is everyone's too busy in their own lives. (CRP member) People are too busy and they want quick solutions to complex social problems. (CRP member) [One barrier is the] lack of resources including time, money, and child protection workers, which directly impacts our ability to act proactively. (CRP member) [One barrier is the ] government's inability to be able to meet all the recommendations of the panels due to budget constraints or disagreement about policy direction. (agency liaison) State system is so large that common sense changes and decisions can't be made at the local level. (CRP member)

Have a good understanding of child welfare practices so they can make informed recommendations. (agency liaison) Ensure that members of the citizen review panels are well trained on the system and [on] limitations of child protective services. (CRP member) The two groups agreed that CPS information needed to be shared more openly with CRPs for them to fulfill their responsibilities more effectively, and that CRPs should spend more time advocating for needed child welfare services, resources, and prevention initiatives as agency partners. Both sets of respondents also agreed the CRPs needed clearer focus on their panel objectives as described in the mandate. Members also noted ways that child welfare agencies could facilitate a more effective partnership: making practice changes to improve effectiveness, collaborating with community agencies, and more consistently and completely responding to CRP recommendations. They also believed that they could more effectively contribute to improving outcomes by spending more time reviewing individual cases. Last, CRP members recommended that a clearer charge be provided to CRPs and child welfare agencies, either federally or by states, as to what the explicit responsibilities of CRPs are and the extent to which state agencies are accountable to their review efforts. 5.2.6. Consequences of potential strategic improvements The most important consequence of the successful implementation of these strategies expressed by participants appeared to be that citizen volunteers would be in a better position to effectively contribute in a knowledgeable way to the partnership. By far, the thematic category most often referenced and reflected upon was the knowledge base that CRP members and new recruits possess. Ways in which volunteers could become more educated and informed about relevant child welfare issues and systemic concerns varied widely across responses, but many involved a combination of volunteers proactively seeking out this information and agencies offering more direct training and access to knowledge. The resounding implication of the importance of this theme to participants is that both CPS and CRPs believe that CRP members are not all equally prepared or knowledgeable enough to competently engage as partners with child welfare agencies to protect children and to improve the system. Have a good understanding of child welfare practices so they can make informed recommendations. (agency liaison) To be cognizant of agency policy and procedures so that recommendations are meaningful. (agency liaison) In order to make suggestions that will improve the system, you need to understand where the problems lie. (CRP member) Better education/understanding regarding poverty, drug addiction and recovery, racial inequalities, and resource availability. (CRP member)

[One barrier is] prejudice against “welfare.” (CRP member) 5.2.5. Strategies to improve CRP–CPS partnership effectiveness Many strategies were offered as ways to more effectively work together as partners and to improve the CRP–CPS relationship. Both

Getting to know the CPS process stemming from calls received up to but not limited to what would institute removal of children. (CRP member) A second potential consequence or outcome of greater importance to CRP members than to liaisons was that of forming a shared vision

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

within panels and with CPS about how to work together and strategically plan initiatives that serve children within their communities and improve child welfare outcomes. Many CRP members expressed a strong desire to be viewed as genuine partners in this challenging endeavor, and not as adversaries or only as watchdogs. [We have] different agendas though child protection is ultimate goal of each. (CRP member) Disparity between agency priorities for the CRP and the priority areas selected by the CRP. (CRP member) Work from the perspective that we are all in this together, looking at how we as a community can improve child safety, and not as one group reviewing and critiquing the role of another. (CRP member)

619

vision exists to strengthen families and communities. In this relationship, state agents desire and benefit from an authentic partnership with citizens, as its value is recognized. The authenticity of the partnership is demonstrated to citizen volunteers by state leadership through honest and timely communication through which useful information is shared in preparation of and in response to the recommendations for systems change made by the panels. Genuinely engaged citizen volunteers contribute thoughtful, carefully considered insights and recommended solutions to improve the child welfare system that are in turn, seriously deliberated upon by state agents and adopted when possible and practical to do so. When key components of mutual respect, legitimacy, child welfare knowledge, shared vision, authenticity, citizen engagement, honest and open communication, and a serious, deliberative process are in any way impeded or absent, the partnership is vulnerable to breakdowns and ineffectiveness and its intended outcomes may be compromised.

Provide a framework of mutual goals for increased effectiveness. (CRP member) 6. Discussion and conclusion The emergent theoretical model developed from and grounded within these data is summarized and displayed in a logic diagram, Fig. 1.

5.3. An emergent, substantive-level theoretical statement on citizen–state child welfare partnerships These data imply the following substantive-level theory: the CRP– CPS citizen–state partnership is more likely to be successful when knowledgeable citizens are legitimately engaged in a working relationship reflecting mutual respect with state agents and a shared

A potentially interesting dynamic has surfaced from the analysis. While CRP members appear to want to be respected and valued by agencies, concerning themselves with their acceptance by CPS agencies may detract from their ability to fulfill perhaps their most important function in relation to child welfare, that of external reviewer and evaluator of services and system outcomes. This was clearly the main role they were intended to fulfill as explicated by the CAPTA mandate. The fact that they were designed to fulfill an evaluative function predestines a certain amount of tension between the parties. The nature

Context

Causal Conditions Agency attitude toward partnership Lack of agency commitment to partnership CPS management attitude Unclear role and confusion Confidentiality concerns Distrust Differing priorities Negative CPS public image Territorialism CRP recruitment problems CRP member engagement Lack of system transparency

CRP member competence CRP leadership skills Diverse membership Funding Level of open communication CRP lack of independence from child welfare agency Dedicated agency liaison CPS honesty/truthfulness Citizen apathy Adversarial agency/panel relationship

Phenomenon Citizens partnering with CPS in a complex, evaluative relationship through CRP mandate

Intervening Conditions

Key Plain text: key themes of both members and liaisons Italics: key theme of liaisons Underline: key themes of CRP members

Time to devote to partnership Resources Scheduling Agency demands on workers Level of trust with agency Fear of the system Bureaucracy Worker turnover Prejudices/biases toward system clients

Strategies CRP members educating themselves on policy/practice issues Raising awareness and promoting CRPs Advocating for agencies and children Allowing access to information for CRPs Improving panel focus on objectives Agencies making practice changes and improving effectiveness Agencies collaborating with other agencies and communities Agencies responding to recommendations Reviewing cases Providing a clearer charge for CRPs

Fig. 1. Logic diagram of CRP–CPS relationship.

Consequences Improving ability of citizen volunteers to knowledgeably contribute Sharing a vision and goals

620

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621

and purpose of evaluation in any context tends to foster resistance, and in some unfortunate circumstances, outright hostility and antagonism toward those charged with the monitoring or evaluative responsibility. Resistance to evaluation is so common, in fact, that professional evaluators have coined a term for it, evaluation anxiety (Scriven, 1991). Negative tensions between the evaluator and the entity under evaluation may result from concerns regarding the legitimacy of the evaluation and those conducting it (Donaldson, Gooler, & Scriven, 2002). Evaluation resistance may also result from a fear that one's reputation or even livelihood may be in jeopardy if findings are negative. Herein may exist the largest source of tension between CPS and CRPs. Perhaps at least some of the concerns about the preparation and knowledge base of CRP members are not really about their competence to participate, but rather their level of perceived legitimacy to evaluate child welfare services. Given the level of education and experience of panelists and data indicating that training is provided to them, the persistent desire for more knowledge seems to have a deeper and more complex source. Statements indicating that volunteers need more basic knowledge about how things operate in child welfare bureaucracies might serve a self-protective function, by maintaining the status quo of power distribution. However, this speculation does not explain why so many CRP members themselves identified a fundamental lack of child welfare knowledge as a barrier. Additionally, this perception may be an effort to explain why the panel's considered review and recommendations may not be resulting in the desired response by the agency. It is possible that volunteers may have perceived CPS agencies' lack of feedback or follow through on CRP recommendations as indications they were not offering helpful or useful information. Additionally, the complexity of child welfare bureaucracies is unquestionably challenging to unravel and interpret, and it is reasonable to assume that it would be intimidating to anyone, particularly to those new to these systems. Nonetheless, it is important to note that volunteers in this study averaged 18 years of education, equivalent to a master's degree, suggesting that they are as a group highly capable of learning. In many cases, they possess far more education than front-line workers providing child protective services. If lack of volunteer knowledge impedes competent participation, CPS should be obligated to offer access to that knowledge, as this clearly trainable group would benefit from that, as may the perceived value of the recommendations made. If embedded skepticism about evaluation and perceived legitimacy of the CRP evaluator role are also at issue, however, more open discussion regarding this complex matter should occur to bring it to light. The perspective of agency liaisons might suggest a third, reasonable possibility: the perceived lack of CRP knowledge and agency skepticism toward CRP evaluation may be inextricably linked in terms of how the partnership is viewed by agencies. As the key information conduit between CPS and CRPs, liaisons are the only involved individuals with first-hand knowledge of how agencies receive CRP input and how CRPs view the CPS agencies they evaluate. Knowledge of this difference in perceptions may very well have helped create the weak relation between liaisons' CRP effectiveness ratings and CRPs' self-ratings of effectiveness identified in the 2010 study (citation withheld for blind review, added upon notice of acceptance).While both CRP members and liaisons raised questions about perceived CRP members' competence to contribute meaningful input, liaisons commonly identified a lack of agency commitment to the partnership and transparency, as well as negative and disengaged attitudes of agency leadership toward citizen involvement in child welfare, as critical barriers to successful relationships with CRPs. Perhaps liaisons strongly suggested that CRP members should acquire more child welfare education in order to heighten CRP members' level of perceived legitimacy as participants and evaluators in the eyes of agency leaders. The study is limited by the fact that though these and other emergent areas from the analytic results could offer more pointed

direction for further study, the protocol required closing of the project in December 2008. However, a path for targeted inquiry has been laid, and it appears evident that more direct recruitment of CPS workers and administrators at various levels in future studies is necessary to explore these issues from the agency perspective. Results of this study help place into context prior findings revealing a disparity in panel and state agency impressions of CRP effectiveness. These findings have uncovered several important relational difficulties that have formed barriers between volunteers and CPS agencies, and suggest specific issues to target to improve involvement of CRPs in child welfare, and to foster better citizen–state relationships. These results can prompt a more informed discourse about the challenges and opportunities presented when attempting to democratically engage citizens in child welfare practice and policymaking, and can lead to interesting, new research paths. It is suggested that it may be a worthwhile venture to next investigate these relationships by studying cases of successful CRP–CPS partnerships, to further test and evaluate the emergent conceptual model derived from these findings. These types of studies may in time reveal if effective citizen–state partnerships can help build better child welfare systems, and ultimately may allow stakeholders to truly assess the worth of the citizen review panel initiative. References Administration for Children and Families (1998). Establishment of the Citizen Review Panel Requirement Under the Child Abuse Prevention and Treatment Act. January 7, 1998. U.S. Department of Health and Human Services. Barton, P. G. (1970). Civilian review boards and the handling of complaints against the police. University of Toronto Law Journal, 20(4), 448−469. Blome, W. W., & Steib, S. (2007). An examination of oversight and review in the child welfare system: the many watch the few serve the many. Journal of Public Child Welfare, 1(3), 3−26. Box, R. C. (1998). Citizen governance: Leading American communities into the 21st century. Thousand Oaks, CA: Sage Publications. Bryan, V., Jones, B. L., Allen, E., & Collins-Camargo, C. (2007). Civic engagement or token participation? Perceived impact of the citizen review panel initiative. Children and Youth Services Review, 29(10), 1286−1300. Bryan, V., Jones, B., & Lawson, E. (2010). Key features of effective citizen-state partnerships: Findings from a national study of citizen review panels. Children and Youth Services Review, 32(4), 595−603. Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: Sage. Creighton, J. L. (1999). Public participation in federal agencies' decision making in the 1990's. National Civic Review, 88(3), 249−257. Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage Publications. Denhardt, R. B., & Denhardt, J. V. (2000). The new public service: serving rather than steering. Public Administration Review, 60, 549−559. Donaldson, S., Gooler, L., & Scriven, M. (2002). Strategies for managing evaluation anxiety: toward a psychology of program evaluation. American Journal of Evaluation, 23(3), 261−273. Glaser, B. (1964-1965). The constant comparative method of qualitative analysis. Social Problems, 12, 436−444. Jones, B. (2004a). Citizen participation in public child welfare: a multi-state study of citizens review panel members' perceptions of effectiveness. Dissertation Abstracts International, 65 (07), 2765A. (UMI No. 3140108). Jones, B. (2004b). Effectiveness of citizen review panels. Children and Youth Services Review, 26(12), 1117−1127. Jones, B., Litzelfelner, P., & Ford, J. (2003). The value and role of citizen review panels in child welfare: perceptions of citizen review panel members and child protection workers. Child Abuse & Neglect: The International Journal, 27(6), 699−704. Jones, B. L., & Royse, D. (2008). Citizen review panels: the connection between training and perceived effectiveness. Child Abuse & Neglect: The International Journal, 32(10), 918−919. Jordan, C., & Franklin, C. (1994). Have external review systems improved the quality of care for children? In E. Gambrill & S. Stein (Eds.), Controversial Issues in Child Welfare (pp. 136−149). Boston, MA: Allyn & Bacon. King, C. S., Felty, K. M., & Susel, B. O. (1998). The question of participation: toward authentic public participation in public administration. Public Administration Review, 58(4), 317−326. Koontz, T. (1999). Administrators and citizens: measuring agency officials' efforts to foster and use public input in forest policy. Journal of Public Administration Research, 9(2), 251−269. Kot, V., Bruner, C., & Scott, S. (1998). Citizen review panels for the child protective services system: Guidelines and protocols. Chicago, IL: National Committee to Prevent Child Abuse. LeCompte, M. D., & Schensul, J. J. (1999). Analyzing & interpreting ethnographic data. Walnut Creek, CA: Sage Publications.

V. Bryan et al. / Children and Youth Services Review 33 (2011) 612–621 Lynn, F. M., & Kartez, J. D. (1995). The redemption of citizen advisory committees: a perspective from critical theory. In O. Renn, T. Webler, & P. Wiedemann (Eds.), Fairness and competence in citizen participation: evaluating models for environmental discourse. Boston: Kluwer Academic Publishers. Morrow, S. L., & Smith, M. L. (1995). Constructions of survival and coping by women who have survived childhood sexual abuse. Journal of Counseling Psychology, 42, 24−33. Osborne, D., & Gaebler, T. (1992). Reinventing government: how the entrepreneurial spirit is transforming the private sector from school house to state house, city hall to Pentagon. Reading, MA: Addison-Weseley. Putnam, R. D. (1993). Making democracy work: civic traditions in modern Italy. Princeton, NJ: Princeton University Press. Schacter, H. L. (1995). Reinventing government or reinventing ourselves: two models for improving government performance. Public Administration Review, 55(6), 530−537. Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage Publications.

621

Stivers, C. (1990). Active citizenship in public administration. In G. L. Wamsley, R. N. Bacher, C. T. Goodsell, P. S. Kronenberg, J. A. Rohr, C. M. Stivbers, O. F. White, & J. F. Wolf (Eds.), Refounding public administration (pp. 246−273). Thousand Oaks, CA: Sage Publications. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage. Strauss, A., & Corbin, J. (1998). Basics of qualitative research: techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage. Timney, M. M. (1998). Overcoming administrative barriers to citizen participation: citizens as partners, not adversaries. In King, & Stivers (Eds.), Government is us: public administration in an anti-government era. Thousand Oaks, CA: Sage. Verba, S., Schlozman, K., & Brady, H. (1995). Voice and equality: civic voluntarism in American politics. Cambridge, MA: Harvard University Press. Waldfogel, J. (1998). The future of child protection: how to break the cycle of abuse and neglect. Cambridge, MA: Harvard University Press.