Building a unified system for universal Pre-K: The case of New York City

Accepted Manuscript Building a unified system for universal Pre-K: The case of New York City

Jeanne L. Reid, Samantha A. Melvin, Sharon Lynn Kagan, Jeanne Brooks-Gunn PII: DOI: Reference:

S0190-7409(18)30956-3 https://doi.org/10.1016/j.childyouth.2019.02.030 CYSR 4224

To appear in:

Children and Youth Services Review

Received date: Revised date: Accepted date:

30 October 2018 15 February 2019 16 February 2019

Please cite this article as: J.L. Reid, S.A. Melvin, S.L. Kagan, et al., Building a unified system for universal Pre-K: The case of New York City, Children and Youth Services Review, https://doi.org/10.1016/j.childyouth.2019.02.030

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT Building a Unified System for Universal Pre-K: The Case of New York City Jeanne L. Reid, Ed.D., Samantha A. Melvin, Sharon Lynn Kagan, Ed.D., Jeanne Brooks-Gunn, Ph.D.

T

Corresponding Author: Jeanne L. Reid, Ed.D. Research Scientist, National Center for Children and Families, Teacher’s College, Columbia University, 525 West 120th Street, Box 226, New York, NY 10027 [email protected]

CR

IP

Samantha Melvin Graduate Research Fellow, National Center for Children and Families, Teachers College, Columbia University, 525 West 120th Street, Box 226, New York, NY 10027 [email protected]

AN

US

Sharon Lynn Kagan, Ed.D. Virginia and Leonard Marx Professor of Early Childhood and Family Policy, Co-Director of the National Center for Children and Families, Teachers College, Columbia University, 525 West 120th Street, Box 226, New York, NY 10027 [email protected]

ED

M

Jeanne Brooks-Gunn, Ph.D. Virginia and Leonard Marx Professor of Child Development and Education, Co-Director of the National Center for Children and Families, Teachers College, Columbia University, 525 West 120th Street, Box 226, New York, NY 10027 [email protected]

PT

The study was generously supported by the Foundation for Child Development in New York City. The Foundation did not play a role in the design and conduct of the research, or in the interpretation of the results presented here.

CE

The authors have no competing interests related to the content or purpose of the manuscript. Abstract

AC

In recent decades, state funding for preschool programs has more than tripled as more states are trying to create universal access to pre-k. Efforts to expand pre-k access typically include the use and coordination of multiple preschool settings, notably those found in public schools and private community-based organizations, which often have distinct goals, resources, requirements, funding sources, and reporting systems. In this challenging context, the present study examined the empirical example of an ambitious policy initiative to provide high-quality universal pre-k (UPK) in New York City. Inheriting a mixed-delivery system for UPK provision, the city employed substantial resources to increase and align quality across programs in different settings and with varied auspices. Comparing program and classroom data, the analysis identified significant variation by program setting and auspice that appears to have roots in systemic obstacles that are far from unique to New York City. The findings could inform policymakers

ACCEPTED MANUSCRIPT nationwide who are striving to build effective systems for the provision of equitable, high-quality early childhood education.

AC

CE

PT

ED

M

AN

US

CR

IP

T

Keywords: pre-kindergarten, child care, mixed delivery, equity, quality

ACCEPTED MANUSCRIPT Building a Unified System for Universal Pre-K: The Case of New York City February 15th, 2019 Keywords: pre-kindergarten, child care, mixed delivery, equity, quality 1. Introduction and background

AN

US

CR

IP

T

In recent decades, the provision of early childhood education has expanded dramatically across the country (Phillips, Anderson, Rupa Datta, & Kisker, 2018). State funding for pre-kindergarten programs that focus mostly on 4-year-olds has more than tripled from $2.4 billion in 2002 to over $7.6. billion in 2017 (Friedman-Krauss et al., 2018). As public investment has grown, more states and cities are trying to create universal access to their pre-k programs. About half of the 50 states in the U.S. and the District of Columbia have established “universal” pre-k programs for which eligibility is determined regardless of family income (Friedman-Krauss et al., 2018). Ten states enrolled at least 50% of their 4-year-olds in public preschool programs in 2017, compared to only two states in 2002, and five states have sufficiently funded their programs to enroll at least 70% of their 4-year-old children.

CE

PT

ED

M

Efforts to expand access to pre-k typically include the use and coordination of multiple preschool settings, most notably those found in public schools and private community-based organizations. Historically, programs in these settings have distinct goals, resources, requirements, funding sources, and reporting systems. Supported by federal funding from diverse sources, including Title I school funding, Head Start funding, and child care subsidies via the Child Care and Development Block Grant, services for young children may be further augmented by state and local contributions, as well as private funding. Often these funds are administered by different government authorities. Such a heterogenous context can offer varied choices for parents, but it also poses steep challenges for policymakers who seek to establish a coordinated, efficient, and equitable approach to universal pre-k (Schulman & Blank, 2007; Wat & Gayl, 2009). One concern is that disparate funding levels, which are often higher for schools than for communitybased organizations, may contribute to a two-tiered system of funding, leading to unequal pre-k quality (Holcomb, 2006; Weiland, 2018).

AC

Given this fragmented landscape, state and local policymakers are striving to establish universal pre-k (UPK) programs that offer high levels of quality system-wide. The present study empirically examines an example of one such policy initiative, Pre-K for All, in New York City. Launched in 2014-15, Pre-K for All (PKA) sought to offer consistently high-quality UPK services to every 4-year-old in the city. Like many other cities and states, the city inherited a mixed-delivery system for PKA provision in which the city’s Department of Education oversees school-based programs and the city’s Administration for Children’s Services oversees most community-based programs. To identify similarities and differences in PKA programs across school and community-based locations, we collected data via surveys of PKA administrators and lead teachers in both settings, and observations of their classroom environments and teacherchild interactions. We then used a systems lens to analyze the data. With this descriptive study,

ACCEPTED MANUSCRIPT we aim to inform policymakers across the country who are striving to build effective systems for the provision of equitable, high-quality early childhood education. 1.1. The potential of high-quality pre-k programs

US

CR

IP

T

As the advent of UPK is fairly recent, the evaluation literature is based on small-scale pre-k programs as well as the national Head Start program. This body of research is typically based on pre-k programs that target low-income children and cohorts of children from the 1960s through the 1980s, when fewer center-based options were available. Together, the research indicates that high-quality pre-k programs are associated with short-, medium-, and long-term impacts on children’s learning and achievement (Heckman, 2011; Yoshikawa, Weiland, & Brooks-Gunn, 2016). The gains accrue to all groups of children, although low-income children appear to benefit the most, and as children move through elementary school, the gains tend to get smaller (Yoshikawa et al., 2016; Gormley, Gayer, Phillips, & Dawson, 2005). At the same time, highquality preschool programs have been linked to positive long-term life outcomes that outweigh the initial costs of preschool provision (Bartik, Belford, Gormley, & Anderson, 2016; Duncan & Magnuson, 2013; Karoly, 2016).

AN

1.2. The expansion of pre-k programs

PT

ED

M

Such evidence has motivated policy efforts to extend access to all 4-year-old children, regardless of their family income. While most pre-k programs still target poor children, more states and cities are trying to establish universal access to high-quality programs (Friedman-Krauss et al., 2018; Rebell & Wolff, 2017). Compared to targeted programs, universal programs may attract a population of children that is more diverse by family income as well as by children’s skills, which may in turn attract better teachers who provide the responsive interactions that promote children’s learning (Burchinal, 2018; Hanushek, Kain, & Rivkin, 2004). Consistent with social learning theory (Bandura, 1971), preschool classrooms with a mix of skill levels may also confer peer effects that foster children’s learning (Henry & Rickman, 2007; Justice, Petscher, Schatschneider, & Mashburn, 2011; Weiland & Yoshikawa, 2014).

AC

CE

If states and cities can create universal access to programs of consistently high levels of quality, they could narrow the substantial disparities in early skills that have been identified among poor and non-poor children at kindergarten entry (Cascio, 2017, Magnuson & Waldfogel, 2005; Valentino, 2018). This statement assumes that poor children benefit more from their participation in high-quality programs than higher-income children. Yet, poor children have been more likely to be in lower quality programs than middle-income children, at least prior to the advent of UPK. Relegating poor children to programs of generally lower quality would likely aggravate disparities between lower and higher income children (Kagan, 2009). A central question is: how feasible is it to scale up a pre-k program to serve all children in a state or city and simultaneously achieve consistent levels of quality? The findings on the scaling-up of pre-k programs are mixed. While small-scale experimental preschool programs have been associated with substantial gains in children’s learning and achievement, more recent evaluations of broader-based programs are less consistent (Duncan & Magnuson, 2013; McCoy et al., 2017). Evaluations of universal programs in Boston and Tulsa have found the strongest gains, while an

ACCEPTED MANUSCRIPT evaluation of Tennessee’s expanded pre-k program, which targets poor children, found either no effects or adverse effects (Gormley, Phillips, & Anderson, 2018; Lipsey, Farran, & Durkin, 2018; Weiland & Yoshikawa, 2013). Such outcomes have renewed efforts to identify effective policies to take pre-k to scale while fostering its quality (Weiland, 2018; Yoshikawa et al., 2018). 1.3. What constitutes quality?

M

AN

US

CR

IP

T

To inform these policy efforts, abundant empirical research in preschool settings has sought to identify what exactly constitutes preschool “quality” and how to measure it. The scholarship on early childhood education has discerned structural elements of program quality, such as teacher qualifications, class size, curriculum use, and teacher-child ratios, which are comparatively easy to measure (Burchinal, 2018). In turn, such structural components support process components of quality, i.e., the sensitive and stimulating teacher-child interactions that are harder to measure, but the most reliable predictors of children’s learning (Burchinal, 2018; Johnson, Markowitz, Hill, & Phillips, 2016; Pianta, Downer, & Hamre, 2016). Structural and process components thus interact to render overall “pedagogical quality,” the nurturance of young children and promotion of their learning and development. In addition, the Head Start model emphasizes family engagement in program activities and governance, as well as the provision of comprehensive social and health services to children and families as components of program quality (Administration for Children & Families, 2017). How family engagement and such services relate to gains in children’s learning is not well understood (Camilli, Vargas, Ryan, & Barnett, 2010; Magnuson & Schindler, 2016).

CE

PT

ED

Pedagogical quality is supported by well-trained teachers who are adequately compensated (Phillips, Austin, & Whitebook, 2016). Though teacher education levels have not been consistently linked to children’s learning gains (Early et al, 2007), low compensation for the early education workforce, relative to K-12 teachers, has been found to foster teacher stress and turnover, making it difficult for providers to attract and retain their best teachers (King et al, 2015; Whitebook, Phillips, & Howes, 2014). Less attention has been paid to the qualifications and compensation of preschool program directors, though effective program leadership is thought to be a critical accelerator of program quality (Institute of Medicine & National Research Council, 2015; Rohacek, Adams, & Kisker, 2010).

AC

In an effort to improve quality, all 50 states and the Head Start program have developed early learning standards to guide decisions regarding curriculum, assessments, and teacher practices (ACF, 2015; Scott-Little, Kagan, & Frelow, 2003). Many states and localities have also devoted greater resources to in-service training and individualized coaching for early education teachers (Kraft, Blazar, & Hogan, 2018; IOM & NRC, 2015; Cannon, Zellman, Karoly, & Schwartz, 2017). Coaching has been found to support teachers’ effective use of domain-specific curricula (Marti, Melvin, Noble, & Duch, 2018; Phillips et al., 2017). Additionally, the 2007 reauthorization of the Head Start program required that half of its lead teachers have B.A. degree (ACF, 2008). To sustain preschool learning gains, policymakers have further sought to foster children’s transitions to formal schooling with preschool-to-school collaborations and improved curricular alignment (Kagan & Tarrant, 2010; LoCasale, Mashburn, Downer, & Pianta, 2008). 1.4. Systemic obstacles to pedagogical quality

ACCEPTED MANUSCRIPT

CR

IP

T

Despite such efforts, systemic obstacles may hinder the achievement of pedagogical quality. As noted, early education services are often overseen by diverse ministries, departments and/or agencies, such as those with responsibility for education, human service, community development, welfare, and/or social services. This divided governance structure is often characterized by quite different policies and regulations that apply to programs with varied histories, missions, and capacity. A lack of administrative cohesion translates into services for children and families that are less than optimal quality, inequitably distributed, and inefficiently operated (Kagan & Gomez, 2015). Policies to extend UPK are likely to confront and potentially confound these challenges when they engage schools and community-based organizations to meet broad-based demand (McCabe & Sipple, 2011; Schulman & Blank, 2007; Stephens, 2014). Efforts to establish UPK programs, such as Pre-K for All in New York City, thus represent an opportunity to consider how strategies to scale-up pre-k programs may be differentially realized within mixed-delivery systems and among diverse providers. 1.5. New York City’s Pre-K for All initiative and the present study

M

AN

US

New York City’s Pre-K for All initiative sought to make the state’s UPK program, inscribed into New York State law in 1997, a reality in the city. Starting in the 2014-15 school year, the city more than tripled the number of children enrolled in full-day pre-k from 19,287 children prior to the expansion to 68,647 children in 2015-16, representing about 70% of all New York City 4year-olds (Barnett et al., 2016). In 2014-15 alone, funding for pre-k in the city increased by some $358 million, representing almost two-thirds of the national increase in pre-k funding that year (Barnett et al., 2016).

AC

CE

PT

ED

Implementing PKA demanded coordination across the two city agencies tasked with primary operational responsibility for the program, the Department of Education and ACS, as well as the Office of the Mayor and the Department of Health and Mental Hygiene, which licenses child care sites in the city. Within this governance structure, PKA slots exist in two very different settings: 1) those within public, charter, and private schools; and 2) those within private community-based organizations. Further complicating matters, PKA funds flow to communitybased organizations on two paths: through a contract with the Department of Education, or through a contract with the Administration for Children’s Services, which combines various funding sources, primarily Child Care, Head Start, and/or PKA. Community-based organizations may also have separate but concurrent contracts for PKA with the Department of Education and the Administration for Children’s Services. Given these multiple contracting arrangements, the auspices that govern community-based organizations can vary considerably. Compounding these challenges, variation in the resources and goals of long-standing programs complicate efforts to create system-wide fidelity to the PKA model and consistently high levels of quality. The city responded with a multi-pronged strategy to increase access while enhancing and aligning program quality (New York City Department of Education & Administration for Children’s Services, 2015; Westat et al., 2016), which included:  Providing policy statements that applied to all settings (New York City DOE & ACS, 2015); additionally, Head Start programs must abide by federal Head Start program requirements and standards (e.g., Administration for Children & Families, 2016).

ACCEPTED MANUSCRIPT

   

T

IP

PT

 

CR



US



AN



M



Raising education and credential requirements for all lead teachers, regardless of setting. Lead teachers must have a teaching license or certificate valid for service in early childhood; or a teaching license or certificate for students with disabilities valid for service in the early childhood grades; or a bachelor’s degree in the early childhood grades or related field and a study plan to obtain early childhood certification within three years. Requiring all programs to implement a pedagogy that emphasizes “purposeful play and exploration through student-initiated and teacher facilitated activities [and] opportunities for students to engage in individual, small, and whole-group learning through a balance of student-initiated and teacher-facilitated activities.” Providing early learning standards that applied to all settings (New York State Department of Education, 2011); additionally, Head Start programs must use the federal Head Start early learning standards (ACF, 2015). Requiring all PKA teachers to use curricula that align with the PKA early learning standards. In addition to publicly available curricula such as the Creative Curriculum, PKA programs could use curricular “units of study” developed by the Department of Education for PKA programs. Requiring all programs to use an authentic assessment system to monitor developmental progression and plan instruction. Providing differentiated professional development for PKA teachers, including instructional coaching. Providing professional development opportunities to PKA administrators. Offering additional “enhancement” funding to community-based organizations that could be used for multiple purposes, including to increase teacher salaries. Encouraging all sites to engage families in program activities by establishing parentfriendly communication channels and initiating regular communication. Employing social workers who each support teachers in several PKA sites. Providing funding for “full-day” services (6 hours and 20 minutes per day day) for all children for 180 days per year.

ED



AC

CE

The purpose of the study is to identify similarities and differences in PKA programs across school and community-based programs (CBOs). Specifically, using data from administrators, teachers, and classroom observations in varied settings, we compare programs located in schools and CBOs, and programs located in CBOs with different funding auspices, i.e., CBOs that receive only PKA funds from the Department of Education, CBOs that receive PKA and Child Care funds from the Administration for Children’s Services, and CBOs that receive PKA and Head Start funds (and usually Child Care funds as well) from the Administration for Children’s Services. The analysis is grounded in the hypothesis that PKA programs are likely to vary by setting, given differences in the capacity, resources, and missions associated with schools and CBOs, as well as by auspice, given the inherent administrative and pedagogical challenges of managing different funding streams with myriad purposes and requirements. 2. Conceptual framework The analysis is framed by a theory of change (Kagan, 2015), which takes a systems perspective on the provision of high-quality early education. The theoretical framework identifies eight components that are thought to be necessary to propel the realization of an effective early

ACCEPTED MANUSCRIPT education system: 1) direct services to children and families, and seven other components that support those services, known collectively as the system’s infrastructure: 2) governance; 3) finance; 4) pedagogical quality; 5) workforce development; 6) accountability; 7) family engagement; and 8) linkages to other services and children’s transitions to kindergarten. As elaborated by Kagan and Gomez (2015), pedagogical quality is a component of the infrastructure, while overall systemic quality is an outcome associated with the constellation of services and their supportive infrastructure.

US

CR

IP

T

We collected data relevant to these eight components and then used them as an analytic frame for interpreting the data. Importantly, we recognize that the present study collected data at the classroom and program level, including many but not all elements of an early education system. In analyzing the rich set of data, we found multiple results that appeared to reflect systemic challenges. For example, questions on program characteristics and experience with PKA rendered data that pertained to system-level governance and finance; questions on staffing and compensation rendered data pertaining to the financing and development of the early education workforce; and observations regarding classroom quality rendered data related to system-wide differences in child enrollment, teacher recruitment, working conditions, and teacher retention.

M

AN

In sum, we use a systems framework for analyzing the data with the rationale that the PKA effort: 1) is part of a system for early education services; 2) is affected by this system; and 3) needs to be considered within a systems framework if it is to achieve its entwined goals of quality and equity.

ED

3. Data and method

We use a systems lens to look at PKA programs in different settings and auspice. The analyses were guided by two research questions:

CE



How do PKA programs compare by setting in terms of key components of an effective early education system? How do PKA programs compare by auspice in terms of key components of an effective early education system?

PT



AC

To answer these questions, we utilized a mixed-methods design, incorporating: 1) quantitative and qualitative survey data from administrators and lead teachers at PKA sites; 2) Classroom Assessment Scoring System (CLASS) observations in each teacher’s classroom; and 3) child demographic data. 3.1. Site sampling Because this study was one of several of a consortium of studies on the city’s PKA initiative, site sampling was conducted in two stages. First, an independent research organization (the MDRC) was engaged by the funder to conduct a coordinated sampling approach to assign PKA sites across nine Community Districts (CDs) in New York City to each of the study teams. The nine CDs represented three resource levels (three CDs at each resource level), which MDRC categorized as low, moderate, and high, using data on residents’ income-to-needs ratio,

ACCEPTED MANUSCRIPT concentration of racial/ethnic minorities, and passing rates on an English Language Arts assessment. Lower-resource communities had higher income-to-needs, higher minority concentrations, and lower passing rates on the English Language Arts assessment, while higher resource-communities were the opposite on each of these dimensions. This approach targeted three goals: 1) to collect data in neighborhoods with diverse resource levels; 2) to foster generalizability across study teams; and 3) to avoid burdening programs with multiple research activities. Through the coordinated effort, we were assigned a pool of 150 sites (distributed roughly equally across the three resource-level categories) and began recruiting participants.

IP

T

3.2. Site recruitment

ED

M

AN

US

CR

The study was implemented with support from the New York City Department of Education (DOE), Administration for Children’s Services (ACS), and Department of Health and Mental Hygiene during the 2016-17 school year. In the fall of 2016, the three agencies co-authored and sent a letter to the director or principal at each of the 150 sites to inform them about the study and encourage them to participate. The research team then contacted the designated person by phone or email, provided information about the study, determined basic interest, and identified the target “administrator” who had primary responsibility for staffing and operations at the site. At schools, this person was typically the Principal or an Assistant Principal who had primary oversight of the school’s PKA program; at CBOs, this person was typically the “Director” or sometimes the “Educational Director,” who often oversaw the site as an employee of a larger chain or organization of affiliated sites. This initial contact was followed by an in-person meeting to explain the study in more depth, obtain informed consent, and collect data regarding the funding received by the site for each classroom serving 4-year-old children. Because seven sites were no longer offering PKA services, our sample universe was 143 sites.

AC

CE

PT

For our sampling frame, we stratified sites to approximate the distribution of all school-based and CBO-based PKA sites across the nine CDs chosen for sampling, which in the aggregate was 37% in schools and 63% in CBOs. Accordingly, the final analytic sample of 57 sites was comprised of 22 schools (39%) and 35 CBOs (61%). Among the CBOs, we sampled to ensure that we had sufficient numbers of the three auspice-types to allow for statistical comparison. Of the 143 sites in the full sample, 41 sites (29%) declined to participate. Of these, 21 sites stated that they were too busy or overburdened, 13 sites declined for other reasons, and seven sites gave no reason. Additionally, despite concerted efforts, the research team could not sustain contact with 45 sites (31%). The overall response rate was therefore 40%, or 57 sites. We concluded the recruitment phase when we determined that we had approximated the actual distribution of sites in schools and CBOs, and when we had sufficient sites to compare each auspice type. As shown in Table 1, for analytic purposes, we grouped sites into two types of settings, schools and CBOs, and then among the CBOs, in three auspice categories. Table 1 here Of the 22 schools, 17 were public schools, three were private schools, one was a charter school. (One was a Pre-K Center, a stand-alone PKA center created by the DOE in which staff report to DOE superintendents and teachers belong to the same union as teachers in public schools.) Of

ACCEPTED MANUSCRIPT the 35 CBOs, 29 were affiliated with a chain or larger organization and six were independently operated. We categorized 13 of the CBOs as Head Start sites because they received Head Start funding, in addition to PKA and any Child Care funding, to distinguish them from the 16 CBOs that received only PKA and Child Care funding. DOE-only CBOs received no Child Care or Head Start funding for 4-year-olds. 3.3 Classroom sampling and teacher recruitment

AN

US

CR

IP

T

In schools and DOE-only CBOs, one classroom was randomly selected (28 classrooms from 28 sites). In CBOs with multiple funding streams (i.e., funding from a mix of PKA, Child Care, and/or Head Start) that were equally distributed across classrooms (i.e., all 4-year-old classrooms contained slots from the same set of funding sources), one classroom was randomly selected (21 classrooms from 21 sites). In CBOs with multiple funding streams that were differentially distributed across classrooms (e.g., some classrooms contained PKA-only slots and others contained Child Care and Head Start slots), multiple classrooms were selected to capture the variation in funding sources (17 classrooms from eight sites). Following classroom selection, a recruitment letter was provided to the lead teacher in each classroom. Research staff followed up with an email that included links to the consent form, which was completed electronically. In total, the final sample was comprised of 66 classrooms (22 at schools, 44 at CBOs).

3.4.1. Administrator and teacher surveys

M

3.4. Data instruments and collection

CE

PT

ED

We fielded a survey of the sampled administrators (n=57) and lead teachers (n=66) at each site. The study team drafted hypotheses and then discerned several content areas for questions: program characteristics and experience with PKA, administrator characteristics, teacher characteristics, instructional practices, and professional development, each with relevance to the eight components of the Kagan (2015) systems framework, i.e., direct services to children and families; governance; finance; pedagogical quality; workforce development; accountability; family engagement; and linkages to other services and children’s transitions.

AC

In both surveys, the questions asked for a mix of close-ended and open-ended responses. Where possible, questions were drawn from existing surveys from large-scale research studies, such as the federal Early Childhood Longitudinal Study – Birth Cohort (Nord et al., 2006) and the National Survey of Early Care and Education (NSECE, 2012). When existing questions were not available to capture desired content, the research team created new items to do so. Both surveys were piloted with four PKA-site administrators and four PKA-site teachers outside of the sample. Administrators (n=57) and teachers (n=66) completed surveys over several months, with the first surveys completed in December 2016 and the last completed in June 2017. Most respondents completed the survey online, although a few opted to complete the survey on paper. The administrator survey required roughly 40 minutes and the teacher survey required roughly 25 minutes. Survey responses were confidential. A strict protocol was followed to de-identify all data for analytic purposes. Qualitative items were coded in NVivo by a single member of the research team to allow for quantitative analysis.

ACCEPTED MANUSCRIPT

3.4.2. CLASS observations

IP

T

Following the survey, we conducted Classroom Assessment Scoring System Pre-K (CLASS) observations in each classroom. The CLASS is an observational rating tool that was developed to assess process quality and has been associated with child learning (Pianta, La Paro, & Hamre, 2008). Full-length CLASS observations were conducted in 65 of the 66 participating classrooms. When one site declined to have a CLASS observation, we used the DOE’s own CLASS data, which had been collected in more than one classroom at the site and then aggregated to produce a site-level score. Because analyses using the site-level score were not significantly different from analyses that excluded this classroom, the site-level score was used for all analyses.

US

CR

Each CLASS observation was comprised of four 20-minute cycles, typically beginning during a morning meeting and lasting until all cycles were complete. Each observation was scored on a scale of 1 to 7 in three domains: Classroom Organization, Instructional Support, and Emotional Support. Scores for each domain were averaged to render an overall score for the classroom.

ED

M

AN

The field research staff participated in CLASS Pre-K training provided by Teachstone and completed Teachstone’s reliability assessment with a score of 80% or higher. To further establish reliability, each staff member then completed two cycles of observation with an experienced CLASS observer from the research team. After concurrently observing the two cycles, the team members compared scores and all observers were within one point of the experienced observer on 8/10 of the dimensions in each cycle; each were then cleared to observe on their own. CLASS observations were conducted over a nine-week period from mid-March 2017 to mid-May 2017. On average, CLASS observations lasted 78.5 minutes. 3.4.3. Demographic data

AC

3.5. Data analysis

CE

PT

The DOE provided the research team with de-identified PKA site-level data regarding child race/ethnicity, Individualized Education Programs (IEPs), Dual Language Learners (DLLs), and poverty status as measured by eligibility for the Free and Reduced Price (FRP) lunch program. These data were used to derive site-level percentages of these demographic groups.

To identify variation among programs by setting and auspice, preliminary analyses compared survey responses, CLASS scores, and demographic data in the setting and auspice categories. Specifically, descriptive statistics were run for the following setting types: schools, CBOs, and CBO auspice types (DOE-only CBOs, Child Care CBOs, and Head Start CBOs). To discern significant variation by setting and auspice, we ran independent samples t-tests and one-way analyses of variance (ANOVAs). For the sake of simplicity, all comparisons by setting and auspice used schools as the reference category. Because the sample is fairly small (n=57 sites and 66 classrooms) and the study is exploratory in nature, we recognized statistical significance at four levels: p<.10, p<.05, p<.01, and p<.001. Indeed, we make no claim to causal effects. 4. Results

ACCEPTED MANUSCRIPT

The results are organized into the eight elements of an effective early childhood system (Kagan, 2015): 1) direct services to children and families; 2) governance; 3) finance; 4) pedagogical quality; 5) workforce development; 6) accountability; 7) family engagement; and 8) linkages to other services and children’s transitions to kindergarten. Data on the comparisons of schools and CBOs are presented in Table 2. Data on the comparisons of CBOs of varied auspice are presented in Table 3. For the sake of simplicity, all statistical comparisons (by either setting or auspice) use schools as the reference category.

IP

T

4.1. Direct services

CR

4.1.1. Duration of programming

US

As shown in Table 2, on average, CBOs were open for nine hours per day (9:01; p<.001), while schools were open for about seven hours each day (6:57; p<.001). Almost all CBOs were open during the summer months, while no schools reported being open then (94% vs. 0%, respectively; p<.001). Across the auspice categories, no significant differences in the duration of programming were evident.

AN

4.1.2. Child services

PT

4.1.3. Family services

ED

M

In addition to the regular PKA day, schools and CBOs provided a similar array of services on site for children, such as health screenings, developmental assessments, mental health services, and medical services (an average of 2.0 types of child services at schools and 1.4 at CBOs). However, schools were more likely than CBOs to provide therapeutic services, such as speech or occupational therapy, to children (50% vs. 11%, respectively; p<.01). Across the auspice categories, no differences in the services provided to children were evident.

AC

CE

In addition to child services, CBOs provided a wider array of services on site to children’s families, such as mental health, housing and/or food assistance, employment assistance, parenting classes, and government application assistance, than schools (an average of 1.2 types of family services at CBOs vs. 0.5 at schools; p<.05). In particular, CBOs were more likely to provide on-site mental health services (31% of CBOs vs. 9% of schools; p<.10), and housing and/or food assistance (12% vs. 0% respectively; p<.10). As shown in Table 3, Head Start sites provided the widest array of services to families of any setting (1.5 on average; p<.10), 4.1.4. Child enrollment Along several dimensions, schools and CBOs provided these services to different populations of children and families. On average, CBO administrators served a higher percentage of children who were DLLs (43% of children in CBOs vs. 24% in schools; p<.05) and children with IEPs (11% vs. 5%, respectively; p<.01). CBOs and schools had virtually equal enrollment of children living in poverty (58% and 57%, respectively), although the percentage of poor children at CBOs is likely to be an underestimate because data were taken from FRP lunch forms, which many

ACCEPTED MANUSCRIPT CBOs do not submit. The larger proportion of high-need children was most pronounced at Head Start sites, which served the highest percentage of DLLs (52% at Head Start sites; p<.10) and children with IEPs (13% at Head Start sites; p<.05). Across the auspice categories, no differences in the percentage of poor children were evident.

CR

IP

T

Compared to schools, CBOs also served a higher percentage of African American children (38% vs. 25%, respectively; p<.10) and a lower percentage of White children (7% vs. 19%, respectively; p<.05). Racial/ethnic minority children were more likely to be concentrated in CBOs; in 83% of CBOs, racial/ethnic minority children presented 90% or more of PKA enrollment at the site, compared to 55% of schools (p<.05). Once again, Head Start sites were distinguished by serving the highest percentage of children who were African American or Hispanic (89% at Head Start sites p<.10). Of the children enrolled at Head Start sites, only 2% were White (p<.05), and every Head Start site enrolled a population that was 90% or more racial/ethnic minority children (p<.05).

US

Tables 2 and 3 here 4.2. Governance

ED

M

AN

The governance of sites was predictably straightforward at schools, which invariably contracted directly with the Department of Education to provide Pre-K for All services to children and families. Governance of the CBOs was more complex. While by definition, all six of the DOEonly sites had DOE contracts, 22 of the Child Care and Head Start CBOs had only ACS contracts and seven of CBOs had concurrent contracts for Pre-K for All services with both Department of Education and Administration for Children Services. Child Care sites were the most likely to have dual contracting arrangements (31% of Child Care sites; p<05).

PT

4.3. Finance 4.3.1. Teacher compensation

AC

CE

Funding for staff compensation differed significantly among schools and CBOs. On average, school teachers earned more per year than CBO teachers ($73,471 per year vs. $43,660, respectively; p<.001). School teachers were also more likely to have household incomes over $100,000 (50% vs. 3%, respectively; p<.001). In the sites that used the “enhancement” funding offered by the DOE (82% of CBOs), average teacher salaries were $10,000 higher than in sites that did not. Compensation disparities extended to the benefits teachers received. Although nearly all school and CBO teachers had health insurance, CBO teachers were less likely to have it through their employer or union (48% of CBO teachers vs. 86% of school teachers; p<.01). CBO teachers were also less likely to have a retirement plan through their employer or union (86% of school teachers vs. 61% of CBO teachers; p<.05). Corresponding with this variation in compensation and benefits, union membership differed among school and CBO teachers. Most school teachers (86%) belonged to a union, compared to 66% of CBO teachers (p<.10). No differences in teacher compensation were evident among the CBOs of different auspice.

ACCEPTED MANUSCRIPT

4.3.2 Administrator compensation Differences in administrator compensation were also evident. On average, school administrators earned nearly twice as much per year as CBO administrators ($117,044 vs. $64,016, respectively; p<.001). School administrators were also more likely to have household incomes over $100,000 (85% vs. 33%, respectively; p<.001).

CR

IP

T

Once again, compensation disparities extended to the benefits administrators received. While nearly all school and CBO administrators had health insurance, school administrators were more likely to have insurance through their employer or union (91% vs. 71%, respectively; p<.10). School administrators were also more likely to have a retirement plan through their employer or union than CBO administrators (96% vs. 57%, respectively; p<.01). No differences in administrator compensation were evident among the CBOs of different auspice.

US

4.4. Pedagogical quality

AN

The results presented below explicate two dimensions of pedagogical quality: first, aspects of process quality (i.e., CLASS observations and teacher practices), and second, aspects of structural quality (i.e., the use of curriculum, assessments, and early learning standards).

M

4.4.1. CLASS observations

PT

ED

School classrooms had higher overall scores on the CLASS than CBO classrooms (5.4 vs. 5.0, respectively; p<.05). Schools scored higher, on average, than CBOs on the domains of Classroom Organization, (6.1 vs. 5.7, respectively; p<.05) and Instructional Support (3.6 vs. 2.9, respectively; p<.01), but no difference was found on Emotional Support. In both schools and CBOs, overall CLASS scores were not significantly correlated with teacher salaries.

CE

Head Start classrooms had the lowest overall CLASS scores (4.6 in Head Start classrooms: p<.01), reflecting the lowest average score on Classroom Organization (5.1 in Head Start classrooms; p<.01) and Instructional Support (2.7 in Head Start classrooms; p<.05). No differences in the three auspice categories were evident on Emotional Support.

AC

4.4.2. Teacher practices

To learn more about classroom practices, teachers were asked how closely their practices matched what was expected of them, CBO teachers were more likely than school teachers to say, “not at all” or “somewhat closely” (an average 2.0 for CBO teachers vs. 2.4 for school teachers on a scale of 1 to 4; p<.10). Many CBO teachers further reported a lack of designated, out-ofclassroom time to prepare their lessons and reflect on their practices; school teachers, who are guaranteed five such prep-times per week in their union contract, did not. No differences in these teacher views and prep-time were evident among the CBOs of different auspice. Regarding teacher practices with DLLs, CBO teachers were more likely than school teachers to practice bilingual instruction rather than English-only instruction (55% of CBO teachers vs. 27%

ACCEPTED MANUSCRIPT of school teachers; p<.05). Bilingual instruction occurred in their classrooms or via pull-out instruction in a child’s home language. No differences in bilingual instruction were evident among the CBOs of different auspice. 4.4.3. Curriculum, assessments, and early learning standards

CR

IP

T

School and CBO administrators did not differ significantly in the difficulty they experienced meeting PKA’s requirements on the use of curriculum and assessments, but CBO administrators were more likely to report difficulty complying with the PKA early learning standards than school administrators (31% vs. 9%, respectively; p<.10). CBO administrators commonly cited the need to meet different agency standards or having inexperienced or insufficiently trained staff as the reasons. School administrators commonly cited the challenge of meeting standards with high-need children.

AN

US

Head Start sites were the most likely of any setting to report difficulty complying with such PKA requirements. Almost half of Head Start administrators (46%; p<.10) reported difficulty with PKA early learning standards and more than a third (39%; p<.10) said that PKA curriculum requirements were difficult. When asked how they manage both Head Start and PKA requirements, Head Start administrators said they follow both, follow whichever is highest, follow Head Start, or follow their own requirements.

M

4.5. Workforce development

4.5.1. Teacher characteristics

ED

The results presented below address characteristics of the PKA workforce, teacher recruitment and retention, and experiences with professional development.

AC

CE

PT

School and CBO teachers had similar levels of experience teaching children under age five (an average 9.0 and 9.4 years, respectively). However, teachers’ education, certification, and credentials differed. All school teachers had a master’s degree or higher, compared to two-thirds of CBO teachers (100% vs. 66%, respectively; p<.001). Nearly one-third of CBO teachers (32%; p<.01) had a bachelor’s degree, and 2% had a high school degree. CBO teachers’ education levels were moderately correlated with their salaries (r=0.63; p<.01), but not with their overall CLASS scores. Because all school teachers had master’s degrees, no analyses of correlation between their educational levels and salaries or CLASS scores were conducted. School teachers were also more likely than CBO teachers to be state certified (91% vs. 66%, respectively; p<.05), and to be certified in early childhood teaching (82% vs. 55%, respectively; p<.05) and/or elementary teaching (55% vs. 21%, respectively; p<.01). Fewer school teachers than CBO teachers had no certification or credential (5% vs. 20%, respectively; p<.10). Among the CBOs of varied auspice, Child Care teachers were the least likely to have a master’s degree or higher (61% of teachers at Child Care sites; p<.05), to be state certified (52%; p<.05), and to be certified in early childhood (39%; p<.05) or elementary school teaching (13%; p<.05).

ACCEPTED MANUSCRIPT The racial/ethnic composition of the school and CBO teaching workforce differed as well. CBO teachers were more likely than school teachers to be Hispanic/Latinx (27% vs. 9%, respectively; p<.10) and less likely to be White (23% vs. 50%, respectively; p<05). Even so, CBO and school teachers were equally likely to speak Spanish in addition to English (43% and 36%, respectively). More than half of teachers in both settings spoke any second language in addition to English (64% of CBO teachers and 55% of school teachers). Even so, many teachers in both settings said they sometimes needed an interpreter to help them talk to parents (48% of CBO teachers and 59% of school teachers).

CR

IP

T

The race/ethnicity of teachers at Head Start and Child Care sites were distinctive. Almost every Head Start teacher (87%; p<.10) and Child Care teacher (87%; p<.05) was a racial/ethnic minority. Even so, no differences across the auspice categories were evident in teachers’ bilingualism and need for interpreters. 4.5.2. Teacher recruitment and retention

ED

M

AN

US

These disparate compensation levels appear to have affected the ability of CBOs to recruit and retain qualified teachers. CBO administrators were more likely than school administrator s to say that recruiting teachers who met PKA requirements regarding teacher credentials and education was difficult (response of 3 or 4 on a 4-point scale; 66% vs. 9%, respectively; p<.001). Retaining teachers was also harder for CBOs. Two thirds of CBO administrators (69%) vs. one third of school administrators (36%) said that at least one of their PKA teachers had left in the prior year (p<.05). Most of the CBO administrators (81%) who reported such difficulty recruiting and retaining qualified teachers cited insufficient salaries as the reason. Half of them (52%) said that their teachers left to pursue better-paying jobs at schools.

CE

PT

Among the CBOs, these funding challenges were most common at Head Start and Child Care sites. Almost every Head Start administrator (85%; p<.001) and most Child Care administrators (69%; p<.001) said that hiring teachers with the requisite education and credentials was difficult, and most of them cited low salary levels. Head Start administrators were also the most likely of any setting to report having at least one teacher leave their site during the previous year (85% at Head Start sites; p<.05).

AC

4.5.3. Teacher participation in professional development In both settings, participation in professional development workshops was high (96% of teachers in both schools and CBOs participated in the past year). However, CBO teachers were more likely than school teachers to attend workshops led by members of their program’s staff, rather than by DOE trainers (55% vs. 23%, respectively; p<.05), and to attend workshops held on site (80% vs. 52%, respectively; p<.05). Several CBO administrators said that staff shortages prevented them allowing teachers to attend off-site trainings. Almost all school and CBO teachers reported receiving some form of instructional coaching in the past year, but fewer than half in each setting (36% of school teachers and 46% of CBO teachers) said they received coaching at least monthly. Most teachers in schools and CBOs received coaching from DOE-affiliated coaches, while about a quarter had a coach based in their

ACCEPTED MANUSCRIPT own program. No differences in either workshop attendance or coaching receipt were evident among the CBOs of different auspice. 4.5.4. Changes in teacher practice

CR

IP

T

Overall, teachers in both settings were more likely to say that coaching changed their classroom practice “a lot” (52%) than did workshops (35%; p<.05). Among CBO teachers, the frequency of coaching was positively associated with teacher reports of changes in their practice. CBO teachers who received coaching at least monthly were more likely than those who received it less often to say that coaching changed their classroom practice “a lot” (75% of CBO teachers who received coaching at least monthly vs. 43% of CBO teachers who received coaching less often; p<.05). This result was not evident for school teachers, however.

US

Across the auspice categories, Child Care teachers who received coaching at least monthly said that coaching substantially changed their classroom practices (73%), compared to Child Care teachers who received coaching less frequently (36%; p<.10). This difference was not evident among either Head Start or DOE-only teachers.

AN

4.5.5. Administrator characteristics

CE

4.6. Accountability

PT

ED

M

School and CBO administrators had similar years of experience at their sites (an average 5.0 and 4.0 years, respectively), and similar years of experience as an early childhood director at any site (7.6 years and 10.7 years, respectively). All school administrators and most CBO administrators (89%) had a master’s degree or higher. (Master’s degrees could be in any area, including but not limited to education.) CBO administrators’ education levels were moderately correlated with their salaries (r=0.56; p<.01). Because all school administrators had master’s degrees, no analyses of correlation between their education levels and salaries were conducted.) Across the auspice categories, CBOs administrators had similar years of experience and most had a master’s degree or higher.

AC

Schools and CBOs did not differ significantly in the difficulty they experienced regarding compliance with PKA requirements on reporting and monitoring. However, CBO administrators reported having to devote significantly more time to budgeting and allocating costs than school administrators. On average, CBO administrators spent 4.3 hours per week on budgeting and cost allocation, compared to 1.6 hours per week spent by school administrators (p<.05). More broadly, the time per week spent by the entire program staff on budgeting and cost allocation was 18.0 hours in CBOs compared to 3.4 hours in schools (p<.05). CBO administrators also agreed more strongly that the PKA rules regarding budgeting and cost allocation were complicated (p<.01), that the rules were confusing (p<.001), and that PKA administrators lacked sufficient budgeting support (p<.05). Head Start administrators, most of whom were navigating the requirements of Head Start, Child Care, and PKA, agreed most strongly that the PKA rules regarding budgeting and cost allocation were complicated (p<.001), that the rules were confusing (p<.001), and that they lacked sufficient budgeting support (p<.05).

ACCEPTED MANUSCRIPT

4.7. Family engagement

IP

T

Many school (32%) and CBO (43%) administrators reported that complying with PKA requirements regarding family engagement was difficult, saying that parents were too busy or uninterested. CBO administrators also cited insufficient staff time and resources to engage parents. Nearly every school and CBO used parent/teacher conferences, parent board/councils, class events, and parent groups or workshops to engage parents. Schools were more likely than CBOs to send parents materials to support children’s at-home learning at least weekly (77% vs. 49%, respectively; p<.05).

AN

4.8.1. Child services

US

4.8. Linkages to other services and transitions

CR

Child Care sites were the least likely of any setting to send parents materials to support at-home learning at least weekly (25% of Child Care sites; p<.05), while DOE-only sites were least likely to engage parents via parent board/councils (50% of DOE-only sites; p<.10). Consistent with the Head Start model, every Head Start site had a parent board/council.

CE

4.8.2. Family services

PT

ED

M

In addition to the providing the direct on-site services for children described in Sections 4.1.2, both schools and CBOs referred children to off-site services, with CBOs referring children to a wider array of services than did schools (an average 3.9 services vs. 2.3 services, respectively; p<.01). In particular, CBOs were more likely than schools to refer children to off-site screenings (69% of CBOs vs. 38% of schools; p<.05), therapeutic services, such as speech or occupational therapy (89% vs. 41%, respectively; p<.001), and medical services (86% vs. 50%, respectively; p<.01). Among the CBOs of different auspice, Child Care and Head Start sites referred children to the widest array of services (4.1 services at Child Care sites; p<.05; 3.9 services at Head Start sites; p<.10) and were most likely to utilize off-site therapeutic services and medical services.

AC

Similarly, in addition to providing the on-site services for families described in Sections 4.1.3, both schools and CBOs referred families to services off-site, though once again, CBOs referred families to a wider array of services than did schools (an average 4.2 services vs. 2.5 services, respectively; p<.01). In particular, CBOs were more likely to refer families to off-site legal services (69% of CBOs vs. 36% of schools; p<.05), housing and/or food assistance (81% of CBOs vs. 41% of schools; p<.01), employment/education assistance (79% vs. 41%, respectively; p<.01), and assistance with government applications (74% vs. 36%, respectively; p<.01). Among the CBOs of different auspice, Head Start sites referred families to the widest array of services (5.2 services; p<.01) and were most likely to refer families to off-site housing and/or food assistance, employment/education assistance, and assistance with government applications. 4.8.3 Transitions to kindergarten

ACCEPTED MANUSCRIPT Schools and CBOs helped children transition to kindergarten in similar ways, such as sending kindergarten information to parents, providing workshops for parents on the transition, helping parents apply to kindergarten, and having teachers discuss the transition with children. Most school administrators (73%) also said their PKA teachers participated in professional development with their kindergarten teachers. The content of these joint trainings included literacy and math instruction, standards, the Common Core, assessments, using data, and aligning PKA and kindergarten curricula.

US

CR

IP

T

Additionally, schools were more likely than CBOs to have all their PKA children visit kindergarten classrooms (96% of schools vs. 71% of CBOs; p<.05). Among the CBOs of varied auspice, DOE-only sites were the least likely of any setting to have all their children visit kindergarten classrooms (50% of DOE-only sites; p<.01). Such visits appear to be more formidable for CBOs than for schools because of their children’s geographic dispersion when they move on to kindergarten. Three-quarters of school administrators (76%) said that their PKA children usually stayed at their school for kindergarten. In contrast, only 21% of CBO administrators said that their PKA children typically went on to the same school; among those who said they did not, children dispersed among five or six schools (5.6 on average). Across the auspice categories, no differences were evident in the dispersion of children to kindergarten.

AN

5. Discussion and policy implications

PT

ED

M

The results indicate substantive differences between school- and CBO-based PKA programs in New York City. Along several dimensions (e.g., budgeting and cost allocation, teacher recruitment and retention, classroom quality, early learning standards, and transitions), CBOs had greater difficulty with the PKA model, though they simultaneously possess important strengths (e.g., longer hours, summer hours, on-site and off-site services) that schools often do not. Head Start sites, while exemplary in terms of services to economically disadvantaged families, appear to have had the greatest difficulty with adding PKA to their programming. Together, the findings, though rendered from purely descriptive data, suggest the presence of systemic obstacles to the provision of consistently high-quality UPK programs.

AC

CE

What do the results say about building a unified mixed-delivery system for UPK? To address this question, we return to the eight components of effective early education systems presented by Kagan (2015). 5.1. Direct services

The core direct service in this example is PKA for 4-year-olds for 6 hours and 20 minutes per day and 180 days per year. The results indicate that in fundamental ways, schools and CBOs deliver this service differently. The typical school day aligns closely with the PKA day and year, with schools in the sample open for about seven hours a day on average and 9 months per year. CBOs look quite different, staying open for nine hours a day on average and 12 months per year. These longer hours place PKA’s CBOs, on average, among the minority of early education centers nationwide (42%) that are open at least 30 hours per week (NSECE, 2014).

ACCEPTED MANUSCRIPT The extended day and months at CBOs reflect their broader mission to serve families comprehensively, offering the additional hours of child-care coverage that their jobs may require. Unlike schools, CBOs multiple funding streams (PKA, Child Care, and Head Start) allow them to pursue this mission by offering days that extend beyond what PKA covers. National survey data indicate that families headed by low-income parents, and particularly single parents, are most likely to need extended hours of coverage (NSECE, 2017).

IP

T

CBOs also distinguished themselves by offering a wider array of on-site services to families, a reflection of their holistic approach to family well-being. CBOs and schools offered a similar array of on-site services to children, though schools were more likely then CBOs to provide children on-site therapeutic services, such as speech and occupational therapy.

M

AN

US

CR

The results indicate that schools and CBO provide these services to quite different populations of children. CBOs were more likely to enroll DLLs and children with IEPs. Head Start sites served the highest need population, with the highest percentages of children who are DLLs and children with IEPs. Though the data on children in poverty at CBOs and schools look similar, they are likely to be different in reality, given the income-eligibility that governs non-UPK sources of CBO funding. Both Head Start and Child Care sites were also likely to serve high numbers of children in poverty, given the low-income thresholds required of many, if not most, of their enrollees. Together, the results indicate that CBOs were providing PKA to a higher-need population, which exerts a greater demand on program and workforce capacity at CBOs, relative to schools that generally serve a lower-need population.

PT

ED

Importantly, the results suggest that CBOs were, in many ways, meeting this demand. CBOs— and especially Head Start sites—excel at services to families, again reflecting their broader mission of holistic service to their communities. CBOs offered a wider array of family services, such as mental health, housing and/or food assistance, employment assistance, parenting classes, and government application assistance, and provided more of them on site.

AC

5.2. Governance

CE

The CBO focus on the varied needs of their families extends to instructional practices within classrooms. The results indicate that despite equal levels of bilingua lism among CBO and school teachers, CBO teachers were more likely to use these skills to practice bilingual instruction with their children, who are more likely to be DLLs than their school-based peers.

The multiple contracting arrangements evident among PKA sites reflect a history of UPK governance that has long been divided between Department of Education and Administration for Children’s Services, and among multiple other agencies that exert authority over UPK sites. As PKA policies continue to evolve, the city is taking steps to consolidate the contracting of its early education services within the DOE, including but not limited to those at PKA sites. 5.3. Finance The correlates of divided governance are perhaps most evident in the different compensation schedules for teachers and administrators in CBOs and schools. On average, school teachers,

ACCEPTED MANUSCRIPT who overall were more degreed and certified, earned $30,000 more per year than CBO teachers and were more likely to have health insurance and retirement plans through their employer or union. These disparities certainly reflect differences in teacher qualifications, but also different compensation schemes instituted long ago by DOE and ACS, and different unions representing teachers in the two settings.

CR

IP

T

These salary disparities are consistent with national data indicating that preschool teachers working in schools typically earn more on average than the general population of preschool teachers. While the median annual earnings for preschool teachers in all settings nationally was only $28,579 in 2015, preschool teachers in schools earned $42,890 annually (Whitebook, McLean, & Austin, 2016). In New York City, salary levels in both schools and CBOs are higher than this national median, likely a reflection of the city’s relatively high standard of living.

AN

US

While such gaps in teacher compensation have garnered due national attention, the results indicate substantial differences in administrator compensation that also deserve recognition. On average, school administrators earned $50,000 more per year than CBO administrators and were more likely to have insurance and retirement plans through their employer or union. Research and policy efforts have understandably focused on developing core competencie s for preschool program leadership, while designing the education and training to develop such capabilities (IOM & NRC, 2015). Assuring that administrators in all settings are also adequately compensated deserves further attention as well.

M

5.4. Pedagogical quality

AC

CE

PT

ED

In several ways, the results indicate that CBO administrators have greater difficulty fostering program quality. Most saliently, CBO administrators were more likely than school administrators to say that complying with PKA requirements regarding teacher qualifications was difficult because of the lower salaries, longer workdays, and longer work-years at CBOs. Teacher retention was also harder at CBOs—and most difficult at Head Start sites—because teachers who gained the requisite education and credentials leave to pursue better-paying jobs at schools and/or to teach older children. This compensation structure amounts to a de facto career ladder that draws qualified talent from CBOs to school. In turn, the revolving door absorbs CBO administrator time, disrupts the continuity that supports children’s learning, and creates chronic staff shortages. CBO administrators further reported that lack of staff coverage prevented them from allowing teachers to attend off-site trainings. CBO teachers also reported a lack of time to prepare their lessons, time that is guaranteed in the school teacher union contract, but not in CBO teacher union contracts. Largely beyond the control of program administrators, these systemic issues thus hinder program quality. At the classroom level, differences in quality, as measured on the CLASS, between schools and CBOs are an enduring concern. The finding of the lowest scores, on average, at Head Start sites may reflect the intersection of multiple systemic issues, such as a higher need population of families, lower workforce compensation, longer work-days and work-years, higher teacher turnover, and the challenges of managing multiple program requirements. By comparison, the CLASS results for PKA Head Start sites were equal to or below the national average Head Start CLASS scores in 2017 (ACF, 2018).

ACCEPTED MANUSCRIPT

IP

T

The greater mismatch at CBOs, as reported by teachers, between what PKA expects of them and their practices in the classroom is also troubling. This difference may reflect well-established teacher practices at CBO programs that have been serving young children long before PKA arrived (Brown & Gasko, 2012; Wilinski, 2017). CBO administrators further reported more difficulty complying with PKA’s early learning standards, a challenge that may be aggravated, especially at Head Start sites, by the multiple and varied sets of standards that apply to them. Even so, the CLASS results indicate that the quality of instruction was relatively low in both schools and CBOs, consistent with other research using multi-state data (Burchinal, Vandergrift, Pianta, & Mashburn, 2010). This finding is a concern given the centrality of instructional quality in promoting children’s learning (Burchinal, 2018; Johnson et al., 2016).

CR

5.5. Workforce development

ED

M

AN

US

The results are encouraging in that CBO teachers were often positive about the coaching they received, and especially more frequent coaching, although fewer than half of teachers in both settings said they received coaching at least monthly. This finding is consonant with research that associates more frequent coaching with changes in teacher practices and children’s learning gains (Egert, Fukkink, & Eckhardt, 2018; Kraft et al., 2018; Markussen-Brown et al., 2017). The results also indicate that CBO teachers were more likely to attend workshops held on site. This seems to reflect a more systemic problem, i.e., the resource and staff shortages that appear to be more common in CBOs and thus shortchange professional development efforts to improve classroom quality. For teachers to use the full range of professional development opportunities, administrators require adequate staff coverage, which the results suggest CBOs are lacking.

CE

PT

At the same time, the results do not suggest that variation in pedagogical quality can be explained solely by differences in teacher education, certification, credentials, or experience, consistent with research in other locations (Early et al., 2007). While more frequent coaching may help address gaps in quality, systemic differences between schools and CBOs, which define the compensation and conditions under which teachers work and develop their skills, appear to help explain these disparities as well. More deeply rooted, uneven quality will not be solved solely by increased coaching or higher teacher qualifications.

AC

5.6. Accountability

CBO administrators and their staff spent far more time each week than those in schools on budgeting and cost allocation, a drain on administrator time that could otherwise be used to promote quality programming and teaching. Head Start sites had the most difficulty with budgeting, cost allocation, and the management of multiple program requirements. Additional support, whether via increased staffing or technical support, for example, appears needed to allow CBO administrators to turn more of their attention to program quality. 5.7. Family engagement Schools and CBOs try to engage their families in similar ways and report similar difficulty engaging families who are either too busy or insufficiently willing. The results thus suggest that

ACCEPTED MANUSCRIPT both settings have heard the signal from PKA that family engagement is an important component of program quality and are striving to meet this goal. 5.8. Linkages to other services and transitions

US

CR

IP

T

Both schools and CBOs linked children to a similar array of services, but CBOs connected their families to a wider array of services than did schools. Once again, Head Start sites were the leader in this respect. In contrast, schools and CBOs helped their children transition to kindergarten in similar ways, although the challenges of doing so appear to be much steeper at CBOs. Schools enjoy an inherent advantage in being able to foster smooth transitions for children who mostly stay in the same building for kindergarten, while CBOs must promote transitions among children who disperse widely among multiple schools. Moreover, schools can more easily conduct the joint professional development between their pre-k and kindergarten teachers that fosters pedagogical and curricular continuity for children. Together, these structural differences suggest that CBOs must exert (and in this example, are exerting) greater effort than schools to promote the smooth transitions that support children’s preschool learning gains. 5.9. Policy implications

ED

M

AN

In these multiple ways, the results indicate the presence of systemic differences in school- and CBO-based PKA programs that will not be solved by narrow policy and programmatic changes. At the same time, the results indicate that different settings possess varied program assets. The findings are thus consistent with other research that has identified both positives and negatives in a mixed-delivery approach to preschool provision (Clothier, 2006; Schumacher, Ewen, Hart, & Lombardi, 2005; Stephens, 2014; Wat & Gayl, 2009). Cognizant of this complex and, in many ways, promising context, we offer three policy considerations regarding policies to establish effective systems for UPK provision that emanate from the results.

PT

5.9.1. Recognize that achieving equal UPK outcomes requires different inputs that respond to site-level needs.

AC

CE

Different levels of inputs are necessary to foster “high-quality” programs in all settings that offer both exemplary pedagogy and the comprehensive services required to sustain children and families. The results here suggest that CBOs are being asked, in effect, to do more with less—to serve a higher need population with inadequate resources to hire needed staff and pay them sufficiently. At the same time, schools that are strong on pedagogy but perhaps weaker on family services may need additional resources to serve their families adequately. In either case, policies should recognize that the higher the need, the higher the required resources. The need to adapt to site-level needs extends to the city’s professional development efforts, particularly given the evident need to promote instructional quality across all settings. The results here suggest that some teachers might be more likely to change their practice if they have more voice in choosing and crafting their own training. Cognitive and adult learning theories point to the limitations of “telling” practitioners how to change what they do, and instead suggest engaging them in professional learning communities that use a distributive, rather than hierarchical, structure of leadership (Coburn, 2003; Resnick & Spillane, 2010; Spillane, Reiser,

ACCEPTED MANUSCRIPT & Reimer, 2002). Rethinking how to engage both administrators and teachers in improving teacher practices could support the depth and quality of professional learning. 5.9.2. Establish equitable compensation for UPK teachers and administrators that apply to all settings and build system-wide career ladders for teachers that offer rising compensation.

CR

IP

T

The results indicate that qualified teachers are leaving CBOs to pursue jobs in schools that pay more for fewer working hours, with schools open an average 35 hours per week, compared to 45 hours per week at CBOs. To staunch this movement of talent along this de facto career ladder, policymakers should establish hourly wages that apply to PKA teachers and administrators in all settings. Given the scope of both teacher and administrator compensation gaps, addressing these disparities requires the identification of new and durable financing mechanisms. Policymakers should also consider building career ladders with job titles and financial incentives that apply in all settings. Often done in other countries, the ladder would guide the professional trajectories of UPK teachers and reward them for developing their skills (Kagan, 2018).

ED

M

AN

US

Left unaddressed, inequitable compensation will hinder policy efforts to develop a core asset of CBOs: the racially and ethnically diverse teaching workforce. While many of the results here point to the challenges of a lesser educated, lesser compensated CBO workforce, its relative racial/ethnic diversity, compared to school-based PKA teachers, represents a strength in light of the growing diversity of children enrolled in pre-k programs in New York City and indeed, nationwide (Aud, Fox, & KewalRamani, 2010; Vespa, Armstrong, & Medina, 2018). Developing the full capacity of the CBO workforce by providing adequate compensation and financial incentives to propel their career advancement remains a critical challenge in the pursuit of equitably high levels of quality (Whitebook et al., 2008; U.S. Dept. of Health and Human Services & U.S. Dept. of Education, 2016).

AC

CE

PT

In recent years, several states and localities have responded with policy strategies to improve teacher compensation, largely focused on establishing parity between all preschool teachers (whether in schools or CBOs) and teachers in K-12 classrooms (Barnett & Kasmin, 2017: McLean, Dichter, & Whitebook 2017; Whitebook, Phillips, & Howes, 2014). Some seek to raise teachers’ base salaries, while others focus on bonuses to teachers or stipends tied to gains in teacher qualifications. Some affect all settings, while others target “eligible” centers, such as those that serve mostly poor children. Strategies to improve base salaries and benefits in all settings represent the most comprehensive and durable approach (Whitebook & McLean, 2017). Extending such efforts to the compensation of program leadership is an important next step. 5.9.3. Create transcendent governmental entities to craft coherent policies that apply to all early education programs and settings. To bridge deeply-rooted structural distinctions, policymakers should consider creating a ministry or other governmental entity, whether by consolidating existing entities or creating a new one, that transcends departmental boundaries and has jurisdiction over early education programs in all settings. Other countries have successfully created such durable entities that bridge the structural divide (Kagan, 2018). This entity can facilitate the development of coherent policies that transcend settings, promote equity and sustainability, and generate public support when they are charged with data collection and dissemination. Other countries and states within the U.S. have

ACCEPTED MANUSCRIPT used such entities to adopt a single set of policies, often articulated in policy documents that apply across the early education system, specifying: i) quality standards, ii) aligned curricula and assessments, iii) workforce development and compensation, iv) program monitoring, v) comprehensive service plans; and vi) data systems.

CR

IP

T

However feasible, these innovations require careful thought among early education policymakers, constituents, and scholars. In New York City, for example, the city’s consolidation of all early education contracts within its Department of Education represents an ambitious attempt to create more coherent policies and create a birth-to-five continuum of publicly provided early childhood services. A multi- year process that introduces new complexities even while trying to mitigate them, the city’s strategy nonetheless represents a laudable effort to address systemic issues that deserves further study. 6. Limitations and conclusion

AN

US

Given the relatively small size of the sample, it is possible that some of the associations we found were spurious and should be interpreted with due caution. Additionally, because the sample is rooted in the particular context of New York City, the results are not directly generalizable to other cities and states. The fairly low response rate among sites also evokes caution regarding the extent to which the sample reflects the complete distribution of sites in the city’s PKA landscape.

ED

M

We further emphasize that the CLASS is not intended to be a comprehensive measure of quality. The instrument privileges responsive teacher-child interactions, classroom organization, and emotional dimensions of teaching practice that are supported by the research literature, but not meant to be definitive indicators of quality. Instead, the results can be used to inform resource allocation, as well as site-level and teacher-level professional development and coaching.

AC

CE

PT

Using descriptive data, we make no claim to causal effects. Further research is needed to discern when and how the components of pedagogical quality, and policy strategies to promote them, are predictive of child learning. The descriptive data here regarding participation in qualityenhancement efforts does not imply their effectiveness. Moreover, any effort to implement a model system-wide confronts a tension between the dual pursuit of model fidelity and appropriate variation at the site level to adapt to community strengths, needs, and preferences. This empirical example is no exception and points to the need for additional study on how to craft system-wide and coherent policies that allow for the local variation that enhances quality. In sum, the study points out similarities and distinctions in program characteristics that deserve consideration as the UPK expansion continues in New York City and elsewhere. Many of these distinctions appear to be rooted in systemic issues, far from unique to the city, and addressing them will require innovative policies and funding mechanisms. Taking a systems approach to the quality of early education could enable policymakers to identify and overcome the inequities created by a policy landscape fractured among different agencies and diverse providers. As theoretical and empirical scholarship progresses to support a systems perspective, the present study aims to contribute to this work.

ACCEPTED MANUSCRIPT 7. References Administration for Children and Families. (2008). Head Start Act (as amended December 12, 2007). Washington, DC: U.S. Department of Health and Human Services. Retrieved from https://eclkc.ohs.acf.hhs.gov/policy/head-start-act.

T

Administration for Children and Families. (2015). Head Start early learning outcomes framework: Ages birth to five. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Head Start.

CR

IP

Administration for Children and Families. (2016). Head Start program performance standards: 45 CFR Chapter XIII. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Head Start.

US

Administration for Children and Families. (2017). Head Start policy & regulations: Policy council and policy committee. Retrieved from https://eclkc.ohs.acf.hhs.gov/policy/45-cfr-chapxiii/1301-3-policy-council-policy-committee.

M

AN

Administration for Children and Families. (2018). A national overview of grantee CLASS scores in 2017. Washington, DC: U.S. Department of Health and Human Services. Retrieved from https://eclkc.ohs.acf.hhs.gov/data-ongoing- monitoring/article/national-overview-grantee-classrscores-2017.

ED

Aud, S., Fox, M., and KewalRamani, A. (2010). Status and Trends in the Education of Racial and Ethnic Groups (NCES 2010-015). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

PT

Bandura, A. (1971). Social learning theory. Morristown, NJ: General Learning Press.

CE

Barnett, W. S., Friedman-Krauss, A. H., Gomez, R., Horowitz, M., Weisenfeld, G. G., Brown, K. C., & Squires, J. H. (2016). The state of preschool 2015: State preschool yearbook. New Brunswick, NJ: NIEER.

AC

Barnett, W. S., & Gomez, R. (2016). Universal pre-k: What does it mean and who provides it? New Brunswick, NJ: National Institute for Early Education Research. Retrieved from http://nieer.org/2016/01/06/universal-pre-k-what-does-it-mean-and-who-provides- it. Barnett, W. S., & Kasmin, R. (2016). Teacher compensation parity policies and state-funded pre-k programs. New Brunswick, NJ: NIEER and Berkeley, CA: Center for the Study of Child Care Employment, University of California, Berkeley. Bartik, T., Belford, J. A., Gormley, W. T., & Anderson, S. (2016). A benefit-cost analysis of the Tulsa universal pre-k program: Center for Research on Children in the United States (CROCUS) Working Paper 19. Washington DC: Georgetown University.

ACCEPTED MANUSCRIPT Brown, C., & Gasko, J. W. (2012). Why should pre-k be more like elementary school? A case study of pre-k reform. Journal in Early Childhood Education, 26, 264-290. Burchinal, M. (2018). Measuring early care and education quality. Child Development Perspectives, 12(1), 3-9.

T

Burchinal, M., Vandergrift, N., Pianta, R., & Mashburn, A. (2010). Threshold analysis of association between child care quality and child outcomes for low-income children in prekindergarten programs. Early Childhood Research Quarterly, 25, 166-176.

CR

IP

Butrymowicz, S., & Mader, J. (2016). How the military created the best child care system in the nation: And what states can learn from its significant turnaround. The Hechinger Report. Retrieved from https://hechingerreport.org/how-the- military-created-the-best-child-care-systemin-the-nation/.

US

Cannon, J. S., Zellman, G. L., Karoly, L. A. & Schwartz, H. L. (2017). Quality rating and improvement systems for early care and education programs: Making the second generation better. Santa Monica, CA: Rand Corporation.

AN

Cascio, E. U. (2017). Does universal preschool hit the target? Program access and preschool impacts, Working paper 23215. Washington, DC: National Bureau of Economic Research.

ED

M

Clothier, S. (2006). Effective strategies for prekindergarten expansion: Collaboration with community providers. Denver, CO: National Conference of State Legislatures. Retrieved from http://www.ncsl.org/print/cyf/effective_strategies.pdf.

PT

Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), pp. 3-12.

CE

Duncan, G., & Magnuson, K. (2013). Investing in preschool programs. Journal of Economic Perspectives, 27(2), 109-132.

AC

Early, D. M., Maxwell, K. L., Burchinal, M., Alva, S., Bender, R. H., Bryant, D., … & Zill, N. (2007). Teachers’ education, classroom quality, and young children’s academic skills: Results from seven studies of preschool programs. Child Development, 78(2), 558-580. Egert, F., Fukkink, R. G., & Eckhardt, A. G. (2018). Impacts of in-service professional development programs for early childhood teachers on quality ratings and child outcomes: A meta-analysis. Review of Educational Research, 88(3), 401-433. Friedman-Krauss, A. H., Barnett, W. S., Weisenfeld, G. G., Kasmin, R., DiCrecchio, N., & Horowitz, M. (2018). The state of preschool 2017: State preschool yearbook. New Brunswick, NJ: National Institute for Early Education Research. Gormley, W., Gayer, T., Phillips, D.A., & Dawson, B. (2005). The effects of universal pre-k on cognitive development. Developmental Psychology, 41, 872-884.

ACCEPTED MANUSCRIPT

Gormley, W. T., Phillips, D., & Anderson, S. (2018). The effects of Tulsa’s pre-k program on middle school student performance. Journal of Policy Analysis and Management, 37(1), 63-87. Heckman, J. (2011). The economics of inequality: The value of early childhood education. American Educator, Spring, 31- 47.

T

Henry, G. T., & Rickman, D. K. (2007). Do peers influence children’s skill development in preschool? Economics of Education Review, 26, 100-112.

CR

IP

Holcomb, B. (2006). A diverse system delivers for pre-k: Lessons learned in New York State. Washington, DC: Pre-K Now.

US

Institute of Medicine and National Research Council. (2015). Transforming the workforce for children birth through age 8: A unifying foundation. Washington, DC: The National Academies Press.

AN

Johnson, A. D., Markowitz, A. J., Hill, C. J., & Phillips, D. A. (2016). Variation in impacts of Tulsa pre-k on cognitive development in kindergarten: The role of instructional support. Developmental Psychology, 52(12), 2145-2158.

ED

M

Justice, L. M., Petscher, Y., Schatschneider, C., & Mashburn, A. (2011). Peer effects in preschool classrooms: Is children’s language growth associated with their classmates’ skills? Child Development, 82(6), 1768-1777. Kagan, S. L. (2009). American early childhood education: Preventing or perpetuating inequity? New York: Teachers College at Columbia University.

CE

PT

Kagan, S. L. (2015). Conceptualizing ECE governance: Not the elephant in the room. In S. L. Kagan & R. E. Gomez (Eds.), Early childhood governance: Choices and consequences (pp. 929). New York: Teachers College Press.

AC

Kagan, S. L. (Ed.) (2018). The early advantage I: Early childhood systems that lead by example. New York: Teachers College Press. Kagan, S. L., & Gomez, R. E. (2015). Early childhood governance: Choices and consequences. New York: Teachers College Press. Kagan, S. L., & Tarrant, K. (2010). Transitions for young children: Creating connections across early childhood systems. Baltimore, MD: Brookes Publishing. Karoly, L. A. (2016). The economic returns to early childhood education. Future of Children, 26(2), 37-55. King, E. K., Johnson, A. V., Cassidy, D. J., Wang, Y. C., Lower, J. K., & Kintner-Duffy, V. L. (2016). Preschool teachers’ financial well-being and work time supports: Associations with

ACCEPTED MANUSCRIPT children’s emotional expressions and behaviors in classrooms. Early Childhood Education Journal, 44, 545-553. Kraft, M. A., Blazar, D., & Hogan, D. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547-588.

IP

T

Lipsey, M. W., Farran, D. C., & Durkin, K. (2018). Effects of the Tennessee prekindergarten program on children’s achievement and behavior through third grade. Early Childhood Research Quarterly, 45, 155.176.

CR

LoCasale, J., Mashburn, A. J., Downer, J. T., & Pianta, R. C. (2008). Prekindergarten teachers’ use of transition practices and children’s adjustment to kindergarten. Early Childhood Research Quarterly, 23, 124-139.

US

Magnuson, K., & Schindler, H. S. (2016). Parent programs in pre-k through third grade. Future of Children, 26(2), 207-221.

AN

Magnuson, K., & Waldfogel, J. (2005). Early childhood care and education: Effects on ethnic and racial gaps in school readiness. The Future of Children, 15(1), 169-196.

ED

M

Markussen-Brown, J., Juhl, C. B., Piasta, S. B., Bleses, D., Hojen, A., & Justice, L. M. (2017). The effects of language- and literacy- focused professional development on early educators and children: A best-evidence meta-analysis. Early Childhood Research Quarterly, 38, 97-115.

PT

Marti, M., Melvin, S., Noble, K. G., & Duch, H. (2018). Intervention fidelity of Getting Ready for School: Associations with classroom and teacher characteristics and preschooler’s school readiness skills. Early Childhood Research Quarterly, 44, 55-71.

CE

McCabe, L. A., & Sipple, J. W. (2011). Colliding worlds: Practical and political tensions of prekindergarten implementation in public schools. Educational Policy, 25(1), 1-26.

AC

McCoy, D. C., Yoshikawa, H., Ziol-Guest, K. M., Duncan, G. J., Schindler, H. S., Magnuson, K., … Shonkoff, J. P. (2017). Impacts of early childhood education on medium- and long-term educational outcomes. Educational Researcher, 46(8), 474-487. McLean, C., Dichter, H., & Whitebook, M. (2017). Strategies in pursuit of pre-k teacher compensation parity: Lessons from seven states and cities. New Brunswick, NJ: NIEER and Berkeley, CA: Center for the Study of Child Care Employment, University of California, Berkeley. National Survey of Early Care and Education. (2014). Fact sheet: Characteristics of centerbased early care and education programs. Washington DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved from http://www.acf.hhs.gov/programs/opre/research/project/nationalsurvey-of-early-care-and education-nsece-2010-2014.

ACCEPTED MANUSCRIPT National Survey of Early Care and Education. (2017). Snapshot: Parent work schedules in households with young children. Washington DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved from http://www.acf.hhs.gov/programs/opre/research/project/national-survey-of-earlycare-and-education- nsece-2010-2014.

T

National Survey of Early Care and Education Project Team (National Opinion Research Center). (2012). National Survey of Early Care and Education (NSECE), 2012 Codebook . Ann Arbor, MI: Inter-university Consortium for Political and Social Research.

CR

IP

New York State Department of Education. (2011). New York State prekindergarten foundation for the common core. Albany, NY: New York State DOE.

US

New York City Department of Education & New York City Administration for Children’s Services. (2015). Pre-K for all policy handbook: Part I, 2015-2016. New York: NYC DOE

AN

Nord, C., Edwards, B., Andreassen, C., Green, J. L., & Wallner-Allen, K. (2006). Early Childhood Longitudinal Study, Birth Cohort (ECLS-B), user’s manual for the ECLS-B longitudinal 9-month–2-year data file and electronic codebook (NCES 2006–046). Washington, DC: National Center for Education Statistics.

ED

M

Phillips, D. A., Anderson, S., Rupa Datta, A., & Kisker, E. E. (2018). The changing landscape of publicly- funded center-based child care: 1990-2012. Children and Youth Services Review, 91, 94-104. Phillips, D., Austin, L. J. E., & Whitebook, M. (2016). The early care and education workforce. Future of Children, 26(2), 139-158.

CE

PT

Phillips, D., Lipsey, M., Dodge, K. A., Haskins, R., Bassok, D., Burchinal, M. R., . . . & Weiland, C. (2017). Puzzling it out: The current state of scientific knowledge on prekindergarten effects. Washington, DC: Brookings Institution. Retrieved from https://www.brookings.edu/wp-content/uploads/2017/04/consensus-statement final.pdf.

AC

Pianta, R., Downer, J., & Hamre, B. (2016). Quality in early education classrooms: Definitions, gaps, and systems. The Future of Children, 26(2), 119-137. Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System: Manual Pre-K. Baltimore, MD: Brookes Publishing. Rebell, M. A., & Wolff, J. R. (2017). Establishing universal access to prekindergarten as a constitutional right. New York: The Center for Educational Equity, Teachers College at Columbia University. Resnick, L. B., Spillane, J. P., Goldman, P., & Rangel, E. S. (2010). Implementing innovation: From visionary models to everyday practice. In H. Dumont, D. Istance, & F. Banavides (Eds.),

ACCEPTED MANUSCRIPT The nature of learning: Using research to inspire practice (pp. 285-315). OECD Center for Educational Research Schulman, K., & Blank, H. (2007). A center piece of the pre-K puzzle: providing state prekindergarten in child care centers. Washington, DC: National Women’s Law Center. Retrieved from https://www.nwlc.org/wp-content/uploads/2015/08/NWLCPreKReport2007.pdf.

IP

T

Schumacher, R., Ewen, D., Hart, K., & Lombardi, J. (2005). All together now: State experiences in using community-based child care to provide pre-kindergarten. Washington, DC: Center for Law and Social Policy. Retrieved from https://files.eric.ed.gov/fulltext/ED484645.pdf.

CR

Scott-Little, C., Kagan, S. L., & Frelow, V. S. (2003). Standards for preschool children’s learning and development: Who has standards, how were they developed, and how are they used? Greensboro, NC: SERVE.

US

Spillane, J. P., Reiser, B. J., & Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72(3), 387-431.

AN

Stephens, S. A. (2014). Community-level challenges in implementing a mixed delivery prekindergarten system: A brief review of research and field experience. New York: Center for Children’s Initiatives. Retrieved from https://eric.ed.gov/?id=ED562091.

ED

M

Ullrich, R., Hamm, K., & Herzfeldt-Kamprath, R. (2016). Underpaid and unequal: Racial wage disparities in the early childhood workforce. Washington, DC: Center on American Progress. Retrieved from https://cdn.americanprogress.org/wp-content/uploads/2016/08/01073800/NSECEreport2.pdf.

CE

PT

U.S. Department of Health and Human Services and U.S. Department of Education. (2016). High-quality early learning settings depend on a high-quality workforce: Low compensation undermines quality. Washington, DC: U.S. Depts. of Health and Human Services and Education. Retrieved from https://www2.ed.gov/about/inits/ed/earlylearning/files/ece- low-compensationundermines-quality-report-2016.pdf.

AC

Valentino, R. (2018). Will public pre-k really close achievement gaps? Gaps in prekindergarten quality between students and across states. American Educational Research Journal, 55(1), 79116. Vespa, J., Armstrong, D. M., & Medina, L. (2018). Demographic turning points for the United States: Population projections for 2020 to 2060. Washington, DC: U.S. Census Bureau. Retrieved from https://www.census.gov/content/dam/Census/library/publications/2018/demo/P25_1144.pdf. Wat, A., & Gayl, C. (2009). Beyond the school yard: Pre-k collaborations with communitybased partners. Washington, DC: Pew Center on the States. Retrieved from https://www.nmefoundation.org/getattachment/Resources/Past-Work/Beyond-the-SchoolYard/BeyondtheSchoolYard.pdf.

ACCEPTED MANUSCRIPT

Weiland, C. (2018). Commentary: Pivoting to the “how”: Moving preschool policy, practice, and research forward. Early Childhood Research Quarterly, 45, 188-192 Weiland, C., & Yoshikawa, H. (2013). Impacts of a prekindergarten program on children’s mathematics, language, literacy, executive function, and emotional skills. Child Development, 84(6), 2112-2130.

IP

T

Weiland, C., & Yoshikawa, H. (2014). Does higher peer socio-economic status predict children’s language and executive function skills in prekindergarten? Journal of Applied Developmental Psychology, 35, 422-432.

US

CR

Westat, Metis Associates, & Branch Associates. (2016). Evaluation of the New York City Pre-K for All initiative, 2014-15: Implementation study report: Expansion rollout. Rockville, MD: Westat.

AN

Whitebook, M., Kipnis, F., Sakai, L., Almaraz, M., & Bellm, D. (2008). Learning together: A study of six B.A. completion cohort programs in early care and education: Year 1 report. Berkeley, CA: University of California, Berkeley, Center for the Study of Child Care Employment, Institute of Industrial Relations.

ED

M

Whitebook, M., & McLean, C. (2017). In pursuit of pre-k parity: A proposed framework for understanding and advancing policy and practice. Berkeley, CA: Center for the Study of Child Care Employment, University of California, Berkeley and New Brunswick, NJ: National Institute for Early Education Research.

PT

Whitebook, M., McLean, C., & Austin, L. J. E. (2016). Early childhood workforce index – 2016: Executive summary. Berkeley, CA: Center for the Study of Child Care Employment, University of California, Berkeley.

CE

Whitebook, M., Phillips, D., & Howes, C. (2014). Worthy work, STILL unlivable wages: The early childhood workforce 25 years after the National Child Care Staffing Study. Berkeley, CA: Center for the Study of Child Care Employment, University of California, Berkeley.

AC

Wilinski, B. (2017). When pre-k comes to school: Policy, partnerships, and the early childhood workforce. New York: Teachers College Press. Yoshikawa, H., Weiland, C., & Brooks-Gunn, J. (2016). When does preschool matter? Future of Children, 26(2), 21-35. Yoshikawa, H., Wuermli, A., Raikes, A., Kim, S., & Kabay, S. B. (2018). Toward high-quality early childhood development programs and policies at national scale: Directions for research in global contexts. Society for Research in Child Development Social Policy Report, 31(1), 1-36.

ACCEPTED MANUSCRIPT

Table 1. Analytic Sample of Schools and Community-Based Organizations (n=57) Total Schools

Types of CBOs

Total CBOs

DOE-only CBOs

Child Care CBOs

Head Start CBOs

7

15

2

3

10

Moderate Resource

9

9

1

6

2

High Resource

6

11

3

7

1

Total

22

35

6

16

T

Low Resource

AC

CE

PT

ED

M

AN

US

CR

IP

13

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN

US

CR

IP

T

Table 2. Comparisons of Pre-K for All Programs in Schools and Community-Based Organizations (n=57) Schools CBOs n=22 n=35 I. Direct services: Duration of programming Hours/minutes open daily 6:57 9:01*** (1:19) (1:32) Open in summer 0 94.0*** On-site child service provision Basic screenings 42.9 42.9 Developmental assessments 54.6 42.9 Therapeutic services 50.0 11.4** Mental health services 27.3 35.3 Medical services 22.7 8.6 On-site family service provision Mental health services 9.1 31.4~ Housing and/or food assistance 0 11.8~ Employment/education assistance 4.6 14.3 Parenting classes 27.3 42.9 Assistance with government applications 4.6 17.1 Child enrollment Dual Language Learners 24.1 43.7* Individual Education Plans 4.9 10.5** Poor 56.6 57.7 Hispanic 40.7 44.1 African American 24.8 38.0~ White 18.6 7.3* Asian 12.8 7.8 Mixed/other 3.3 3.0 Percent of sites >90 percent non-white 54.6 82.9* II. Governance Pre-K for All contracting Contract with Department of Education 100 17.1 Contract with Administration for Children’s Services 0 62.9 Contracts with both Department of Education and 0 20.0 Administration for Children’s Services III. Finance Teacher compensation Annual salary $73,471 $43,660*** (19,990) (8,519) Household income over $100,000 50.0 2.5*** Health insurance from employer/union 86.4 47.7** Retirement plan from employer/union 86.4 61.4* Union member 86.4 65.9~ Administrator compensation Annual salary $117,044 $64,016*** (30,078) (16,177) Household income over $100,000 85.0 33.3*** Health insurance from employer/union 90.9 71.4 ~ Retirement plan from employer/union 95.5 57.1** IV. Pedagogical quality CLASS scores Overall score 5.40 4.99* (0.50) (0.40)

ACCEPTED MANUSCRIPT

Instructional Support Emotional Support Teacher practices Classroom practices match what is expected

1.98~ (0.76) 54.5*

IP

CR

AN

M

ED

PT

CE

2.36 (0.73) 27.3

22.7

Education: Highest degree Bachelor’s Master’s or higher Certification/credentials Has state certification Early childhood teaching Special education Elementary teaching No certification/credential Race/Ethnicity: Hispanic/Latinx African American White Asian Bilingual in English and: Spanish Any second language Ever needed interpreter to talk to parents Teacher recruitment and retention Percent of administrators who said compliance with credential and education requirements was difficult Percent of administrators who said at least one teacher left in prior year Teacher participation in professional development Attended workshops in past year Attended workshops at least monthly Attended on-site workshops Workshops led by Department of Education Workshops led by program staff Received coaching in past year Frequency of coaching sessions Less than once/month

AC

5.71* (0.92) 2.85** (1.03) 6.40 (0.54)

9.1

US

Provide bilingual/multilingual instruction Curriculum, assessments, and early learning standards Percent of administrators who said complying with curriculum requirements was difficult Percent of administrators who said complying with assessment requirements was difficult Percent of administrators who said complying with early learning standards was difficult V. Workforce development Teacher characteristics Years of experience with children under age 5

6.14 (0.49) 3.59 (0.94) 6.48 (0.40)

T

Classroom Organization

20.0 17.1

9.1

31.4~

8.98 (7.04)

9.36 (8.46)

0 100

31.8** 65.9***

90.9 81.8 27.3 54.5 4.5

65.9* 54.5* 18.2 20.5** 20.4~

9.1 18.2 50.0 13.6

27.3~ 34.1 22.7* 11.4

36.4 54.6 59.1

43.2 63.6 47.7

9.1

65.7***

36.4

68.6*

95.5 31.8 52.4 90.9 22.7 100

95.5 65.9** 80.1* 77.3 54.6* 93.2

63.6

54.6

ACCEPTED MANUSCRIPT

M

AN

Rules are confusing

ED

Not enough support

42.9 40.9

31.0 58.5

5.00 (5.62) 7.55 (6.84)

3.97 (4.14) 10.66 (8.86)

0 100

8.6 88.6

18.2

28.6

1.58 (1.44) 3.38 (2.86)

4.27* (5.70) 17.98* (30.89)

2.18 (0.50) 2.05 (0.38) 2.18 (0.59)

2.78** (0.75) 2.88*** (0.75) 2.66* (0.79)

CR

US

Education: Highest degree Bachelor’s Master’s or higher VI. Accountability Percent of administrators who said complying with reporting/monitoring was difficult Number of hours/week administrator spends on budgeting and cost allocation Number of hours/week entire staff spends on budgeting and cost allocation Agree with following statements re budgeting and cost allocation Rules are complicated

45.5 4.6 81.4 27.9

IP

Years of experience as director at any site

36.4 13.6 90.9 22.7

T

At least monthly At least once/week Coaches employed by Department of Education Coaches employed by program Changes in teacher practice Workshops changed practice “a lot” Coaching changed practice “a lot” Administrator characteristics Years of experience at site

AC

CE

PT

VII. Family engagement Percent of administrators who said family engagement was 31.8 42.9 difficult Parents receive materials for at-home learning at least weekly 77.3 48.6* Parent board/council meetings 90.9 85.7 VIII. Linkages to other services and transitions Off-site child service referrals Basic screenings 38.1 68.6* Developmental assessments 45.5 68.6~ Therapeutic services 40.9 88.6*** Mental health services 59.1 79.4 Medical services 50.0 85.7** Off-site family service referrals Mental health services 45.5 65.7 Legal services 36.4 68.6* Housing and/or food assistance 40.9 79.4** Employment/education assistance 40.9 74.3* Parenting classes 45.5 62.9 Assistance with government applications 36.4 74.3** Transitions to kindergarten Visit kindergarten classrooms 95.5 71.4* Note: ~p<.10, *p<.05, **p<.01, ***p<.001. All data are either percentages or means; means are accompanied by standard deviations in parentheses. All significance tests use schools as the reference category. In several

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN

US

CR

IP

T

instances, numbers do not reflect the full sample due to missing data. Some percentages sum to more than 100 because participants gave more than one respons e. “Poor” children are eligible for the Free and Reduced Price lunch program. Scale for match of teacher practice and expectations is 1=not at all to 4=extremely closely. Scale for agreement with statements is 1=strongly disagree to 4=strongly agree. Respo ndents who said program compliance was difficult chose 3 or 4 on a 4-point scale of ease/difficulty. Data sources: Administrator and teacher survey responses, and DOE demographic data.

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN

US

CR

IP

T

Table3: Comparisons of Pre-K for All Programs in Community-Based Organizations with Different Auspices (n=57) DOE-only Child Care Head Start CBOs CBOs CBOs n=6 n=16 n=13 I. Direct services: Duration of programming Hours/minutes open daily 9:02* 9:13*** 8:46* (2:23) (1:19) (1:26) Open in summer 100*** 94.0*** 92.0*** On-site child service provision Basic screenings 0 50.0 53.9 Developmental assessments 0 50.0 53.9 Therapeutic services 16.7 0** 23.1 Mental health services 16.7 40.0 38.5 Medical services 0 12.5 7.7 On-site family service provision Mental health services 0 43.8~ 30.8 Housing and/or food assistance 0 13.3 15.4 Employment/education assistance 0 6.3 30.8 Parenting classes 50.0 25.0 61.5 Assistance with government applications 16.7 18.8 15.4 Child enrollment Dual Language Learners 25.6 44.0 51.7~ Individual Education Plans 9.2 9.3 12.6* Poor 42.0 55.1 68.2 Hispanic 38.8 42.6 48.3 African American 26.3 40.6 40.2 White 20.8 6.6 1.9* Asian 4.5 8.5 8.5 Mixed/other 9.8* 1.9 1.2 Percent of sites >90 percent non-white 33.3 87.5 100* II. Governance Pre-K for All contracting Contract with Department of Education 100 0 0 Contract with Administration for Children’s Services 0 68.7 84.6 Contracts with both Department of Education and 0 31.3* 15.4 Administration for Children’s Services III. Finance Teacher compensation Annual salary $43,999*** $41,829*** $47,000*** (7,378) (8,283) (9,095) Household income over $100,000 0 4.3*** 0 Health insurance from employer/union 0** 47.8* 66.7 Retirement plan from employer/union 50.0 65.2 60.0 Union member 16.7 82.6* 60.0 Administrator compensation Annual salary $62,000*** $62,062*** $67,963*** (29,927) (10,594) (12,556) Household income over $100,000 33.3 33.3* 33.3~ Health insurance from employer/union 33.3* 81.3 76.9 Retirement plan from employer/union 50.0 62.5 53.9~ IV. Pedagogical Quality CLASS scores

ACCEPTED MANUSCRIPT

Instructional Support Emotional Support Teacher practices Classroom practices match what is expected

AC

CE

PT

ED

M

Education: Highest degree Bachelor’s Master’s or higher Certification/credentials Has state certification Early childhood teaching Special education Elementary teaching No certification/credential Race/Ethnicity: Hispanic/Latinx African American White Asian Bilingual in English and: Spanish Any second language Ever needed interpreter to talk to parents Teacher recruitment and retention Percent of administrators who said compliance with credential and education requirements was difficult Percent of administrators who said at least one teacher left in prior year Teacher participation in professional development Attended workshops in past year Attended workshops at least monthly Attended on-site workshops Workshops led by Department of Education Workshops led by program staff Received coaching in past year

4.64** (0.70) 5.13** (0.94) 2.66* (0.99) 6.12 (0.57)

2.00 (0.89) 50.0

1.96 (0.77) 52.2

2.00 (0.76) 60.0

12.5

38.5~

16.7

12.5

23.1

16.7

25.0

46.2~

8.83 (12.97)

9.09 (6.80)

10.00 (9.30)

33.3 66.7

34.8* 60.9*

26.7 73.3

65.9* 54.5* 18.2 20.5** 20.4~

83.3 83.3 33.3 33.3 16.7

52.2* 39.1* 17.4 13.0* 26.1

16.7 0 83.3 0

39.1 30.4 13.0* 13.0

13.0 53.3 13.3~ 13.3

50.0 100 33.3

43.5 60.9 39.1

40.0 53.3 66.7

16.7

68.8***

84.6***

50.0

62.5

84.6*

100 50.0 83.3 100 33.3 100

95.7 73.9* 81.8 73.9 60.9~ 95.7

93.3 60.0 78.6 73.3 53.3 86.7

US

CR

0

AN

Provide bilingual/multilingual instruction Curriculum, assessments, and early learning standards Percent of administrators who said complying with curriculum requirements was difficult Percent of administrators who said complying with assessment requirements was difficult Percent of administrators who said complying with early learning standards was difficult V. Workforce development Teacher characteristics Years of experience with children under age 5

5.13 (0.65) 5.99 (0.78) 2.88 (1.06) 6.53 (0.49)

T

Classroom Organization

5.29 (0.68) 6.06 (0.74) 3.23 (1.05) 6.58 (0.51)

IP

Overall score

Years of experience as director at any site

M

ED

Rules are confusing Not enough support

AC

CE

PT

VII. Family engagement Percent of administrators who said family engagement was difficult Parents receive materials for at-home learning at least weekly Parent board/council meetings VIII. Linkages to other services and transitions Off-site child service referrals Basic screenings Developmental assessments Therapeutic services Mental health services Medical services Off-site family service referrals Mental health services Legal services Housing and/or food assistance Employment/education assistance Parenting classes Assistance with government applications Transitions to kindergarten

60.0 40.0 6.7 64.3 42.9

33.3 50.0

40.9 54.6

14.3 69.2

4.17 (5.08) 6.83 (4.02)

3.13 (3.38) 10.63 (10.86)

4.92 (4.65) 12.46 (7.58)

18.8~ 81.3

0 100

0

37.5

30.8

6.2 (6.18) 27.0 (30.34)

3.8 (2.98) 11.3 (8.95)

3.7 (7.14) 19.4 (42.12)

2.40 (0.55) 2.40 (0.55) 2.40 (0.55)

2.50 (0.65) 2.71* (0.73) 2.43 (0.85)

3.23*** (0.73) 3.23*** (0.73) 3.00* (0.71)

16.7

50.0

46.2

66.7

25.0*

69.2

50.0~

88.0

100

66.7 66.7 66.7 50.0 66.7

75.0 75.0 100** 80.0 87.5~

61.5 61.5 84.6* 92.3 92.3~

50.0 50.0 50.0 33.3 16.7 16.7

56.3 68.8 73.3 75.0 75.0 75.0~

84.6 76.9 100** 92.3* 69.2 100**

US

T

52.2 47.8 4.4 87.0 26.1

0 83.3

AN

Education: Highest degree Bachelor’s Master’s or higher VI. Accountability Percent of administrators who said complying with reporting/monitoring was difficult Number of hours/week administrator spends on budgeting and cost allocation Number of hours/week entire staff spends on budgeting and cost allocation Agree with following statements re budgeting and cost allocation Rules are complicated

50.0 50.0 0 100 0

IP

Frequency of coaching sessions Less than once/month At least monthly At least once/week Coaches employed by Department of Education Coaches employed by program Changes in teacher practice Workshops changed practice “a lot” Coaching changed practice “a lot” Administrator characteristics Years of experience at site

CR

ACCEPTED MANUSCRIPT

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN

US

CR

IP

T

Visit kindergarten classrooms 50.0** 75.0 76.9 Note: ~p<.10, *p<.05, **p<.01, ***p<.001. All data are either percentages or means; means are accompanied by standard deviations in parentheses. All significance tests use schools as the reference category. In several instances, numbers do not reflect the full sample due to missing data. So me percentages sum to more than 100 because participants gave more than one response. “Poor” children are eligible for the Free and Reduced Price lunch program. Scale for match of teacher practice and expectations is 1=not at all to 4=extremely closely. Sc ale for agreement with statements is 1=strongly disagree to 4=strongly agree. Respondents who said program compliance was difficult chose 3 or 4 on a 4-point scale of ease/difficulty. Data sources: Administrator and teacher survey responses, and DOE demographic data.

ACCEPTED MANUSCRIPT Highlights

CE

PT

ED

M

AN

US

CR

IP

T

UPK programs in New York City differ by program setting and auspice. Schools and CBOs serve different child populations with different services. Classroom quality, compensation, and working conditions vary significantly. Head Start sites struggle the most with UPK implementation. Many differences are rooted in a mixed-delivery system for UPK provision.

AC

    