Family selection and child care experiences: implications for studies of child outcomes

Early Childhood Research Quarterly, 15, No. 3, 385– 411 (2000) ISSN: 0885-2006

© 2000 Elsevier Science Inc. All rights of reproduction in any form reserved.

Family Selection and Child Care Experiences: Implications for Studies of Child Outcomes Margaret R. Burchinal and Lauren Nelson Frank Porter Graham Child Development Center and Psychology Department, University of North Carolina-Chapel Hill

Studies of the impact of child care experiences on child outcomes must consider family selection factors because children from more advantaged families tend to attend higher quality child care and are more likely to be in center care than children from less advantaged families. Although this issue is widely recognized, developmentalists and economists have used different statistical methods when testing whether child care experiences are related to child outcomes and have drawn different conclusions from their analyses. This paper discusses some of the family selection issues that should be considered in child care research and provides empirical evidence demonstrating why each issue should be considered. These issues include whether causal inferences can be made from observational studies and the impact on conclusions from regression analyses that include highly correlated measures of child care experiences, nonrepresentative samples, and family covariates with bi-directional effects on child care quality. The secondary data analysis was supported by a grant from Office of Educational Research and Improvement. The Cost, Quality, & Outcomes study was funded by grants from the Carnegie Corporation of New York, the William T. Grant Foundation, the JFM Foundation, the A. L. Mailman Family Foundation, the David and Lucille Packard Foundation, the Pew Charitable Trusts, the USWEST Foundation, and one anonymous foundation, and was conducted by a team of researchers including Donna Bryant, Peg Burchinal, Richard Clifford, Debby Cryer, Mary Culkin, Suzanne Helburn, Carollee Howes, Sharon Lynn Kagan, H. Naci Mocan, John Morris, Leslie Phillipsen, Ellen Peisner–Feinberg, and Jean Rustici. The Carolina Otitis Media Project was supported by the Maternal and Child Health Bureau, Health Resources and Services Administration, Department of Health and Human Service and by the Spencer Foundation. We are grateful to the child care staffs, the children and their families for their participation in these projects. Portions of the paper were presented at the 1999 Biennial Meeting of the Society for Research in Child Development. Direct all correspondence to: Peg Burchinal, FPGCDC, CB# 8185, UNC-CH, Chapel Hill, NC 27599-8185.

385

386

Burchinal and Nelson

There is general agreement between psychologists, educators, and economists that studies of child care must consider family selection factors when examining whether child care experiences are related to child outcomes. Research has clearly demonstrated that parents do not choose child care arrangements randomly (Lamb, 1998). Children from more advantaged families are more likely to attend higher quality child care settings (NICHD Early Child Care Research Network, 1996; Lamb, 1998). Accordingly, analyses that fail to take family selection issues into account are likely to draw faulty or biased conclusions regarding the relations between child care experiences and child outcomes. However, although there is clear consensus that family selection must be considered, there are substantial differences in how economists and developmentalists have approached this problem analytically and in the conclusions they have drawn from their analyses. Family selection is an important issue to consider in child care research because the type and quality of child care are related to demographic and family characteristics that predict child outcomes (Lamb, 1998). In an extensive review of the child care literature, Lamb concluded that parental selection of child care is related to family characteristics. Children from more economically advantaged families in which parents have more education and income are more likely to experience center-based child care as well as higher quality care. In addition, higher quality care has been linked with more responsive parenting and less authoritarian child-rearing beliefs. Parents also appear to make decisions about child care based on child characteristics. Parents are more likely to select center care for preschoolers than infants (Lamb, 1998; Singer, Fuller, Keiley, & Wolf, 1998) and higher quality care for girls than boys (NICHD ECCRN, 1996). In addition to family selection issues, there is growing concern that child care quality may also be directly influenced by child characteristics. Child care quality can be either measured for an entire setting (e.g., center classroom or child care home) or for an individual child. The classroom level measures describe the quality of care by focusing on issues such as the sensitivity of all caregivers in general and the appropriateness of activities for all children in that setting (e.g., Early Childhood Environment Rating Scale; Harms & Clifford, 1980). It is implicitly assumed that all children in the setting receive care of the same quality. The child level measures describe the quality of care provided to an individual child by focusing on the sensitivity of a caregiver to a specific child and the appropriateness of the activities for that child (e.g., Observational Record of the Caregiving Environment, NICHD ECCRN, 1996). These measures explicitly assume that children in the same setting receive care of different quality. Whereas measures collected at the individual child level provide a more accurate description of that child’s experiences, such measures are more likely to be influenced by the child. For example, it is likely that a caregiver will appear more sensitive with a child who is responsive and extroverted than with a child who is hostile and angry. The degree to which family and child characteristics are strongly linked to child care characteristics has been debated. Empirical evidence suggests there are modest associations, especially with child care quality. For example, two large childcare studies reported only modest correlations between family characteristics

Family Selection and Child Care Research

387

and childcare quality measures. In the Cost, Quality, and Outcome Study of over 700 children from four U.S. states in center care, observed child care quality as measured with setting level measures correlated only modestly with maternal education (r ⫽ .24), parent’s progressive attitudes about child rearing (r ⫽ .22), a measure of the home environment (r ⫽ .15), and ethnicity (r ⫽ .06) (Peisner– Feinberg & Burchinal, 1997). Similarly, correlations between observed quality and family characteristics were modest to moderate in the NICHD Study of Early Child Care, another large, multisite study of over 1,200 children in ten U.S. sites (NICHD ECCRN, 1996). This study included care by fathers, grandparents, and other relatives as well as care by unrelated adults; thus, associations between child care and family measures reflect similarities between members of the same family because of shared environments and genetics in addition to reflecting selection effects. Correlations between family characteristics and child care quality measured with a child-level measure at 36 months included modest to moderate correlations with ratings of the home environment (r ⫽ .20 for care by nonrelatives and r ⫽ .46 for care by relatives), observed maternal sensitivity in interactions with child (r ⫽ .26 for care by nonrelatives and r ⫽ .34 for care by relatives), maternal education (r ⫽ .19 for care by nonrelatives and r ⫽ .30 for care by relatives), caregiving attitudes (r ⫽ .12 for care by nonrelatives and r ⫽ .25 for care by relatives) and family income (r ⫽ 0.20 for care by nonrelatives and r ⫽ .27 for care by relatives). The correlations between child characteristics and observed care quality were smaller. They ranged from very small correlations with maternal rating of child temperament (r ⫽ ⫺.01 for care by nonrelatives and r ⫽ ⫺.08 for care by relative), behavior problems (r ⫽ ⫺.08 for care by nonrelatives and r ⫽ ⫺.02 for care by relatives), to modest correlations with a prior measures of cognitive level (r ⫽ .17 for care by nonrelatives and r ⫽ ⫺.02 for care by relatives) and sociability (r ⫽ .17 for care by nonrelatives and r ⫽ .19 for care by relatives). Overall, these child and family characteristics accounted for 11% of the variance in the child care quality composite score in observations of nonrelative care and for 26% of the variance in observations of relative care. Other smaller childcare studies have reported similar, modest correlations between family and child characteristics and child care quality (cf., Lamb, 1998).

IMPLICATIONS FOR CHILD CARE STUDIES Family selection factors are almost always considered in current child care research. Although the initial child care studies published in the 1970s and 1980s by developmentalists often did not consider family selection factors, all studies now must include family selection factors as control variables in their analyses to be credible (Lamb, 1998). Developmentalists identify family selection variables to include as control variables by identifying family characteristics that are significantly related to both the child care experiences and to child outcomes. These “family selection” characteristics, along with other family and child characteristics are entered as covariates in regression analyses. For example, both large scale child care studies, the Cost, Quality, and Outcomes Study (CQO;

388

Burchinal and Nelson

Helburn, 1995) and the NICHD Study of Early Child Care (NICHD ECCRN, 1998; 2000), identified characteristics such as maternal education and parenting to include as family selection factors after demonstrating they were significantly correlated with both child care quality and child outcomes. Although these analytic methods do not eliminate bias because of family selection issues, they do reduce that bias without substantially reducing the precision of the childcare quality measure. Most developmental studies have concluded that child care quality is modestly to moderately related to child outcomes, even after considering family selection issues (Lamb, 1998). Other disciplines, such as economics, have approached these issues differently. They employed specialized regression methods and focused on whether the predictors should be endogenous or exogenous. These specialized methods are useful when a researcher has sufficient confidence in the underlying theory to believe that the correct equation for predicting the dependent variable is known, even if all predictor variables were not measured. Based on the assumption that the correct regression model is known, the statistical methods correct for bias in the estimated regression coefficients because of the omission of unmeasured predictor variables (i.e., omitted variable bias). Two such approaches, the instrumental variable regression and fixed-effect regression, are described in more detail below; both approaches provide unbiased estimates of the relationship between a given predictor and the outcome variable when the prediction model is correctly specified. However, both methods provide considerably less power for hypothesis testing than ordinary multiple regression methods. After using these methods, some economists have concluded that childcare experiences are, at best, trivially related to child outcomes after considering family selection issues (e.g., Blau, 1999). Because economists and developmentalists have very different traditions regarding modeling or analyzing child care data, it has been difficult for each discipline to understand the issues and methods used by the other. We believe there are four statistical issues that have implications for analyzing child care data and that explain in part why developmentalists and economists draw different conclusions regarding the relationship between child care quality and child outcomes. The four issues include two issues that represent fundamental differences between the two disciplines, and two other issues that both disciplines should consider. The two disciplines make different assumptions when analyzing observational data. First, some economists, but not most developmentalists, believe that causality can be inferred from observational data when analyzed appropriately. Second, some economists argue that all measured family and child care variables should be included in analyses to decrease the bias in parameter estimation, whereas most developmentalists believe that highly correlated predictors should not be included in the same analysis to increase power to test hypotheses. Furthermore, there are at least two fundamental problems in analyzing child care data that are not always considered by either discipline. First, both disciplines use nonrepresentative samples that often include under-representation of lowquality child care. Reducing the range and distribution of quality scores in the

Family Selection and Child Care Research

389

sample also reduces power to test hypotheses about associations between child care quality and outcomes. Second, both economists and developmentalists assume that family characteristics influence the selection of higher or lower quality care. However, conclusions from both of their analyses may be flawed if quality of care influences family characteristics. The following sections discuss these four issues with empirical examples provided from two child care studies.

DATASETS/CHILD CARE STUDIES CQO The first dataset is from the Cost, Quality, and Outcomes Study (CQO), a large-scale study conducted in child care centers in California, Connecticut, Colorado, and North Carolina (for more details see Helburn, 1995; Pesiner– Feinberg & Burchinal, 1997). These states varied widely in both economic climate and the stringency of state child care regulations. For example, regulations regarding staff:child ratios for 4-year-olds varied from 1:10 in Connecticut to 1:20 in North Carolina. The initial sample of centers for the cost and quality component of the study was randomly selected from state day care licensing lists, including approximately half for-profit (n ⫽ 50) and half nonprofit (n ⫽ 50) in each state. To be included in the study, centers had to provide full-time care. Full-time care was operationally defined as care available at least 11 months of the year, and served the majority of the children at least 30 hours (or 5 days) per week. Once a center and classroom teacher agreed to participate, consent forms were given to the teacher to send home to parents of all potentially eligible children, with up to 12 children randomly selected from each classroom. Four criteria were applied for inclusion in the sample: (a) children were eligible to enter kindergarten in the fall of 1994 (i.e., in the same age cohort); (b) children were enrolled in the classroom during the time it was observed for the quality data collection; (c) parents reported they expected the child to continue attending that center the following year to minimize changes in the independent variable during the subsequent year of data collection (not reported here); and (d) the primary language spoken in the child’s home was English. Each classroom was observed for approximately 3 to 4 hours in a single visit. Four observational measures of process quality were used: (a) global quality was measured using the ECERS (Harms & Clifford, 1980); (b) teacher sensitivity was measured with the Caregiver Interaction Scale (CIS) (Arnett, 1989); (c) childcenteredness was measured by the UCLA Early Childhood Observation Form (ECOF) (Stipek, 1993); and (d) teacher responsiveness was measured with the Adult Involvement Scale (AIS) (Howes & Stewart, 1987). Two assessors gathered the data; one collected ECERS, CIS, and ECOF data, and the other collected the AIS data. Individual assessments of each child were conducted at the child care center in a session that lasted approximately 30 minutes. After the assessments, questionnaires were given to teachers. The parent demographic questionnaires were sent

390

Burchinal and Nelson

home with the original consent form. Preaddressed, stamped envelopes allowed parents and teachers to return the forms to the investigators. The return rates were high: 98% for the parent surveys and 96% for the teacher surveys. Parents provided demographic information about the child and family. Individual assessments of the children’s cognitive and socio-emotional developmental status were conducted by the child’s teacher and by a trained research assistant who had not participated in the classroom ratings. These assessments were conducted using four instruments: (a) receptive language ability was measured using the Peabody Picture Vocabulary Test-Revised (PPVT-R) (Dunn & Dunn, 1981); (b) preacademic skills were measured using the Woodcock-Johnson Tests of Achievement-Revised (WJ-R) (Woodcock & Johnson, 1990); (c) children’s self-perceptions of competence and attitudes toward child care were measured using The Attitudes/Perceptions of Competence (Stipek, 1993); and (d) teachers rated children’s social and cognitive skills on the Classroom Behavior Inventory (CBI) (Schaefer, Edgerton, & Aaronson, 1978). In addition, another measurement of child care quality was based on teacher ratings of the teacher-child relationship using the Student-Teacher Relationship Scale (STRS) (Pianta, 1992; Pianta & Steinberg, 1992). See Table 1 for the descriptive statistics on the measures of child care quality, the family factors and child outcomes for the CQO study. COMP The Carolina Otitis Media Project included 89 African American infants attending community-based child care centers who were part of a larger prospective longitudinal project examining the effects of otitis media and associated hearing loss on African American children’s language development (for more details see Burchinal et al., 1996; Roberts et al., 1995). Initially, nine child care centers in two adjacent small southeastern cities were selected to participate in the study because each center enrolled at least two African American infants. The families of all African American infants less than 12 months of age attending those centers were invited to enroll. The selection criteria involved were ethnicity and the child’s age: only African American children who were less than 12 months of age at enrollment and apparently developing normally were recruited. The children were followed prospectively if they remained in child care for at least 3 months, or if their parents reported they were going to place their child in another child care center within the next year. By 3 years of age, these children had attended at least one of 27 local centers; some children remained in the same center for all 3 years, whereas others changed centers frequently. The 89 children included in this study entered their child care program between the ages of 1 and 11 months (mean of 5.4 months), and all attended child care at least 30 hours per week. Most of the children (69%) were from families with incomes that qualified them for federal programs to assist poor families (i.e., incomes less than 185% of the federal poverty threshold). Single parents (68%) headed many of these families. Family characteristics were measured annually. The annual interview with the primary caregiver included administration of the Parenting Stress Index (PSI)

Family Selection and Child Care Research Table 1.

391

Cost, Quality, and Outcomes Study: Descriptive Statistics on Measures of Child Care Quality, Family Factors, and Child Outcomes n

Child Care Structure Profit Status For-Profit Nonprofit Child-Teacher Ratio Group Size Child Care Quality ECERS-Global Quality CIS-Teacher Sensitivity ECOF-Child Centeredness AIS-Teacher Responsiveness Teacher-Child Relationship STRS-Closeness STRS-Conflict STRS-Overdependency Child and Family Factors Child Age Child Gener Male Female Child Ethnicity African-American Latino Other White Maternal Education Parental Marital Status Married Single Other Family Income Child Outcomes PPVT-R WJ-R Reading WJ-R Math CBI-Sociability CBI-Problem Behavior CBI-Positive Behavior

Mean

SD

7.72 14.12

3.59 8.29

1.8–28 3–71

177 177 177 177

4.25 2.96 3.49 0.31

1.03 0.55 0.82 0.27

1.7–6.6 1.4–3.8 1.4–5.0 0.0–1.0

730 730 730

4.17 1.80 2.09

0.58 0.77 0.73

1.9–5.0 1.0–4.8 1.0–4.6

757 757 387 370 757 120 35 88 514 757 754 525 105 124 511

4.30

0.36

3.1–5.7

14.22

2.24

10–20

38,900

21,200

2,400–72,000

93.59 99.65 102.38 3.98 2.45 3.62

18.48 13.02 13.45 0.71 0.86 0.71

40–144 63–161 46–139 1.3–5.3 0.8–4.9 1.4–5.0

177 82 95 173 176

757 755 720 729 728 728

%

Range

46.3 53.7

51.1 48.9 15.9 4.6 11.6 67.9

69.6 13.9 16.5

(Abidin, 1990) at entry, 18 months, 30 months, and 42 months. The PSI is a rating scale designed to assess levels of parenting stress in the family. It has 12 scales that measure characteristics of the child and the parent or primary caregiver, including a scale that describes the mother’s overall sense of competence as a

392

Burchinal and Nelson

parent. Maternal sensitivity was measured from interactions between mother and their infants. Mother and children interacted during a unstructured free play at 1 year and book reading at ages 2, 3, and 4 years, and their interactions were coded from videotapes using the qualitative ratings of MULTI-PASS (Marfo, 1992). MULTI-PASS is a coding scheme that includes both behavioral counts and qualitative ratings to examine parent-child interactions. A composite score was created from ratings of warmth, sensitivity, responsiveness, and encouragement of initiative, stimulation value, and elaborativeness after factor analysis indicated these ratings could be combined into a single score. The child-rearing environment at home was assessed with two versions of the Home Observation for Measurement of the Environment (HOME) (Elardo & Bradley, 1981). The HOME examines the quality and responsiveness of the home environment during a semistructured interview, measuring aspects of the child’s home environment that are believed to foster cognitive and social development. The HOME total, computed as the proportion of items passed, was used in the analyses. Children’s cognitive development was assessed with the original Bayley Scales of Infant Development (Bayley, l969) at 1 and 2 years of age and with the revised Bayley (Bayley, 1993) at 3 years of age. The overall cognitive level is represented by the Mental Developmental Index (MDI). These scores are norm-referenced standard scores (M ⫽ 100, SD ⫽ 15), with lower scores in general on the revised scales because of the renorming of the test (Bayley, 1993). See Table 2a for a description of the child and family characteristics of this sample and Table 2b for a description of the children’s child care experiences.

STATISTICAL ISSUES Inferring Causality From our perspectives as developmentalists, much of the controversy between developmentalists and economists regarding selection effects is because of fundamental differences in analytic traditions. Psychologists regard the experimental design with random assignment to treatment and control groups as the gold standard against which all other designs are compared (Kirk, 1982). Most psychologists believe that it is not possible to infer cause and effect from observational studies, with the caveat that overwhelming evidence across studies can be interpreted causally such as with studies linking smoking to cancer and lung disease. It is likely that random assignment designs are not possible in economics. Instead, some economists believe causality can be inferred from observational data when analysis models accurately reflect theoretical models. They argue that statistical methods can produce accurate descriptions of true relationships between predictor and outcome variables based on the assumption that the theory correctly delineates the complete prediction equation for the dependent variable. Perhaps because they disagree about whether observational data can be used to infer causality, the two disciplines place very different emphasis on the consequences of making mistakes when testing hypotheses. Many of the differences in

Family Selection and Child Care Research Table 2a.

393

Carolina Otitis Media Project: Descriptive Statistics on Measures of Child Care Quality, Family Factors, and Child Outcomes

Family Characteristics Ethnicity-African-American Marital status-Married Mother’s education, years completeda Mother’s age, yearsa Poverty status-Below poverty lineb Quality of Home Environment HOME Total-Infant/Toddler version HOME 12 m HOME 24 m HOME 36 m Child characteristics Gender-Male Age began child care (m) Bayley-Mental Developmental Index MDI (original) 12 m MDI (original) 24 m MDI (revised) 36 mc

N

%

89 29 89

100% 32%

89 61

86 65 51

SD

Range

12.5

2.3

8–19

25.8

7.7

14–63

34.15 35.88 36.53

6.08 4.99 5.18

19–44 19–45 19–42

5.4

3.0

1–11

114.26 110.83 95.74

13.45 20.41 10.15

69%

86 65 51 42 89

M

47%

81–140 62–150 79–122

Notes a Reported the primary caregiver, who was not the mother in several cases. b Poverty based on 185% federal guidelines for determining poverty. c Revision of the Bayley MDI resulted in substantially lower MDI scores due to updating the standardization population.

statistical orientation between economists and developmentalists appear to involve differences between the disciplines in concerns about Type I and II errors. The Type I error occurs when analyses falsely indicate a predictor is reliably related to an outcome when it is not really related in the population. The Type II error occurs when analyses falsely indicate the predictor is not reliably related to the outcome when it is really related in the population. Some economists are very concerned about Type I errors because they believe that analysis of observational data can be used to draw causal inferences. Most developmentalists worry about both Type I and II errors because they believe analysis of any single sample of observational data can be used to describe associations, not draw causal inferences. This use of random assignment as the basis for causal inference could not be examined by either the CQO or COMP study because both are observational studies. The only major child care studies that involved random assignment to a child care setting or a comparison group were the early intervention studies such as the twelve studies conducted in the 1960s discussed in the Lazar monograph

394 Table 2b.

Burchinal and Nelson Carolina Otitis Media Project: Child Care Classroom Description of Infant, Toddler, Two- and Three-Year-old Classrooms

n Process Quality ITERS Total ECERS Total

M SD M SD

Infant

Toddler

18

22

3.1 0.9

3.4 0.9

Two 22 3.1a 1.0 4.0a 0.9

Three

Mixed

21

3 4.7b

4.0 0.8

5.6b 0.8

Notes a There were five 2-year-old classes with ITERS scores and 17 with ECERS scores. b There was one mixed-age class with an ITERS score and two mixed-age classes with ECERS scores.

(Lazar & Darlington, 1982), Perry Preschool/High School Project (Schweinhart, Weikart, & Larner, 1986), Infant Health and Development Project (IHDP, 1990), and Abecedarian Project (Campbell & Ramey, 1994). Among these studies, the projects with the most intensive interventions have demonstrated nontrivial longterm child care effects on cognitive, academic, and social outcomes for children from families living in poverty. Some economists believe that special types of regressions allow one to make causal inferences with observational data by adjusting statistically for “omitted variable bias” (Davidson & Mackinnon, 1993). This is an important issue for child care research since this type of bias is likely because child care selection is not random. Econometricians have used at least two statistical methods to adjust for this omitted variable bias in observational data, “instrumental variables” and “fixed-effects” regression. The instrumental variables approach is based on an Aiken two-stage least square model (for more details see Doran, 1989; Foster & McLanahan, 1996). This approach assumes that the variance in a predictor of interest, which is shared with variables unrelated to the selection factors, will provide an unbiased predictor of that outcome. As with estimating latent variables from observed variables in structural equation models, the logic of this approach involves replacing the observed measure of child care quality with an estimate that corrects for both bias and error in the observed measure. The observed values of child care quality are predicted from exogenous indicators, called instrumental variables, which are indirectly related to outcome through their relationship with child care quality. Thus, these indicators are completely independent of the family. The estimated “latent” quality variable should contain all of the reliable information about child care quality that overlaps with the selected indicators. This “latent” variable will not include the random error in measurement in the original child care quality variable, but it will only represent the reliable aspects of quality correlated with the selected instrumental variables. This new “latent” variable will be more reliable if all reliable predictors of child care quality are included as instrumental variables, but will be considerably less reliable if reliable independent predictors of quality were not included in creating the “latent” variable.

Family Selection and Child Care Research

395

Specifically, this approach involves two steps. In the first step, the primary predictor of interest is predicted from instrumental variables assumed to be directly related to the primary predictor of interest but not to the outcome measure. It is also assumed that the true measure of quality can be predicted, with an equation such as the one below: T ⫽ ␤0 ⫹ ␤1 state regulation ⫹ ␤2 availability ⫹ ␤3 staff:child ratio ⫹ ␤4 CG training ⫹ ␤5 CG attitudes ⫹ ␤6 CG experience ⫹ . . . where T is the true child care quality score for that child, ␤ is a regression coefficient, representing the unique contribution of that measure, state regulation is the level of regulation in that child’s state, availability is number of child care arrangements available in the community, staff:child ratio is the number of adults to the number of children in the classroom, and CG training, attitudes, experience, represent the level of training, the attitudes about caregiving, and the experience of the caregiver (CG). The “latent” quality scores are then estimated only from the instrumental variables, as

␤*0 ⫹ ␤*1 state regulation ⫹ ␤*2 availability. These regression coefficients (␤*) represent the unique contribution of the instrumental variables and any overlapping variance with other predictors of true quality. Excluded from this “latent” variable is any information about quality provided by the other predictors that is not overlapping with the instrumental variables. In the above example, information about child care quality represented by

␤3 staff:child ratio ⫹ ␤4 CG training ⫹ ␤5 CG attitudes ⫹ ␤6 CG experience ⫹ . . . would not be included in the “latent” variable to the extent that those variables are uncorrelated with state regulation and availability. According to economists, the instrumental variables must be exogeneous- that is, they have an indirect effect on the outcome via the child care quality, but not a direct effect. They also must be variables that represent aspects of child care that are not involved in the parental selection process. Typically, instrumental variables include availability of child care in the community and level of regulation in the state. In the second stage, a hybrid regression analysis tests whether this latent quality variable is related to the outcome. Two regressions are performed and combined. Child outcomes are predicted twice, from the “latent” quality variable in the first regression and from the original observed quality measure in the second regression. The parameter estimates from the analysis involving the “latent” variable are combined with the estimated error variance from the model involving the original quality variable. This hybrid approach is designed to compensate for the increase in error that occurred from discarding information about child care quality,

396

Burchinal and Nelson

increasing the standard error in the analysis involving the “latent” quality variable. Evidence of an unbiased or causal relationship is obtained if the predicted values are significantly associated with the outcome in this analysis and the residuals are not correlated with any of the predictors from the first or second stage. With this approach, parameter estimates of the association between child care quality and outcomes should be unbiased when omitted variables have only main effects on quality, not interactive effects. In contrast, the reliability or precision of the measures of child care quality is markedly decreased when there is substantial variance in the “true” measure of quality that is not related to the instrumental variables. In that case, then power to detect associations between child care experiences and outcomes is decreased. This is a major concern for child care research since the variables used to create these “latent” quality measures tend to account for a very small portion of the variance in the observed quality measures. If these “latent” variables account for a very small portion of the “true” variance of the quality measures, the likelihood of detecting associations between those “latent” measures and child outcomes is very small unless either samples or effect sizes are large. Power to detect “true” associations is decreased when “true” variance is discarded by computing the predicted values from the instrumental variables. The extent to which the correlation or partial correlation reflecting the association between child care quality and child outcome can be detected in any study depends on the “true” correlation (i.e., the correlation one would obtain if we could measure everyone in the population without bias or error) and the reliability of measurement of child care quality and child outcomes. The correlation that one would expect to obtain in a sample would be, where r (X,Y) ⫽ ␳ (X,Y) ⫻ Rel (X)1/2 ⫻ Rel(Y)1/2, r (X,Y) is the observed correlation, ␳ (X,Y) is the true correlation between X and Y, Rel (X)1/2 is the square root of the reliability in measurement of X, and Rel (Y)1/2 is the square root of the reliability in measurement of Y. If we anticipate that the true correlation between child care quality and vocabulary skills is r ⫽ .25 and we have measured child care quality and vocabulary skills with 90% reliability (i.e., reflecting inter-rater differences as well as failure by the assessment tools to completely and accurately measure the quality of the classroom and the child’s vocabulary skills), then we expect the observed correlation to be 0.25 ⫻ 0.901/2 ⫻ 0.901/2 ⫽ 0.225 (Cohen, 1988). The power to detect a correlation of 0.225 would be 80% or greater in studies with sample sizes of 150 children or more. The instrumental variable approach would replace observed quality with latent variable based on exogenous measures such as level of regulation of child care in the state and number of child care arrangements available in a community. Such measures likely account for 10% to 25% of the variance in child care quality. If observed quality scores are replaced with predicted quality scores, thereby discarding about 75% to 90% of the true variance, the expected correlation is 0.25 ⫻ 0.251/2 ⫻ 0.901/2 ⫽ 0.119. Use of the

Family Selection and Child Care Research

397

estimated error variance from the analysis model involving observed quality increases the reliability somewhat. The reliability of the child care quality measure in these analyses that used the latent quality variable for estimating parameters and the observed quality variable for estimating error would be between 0.25 and 0.90. If the reliability was 0.60, then expected correlation is 0.25 ⫻ 0.601/2 ⫻ 0.901/2 ⫽ 0.184. The power to detect a correlation of 0.19 would be 80% or greater in studies with sample sizes of over 215 children. Thus, these methods can provide unbiased estimates of the relationship between child care quality and child outcomes, but the ability to accurately test whether those associations differ from zero (i.e., the test of significance of the correlation) is substantially reduced when information is discarded. Neither of our studies collected sufficient exogenous information to permit an empirical examination of this type of analysis. Another approach advocated by economists to adjust for selection bias is the fixed-effect regression. This approach can be used when the study design involves repeated measures of child care quality and child outcomes, and theory posits the omitted variables have only main effects on the predictors and the outcomes (Greene, 1997). The nested factor could be repeated assessments over time of the same child, multiple assessments of children within a classroom, or multiple assessments of siblings within a family. This approach involves replacing observed values with difference scores for all predictor and outcome variables to correct for omitted variable bias. It is believed that the effects of the unmeasured variables are removed from both the predictors and the outcome because the main effects of the omitted variables would be removed from both predictor and outcome measures as the difference scores are computed. Each difference score is computed as the difference between the observed value for that individual and the mean score on that variable for that individual’s level of the nested factor. For example, fixed-effect regressions can be based on separate assessments of child care quality and child outcome for multiple children within a classroom, making classroom the repeated factor. Then, each difference score is computed as the difference between the child’s scores and that child’s classroom mean scores for each predictor and outcome variable. Similarly, fixed-effect regressions are possible if repeated measures of child care quality and child outcomes were collected over time, making time the repeated factor. In those analyses, the scores for each individual at each assessment time would be replaced by the difference between the child’s scores at each time point and the across-time mean scores for that child for all predictor and outcome variables. Use of the difference scores eliminates statistical effects because of unmeasured or omitted variables as long as the omitted variables do not interact with the nested factor (i.e., only main effects on the outcome and predictors). However, the precision of the difference scores will be substantially less than the precision of the original measures of child care experience or child outcomes when the repeated measures are highly correlated. Use of difference scores for independent and dependent variables often increases error and decreases power to test hypotheses (Cronbach & Furby, 1970). The reliability of the difference score depends on the reliability of original scores, the amount of true change (i.e., difference between observed and mean scores), and the correlation between the repeated

398

Burchinal and Nelson

measures use to create the difference scores. Difference scores will be completely unreliable if the assessment tools are unreliable or there is no true change. No true change would occur if a child’s vocabulary scores over time do not change or if all children in a classroom achieve the same vocabulary score. Difference scores will be up to half as reliable if the two repeated measures are positively correlated, even with substantial true change. Like the instrumental variable approach, the fixed-effects regression approach is likely to have reasonable power to detect associations only if sample sizes are large or the “true” association between child care quality and a child outcome is substantial and uncorrelated with initial observations of quality and outcome. We extended the previous power analysis example for estimating the correlation between observed child quality and child vocabulary. We estimated power for a study that collected repeated measures over time and for a study that collected multiple assessments within the same classroom. In the first example, we assumed that a study had measured vocabulary and child care quality on multiple occasions for a sample of children, and that the repeated measures of both vocabulary and child care quality showed nontrivial across-time correlations. If there are moderate to large across-time correlations, the decrement in reliability would be moderate to large. If the reliability of difference score was 60% of the original score, then we would expect to obtain a correlation of 0.25 ⫻ (0.60 x 0.90)1/2 ⫻ (0.60 x 0.90)1/2 ⫽ 0.135. The power to detect a correlation of 0.135 would be 80% or greater in studies with sample sizes of over 375 children. In the second study, we assumed that assessments had been collected on multiple children within the same classroom and that the within-classroom correlations were modest. With modest within-classroom correlations, the decrement in reliability is modest (e.g., 10%), then we would expect to obtain a correlation of 0.25 ⫻ (0.90 ⫻ 0.90)1/2 ⫻ (0.90 ⫻ 0.90)1/2 ⫽ 0.2025. A sample of 190 children or more would be necessary to have 80% power to detect such a correlation. To demonstrate one of these methods, we conducted a fixed-effects regression using the CQO data. We were able to treat the classroom as a nested factor because more than one child was recruited from most classrooms in this study. The major research question was the extent to which child care quality was related to child outcomes after adjusting for family selection biases. We were able to implement this approach because we had a separate measure of quality for each child in each class. The teacher’s rating of her closeness with each study child in her class was one of the two measures of child care quality used in this study (for details see Peisner–Feinberg & Burchinal, 1997). We attempted to eliminate the impact of omitted variables, including family selection variables, by subtracting the classroom mean score from each child’s predictor and outcome variable scores. Specifically, in the original analysis we predicted the child’s language skills from child care quality, state, mother’s education, gender, and ethnicity (see the following equation for Model 1). Model 1: E(PPVTij) ⫽ B0 ⫹ B1(T-C Closeij) ⫹ B2(M. Edij)⫹ B3(Genderij) ⫹ B4(Ethnicityij) ⫹ B5 (Statej),

Family Selection and Child Care Research

399

where PPVTij is the language score of the i-th child in the j-th classroom, T-C Closeij is teacher rating of closeness with the i-th child in the j-th classroom, M-Edij is the mother’s education of the i-th child in the j-th classroom, Genderij is the gender (male ⫽ 1, female ⫽ 0) of the i-th child in the j-th classroom, Ethnicityij is the ethnicity (white/nonHispanic ⫽ 1, nonwhite ⫽ 0) of the i-th child in the j-th classroom, and Statej refers to whether the center was in CA, CO, CT or NC. Furthermore, B0 is the overall intercept, B1 is the estimated increment in language scores associated with teacher-child closeness, B2 is the estimated increment associated with maternal education, B3 is the estimated increment associated with being male, B4 is the estimated increment associated with being white, and B5 is the estimated increment associated with being from a specific state. In the second model, we computed a difference score for the outcome variable as the difference between the study child’s language skills and the overall classroom language skill. Predictor difference scores were computed: (a) the difference between that child’s teacher-child closeness rating and the overall classroom rating; and (b) the difference between the child’s mother’s education and the mean mother’s education for the classroom; (c) the difference between the child’s gender and the mean gender for the classroom; and (d) the difference between the child’s ethnicity and the mean ethnicity for the classroom. Model 2: E(PPVTij- PPVTj) ⫽ B0 ⫹ B1 (T-C Closeij -T-C Closej) ⫹ B2 (M. Edij -M. Edj) ⫹ B3 (Genderij - Genderj) ⫹ B4 (Ethnicityij - Ethnicityj ), where PPVTij is the language score of the i-th child in the j-th classroom and PPVTj is the mean language score of the recruited children in the j-th classroom; T-C Closeij is teacher rating of closeness with the i-th child in the j-th classroom and T-C Closej is mean rating for the j-th classroom; M-Edij is the mother’s education of the i-th child in the j-th classroom and M-Edj is the mean of the j-th classroom; Genderij is the gender (male ⫽ 1, female ⫽ 0) of the i-th child in the j-th classroom and Genderj is the mean gender in the j-th classroom; Ethnicityij is the ethnicity (White/nonHispanic ⫽ 1, nonwhite ⫽ 0) of the i-th child in the j-th classroom and Ethnicityj is the mean ethnicity in the j-th classroom. B0 is the overall intercept, B1 is the estimated increment in language scores associated with teacher-child closeness, B2 is the estimated increment in language scores associated with maternal education, B3 is the estimated increment in associated being male, B4 is the estimated increment in language scores associated with being white, and B5 is the estimated increment in language scores associated with being from a specific state (CA, CO, CT, or NC). Model 2 includes all predictors from Model 1 that varied within the classroom. One variable, state, was excluded because all children within a classroom were from the same state. The analysis involving Model 2 should provide an unbiased

400 Table 3.

Burchinal and Nelson Ordinary OLS and Fixed-Effect Regression Analysis of CQO Data: Predicting Language Skills from Child Care Quality Multiple Regression

R2 Regression Coefficients (se) T-C Closeness Maternal Education Gender (male⫽1) Ethnicity (white⫽1) Statea CA CO CT

.31*** 3.56*** (.97) 2.07*** (.25) ⫺2.75* (1.11) 12.27*** (1.24) *** 6.31*** (1.55) 8.29*** (1.55) 12.90*** (1.61)

Fixed-Effect Regression .11*** 4.04*** (1.00) .76** (.25) ⫺1.86 (1.01) 10.00*** (1.39)

Notes aIn the first regression, NC was the reference cell so the other parameter estimates are comparisons between each of the other states and NC. State was not included in the fixed-effect regression because all children within the same class are from the same state, therefore that predictor reduced to zero when difference scores were computed. * p ⬍ .05; ** p ⬍ .01; *** p ⬍ .001.

estimate of the relations between teacher-child closeness and language skills under the assumption that all omitted variables have the same impact at the classroom level because their influence has been subtracted out of both the predictors and the outcome. That is, we assumed each omitted variable has a main effect, but that none interact with included or omitted variables. The results of the two analyses indicate that children with closer relationships with their teachers tended to have higher language scores (see Table 3). The analysis of Model 1 was an ordinary least squares (OLS) regression or typical multiple regression. The results from this analysis are listed under the heading multiple regression. The analysis of Model 2 was a fixed effect regression, and results from this analysis are listed under that heading. Comparisons of the results of the two analyses indicate that the fixed-effect regression accounts for less variance (R2 ⫽ 0.11, p ⬍ .001) than the multiple regression (R2 ⫽ 0.31, p ⬍ .001) because the child and family variables are stronger predictors in the multiple regression (e.g., B ⫽ 2.07, p ⬍ .001 for maternal education) than in the fixedeffect regression (e.g., B ⫽ 0.76, p ⬍ .01 for maternal education). Nevertheless, the child care quality measure provides similar prediction of language skills with the multiple regression (B ⫽ 3.56, p ⬍ .001) and the fixed-effect regression (B ⫽ 4.04, p ⬍ .001). These results indicate that the observed association between the child’s language skills and teacher-child closeness observed in multiple regression analyses of the CQO data are not simply because of omitted family selection variables. There was reasonable power for this fixed-effects regression because within-classroom correlations were very modest (i.e., within classroom correlations of r ⬍ .10) for both child outcomes and the child care quality measure.

Family Selection and Child Care Research

401

Correlations Among Predictors The second statistical issue that separate some economists and developmentalists is whether moderately to highly correlated child care predictors should be included in regression analyses. At least some economists argue that child care variables should be included in any analyses (e.g., Blau, 1999), whereas at least some developmentalists argue that child care variables should be selected carefully to avoid highly correlated predictors (e.g., NICHD ECCRN, 1998). This is an important issue because conclusions drawn from examining regression coefficients must be tempered by the degree of overlap between predictors. The regression coefficient describes the degree to which a predictor variable provides independent linear prediction of the outcome variable. So including moderately to highly correlated predictors is a problem because it is not possible for a predictor to independently account for substantial variance if the predictors are moderately to highly correlated. Indeed, it is possible to have an overall model that is highly significant in which none of the individual variables is a significant predictor. This problem exists regardless of whether the relation between the predictor and outcome is causal or observational. For this reason, statisticians recommend using fewer predictors or forming a single composite or a set of uncorrelated composites through factor analysis when predictor variables are moderately to highly correlated (Mosteller & Tukey, 1977). This is an issue for child care researchers. It can be a problem either when regression models include highly correlated indicators of the child care experience or when both structural and process quality measures are used to predict child outcomes. Typically the regression coefficient is interpreted as indicating whether that aspect of child care is reliably related to child outcomes. That interpretation is probably wrong when highly correlated measures such as caregiver education and training or class size and ratio are all included in the same analysis. For example, caregiver education and training are highly correlated since almost all levels of child care training requires formal education. In that case, a regression analysis that includes both education and training is asking the extent to which caregiver training is related to outcomes if teachers had the same education. In that case, it is impossible to increase training without increasing education, so it is of limited value to test the extent to which training predicts child outcome after adjusting for education. Use of both structural and process measures of child care as predictors can also cause problems with interpreting results. If one believes that structural measures such as better adult-child ratios or more caregiver training provide the underpinnings for sensitive and responsive child care, then including all three measures in the same analysis is problematic. There are logical inconsistencies in analyses that ask about the impact of child care quality on child outcomes after statistically adjusting for some of the factors that lead to higher quality care. This issue applied to child care research involves the extent to which associations between child care experiences and child outcomes may be underestimated because of the inclusion of highly correlated predictors. We examined this question with the Cost, Quality, and Outcomes data by looking at the degree to

402

Burchinal and Nelson Table 4.

Cost, Quality, and Outcomes Project: Predicting PPVT from Correlated Measures of Child Care Quality Model 1

Variance Accounted For Total Model R2 Child Care Variables R2 Regression Coefficients Intercept B(se) State (NC is reference cell) CA CO CT Ethnicity (White ⫽ 1) Gender (Male ⫽ 1) Child Care Block Child Care Quality Composite ECERS CIS % T. Responsive Child Centered Group Size Ratio T. Education

Model 2

Model 3

.31*** .05***

.31*** .05***

.31*** .05***

63.10 (6.73)

77.67 (2.55)

77.43 (1.44)

6.99*** (3.95) 8.85*** (1.82) 12.62*** (1.99) 15.19*** (1.31) ⫺3.31** (1.20)

0.63 (.98) 3.62 (2.22) 3.82 (2.62) 1.94 (2.03) ⫺.08 (.06) 6.87 (8.07) 0.05 (.36)

7.05*** (1.69) 8.73*** (1.76) 13.03*** (1.88) 15.10*** (1.30) ⫺3.26** (1.20)

7.50*** (1.55) 8.77*** (1.59) 13.84*** (1.62) 14.77*** (1.22) ⫺3.12** (1.12)

2.32*** (.40)

2.40*** (.34)

⫺.09 (.06) 6.40 (8.02) 0.08 (.36)

Notes * p ⬍ .05, ** p ⬍ .01; *** p ⬍ .001.

which child care outcomes can be predicted from factors such as multiple measures of observed classroom quality in regressions that included family selection factors as covariates. To illustrate this point, we regressed multiple measures of child care quality onto the children’s vocabulary score (PPVT), adjusting for state, maternal education, ethnicity, and gender. Table 4 shows the results from three models that included family selection variables and child care quality measures. The correlations between child care measures ranged from modest to large, ranging from r ⫽ .2 to r ⫽ .8, with the observed process quality measures correlating more highly, ranging from r ⫽ .7 to r ⫽ .83. The first model included all measures of child care quality. The second model replaced the four process measures of quality with a composite measure created using factor analysis of those four process measures, but still included the three structural measures as separate predictors. This model corresponds to Mosteller and Tukey’s (1977) suggestion regarding replacing correlated predictors with a composite score that retains much of the information in the original measures. The final model included the composite measure as the only child care measure. It is interesting to note that all three models fit about equally well and child care quality seems to account for comparable amounts of variance across the three

Family Selection and Child Care Research

403

models. The major point to be drawn from the table is that including all possible predictors is not always helpful, and indeed when one does, the regression coefficients can be misleading. Individually, each regression coefficient was clearly insignificant in Model 1; however, the composite process quality score accounted for a substantial amount of variance in Models 2 and 3. Representativeness of Sample A statistical issue that both economists and developmentalists should consider involves drawing conclusions from samples that were not randomly selected. Conclusions about associations between predictor and outcome measures need to be tempered by the representativeness of the sample and the degree to which the distribution of family selection, child care, and outcomes are represented similarly in the sample and the population. Statistical text books warn about the consequences: for example, Hays (1973) states (p. 527) “We can have no confidence that the sample regression equation is even remotely like the best way to predict for some population unless the proportional representation of X (predictor variable) values in population is like that in the sample, or unless X values themselves are selected at random.” Truncated distributions are a problem for child care research. Child care studies almost certainly have truncated distributions in terms of child care quality because it appears that lower quality child cares are much less likely to consent to participate in child care research (e.g., NICDH Early Child Care Research Network, 1996). This truncation in the distribution of child care quality likely results in the underestimation of child care effects. Furthermore, family selection variables are probably truncated in the regression analyses, with more at-risk families being less likely to have complete measures on both observational and questionnaire measures (e.g., Peisner–Feinburg & Burchinal, 1997). Thus, it is disproportionately likely that children attending low quality child care settings will not be represented in analyses either because their child care provides refused to participate in the study or because their family selection variables or child outcomes contain missing data. Both developmentalists and economists have not adequately addressed this issue in examining child care effects in general and family selection issues in specific. This issue was examined empirically in two ways using the larger dataset, CQO. First, we computed correlations and partial correlations between three measures of child care quality and a child outcome, vocabulary (PPVT). The zero-order correlation is labeled Model 1 in Table 5. The first partial correlation controlled for state, mother’s education, ethnicity, and gender (labeled Model 2 in Table 5). The second partial correlation controlled for the same variables, but excluded any child who was missing another family selection factor, the measure of parental attitudes about child rearing (labeled Model 3 in Table 5). The third partial correlation controlled for state, mother’s education, ethnicity, gender, and parental attitudes (labeled Model 4 in Table 5). Second, we deleted data using a censuring mechanism that we felt may approximate how missing data occurs in child care research (i.e., one that fails to

404

Burchinal and Nelson Table 5.

Cost, Quality, Outcomes Study: Relating Child Care Quality to Children’s Language—Before and After Adjusting for Family Selection Factors

Model 1—Zero-order correlation between child care quality and the child’s vocabulary All data - r Reduced sample - r Model 2—child care quality, given state, maternal education, ethnicity, and gender All data - r Reduced sample - r Model 3—child care quality, given state, maternal education, ethnicity, and gender and the family completed the parental attitudes measure All data - r Reduced sample - r Model 4—child care quality, given state, maternal education, ethnicity, gender, and parental attitudes All data - r Reduced sample - r

Composite Quality

Global Quality (ECERS)

CG Sensitivity (CIS)

.29*** .22***

.24*** .20***

.29*** .20***

.20*** .15***

.16*** .13***

.19** .12**

.19*** .15***

.15*** .12**

.19*** .12**

.17*** .11***

.13*** .09*

.16*** .09**

Notes * p ⬍ .05, ** p ⬍ .01; *** p ⬍ .001.

sample lower quality care more often than high quality care). We randomly deleted 30% of the classrooms in which quality was considered to be at least 1 SD below the mean, 20% of the classrooms in which quality was considered to be between the mean and 1 SD below the mean, and 10% of the classrooms in which quality was above average. The same correlations and partial correlations were computed and results are reported in the second row under each model in Table 5. Results shown in Table 5 indicate the representativeness of the sample affect the magnitude of the correlations and partial correlations estimated from the data. As expected, the correlations are smaller when we consider family selection variables than when we ignore them (compare the first rows under Models 1– 4). The partial correlations are slightly smaller when we exclude children whose families did not complete one of the family selection variables, even before that variable was considered as a covariate. Thus, some of the reduction in the correlation between child care quality and child outcome associated with adjusting for this family selection factor is actually related to who did not complete this measure of parental attitudes. The major point to be made from Table 5, however, is the impact that differential deleting of data has on the correlations. There were marked decreases in the correlations in the artificially reduced sample from those

Family Selection and Child Care Research

405

in the origin sample (compare the correlations in the two rows under each model), even after adjusting for family selection variables. This suggests that differential representation of higher quality child care in child care samples probably underestimates child care effects, even after adjusting for family selection factors. Endogenous or Exogenous Effects The final issue involves drawing conclusions when some predictors may be mediators or have bi-directional relationships. Many psychologists and sociologists are concerned about whether control variables in regression or path analyses may serve as mediators. Within the child care literature, there is growing concern that “family selection factors” may mediate part of child care effects on child outcomes. For example, there is some evidence that a factor like maternal sensitivity to the child is influenced by the child care experience (NICHD ECCRN, 1999a). Child care experiences could influence parenting in at least two ways. The child in higher quality care may elicit more sensitive parenting because of enhanced cognitive or social skills acquired through child care experiences or the parent learn more sensitive caregiving practices from child care caregivers. Similarly, the child in lower quality care may be viewed as more difficult because of behaviors learned in the child care environment. If the direction of effects between some family selection variables and child care quality is bidirectional, then use of those selection variables as covariates will result in underestimating the association between child care quality and child outcomes because of the overlapping variance. We empirically examined the issue of whether parenting beliefs and practices could be influenced by child care experiences, and thus, serve as a mediator of some of the observed association between child care quality and child outcomes using the data from the Carolina Otitis Media Project. This study included annual assessments of child outcomes, multiple aspects of parenting, and child care quality. We selected the most general child outcome, the Mental Developmental Index (MDI) from the Bayley Scales of Infant Development. Three parenting measures were selected, a semistructured observation of the home environment (HOME), a composite rating of maternal sensitivity, responsiveness, and stimulation from a videotaped mother-child interaction session, and the parent’s rating of sense of competence as a parent from the Parenting Stress Index. These three measures were combined into a single aggregate score using principle components analysis. The quality of caregiving in child care was measured with the most commonly used measure of quality, the Infant-Toddler Environmental Rating Scale (ITERS) and the Early Childhood Environment Rating Scale (ECERS). Cumulative measures of the parenting and childcare measures were computed as the mean of that child’s assessments over the first 3 years. Two sets of analyses were conducted. First, a path analysis (shown in Figure 1) related the child’s MDI at 36 months to the cumulative measures of parenting and child care quality. As shown, there are direct effects of both the parenting and child care measures on the MDI, and considerable overlap between the parenting and child care measures. We were unable to test the direction of the effect within

406

Burchinal and Nelson

Figure 1. Path analysis of COMP data showing overlap in variance between measures of child care and home environments.

the path analysis so we computed across-time correlations (shown in Table 6). Comparing the cross-lagged correlations suggests that children with better parenting practices and beliefs in Year 1 tend to be in child care of higher quality in Year 2 (r ⫽ .41). It also appears that children with better child care quality in Year 2 are likely to experience better parenting practices and beliefs at home in Year 3 (r ⫽ .37). Thus, these data suggest that child care quality may have both a direct effect on cognitive development within this sample and an indirect effect through parenting practices and beliefs.

DISCUSSION Child care researchers must attend to family selection factors. It has been clearly demonstrated that children from families with higher incomes and more education tend to experience somewhat higher quality child care than children with fewer advantages. These family selection effects on child care quality appear to be relatively modest, but must be considered when asking whether children’s development could be influenced by their child care experiences. How to appropriately adjust for family selection effects without compromising statistical power in-

Family Selection and Child Care Research Table 6.

407

Carolina Otitis Media Project: Across-time Correlations among Parenting and Child Care Measures

Descriptive Statistics Child Care ITERS, Year 1 ITERS, Year 2 ECERS, Year 3 Parenting Parenting, Year 1 Parenting, Year 2 Parenting, Year 3

Mean

SD

Min.

Max.

3.02 3.33 4.00

0.88 0.90 0.89

2.03 2.03 2.28

4.86 5.13 6.18

0 0 0

1.29 1.28 1.29

⫺3.40 ⫺3.41 ⫺3.68

2.67 2.44 2.42

Parenting Year 2 Year 3

Correlations Child Care Year 1 Child Care ITERS, Year 1 ITERS, Year 2 ECERS, Year 3 Parenting Parenting Composite, Year 1 Parenting Composite, Year 2 Parenting Composite, Year 3

Year 2

Year 3

Year 1

.86***

.73*** .78**

.38** .41** .30*

.31** .32* .19

.30** .37** .13

.78***

.69*** .81***

Cross-lagged correlations Year 1

Year 2

Year 3

Parenting

.31**

.37**

Child Care

.41**

.19

Notes * p ⬍ .05, ** p ⬍ .01; *** p ⬍ .001.

volves balancing issues of bias because of unmeasured family selection factors against loss of power because of discarded information. Furthermore, analysis models should not include moderately to highly correlated measures of child care experiences if individual regression coefficients are to be interpreted. Finally, the possibility of both bidirectional effects between child care and family styles of caregiving and truncation of range in quality should be considered when results are interpreted. Multiple market forces may explain why family selection effects are more modest than expected based on economic and developmental theories. Helburn (1999) listed three market forces that may limit family selection effects. First, many parents are first-time purchasers of child care with little experience and very

408

Burchinal and Nelson

immediate needs. Despite the work of resource and referral agencies, the market is not well organized to inform parents of their options and parents may assume they have very few choices. Second, the purchasers of care (the parents) are not the consumers of the care, and thus, are not in the best position to judge the quality of the care they are purchasing. Young children probably are not good informants regarding what happens to them during the day in child care. Large discrepancies between parent and observer reports on the quality of child care likely reflect the inability of parents to adequately judge the care they are purchasing for their child. The CQO found that parents and professionals valued similar characteristics of child care, but disagreed about whether the parents were purchasing high quality care. Although almost all parents thought that their children were attending high quality child-care, the trained observers rarely agreed (Cryer & Burchinal, 1997). Third, to date, nonparental purchasers of care may reduce the availability of quality care in their communities by downplaying quality in their decisions. Subsidies for child care paid for by the local, state, or federal government are usually linked to some estimate of the market rate that is too low in most communities to pay fees at high-quality child care settings. In summary, there are multiple reasons why families may incorrectly think they are selecting high quality child care settings. Because of the modest association observed between family characteristics and child care quality, the manner in which selection factors are addressed in analyses can affect the results of those analyses. Use of econometric methods should result in more accurate (i.e., less biased) measures of association between child care quality and child outcomes, but they provide much less power to test whether those associations differ significantly from zero. Such methods will reliably detect the anticipated modest to moderate association between child care quality and child outcomes only in studies with very large samples. Use of covariates in regression models to adjust for selection factors is likely to provide better power to detect modest or moderate association, but the actual estimates of the qualityoutcome association may be less accurate. Furthermore, results from the analyses presented here suggest that even those regression analyses may underestimate the quality-outcome association unless close attention is paid to the representativeness of the sample, bi-directional associations between family characteristics and child care quality, and overlap between child care measures. Consideration of all of these statistical issues is important because child care quality is unlikely to have large effects on child outcomes according to developmental theory (Bronfrenbrenner & Morris, 1998), and because indices of associations are likely to be underestimated or attenuated. According to theory, children’s development is influenced most strongly by the most proximal interpersonal relationships. Parents and other family members are more likely to be heavily invested in their children than are caregivers, and thereby are more likely to have stronger, more persistent effects on children’s development. Therefore, one would expect modest to moderate child care effects after considering family influences on development, except perhaps when high-quality compensatory child care is provided to at-risk children. Furthermore, observed associations between child care quality and child outcomes are likely to be underestimated or

Family Selection and Child Care Research

409

attenuated because of limited reliability in measurement of quality and outcomes and the need to consider family selection factors as covariates. Attenuation is likely because observations of caregivers, even collected over 2 days, cannot completely describe the child’s child care experience throughout the year. In addition, assessments of young children are not completely accurate. Even the most reliable assessments such as measures of cognitive development are more reliable when collected on school-age children than preschoolers, and on preschoolers than infants and toddlers (Neisser et al., 1996). As discussed above, observed correlations become smaller when reliability of assessment is lower. Finally, inclusion of family selection variables as covariates or use of econometric methods that adjust for selection bias is also likely to underestimate the association between child care experiences and child outcomes. This loss of power in hypothesis testing will occur to the extent that meaningful individual differences in child outcomes that are related to child care experiences in the population are removed during analysis because they also relate to the family selection variables. Therefore, we believe that even “modest” effects sizes provide reasonable evidence of meaningful associations between child care quality and child outcomes based on developmental theory and problems with attenuation. It is unreasonable to expect that child care experiences will account for 5% or more of the variance in outcomes for very young children when we know that factors such as sensitivity in parenting often account for less than 10% of variance in analyses that also adjust for family demographics (for example see NICHD ECCRN, 1999b). In addition, it is unlikely that child care experiences will account for 5% of the variance in preschool outcomes as they did in the CQO analysis if extensive family selection measures had been collected and included in those analyses. Cohen (1988) defines as modest effect sizes either regression coefficients that correspond to partial correlations of r ⱖ .10 or to standardized differences between group means of d ⱖ .30. Use of effect sizes rather than statistical significance as evidence of associations between child care quality and child outcomes will more robust indicators, especially when power to detect associations is lowered by practices such as using analytic methods that dramatically decrease reliability, include family measures with bidirectional effects, or include moderately to highly correlated measure of child care experiences. In summary, child care researchers need to consider family selection factors whenever they relate child care experiences to child outcomes. In doing so, they should pay careful attention to factors that may influence conclusions such as: failure to represent the full range of child care in the sample; differential attrition of children from less advantaged families in poorer quality care; bi-directional effects of child care experiences on families as well as family characteristics on the selection of child care; and the use of correlated indicators of child care quality. Finally, it seems likely that developmental researchers may need to employ some of the econometric analytic methods to be credible to scientists in other disciplines and policy makers. Cooperation between developmentalists and economists in this venture could result in sensitive crafting of analysis models that are acceptable to both disciplines.

410

Burchinal and Nelson

REFERENCES Abidin, R. R. (1990). Parenting Stress Index (3rd ed.). Charlottesville, VA: Pediatric Psychology Press. Arnett, J. (1989). Caregivers in day-care centers: Does training matter? Journal of Applied Developmental Psychology, 10, 541–552. Bayley, N. (1969). Bayley Scales of Infant Development. New York: Psychological Corp. Bayley, N. (1993). Bayley II Scales of Infant Development. New York: Psychological Corp. Blau, D. M. (1999). The effect of child care characteristics on child development. Journal of Human Resources, 34, 786 – 822. Bronfrenbrenner, U. & Morris, P. A. (1998). Ecology of developmental processes. In W. Damon & R. M. Lerner (Eds.), Handbook of child psychology–5th edition (pp. 993–1028). New York: John Wiley & Sons, Inc. Burchinal, M.R., Roberts, J.E., Nabors, L., & Bryant, D. M. (1996). Quality of center infant care and infant cognitive and language development. Child Development, 67, 606 – 620. Campbell, F., & Ramey, C. T. (1994). Effects of early intervention on intellectual and academic achievement: A follow-up study of children from low-income families. Child Development, 65, 684 – 698. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: L. Erlbaum Associates. Cronbach, L.J., & Furby, L. (1970). How should we measure “change”— or should we? Psychological Bulletin, 24, 68 – 80. Cryer, D, & Burchinal, M. (1997). Parents as child care consumers. Early Childhood Research Quarterly, 12, 35–58. Davidson, R., & MacKinnon, J.G. (1993). Estimation and inference in econometrics. New York: Oxford. Doran, H. E. (1989). Applied regression analysis in econometrics. New York: Marcel–Dekker, Inc. Dunn, L. M., & Dunn, L. M. (1981). Peabody picture vocabulary test–revised. American Guidance Service, Circle Pines, MN. Elardo, R., & Bradley, R. H. (1981). The Home Observation for Measurement of the Environment (HOME) scale: A review of research. Developmental Review, 1, 113–145. Foster, E. M., & McLanahan, S. (1996). An illustration of the use of instrumental variables: Do neighborhood conditions affect a young person’s chance of finishing high school? Psychological Bulletin, 1, 249 –260. Greene, W. H. (1997). Econometric analysis. Englewood Cliffs, NJ: Prentice–Hall, Inc. Harms, T., & Clifford, R. (1980). Early childhood environment rating scale. New York: Teachers College. Hays, W. L. (1973). Statistics for the social sciences. New York: Holt, Rinehart and Winston, Inc. Helburn, S. W. (1995). Cost, Quality, and Child Outcomes in Child Care Centers, Technical Report. Denver, Department of Economics, Center for Research in Economic and Social Policy, University of Colorado at Denver. Helburn, S. W. (1999). The annals of the American Academy of Political and Social Science. Thousand Oaks, CA: Sage Publications, Inc. Howes, C., & Stewart, P. (1987). Child’s play with adults, toys, and peers: An examination of family and child care influences. Developmental Psychology, 23, 423– 430. IHDP. (1990). Enhancing the outcomes of low-birth-weight, premature infants: A multisite, randomized trial. Journal American Medical Association, 263, 3035–3042. Kirk, R. E. (1982). Experimental design: Procedures for the behavioral sciences. Belmont, CA: Wadsworth, Inc. Lamb, M. E. (1998). Nonparental child care: Context, quality, correlates. In W. Damon, I. E. Sigel, & K. A. Renninger (Eds.), Handbook of child psychology: Vol. 4. Child psychology in practice (5th ed., pp. 73–134). New York: Wiley. Lazar, I., & Darlington, R. (1982). Lasting effects of early education: A report from the Consortium for Longitudinal Studies. Monographs for the Society for Research in Child Development, 47(2–3:Serial No.195).

Family Selection and Child Care Research

411

Marfo, K. (1992). Parent-Child Interaction, MULTI-PASS, Version 3.0. Unpublished instrument, Department of Educational Psychology and Leadership Studies, Kent State University, Kent, OH. Mosteller, F., & Tukey, J. W. (1977). Data analysis and regression. Reading, MA: Addison–Wesley Publishing Co. Neisser, U., Boodoo, G., Bouchard, T. J., Boykin, A. W., Brody, N., Ceci, S. J., Halpern, D. F., Loehlink, J. C., Perloff, R., Steinberg, R., & Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77–101. NICHD Early Child Care Research Network. (1996). Characteristics of infant child care: Factors contributing to positive caregiving. Early Childhood Research Quarterly, 1, 269 –306. NICHD Early Child Care Research Network. (1998). Early child care and self-control, compliance, and problem behavior at 24 and 36 months. Child Development, 69, 1145–1170. NICHD Early Child Care Research Network (1999a). Child care and mother-child interactions in the first three years of life. Developmental Psychology, 35(6), 1399 –1413. NICHD Early Child Care Research Network. (1999b). Effect sizes from the NICHD Study of Early Child Care. In K. McCartney & A. Clarke–Stewart (chairs), Does child care matter? Symposium presented at the Biennial Meeting of the Society for Research in Child Development. Albuquerque, NM. NICHD Early Child Care Research Network (2000). The relation of child care to cognitive and language development. Child Development, 71, 339 –357. Peisner–Feinberg, E., & Burchinal, M. (1997). Concurrent relations between child care quality and child outcomes: The study of cost, quality, and outcomes in child care center. Merrill-Palmer Quarterly, 43, 451– 477. Pianta, R. C. (1992). The Student-Teacher Relationship Scale. Charlottesville, VA: University of Virginia. Pianta, R. C., & Steinberg, M. (1992). Teacher-child relationships and the process of adjusting to school. In R. C. Pianta (Ed.), Beyond the parent: The role of other adults in children’s lives, No. 57 (pp. 61– 80). San Francisco, CA: Jossey-Bass, Inc. Roberts, J. E., Burchinal, M. R., Medley, L., Zeisel, ,S. A., Mundy, M., Roush, J., Hooper, S., Bryant, D., & Henderson., F. W. (1995). Otitis media, hearing sensitivity, and maternal responsiveness in relation to language during infancy. Journal of Pediatrics, 126, 481– 489. Schaefer, E. S., Edgerton, M., & Aaronson, M. (1978). Classroom behavior inventory. Unpublished rating scale. Schweinhart, L. J., Weikart, D. P., & Larner, M. B. (1986). Consequences of three preschool curriculum models through age 15. Early Childhood Research Quarterly, 1, 15– 45. Singer, J. D., Fuller, B., Keiley, M. K., & Wolf, A. (1998). Early child-care selection: Variation by geographic location, maternal characteristic, and family structure. Developmental Psychology, 34, 1129 –1144. Stipek, D. (1993). Attitudes/perceptions of competence. Unpublished rating scale. Woodcock, R. W., & Johnson, M. B. (1990). Woodcock–Johnson psycho-educational batteryrevised. Allen, TX: DLM Teaching Resources.