Evaluation of Hull house teen pregnancy and parenting program

E~alua~~~~ and Program P[ann~~g,Vol. 14, pp. 49-61, 1991

0149-7189/91 $3.00 + .oo Copyright 0 1991 Pergamon Press plc

Printed in the USA. All rights reserved.

EVALUATION OF HULL HOUSE TEEN PREGNANCY AND PARENTING PROGRAM JEANNE C. MARSH and MOLLY A. WIRICK The University

of Chicago

ABSTRACT An institutional cohort design was used in the evaluation of the Hull House Adolescent Family Life Project, a program for pregnant and parenting teenagers located in two public housing projects in Chicago. In 4 years, 335 teenagers, extremely disadvun faged in terms of income and opportunities, participated in the program. Services were characterized by an emphasis on the acquisition of knowledge of parenting and child deve~opmenf. Services also emphasized heaffh care and assistance with prucfi~af fife probfems. Study findings reveuf the program achieved some of the gains found by other programs: Knowledge increases in the areas of sexuality, contru~eption, child development, and parenting. Afthough there was no detectable change in cfients’sexuaf activity, there was a signz$cunt increase in the numbers using birth control. The findings reveal change in the direction of more clients in-school and employed post program, but these changes were not statistically significant. Further, the fertility measure, number of repeat pregnancies in I year, was completely unaffected by the intervention. In considering which aspects of the program were most effective, counseling was the most important in influencing young women to avoid subsequent pregnancies. Assistance with health problems was particularly important for enabling young people to stay in school, employed, or in the labor force. Counseling and health services were strengths of the program, measured in terms of services most frequently delivered. Program participants expressed greatest dissatisfaction with the lark of job training, one of the weakest aspects of the program. Overall, the gains doeumented in this study are consistent with those found in evaluations of similar programs, and afthough modest, provide the basis for continued support and evaluation of programs for teenage parents.

INTRODUCTION 1987). Although there is relatively little discussion based on relevant scientific evidence of the most promising directions for service provision, there is growing evidence of the antecedents and consequences of adolescent parenthood and of the types of factors that may be useful in preventing and ameliorating the problem. Hofferth (1987) has reviewed what has been learned from a small number of evaluations of comprehensive programs designed for pregnant teenage women, In general, the evaiuations reveal that programs can achieve short-term positive gains in mothers’ and infants’ health and in knowledge of child development. Longer term effects are documented in some studies for

In the last 15 years, the growing number of teenagers who are becoming parents has caused increasing concern. The concern derives from the association of teenage childbearing with a range of negative life consequences: greater health risks for mother and child, reduced educational attainment and success in the labor market, and increased duration on welfare. Commensurate with this concern has been an increase in efforts to ameliorate problems related to early parenting. So far, there has been limited evaluation of these efforts. We are only beginning to understand “how various interventions work, for whom, under what circumstances, with what intended and unintended effects” (Hofferth,

This research was supported by a subcontract from a grant to Hull House Association from the Office of Adolescent Pregnancy Programs, 1982-87. The authors are grateful to the Hull House administration and staff for their support and cooperation with the evaluation and for their remarkable commitment to the well-being of program participants. Requests for reprints shouid be sent to Jeanne C. Marsh, The University of Chicago, Schooi of Social Service Administration, Chicago, IL 60637.

49

SO

JEANNE C. MARSH and MOLLY A. WIRICK

school completion and employment (either enrollment in employment and training or participation in the work force) (see Burt, Kimmich, Goldmuntz, & Sonenstein, 1984; Edwards, Steinman, Arnold, & Hakanson, 1977, 1980; Klerman & Jekel, 1973; Polit, Tannen, & Kahn, 1983). In almost all studies, the delay in subsequent pregnancy, a key outcome variable, has remained stubbornly immune to intervention efforts. Specific aspects of programs that are related to positive outcomes were identified in the Urban Institute evaluation of selected projects of the Office of Adolescent Pregnancy Programs (Burt, et al., 1984). They found child care to be an important service. It was related to reduced chances of a repeated pregnancy at 12 months after delivery for those pregnant at entry. Child care was also related to school completion and employment 12 months after the birth. This research also found the length of time in the program was an important mediating factor. The longer a participant remained in the program, the less likely she was to have a subsequent PROGRAM The Adolescent Family Life Program (AFLP) of Hull House Association Chicago, IL was funded in 1982 by the U.S. Department of Health and Human Services, Office of Adolescent Pregnancy Programs (OAPP). Originally a 3-year pregnancy prevention and care demonstration grant, in the second year, the project was funded for 5 years to provide care services only to pregnant or parenting teenagers. Context The AFLP was structured as a comprehensive service program for pregnant and parenting teenagers in two different Hull House-affiliated neighborhood centers in public housing complexes in the City of Chicago. For purposes of this report, these centers are referred to as the Southwest Side Center and the Near South Center. The Southwest Side Center serves a housing complex that is a group of low-rise structures within a well-contained commercial area in a middle and upper-middle class neighborhood on the southwest side of Chicago. The deveIopment has a population of about 5,000, virtually 100% black. It is isolated markedly by major roads and highways from the totally white community that composes the rest of the community. The staff at the center tell the anecdote that the president of the local Chamber of Commerce once told the director of the center “We”- meaning the white community - “don’t recognize that (the development) exists.” The nearest medical facility is Cook County Hospital, 10 miles away. Prenatal care is problematic because of this distance and difficulty with transportation. It is common for a woman in labor to take a bus to County Hospital to deliver her baby.

pregnancy at 12 months and the better her educational attainment. They also found assistance with transportation to be related to school attainment. Two major problems can be identified in the evaluation research conducted thus far: (1) Most studies lack relevant control or comparison groups, and (2) with the exception of the Urban Institute study, research has failed to link specific aspects of programs to specific outcomes. The latter phenomenon has been referred to as the “black box” approach to evaluation where a program is treated as a single unitary force and no effort is made to sort out specific “active ingredients” or aspects of the program that are particularly important. The present study, an evaluation of the Hull House Adolescent Family Life Program is an effort to address these issues. Within the resource constraints of the study, a research design was selected that afforded optimum control. And specific services delivered were described and documented and ultimately related to the outcomes of interest. DESCRIPTION The Near South Center serves a housing complex that is south of downtown Chicago in a deteriorating industrial district in a neighborhood it shares with Chinatown and a middle-income black community. This section of the city is similar to the other neighborhood having major roads on two sides and elevated railroad tracks on a third. The three projects served by the center are isolated from the middle-class neighborhoods, although there is an amicable relationship with residents of Chinatown, which borders the housing projects. It is a sentiment among residents, however, that Chinatown merchants won’t hire young blacks from the neighboring highrises, a sentiment borne out by employment statistics. The Near South Center serves several of the most concentrated areas of poverty in the U.S. (Ziemba, 1984). One of the projects, Dearborn Homes, ranks 1 lth in the nation among the most poor areas, with a per capita income of $1782. Virtually 100% of the families in the three projects are black, about three-quoters are headed by women, about a quarter of the work-age adults are employed, and about 60% are 30 years old and younger (IIlinois Department of Public Health, 1981; Ziemba, 1984). One advantage of this area for teen mothers is that it is close to several major medical centers. Goals In keeping with the legislative mandate, the goals of the program were to assist adolescents to delay initial and repeat pregnancies and to become more productive and independent contributors to society. In the short term, the program architects hoped to increase birth weights

51

Evaluation of Hull House and reduce medical complications for mother and child. They wished to improve knowledge of child development, family planning, and parenting and to encourage contraceptive use. In the long term, they hoped to reduce repeat pregnancies, improve educational attainment, and employment. To achieve these goals, the programs provided comprehensive health services ineluding preventive services and family planning services for adolescents, family counseling, and family life education (including information about adoption, health INTERVENTION Program Services

To improve

Health Family

Intermediate Outcomes

Planning

Counseling/ child care Education/Job training Life Skills

knowledge

of fa-

mily planning, child development and parenting To improve contraceptive behavior To improve health of mother and child

MODEL

AND EVALUATION

Ultimate Outcomes To reduce fertility To remain in school To find employment

This model is a rough model devised from available descriptions of program intent and a basic knowledge of service delivery, but the causal relationships specified in this model were inferred by the evaluators and used to guide decisions about design and analysis. Application of the model led to an evaluation designed to assess both the implementation of the program and its outcome. The following evaluation questions were specified: EVALUATION

care, and parenting} along with education and employment counseling and job placement. As the program evolved, plans were made for a young fathers component as well as a home visitor parent support component with appropriate objectives. See Wirick and Marsh (1986) for a description and evaluation of these specific aspects of the program. A basic intervention model is implicit in the objectives outlined. This model provides a rationale and a guide for the evaluation findings reported here.

Implementation Assessment 1. What are the characteristics of the population receiv-

ing services? 2. What services are participants receiving? 3. How are these services being delivered? Outcome Assessment 1. What knowledge about parenting, reproduction, and contraception have the participants gained over the life of the program? 2. What changes do participants report in their use of contraceptives? 3. How many participants return to school? 4. How many participants find jobs? 5. What participant characteristics are related to program outcome? 6. What program participation factors are related to program outcome?

RESEARCH

Several considerations influenced the design of the study. First, the research sought to provide ongoing, relevant feedback to the program as well as to advance understanding of services for adolescent parents. As a result, the evaluation reports for the initial years of the program focused on formative evaluation, that is, information relevant to the characteristics of clients and services provided (see Love & Marsh, 1983; Wirick & Marsh, 1986). Results of the evaluation were regularly and systematically reviewed with program staff in terms of their implications for program direction. Assessment of outcome, that is, examination of which specific aspects of the services are most effective for which clients, was conducted only in the Final program year. Second, resources for this project (and all AFLP projects, see Hayes, 1987, p. 250) were limited so that it was necessary to identify a research design and data collection method that could be implemented inexpensively. The institutional cohort design was selected be-

QUESTIONS

METHODS

cause it takes advantage of the fact that a new, but basically similar, group of clients was served by the program each year. Prior to receiving program services, each new group of clients provided a useful comparison for the group of clients who had received 1 year of services. This design, elaborated below, includes a relevant comparison without the cost of seeking groups outside the program. Further, the study relied on measures that had been developed for the Adolescent Family Life Program by the Urban Institute (Urban Institute, 1982) so that only a modest amount of instrument development was necessary as part of the evaluation. This, of course, had the additional advantage of making study data compatibfe with data from other studies. Finally, the service program staff carried heavy data collection responsibilities. They were responsible for insuring the information on each client was collected accurately and completely and was compiled for transfer to the evaluation staff at appointed times. All research responsibil-

52

JEANNE

C. MARSH

ities of service staff were completed in addition to ongoing conduct of the service programs. A vigorous effort was made by evaluation staff to provide adequate training and feedback in order to maintain consistent data collection methods over the course of the evaluation. Finally, the study was designed to build on the relevant research in the field. Specifically, previous research has been limited by inadequate control or comparison groups and by a failure to adequately assess the character of the programs and the relation of specific services or approaches to specific outcomes (Hofferth, 1987). Thus, within the constraints of resources, this study was designed to include relevant comparisons. And, both as a function of the ongoing communication between the program and evaluation staff (through joint monthly meetings sessions rotated between service sites and University, numerous phone calls and meetings between sessions) and the data collected, the evaluation attempted to understand and document carefully the character of the services being provided. Clients

The criteria for accepting clients into the program were (a) clients had to be 18 years of age or Iess and (b) clients had to have recently delivered a child. Most clients lived in the housing developments served by the programs although occasionally individuals outside these developments were served. Clients used program services on an as-needed basis. The criterion for including clients in the analysis was a complete set of pre-program and post-program data. Thus, this evaluation provides evidence of the nature and impact of this pregnancy and parenting program for those who complered the program in the sense of completing information about their experience. It does not provide evidence of the impact of the program for all young people in the targeted housing projects who were eligible, nor does it provide evidence of the impact of the program on all young people who ever crossed the program threshold. This evaluation provides information about the impact of a comprehensive service program for teen parents who took advantage of the services provided. In total, 335 teens met the criterion: 1I3 in year 1, 92 in year 2, 96 in year 3, and 34 in year 4; 186 from Near South and 149 from Southwest Side Center. Previous research has shown several client characteristics to be related to program outcome: race, age, previous pregnancies, work, and school status. Data on these characteristics of clients were collected and taken into account in order to assess program impact. Since this program was conducted over 4 years at two program sites in Chicago, differences over time and over sites were also tracked. Procedure

Research Design. Data were collected and analyzed using an institutional cohort quasi-experimental design.

and MOLLY A. WIRICK As mentioned previously, this design takes advantage of the fact that a new group of clients entered the program each fall. And although the groups differed in terms of entry year into the program, they had very similar socioeconomic characteristics and home environments. It would be very difficult to find comparison groups more similar on relevant characteristics than the groups entering the program in subsequent years. Degree of comparability is never as high with cohorts as with random assignment. Random assignment was not possible, however, and the cohort design provides the basis for reasonably strong inferences. Additionaliy, data analyses described below, completed to assess (and hopefully rule out) the influence of selection bias, strengthened the inferences. There is one major concern about the validity of inferences based on an institutional cohort design: the inferences depend on a pattern of contrasts that must be statistically reliable. In this study, concern was alleviated to some extent because the major outcome variables were attributes such as whether or not a client was in school or working. Thus, reliability depended primarily on consistency of reporting (and recording) of these facts rather than on the reliability of a measur~tlg instrument. It was recognized that in a demonstration program such as this, neither the first nor last year of the program is representative. In the first year, program staff are gearing up, identifying clients, and developing procedures; in the last year, they are gearing down, determining under what auspice they will continue, or transferring clients and staff to other programs. As a result, for this evaluation, data were analyzed to evaluate outcomes from the Last 4 program years. As expected, only a limited amount of information was available from the last year, which was only 9 months long. As a result, some of the analyses are based on only the middle 3 years of the program. The basic design was configured as follows: (1) (2) (3) (41

0,

x

02 03

x

04 0,

x

0, 0;

x

Ox,

where the numbers in parentheses represent program years, X represents the treatment, 0 stands for an observation, and the subscripts l-n indicate the sequential ordering of observations. The logic of the design is that each cohort is assumed to have “quasi-comparability” and to serve as a comparison group for each of the others. This particular design is different from the institutional cohort design originally discussed by Campbell and Stanley (1963) because pre- and post-program data on four (in some analyses three) full cohorts are used. In some ways, this

Evaluation of Hull House design can be viewed as a series of replications of the institutional cohort design. The replications enhance the validity by reducing the possibility of a history threat, that is that any apparent program effect is really the result of some unique event coincident in time with the program (such as an infusion of resources into the Chicago Housing Authority). Concern about history is reduced if, in addition to O2 surpassing 0, , O4 is greater than 03, O6 is greater than OS, and so on. This comparison suggests that the program has been effective at two or more times and it is unlikely that any historical force would operate more than once or sustain itself over an extended period of time. The major concerns with selection are eliminated since the major pre-post comparisons concern the same individuals. And testing is not a concern since the key outcome variables are facts about school status and work status that are not vulnerable to conventional testing effects of practice or demand. In addition, for the first 3 of the 4 years described above, follow-up data were collected approximately 1 year after post data were collected. Locating participants I year after they left the program proved exceedingly difficult so the numbers of clients for whom follow-up data are available are small: 2 in the first year, 43 in the second year, and 20 in the third year. These data can provide further evidence against the history bias and some information, albeit from a very small and select sample, about the long-term effects of the program. In order to enrich the quantitative analyses, in the final year of the program, intensive in-depth interviews with nine randomly selected clients were conducted. The purpose of these interviews was to attempt to understand clients’ perceptions of the experience of being teen parents-its pluses and minuses- and their view of the helping resources available to them and those they

53 TABLE 1 DATA COLLECTION SCHEDULE Pre

Follow-up

*

Demographic data Fertility/contraceptive behavior

Post

knowledge

and *

*

Parenting knowledge

*

l

Work experience/status

*

*

t

School experience/status

*

*

*

could have used but did not have. The quantitative analyses were designed to reveal what aspects, if any, of the program were helpful in terms of clients’ capacity to remain in school or become employed. The qualitative data provide a means to assess what was missing in the program from clients’ points of view. Data Collection. The data related to client characteristics and behaviors were collected on three staffadministered (preprogram) questionnaires at (1) the time the clients came into the project, (2) 6 months later (postprogram) and (3) 1 full year later (follow-up). These questionnaires elicited information on client characteristics such as race, marital status, pregnancy status, household composition, source of income, work, and school status. The post- and follow-up forms focused on work, school, and pregnancy status. At the pre- and postprogram data collection only, information was gathered on contraceptive behavior, knowledge of sexuality and contraception, and knowledge of child development. Throughout the program, staff recorded every episode of service that each client received. The specific data collected on pre-, post- and follow-up measures are summarized in Table 1.

IMPLEMENTATION ASSESSMENT Client Characteristics Table 2 provides a summary of basic client characteristics broken down by center and by entry year. This table provides a basis for judging differences in clients served in different centers or in different years. The data reveal that as a group, clients were female, 17 years of age approaching 18, black, and never married. Half were pregnant at the time they entered the program. Few (7Vo) were heads of their own households; most lived in households headed by women, typically their mothers. An average of 5 people lived in these households of whom 1.5 were children. The annual household income of clients was low with 98% of clients below the poverty line (defined as $13,000 for families of five in 1985). Most were supported by a parent or Aid to Families with Dependent Children (AFDC).

Half were in school, a few (4%) had completed school, and the rest (47%) had dropped out of school. Significant differences in center and entry year cohorts were minimal. There was a difference in age that was expected due to the fact that the southwest side center served young women 19 and over in the early years of the program until this was forbidden by the funding agency. Since there were few differences between centers, the two samples were aggregated for analysis. Service Characte~stics This evaluation of a program for pregnant adolescents can be distinguished from most others by its effort to document type and level of services received by clients. The service categories used were those developed by the Urban Institute for the Office of Adolescent Pregnancy

54

JEANNE

C. MARSH

and MOLLY

A. WIRICK

TABLE 2 CLiENT CHARACTERISTICS Year

Race Black Hispanic Native American I?=

Age ”

=

Near South

Southwest Side

98%

1000/o***

Signif.

1

2

3

99% 0 1 103

99% 1 0 92

99% 0 1 89

n.s.

18.5 113

17.5 92

17.4 96

0

4

1

169 17.2 186

18.3’ 149

98%

ns.

95% 3 1 1 110

98% 2 0 0 92

98% 1 1 0 93

39% 61 108

57% 43 92

600/o 40 94

47% 53 32 12% 88 34

*

Total

98%

0 147

ns.

Signif.

ns.

1

316 n..s.

17.7 335

ns.

96% 2 1 1 329

Marital Status Never Currently married Separated Living together n=

180

96% 3 1 0 149

Pregnant now Yes No n=

49% 57 180

53% 47 146

R.S.

6% 94 174

8% 92 147

n.s.

7% 93 99

5% 95 92

7% 93 96

91% 8

n-s.

170

86% 12 1 147

95% 5 0 95

88% Ii 1 92

84% 14 2 96

4.8 173

5.3* 148

5.2 99

5.3 92

49 96

4.3 34

1.5 146

1.4 107

2.3 39

.8 84

1.1 56

1.1” 24

38% 45% 12 3 3 0 0 97

26% 54% 8 3 2 5 1 87

27% 48% 13 4 7 3 2 92

54% 35% 3 0 0 3 3 31

ns.

33% 47% 10 3 3 3 1 287

ns.

66% 34 61

62% 38 91

69% 31 70

92% 8 25

n.s.

68% 32 247

IIS.

66% 34 58

69% 31 48

86% 14 36

82% 9 11

n.s.

73% 27 153

9% 1 5 16 0 1 63 6 88

3% 0 0 49 5 3 40 0 97

3% 0 0 26 0 8 60 3 93

3%” 0 0 25 0 3 69 0 32

Client as household head Yes No “r=

Sex of household head Female Male Joint n= Average number of people in household

f?= Average number of children in household Annual household income o-3,999 4,000-5,999 6,000-7,999 8,000.-9.999 10,000-l 4,999 15,000-l 9‘999 20,000 or more n=

Client receives Medicaid Yes No n= Baby receives Medicaid Yes No n= Primary source of income Self Husband Father of baby Parent Other relatives SS, UN, WK CP AFDC, SSI, GA Other i?=

0

36% 46% 11 4

151

29%* * 46% IO 2 7 4 2 136

72% 28 147

61% 39 100

75% 25 96

68% 32 57

4% 0 19

3 69 4 159

6%* * *

1 2 41 3 6 41 I 145

n.s.

100% 0 0 0 34

85% 12 3 34

51% 49 326 i-is.

7% 93 321

n.s.

8 7% 10 1 317

n.s.

5.1 321 1.5 253

5%

1 1 30 2 4 56 3 304 continued

Evaluation

55

of Hull House

TABLE 2 CONTINUED Year Near South

Southwest Side

AFDC to household Yes No n-

61% 41 186

39% 61 149

School status In school Not in school Compieted Dropped out n=

58% 42% 3 39 186

37%’ 63% ? 56 149

Work status Currently working Seeking work Not seeking work ff=

20 72 117

*p < .05; l*p
Programs.

8%

““P<

The Urban

l

7% * *

l

l

46 47 127

Signif.

1

2

3

ns.

73% 27 113

72% 28 92

64% 36 96

79% 21 34

70% 30 335

52% 48% 6 42 113

50%

50% 2 48 92

35% 659/o 6 59 96

76% * * 24% 0 24 34

49% 51% 4 47 335

7% 21 72 90

5% 44 51 81

3% * * 34 62 32

7% 34 59 244

17% 39 44 41

4

Signif.

Total

.001

Institute

evaluation

utilizing

these categories

used only one measure of service level: length of service (Burt et al., 1984). Two measures of service level were developed and used in this study: inlensi@ of service measured in average number of contacts for each service category and length or duration of service measured as the difference between the first and Iast contact (in months) for each category. As shown in Table 3, the largest number of client service contacts was provided in the area of life skills, an ambiguous label for assistance with practical life problems in the areas of family relationships, nutrition, finances, and housing. The lowest number of contacts was in the areas of family planning and job training/education. The low level of service in the family planning area could be accounted for by the heavy emphasis placed on abstinence as a birth control mechanism in this federal program. Assistance with health problems and counseling, primarily in relation to interpersonal problems, were somewhere in the middle. These data support what was obvious from any visit to these programs: The strength of the staff and their orientation was to provide practical assistance with clients’ concrete life problems. They were prepared (through the establishment of a clinic at the Southwest Side Center and good referral linkages to nearby hospitals at the Near South Center) to provide help with health care required by new mothers and their children as well as some counseling. The programs were not well equipped to provide job training nor, it appears, family planning. OUTCOME Given the goals of the program, the primary outcomes of interest were related to fertility, and to client’s school

Such an orientation toward practical helping is consistent with the settlement house tradition represented by Hull House, the sponsoring organization. The data on length of service were less informative. They indicated that clients received most services for an average of 3 to 4 months. There was relatively little variability among the service categories. Further, when total number of service contacts and total length of contacts were examined for each center and each entry year, the evidence suggests the Near South Center was providing more services in a longer average period of time. In addition, there was some minor variability of frequency and length of services provided over the years of the program. Table 4 provides a different perspective on the nature of the program by examining what proportion of clients took advantage of “key” program components. These data are compared with similar data from Project Redirection, another carefully evaluated program with a heavy employment component (Polit, Kahn, & Stevens, 1985). The data in Table 4 support the findings on frequency of client contacts. AFLP clients were least likely to receive birth control counseling, job training services, or housing assistance. However, AFLP clients exceeded Project Redirection clients in their receipt of housing assistance, education counseling, and help with personal and interpersonal problems. These data provide additional evidence that the primary emphasis in the Hull House AFLP program was counseling and help with practical problems. ASSESSMENT and work status. There were fundamentally six outcome variables of interest: uses birth control always, uses

JEANNE

56

C. MARSH and MOLLY

TABLE3

TABLE4

NUMBER AND LENGTH OF CONTACTS

PERCENTAGE OF CLIENTS USING SELECTED SERVICES

Service Type

Average Number of Contacts

Average Length of Contacts

Pregnancy test Maternity counseling VD test VD treatment VD education Prenatal health care Childbirth education Maternal post-partum health Infant first pediatric visit Maternal 12-month medical check-up Other health care-Mother Other health care-Infant

-2.2 .02 .9 .004 .07 -01 .54 .22 .02 .03 ,004 .17 .21

-4.3 .5 5.9 0.0 3.0 0.0 2.6 2.6

Health

Family planning

Counseling/child

care

Counseling”extended family Counseling-male partner Counseling-client Adoption counseling Assistance to find child care Child care-licensed, regular Child care-private, family Transportation Job training Educattonaiivocational training or referral Entered educational/vocational program Entered job traintng program Other Life skills Family relations/parenting education Consumer/homemaking education Nutrition counseling/education Enrolled In WIC Enrolled in food stamps Other: Financial assistance/AFDC Other: Housing Other Total

* *p < .Ol;

“““p<

.10 .62

1.4 2.2

-1.3 .20 .Ol .71 .06 .Ol .oo .004 .34

4.0 2.0 6.0 3.5 4.0 2.0 0.0 0.0 3.2

s

3.9

30 .04 ,I 1 .34

2.7 3.0

Services Used Ever

1

2

3

4

Project Redirection Polit et al., Total 1985

Parenting classes

65% 48% 13% 50%

46%

Brrth control counseling

52

31

6

0

33

74

Educational counseling

66

47

35

0

51

41

Nutrition counseling

61

42

18

50

44

56

Personal counseling

79

79

48

50

71

44

Housing assistance

24

33

35

50

30

75

23 107

19 86

13 68

0 2

19 263

64 285

Job (employability training) n=

64%

4.8

2.4

with several other outcomes considered intermediate. In particular, given the goals of the program to provide participants with information on family planning and parenting, it was of interest whether they learned anything about these subjects. Therefore, changes in knowledge about sexuality and parenting also were examined. Additionally, any reported changes in contraceptive behavior were reviewed. Beyond this, it was of interest to understand the relation between specific program inputs-in terms of the length and duration of specific services-and the ultimate outcomes. ‘Thus, in this section, both intermediate and ultimate outcomes are examined, and then the relation of program characteristics to these outcomes is considered. intermediate

Outcomes

4.1

-4.8

Contracepfive Knowledge and Behavior. Considering

t .2

23 2.1 25 .25 .5 20 5.2 3.8

knowledge of sexuality and contraception as necessary (though not sufficient) for effective decisions and behavior related to fertility, the program attempted to increase knowledge in these areas and modify contraceptive behavior. Table 5 shows there was a significant increase in sexuality and contraceptive knowledge for the total sample of program participants. The proportion of participants using birth control aIso increased. Table 5 reveals significant pre-post increases in the number of clients who reported they used birth control sometimes or always over the course of the program. And the individuals who reported they never used birth control decreased from 40% of the sample to 23%. This is an impressive drop, but it must be noted that there was still, in absolute terms, a notable portion of the sample not using birth control at the close of the program. There were significant differences in the pre to post within cohort comparisons as well as the post to pre between cohort comparisons (the 0:-03, 04-O5 comparisons using CampbeIl and Stanley notation). Level of sexual activity, as revealed by the variable

.54 48 .l 1 .03 .OT -11 1.63 -5.9 7.3 5.4 6.3 7.6 5.3 4.5

Near South Center Southwest Side Center Year 1 Year 2 Year 3 Year 4 05;

15 12 1 .o 1.7 2.6

AFLP Year

-3.2

Contraceptive information and/or counseling Counseling for sexual decision-making

‘p <

A. WIRICK

3.2 4.7 2.4 3.9 4.2 2.6 2.1

,001

birth control at least sometimes, repeat pregnancy, enrolled in or completed school, employed, or in the iabor force (defined as either employed or looking for work). These might be considered ultimate outcomes

57

Evaluation of Huh House TABLE 5

CONTRACEPTIVE

KNOWLEDGE AND BEHAVIOR Year

1

Services Used Ever

Post-Pre .96

Sexual knowledge n=

46 -Pre

Percent used birth control who had intercourse Never Sometimes Always n= Percent sexually active in last month Never One or more times n=

3

2

Post

f&

Total

Post-Pre

posl

3

1.42 49

1 .D 27

n.s.

Pre

Post

9.9 122

11.2*** 122

&%PostE

post

i-f 37% 44% 20% 41

22%:: 54% 24% 41

45% 41% 14% 44

30% ** 48% 23% 44

36% 48% 16% 25

12% 60% 28% 25

40% 44% 16% 110

23%*” 53% 24% 110

36% 64% 41

43% 66% 41

42% 58% 24

33% 67% 24

39% 61% 108

32% 68% 108

+ 40% 60% 43

30% * 70% 43

x2 tests within cohorts (pre-post, year1 ; pre-post, year 2); ‘p < .05: “p < .Ol ; “**p < ,001. Cochran-Mantel-Haenszel test across cohorts (post year 1 vs. pre year 2 and post year 2 vs. pre year 3); +p <: .05; ++p

Percent sexually active in the last month, in Table 5 shows some decrease from pre- to post- but fundamentally indicates very little significant change over the course of the program and very little change from year to year. To summarize, although the program appeared to have very little impact on the level of sexual activity, there was some change over the course of the program in the use of contraceptives. However, despite an overall increase in the number of clients using contraceptives at least sometimes or always, a reasonably large proportion continued to report they never used birth control. Knowledge of Child ~eve~o~~ent and Parenting. The total sample of young parents significantly improved their knowledge of child development and appropriate parenting techniques during the program period. Birth Outcomes for New Babies. Because one of the program goals was to help young women give birth to healthier babies, information was collected on basic indicators of infant health at birth: birth weight, gestation period, month at which prenatal care began, number of prenatal visits, and number of hospitalizations during pregnancy. To provide a standard of comparison, babies born to mothers enrolled in the program were compared to their next eldest sibling. Although such a comparison fails to control for birth-order effects, it does control for general health of the mother. Because there was a small number of babies born during the program who had older siblings, these data were not analyzed by program year. As shown in Table 6, there were no significant differences on birth indicators for babies born during the study and their next eldest sibling. In absolute terms, the averages for both groups were reasonably good for a high-risk population, The

<

.Ol

average weight of babies approached 7 pounds, gestation periods were 37-38 weeks on average, prenatal care began in the third month on average resulting in 11 prenatal visits. Number of hospital visits during pregnancy was less than .5 of one visit. In summary, these young parents seemed to benefit from the educational aspects of the program that focused on child development and parenting. They significantly improved their knowledge in these areas over the course of the program. They also had reasonably normal pregnancies and deliveries, given their status as members of a high-risk group. It is not appropriate to attribute this to the program since a comparison of “program” babies with older “nonprogram” siblings revealed no differences. However, these young parents seemed to benefit from the educational aspects of the program that focused on child development and parenting. They significantly improved their knowledge in these areas over the course of the program.

TABLE 6

BIRTH OUTCOMES

FOR NEW BABIES

Study Baby Birth weight (grams) Gestation weeks Month started prenatal care Number of prenatal visits Number of hospitalizations pregnancy

Next Eldest (n)

Baby

(n)

6.9

(115)

6.7

(23)

37.6

(107)

36.8

(191

2.8

(109)

2.6

(22)

11.4

(103)

11.1

(19)

during .5

(77)

.3

(23)

58

JEANNE

C. MARSH

and MOLLY

TABLE 7 AND EMPLOYMENT

EDUCATIONAL

A. WIRICK

OUTCOMES Year

1

Percent In/completed

Pre

Post

Pre

Post

62% 80

65% 88

51% 59

68% **

76% 59

76% 59

62% 50

91 % 57

96% 57

69% 51

Total

Pre

Post

Pre

Post

27% 48

85% 48

59% 187

70% 187

80% =* 50

33% 45

88% 45

59% 154

81% 154

96% 51

65% 44

98% 44

76% 152

96% 152

++ 59 ++

school or employed

n= Percent in/completed n=

3

school

n= Percent in/completed

2

school or in labor force

x2 tests wlthin cohorts (pre-post. year 1; pre-post, year 2); lp < .05; “p < .Ol ; ***p < ,001. y * tests across cohorts (post year 1 vs. pre year 2 and post year 2 vs. pre year 3); +p < .05; ++p < .Ol

Ultimate

Outcomes

Fertility. The l-year follow-up revealed that none of the young women in the first program year, 23% in the second, and 10% in the third had repeat pregnancies and births. The overall repeat-pregnancy rate of 18% repeat pregnancy after 1 year is remarkably similar to the rate found in other comprehensive care programs (Burt et al., 1984; Klerman & Jekel, 1973). Since these data were collected at follow-up on a select group of participants in the AFLP study, it is quite likely that this is an underestimate for the AFLP program. Of the outcomes of interest to intervention programs, the rate of repeat pregnancy appears the most resistant to ameliorative efforts. Education and Employment

Outcomes. As with many programs for teen parents, the goals of the program focused on education and employment. Employment and education status were measured preprogram and 6 months later (postprogram) and 1 year later (at followup). Educational status, shown in Table 7 shows that there was an increase in the proportion of clients who were in school or had completed school by the postprogram measurement. These increases were significant (both for the within cohort analysis and the between cohort analysis) only for year 2. An examination of clients’ educational aspirations indicated that a third reported they desired more than a high school degree. Follow-up data are not reported since only a subset of clients completed follow-up data and it is likely they PROGRAM

IMPACT

represented a special group easier to find perhaps because they remained connected to the program as a result of remaining in school or employed. Indeed, the follow-up measures show that 100% of those who participated in the follow-up measure were in school or had completed school. Similarly, Table 7 shows an increase at the postprogram measure in employment status, either in school/completed school or employed or in school/completed school or looking for work. Again, the increases were statistically significant only in year 2. Overall, the gains in education and employment status were not statistically significant. Cross-Cohort Analysis The institutional cohort design strengthens the conclusions that can be drawn with simple pre-post comparison within cohorts by permitting additional comparisons between cohorts. Again, the design was configured as follows: (Year 1) 0, (Year 2) (Year

3)

(Year

4)

O2 03

x

0, of,

x

0, 07

x

Ox

For each of the outcome variables, within-cohort comparisons were made (Or-02, 03-04, OS-06) as well as between-cohort comparisons (02-03, 0,-O,). As revealed in the findings discussed above, significant differences found in the within cohort analyses were also found in the between cohort analyses.

ON SELECTED

The foregoing documents some gains made by program participants; we now examine whether there is a relation between program inputs and the outcomes achieved. In

X

OUTCOMES

order to assess program impact, it is necessary to control for relevant characteristics of clients. Client characteristics found important in other studies are age,

Evaluation of Hull House number of times pregnant, school status, work status, and work experience. Race has been an important variable in other studies, but since virtually all clients were black in this study, the effect of race cannot be examined. Program input is defined in terms of service episodes (both the number of episodes overall and in various categories), and the duration of service (overall and in specific categories). Duration is defined as the time between first service and last service received. The analyses examine whether there was an overall impact of the program on selected outcomes, as well as whether there was an impact of particular services in the areas of health, family planning, counseling, job training, and life skills (defined as financial and housing assistance and consumer, nutrition, and homemaking education). Overall Program Impact

A regression analysis revealed that the impact of the program as measured in terms of total number of program contacts and total length of contact with the program was minimal. There was virtually no detectable relation between these global measures of program input and any of the outcomes measures of interest. The analysis using both measures of program inputs documents the futility of using a “black box” approach to assess the program effects, in which the program overall is expected to influence specific outcomes. Programs such as the one evaluated here have a number of different components and the effects of each must be assessed if effects are to be detected. Impact of Specific Services

Table 8 summarizes regression weights from regression analyses that related individual client characteristics and specific program services (health, family planning,

RELATION OF CLIENT CHARACTERISTICS

59

counseling, job counseling, and employment training, and the development of life skills for handling finances, housing, and welfare) to the outcomes of interest controlling for individual characteristics. Individual characteristics were always included in the model and program characteristics were included using a stepwise regression procedure. In the regression analyses, program input was assessed in terms of number of service episodes in each of the five service areas for the nine outcomes of interest. The results in Table 8 for the first three outcome measures suggest that family planning services had very little impact on fertility outcomes. There was a relation between avoiding a subsequent pregnancy and (1) age, (2) being in school or (3) being employed or looking for work, and (4) counseling. Being in school or employed at the beginning of the program seems to have an inoculation effect against subsequent pregnancies. Additionally, the socioemotional and motivational factors influenced in counseling (rather than the knowledge acquisition influenced in family planning services) seems important in avoiding subsequent pregnancies. It must be remembered, however, that many fewer family planning than counseling services were provided. In Table 8 the results for the last three outcome variables of school and employment status suggests that the capacity to remain in school or employed at the postprogram measure is associated with number of previous pregnancies and having received help with health problems. The findings here are consistent with a number of other studies showing number of early pregnancies is an important determinant of life outcomes. While it may be possible to finish school and find a job with one child, the responsibilities related to the care of two or

TABLE 8 AND LENGTH OF SERVICE CONTACTS ON OUTCOME MEASURES Outcome Measures

Uses Birth Control Always

Uses Birth Control at Least Sometimes

Age

-.Ol

Number of pregnancies

-.Ol

-.02

.OOOl

In school at pre

-.OOl

-.Ol

.05**

Employed or seeking work

-.02

-.ll

-.28’

-.06

-.02

.19 .09 -.04 -.05 -.14=

.12 .27 1.47” -.93 -.07

Ever worked Frequency service contacts Health Family planning Counseling Job training/education Life skills ‘p < .05;

l*p < .Ol

.03 .25 -.03 .30 -.12 -.05

.04

Avoided Subsequent Pregnancy .27**

In/completed School - Post .03 -.21

Employed Post -.03

l

*

In Labor Force-Post .006 -.22**

,006

,002

-.12*

.07

.Ol

.ll

.07

.04

.34’ .09 -.13 -.15 -.06

.29’

.lO -.06 .09 -.07 .002

,006

.07 -.15 -.ll -.07

.IEANNE

60

C. MARSH

more children prevent such activities. Further, the relative ease of obtaining health services- no doubt related to prevention of problems-appeared an important aid for remaining in school or employed. A similar analysis was conducted using Xength of service, both overall and for specific services, as independent variables. The relationships found were weaker or not d~scernabl~ when the intensity of the service variable was replaced by the length of service variable. The resuIts are not presented. AIthough this measure of service provision has been used in other studies (Burt et al., WHAT

HAVE WE LEARNED

ABOUT

and MOLLY

1984), it is not clear what this service variable means substantively since it would be possible for a client to have mi~limal and Ioose contact with the program and still have a long record of service if the first and last episodes were separated in time. In summary, the data on program contact suggest the program has very targeted effects with the provision of couns~Ii~g services proving to be the most he~pfui for enabiing dients to avoid subsequent pregnancies. Health services were a key component for assisting clients in pursuing schooling and employment.

PROVIDING

This evaI~~tion of the Hull House Adolescent Family Life Program which served young parents in two public housing projects in Chicago adds to the accumulating empirical evidence indicating the specific services that are usefuf and effective for teenage mothers, The resuhs show that this program served young women who were extremely djsadvan~aged in terms of income and opportunities. The prograin had a strong edu~ationaI component devoted to the acquisition of knowledge of parenting and child development. Services emphasized health care and assistance with practical life problems. Study findings reveaf the program achieved some of the gains found by other programs: knowledge increases in the areas of sexuality, contraception, child development, and parenting. Although there was no detectable change in ctients sexuaf activity, there was a significant increase in the numbers using birth control. Referring back to the intervention model for the program, positive and significant gains were achieved for the ~~~~r~n~~~~~~ o~ico#?~s. Program impact was more difficult to detect in the ~~~~~~~~oulco?yIes. Although the findings reveal change in the direction of more clients in school and employed post program, these changes were not statistically significant. Nevertheless, at the postprogram measure, 96% of clients providing information had completed school, were in school, or were in the labor force. The fertility measure, number of repeat pregnancies in 1 year, was completely unaffected by the intervention. Over 3 years, the average number of repeat pregnancies was 18Di0,a rate similar to that found in other pro~rams~ In co~sjd~rir~g which aspects of the program were most effective, counsehng was the most important in influencing young women to avoid subsequent pregnancies_ Assistance with health problems was particularly important for helping young people stay in school, employed, or in the labor force. The most effective program components were also the most frequently delivered components - those strongest aspects of the program. ~though the individuals interviewed reported a general satisfaction with these aspects of the program, they also mentioned, in a number of different ways, the impor-

A. WIRICK

SERVICES

TO TEEN

PARENTS?

tance of job training in a program for young parents, a companent that was generally weak in the AFLP. Findings from this study were not completely consistent with findings from previous studies. For example, the Project redirection findings at 5 years showed positive outcomes in ~nlp~oyrn~nt, a strong component of that program. In the Urban Institute study, the only other study examining the impact of specific program components, child care was an important factor for enabling young mothers to remain in school. This factor did not emerge in this study although a number of young women mentioned the need for child care during their interviews. It is noteworthy that in this population, a large number of clients’ mothers cared for their children while they returned to school or work, It is not apparent whether this also was the case for clients in the Urban Institute study, transportation also emerged as an important variabIe in the Urban Institute study but not in this one. There was some evidence in the in-depth interviews, however, that chents did not use services outside their communities because they could not afford public transportation. Clearly, different programmatic strengths result in different program outcomes. One finding that emerged here, that also was found in other studies, is the importance of warm, positive relationships with staff. Zabin (personal communication, 1986) reported this to be a key factor in the success of school-based clinics in Baltimore. Although this result does not ~rner~~ from the quantitative analysis (except perhaps in the importance of counseIin~ for avoiding subsequent pre~naIlcy), the clients interviewed were nearly unanimous in mentioning the value to them of staff members they respected and Iiked. Thus, to attempt to distill the key factors t,hat seemed to operate in these programs in Chicago, there is support for the conclusions that chents found greatest benefit in programs (1) with staff they enjoyed and respected, and (2) that provided them with health care and assistance in deveIoping skills to handle practical problems with finances, housing, welfare, and family reI~tionships” These conclusions must be considered, of course, in terms of the research on which they are based. The

Evaluation of Hull House present study improved upon others in this area by including comparison groups and by examining the relations between specific aspects of the program and outcomes of interest. The major problem with the study- and most studies in the field-is the failure to measure long-term goals in the long term. In this study, the results on measures of education and employment status after 6 months (with the data indicating an average service utilization of 3 months) were quite unsatisfactory. It is unlikely such effects could be felt after 6 months even if the program influenced these factors. In the Project Redirection analyses, the program impacts on employment were detected only 5 years after the program baseline (Polit, Quint, & Riccio, 1988). In this study, locating program participants 1 and 2 years post program with limited resources proved insurmountable. The longitudinal data required for adequate measures of impact are very costly. A related problem is the adequacy of available measures of both program input and outcome. Although this study was one of the few that has attempted to examine the effects of specific service components, the

61

measure proved inadequate in terms of the meaningfulness of categories and level of detail. More work on measures of program input is clearly needed. Similarly, more sensitive, short-term measures that could serve as proxies for the long-term measures of interest would also be useful. Clearly, we are beginning to understand how to provide services for young parents. Intensive, comprehensive programs can prove effective for disadvantaged young mothers. This evaluation of the Hull House Adolescent Family Life Program serving young parents in two public housing projects in Chicago adds to the accumulating empirical evidence of specific services that are useful and effective for teenage mothers. Specifically, the results of this study, along with a handful of others, show that in the short-term, efforts to increase knowledge of contraception, child development, and parenting can be effective. Evidence on the long-term effects on education and employment are encouraging but not definitive. Overall, the gains are consistent, if modest, and provide the basis for continued support and evaluation of programs for teenage parents.

REFERENCES BURT, M., KIMMICH, M., GOLDMUNTZ, J., &SONENSTEIN, F. (1984). Helping pregnant adolescents: Outcomes and costs of service delivery. Final report to the Office of Adolescent Pregnancy Programs, Department of Health and Human Services. Washington, DC: Urban Institute.

J.C. (1983). Adolescentfamily lifeprogram University of Chicago School of Social Service Administration.

EDWARDS, L., STEINMAN, M.. ARNOLD, K., & HAKANSON, E. (1977). An experimental comprehensive high school clinic. Amer-

York: Manpower

ican Journal of Public Health, 67, 765-766.

POLIT, D.F., KAHN, J.R., & STEVENS, D. (1985). Final impacts from project redirection: A program for pregnant and parenting teens. New York: MDRC.

EDWARDS, L., STEINMAN, M., ARNOLD, K., d%HAKANSON, E. (1980). Adolescent pregnancy prevention services in high school clinics. Family Planning Perspecfives, 12, 6-14.

LOVE, L.J., & MARSH,

evaluation report: Year 1. Chicago:

POLIT, D.F., TANNEN, M.B., & KAHN, J.R. (1983). School, work and family planning: Interim impacts in projeci redirection. New

POLIT,

Demonstration

D.F., QUINT,

Research

J.C., & RICCIO,

Corporation.

J.A. (1988). Thechahenge

of serving teenage mothers. New York: MDRC. HAYES, C. (Ed.) (1987). Risking the future: Adolescent se.uuality, pmgnancy, and childbearing (Vol. I). Washington, DC: National Academy

Press.

The effects of programs and policies on adolescent pregnancy and childbearing. In S. Hofferth & C. Hayes (Eds.), Risking the future: Adolescent sexuality, pregnancy and childbearing (Vol. II). Washington, DC: National Academy Press. HOFFERTH,

ILLINOIS KLERMAN,

S.L. (1987).

DEPARTMENT

OF PUBLIC

HEALTH,

1981.

L.V., & JEKEL, J. (1973). School-age mothers: ProbHamden, CT: Shoe String Press (Linnet).

lems, programsandpoiicy.

URBAN

INSTITUTE.

(1982). Evaluation of adolescent programsDC: The Urban Institute.

first year report. Washington,

M., & MARSH, J.C. (1986). Adolescent family lifeprogram evaluation reports: Years Z-4. Chicago: University of Chicago WIRICK, School

of Social Service Administration.

WIRICK, M., & MARSH, J.C. (1987). Final report: Hull House adolescent fami[v life project, 1983-87. Chicago: University of Chicago School

of Social Service Administration.

ZIEMBA. (1984, April 29). 10 of the Poorest U.S. Areas in the CHA: Study. Chicago Tribune. 6.