A study in failure

A Study in Failure Bess Udell

F

AILURE CAN BE both chastening and instructive, hut not while it is rendered undiscussable by its taboo status which itself is unwittingly supported by dilemmas of careers and jobs, by vagaries of funding, and not least, by ideological investments. The impetus for this study derives from a recently completed demonstration project which had been designed to evaluate the impact on child development of group counseling of young, first-pregnancy mothers. The project was directed toward improving the social-emotional-cognitive interaction between mother and child. Its larger aim was to demonstrate the effectiveness of a preventive mental health component of a comprehensive health program for children. A complementary objective was the training of public health clinic nurses to become group counselors with a view toward enlarging their roles, skills, and services at public health pre-natal and well-baby clinics. Demographically matched experimental (N = 94) and control (N = 89) groups were drawn from first-pregnancy patients at public prenatal clinics. Both groups were visited and interviewed every two weeks at home by trained interviewers who administered a complex, time-staggered set of test-retest instruments. Extensive demographic and psychosocial information was collected, including data about child-rearing practices. Mothers in the experimental group were to attend (prenatally and postnatally) 20 weekly counseling meetings conducted by public well-baby clinic nurses who had been assigned by their clinic staff to participate in the project and who received intensive training by three mental health clinicians (two child psychiatrists and a clinical psychologist). It was expected that after 20 group-counseling sessions the experimental and control groups of paired mothers/infants could be compared on a number of mental health and developmental measures based on a battery of informal, openended interviews as well as established test instruments. The clientele was prototypically black, teen-age, unmarried, and of lower socioeconomic status; the very “problem” group presumably in need of, and to whom, such programs are addressed. PROGRAMMATIC

RESULTS

Although the formidable logistical obstacles that attended implementation (described in detail in the final report) are themselves interesting and instructive, they are by-passed here inasmuch as they are not relevant to the issues under discussion. On the other hand, the programmatic results of the project in terms of

From the Greaier Kansas City Mental Health Foundation, Kansas City, Miwouri. Bess Udell, B.A.: Research Associate, Greater Kansas City Mental Health Foundation. City. MO. Supported by NIMHgrant ROI-MNI9451. Reprint requests should be addressed to Bess L/dell, B.A., Research Associate, Greater City Mental Health Foundation, 600 East 22nd Street, Kansas Ciiy. MO. 64108. 0 1977 by Grune & Stratton. Inc. ISSN 0010-440X.

Comprehensive Psychiatry, Vol 18, No. 6 (November/December),

1977

Kansas

Kansas

573

574

BESS UDELL

what it sought to demonstrate as compared and what it actually did demonstrate, or failed to demonstrate, can be readily summarized. Despite exhaustive and flexible strategies (e.g. adding door-to-door transportation, as well as an intensified home visit schedule), the project failed to elicit from the mothers sufficient interest in group counseling. Group attendance was dismally poor, and even control-group dropout rates, aIthough lower than those of the experimental group, were high. As a result, groups were disbanded or more usually evaporated on their own. The project thus failed to acquire an anticipated and required experimental group of mothers who attended at least 20 counseling sessions (actually attained by only one mother; median number of sessions attended was 3). This made unlikely any meaningful comparisons of experimental versus control groups of children. Without viable counseling groups, we could demonstrate neither the possibility of training public health nurses to become competent group counselors of mothers, nor the impact of such counseling on the development and “mental health” of young children, nor the feasibility of such programs being conducted in various public health clinics. Although experimental versus control comparisons were attempted without success, the collected data were nevertheless explored for any possible new information. They were subjected to complex statistical analyses, treating as a single group those 66 paired mothers/infants who were available for analysis at infant age 6 months. The “~nd~gs” were weak and ambiguous. They pertained mostly to feeding-and-rearing practices in complex interaction with psychocultural characteristics. Yet, even had these findings been stronger or more interesting they would still have been judged as inconsequential alongside what in our view is the major “finding” of this study, namely: the poor response to and poor use of a “mental health” program which, it had been assumed, was needed, was wanted, and would hence be used. DISCUSSION

As part of an intensive effort to understand more fully the project and the reasons for its failure, it was only natural to turn to the research literature with a view toward learning what kind of outcomes can be reasonably and re~istic~ly expected for programs such as this one. What emerged from a study of the literature was the interesting recurrence, gradually discernable, of several themes and patterns of research reporting. Compared to the literature on infant development (physical, psychological and educational) which is both voluminous and rigorous, the evaluation literature which reports the effectiveness of ameliatorive programs for infants is scantier and, understandably, more tentative, unclear, and contradictory. A complimentary and mutually interlocking body of literature is concerned with documenting the prevailing extent of poor infant care, the identification of high risk populations, and the need and desirability (and desire) for “preventive” services. The “causes” of infant morbidity, mortality, and inadequate care are also contradictory and inconsistent. These “causes” seem upon critical reflection to be less well understood than is sometimes asserted. Inferences are drawn with considerable confidence from “correlations,” for example, with poverty. In his

STUDY

IN FAILURE

575

uncommon respect for the inherent obscurities and interactive complexities of social factors, Odin Anderson’ constitutes a welcome exception. While noting the (insufficiently noted) remarkably low child mortality rate of turn-of-the-century, slum-dwelling, immigrant Jews as compared to other groups (even as compared to those of higher socioeconomic status), he alludes to what he calls “mothercraft,” inviting speculation as to its relation to those variations in cultural values and styles that embrace parental responsibility and child care. A common theme in the literature thus focuses on the identification of high risk mother/infant groups which are usually found to be associated with a familiar array of sociodemographic variables, e.g. adolescence, out-of-w~lock pregnancies, inadequate nutrition, inadequate prenatal care, and so on. Emphasized is the inadequacy of a spectrum of health and social services ranging from those related to family planning, pre- and postnatal care, school and vocational guidance, mother and infant’s health and medical care, to those offering counseling in maternal care and child-rearing. Concomitantly emphasized is the need for just such services, including many that are in the nature of “remedial,” “compensatory,” and “interventionist” efforts. The assumption, generally unquestioned, is that “needed” services are also “wanted” services which are then confidently expected to be readily used. At the same time, a variety of identified health deficiencies are assumed to flow from inadequate services. As another common theme in the literature, under-users of services are thus assumed to be “neglected.” This is illustrated, for example, by Fiedler et al,’ when they state “Teen-agers are the most neglected age group in the health care system,” or in their further assertion that “adolescents are acutely aware of their health needs and with guidance and interest, are readily motivated to improve their health status.” Interestingly, such assertions remain unquestioned, uninvestigated, and uncritically accepted. With respect to program effectiveness, the literature contains some occasionally-voiced notes of caution. 3-5 For the most part, however, reported findings are predominantly those of success and positive gains.“-I5 Whether such claims may or may not be valid, albeit exaggerated, they nonetheless remain difficult to weigh against a nontypical report like Fowler’sL6 which pointedly notes the difficulty of involving “disadvantaged” families in remedial programs, and goes on to report that the infants of 6 out of 11 such families that were enrotled in a special program either showed no gains, or regressed, or dropped out. Consequently, the preponderance of programs that are reported as successful and effective is hard to assess without knowing to what extent negative findings and poor outcomes are under-reported, whether because they are less likely in the first place to be written up as articles, or because of the preferential publication of positive findings, or both. In any case, misleading claims turn out to be of dubious service since in the long run they prove self-discrediting while in the short run they subvert effort and energy. Even then, this tendency to selectively focus on success, if true, would constitute only one type of what may be described as under-reporting. There is yet another, more serious, type of under-reporting. This takes the form of a conspicuous absence of utilization data. Pertinant details are neglected, ignored, or omitted. The reader is usually left to puzzle over what proportion of those invited to participate in a program actually did so (and, crucially important, for how long), how many declined, and how many dropped out. It follows that a highly

576

BESS WELL

“successful” program, on closer inspection, may turn out to mean that a particular program was successful for onfy those self-selected few who attend or utilized that program. And yet these “users” could well constitute no more than a negligible proportion of the population “in need.” Among the few investigators who allude even briefly and indirectly (and only in passing) to the facts of utilization is Bracken,14 who reports that out of 17 girls invited to attend 6 one-hour group meetings (transportation furnished), only 6 accepted. Fowler,rs as mentioned earlier, pointed to similar obstacles when he stated that “disadvantaged families are diflicult to involve.” In another study, Willi~si5 outlined how premature and underweight newborns benefited from the extra stimulation and attention provided by hospital staff, he then goes on to remark how most mothers successfully resisted staff efforts to involve them in the program. Badger et al. in their description of a successful project do include utilization data, but these are too unclear to provide a needed picture of usage. In a report by Gutelius and Kirsch, favorably describing another program, important specifics are again omitted but at least mention is made, however casual, of poor group attendance. Even though in these few examples under-utilization gets but a passing glance, such reports are in contrast to the prevailing mode of reportage in which under-utilization is barely acknowledged, let alone discussed. This pattern is pervasive throughout (published) evaluation research, even though the way a treatment and service program is used (not to say, misused or underused) would clearly seem to be critically linked to any accurate appraisal of its outcome. Nor, for that matter, are the realities of utilization without implication for a variety of areas related to health/mental health care. These would include prevention, public health, rehabilitation, self-help treatment programs, cost-effectiveness analysis, and mental health coverage under private as well as national health insurance plans. For example, as is probably known and intramurally acknowledged,‘7*“g*30the uneven utilization of services poses a problem for prevention and public health. Such unevenness can take the form, for instance, of failing to secure treatment for, say, venereal disease, or neglecting to get a child innoculated. The U.S. Public Health Center for Disease Control has reported that the percentage of children aged l-4 years who were innoculat~ against all infectious diseases declined from 73% in 1964 to 63% in 1974. Whether this reflects inadequate services rather than (or alongside with) the inadequate use of them is not as self-evident as the conventional outcry for more and better services would suggest. In other words, as with some other groups (e.g. alcoholics or drug addicts) who are likewise socially problematic and likewise targeted for rehabilitative/preventive/interventive efforts, the youthful “high-risk” mothers may likewise be indifferent users or poor users of health services. These groups serve to unmask the curious paradox whereby effective prevention/rehabilitation requires, even depends upon, a nonemergency mode of utilization and yet is frequently addressed to precisely those high-risk groups who are predominantly emergency users of services. For similar reasons, the acclaimed benefits of self-help treatmentI programs have been hard to assess over the years in the absence of utilization information; a closer, more sober appraisal is increasingly warrented in the face of recently estimated drop-out rates of 7%90%. lg In the context of these particular and persistent problems, it can be seen that

STUDY

IN FAILURE

577

mental health care, no less than health care generally, must tacitly cope with the problem of public/private “needs” versus “wants” as it is manifested and expressed, among numerous other ways, in identifiable and traceable patterns of “user behavior,” i.e., in diverse but consistent styles of utilization which characterize varying client groups. Hence the facile presuppositions of intervention and rehabilitation conducted for a variety of social problems become ever harder to sustain in the face of serious utilization problems, some of them only currently starting to be reported by a few investigators, including Anderson et al.,20*zlDerman et al.,“” Shefeet et aLZ3 Glasscote et al.,“” as well as the author and coworkers2*-“’ This interest in utilization may increase in the wake of the growing debate around mental health benefits to be included in national health insurance and as a result of which greater demands will probably be imposed on evaluation research. And to the degree that effectiveness cannot be entirely divorced from utilization, the latter becomes a relevant dimension in the assessment of outcome. In other words, evaluation research, whatever else it may or may not be, is also utilization research. It also follows that utilization is a needed component in cost-effectiveness analysis. That is, just as services are expensive or inexpensive, efficient or inefficient, effective or ineffective, they are also well-used or poorly-used (both in quantity and quality). Not only can util~ation differentials lead to outcome differentials, but poor utilization can make even good services look bad. At the same time, effectiveness may to an unknown degree be confounded by motivation which is itself expressed in part via utilization. Sharfstein et al.,“” in a recent discussion of the difficulties of analyzing the comparative costs of private versus public psychiatric care, conclude that social science research will aid in peer review, in the planning of new services, and in cost-effectiveness research. This is a tall order, even apart from formidable difficulties of assessing effectiveness in the absence of consensus on the criteria of treatment success and effectiveness. Whether such generous faith in the power of social science research is perhaps extravagant remains to be seen. At the least, it deserves that presumeably well-conducted investigative work be reported as fully and accurately as possible. REFERENCES 1. Anderson 0: Infant mortality and social and cultural factors: Historical trends and current patterns, Jaco EG (ed): in Patients, Physicians and Illness. New York, Free Press, Div Macmillan, 1958, pp IO-24 2. Fiedler DE, Lang DH, Carlson JM: Pathology in the healthy teenager. Am J Public Health 63:116- 121, 1973 3. Lichtenberg P, Norton DG: Cognitive and mental development in the first five years of life. Rockville, Maryland, NIMH, 1970 4. Caldwell BM: A decade of early intervention programs. Am J Orthopsychiatr 441491~496, 1974 5. Costello J: Review and summary of a national survey of the parent-child center pro-

grams. Office of Child H.E.W., August, 1970

Development

U.S.

6. Osofsky HJ, Osofsky JD. Adolescents as mothers. Am J Orthopsychiatr 40:825--833, 1970 7. Hunt JMcV: Has Compensatory Education Failed? Harv Educat Rev 39:278-300, 1964 8. Gutelius MF, Kirsch AD Factors promoting success in infant education. Am J Public Health 65:384--387, 1975 9. Pavenstedt E: An intervention program for infants from high risk homes. Am J Public Health 63:393 -400, 1973 IO. Flopan M, Schoenfeld H: Procedures for exploring women’s childbearing motivations, alleviating childbearing conflicts and enhancing ma-

578

temal role development. Am J Orthopsychiatr 42389-3981972 Il. Wright C, Zally JR, Dibble M: PrenataI and postnatal intervention. Paper presented annual meeting American Psychologic~ Association, Miami, September, 1970 12. Schaefer ESA, A Home Tutoring Program. Children 16:59-61,1969 13. Klaus RA, Gray SW The early training project for disadvantaged children. Monographs of the Society for Research in Child Development vol33,1968 14. Bracken M: Lessons learned from a baby care club for unmarried mothers. Children l&133-137,197l 15. Williams ML: Effects of early stimulation on low-birth weight infants, Child Development 44:94-101,1973 16. Fowler W: A Developmental learning approach to infant care in a group setting. MerrillPalmer Qu l&145-175, 1972 17. Graham S: Studies of behavior change to enhance public health. Am J Public Health 63:327-333,1973 18. Dumont MP: Self-help treatment programs. Am J Psychiatr 131:636640, 1974 19. Khantzian EJ: Perspectives on self-help: Psychiatry controversy in addiction treatment. Psychiatr Anns 6%16,1976 20. Anderson WH Lt Cdr, O’MaBey JE, Lazare A: Failure of out-patient treatment of drug abuse: I. Heroin. Am J Psychiatr 12812, 1972 21. Anderson WH Lt Cdr, O’Malley JE, Lazare A: Failure of out-patient treatment of drug

BESS

UDELL

abuse: II. Amphetamines, barbituates, hallucinogens. Am J Psychiatr 128:12, 1972 22. Derman RM, Mascia GV: Report to annual meeting of American Orthopedic Association, 1974, reported in Psychiatric News, May 15, 1974 23. Shefeet A, Quinones M, Lavehar MA, et al: An evaluation of detoxification as an initial step in the treatment of heroin addiction. Am J Psychiatr 133:337-339, 1976 24. Glasscote R, Susex J, JalTee JH, et al: Treatment of drug abuse: programs, problems projects. Washington, D.C., Joint Information Service of the American Psychiatric Association; National Association for Mental Health, 1972 25. Udell B, Hornstra RK: The uneveness of psychiatric care. Compr Psychiatr, 18:1-l 1, 1977 26. Udell B, Hornstra RK: Three utilization styles at an urban mental health center. Hosp Commun Psychiatr 1977 (in press) 27. Udell B, Hornstra RK: Good patients and bad: Therapeutic assets and liabilities. Arch Gen Psychiatry 32:1533-1539, 1975 28. Sharfstein SS, Taube CA, Goldberg ID: Problems in analyzing the comparative costs of private versus public care. Am J Psychiatr 13429-33, 1977 29. Badger E, Burns BS, Rboads B: Education of adolescent mothers in a hospital setting. Am J Public Health 66:469-473, 1976 30. Anderson OW: Syphilis and society: Problems of control in U.S. 1912-1964. Center for Health Administration, University of Chicago Research Series 22, 1965