- Email: [email protected]
Research and policy-making in education
STUDIES IN EDUCATIONAL E V A L U A T I O N
RESEARCH
Volume
AND POLICY-MAKING
2. No. 3
Winter 1976
IN E D U C A T I O N *
AHARON F. KLEINBERGER
The slogan of the intellectuals in our generation is "relevance". In an epoch in which the intellectuals have lost their self-confidence, and have retreated to defence positions before the assault of vitalism and irrationalism, they have no longer the courage to believe or at least to proclaim what their predecessors in earlier ages believed and proclaimed: that knowing the truth and understanding ourselves and the universe are values in themselves and worthy purposes of human existence, and that the intellectual activity which aspires to these aims is the summum bomun, the highest good, and that it comprises the supreme happiness and tile most sublime human perfection. From their defence positions they seek the justification for their intellectual activity by showing it to be "relevant" to our most vital needs and our most violent desires. The constant approximation to tile knowledge of truth and to a fuller understanding, instead of serving as an end in itself, is being justified as an effectful means for solving practical problems and for the satisfaction of urgent needs. One can hardly find more striking evidence of this attitude than tile proclamations issued by the participants in an international meeting of experts on educational research which was convened in 1967 under the sponsorship of the UNESCO Institute for Education in Hamburg, and which resulted in a report published in 1971 under the title The Role of Research in Educational Change. q-here we read that tile sponsors and the participants of the conference are committed to the view that the attainment of knowledge for its own sake is an inadequate objective for educational research, and that its justification must rest on the contribution it can make to the improvement of educational practice. This is true not only with respect to what is called "applied research". The value of so-called "'basic" or "'pure" research, as well, rests not on the extension of our knowledge and understanding, but on its demonstrated contribution to the improvement of educational practices, m that it will ultimately lead to the introduction into the educational system of new and more effective forms of organization and procedures. Indeed, there is wide agreement within the ranks of those engaged in educational research that their activity, if it is to be "'relevanl", must produce important changes in all the major aspects of tile educational system - in its organization, its objectives, its procedures, and its achievements, and that massive, lasting changes in education cannot safely be made except on the basis of objective scientific inquiry. This, of course, requires that educational research ought to influence and to guide the decisions of educational policy-makers at all levels, from the teachers in their classrooms and the school principals to the politically authorized decision-makers at the level of the *A previous version of this paper was given as the opening lecture at the first congress of the Israel Educational Research Association at Tel A',iv University in April 1976. 215
_.2'[ b
KI,FINBI R(;I.IR
national legislatt,re and executive (Cohen & (.k,ret. 1975: Crtmbach & Suppes. I~(~): Smith & James, 1975: Yates. 1971 ). Some educational researchers have deveh)ped models of what m their view COllStltutes the rational process of change and reform m the sphere of education, such as tire model suggested by the Ame,-ican professors David ('lark and Egon Guba (1965). According to this model, the rational process of change and improvement in education is seen as involving four major stages. (a) Basic research is the starting point m the chain of events leading to the introduction of a new practice, in that its insights may suggesl the possibility of an improvement in tire processes and procedures of education. (b) On this foundation the stage of derelopme;zt follows in which, by utilization of relevant research results, a new solution is being invented and elaborated for some practical problem confronting those responsible for tire management of educational affairs. (c) When, after repeated experiments and revisions, an adequate solution has been found in a form suitable for large-scale use. the state of d~lJi~sbm folh)ws, in which the innovation is brought to tire attention of educational practitioners, and its usability in real situations is demonstrated. ( d ) F i n a l l y , there is the state ol'adoptio;t of the new practice which is but a series of experiments under tire spe q'ic circumstances of different educational institutions. Not only at the first stage, but at all four stages, objective scientific inquiry plays a decisive role. From stage to stage, however, it becomes more "'applied" and directly oriented towards the specific needs, conditions, and problems of particular situations and institt, tious. If we are still rather remote l'roflt applying this rational method by which educational policy-making can be directed by research towards educational change and improvement. according to current opinion among experts on educational research this is merely the result of contingent and temporary difficulties. On the one hand, systematic research on education and the application of basic scientific disciplines to educational issues represent a relatively recent development. In consequence, it is only natural that the relevant scientific inquiry has not yet been able to offer tested and reliable answers to many of the questions and problems with which educational practitioners arc confronted. ()n the other hand, there are regrettable misunderstandings and failures of communication between researchers and educational decisi
RESEARCH AND POLICY-MAKING
217
and better practices and organizational patterns by educational institutions or systems (Husen & Boalt, 1967; Yates, 1971). 1 shall adduce here sections from two reports submitted by Swedish commissions. The 1946 School Commission, which consisted mainly of members of Parliament, remarked in the report which it submitted to the Swedish government in 1948 that the results of psychological and educational research have to play a decisive role with respect to those problems which have already been systematically investigated. In a section of the report, entitled The Need of Analyzing Instructional Objectives, this cdmmission has demanded to undertake comprehensive scientific inquiries on which the detailed goals of the school can be based, for practical experience in education is not sufficient for the selection of those items of subject-matter which can be considered valuable (Husen & Boalt, 1967). Likewise the 1957 School Committee which undertook the development of a new curriculum for. the reformed nine-year comprehensive school initiated an extensive program of scientific research with a view to determining empirically the demands made by society of the individual, both as the occupant of a certain vocation and as a citizen, with respect to specific items of knowledge and skills (Husen & Boalt, 1967). Likewise in Israel the conviction has prevailed that scientific research has the authority to guide and direct those responsible for making educational policy decisions even with regard to the determination of goals and objectives. A team of experts in the Ministry of Education, which in February 1974 handed in its report concerning the policy of fostering culturally disadvantaged social groups in the educational system, wrote in one of its recommendations: "Fostering projects are to be carried out only provided that the following conditions have been satisfied: a precise determination of goals and objectives which are based on theoretical and empirical knowledge." (p. 5) It is the central thesis of this essay that the conception of the function of educational research (including relevant inquiry in basic disciplines, like psychology, sociology, economics, etc.) to direct and steer educational policy decisions is false, and that the accepted model of a straight and consistent chain of events that leads from basic and applied research directly to decisions about the introduction of new practices or patterns of organization into educational institutions or systems is entirely wrong and impossible. 'My argument is based on three considerations which 1 shall term (a) methodological, (b) logical, and (c) sociological. (a) The methodological consideration does not concern the difficulties and problems by which social research in general and educational research in particular are affected in various methodological respects with regard to research design, the operational definition of concepts and variables, the selection of samples, measuring instruments and methods of observation, statistical data processing and its interpretation, etc. Neither does it concern the corresponding methodological deficiencies and defects for which many of the investigations undertaken in the sphere of education have been criticized (Averch et al., 1972). Even if all these methodological problems find their satisfactory solutions, and the investigations are sound and faultless in every methodological respect, there is an inherent limitation of scientific method which prevents the derivation of educational policy decisions as inferences from the results of specific empirical inquiries. Scientific method achieves maximum accuracy by resolving complex phenomena or situations into their analytical components, isolating from them a limited number of exactly definable and observable variables (that holds also for so-called "multi-variate"
21B
KLI.INBI.R(,IR
research designs), under the strictest possible control of other variables, which are liable to influence tile investigated phenomenon or situation, and undertaking controlled observations, planned experimental manipulations, and exact measurements of the relatively few variables that have been isolated from the total situation, in order to discover regular and systematic relationships between them. IIowever, the persons who are responsible for decisions about the introduction of new practices and models into the educational system and tbr their implementation cannot disregard the intricate web of complex relationships among the many and variegated factors involved in real educational situations. This was pointed out already a number of years ago by a Committee on Educational Research under the chairmanship of Lee Cronbach and Patrick Suppes, which was appointed by the National Academy of Education in the United States, and pnblished its report in 1969 under the title Res'earch for Tomorrow's Schools. As the authors of this book have made clear, a new educational practice may be admirable in one respect and detrimental in another, for it is liable to influence the pupils in various ways. Apart from its (often exclusively studied) effect on the mastery of subject-matter, it is apt to produce results with regard to the pupils" interests ~,nd motivation for learning, their self-direction or dependence upon direction by others, cooperativeness or competitiveness among them, and many other sets of dispositions and propensities. Yet not all possible effects of the introduction of a new educational practice - in particular not all its accumulative hmg-range effects can be the object of systematic observations in a specific investigation. Therefore, no specific finding obtained m some partial inquiry can be translated directly into a policy conclusion. Tiffs problem becomes even more complicated, when interaction is present between variables which cannot all be kept constant at the same time. For instance a given method of instruction, which is successft,1 with one kind of pupil will not necessarily influence others equally favorably. It may be of advantage when applied by a certain type of teacher, but not st) in the hands of another" it may produce desired effects in some schools, but not in others: etc. This problem is less troublesome in the natural sciences for they are as a rule able to isolate systems of variables ahnost completely. The social sciences, however, have had only slight success with the isolation of systems. In such a casc. when interactions between partly uncontrolled variables are present, it is rarely safe to generalize, about a relationship between few variables which have been investigated without consideration of additional factors. In view of these difficulties which are inherefit in scientific method, when it is applied to social and educational problems, the authors of the afore-mentioned book, too, speak of the myth that practical measures can be directly derived as conclusions from research findings (Cronbach & Suppes, 1969; of. also Averch el al., 1972; Cohen & Garet, 1975). l lence it is reasonable for educational decision-makers to take into consideration the tested and reliable results of systematic scientific investigations. But they act irresponsibly and are liable to cause great harm to the educational system, if they attempt to translate specific research findings directly into policy conclusions. This danger shall be illustrated by three examples. A well-known psychologist wrote an expert opinion for the German Council on Education {Deutscher Bildungsrat a body consisting o f scientists, representatives of the public, politicians, and educational administrators which has the function to prepare lbr the authorized decision-makers recommendations concerning the planning and develop-
RESEARCH AND POLICY-MAKING
219
merit of the educational system as well as its structure). This opinion, which was based on the findings of existing psychological investigations dealt with the promotion of the motivation for learning and of intellectual achievements. One of its conclusions was that in order to increase the motivation for learning and intellectual attainments, it would be necessary to secure an "optimal fit" between the grade of difficulty in instruction and the level of intellectual development reached by each individual pupil. The fit between them is said to be optimal, when the grade of difficulty in the instructional program is ~uch as to exceed in its demands only slightly the pupil's present level of achievement. On the basis of these research results and considerations the psychologist recommended to introduce homogenous grouping by achievement in several subjects of instruction which possess a logical-systematic structure, primarily in mathematics, natural sciences, and foreign languages. The remaining subjects -- above all the mother tongue - were to be taught in ungrouped (i.e intellectually heterogenous) age-classes (Heckhausen, 1969). But the attempt to infer practical organizational measures as conclusions from the t'mdings of empirical investigations, which had isolated a few exactly defined motivational variables from the totality of factors which are involved in the complex school situation, appears highly questionable. For such an attempt disregards all other potential effects of the proposed homogenous achievement grouping -- such as its influence on teachers' behavior, on the interaction among the pupils, on their social esteem, on their emotional development, etc. Hence, there exists a serious danger that deriving a modification in the school structure, which forms part of a highly complex social context, from research findings which were obtained by the isolation of a few psychological variables is liable to result in unforeseen detrimental consequences (cf. Becker, 1975). A similar danger is illustrated by the second example. Psychologists endeavour to improve the reliability and validity of texts and examinations as instruments for predicting chances of success at subsequent stages of formal education and in various occupational careers. But it would be wrong to take decisions about making use of improved tests and examinations solely on the basis of their demonstratedly high predictive validity, without consideration of their possibly harmful and perverting backlash effects on what is going on inside the schools which will now prepare their pupils for the requirements of the new tests and exams (Young, 1965). The third example has been taken from the afore-mentioned report of the expert team charged with proposing a policy for the advancement of culturally disadvantaged pupils in Israel. The team reviewed the findings of empirical investigations which had studied the relative effectiveness of different methods for the promotion of cognitive development in the preschool, both by means of direct instruction and in indirect ways (e.g. through dance and play). It reached the conclusion that the indirect methods showed no advantage over direct instruction as regards the promotion of cognitive development. However, the team remarked that the two types of techniques might possibly contribute in different measures to the development of the children's personalities in non-cognitive respects which had not been investigated in the studies under review (Israel, Ministry of Education, 1974). Hence it would be irresponsible to draw policy conclusions directly from these research findings. This problem becomes even more complicated by two additional factors. First of all, educational problems are multi-disciplinary in character because the variables involved in them (such as pupils' and teachers' attributes and behaviors, methods of instruction and curricular subject-matter, the social environment, etc.) are frequently investigated
220
K 1.t.INI'~IIR(;1i R
by separate scientific disciplines. Therefore, it is unreasonable to infer practical consequences immediately from the findings and theories of tmc single scientific discipline. Secondly, even within a given scientific discipline (e.g. psychology) rival theories and schools have developed, and each of them concentrates its attention one-sidedly on one particular aspect of the nmlti-dimensional human personality and mode of acting. Ira view of these facts, .Joseph Schwab in an essay about "the corruption of education by psychology", published ahnost twenty years ago, already warned that the factual and theoretical results of many psychological studies concerning education were nothing but "'partial truths", and that from partial truths no reliable practical conclusions could be drawn (Schwab, 1958). (b) The logical co~tsiderati(m is quite ancient and extremely simple, yet is nevertheless often disregarded both by scientists and by practitioners. It refers to the rule that in a logically valid inference one cannot derive as a conclusion more than has been given in the premises. In this respect a logical inference is analogous to a mincer: if at one end you put in merely bread and l\md coloring, it is impossible to get out at the other end meat balls. Similarly, from the propositions of an empirical science about what is the case, why it is the case, and under what conditions it may possibly or must necessarily be the case no policy conclusions about what ought to be the case or to be done could be validly derived. In order to draw logically valid inferences about measures which ought to be taken, we need as additional premises propositions or judgments about values and aims which empirical research is not competent to provide. This consideration shall be illustrated by the empirical curriculum research which ',,,'as undertaken in Sweden upon the initiative of the 1957 School ('ommittee. The fundamental idea of these investigations was to examine the relevance which the items of knowledge and skills included in the syllabi for mathematics and the mother language had for the demands posed by vocational education and by the world of work, especially in industrial and lower "white collar" occupations. On account of the measuring instruments and teclmiques at their disposal, the investigators limited their attention to the cognitive effects of instruction in these two school subjects, and did not deal with its non-cognitive results in the form of attitudes, interests and values. Data were obtained with regard to three sets of facts: (I) the items of knowledge and skills which were required in various courses of vocational education and in different occupational careers; (2) the actual use made in these areas of the items of knowledge and the skills included in the existing syllabi; and (3) the differential levels of their mastery several years after the termination, t)f schooling. From the findings of these empirical investigations the researchers drew "'implications" regarding the relative priority of different classes of topics which ought to be the basis for constructing the curricula ira the two subjects within the framework of universal compulsory schooling. Thus, for instance, they identified topics which ought to be included in the common core mathematics course -- e.g., the four basic operations with integers, decimal fractions, measurements in the decimal system, percentages, interest, and averages. By contrast, topics with low usability for the maiority of occupational training courses and careers, such as algebra and formal demonstrative geometry, ought to be omitted from the obligatory programme in mathematics. According to the same principle, they recommended to give ira the curriculum for the mother language greater weight to non-fictional prose, in contrast with fiction and poetry which had occupied a dominant position in conventional syllabi (Husen & Boalt, 1967).
RESI-ARCH AND POLICY-MAKING
221
It is obvious that these practical recommendations as to what ought to be done and changed in teaching do not follow from the empirical research findings, unless two value judgments are presupposed wtuch by no means can be accepted as self-evident nor have been established by the investigations under review but which have been underlying them (though only implicitly). They are (1) that the school ought primarily to impart knowledge and skills which are useful for the world of occupations, and (2~ that a human being needs neither a widening of his intellectual horizon nor familiarity with either the basic forms of knowledge and understanding or the creations of the human mind, unless these are usable and relevant for his personal and vocational needs. Sometimes the extra-scientific value premisses have been built into the empirical research design itself. Richard Atkinson from Stanford University has developed a model for the analysis of instruction which is based upon decision theory, and has the major advantage that it takes into account the relationship between the cost of a treatment and its effects which has been disregarded by the usual kind of evaluation studies in education. With the aid of this model he has examined the optimal use to be made of a computer in initial reading instruction. For this purpose he required a specification of the teaching objective which could serve as a criterion for the optimalization of use. He had a choice between several possibilities, for instance: (a)to maximize the average test achievement for the class as a whole; (b)to reduce as far as possible the variance in test achievements among the pupils of the class: (c)to maximize the average test achievement, with the provision that the resulting variance does not exceed the measure which would have been obtained without the use of a computer: Id)to maximize the number of pupils who achieve the norm of their grade. It was obvious to him that these different objectives implied different strategies o~ optimalization for the use of the computer, llowever, what apparently escaped his notice was that these different conceptions of the teaching objective represent contradictory social philosophies and value attitudes between which no empirical research, not even one based on decision theory, is competent to decide (Atkinson, 1973). In view of this logical consideration, it is not surprising that different persons can "conclude" contradictory policies from the same research findings. This shall again be illustrated by an example taken from Sweden. Upon the evidence given by a number of psychologists and as a result of investigations that have been commissioned by several School Committees, it was accepted as a scientifically valid finding that success in academic-theoretical studies was primarily dependent upon, and predictable by, general intelligence which could be determined with adequate certainty at the age of 11, whereas specific vocational aptitudes and interests could not be diagnosed before the age of 14. Part of the members of the 1940 School Committe, which consisted mainly of teachers of all levels of education, many of whom held somewhat conservative views, "concluded" from this finding that intellectualy gifted pupils ought to transfer at the age of 11, after completing the fourth grade, to a selective secondary school (the realskola). By contrast, the 1946 School Commission, which consisted of members of Parliament with rather radical opinions, "inferred" from the same finding that an early transfer of pupils with high general intelligence to selective secondary schools was unjustified because in that way the practical and technical occupations would be deprived of intellectually gifted pupils, and in consequence were going to suffer a
222
K LEINB I:R(;I. R
dangerous loss of prestige; t\)r this reason the Commission recommended a later organizational differentiation in the school system (Husen & Boalt, 1967). We may sum up the logical consideration with Boyd H. Bode's well-known saying: "Statistics can inform us how many policemen and how many thieves exist in a given society, llowever, they cannot tell use whether there ought to be more policemen or more thieves". (c) The sock)logical consideration points to the fact that actually the relationship between research and policy-making is frequently contrary to the model according to which the former directs and steers the laner. In other words, the policy decisions of the political estabfishment and the accepted social nomls underlying them determine to a considerable extent tire directions taken by social and educational research, the questions it raises, and those which it fails to deal with or even to ask. 1! is impossible to go here into the methods and mechanisms of exercising financial and poqtical power by means of which the political establishment influences tile directions of social research. But the fact of this influence is beyond doubt. It becomes manifest primarily in sudden and extreme changes in tile fashions of educational research which occur in the wake of new policy trends on the part of the political establishment. A dramatic example has been provided by the "'shock" caused by the first "Sputnik" in the western world and especially in the United States. Under its inflt, ence the educational policy of those countries was directed towards the qualitative and quantitative improvement of the training of natural scientists and technt)logists. This reversal in educational policy brought in its wake decisive changes in the directions and fashions of educational research. In comparative education a series of investigations about the education of scientists and technologists in the Soviet Union was started, followed by the series of studies analyzing the preparation ot" scientific and engineering manpower in the OECI) member countries. An intensive and comprehensive activity began with a view to developing new mathematics and science curricula the common approach of which consisted in teaching these subjects by way of inquiry, so that the pupils should learn how to " d o " mathematics, physics, etc. in the manner in which they are pursued by mathematicians and scientists (Young, lt)73). A new fashion of dealing with "creativity" sprang up, both by developing new kinds of tests for measuring its abilities and predicting its chances, and by investigating methods of its nurture, while in the background there was the need to identify future "'creative" scientists and inventors, and to actualize their promising potentialities (Young, 1973). Further examples can be touched here only briefly: investigations of foreign language instruction, when tile gove,-nment of the United States recognized an increased need for personnel mastering foreign languages both for its intelligence and espionnage services and for technical aid to developing countries; the discovery of education by economic research as a profitable investment in human capital, when it became obvious that success in international economic competition was dependent on tile quality and quantity of skilled manpower (Wall & Husen, 1968); and finally the fact that the National Foundation for Educational Research in England and Wales which has been founded after the promulgation of the English Education Act of 1944 devoted during the first decade of its existence nearly all its resources to research on the allocation of primary school leavers to suitable types of secondary schools and on selection for academic secondary schools (grammar schools) that is to say, to the study t)f problems with which that Act had confronted the schools and local education authorities (Yates, 1971 ).
RESEARCH AND POLICY-MAKING
223
The last example reflects already a deeper sense of the sociological consideration. Not only the directions of research are largely determined by the decisions of the political establishment, but empirical investigators frequently tend to accept as natural and self-evident the status quo and the unexamined normative presuppositions of current educational practices and policies. Characteristic in this respect is the extensive research effort that has been devoted in a number of countries to the inequality of educational opportunities between socio-economic classes, races, and socio-cultural groups, and to the problems of cultural deprivation or environmental disadvantage. This research has focused on the social characteristics of the school failures and drop-outs and on the correlation between these characteristics and the differential rates of success and failure in formal education, while at the same time accepting as natural and selfevident the present curricula and their organization and structure, the usual procedures and methods of teaching, as well as the kinds of abilities and aptitudes which are fostered and promoted by traditional schooling, and which are required for success in it - in short, the entire system of recognized norms and criteria which determine who is considered a success and who a failure. In-consequence, failure in school appears, in relation to these norms and criteria, as a sort of deviance or misadaptation (Young, 1973; Glaser, 1973; Israel, Ministry of Education, 1974). Further examples can only be touched here briefly: validation of intelligence and ability tests by the criterion of success with the modes of teaching and learning of the existing school, and their utilization for the selection of pupils who are suitable for an accepted and immutably fixed educational environment (Cronbach & Suppes, 1969; Glaser, 1973); the comparison between the achievements of comprehensive shools in Sweden and England and of selective secondary schools in the traditional school system by the conventional criteria of success (Husen & Boalt, 1967; Wall & Husen, 1968); investigations of drop-outs which are based on an uncritical acceptance of the dominant assumption that an extended period of school attendance is beneficial also for those adolescents who are not interested in it and do not enjoy it. In short, educational research tends to ask how the existing things are being done, how they can be done better and more effectively, and how the educational achievements can be improved with reference to the currently prescribed aims of schooling. It accepts what is usual and existing as if it were a law of nature. As a rule, however, it refrains from asking heretical and unorthodox questions with regard to the dominant conceptions and assumptions, and from examining radical alternatives to the status quo (Brunet, 1975; Rowher, 1973; Young, 1965; cf. also Argyris, 1969; Kelley, 1971). If the conception that research is competent to steer educational policy-making, must, then, be rejected, the question arises: what can educational research contribute to an improved and more rational educational policy? Within the limits of a short essay, one can merely attempt to answer this question with three brief hints. (1) Research can create a more secure and reliable foundation for decisions concerning the introduction of reforms and innovations by evaluating their relative effectiveness. Of course, the findings of such evaluative studies can never provide the sole conclusive consideration for those kinds of decisions,and all the reservations which have been raised above in my methodological consideration are also valid in this context for the potential contribution of research. Here a further reservation shall be raised against a certain kind of research which I shall call "fetishistic" investigations of "fetishistic" policies. "Fetishism" in Freud's
224
K Lt.I NIJt- P,(;I.R
terminology means that sexual satisfaction is not sought in the genitals, but through other parts of the body or through objects which normally serve only as means for the arousal of sexual passion, but not as tile goal of the latter. In analogy, fetishistic policies and the fetishistic research subserving them deal only with external frameworks and arrangements which are destined to make certain improved ways and modes of treatment possible, but not with tile treatments themselves which represent their purpose. I have in mind evaluative studies about the effectiveness of such arrangements as educational institutions for tile preschool age, provision of increased financial and other resources for the schools, reduction of class size. homogenous ability grouping, an extended school day, observation cycles, comprehensive schools, etc. As is well known, evaluative studies conducted in many countries have as a rule failed to discover any significant differences in school achievements between the childrexl who have enjoyed the benefit of such or similar arrangements and those who have not (Aver, (I et al.. 1972; Elvin, 1975; Husen & Boalt, 1967; Israel, Ministry of Education, 1974: Rossi & Williams, 1972: Smith & James, 1975; Tizard, 1975: Wall & Husen, 1968). Why shotdd that come as a surprise? These investigations have not examined what was happening within tile frameworks in question, or how the arrangements under review were being utilized, and whether c)r not any innovative or improved treatments which it had been their function to make possible had been introduced at all. (Indeed those who make the educational decisions and who carry them out have also not infrequently been satisfied with making available the external frameworks and arrangements without providing for the improved treatments). Imagine that in some courltry multi-lane highways have been built which are devised to make possible faster and safer driving than on the old roads. Yet the traffic on the new super highways proceeds with the same mule carts as was the case on the old dust roads. If in that country azl evaluative study after the model of the afore-mentioned educational investigations is undertaken about the relative effectiveness of the new highways, it, it)o, will come to the conclusion that no significant differences exist with regard to the speed and the safety of transportation between these two kinds of roads. (2) On the basis of their research, investigators can suggest to the educational decision-makers more pertinent and fruitful questions than tho3e they had been used to ask hitherto, and thus turn educational research, too, towards new and more promising lines of inquiry. Only a few examples of what may be termed with a Platonic expression the "turning of the soul" can be mentioned here very briefly. Instead of asking at which age it would be advisable to start with teaching a first foreign language, educational research has raised the question, by means of which methods of t\)reign language instruction is it possible to achieve success already with younger children? (Wall. & lhisen. 1968). Instead of looking for improved tests for diagnosing general ability which are capable of predicting success in fixed courses of study that require only certain kinds of capacity, and of selecting a relatively small number of candidates who are suited for these courses of study, some psychologists (among them Lee Cronbach) have recommended asking a new question: How is it possible to develop differentiated tests which can help us to choose for each individual a teaching method and learning style which are suited t() his particular cognitive style and to his individual aptitudes, inclinations and interests? (Cronbach & Suppes, 1969: Elvin, 1975).
RESEARCH AND POLICY-MAKING
225
Economic theory has drawn our attention to the costs of education not merely in terms of money and manpower which are directly invested in it, but also in terms of income which the young persons forgo while attending educational institutions, and in terms of their non-participation in the labour force and creation of the national product. Framing the question about the costs of education from this novel viewpoint may perhaps encourage a search for ways to a more economical utilization of the pupils' time and of shortening the duration of school attendance (Cronbach & Suppes, 1969). (3) The most important potential contribution of research to the improvement of educational practice lies in the production of revolutionary ideas which can serve as foundations for the invention of innovative procedures, models, and teclmologies. There is, however, little hope that it will produce such fertile new ideas as long as educational research is required to deal only with current practical concerns and to achieve specific results which can be immediately translated into useful solutions. D.E. Berlyne has illustrated this by a striking example: if around the year 1800 somebody had been requested to conduct research with a view to improving transportation, he would probably have sought ways of breeding a faster running race of horses or of installing more elastic springs in coaches. However, the research which in fact led ultimately to a revolutionary progress in transportation did not deal with horses and coaches. An Englishman named James Watt was occupied with steam engines for pumping water from mines, and an Italian named Galvani detected a strange phenomenon in frogs' legs. Nobody at that time could have imagined that the investigation of frogs' legs -- and not of horses' legs - was destined to bring about a revolution in transportation (Yates, 1971 ). The moral of this story is that our best hope rests in encouraging basic research - or in the terminology preferred by Cronbach and Suppes, conclusion-oriented in contrast with decision-oriented inquiry (Cronbach & Suppes, 1969). That means that scientists who propose research projects that appear well-founded and promising are granted the freedom to follow their own ideas and to frame their own questions, and that they are not invariably and exclussively harnessed to the immediate practical exigencies of educational practitioners, nor required to deliver within a specified time specific results which can be immediately translated into usable practical solutions. That, of course, involves a risk. Nobody including the investigator himself - can possibly foresee which direction of basic research not oriented towards immediate needs, may yield ideas that will ultimately turn out to be fertile ground for revolutionary innovations in education. But without the readiness to run risks, there can be no chance that perhaps one in a hundred inquiries and investigations may prove pregnant with consequences. "Cast thy bread upon the waters, for thou shall find it after many days." (Ecclesiastes XI, 1)
REFERENCES
ARGYRIS, C. The Incompleteness of Social-Psychological Theory. American Psychoh>gist, 24, 1969, pp. 893-908. ATKINSON, R.C. Ingredients for a Theory of Instruction. In Wittrock, M.('. (ed.), ChaJlging Education. Alternatives J~'om Educational Research. Englewood Cliffs, N.Y.: Prentice Hall, 1973, pp. 6 5 - 8 2 . AVERCH, H.A. et al. How Effective is Schooling? A Critical Review and Synthesis o f Research Findings. Santa Monica, Cal.: Rand Corporation, 1972.
22{,
K I,I. INBI!R(;I. R
BECKER. H. Beitrag und Einfluss dcr Bildungsforschung auf die Arbeit tics "'l)culschcn Bildungsrats'" Zeitschrift fiir Pddagogik, 2 I. 1975. pp. 15'4 172. B R U N E R , J. The Role of the Researcher us an Adviser Io the Educational Policy Maker. Oxfi)rd R e v i e w o.! Education. 1, 1975. pp. 183 188 (_?LARK. D.L. and (;UBA. t-.I;. ,4n Examination ~),I Potential ('hangc Roles m l:ducation. P~,per presented to the N E A - - ( ' S I Seminar on Innovation in Planning School (urricula. Aerlie Ilouse, Virginia. l t~()5. ('OHEN. D.K. and (;ARI-T. M.S. Reforming Educali
RESEARCH
Volume
AND POLICY-MAKING
2. No. 3
Winter 1976
IN E D U C A T I O N *
AHARON F. KLEINBERGER
The slogan of the intellectuals in our generation is "relevance". In an epoch in which the intellectuals have lost their self-confidence, and have retreated to defence positions before the assault of vitalism and irrationalism, they have no longer the courage to believe or at least to proclaim what their predecessors in earlier ages believed and proclaimed: that knowing the truth and understanding ourselves and the universe are values in themselves and worthy purposes of human existence, and that the intellectual activity which aspires to these aims is the summum bomun, the highest good, and that it comprises the supreme happiness and tile most sublime human perfection. From their defence positions they seek the justification for their intellectual activity by showing it to be "relevant" to our most vital needs and our most violent desires. The constant approximation to tile knowledge of truth and to a fuller understanding, instead of serving as an end in itself, is being justified as an effectful means for solving practical problems and for the satisfaction of urgent needs. One can hardly find more striking evidence of this attitude than tile proclamations issued by the participants in an international meeting of experts on educational research which was convened in 1967 under the sponsorship of the UNESCO Institute for Education in Hamburg, and which resulted in a report published in 1971 under the title The Role of Research in Educational Change. q-here we read that tile sponsors and the participants of the conference are committed to the view that the attainment of knowledge for its own sake is an inadequate objective for educational research, and that its justification must rest on the contribution it can make to the improvement of educational practice. This is true not only with respect to what is called "applied research". The value of so-called "'basic" or "'pure" research, as well, rests not on the extension of our knowledge and understanding, but on its demonstrated contribution to the improvement of educational practices, m that it will ultimately lead to the introduction into the educational system of new and more effective forms of organization and procedures. Indeed, there is wide agreement within the ranks of those engaged in educational research that their activity, if it is to be "'relevanl", must produce important changes in all the major aspects of tile educational system - in its organization, its objectives, its procedures, and its achievements, and that massive, lasting changes in education cannot safely be made except on the basis of objective scientific inquiry. This, of course, requires that educational research ought to influence and to guide the decisions of educational policy-makers at all levels, from the teachers in their classrooms and the school principals to the politically authorized decision-makers at the level of the *A previous version of this paper was given as the opening lecture at the first congress of the Israel Educational Research Association at Tel A',iv University in April 1976. 215
_.2'[ b
KI,FINBI R(;I.IR
national legislatt,re and executive (Cohen & (.k,ret. 1975: Crtmbach & Suppes. I~(~): Smith & James, 1975: Yates. 1971 ). Some educational researchers have deveh)ped models of what m their view COllStltutes the rational process of change and reform m the sphere of education, such as tire model suggested by the Ame,-ican professors David ('lark and Egon Guba (1965). According to this model, the rational process of change and improvement in education is seen as involving four major stages. (a) Basic research is the starting point m the chain of events leading to the introduction of a new practice, in that its insights may suggesl the possibility of an improvement in tire processes and procedures of education. (b) On this foundation the stage of derelopme;zt follows in which, by utilization of relevant research results, a new solution is being invented and elaborated for some practical problem confronting those responsible for tire management of educational affairs. (c) When, after repeated experiments and revisions, an adequate solution has been found in a form suitable for large-scale use. the state of d~lJi~sbm folh)ws, in which the innovation is brought to tire attention of educational practitioners, and its usability in real situations is demonstrated. ( d ) F i n a l l y , there is the state ol'adoptio;t of the new practice which is but a series of experiments under tire spe q'ic circumstances of different educational institutions. Not only at the first stage, but at all four stages, objective scientific inquiry plays a decisive role. From stage to stage, however, it becomes more "'applied" and directly oriented towards the specific needs, conditions, and problems of particular situations and institt, tious. If we are still rather remote l'roflt applying this rational method by which educational policy-making can be directed by research towards educational change and improvement. according to current opinion among experts on educational research this is merely the result of contingent and temporary difficulties. On the one hand, systematic research on education and the application of basic scientific disciplines to educational issues represent a relatively recent development. In consequence, it is only natural that the relevant scientific inquiry has not yet been able to offer tested and reliable answers to many of the questions and problems with which educational practitioners arc confronted. ()n the other hand, there are regrettable misunderstandings and failures of communication between researchers and educational decisi
RESEARCH AND POLICY-MAKING
217
and better practices and organizational patterns by educational institutions or systems (Husen & Boalt, 1967; Yates, 1971). 1 shall adduce here sections from two reports submitted by Swedish commissions. The 1946 School Commission, which consisted mainly of members of Parliament, remarked in the report which it submitted to the Swedish government in 1948 that the results of psychological and educational research have to play a decisive role with respect to those problems which have already been systematically investigated. In a section of the report, entitled The Need of Analyzing Instructional Objectives, this cdmmission has demanded to undertake comprehensive scientific inquiries on which the detailed goals of the school can be based, for practical experience in education is not sufficient for the selection of those items of subject-matter which can be considered valuable (Husen & Boalt, 1967). Likewise the 1957 School Committee which undertook the development of a new curriculum for. the reformed nine-year comprehensive school initiated an extensive program of scientific research with a view to determining empirically the demands made by society of the individual, both as the occupant of a certain vocation and as a citizen, with respect to specific items of knowledge and skills (Husen & Boalt, 1967). Likewise in Israel the conviction has prevailed that scientific research has the authority to guide and direct those responsible for making educational policy decisions even with regard to the determination of goals and objectives. A team of experts in the Ministry of Education, which in February 1974 handed in its report concerning the policy of fostering culturally disadvantaged social groups in the educational system, wrote in one of its recommendations: "Fostering projects are to be carried out only provided that the following conditions have been satisfied: a precise determination of goals and objectives which are based on theoretical and empirical knowledge." (p. 5) It is the central thesis of this essay that the conception of the function of educational research (including relevant inquiry in basic disciplines, like psychology, sociology, economics, etc.) to direct and steer educational policy decisions is false, and that the accepted model of a straight and consistent chain of events that leads from basic and applied research directly to decisions about the introduction of new practices or patterns of organization into educational institutions or systems is entirely wrong and impossible. 'My argument is based on three considerations which 1 shall term (a) methodological, (b) logical, and (c) sociological. (a) The methodological consideration does not concern the difficulties and problems by which social research in general and educational research in particular are affected in various methodological respects with regard to research design, the operational definition of concepts and variables, the selection of samples, measuring instruments and methods of observation, statistical data processing and its interpretation, etc. Neither does it concern the corresponding methodological deficiencies and defects for which many of the investigations undertaken in the sphere of education have been criticized (Averch et al., 1972). Even if all these methodological problems find their satisfactory solutions, and the investigations are sound and faultless in every methodological respect, there is an inherent limitation of scientific method which prevents the derivation of educational policy decisions as inferences from the results of specific empirical inquiries. Scientific method achieves maximum accuracy by resolving complex phenomena or situations into their analytical components, isolating from them a limited number of exactly definable and observable variables (that holds also for so-called "multi-variate"
21B
KLI.INBI.R(,IR
research designs), under the strictest possible control of other variables, which are liable to influence tile investigated phenomenon or situation, and undertaking controlled observations, planned experimental manipulations, and exact measurements of the relatively few variables that have been isolated from the total situation, in order to discover regular and systematic relationships between them. IIowever, the persons who are responsible for decisions about the introduction of new practices and models into the educational system and tbr their implementation cannot disregard the intricate web of complex relationships among the many and variegated factors involved in real educational situations. This was pointed out already a number of years ago by a Committee on Educational Research under the chairmanship of Lee Cronbach and Patrick Suppes, which was appointed by the National Academy of Education in the United States, and pnblished its report in 1969 under the title Res'earch for Tomorrow's Schools. As the authors of this book have made clear, a new educational practice may be admirable in one respect and detrimental in another, for it is liable to influence the pupils in various ways. Apart from its (often exclusively studied) effect on the mastery of subject-matter, it is apt to produce results with regard to the pupils" interests ~,nd motivation for learning, their self-direction or dependence upon direction by others, cooperativeness or competitiveness among them, and many other sets of dispositions and propensities. Yet not all possible effects of the introduction of a new educational practice - in particular not all its accumulative hmg-range effects can be the object of systematic observations in a specific investigation. Therefore, no specific finding obtained m some partial inquiry can be translated directly into a policy conclusion. Tiffs problem becomes even more complicated, when interaction is present between variables which cannot all be kept constant at the same time. For instance a given method of instruction, which is successft,1 with one kind of pupil will not necessarily influence others equally favorably. It may be of advantage when applied by a certain type of teacher, but not st) in the hands of another" it may produce desired effects in some schools, but not in others: etc. This problem is less troublesome in the natural sciences for they are as a rule able to isolate systems of variables ahnost completely. The social sciences, however, have had only slight success with the isolation of systems. In such a casc. when interactions between partly uncontrolled variables are present, it is rarely safe to generalize, about a relationship between few variables which have been investigated without consideration of additional factors. In view of these difficulties which are inherefit in scientific method, when it is applied to social and educational problems, the authors of the afore-mentioned book, too, speak of the myth that practical measures can be directly derived as conclusions from research findings (Cronbach & Suppes, 1969; of. also Averch el al., 1972; Cohen & Garet, 1975). l lence it is reasonable for educational decision-makers to take into consideration the tested and reliable results of systematic scientific investigations. But they act irresponsibly and are liable to cause great harm to the educational system, if they attempt to translate specific research findings directly into policy conclusions. This danger shall be illustrated by three examples. A well-known psychologist wrote an expert opinion for the German Council on Education {Deutscher Bildungsrat a body consisting o f scientists, representatives of the public, politicians, and educational administrators which has the function to prepare lbr the authorized decision-makers recommendations concerning the planning and develop-
RESEARCH AND POLICY-MAKING
219
merit of the educational system as well as its structure). This opinion, which was based on the findings of existing psychological investigations dealt with the promotion of the motivation for learning and of intellectual achievements. One of its conclusions was that in order to increase the motivation for learning and intellectual attainments, it would be necessary to secure an "optimal fit" between the grade of difficulty in instruction and the level of intellectual development reached by each individual pupil. The fit between them is said to be optimal, when the grade of difficulty in the instructional program is ~uch as to exceed in its demands only slightly the pupil's present level of achievement. On the basis of these research results and considerations the psychologist recommended to introduce homogenous grouping by achievement in several subjects of instruction which possess a logical-systematic structure, primarily in mathematics, natural sciences, and foreign languages. The remaining subjects -- above all the mother tongue - were to be taught in ungrouped (i.e intellectually heterogenous) age-classes (Heckhausen, 1969). But the attempt to infer practical organizational measures as conclusions from the t'mdings of empirical investigations, which had isolated a few exactly defined motivational variables from the totality of factors which are involved in the complex school situation, appears highly questionable. For such an attempt disregards all other potential effects of the proposed homogenous achievement grouping -- such as its influence on teachers' behavior, on the interaction among the pupils, on their social esteem, on their emotional development, etc. Hence, there exists a serious danger that deriving a modification in the school structure, which forms part of a highly complex social context, from research findings which were obtained by the isolation of a few psychological variables is liable to result in unforeseen detrimental consequences (cf. Becker, 1975). A similar danger is illustrated by the second example. Psychologists endeavour to improve the reliability and validity of texts and examinations as instruments for predicting chances of success at subsequent stages of formal education and in various occupational careers. But it would be wrong to take decisions about making use of improved tests and examinations solely on the basis of their demonstratedly high predictive validity, without consideration of their possibly harmful and perverting backlash effects on what is going on inside the schools which will now prepare their pupils for the requirements of the new tests and exams (Young, 1965). The third example has been taken from the afore-mentioned report of the expert team charged with proposing a policy for the advancement of culturally disadvantaged pupils in Israel. The team reviewed the findings of empirical investigations which had studied the relative effectiveness of different methods for the promotion of cognitive development in the preschool, both by means of direct instruction and in indirect ways (e.g. through dance and play). It reached the conclusion that the indirect methods showed no advantage over direct instruction as regards the promotion of cognitive development. However, the team remarked that the two types of techniques might possibly contribute in different measures to the development of the children's personalities in non-cognitive respects which had not been investigated in the studies under review (Israel, Ministry of Education, 1974). Hence it would be irresponsible to draw policy conclusions directly from these research findings. This problem becomes even more complicated by two additional factors. First of all, educational problems are multi-disciplinary in character because the variables involved in them (such as pupils' and teachers' attributes and behaviors, methods of instruction and curricular subject-matter, the social environment, etc.) are frequently investigated
220
K 1.t.INI'~IIR(;1i R
by separate scientific disciplines. Therefore, it is unreasonable to infer practical consequences immediately from the findings and theories of tmc single scientific discipline. Secondly, even within a given scientific discipline (e.g. psychology) rival theories and schools have developed, and each of them concentrates its attention one-sidedly on one particular aspect of the nmlti-dimensional human personality and mode of acting. Ira view of these facts, .Joseph Schwab in an essay about "the corruption of education by psychology", published ahnost twenty years ago, already warned that the factual and theoretical results of many psychological studies concerning education were nothing but "'partial truths", and that from partial truths no reliable practical conclusions could be drawn (Schwab, 1958). (b) The logical co~tsiderati(m is quite ancient and extremely simple, yet is nevertheless often disregarded both by scientists and by practitioners. It refers to the rule that in a logically valid inference one cannot derive as a conclusion more than has been given in the premises. In this respect a logical inference is analogous to a mincer: if at one end you put in merely bread and l\md coloring, it is impossible to get out at the other end meat balls. Similarly, from the propositions of an empirical science about what is the case, why it is the case, and under what conditions it may possibly or must necessarily be the case no policy conclusions about what ought to be the case or to be done could be validly derived. In order to draw logically valid inferences about measures which ought to be taken, we need as additional premises propositions or judgments about values and aims which empirical research is not competent to provide. This consideration shall be illustrated by the empirical curriculum research which ',,,'as undertaken in Sweden upon the initiative of the 1957 School ('ommittee. The fundamental idea of these investigations was to examine the relevance which the items of knowledge and skills included in the syllabi for mathematics and the mother language had for the demands posed by vocational education and by the world of work, especially in industrial and lower "white collar" occupations. On account of the measuring instruments and teclmiques at their disposal, the investigators limited their attention to the cognitive effects of instruction in these two school subjects, and did not deal with its non-cognitive results in the form of attitudes, interests and values. Data were obtained with regard to three sets of facts: (I) the items of knowledge and skills which were required in various courses of vocational education and in different occupational careers; (2) the actual use made in these areas of the items of knowledge and the skills included in the existing syllabi; and (3) the differential levels of their mastery several years after the termination, t)f schooling. From the findings of these empirical investigations the researchers drew "'implications" regarding the relative priority of different classes of topics which ought to be the basis for constructing the curricula ira the two subjects within the framework of universal compulsory schooling. Thus, for instance, they identified topics which ought to be included in the common core mathematics course -- e.g., the four basic operations with integers, decimal fractions, measurements in the decimal system, percentages, interest, and averages. By contrast, topics with low usability for the maiority of occupational training courses and careers, such as algebra and formal demonstrative geometry, ought to be omitted from the obligatory programme in mathematics. According to the same principle, they recommended to give ira the curriculum for the mother language greater weight to non-fictional prose, in contrast with fiction and poetry which had occupied a dominant position in conventional syllabi (Husen & Boalt, 1967).
RESI-ARCH AND POLICY-MAKING
221
It is obvious that these practical recommendations as to what ought to be done and changed in teaching do not follow from the empirical research findings, unless two value judgments are presupposed wtuch by no means can be accepted as self-evident nor have been established by the investigations under review but which have been underlying them (though only implicitly). They are (1) that the school ought primarily to impart knowledge and skills which are useful for the world of occupations, and (2~ that a human being needs neither a widening of his intellectual horizon nor familiarity with either the basic forms of knowledge and understanding or the creations of the human mind, unless these are usable and relevant for his personal and vocational needs. Sometimes the extra-scientific value premisses have been built into the empirical research design itself. Richard Atkinson from Stanford University has developed a model for the analysis of instruction which is based upon decision theory, and has the major advantage that it takes into account the relationship between the cost of a treatment and its effects which has been disregarded by the usual kind of evaluation studies in education. With the aid of this model he has examined the optimal use to be made of a computer in initial reading instruction. For this purpose he required a specification of the teaching objective which could serve as a criterion for the optimalization of use. He had a choice between several possibilities, for instance: (a)to maximize the average test achievement for the class as a whole; (b)to reduce as far as possible the variance in test achievements among the pupils of the class: (c)to maximize the average test achievement, with the provision that the resulting variance does not exceed the measure which would have been obtained without the use of a computer: Id)to maximize the number of pupils who achieve the norm of their grade. It was obvious to him that these different objectives implied different strategies o~ optimalization for the use of the computer, llowever, what apparently escaped his notice was that these different conceptions of the teaching objective represent contradictory social philosophies and value attitudes between which no empirical research, not even one based on decision theory, is competent to decide (Atkinson, 1973). In view of this logical consideration, it is not surprising that different persons can "conclude" contradictory policies from the same research findings. This shall again be illustrated by an example taken from Sweden. Upon the evidence given by a number of psychologists and as a result of investigations that have been commissioned by several School Committees, it was accepted as a scientifically valid finding that success in academic-theoretical studies was primarily dependent upon, and predictable by, general intelligence which could be determined with adequate certainty at the age of 11, whereas specific vocational aptitudes and interests could not be diagnosed before the age of 14. Part of the members of the 1940 School Committe, which consisted mainly of teachers of all levels of education, many of whom held somewhat conservative views, "concluded" from this finding that intellectualy gifted pupils ought to transfer at the age of 11, after completing the fourth grade, to a selective secondary school (the realskola). By contrast, the 1946 School Commission, which consisted of members of Parliament with rather radical opinions, "inferred" from the same finding that an early transfer of pupils with high general intelligence to selective secondary schools was unjustified because in that way the practical and technical occupations would be deprived of intellectually gifted pupils, and in consequence were going to suffer a
222
K LEINB I:R(;I. R
dangerous loss of prestige; t\)r this reason the Commission recommended a later organizational differentiation in the school system (Husen & Boalt, 1967). We may sum up the logical consideration with Boyd H. Bode's well-known saying: "Statistics can inform us how many policemen and how many thieves exist in a given society, llowever, they cannot tell use whether there ought to be more policemen or more thieves". (c) The sock)logical consideration points to the fact that actually the relationship between research and policy-making is frequently contrary to the model according to which the former directs and steers the laner. In other words, the policy decisions of the political estabfishment and the accepted social nomls underlying them determine to a considerable extent tire directions taken by social and educational research, the questions it raises, and those which it fails to deal with or even to ask. 1! is impossible to go here into the methods and mechanisms of exercising financial and poqtical power by means of which the political establishment influences tile directions of social research. But the fact of this influence is beyond doubt. It becomes manifest primarily in sudden and extreme changes in tile fashions of educational research which occur in the wake of new policy trends on the part of the political establishment. A dramatic example has been provided by the "'shock" caused by the first "Sputnik" in the western world and especially in the United States. Under its inflt, ence the educational policy of those countries was directed towards the qualitative and quantitative improvement of the training of natural scientists and technt)logists. This reversal in educational policy brought in its wake decisive changes in the directions and fashions of educational research. In comparative education a series of investigations about the education of scientists and technologists in the Soviet Union was started, followed by the series of studies analyzing the preparation ot" scientific and engineering manpower in the OECI) member countries. An intensive and comprehensive activity began with a view to developing new mathematics and science curricula the common approach of which consisted in teaching these subjects by way of inquiry, so that the pupils should learn how to " d o " mathematics, physics, etc. in the manner in which they are pursued by mathematicians and scientists (Young, lt)73). A new fashion of dealing with "creativity" sprang up, both by developing new kinds of tests for measuring its abilities and predicting its chances, and by investigating methods of its nurture, while in the background there was the need to identify future "'creative" scientists and inventors, and to actualize their promising potentialities (Young, 1973). Further examples can be touched here only briefly: investigations of foreign language instruction, when tile gove,-nment of the United States recognized an increased need for personnel mastering foreign languages both for its intelligence and espionnage services and for technical aid to developing countries; the discovery of education by economic research as a profitable investment in human capital, when it became obvious that success in international economic competition was dependent on tile quality and quantity of skilled manpower (Wall & Husen, 1968); and finally the fact that the National Foundation for Educational Research in England and Wales which has been founded after the promulgation of the English Education Act of 1944 devoted during the first decade of its existence nearly all its resources to research on the allocation of primary school leavers to suitable types of secondary schools and on selection for academic secondary schools (grammar schools) that is to say, to the study t)f problems with which that Act had confronted the schools and local education authorities (Yates, 1971 ).
RESEARCH AND POLICY-MAKING
223
The last example reflects already a deeper sense of the sociological consideration. Not only the directions of research are largely determined by the decisions of the political establishment, but empirical investigators frequently tend to accept as natural and self-evident the status quo and the unexamined normative presuppositions of current educational practices and policies. Characteristic in this respect is the extensive research effort that has been devoted in a number of countries to the inequality of educational opportunities between socio-economic classes, races, and socio-cultural groups, and to the problems of cultural deprivation or environmental disadvantage. This research has focused on the social characteristics of the school failures and drop-outs and on the correlation between these characteristics and the differential rates of success and failure in formal education, while at the same time accepting as natural and selfevident the present curricula and their organization and structure, the usual procedures and methods of teaching, as well as the kinds of abilities and aptitudes which are fostered and promoted by traditional schooling, and which are required for success in it - in short, the entire system of recognized norms and criteria which determine who is considered a success and who a failure. In-consequence, failure in school appears, in relation to these norms and criteria, as a sort of deviance or misadaptation (Young, 1973; Glaser, 1973; Israel, Ministry of Education, 1974). Further examples can only be touched here briefly: validation of intelligence and ability tests by the criterion of success with the modes of teaching and learning of the existing school, and their utilization for the selection of pupils who are suitable for an accepted and immutably fixed educational environment (Cronbach & Suppes, 1969; Glaser, 1973); the comparison between the achievements of comprehensive shools in Sweden and England and of selective secondary schools in the traditional school system by the conventional criteria of success (Husen & Boalt, 1967; Wall & Husen, 1968); investigations of drop-outs which are based on an uncritical acceptance of the dominant assumption that an extended period of school attendance is beneficial also for those adolescents who are not interested in it and do not enjoy it. In short, educational research tends to ask how the existing things are being done, how they can be done better and more effectively, and how the educational achievements can be improved with reference to the currently prescribed aims of schooling. It accepts what is usual and existing as if it were a law of nature. As a rule, however, it refrains from asking heretical and unorthodox questions with regard to the dominant conceptions and assumptions, and from examining radical alternatives to the status quo (Brunet, 1975; Rowher, 1973; Young, 1965; cf. also Argyris, 1969; Kelley, 1971). If the conception that research is competent to steer educational policy-making, must, then, be rejected, the question arises: what can educational research contribute to an improved and more rational educational policy? Within the limits of a short essay, one can merely attempt to answer this question with three brief hints. (1) Research can create a more secure and reliable foundation for decisions concerning the introduction of reforms and innovations by evaluating their relative effectiveness. Of course, the findings of such evaluative studies can never provide the sole conclusive consideration for those kinds of decisions,and all the reservations which have been raised above in my methodological consideration are also valid in this context for the potential contribution of research. Here a further reservation shall be raised against a certain kind of research which I shall call "fetishistic" investigations of "fetishistic" policies. "Fetishism" in Freud's
224
K Lt.I NIJt- P,(;I.R
terminology means that sexual satisfaction is not sought in the genitals, but through other parts of the body or through objects which normally serve only as means for the arousal of sexual passion, but not as tile goal of the latter. In analogy, fetishistic policies and the fetishistic research subserving them deal only with external frameworks and arrangements which are destined to make certain improved ways and modes of treatment possible, but not with tile treatments themselves which represent their purpose. I have in mind evaluative studies about the effectiveness of such arrangements as educational institutions for tile preschool age, provision of increased financial and other resources for the schools, reduction of class size. homogenous ability grouping, an extended school day, observation cycles, comprehensive schools, etc. As is well known, evaluative studies conducted in many countries have as a rule failed to discover any significant differences in school achievements between the childrexl who have enjoyed the benefit of such or similar arrangements and those who have not (Aver, (I et al.. 1972; Elvin, 1975; Husen & Boalt, 1967; Israel, Ministry of Education, 1974: Rossi & Williams, 1972: Smith & James, 1975; Tizard, 1975: Wall & Husen, 1968). Why shotdd that come as a surprise? These investigations have not examined what was happening within tile frameworks in question, or how the arrangements under review were being utilized, and whether c)r not any innovative or improved treatments which it had been their function to make possible had been introduced at all. (Indeed those who make the educational decisions and who carry them out have also not infrequently been satisfied with making available the external frameworks and arrangements without providing for the improved treatments). Imagine that in some courltry multi-lane highways have been built which are devised to make possible faster and safer driving than on the old roads. Yet the traffic on the new super highways proceeds with the same mule carts as was the case on the old dust roads. If in that country azl evaluative study after the model of the afore-mentioned educational investigations is undertaken about the relative effectiveness of the new highways, it, it)o, will come to the conclusion that no significant differences exist with regard to the speed and the safety of transportation between these two kinds of roads. (2) On the basis of their research, investigators can suggest to the educational decision-makers more pertinent and fruitful questions than tho3e they had been used to ask hitherto, and thus turn educational research, too, towards new and more promising lines of inquiry. Only a few examples of what may be termed with a Platonic expression the "turning of the soul" can be mentioned here very briefly. Instead of asking at which age it would be advisable to start with teaching a first foreign language, educational research has raised the question, by means of which methods of t\)reign language instruction is it possible to achieve success already with younger children? (Wall. & lhisen. 1968). Instead of looking for improved tests for diagnosing general ability which are capable of predicting success in fixed courses of study that require only certain kinds of capacity, and of selecting a relatively small number of candidates who are suited for these courses of study, some psychologists (among them Lee Cronbach) have recommended asking a new question: How is it possible to develop differentiated tests which can help us to choose for each individual a teaching method and learning style which are suited t() his particular cognitive style and to his individual aptitudes, inclinations and interests? (Cronbach & Suppes, 1969: Elvin, 1975).
RESEARCH AND POLICY-MAKING
225
Economic theory has drawn our attention to the costs of education not merely in terms of money and manpower which are directly invested in it, but also in terms of income which the young persons forgo while attending educational institutions, and in terms of their non-participation in the labour force and creation of the national product. Framing the question about the costs of education from this novel viewpoint may perhaps encourage a search for ways to a more economical utilization of the pupils' time and of shortening the duration of school attendance (Cronbach & Suppes, 1969). (3) The most important potential contribution of research to the improvement of educational practice lies in the production of revolutionary ideas which can serve as foundations for the invention of innovative procedures, models, and teclmologies. There is, however, little hope that it will produce such fertile new ideas as long as educational research is required to deal only with current practical concerns and to achieve specific results which can be immediately translated into useful solutions. D.E. Berlyne has illustrated this by a striking example: if around the year 1800 somebody had been requested to conduct research with a view to improving transportation, he would probably have sought ways of breeding a faster running race of horses or of installing more elastic springs in coaches. However, the research which in fact led ultimately to a revolutionary progress in transportation did not deal with horses and coaches. An Englishman named James Watt was occupied with steam engines for pumping water from mines, and an Italian named Galvani detected a strange phenomenon in frogs' legs. Nobody at that time could have imagined that the investigation of frogs' legs -- and not of horses' legs - was destined to bring about a revolution in transportation (Yates, 1971 ). The moral of this story is that our best hope rests in encouraging basic research - or in the terminology preferred by Cronbach and Suppes, conclusion-oriented in contrast with decision-oriented inquiry (Cronbach & Suppes, 1969). That means that scientists who propose research projects that appear well-founded and promising are granted the freedom to follow their own ideas and to frame their own questions, and that they are not invariably and exclussively harnessed to the immediate practical exigencies of educational practitioners, nor required to deliver within a specified time specific results which can be immediately translated into usable practical solutions. That, of course, involves a risk. Nobody including the investigator himself - can possibly foresee which direction of basic research not oriented towards immediate needs, may yield ideas that will ultimately turn out to be fertile ground for revolutionary innovations in education. But without the readiness to run risks, there can be no chance that perhaps one in a hundred inquiries and investigations may prove pregnant with consequences. "Cast thy bread upon the waters, for thou shall find it after many days." (Ecclesiastes XI, 1)
REFERENCES
ARGYRIS, C. The Incompleteness of Social-Psychological Theory. American Psychoh>gist, 24, 1969, pp. 893-908. ATKINSON, R.C. Ingredients for a Theory of Instruction. In Wittrock, M.('. (ed.), ChaJlging Education. Alternatives J~'om Educational Research. Englewood Cliffs, N.Y.: Prentice Hall, 1973, pp. 6 5 - 8 2 . AVERCH, H.A. et al. How Effective is Schooling? A Critical Review and Synthesis o f Research Findings. Santa Monica, Cal.: Rand Corporation, 1972.
22{,
K I,I. INBI!R(;I. R
BECKER. H. Beitrag und Einfluss dcr Bildungsforschung auf die Arbeit tics "'l)culschcn Bildungsrats'" Zeitschrift fiir Pddagogik, 2 I. 1975. pp. 15'4 172. B R U N E R , J. The Role of the Researcher us an Adviser Io the Educational Policy Maker. Oxfi)rd R e v i e w o.! Education. 1, 1975. pp. 183 188 (_?LARK. D.L. and (;UBA. t-.I;. ,4n Examination ~),I Potential ('hangc Roles m l:ducation. P~,per presented to the N E A - - ( ' S I Seminar on Innovation in Planning School (urricula. Aerlie Ilouse, Virginia. l t~()5. ('OHEN. D.K. and (;ARI-T. M.S. Reforming Educali