- Email: [email protected]
Towards an evaluation of a business school graduate programme
OMEGA,
The Int. JI of MgmtSci., Vol. 3, No. 4, 1975
Towards an Evaluation of a Business School Graduate Programme JG BURGOYNE University of Lancaster x (Recei~d Ncnmvd~,r 1974; ~ r~Jaed form January 1975)
An evaluation research methodology, which has been validated in a previous study, and which was developed to be both scientifically defensible and feasible from a practical point of view, is described. This involves collecting course participants' judgements about the effects of the course, and taking steps to ensure that the process by which the participants arrive at these judgements is sound. The application of this methodology to a course in business administration revealed the following effects or consequences: (1) Access to and progress in succe~ful careers; (2) Flexibility and mobility in careen; (3) Ability to take overall view of problems; (4) Personal confidence; (5) Awareness of own aspirations and care~ fit; (6) Frustration from unused skills and unfulfilled expectations; (7) High salaries; (8) General reasoning and problem solving ability; (9) Economic/commercial understanding; (10) Understanding of organisation and human behaviour; (I1) Useful work, care~" and pexsonal contacts; (12) Social skills.
W H I L E debate a b o u t the value o f Business Schools and their p r o g r a m m e s continues, and decisions on their f o r m and use are still made on the same basis o f ignorance as they always were, the cry for systematic evaluation o f such programmes seems to have diminished. In the latter 1960's, when sophisticated new m a n a g e m e n t development p r o g r a m m e s were launched as a result o f the establishment o f Business Schools, the Training A c t and other forces, the relevant institutions were keen to practise what they preached by evaluating their activities t h r o u g h applied scientific research. Since then interest in evaluation research has diminished, not because it has not been done, but because it appears, in some hard to define way, to have failed to live up to the expectations that were held o f it in terms o f guiding action and vindicating m a n a g e m e n t education. Part o f this problem is due to two unrealistic expectations o f evaluation research: firstly that one project will answer all questions rather than a specific one or two to which it is addressed, and secondly that such research will reveal nothing less than the ultimate ends to which individuals, organisations and society should aspire, rather than some evidence on goal achievement. A n o t h e r I The research reported was carried out while I was lecturer in Management Development, at the Manchester Business School, U.K. 475
Burgoyne--Evaluation of a Graduate Programme aspect of the problem has been the propensity to carry out what Rackham has called 'archaeological' evaluation (evidence about past events with very little relevance for future decisions) rather than evaluation designed to provide feedback in a control process (Hamblin gives a comprehensive review of current evaluation research and methodology from this perspective [5]). A third factor appears to have been a tendency on the part of people with theoretical and practical interest to become preoccupied with critical analysis of evaluation methodology rather than actually carrying out any research. This seems to be part of a wide phenomenon to be observed in many disciplines: the refinement of methodology is seen as a higher status activity than actual investigation of the substantive problems of the disciplines, and the latter become neglected. Obviously methodological issues are important, and methodological decisions are prior to investigation, but surely methodologies must be judged ultimately by their usefulness in tackling the focal problems. Presumably the astronomer who becomes more interested in his telescope than the stars becomes a technician. The effect of this process in the field of evaluation research has been to create a gulf between unrealistic discussion of methodology and actual practical attempts to evaluate programmes. Thus Campbell and Stanley published an elaborate analysis of a variety of experimental and other research designs [4], and Andrews, faced with the challenge of evaluating Harvard Business School post-experience programmes, and having more time and resources available than is usually the ease in evaluation research, goes for a one-off follow-up survey of participants' opinions of the programmes and their usefulness [1]. The aim of this paper is to give the results of some actual evaluation research carried out with a new 'intermediate' methodology which seeks to be both practicable and to embody as many as possible of the features that methodological analysis suggests give validity to results.
METHODOLOGY The methodology used was essentially a refinement of the one-off, opinion collecting follow up approach. The key idea for the refinement came from Campbell et al [3], who, in discussing Andrews" study (op. cit.) wrote: " . . . it is almost implicit in Andrews' argument that if a development programme is evaluated by asking trainees, 'In what ways did it help?' then the linkage of the training activity to the organisation's goals, the comparison of before and after states, the comparison with a placebo, and the examination of interaction between training effects and other organizational forces all must go on within the individual before he renders his opinion. In other words the individual trainee himself must act as his own 476
Omega, Vol. 3, No. 4 before and after measure, control group and systems analyst. These operations must be performed by the individual trainee, or his opinion cannot be considered in any sense a valid means of evaluation." This argument suggests that follow-up opinion surveys give valid results to the extent to which the judgement and decision making processes underlying the formulation of the opinions embody the validity-giving features of scientific research. A preliminary investigation (Burgoyne [2]), using protocol analysis methodology (having respondents "think aloud" as they make a decision or judgement), has produced some evidence that such features frequently do and certainly can, underlie follow-up opinion. This study has been carried out by collecting opinions in such a way that respondents base them on valid judgement, and make the main features of those judgement processes explicit. 48 past members of the Manchester Business School Graduate Course were followed up, 35 of these were visited and interviewed, and in the light of this a carefully designed self-administering questionnaire was designed and used with the remaining 13 respondents. Essentially the respondents were asked in all cases to produce a list of what they saw as having been the significant consequences, if any, of their attending the Course. In forming a view on what the significant effects were they were required to consider and report what had actually changed, what their evidence for this was, and on what basis they felt this was a causal consequence of the Course.
RESULTS As an evaluation study, the aim of this research was to make a start at identifying some of the significant effects and consequences of the Graduate Course. This is the Schoors main post graduate course for people with only a few years or no management experience, and a previous degree or professional qualification. Members stay in the Course one year for a diploma or two years for an MBA, 31 of the follow-up sample had diplomas and the remaining 17 MBA's. The study is concerned with formulating an empirically based list of consequences for discussion and further investigation, rather than testing specific hypotheses. No claim is made that the sample is representative of the total population of about 250 from which it comes. The analysis is primarily inductive, and no attempt has been made to break the sample down according to type of person, job, career or other typology. The results are presented in the form of a list of the twelve significant Course consequences which were reported by five or more of the respondents. The frequencies with which the consequences were mentioned are shown, and these are intended as rough indices only, and not as the highly accurate measures that the use of numbers sometimes suggests. Table 1 shows the 12 consequences, with 477
Burgoyne~Evaluation of a Graduate Programme TABLE1. I~PORZ~ COtn~S~CONSEOV~CeS Rank 1 2 3 4
5,= 5-= 7 8 9= 9-----
9~ 12
Consequences
Access to, and progress in, successful career Flexibility and mobility in career Overallview of problems Personal confidence Awarenessof own aspirations and career fit Frustration from unsatisfied expectations, and unused skills Salary General reasoning and problem solving Economic/commercial understanding Understanding of organisation and human bchaviour Contacts
Social skills
Frequencies 22 20 17 15 12 12 I0 8
6 6 6 5
their frequencies, in rank order. The items in the list are discussed in the remaining text to clarify their meaning and to report some of the relevant impressionistic data coming from the interviews. The two most frequently mentioned effects were both concerned with careers. Access to and progress in successful career refers to getting a good first job and progressing to better ones, in terms of conventional criteria of success such as status, seniority, authority. Flexibility and mobility in career refers to the effect of the Course in widening the variety o f jobs and careers open to individuals. Course members found that because they had an all purpose qualification they were not tied to a particular function, like management accounting, or technical specialism like chemical engineering, or type of industry, like the building trade. The respondents made these judgements with confidence, because they could point to comparable people who had not been to the Business School and did not have these advantages. One issue that many of the respondents raised was the extent to which these effects resulted simply from being known to be a Business School Graduate rather than any actual educational benefit from being at the School. The general conclusion was that the 'certification' or 'ticket' effect certainly had played a part, particularly in getting first jobs, and being at least considered for a wide variety of posts. However the participants felt that having been given the opportunities on the grounds o f the qualification they had then had to prove themselves on the basis o f their skills. Furthermore many of the respondents point out that in the longer term the credibility of the qualifications as indicators of management potential was based on the effectiveness of themselves and their predecessors. The third most frequently mentioned effect was the ability to take an Overall view of problems. Respondents found themselves different from their nonBusiness School contemporaries in that the latter took comparatively parochial views of problems, seeing them only from the perspective of the accountant, the 478
Omega, VoL 3, No. 4 marketing department, or the technical specialist. The Business School Graduates found themselves able to consider these problems from all these perspectives at once and play a major part in helping groups of specialists work constructively together. The fourth effect, Personal confidence, was by its nature a 'subjective' and 'internal' state and difficult to prove with objective evidence. Respondents were however able to indicate situations and tasks that they would now go into and take on, which they felt they would not have dared to do had they not been on the Course. These were particularly situations of dealing with senior management, complex problems, specialists and customers. Much of this confidence was based on the other Course effects, like being able to take an overall view, specific relevant skills and feeling in a position of strength in career terms. However an equally important basis for confidence was a "demystification' effect of the Course. This came from learning, for example, that senior management do not have all the answers and often have even less information than oneself, that there really is not much o f substance behind the jargon of many of the management specialisms, and what there is can be questioned. Awareness of own aspirations and career fit was seen as a consequence by a number of the respondents who felt that attending the Course had given them the time for, and some help in, clarifying their own life and career aspirations. The Course, and the career opening which it gave access to, had provided a vehicle for realising these aspirations. Contrasting themselves with non-Business School contemporaries, the respondents felt that because management covers a very diverse set of activities, and because there are many different relevant areas of knowledge and skill which can be developed and made the basis o f a successful career, they had been able to build on their own particular strengths and interests. Had they not been to a Business School they would have been likely to have developed, either through formal education or on the job experience, a much narrower and specialist set of skills, chosen at a relatively young age which would have been less likely to turn out to relate to enduring life interest and give access to a satisfying career. Frustration from unused skills and unfulfilled expectations was the only 'negative" effect mentioned with any frequency (respondents were asked for every effect they perceived. "Overall, does this suggest that attending the Course was the right decision?"). Frustration being another essentially subjective and internal phenomenon, it was not possible to produce concrete examples of frustration itself. However the situations which were described as frustrating tended to be those in which the person was denied the right to influence decisions that he felt he had been trained to help with. Salary, meaning high relative salary, was a consequence which several respondents felt to be important and substantial in relation to those of nonBusiness Graduates. General reasoning andproblem solving in the business context referred to skills 479
Burgoyne--Evaluation of a Graduate Programme in thinking about and working on complex managerial problems and decisions. The respondents pointed to the fact that they were able to spend some time analysing and thinking about problems, making some necessary assumptions and simplifications, in a way that gave them some kind of a basis for recommending and defending a course of action. They found that their colleagues with scientific or specialist backgrounds seemed to find their models and decision making techniques too simple to cope with real managerial problems, were unwilling to work with rough approximations in order to make some kind of decision and always wanted to wait for the results of further data collection and analysis. Other colleagues more experienced in making managerial decisions tended to do so on the basis of virtually no analysis, and to be unduly influenced by contemporary pressure and events in very recent history. Economic/commercial understanding referred to a general skill in seeing the financial, profitability and competitive advantage implications of problems and alternatives from the organisation's point of view. Particular instances of the exercise of this skill included knowing where to look for and how to interpret financial information and economic trends, how to analyse this information, and present an argument in these terms for a course of action. Understanding of organixation and human behaviour covered the spectrum from having an idea of the reason for certain instances of individual and small group behaviour, and being able to respond appropriately, to being able to analyse and recommend solutions to problems of overall organisation structure. Contacts referred to the establishment of a network of acquaintances of actual or potential usefulness in work or career terms. Contacts included colleagues on the Course, members of other courses in the School at the time, members of the teaching and research staff, people in other organisations where project work was carried out during the Course, and recruiters seen at the job hunting stage. Contacts were useful in a great variety of ways, including direct commercial ones, as sources of information, for advice on problems, for access to career opportunities, for a yardstick against which to measure career progress, and for emotional support in times of difficulty. Lastly social skills referred to immediate face-to-face performance in getting information, communicating, bargaining and negotiating, gaining co-operation and making presentations. The respondents felt that they were at an advantage, in terms of these skills, compared with non-Business School colleagues, and that this could be traced directly back to the fact that the Course itself gave opportunities to develop and exercise these skills.
DISCUSSION These results suggest some conclusions about the state of post graduate management education, raise some questions and throw a little light on some of 480
, Omega, Vol. 3, No. 4
the issues frequently debated concerning Business Schools. Since these results are based on the study of only one course, generalisations from them must be approached with extreme caution. Some such implications, which must be considered with this qualification in mind, are set out below: The Course in relation to its environment (a) It is clear that, despite all the criticism of Business School Courses and their alumni, this course is a well established part of a successful career pattern for young men and women entering management with little or no previous work experience. This suggests that despite the sometimes acrimonious discussion of the failings of Business Schools as educators, and of those who recruit their products as employers, the Schools' placement systems and the employers' systems for recruitment and development, are both operating in a reasonably effective and integrated way.
(b) A major aspect of this Course as an effective part in successful managerial careers seems to be its effect in broadening the range of problems that its members are, and are seen to be, capable of coping with. In this respect the Course seems to serve a happy coincidence of interest between members who want to be, and employers who want to have, generalists. It is possible that Business Schools could make more of this effect in attracting and selecting bright and frustrated new members from narrower specialisms, and finding them new career starting points after the Course. (c) A third aspect of the place of this Course in the context of careers is the role it appears to be playing in helping members clarify their career aims and aspirations and work towards satisfying these within the constraints of their abilities and the opportunities open to them. Most discussions of post graduate management education assumes that all the students are or should be strongly committed to a career in management, or even to a specific type of career. The reality is that many members come to the comse with the objective, at least in part, of clarifying their career aims. It may be that the breadth of the course, and the variety of careers to which it is an entry, suits it for this purpose. The question which this raises is whether this should not be a legitimate aim for post graduate management education, and, in so far as this is not already done, allowed for in the design of the course. (d) Despite the evidence of successful careers compatible with career aspirations, the familiar reports of frustration derived from unused skills and unsatisfied aspirations still emerge. Although this frustration seems real, these results, taken as a whole, suggest the hypothesis that this may be the natural sign of effective young managers seeking to extend their influence as far as possible rather than as evidence of severe disfunctional imbalance between Business School alumni and their employers. 481
Burgoyne--Evaluation of a Graduate Programme Course design (a) Although the list of consequences of the Course bears very little immediate resemblance to any list of subject areas or topics that might describe its content, there is some evidence of appropriateness. Firstly the breadth and generality of the problems and situations which the alumni find themselves able to cope with must be evidence for the value of the equally broad array of subjects, topics, disciplines and approaches from which the Course content is drawn. Secondly, all of the main areas of content, the behavioural and social sciences, those concerned with money and financial matters and those concerned with quantitative and qualitative decision making, can be seen as contributing to the reported benefits. Finally, the evidence for personal adjustments and successful behaviour in practice strongly suggests that the aspects of the Course that go beyond cognitive, abstract and intellectual learning into areas of action skills, self awareness and practical competence have a valuable effect. (b) A number of the effects suggest that the value of the Course goes beyond suitable content into suitable overall design. The nature of the various benefits suggests that the elements of content are to a reasonable degree integrated in an effective educational design for the Course as a whole.
CONCLUSION The intermediate evaluation methodology of asking past members what effects the course has had, encouraging them consider and make explicit the evidence for the effect and its causal relation with the Course, has provided an empirically based list of Course effects. This list may serve both as a framework for further research, and as a basis for more informed discussions of management education.
ACKNOWLEDGEMENTS I am grateful to Tony Clark, Nell Fairhead, Michael Kightley, Crispin Raymond, Rod. Reeves, Alan Sykes and Richard Townsend, members of the 1973-1974 Manchester Business School Graduate Course first year who carried out many of the interviews on which this paper is based.
REFERENCES 1. ANDREWSK R (1966) The Effectiveness of University Mana~,ement Development Programmes. Harvard University Graduate School of Business Administration. Boston. 2. BURGOYNEJG (1973) The judgement process in management students" evaluation of their learning experiences. Unpublished PhD Thesis, to appear in Hum. Relations. 3. CAMPaELLJP et ai (1970) Managerial Behaviour, Performance and Effectiveness. McGrawHill. 4. CAMPaVJ.L DT and STAN~Y JC (1964) Experimental and quasi-experimental design for research in training. In Handbook of Education Research. Rand McNally, Chicago. 5. HAMELINAC (1974) Evaluation and Control of Training McGraw-Hill, London.
482
The Int. JI of MgmtSci., Vol. 3, No. 4, 1975
Towards an Evaluation of a Business School Graduate Programme JG BURGOYNE University of Lancaster x (Recei~d Ncnmvd~,r 1974; ~ r~Jaed form January 1975)
An evaluation research methodology, which has been validated in a previous study, and which was developed to be both scientifically defensible and feasible from a practical point of view, is described. This involves collecting course participants' judgements about the effects of the course, and taking steps to ensure that the process by which the participants arrive at these judgements is sound. The application of this methodology to a course in business administration revealed the following effects or consequences: (1) Access to and progress in succe~ful careers; (2) Flexibility and mobility in careen; (3) Ability to take overall view of problems; (4) Personal confidence; (5) Awareness of own aspirations and care~ fit; (6) Frustration from unused skills and unfulfilled expectations; (7) High salaries; (8) General reasoning and problem solving ability; (9) Economic/commercial understanding; (10) Understanding of organisation and human behaviour; (I1) Useful work, care~" and pexsonal contacts; (12) Social skills.
W H I L E debate a b o u t the value o f Business Schools and their p r o g r a m m e s continues, and decisions on their f o r m and use are still made on the same basis o f ignorance as they always were, the cry for systematic evaluation o f such programmes seems to have diminished. In the latter 1960's, when sophisticated new m a n a g e m e n t development p r o g r a m m e s were launched as a result o f the establishment o f Business Schools, the Training A c t and other forces, the relevant institutions were keen to practise what they preached by evaluating their activities t h r o u g h applied scientific research. Since then interest in evaluation research has diminished, not because it has not been done, but because it appears, in some hard to define way, to have failed to live up to the expectations that were held o f it in terms o f guiding action and vindicating m a n a g e m e n t education. Part o f this problem is due to two unrealistic expectations o f evaluation research: firstly that one project will answer all questions rather than a specific one or two to which it is addressed, and secondly that such research will reveal nothing less than the ultimate ends to which individuals, organisations and society should aspire, rather than some evidence on goal achievement. A n o t h e r I The research reported was carried out while I was lecturer in Management Development, at the Manchester Business School, U.K. 475
Burgoyne--Evaluation of a Graduate Programme aspect of the problem has been the propensity to carry out what Rackham has called 'archaeological' evaluation (evidence about past events with very little relevance for future decisions) rather than evaluation designed to provide feedback in a control process (Hamblin gives a comprehensive review of current evaluation research and methodology from this perspective [5]). A third factor appears to have been a tendency on the part of people with theoretical and practical interest to become preoccupied with critical analysis of evaluation methodology rather than actually carrying out any research. This seems to be part of a wide phenomenon to be observed in many disciplines: the refinement of methodology is seen as a higher status activity than actual investigation of the substantive problems of the disciplines, and the latter become neglected. Obviously methodological issues are important, and methodological decisions are prior to investigation, but surely methodologies must be judged ultimately by their usefulness in tackling the focal problems. Presumably the astronomer who becomes more interested in his telescope than the stars becomes a technician. The effect of this process in the field of evaluation research has been to create a gulf between unrealistic discussion of methodology and actual practical attempts to evaluate programmes. Thus Campbell and Stanley published an elaborate analysis of a variety of experimental and other research designs [4], and Andrews, faced with the challenge of evaluating Harvard Business School post-experience programmes, and having more time and resources available than is usually the ease in evaluation research, goes for a one-off follow-up survey of participants' opinions of the programmes and their usefulness [1]. The aim of this paper is to give the results of some actual evaluation research carried out with a new 'intermediate' methodology which seeks to be both practicable and to embody as many as possible of the features that methodological analysis suggests give validity to results.
METHODOLOGY The methodology used was essentially a refinement of the one-off, opinion collecting follow up approach. The key idea for the refinement came from Campbell et al [3], who, in discussing Andrews" study (op. cit.) wrote: " . . . it is almost implicit in Andrews' argument that if a development programme is evaluated by asking trainees, 'In what ways did it help?' then the linkage of the training activity to the organisation's goals, the comparison of before and after states, the comparison with a placebo, and the examination of interaction between training effects and other organizational forces all must go on within the individual before he renders his opinion. In other words the individual trainee himself must act as his own 476
Omega, Vol. 3, No. 4 before and after measure, control group and systems analyst. These operations must be performed by the individual trainee, or his opinion cannot be considered in any sense a valid means of evaluation." This argument suggests that follow-up opinion surveys give valid results to the extent to which the judgement and decision making processes underlying the formulation of the opinions embody the validity-giving features of scientific research. A preliminary investigation (Burgoyne [2]), using protocol analysis methodology (having respondents "think aloud" as they make a decision or judgement), has produced some evidence that such features frequently do and certainly can, underlie follow-up opinion. This study has been carried out by collecting opinions in such a way that respondents base them on valid judgement, and make the main features of those judgement processes explicit. 48 past members of the Manchester Business School Graduate Course were followed up, 35 of these were visited and interviewed, and in the light of this a carefully designed self-administering questionnaire was designed and used with the remaining 13 respondents. Essentially the respondents were asked in all cases to produce a list of what they saw as having been the significant consequences, if any, of their attending the Course. In forming a view on what the significant effects were they were required to consider and report what had actually changed, what their evidence for this was, and on what basis they felt this was a causal consequence of the Course.
RESULTS As an evaluation study, the aim of this research was to make a start at identifying some of the significant effects and consequences of the Graduate Course. This is the Schoors main post graduate course for people with only a few years or no management experience, and a previous degree or professional qualification. Members stay in the Course one year for a diploma or two years for an MBA, 31 of the follow-up sample had diplomas and the remaining 17 MBA's. The study is concerned with formulating an empirically based list of consequences for discussion and further investigation, rather than testing specific hypotheses. No claim is made that the sample is representative of the total population of about 250 from which it comes. The analysis is primarily inductive, and no attempt has been made to break the sample down according to type of person, job, career or other typology. The results are presented in the form of a list of the twelve significant Course consequences which were reported by five or more of the respondents. The frequencies with which the consequences were mentioned are shown, and these are intended as rough indices only, and not as the highly accurate measures that the use of numbers sometimes suggests. Table 1 shows the 12 consequences, with 477
Burgoyne~Evaluation of a Graduate Programme TABLE1. I~PORZ~ COtn~S~CONSEOV~CeS Rank 1 2 3 4
5,= 5-= 7 8 9= 9-----
9~ 12
Consequences
Access to, and progress in, successful career Flexibility and mobility in career Overallview of problems Personal confidence Awarenessof own aspirations and career fit Frustration from unsatisfied expectations, and unused skills Salary General reasoning and problem solving Economic/commercial understanding Understanding of organisation and human bchaviour Contacts
Social skills
Frequencies 22 20 17 15 12 12 I0 8
6 6 6 5
their frequencies, in rank order. The items in the list are discussed in the remaining text to clarify their meaning and to report some of the relevant impressionistic data coming from the interviews. The two most frequently mentioned effects were both concerned with careers. Access to and progress in successful career refers to getting a good first job and progressing to better ones, in terms of conventional criteria of success such as status, seniority, authority. Flexibility and mobility in career refers to the effect of the Course in widening the variety o f jobs and careers open to individuals. Course members found that because they had an all purpose qualification they were not tied to a particular function, like management accounting, or technical specialism like chemical engineering, or type of industry, like the building trade. The respondents made these judgements with confidence, because they could point to comparable people who had not been to the Business School and did not have these advantages. One issue that many of the respondents raised was the extent to which these effects resulted simply from being known to be a Business School Graduate rather than any actual educational benefit from being at the School. The general conclusion was that the 'certification' or 'ticket' effect certainly had played a part, particularly in getting first jobs, and being at least considered for a wide variety of posts. However the participants felt that having been given the opportunities on the grounds o f the qualification they had then had to prove themselves on the basis o f their skills. Furthermore many of the respondents point out that in the longer term the credibility of the qualifications as indicators of management potential was based on the effectiveness of themselves and their predecessors. The third most frequently mentioned effect was the ability to take an Overall view of problems. Respondents found themselves different from their nonBusiness School contemporaries in that the latter took comparatively parochial views of problems, seeing them only from the perspective of the accountant, the 478
Omega, VoL 3, No. 4 marketing department, or the technical specialist. The Business School Graduates found themselves able to consider these problems from all these perspectives at once and play a major part in helping groups of specialists work constructively together. The fourth effect, Personal confidence, was by its nature a 'subjective' and 'internal' state and difficult to prove with objective evidence. Respondents were however able to indicate situations and tasks that they would now go into and take on, which they felt they would not have dared to do had they not been on the Course. These were particularly situations of dealing with senior management, complex problems, specialists and customers. Much of this confidence was based on the other Course effects, like being able to take an overall view, specific relevant skills and feeling in a position of strength in career terms. However an equally important basis for confidence was a "demystification' effect of the Course. This came from learning, for example, that senior management do not have all the answers and often have even less information than oneself, that there really is not much o f substance behind the jargon of many of the management specialisms, and what there is can be questioned. Awareness of own aspirations and career fit was seen as a consequence by a number of the respondents who felt that attending the Course had given them the time for, and some help in, clarifying their own life and career aspirations. The Course, and the career opening which it gave access to, had provided a vehicle for realising these aspirations. Contrasting themselves with non-Business School contemporaries, the respondents felt that because management covers a very diverse set of activities, and because there are many different relevant areas of knowledge and skill which can be developed and made the basis o f a successful career, they had been able to build on their own particular strengths and interests. Had they not been to a Business School they would have been likely to have developed, either through formal education or on the job experience, a much narrower and specialist set of skills, chosen at a relatively young age which would have been less likely to turn out to relate to enduring life interest and give access to a satisfying career. Frustration from unused skills and unfulfilled expectations was the only 'negative" effect mentioned with any frequency (respondents were asked for every effect they perceived. "Overall, does this suggest that attending the Course was the right decision?"). Frustration being another essentially subjective and internal phenomenon, it was not possible to produce concrete examples of frustration itself. However the situations which were described as frustrating tended to be those in which the person was denied the right to influence decisions that he felt he had been trained to help with. Salary, meaning high relative salary, was a consequence which several respondents felt to be important and substantial in relation to those of nonBusiness Graduates. General reasoning andproblem solving in the business context referred to skills 479
Burgoyne--Evaluation of a Graduate Programme in thinking about and working on complex managerial problems and decisions. The respondents pointed to the fact that they were able to spend some time analysing and thinking about problems, making some necessary assumptions and simplifications, in a way that gave them some kind of a basis for recommending and defending a course of action. They found that their colleagues with scientific or specialist backgrounds seemed to find their models and decision making techniques too simple to cope with real managerial problems, were unwilling to work with rough approximations in order to make some kind of decision and always wanted to wait for the results of further data collection and analysis. Other colleagues more experienced in making managerial decisions tended to do so on the basis of virtually no analysis, and to be unduly influenced by contemporary pressure and events in very recent history. Economic/commercial understanding referred to a general skill in seeing the financial, profitability and competitive advantage implications of problems and alternatives from the organisation's point of view. Particular instances of the exercise of this skill included knowing where to look for and how to interpret financial information and economic trends, how to analyse this information, and present an argument in these terms for a course of action. Understanding of organixation and human behaviour covered the spectrum from having an idea of the reason for certain instances of individual and small group behaviour, and being able to respond appropriately, to being able to analyse and recommend solutions to problems of overall organisation structure. Contacts referred to the establishment of a network of acquaintances of actual or potential usefulness in work or career terms. Contacts included colleagues on the Course, members of other courses in the School at the time, members of the teaching and research staff, people in other organisations where project work was carried out during the Course, and recruiters seen at the job hunting stage. Contacts were useful in a great variety of ways, including direct commercial ones, as sources of information, for advice on problems, for access to career opportunities, for a yardstick against which to measure career progress, and for emotional support in times of difficulty. Lastly social skills referred to immediate face-to-face performance in getting information, communicating, bargaining and negotiating, gaining co-operation and making presentations. The respondents felt that they were at an advantage, in terms of these skills, compared with non-Business School colleagues, and that this could be traced directly back to the fact that the Course itself gave opportunities to develop and exercise these skills.
DISCUSSION These results suggest some conclusions about the state of post graduate management education, raise some questions and throw a little light on some of 480
, Omega, Vol. 3, No. 4
the issues frequently debated concerning Business Schools. Since these results are based on the study of only one course, generalisations from them must be approached with extreme caution. Some such implications, which must be considered with this qualification in mind, are set out below: The Course in relation to its environment (a) It is clear that, despite all the criticism of Business School Courses and their alumni, this course is a well established part of a successful career pattern for young men and women entering management with little or no previous work experience. This suggests that despite the sometimes acrimonious discussion of the failings of Business Schools as educators, and of those who recruit their products as employers, the Schools' placement systems and the employers' systems for recruitment and development, are both operating in a reasonably effective and integrated way.
(b) A major aspect of this Course as an effective part in successful managerial careers seems to be its effect in broadening the range of problems that its members are, and are seen to be, capable of coping with. In this respect the Course seems to serve a happy coincidence of interest between members who want to be, and employers who want to have, generalists. It is possible that Business Schools could make more of this effect in attracting and selecting bright and frustrated new members from narrower specialisms, and finding them new career starting points after the Course. (c) A third aspect of the place of this Course in the context of careers is the role it appears to be playing in helping members clarify their career aims and aspirations and work towards satisfying these within the constraints of their abilities and the opportunities open to them. Most discussions of post graduate management education assumes that all the students are or should be strongly committed to a career in management, or even to a specific type of career. The reality is that many members come to the comse with the objective, at least in part, of clarifying their career aims. It may be that the breadth of the course, and the variety of careers to which it is an entry, suits it for this purpose. The question which this raises is whether this should not be a legitimate aim for post graduate management education, and, in so far as this is not already done, allowed for in the design of the course. (d) Despite the evidence of successful careers compatible with career aspirations, the familiar reports of frustration derived from unused skills and unsatisfied aspirations still emerge. Although this frustration seems real, these results, taken as a whole, suggest the hypothesis that this may be the natural sign of effective young managers seeking to extend their influence as far as possible rather than as evidence of severe disfunctional imbalance between Business School alumni and their employers. 481
Burgoyne--Evaluation of a Graduate Programme Course design (a) Although the list of consequences of the Course bears very little immediate resemblance to any list of subject areas or topics that might describe its content, there is some evidence of appropriateness. Firstly the breadth and generality of the problems and situations which the alumni find themselves able to cope with must be evidence for the value of the equally broad array of subjects, topics, disciplines and approaches from which the Course content is drawn. Secondly, all of the main areas of content, the behavioural and social sciences, those concerned with money and financial matters and those concerned with quantitative and qualitative decision making, can be seen as contributing to the reported benefits. Finally, the evidence for personal adjustments and successful behaviour in practice strongly suggests that the aspects of the Course that go beyond cognitive, abstract and intellectual learning into areas of action skills, self awareness and practical competence have a valuable effect. (b) A number of the effects suggest that the value of the Course goes beyond suitable content into suitable overall design. The nature of the various benefits suggests that the elements of content are to a reasonable degree integrated in an effective educational design for the Course as a whole.
CONCLUSION The intermediate evaluation methodology of asking past members what effects the course has had, encouraging them consider and make explicit the evidence for the effect and its causal relation with the Course, has provided an empirically based list of Course effects. This list may serve both as a framework for further research, and as a basis for more informed discussions of management education.
ACKNOWLEDGEMENTS I am grateful to Tony Clark, Nell Fairhead, Michael Kightley, Crispin Raymond, Rod. Reeves, Alan Sykes and Richard Townsend, members of the 1973-1974 Manchester Business School Graduate Course first year who carried out many of the interviews on which this paper is based.
REFERENCES 1. ANDREWSK R (1966) The Effectiveness of University Mana~,ement Development Programmes. Harvard University Graduate School of Business Administration. Boston. 2. BURGOYNEJG (1973) The judgement process in management students" evaluation of their learning experiences. Unpublished PhD Thesis, to appear in Hum. Relations. 3. CAMPaELLJP et ai (1970) Managerial Behaviour, Performance and Effectiveness. McGrawHill. 4. CAMPaVJ.L DT and STAN~Y JC (1964) Experimental and quasi-experimental design for research in training. In Handbook of Education Research. Rand McNally, Chicago. 5. HAMELINAC (1974) Evaluation and Control of Training McGraw-Hill, London.
482