Locus of control and evaluation use: Does sense of control affect information needs and decision making?

Studies in EducationalEvaluation. Vol.

9, pp. 77-88, 1983 Printed in Great Britain. All rights reserved.

0191-491X/83/010077-12506.00/0 Copyright © 1983 Pergamon Press Ltd.

LOCUS OF CONTROL A N D EVALUATION USE: DOES SENSE OF CONTROL AFFECT INFORMATION NEEDS AND DECISION MAKING? Dianna L. N e w m a n Oklahoma State University, Stillwater, Oklahoma 74074, USA

Robert D. Brown and Linda S. Rivers Department of Educational Psychology and Measurement, Teachers College, University of Nebraska, Lincoln, Nebraska 68588, USA

Making evaluative decisions about programs is a highly complex process. When discussing this process abstractly, evaluators agree that many factors affect the utilization of information for decision-making (Braskamp & Brown, 1980). However, folklore in evaluation is replete with anecdotes by and about disgruntled evaluation practitioners, describing their frustrations. They are upset because a decision-maker did not give sufficient attention to their predictive regression equations, their analysis of variance results, or other forms of statistical findings. These tales of woe suggest that many evaluators believe decisions about programs are and should be made primarily on the basis of empirical data collected during the evaluation process. They fail to recognize that numbers or computer print-outs do not make decisions; people make decisions. This truism has significant implications for evaluators and for the study of the uses of evaluative information. Decision-making is not necessarily a straight-forward linear process, but rather one which is influenced by a multitude of personal and contextual variables. Until there is a fuller understanding of how people process information and make evaluative decisions, evaluators will not be as effective as they might be in obtaining, providing, and reporting useful information for decision-makers. A critical and somewhat neglected dimension of research on evaluation utilization is an examination of the characteristics of the decision-maker. Several questions need study: What personal characteristics influence decision-making? Do personality characteristics affect whether and how data are used? How much are decision-makers influenced by external opinions or their preconceived biases? Do groups of decision-makers process evaluative information differently than individual decision-makers? Answers to these questions could provide evaluators with a better understanding of the decision-making processes. One psychological construct that might help explain decision-makers' behaviors is locus of control (Rotter, 1954, 1966). This construct suggests that persons characterized as externals typically ascribe their fate, success of failure, to external factors over which they have no control. Internals, on the other hand, perceive themselves more in personal control of their fate. It is possible that externals and internals have different information needs, process information differently, have varying needs for expert assistance, and differ in how much they want to know others' opinions. Understanding the relationships

77

78

D . L . Newman eta/.

between locus of control and information use could provide helpful perhaps even guidelines for evaluators.

insights and

Theoretical and Research Background Previous research indicates that characteristics of evaluation audiences are related to how evaluative information is interpreted. Audience characteristics which appear to be related include: (i) Organizational Position. Teachers and administrators differ in their perceptions of the value of the same evaluation information (Braskamp, Brown, & Newman, 1978), (2) Professional Level. Experienced educators differ from student teachers in their views regarding evaluation information (Newman, Brown, & Littman, 1979), (3) Professional Background. Business persons view educational evaluations differently than educators (Newman, Brown, & Littman, 1979), and (4) Perceived Need for Evaluation. Persons strongly in favor of evaluation see more utility in different kinds of information than do those less in favor of evaluation (Brown, Newman, & Rivers, 1980). For the most part, these studies focused on readily identifiable characteristics, such as sex, occupation, and educational background. Psychological variables have also been found to effect decision-making in other contexts (Janis & Mann, 1977). Selective response to new information based on an individual's stand on issues, for example, is affected by certain personality characteristics of the decision-maker (McGuire, 1968; Triandis, 1971). Self confidence (Lowin, 1967), compulsivity (Cameron, 1963), and sense of competence (DeCharm, 1968) are among other personality correlates of various decisionmaking styles. It is reasonable to expect that these variables also affect decision-making in an evaluation context. Morris (1972) concluded that decision-makers have personal styles that are fairly consistent. Development of this style is influenced by how much people are willing to rely on intuition and their tolerance of uncertainty and ambiguity. Congruency of beliefs and actions, as well as sensitivity to stress, are suggested as other personality characteristics affecting decision-making. A major factor affecting decisions and influencing the extent of bias in decision-making is the illusion of control (Hograth, 1980). Sense of control may be especially influential in the planning and evaluation stages of a program. Many decision-makers attribute a successful program to their personal influence and power, but attribute a failure to "bad luck." Locus of control has been found to be related to degree of optimism about outcomes in chance situations (Langer, 1975) and to judges' ratings of gymnastic performance (Scheer & Ansorge, 1979). Personality characteristics of individuals in groups have also been found to affect group decision-making. Bither (1971) found that need for dominance, social recognition, and autonomy were related to group task behavior. The best combination, according to Bither, is a group with a dominant person who has a high need for social recognition, but a low need for autonomy. Janis & Mann (1977) concluded that groups are likely to be more vigilant about seeking new information than individuals, but groups are also more likely to avoid making a decision if at all possible. The four studies reported in this paper examined the relationship of locus of control to evaluation audiences' need for and use of evaluative information. The studies were conducted sequentially and for the most part were exploratory. They sought answers to the question: Does locus of control relate to: (a) preferences for evaluation strategies, (b) kinds of information seen as useful in

79

Locus ofControland Eva~ation Use

decision-making, (c) values placed on the usefulness and (d) group interaction in decision-making?

and

worth

of

evaluation,

METHODOLOGY Participants For the four studies, a total of 189 participants were solicited from class groups and workshops in a variety of educational settings at the University of Nebraska-Lincoln. The participants were almost evenly divided between males and females and the average age was 33. Almost all of the subjects were educators with nearly three-fourths either currently holding or planning to hold administrative positions. Instrumentation Several measures previous studies.

were

used

in

these

investigations;

most

had

been

used

in

Locus of Control. All subjects completed a locus of control measure based on one derived from Rotter's scale by Levenson (1974). It included 12 items reflecting perceptions of control over personal life and decision-making contexts. Respondents used a six-point Likert scale, indicating how often they believed the statements applied to them (I = Hardly ever, i = Seldom, 3 = Sometimes, 4 = Frequently, 5 = Often, and 6 = Almost Always). Half the items reflected internal control (e.g., "I am able to protect my own personal interests.") and half reflected external control (e.g., "My decisions are influenced by those with more power."). Scores on the external items were reversed so that a high overall score reflected internal beliefs, i.e., acceptance of internal statements and rejection of external statements. Alpha coefficient of reliability for Study II was .71. Preference for Recommendations. This measure consisted of two items asking for preference regarding recommendations in an evaluation report. One item asked whether respondents would like to make definite recommendations if they were the evaluator. The other item asked if they would want an evaluator to make recommendations to them if they were the decision-maker. Responses were made using a five-point Likert scale, Strongly Agree = 5 to Strongly Disagree = i. Evaluator Role Preference List. This measure included a list of six possible evaluation roles. Respondents were asked to rank their top three preferred roles for evaluation from the following list: (a) provide information to decision-makers, (b) judge the worth or merit of a program, (c) determine how to improve a program, (d) determine whether to continue or terminate a program, (e) describe or portray a program, and (f) test out a research theory. Usefulness of Information. This instrument included 26 items enumerating information related to program description, program budget, statistical and measurement data, opinions of staff, teachers, and outsiders, and evaluator recommendations. Respondents indicated how often they found the information useful using a six-point Likert scale (i = Hardly ever useful, 2 = Seldom useful, 3 = Sometimes useful, 4 = Frequently useful, 5 = Often useful, and 6 = Almost always useful). Evaluation

Needs

Assessment.

This

grams (I0 civic and i0 educational). believed there was a need to provide

measure

included

a

list

of

twenty

pro-

Respondents indicated the extent they for a budget for an evaluation of the

80

D.L. Newman eta/.

program. They were told that all programs were tax supported and that evaluation information would be used to help improve programs and to decide if the programs would be continued. The alpha coefficient for reliability was about .80 for all administrations. Dollars Budgeted for Evaluation. This measure consisted of two questions asking the respondent to supply the dollar amount they believe should be budgeted for an evaluation. There was no description of the "program". One question asked for the amount to be budgeted for a program costing $i0,000, the other question involved a program costing $i00,000. Simulation Vignettes. This measure consisted of eight questions regarding information needs. Participants had to make decisions about a briefly described program evaluation context. Two evaluation contexts were used, one describing use of television to assist social studies instruction and the other describing a peer drug counseling program for high school students. The eight questions focused on the kinds, amount, and sources of information the respondents would want to have before making a decision about keeping or terminating the respective program. Participants were asked to assume the role of school board members as they deliberated and made their decisions. STUDY PROCEDURES

AND RESULTS

The first three studies were correlational with the primary focus being the examination of the relationships between locus of control and reactions to specific evaluation issues and questions. Participants provided their responses to questionnaires using Likert type items. The fourth study involved manipulation of participants through assignment to different simulated decision-making groups. All subjects provided selected demographic information, including age, sex, occupation, and extent of experience. The procedures and results of each study will be presented, followed by a discussion of the implications of all four. Study

I

This exploratory investigation examined the relationships among locus of control and choice of an evaluation approach. Participants were 87 educators with varying degrees of administrative responsibility and knowledge about evaluation. All were involved in positions necessitating administrative decisionmaking. Each participant completed three sets of measures: (i) Locus of Control, (2) Preference for Recommendations, and (3) Evaluator Role Preference List. Because of the exploratory nature of this study, heuristic purposes were best served by examining the relationship of each of the items on the independent measure, Locus of Control with the two dependent measures, Preference for Recommendations and rank ordering of the evaluator roles on the Evaluator Role Preference List. This was done through chi-square analysis. The results indicated there were significant relationships (p < .05) between three items of the Locus of Control scale and how the participant felt about needing and making recommendations. Responses to "When I make plans, I am almost certain to make them work," (an internal item) were significantly related to preferring to make recommendations if the respondent was conducting an evaluation. The more participants agreed with the statement the more likely they were to have a strong preference for making recommendations if they were acting as an evaluator. Participants who agreed with the internal statement: "I am usually able to protect my personal interests" were also more likely to desire to make recommendations as evaluators. Responses to this same statement were also significantly related to whether respondents believed the evaluator should

Locus of Controland Eva~ation Use

TABLE i:

Means and Standard Deviations of Need for Evaluation A.

Program

F*

for Significant Differences on Ratings

Study II

External Mean S.D.

Internal Mean S.D.

Art education

4.30

3.36

1.02

3.81

.77

School patrol

5.30

2.58

1.16

3.22

1.20

Inservice for math teachers

6.15

3.78

.72

4.22

.80

First aid for police

4.51

3.31

1.24

3.92

1.20

7.29

2.89

1.14

3.53

.84

10.13

2.72

.94

3.42

.19

Police at intersection

4.28

2.97

1.13

3.50

1.03

Modified school lunch program

8.49

3.14

.96

3.81

.98

Career education for handicapped

6.02

4.14

.68

4.50

.56

B i c y c l e safety program Library in parks

81

* p < .05, df = i, 70 B.

Item

F*

Study III External Mean S.D.

Internal Mean S.D.

Remedial writing course

5.24

2.8

1.81

4.4

1.26

Inservice math training for elementary teachers

25.19

2.5

.97

4.4

.70

6.23

4.2

.63

4.8

.42

Career education for handicapped

17.64

4.1

.31

4.8

.42

Enrichment programs for gifted

10.80

4.1

.31

4.7

.48

Alcohol education for teenagers

* p < .05, df=l, 18

make recommendations to them, if they were commissioning an evaluation. The more confident the respondents felt about their personal ability to protect their interests, the more likely they were to want evaluators to make recom-

82

D.L. Newman eta/.

mendations. Responses to "When 1 get what I want, it's usually because I work harder for it," (internal item), indicated those who agreed with this statement wanted the evaluator to make recommendations, whereas those who disagreed were least likely to want the evaluator to make recommendations. There were two Locus of Control items which yielded significant chi-squares (p < .01) with participants' rankings of an evaluator's role. Those who disagreed with the external item "When I get what I want, it's usually because I am lucky," were more likely to believe the primary role of evaluation was either "providing information" or "judging worth or merit." On the other hand, those who agreed with the internal statement, "I am usually able to protect my personal interests," also selected either "providing information" or "judging worth or merit" as the primary evaluator role. Study

II

This study investigated the relationship between locus of control and: (i) the kinds of evaluative information perceived as useful and (2) expressed need for evaluating civic and educational programs. The 72 participants in this study were enrolled in graduate classes and workshops in educational administration. Approximately 60 percent were preparing for administrative roles, others were currently administrators. Their average age was 34 and they averaged nine years in their occupation. Each participant completed the Locus of Control scale, the Usefulness of Information form, and the Evaluation Needs Assessment inventory. Participants were divided into internal and external groups on the basis of a median split on the Locus of Control scale. Those in the external group (n = 36; ~ = 45.69, s.d. = 3.40) could be characterized as viewing themselves to have little control over their destinies, whereas the internal group (n = 36; = 56.19, s.d. = 4.99) believed their decisions affect their lives directly without much external influence. The difference between the two groups on the locus of control measure was statistically significant (p < .01). This study resulted in two major and congruent findings. The first major finding was on the ratings of usefulness of information. Overall, the externals rated evaluation information as less useful than did the internals. Differences were statistically significant on items reflecting usefulness of others' opinions: "what teachers think," "what parents think," "what experts think," and "what school administrators think." Externals also rated information about program participants, program goals, and statistical comparisons as significantly less useful than did internals (p < .01). The second major finding was on the need for evaluation items. The externals gave lower ratings for the need to conduct an evaluation of a variety of civic and educational programs. The differences were statistically significant (p < .05) on nine of the 20 programs listed (See Table i). These results indicate that participants portraying themselves as having less control over their decisions and life, for the most part, did not want to know what other people thought nor were they as supportive of evaluation in general when contrasted to participants who perceived themselves to be in more control of their destiny.

Locus o f C o n ~ o l a n d Eva~adon Use

TABLE

2:

Means and Standard Deviations of Usefulness of Information A.

Item

for Significant

Study

Differences

on Ratings

II

External Mean S.D.

Internal Mean S.D.

4.15

4.53

1.30

5.08

1.00

4.72

4.69

1.26

5.28

1.00

4.94

3.47

1.32

4.14

1.22

F*

Descriptive: What students teachers do

and

Program Statistics:

Comparisons

Opinions: What

teachers

think

5.33

4.28

1.39

4.94

i .04

What

students

think

4.37

4.19

1.47

4.89

1.35

What

experts

think

6.62

3.78

1.22

4.50

1.16

What administrators think

5.49

3.97

1.25

4.58

.94

Information conflicts

4.74

4.17

i .44

4.83

1.13

about

* p < .05, df = I, 70 B.

Item

F*

Study

III

External Mean S.D.

Internal Mean S.D.

Budget: the cost per student in terms of new learning skills

5.63

3.7

.95

4.9

1.29

Statistics - How the information was obtained (sample size, procedures)

4.50

3.4

1.17

4.6

1.35

Descriptive - Statement of specific program goals

7.31

4.5

1.08

5.6

.70

* p

< .05; df = 1,18

83

84

D . L . Newman eta/.

Study I I I This study replicated Study II, which examined the differences between internal and external groups on the Locus of Control scale on evaluation needs. It differed, however, in comparing extreme groups rather than those derived using a median split and also looked at differences on two additional dependent measures. As in Study II, these participants completed the Locus of Control scale but were then blocked into three groups: Internal Locus of Control (n = i0), Middle Locus of Control (n = i0), and External Locus of Control (n = i0). To have extreme groups for comparisons, the middle group was removed from further c o n s i d e r a t i o n for analyses, leaving two subsets of subjects, Internal and External. The differences b e t w e e n the means of the Externals (~ = 47.80) and Internals (f-= 59.40) was significant at the .01 level, (t = 10.13). Besides c o m p l e t i n g the Usefulness of Information form and the E v a l u a t i o n Needs Assessment inventory, participants in Study III also completed the Dollars Budgeted for E v a l u a t i o n items and the questions on the e v a l u a t i o n vignettes. The independent variable was locus of control with the participants classified as either internal or external. The dependent variables were ratings of u s e f u l ness of information, e v a l u a t i o n needs for education and civic programs, dollars budgeted for evaluation, and information needs and sources for making decisions about two simulated programs. Several significant differences were found for types of information rated as useful. Internal subjects found the information on the cost of a new program, how statistics were obtained and d e s c r i p t i o n of program goals more useful than did participants with an external sense of control. There was no d i f f e r e n c e b e t w e e n the groups on the usefulness of the various e v a l u a t i o n strategies described (See Table 2). When asked to indicate the amount of dollars to be budgeted for a $I0,000 program, there was no difference b e t w e e n the two locus of control groups. However, there was a significant d i f f e r e n c e on the $i00,000 project. The internal subjects indicated that a larger amount should be used for e v a l u a t i o n (x = $12,250) than did the external group ( ~ = $3,800). A significant difference was found on five of the ten educational programs w h e n participants indicated the extent to which they saw the need for an evaluation. In all cases, internals were more favorable about the need for a budgeted e v a l u a t i o n than were externals. There were no significant differences between internals and externals on ratings of budgeted e v a l u a t i o n needs for any of the civic programs. Generally, subjects were neutral or u n c e r t a i n about the need for evaluation in the civic program area. There were no significant differences between inLernals and externals on any of the information need items for the peer drug c o u n s e l i n g vignette. In the vignette describing the use of television for instructional purposes, the internals were significantly more supportive of being involved in the decision. There were no other significant differences. Study

IV

This pilot study investigated the r e l a t i o n s h i p of locus of control as a variable affecting group d e c i s i o n - m a k i n g in e v a l u a t i o n contexts. In the education area or in policy settings, p r o g r a m decisions are often made by groups. School boards and others are often r e s p o n s i b l e for making decisions about terminating or retaining a p r o g r a m based on limited or extensive evaluation information. It is possible that locus of control would affect the group process in these decision contexts.

Locus of Control and Evaluation Use

85

Several small group simulations were devised employing locus of control as an independent variable. Four high internals and four externals were selected from the participants in Study III for this study. Two groups of four persons each were established on the basis of the locus of control scores. One group consisted of four externals and the other of four internals. Each group was given one of the evaluation vignettes described earlier and asked to make several decisions: (i) Would they recommend keeping the program? (2) Would they want to consult with someone? and (3) What other information would they find useful? When these decisions were made, the group composition was restructured so that two groups of four persons each contained two externals and two internals. They were then given another evaluation vignette and asked to make the same decisions. Each session was observed by another person who was unaware of the purpose of the investigation. The observers were asked to note the activities of the group and how decisions were made. Observer reports indicated the internal group was fairly task-oriented and as group members considered the complexities in the evaluation context. All participants in this group were comfortable with making the decisions as well as hearing what others in the group felt about the program being evaluated. They arrived at their decisions through consensus. The external group was more hesitant in sharing their opinions and they arrived at several decisions by voting. The two mixed, internal-external groups were less task-oriented than the homogeneous groups. In one group, internal members sought to control the discussion. One mixed group wanted to involve the observer in the d e c i s i o n - m a k i n g and the other group did not arrive at any decision w i t h i n the specified meeting time. DISCUSSION

AND IMPLICATIONS

Discussion These studies were simulations, for the most part, and were narrowly focused on the relevance of locus of control and information needs for decisionmaking in evaluation contexts. Participants were not making decisions about actual programs. The investigators have found, however, that participants in these types of studies take the exercises and questions quite seriously and consider the issues important. Simulation studies provide useful data for both understanding a phenomenon and for practice. The implications for practice depends upon the similarities of the supplied task to that of a real life task. The consistency and congruency of the results of these findings suggest that the construct of locus of control is a viable one to consider in trying to understand an evaluation client or audience's response to and use of evaluative information. More research is needed to determine the extent and nature, of the relationship of locus of control and information use to more natural settings. It is helpful to examine the findings across all the studies as they pertain to: (i) evaluator role and recommendations, (2) need for evaluation, (3) ratings of usefulness, and (4) group decision-making. Evaluator Role and Recommendations. The findings for Study I suggest that locus of control is related to an evaluation audience's preferences for evaluator roles and their interest in either making or receiving an evaluator's recommendations. Internals perceived evaluation as providing information for decision-making and for making judgments about program. They also were more comfortable than externals in making recommendations when they were in an evaluator role or receiving recommendations from an evaluator when they were in a

86

D.L. Newman eta/.

decision-making role. It is possible that internals are more self-confident and open to new information, whereas externals are less self-confident and less responsive to new information which might contradict their biases. Usefulness of Information. The findings of Study II and III provide data on specific information preferences that distinguish the internals from the externals. Internals are not only more open to evaluation information generally but they are also more likely to favor specific kinds of information. Study II results suggest that internals are more likely to welcome opinions from significant others, such as teachers, parents, and staff. Study III indicated internals are more likely to find cost information, goal descriptions, and statistical comparisons with other programs more useful than externals. Need for Evaluation. The findings of Study II and III indicate that ratings of general and specific need for evaluation are consistent with ratings of usefulness of evaluation and specific kinds of evaluative information. Internals are more likely to support the need for evaluation in general and for specific civic and educational programs. This support also manifests itself when participants were asked to specify a dollar amount to be budgeted for program evaluation with internals, in the case of a $i00,000 project, recommending a higher dollar amount for evaluation than externals. Group Decision-Makin$. The results from the pilot study must be interpreted cautiously. The observations suggest that the decision-making process may be different when groups are homogeneous, i.e., totally internal or external and may also differ when the groups are moxed. Further research is needed to validate these findings and to examine differences in the content of the decisions as well as the process. Implications For Research. It appears that locus of control is a viable construct for examination of the decision-making process and the use of evaluative information. The consistency of the findings and their congruency indicate, at least for educators, the variable relates to a number of reactions and opinions affecting use of evaluative information. Among the several research questions possible for further study are: (i) How is locus of control related to information needs among a variety of levels and types of evaluation audiences? Would internal teachers, for example react differently than internal administrators? Would internal decision-makers in industry, business, or health professions react differently than educators? (2) How does locus of control affect group uses of evaluative information? The study reported here was primarily exploratory. Further research should include more groups and examine decision content as well as the process used to make decisions. It would be interesting to note whether the group process leads to different behaviors and information needs before, during, and after a group discussion. (3) Is locus of control a relevant factor in natural settings? Observational techniques of decision-making settings could provide valuable insights into how this variable operates to influence the process and the outcome of decision-making. (4) What strategies might be effective for an evaluator who wishes to enhance utilization of evaluation information in light of these research findings? Both experimental studies involving m a n i p u l a t i o n of various strategies in controlled settings and naturalistic observation should provide clues, if not specific guidelines for enhancement of the uses of evaluation. For practice.

The

implications

of these

findings

for

ator are two-fold. First, decision-making in an evaluation it involves more than straightforward data interpretation.

the practicing

evalu-

context is complex; Personal character-

Locus of Con~oland Eva~ation Use

87

istics, the evaluation context, and group dynamics all influence the process and perhaps the outcome. Second, understanding the extent of personal control perceived by the decision-maker could help the evaluator understand the decisionmaker, if not predict behavior. This should influence evaluator behavior as well. This may appear to make the evaluator's task more complex. However, by being more alert and responsive to personal and contextual issues as well as the directly related evaluation issues, the evaluator can become a more useful consultant.

REFERENCES BITHER, S.W. Personality as a factor in management team decision-making. The Pennsylvania State University, Center for Research, College of Business Administration, 1971. BRASKAMP, L.A., & BROWN, R.D. Utilization of evaluative information. New Directions for Program Evaluation Series. San Francisco: Jossey-Bass, 1980. BRASKAMP, L.A., BROWN, R.D., & NEWMAN, D.L. Credibility of a local evaluation report: Author source and client characteristics. American Educational Research Journal, 1978, 15, 441-450. BROWN, R.D., NEWMAN, D.L., & RIVERS, L. Perceived need for evaluation as an influence on evaluation's impact on decision-making. Educational Evaluation and Policy Analysis, in press. CAMERON, N.A., Personality Development and Psychopathology. Boston: Houghton Mifflin, 1963. DeCHARMS, R., Personal Causation, New York: Academic Press, 1968. HOGARTH, R.M. Judgment and choice; The psychology of decision. Chichester: John Wiley & Sons, 1980. JANIS, I.L., & MANN, L. Decision-making. New York: The Free Press, 1977. LANGER, E. The illusion of control. Journal of Personality and Social Psychology, 1975, 32, 311-328. LEVENSON, H. Activism and powerful others. Journal of Personality Assessment, 1974, 38, 377-383. LOWIN, A. Approach and avoidance: Alternative modes of selective exposure to information. Journal of Personality and Social Psychology, 1967, 6, 1-9. McGUIRE, W.J. Selective exposure: A summing up. In R.P. Abelson et al. (Eds.) Theories of cognitive consistency: a sourcebook, Rand McNally, 1968. MORRIS, W.T. Matching decision aids with intuitive styles. In H.S. Brinkers Decision-making: creativity, judgment, and systems. Columbus: Ohio State University Press, 1972. NEWMAN, D.L., BROWN, R.D., & LITTMAN, M. Evaluator report and audience characteristics which influence the impact of evaluation reports: Does who says what to whom make a difference? CEDR Quarterly, 1979, 12, 14-18. ROTTER, J.B. Social learning and clinical psychology. New York: Prentice-Hall, 1954. ROTTER, J.B. Generalized expectancies for internal vs. external control of reinforcement. Psychological Monographs: General and Applied, 1966, 80(1). SCHEER, J.K., & ANSORGE, C.J. Influence due to expectations of judges: a function of internal-external locus of control. Journal of Sport Psychology, 1979, i(i), 53-58. TRIANDIS, H. Attitudes and attitude change. New York: Wiley, 1971.

88

D. L. N e w m a n et aL

THE AUTHORS DIANNA L. NEWMAN is Assistant Professor of statistics in Applied Behavioral Studies in Education at Oklahoma State University, Stillwater. Her interests are in measurement, program evaluation, and utilization. ROBERT D. BROWN is professor of Educational Psychology and Social Foundations, University of Nebraska-Lincoln. He was formerly Director of the Counseling Center and Assistant Vice President for Academic Affairs. He is currently editor of the Journal of College Student Personnel. His research interests focus on utilization of evaluative information. LINDA S. RIVERS is assistant professor in Human Development and the Family, University of Nebraska-Lincoln and staff member at the Counseling Center. Her interests include human development, counseling, and information utilization.