257
Case Studies
Relationships between Demographics, Training, etc. in a DSS Environment 1. Introduction
Cary T. Hughes North Teras State University, P.O. Box 13677, Denton. TX 7620.? USA
New research findings are reported on the effect of demographics on certain attributes of managers’ decision processes in relation to training received on a DSS generator. A field case study was used to determine if the decision process attributes of time and alternatives generated were influenced by the demographic factors of age, sex, education, and previous programming experience. These factors were evaluated with training. Results indicated that demographics had little effect on decision process attributes in relation to training. Any differences that did exist were eliminated by training. Kqvwords; Decision erator, Training.
process
attributes,
Demographics,
DSS
gen-
In the past ten years, there has been increased interest in developing systems to support managerial decision making. Wagner [lo] refers to the evolving use of computers as “Executive Mind Support”. A more familiar term is Decision Support Systems (Dss). Past research into the area has concentrated on (1) the quality of the decision, (2) the decision making process, and (3) implementation [5]. Little research has been done to determine how training and/or demographics affects the decision process and the decision reached. Yet, the need for this research was described by Chervany [4] when he noted that training experiences can influence the way a manager makes a decision. Several researchers have identified the attributes of age and experience as determinants of information processing ability [6,8]. Further research is needed to determine if training and certain managerial demographics interact to affect managers’ uses of a DSS. 2. Three levels of DSS Sprague [8] proposes three types of DSS: specific DSS generator, and DSS tools. He defines a specific Dss as a system which accomplishes the work; a DSS generator as a software system that is parameterized to cause a specific DSS to be built; and DSS tools as the hardware and software elements, like the operating system, which aid in the development of a specific DSS. The only subject of this investigation is the DSS generator. It will be operationally defined as a financial planning language. The role of a financial planning language is described by Braun [2] as a way to make it possible to move the computer DSS,
Dr. Gary T. Hughes is an assistant professor of BCIS at North Texas State University. His teaching and research interests m&de training and decision support systems. He has done consulting in the area of system analysis and design and given numerous seminars on computer-related subjects.
North-Holland Information & Management
0378-7206/87/$3.50
12 (1987) 257-261
0 1987, Elsevier Science Publishers
B.V. (North-Holland)
258
Information & Management
Case Studies
into the executive suite. Whether this move will occur is debatable, but financial planning languages do have the capability of extending the manager’s thought process.
reach a decision is preferred, given similar quality of the decision.
5. The Experiment 3. Research Question The work was intended to evaluate the effectiveness of a training program using a DSS generator. The object was to determine how certain attributes of managers’ decision processes were initially affected by a training program. Another aspect was to determine how certain demographics - age, sex, education, and previous programming experience - of decision makers affected their use of a DSS generator with respect to the training received. Given this framework, the major research question that was investigated was: Do demographics, either individually or collectively, interact with training on a DSS generator to initially change a manager’s decision process?
4. Attributes and Assumptions
Because of the difficulty in measuring actual decision process change, some attributes of the decision process were used. Possible attributes that may have been considered included: (1) the time to reach the decision; (2) the number of altematives considered; and (3) the confidence placed in the decision. Considerable justification exists in the literature for the use of these particular attributes as appropriate surrogates [3,5,7]. In this research, two attributes were selected to determine if this change had occurred. These were the first two. No attempt was made to classify the number of alternatives generated into good or bad. Rather it was important to determine if there is a significant relationship between the number of alternatives generated, the time to reach a decision, the training received, and the demographics of the manager. Thus, the research was directed at evaluating quantitative changes in attributes of the decision process. However certain implications can be drawn in the context of the manager’s job; e.g. it is generally agreed that examining a larger number of alternatives in reaching a decision is preferable. The time parameter is also important; less time to
An experiment was conducted to compare results of changes in performance variables for managers who participated in the use of a DSS generator and those who did not. The demographic variables were age, sex, education, and programming experience. The performance variables were time and number of alternatives generated. 5. I Experimental
Setting
The experiment was conducted at a major university in the Southwestern USA. Participants were middle and upper level decision makers from a nearby metropolitan city. Demographics were collected before the experiment was begun. Half the managers were randomly assigned to a training group. In all, 63 managers participated in this field study. Two cases were used to determine change. The cases were scored based on a measurement of the two attributes: time and number of alternatives generated. The first case involved a revenue estimate, the second an expenditure estimate. They were designed so the two attributes of interest could be determined. An effort was made to keep the cases as similar as possible in length, degree of difficulty, and scenarios. A pilot test, using graduate students, was used to confirm the similarity of the two cases. Scoring forms were developed and used for the two cases. An independent rater was used to verify correctness of grading. 5.2 Training Program
The training program was one commonly given to organizations using a Dss generator on a trial basis. An experienced trainer administered the one day program. The training was divided into two sessions. The morning session consisted of a presentation of types of decisions that managers must face, the advantages of using modeling in making these decisions, the advantages of using a DSS generator for modeling, and a demonstration of how a par-
Information
C. T. Hughes / DSS Environment
& Management
ticular DSS generator could be used. The afternoon session included a 30 minute recap of the morning session, a one hour group session, in which a model was actually designed, and a one hour terminal session in which participants received “hands-on” experience using a Dss generator.
6. Research Design Interest centers on investigating whether or not demographic variables affect performance variables. In order to accomplish this analysis, the data was organized as the matrix Y+ where Y refers to performance measure, i to the individual manager, j to the group assignment (training or not), and k to the attribute measure (Age, Sex, etc.). Table I gives the variable names, descriptions, and codings used in the analysis.
7. Statistical Analysis
In an attempt to examine the effect of the demographic variables on the performance, three statistical analyses were performed. These are detailed in the Appendix, but the most significant results are reported below. 1. Sex was a significant variable, with males taking more time than females to make a decision.
Table 1 Variable names, descriptions, Variable
Experimental
EXGR
Demographic
Sex of manager Education of manager
EDU
Previous programming Previous programming
PROEXP PROFIN
TIMEP ALTP
2 = some training
Variables
SEX
PALT
In conclusion, when the demographic variables were taken separately, only the sex of the manager and previous programming experience show an effect on the performance variables and then only on the pretest time to reach a decision and on the number of alternatives generated for the pretest, respectively. When the demographic variables are considered in conjunction with the experimental group, only the manager’s sex shows any interaction effect on performance variables. Note that when considered separately or in conjunction with the experimental group, the demographic variables
1 = no training,
group
Age of manager
PTIME
8. Conclusions
Coding
AGE
Performance
However training helped males, but slowed females. 2. People with more experience in using financial planning languages considered more alternatives. Training was not significant in changing the number of alternatives considered. 3. Among all managers (training and no training groups), managers who took the least time and considered the most alternatives tended to be females with previous programming experience in a financial planning language. 4. Among the managers who had training, young highly educated females with previous programming experience tended to evaluate more alternatives before reaching a decision.
and codings Description
name
experience on a DSS generator
1= c 30,2 = 30-40, 3=41-50,4=50+ 1 = male, 2 = female 1 = 2 years college, 2 = 4 years college, 3 = Master’s 4 = More than Master 1= yes, 2 = no 1 = yes, 2 = no
Variables Pretest time to reach decision Number alternatives generated Posttest time to reach decision Number alternatives generated
for pretest for posttest
259
Minutes Integer Minutes Integer
260
Information & Management
Case Studies
which are significant affect only the pretest performance variables - not the posttest ones. In other words, training seemed to have no effect on the way managers scored on the performance variables. When the demographic variables are considered collectively, different demographic effects exist for managers who have had some training and for the complete set of managers. For the complete set of managers, the managers’ sex and previous programming experience in a financial planning language together affect the pretest time, number of alternatives generated and the posttest number of alternatives generated. For only the managers who went through some training, all demographic variables affect the pre and post training number of alternatives generated. Overall, the demographic variables have little effect on the performance variables. Any effects they may have separately seem to be more on the pretest than the posttest. This implies that the separate demographic effects tend to be eliminated by the training program. The implications are that managers need not be concerned about demographics when sending their employees to training programs on the use of a DSS generator. While females may evaluate more alternatives in a shorter period of time than males initially, the training tends to bring both groups to the same level.
References [l] Alpert, M., R.A. Peterson and W.S. Martin: Testing the Significance of Canonical Correlations, Proceedings American Marketing Association, 37 (1975) 117-119. [2] Braun, T.H.: The History, Evolution, and Future of Financial Planning Languages, ZCP Interface Adminisfratiue&Accounting, 5 (1980) 24-33. (31 Brightman, H.J., SE. Harris and W.J. Thompson: An Empirical Study of Computer-Based Financial Modeling Systems: Implications for Decision Support Systems, The DSS Transactions 1981 (1981) 102-110. [4] Chervany, N.L.: Management Information Systems: Design Questions From a User’s Perspective, Proceedings of the Midwest Conference of AIDS (1972) A19-A24. [5] Dickson, C.W., J.A. Senn and N.L. Chervany: Research in Management Information Systems: The Minnesota Experiments, Management Science, 23 (1977) 913-923. [6] Gul, F.A.: A Note on the Relationship Between Age, Experience, Cognitive Styles and Accountants’ Decision Confidence, Accounting and Business Research (Winter 1983) 85-88. 171 Keen, P.G.W.: Decision Support Systems: Translating
Analytic Techniques into Useful Tools, Sloan Managemenf Reuiew (1981) 33-44. 181Sprague, R.H.: A Framework for the Development of Decision Support Systems, Management Information Systems Quarterly, 4 (1980) l-24. as Determinants of [91 Taylor, R.N.: Age and Experience Managerial Information Processing and Decision Making Performance, Academy of Managemem Journal (March 1975) 76-81. 1101Wagner, G.R.: Decision Support Systems: Computerized Mind Support for Executive Problems, Managerial Planning, 30 (1981) 9-16.
Appendix In order to evaluate the data, one-way multivariate analysis of variance (MANOVA), two-way MANOVA, and canonical correlation were used. The first, one-way MANOVA, used the demographic variables as the independent variables and the performance variables as the dependent variables. The results showed that the SEX variable was significant at the 0.033 level. A one-way univariate ANOVA was then run on each of the four performance variables with the result that PTIME was the most significant between males and females. In particular, males scored significantly higher on PTIME (mean = 36.06) than females (mean = 24.35). Follow-up one-way ANOVA'S on each performance variable showed that PALT was the most significant between the two PROFIN groups. In particular, managers who had previous financial programming experience scored higher on PALT (mean = 9.33) than those who did not (mean = 5.47). The second type analysis consisted of a set of two-way MANOVA'S with one factor being the experimental group (EXGR), and the other factor being each demographic variable. The same four performance variables were used in each run. The purpose of these runs was two-fold: (a) to examine the demographic variables simultaneously with the two experimental groups and, in particular, to see if the SEX and PROFIN variables are still significant when the EXGR variable is considered; and (b) to determine if any significant interactions are present. The results showed a significant interaction did exist between EXGR and SEX at the 0.021 level. Follow-up two-way univariate ANOVA'S pointed out that PTIME was the main source of the interaction. In particular, male managers who were in a training program showed a significant decrease in PTlh4Eover those male managers who were not (mean PTIME no training = 45.1, mean PTIME training = 39.8); whereas female managers showed just the reverse (mean PTIME no training = 30.0, mean PTIME training = 39.5). Thus, the effect of the SEX variable is different for the no training and some training groups. None of the other demographic variables had any significant interactions with EXGR nor showed any significant effects in the presence of EXGR. Canonical correlation was the third type analysis used. One set of variables was the four performance variables, and the other set the demographic variables. The purpose of this analysis was to search for interrelationships that may exist between the complete set of demographics and the performances. One
Information & Management Table 2 Canonical
correlations
Variable
C. T. Hughes / DSS Environment
of variables EXGR
= 1
(n = 63) Loadings PTIME
-
PALT TIMEP ALTP AGE
-
SEX EDU
-
PROEXP
-
PROFIN
-
Canonical R Significance of R Redundancy Significance of redundancy
and 2
EXGR
=
2
(n = 42) Loadings
0.476 0.363 0.472 0.287 0.159 0.714 0.050 0.290 0.632 0.492 0.042 0.019
0.281 0.623 -0.131 0.544 - 0.610 0.498 0.604 - 0.380 - 0.494 0.432 0.021 0.052
0.990
0.990
and two-way MANOVA’S can only analyze the demographic variables one or two at a time; whereas canonical correlation analysis can analyze the complete set simultaneously. Three canonical correlation runs were made. The first run
261
was across both experimental groups, the second run was for only the no training group, and the thud for only the training group. One significant canonical correlation existed for the run across both experimental groups and one for the training group. There were no significant canonical correlations for the no training group. The correlations (loadings) between the variables and canonical variates, canonical correlations and significances, and redundancies of the performance variables given the demographics and significances are shown in Table 2. The table shows only the results for the two significant canonical correlations. Interpretation of the significant relationships can be made by an examination of the loadings together with the codings given in Table 1. Only loadings of 0.3 or higher are considered. These findings were presented in the statistical analysis section of this paper. The significance of the canonical correlations does not necessarily imply that the relationships are strong. An indication of the strengths of the relationships can be obtained by determining the redundancies. The redundancies of the dependent variable set (the performance variables) given the independent variable set (the demographic variables) were calculated, aLong with their significances, and are shown in Table 2. The results show that the redundancies are very low. Thus, the demographic variables are having little predictive effect on the performance variables as channeled through the canonical variates.