- Email: [email protected]
Interaction analysis in GDSS research: description of an experience and some recommendations
233
Interaction Analysis in GDSS Research: Description of an Experience and Some Recommendations * 1. Introduction
Ilze Z I G U R S College of Business, University of Colorado, Boulder, CO 80309, USA
An emerging body of research in group decision support systems (GDSS) provides evidence that computer technology impacts the quality of decision making in groups. Most GDSS research focuses on the effects of computer support on outcomes, such as decision quality, commitment, or consensus. This article discusses the analysis of group process, rather than outcomes, via the technique of interaction analysis. Interaction analysis provides a micro-level examination of group behavior by the use of coding schemes for verbal interaction. The application and extension of interaction analysis in GDSS research is reviewed and illustrated. Keywords: Group Decision Support System; Interaction Anal-
ysis.
ilze Zigurs is an Assistant Professor of Management Information Systems in the Management Science/Information Systems Division of the University of Colorado, Boulder. She holds a Ph.D. in management information systems from the University of Minnesota. Dr. Zigurs' primary research interest is in the use and impact of computer and communications technologies in the support of collaborative work. Her research has been published in the
T h e g r o w t h of a p p l i c a t i o n s of c o m p u t e r techn o l o g y in o r g a n i z a t i o n s includes an i n c r e a s i n g interest in c o m p u t e r - b a s e d s u p p o r t for decision m a k i n g in groups. A n e m e r g i n g b o d y of research in g r o u p d e c i s i o n s u p p o r t systems ( G D S S ) p r o vides evidence that c o m p u t e r t e c h n o l o g y can a n d d o e s i m p a c t the q u a l i t y o f g r o u p decision m a k i n g [ G a l l u p e (1985), Lewis (1982), T u r o f f a n d Hiltz (1982), Steeb a n d J o h n s t o n (1981), A p p l e g a t e (1986)], a l t h o u g h results are n o t c o n s i s t e n t [Joyner a n d T u n s t a l l (1970)]. M o s t G D S S research is orie n t e d t o w a r d e x a m i n i n g the effects of c o m p u t e r s u p p o r t on g r o u p o u t c o m e s , t y p i c a l l y d e c i s i o n quality or g r o u p consensus. This p a p e r discusses a t e c h n i q u e t h a t focusses n o t on o u t c o m e s b u t on g r o u p process. T h e t e c h n i q u e is i n t e r a c t i o n a n a l y sis, a n d the p u r p o s e of this p a p e r is to review and illustrate its use in G D S S research. T h e e x a m p l e is f r o m an e x p e r i m e n t that i n v e s t i g a t e d how c o m p u t e r s u p p o r t affects influence p a t t e r n s in groups, b u t the p r i m a r y focus here is on the i n t e r a c t i o n analysis t e c h n i q u e as a tool for G D S S researchers. T h e p a p e r p r o c e e d s with a discussion of the i m p o r t a n c e of a n a l y z i n g i n t e r a c t i o n , followed b y a review o f i n t e r a c t i o n analysis schemes from p r i o r literature. T h e b a c k g r o u n d for the illustrative exa m p l e is p r o v i d e d , a n d the G D S S used in the r e s e a r c h is d e s c r i b e d . D e t a i l s of the i n t e r a c t i o n a n a l y s i s s c h e m e are then p r e s e n t e d , i n c l u d i n g how the c o d i n g was done, w h a t issues arose a b o u t the coding, a n d w h a t d a t a resulted.
Management Information Systems Quarterly and in the Proceedings of the Hawaii International Conference on System Sciences.
* Partial funding for this research was provided by the NCR Corporation, the General Electric Corporation, the MIS Research Center at the University of Minnesota, and the Graduate School of the University of Minnesota. The author thanks Gerardine DeSanctis and Scott Poole for their helpful input to this work. North-Holland Decision Support Systems 5 (1989) 233-241
2. Why Analyze Interaction? R e s e a r c h has d e m o n s t r a t e d the existence Of e m p i r i c a l l y r e d u n d a n t p a t t e r n s of h u m a n interaction [Fisher, D r e c k s e l a n d W e r b e l (1979, p. 3)]. A c c e p t i n g t h a t h u m a n c o m m u n i c a t i o n is pat+ terned,
0167-9236/89/$3.50 © 1989, Elsevier Science Publishers B.V, (North-Holland)
234
L Ztgurs / Interaction Analysis in GDSS Research
... the issue has shifted to whether the analytical system used to observe the interaction reveals significant characteristics of human communication, and whether the system adds to our knowledge of how communication functions to develop and maintain social relationships. A GDSS is a social technology that impacts the pattern of interaction in a group [Poole and DeSanctis (1987)]. As such, the effects of computer support on that interaction process are of interest. With few exceptions [Applegate (1986), Turoff and Hiltz (1982)], GDSS researchers who have looked at group process have done so in terms of participation by counting the number of remarks made, or by self-report measures such as satisfaction [Gallupe (1985), Watson (1987)] or perceived group member behavior [Watson (1987)]. Very few studies have essayed an in-depth, micro-level analysis of group process. Researchers in group dynamics have developed and validated a variety of instruments for detailed interaction analysis, and it seems both prudent and efficient to investigate the application of those schemes in GDSS research where the goal is to understand the impact of technology on the interaction process.
3. A Selected Review of Systems for Interaction Analysis Nearly forty years ago, Bales (1950) developed the now-classic Interaction Process Analysis (IPA) s y s t e m - the first extensive attempt to observe group interaction directly and systematically. That system dominated group interaction research for decades [McGrath (1984)]. The IPA is based upon the theory that groups are continually faced with both task-oriented concerns (focussed on accomplishing the group's task) and social-emotional concerns (centered on maintaining inter-member relations). The IPA itself consists of twelve comprehensive and mutually exclusive categories, as shown in table 1. The categories are linked to Bales' theory of how group interaction develops, through the phases of orientation, evaluation, and control, with constant interplay of task and social-emotional concerns. McGrath (1984) calls the IPA a generic process observation system, with applications to a wide variety of groups. McGrath
Table 1 Categories of Bales' Interaction Process Analysis System. a Positive Social-Emotional/Expressive 1. Shows solidarity 2. Shows tension release 3. Agrees Active Task/Instrumental 4. Gives suggestion 5. Gives opinion 6. Gives orientation Passive Task/Instrumental 7. Asks for orientation 8. Asks for opinion 9. Asks for suggestion Negative Social-Emotional/Expressive 10. Disagrees 11. Shows tension 12. Shows antagonism a From McGrath (1984, p. 141).
notes, however, that data derived from the IPA is of little use in testing theories of interaction other than that of Bales, since Bales' categories and theory are closely linked. In the decades since Bales' pioneering work, a great variety of interaction analysis schemes have been devised. Fisher, Drecksel, and Werbel (1979) developed the Social Information Processing Analysis (SIPA) scheme, based on modern systems theory and intended for use in as wide a variety of social situations as possible. The premise is that information "is the central organizing element of a social system, and communication, or information exchange, is how parts organize into a system" (p. 9). In SIPA, a verbal act is coded in each of four dimensions: (1) source of information, (2) time orientation, (3) information assembly rules, and (4) equivocality reduction. Each dimension contains subcategories for coding, e.g., the dimension of information assembly rules is coded as either (1) generation of information, (2) examining information for selection, or (3) retaining information. Proposed applications of SIPA patterns include examination of leader behavior, normative functioning of social information processing, and the role of social conflict. While the Bales and Fisher coding systems were developed for broad research agendas, other systems were designed for more specific purposes. For example, several coding systems analyze leadership communication. The Carter, Haythorn,
L Zigurs / Interaction Analysis in GDSS Research
Meirowitz, and Lanzetta (1951) scheme codes leader interaction into 72 categories. Mortensen (1964) codes attempted leadership communication and the group's responses. Wood's (1977) system codes how leader comments are structured. Sims and Manz (1984) have a coding system of fourteen categories designed for dyads with a formal leader. Rogers and Farace (1975) code relational control in order to determine dominance in dyadic relationships. Poole and Hirokawa (1986) cite additional coding systems for the classification of orientation and processual statements [Gouran (1969)], process disrupting acts [Leathers (1969)], decision proposals [Fisher (1970)], and the tenor of working relationships [Poole (1983)]. This review of interaction coding systems is by no means exhaustive, but it does indicate the wide variety of types of verbal acts that have been of interest to coders. Interaction analysis schemes have been established as an important research tool for studying communication processes [Poole and Folger (1981)]. With the growing interest in the effects of computer support on the process of group decision making, the technique of interaction analysis has potential as a meaningful addition to the toolkit of the GDSS researcher. Very few studies in the GDSS area have used the interaction analysis method. Applegate (1986) designed a structured observation scheme based on Bales' IPA. Called the Electronic Brainstorming Interaction Analysis instrument, Applegate's version consisted of three categories: task-orlentation, technology orientation, and social-orientation. Within the technology-orientation category, three types of behaviors were coded: (1) entering comments on the system, (2) reading the screen display, and (3) waiting for the next screen display. It is of interest that Applegate made an explicit attempt to incorporate computer use acts into the interaction analysis scheme. Turoff and Hiltz (1982) used Bales' interaction process analysis coding system to analyze the communication differences between computerconferencing versus face-to-face groups. These researchers were looking for patterns of usage among Bales' categories to understand how the computer communication affected a group's process. More common among GDSS researchers who study group process is a variety of individually-devised means of assessing level of participation, usually through counting the number of comments
235
in a group. Beauclair (1987) studied process (in addition to outcomes) by measuring quality of interaction and proclivity to participate. Proclivity to participate was measured as a tally of the number of substantive remarks per member. Quality of interaction was measured by raters who scored each individual's behavior with respect to goal-orientation, relevance, amplification, activity, information use, and critical examination. Siegel, Dubrovsky, Kiesler, and McGuire (1986) measured both participation and a construct called communication efficiency, the latter consisting of time to consensus, number of remarks, percent of task-oriented remarks, and percent of decision proposals. Remarks were classified into the categories of: (1) task-oriented, (2) decision proposals, and (3) uninhibited behavior. Gallupe (1985) supplemented his measures of group outcomes with process measures of number of alternatives generated, amount of member participation (a count of comments), and member satisfaction. The coding of social interaction permits a more complex analysis of process than what has previously been done in most GDSS research. Several questions arise, however. How easily do existing schemes apply to computer-supported groups? What types of behaviors might one want to code? What are the costs and benefits of interaction analysis in GDSS research, and can the expense of such a method be justified? The next section addresses these questions by illustrating an experience with interaction analysis in the context of GDSS research.
4. An Example of Interaction Analysis in GDSS Research The author conducted a study in which interaction analysis was one of the primary means of data capture [Zigurs (1987)]. The objective of this research was to study the impact of computer support on influence behavior in small groups of decision makers. Influence behavior in groups was defined as individual verbal or other acts that attempted to affect or determine the course of group behavior. Influence processes can impact both group process gains and process losses by affecting the group's drive, its cohesion, its goal selection, and its goal attainment (Bass, 1981). Influence is thus an important variable of study
236
GDSS Technology Other Situational Variables
L Zigurs / Interaction Analysis in GDSS Research
--\ /
~
Group Interaction Process
~/
Group Outcomes
f
Fig. 1. A General Model of the Relationship of GDSS Technology to Other Group Variables [adapted from Poole and DeSanctis (1987)].
for GDSS research. The rationale and background to the research are described below, to set the context for the interaction analysis. A group decision support system provides a channel through which interaction takes place, i.e., an electronic communication channel. Other situational factors impact a group's interaction pattern as well, factors such as the task, the group's structure, and member characteristics. The process of group interaction results in outcomes such as group performance or member satisfaction. Fig. 1 is a simplified linear representation of these relationships. Influence behaviors occur through group interaction. More specifically, influence attempts are made through available channels of communication. These channels may be verbal, nonverbal, or written. The introduction of an electronic communication channel via a GDSS provides a new and different means by which members may attempt to influence one another. This study focussed on the interaction process itself, that is, the effect of GDSS technology on influence activity during group interaction, rather than off group outcomes. One important way that individuals attempt to influence each other in groups is through the use of procedural statements [Putnam (1981)]. Five specific categories of procedural statements were of interest here: (1) initiation behavior, which deals with setting or initiating agendas; (2) goal-oriented behavior, concerned with group goals or group jurisdiction; (3) integrative behavior, which summarizes and integrates contributions of others; (4) implementation behavior, representing an action orientation toward getting the task done; and (5) process behavior, dealing with procedural movement of the group. The interaction analysis technique captured group members' use of these categories of influence.
Resulting data allowed an examination of types and patterns of influence behavior.
4.1. Conduct of the Experiment and the GDSS Used The experimental manipulation was type of technological support, with two kinds of groups: (1) GDSS groups (with computer support), and (2) manual groups (with no computer support but an equivalent paper-and-pencil agenda). All groups met in a conference-room type of environment. The groups were assigned an intellective task [McGrath (1984)] that required them to evaluate a set of applicants to an international studies program. The research setting for the GDSS groups consisted of a specially-designed and equipped "decision lab". The room contained a conference table, four chairs with a private computer terminal next to each chair, and a large-screen monitor at the end of the table that acted as the public screen. A camera with a wide-angle lens was positioned unobtrusively in one corner of the room. A chair for the experimenter was positioned in the opposite corner, out of the way of group interaction. The role of the experimenter was strictly to help with questions about the use of the system; the meetings were not facilitated in the sense that the experimenter intervened in or directed the group interaction process. The subjects were novice users of the GDSS, but the training provided was sufficient for relatively error-free operation of the system. The GDSS used in this research was designed and developed by a project team at the University of Minnesota. The programs, collectively called SAMM (Software Aided Meeting Management), are written in the C programming language and run under the Unix operating system. The GDSS consists of two main programs, one running on the public screen and the other on each individual group member's terminal. The two programs communicate with one another through the message exchange facility of Unix. The private program, running at each individual group member's termi-
L Zigurs / Interaction Analysis in GDSS Research
237
AGENDA I. 2. 3. 4. 5. 6. 7. 8.
Define/View Problem a. E n t e r / V i e w Define/View Selection Criteria Define/View Alternatives Rate Alternatives Rank Alternatives V o t e or S t r a w p o l l on A l t e r n a t i v e s Define Decision Conclude Meeting
Comments
Fig. 2. Main Menu for SAMM (SoftwareAided Meeting Management).
nal, collects the keyboard input (messages) of each member and sends it to the public program. Depending upon the type of message, the public program may store the message data, display group information on the public screen, or transmit information to a group member. The private program initially presents to each user a menu of "generic" group agenda items. The choice of which features to include was based on the literature of groups and group effectiveness. For example, research has demonstrated a positive relationship between group effectiveness and attempts to analyze the problem [Hirokawa (1983)]. Thus, one option on the SAMM menu is problem definition, which focusses group attention on examining and agreeing on the problem being addressed in the meeting. Similarly, other features support generation and evaluation of alternative solutions to the problem. To use the system, each individual selects an agenda item and is taken to sub menus that execute appropriate functions for that agenda item. This menu-driven approach makes SAMM relatively quick to learn and easy to use. Fig. 2 reproduces the main agenda of the SAMM software. 4.2. The Interaction Analysis Scheme Working from audiotapes and written transcriptions of the group interaction, coders classified every verbal act of each group member into a ten-category system for coding interaction. The coding system was developed and validated by Putnam (1981) and revised for this research. Its purpose is to code procedural messages during ongoing group interaction. Procedural messages are statements that guide the work of the group,
including what the group is doing, where it is going, and what it should do. Procedural communication "occupies a substantial portion of group talk time, performs vital meta-message functions, and serves as indices of leadership emergence and decision-making processes" [Putnam (1981, p. 332)]. The Putnam Procedural Messages coding system was chosen over other schemes for several reasons. The objective of the interaction analysis was to capture the effect of a technological intervention on overall influence behaviors in relatively leaderless groups. Many of the previously cited coding systems apply to formal leadership situations, and would have required substantial revision to make them relevant. Other coding systems were eliminated as being too general, in that they might not capture the particular effects of a GDSS in groups. A GDSS is likely to have the most impact in how groups structure their work, thus procedural messages are particularly important to capture in such an analysis. Putnam's first five categories record five different types of procedural messages, while the latter five categories record different levels of continuing interaction, including one category for non taskrelated discussion. For this research, every verbal act was coded into one of Putnam's ten categories, but it is the procedural messages (the first five categories) that form the basis for influence measurement. Table 2 shows Putnam's ten categories, with examples of messages in each category. 1 One of the first decisions in interaction coding is to define the unit to be coded. This research 1 A more detailed description of the categories, including instructions for coders, is available from the author on request.
238
L Zigurs / Interaction Analysis in G D S S Research
Table 2 Putnam Coding Scheme for Procedural Messages in Group Interaction [as revised in Zigurs (1987)]. Catg 1 Initiation messages - requesting or suggesting deadlines, agendas, or lists of activities. "Let the meeting begin." "'Let's rank these options now." Catg 2 Goal-oriented messages - requesting or making statements about group goals or group jurisdiction. "We can only pick 3 applicants." "So our task in this meeting is..." Catg 3 Integrative messages - s u m m a r i z i n g and integrating contributions. "So we all agree that Mike should be ado mitted?" "I think we've been saying the same thing here..." Catg 4 Implementation messages - s u g g e s t i n g or requesting division of labor or implementation of a course of action in getting the task done. "Let me write our ideas on the flipchart." "I gave this applicant a score of 40." Catg 5 Process messages - requesting our suggesting procedural direction. "What should we do next?" "Has everyone voted on the last applicant?" Catg 6 Topic change messages - changing task-related topic of discussion by introducing an abstract label or general heading. "The criteria for deciding among these applicants should not include their gender." Catg 7 Topic continuation messages - continues task-related discussion initiated by a general heading or by an agenda category; simple confirmation or acknowledgement. "That makes sense to me." Catg 8 Detail change messages - c h a n g i n g task-related discussion by switching topics via jumping from specific detail to specific detail. "We've been talking about Dorothy's score on the self-concept test but I think the fact that she has already travelled abroad is more relevant." Catg 9 Non task messages - messages which digress from the group task to socio-emotional issues. "' How about those Twins?" Catg 10 Talkovers - Initiates an interruption or multiple conversations such that the content of the interaction is difficult to decipher.
coded the act, i.e., each c o n t i n u o u s u t t e r a n c e b y a n i n d i v i d u a l [Bales (1950)]. O t h e r o p t i o n s are the t u r n [Rogers a n d F a r a c e (1975)], a t h e m e [Larson (1968)], or the interact [Fisher (1970)]. A second decision is what subset of available data to code as a representative sample. Since i n t e r a c t i o n analysis requires a heavy time i n v e s t m e n t a n d c o n s i d e r a b l e
expense, researchers are generally forced to limit c o d i n g in some way. O n e o p t i o n is to code selected segments of i n t e r a c t i o n across all group meetings. Since group meetings typically differ in length a n d character of i n t e r a c t i o n , the o p t i o n used here was to choose a subset of groups a n d code the entire meeting. A total of thirty-two groups were coded. T w o coders worked together o n the verbal coding u n t i l they achieved reliability with each other, a n d t h e n each p e r s o n i n d e p e n d e n t l y coded app r o x i m a t e l y half each of the r e m a i n i n g transcripts. T h e t r a i n i n g process b e g a n with a series of meetings b e t w e e n the two coders, d u r i n g which they discussed the m e a n i n g of each p r o c e d u r a l category. A cycle t h e n e n s u e d i n which the coders worked separately o n a given section of a transcript, t h e n m e t to resolve differences, then coded the next section, m e t again, a n d so on, until reliability was achieved. I n t e r - r a t e r reliabilities ranged b e t w e e n 75% to 85%, a r a n g e that prior researchers c o n s i d e r to be acceptable. 2 P u t n a m ' s basic categories were retained, with m i n o r a d d i t i o n s of c o n t e x t sensitive items to several categories. F o r instance, category 5 consists of messages that request or suggest procedural direction. Messages a b o u t the use of the technology (whether c o m p u t e r or m a n u a l ) were i n c l u d e d i n that category. 4.3. C o d i n g f o r U s a g e o f t h e T e c h n o l o g y
A n i m p o r t a n t e x t e n s i o n of the P u t n a m scheme in this research was to code i n f l u e n t i a l uses of the technology p r o v i d e d to groups - c o m p u t e r technology i n the case of G D S S groups, a n d papera n d - p e n c i l devices in the case of m a n u a l groups. T h e e x p e c t a t i o n was that i n d i v i d u a l s would use the technology to try to i n f l u e n c e the group. A G D S S c a n be used to i n f l u e n c e a group b y changing the c u r r e n t display o n the p u b l i c screen, deleting a n item (a p r o b l e m , criterion, or alternative definition) from the system, or using the group c o m m u n i c a t i o n feature. T h e n u m b e r of times each
2 Berelson (1952, p. 172) surveyed thirty studies and reported a "range of correlation coefficients between 0.78 and 0.99, with a concentration at about 0.90, and a range of percentage agreements between 68% and 96% with a concentration over 90%." Thomdike (1950) and Lasswell et al. (1952) state that 75% to 85% agreement is sufficient to establish reliability when the error is not systematic.
I. Zigurs / Interaction Analysis in GDSS Research
group member undertook one of those three behaviors was counted via an analysis of the computer log files. In manual groups, influence behaviors that corresponded to the three that were measured in GDSS groups were counted through observation of the videotapes. The manual behaviors consisted of changing the currently displayed page of the flipchart, crossing out material on the flipchart, and writing comments on the flipchart. These influence attempts via use of the technology (both GDSS and manual) were combined with the verbal influence attempts, resulting in a total count of influence attempts by each group member. The interaction analysis resulted in counts of statements within each of the first five categories of Putnam's scheme (representing different types of verbal influence behavior), counts on total amount of talk, and counts of influence behavior via use of the technology, for each individual in a group. 3
5. Conclusions and Recommendations
Interaction analysis is an expensive and time-consuming research method. Videotapes must be transcribed to written transcripts, multiple coders must be trained to a reliable level, and the coding itself must be done and periodically crosschecked for consistency. The training and coding processes are intensive and the time investment heavy. It may take four to six weeks to attain reliability on a previously validated coding scheme such as the one used here. Issues of consistency, interpretation and validity associated with the use or development of systems for coding social interaction need to be considered as well [see Folger, Hewes and Poole (1984)]. The question is whether the resource investment required for interaction analysis is worthwhile for GDSS research. The use of Putnam's procedural messages coding scheme in this research contributed in several ways to an understanding of how computer support impacts small groups of decision makers. The capability to examine specific categories of verbal behavior, in addition to overall verbal participation, provided a richer ground for interpreting
3 See Zigurs, Poole. and DeSanctis (1988) for a discussion of the findings.
239
GDSS effects. For instance, the expectation that computer-supported groups would spend more time on processual concerns was borne out by the interaction analysis. The types of messages used by GDSS groups differed significantly from the types used by manual groups, showing that the technology affected the process by which groups reached a decision. This particular result was not a surprising one, but the interaction analysis method provided a means for empirical confirmation. Usage of the computer system itself adds a level of complexity to coding interaction analysis in that one generally wants to capture system usage in addition to verbal acts. The present research coded use of the computer system by recording instances where individuals directed or redirected group attention through the system. Extensions of the interaction analysis in this area might include a more detailed examination of the extent to which verbal versus electronic channels are used and whether different types of channels are used for different purposes or in different ways. Such research would contribute to realizing the full potential of a GDSS as a communication channel in interaction with other channels. This example used the interaction analysis method to look exclusively at influence attempts and patterns in groups. Other researchers might benefit from using the method in a number of ways. The interaction analysis technique has seen considerable use in group behavior research, and there exists a literature on patterns in "naturallyinteracting" groups (i.e., groups without technological support). These prior findings represent a baseline on which GDSS researchers can build in their own analyses. In addition, interaction analysis can reveal problems with which groups strugg l e - problems that might be alleviated through technological support. This method also reveals links between behaviors and outcomes, for instance through examination of patterns of verbal behavior and their relationship to decision quality. The author's opinion is that interaction analysis brings a meaningful new dimension to the GDSS researcher's toolbox, in spite of its high resource cost. Few other techniques provide a micro-level view of group process. A simple i n p u t - o u t p u t model of decision making, which treats the process itself as a black box, is not sufficient to explain the effects of computer support [Todd and Benbasat (1987)]. The ultimate objective of pro-
240
L Zigurs / Interaction Analysis in GDSS Research
viding technological support is to enhance the process and products of group decision making. Interaction analysis helps to open the "black box" of process, and thus has the potential to contribute significantly to our understanding of the impacts of computer support on groups.
References Applegate, L. "Idea Management in Organization Planning." Unpublished doctoral dissertation, University of Arizona, 1986. Bales, R.F., "A Set of Categories for the Analysis of Small Group Interaction." American Sociological Review, 1950, 15, 257-263. Bass, B.M. Stogdill's Handbook of Leadership. New York: The Free Press, 1981. Beauclair, R. "An Experimental Study of the Effects of GDSS Process Support Applications on Small Group Decision Making." Unpublished doctoral dissertation, Indiana University, 1987. Berelson, B. Content Analysis in Communication Research. Glencoe, Illinois: The Free Press, 1952. Carter, L., Haythorn, W., Meirowitz, B., and Lanzetta, J. "A Note on a New Technique of Interaction Recording." Journal of Abnormal and Social Psychology, 1951, 46, 258-260. Fisher, B.A. "Decision Emergence: Phases in Group Decisision-Making." Speech Monographs, 1970, 37, 53-66. Fisher, B.A., Drecksel, G.L. and Werbel, W. "Social Information Processing Analysis (SIPA): Coding Ongoing Human Communication." Small Group Behavior, 1979, 10, 3-21. Folger, J.P., Hewes, D.E., and Poole, M.S. "Coding Social Interaction." In Dervin, B. and Voight, M. (eds.). Progress in Communication Sciences. Vol. 4, 1984. Gallupe, R.B. "The Impact of Task Difficulty on the Use of a Group Decision Support System." Unpublished doctoral dissertation, University of Minnesota, 1985. Gouran, D.S. "Variables Related to Consensus in Group Discussions of Questions of Policy." Speech Monographs, 1969, 36, 387-391. Hirokawa, R. "Group Communication and Problem-Solving Effectiveness II: An Exploratory Investigation of Procedural Functions." Western Journal of Speech Communication, 1983, 47, 59-74. Joyner, R. and Tunstall, K. "Computer Augmented Organizational Problem Solving." Management Science, 1970, 17 (4), B-212-B-225. Larson, C.U. "Leadership Emergence and Attention Span: A Content Analysis of the Time-Devoted-to-Themes Discussed in the Task-oriented Small Group." Unpublished doctoral dissertation, University of Minnesota, 1968. Lasswell, H.D., Lerner, D., and the Sola Pool, I. The comparative study of symbols. Palo Alto: Stanford University Press, 1952. Leathers, D. "Process Disruption and Measurement in Small
Group Communication." Quaterly Journal of Speech, 1969, 55, 287-300. Lewis, L.F. "Facilitator: A Microcomputer Decision Support System for Small Groups." Unpublished doctoral dissertation, University of Louisville, 1982. McGrath, J.E. Groups: Interaction and Performance. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1984. Mortensen, C. "A Content Analysis of Leadership Communication in Small, Task-Oriented Discussion Groups." Unpublished master's thesis, University of Minnesota, 1964. Poole, M.S. "Decision Development in Small Groups II: A Study of Multiple Sequences in Decision Making." Communication Monographs, 1983, 50, 321-341. Poole, M.S. and DeSanctis, G. "Group Decision Making and Group Decision Support Systems: A 3-Year Plan for the GDSS Research Project," working paper, MISRC-WP-8802, MIS Research Center, University of Minnesota, 1987. Poole, M.S. and Folger, J. "A Method for Establishing the Representational Validity of Interaction Coding Systems: Do We See What They See?" Human Communication Research, 1981, 8, 26-42. Poole, M.S. and Hirokawa, R.Y. "Communication and Group Decision Making: A Critical Assessment." In Hirokawa, R.Y. and Poole, M.S. (eds.), Communication and Group Decision-Making. Beverly Hills: Sage Publications, 1986, 15-31. Putnam, L.L. "Procedural Messages and Small Group Work Climates: A Lag Sequential Analysis." In Burgoon, M. (ed.), Communication Yearbook 5. New Brunswick, N.J.: Transaction Books, 1981, 331-350. Rogers, L.E. and Farace, R.V. "Analysis of Relational Communication in Dyads: New Measurement Procedures." Human Communication Research, 1975, 1, 222-239. Siegel, J. Dubrovsky, V., Kiesler, S., and McGuire, T.W. "Group Processes in Computer-Mediated Communication." Organizational Behavior and Human Decision Processes, 1986, 37, 157-187. Sims, H.P., Jr. and Manz, C.C. "Observing Leader Verbal Behavior: Toward Reciprocal Determinism in Leadership Theory." Journal of Applied Psychology, 1984, 69 (2), 222-232. Steeb, R. and Johnston, S.C. "A Computer-Based Interactive System for Group Decision making." IEEE Transactions on Systems, Man, and Cybernetics, August 1981, SMC-11 (8), 5a4-552. Thorndike, R.L. "Reliability." In Lindquist, E.G. (ed.). Educational Measurement. Washington, D.C.: American Council on Education, 1950. Todd, P. and Benbasat, I. "Process Tracing Methods in Decision Support System Research: Exploring the Black Box," MIS Quarterly, 1987, 11 (4), 493-512. Turoff, M. and Hiltz, S. "Computer Support for Group Versus Individual Decisions." IEEE Transactions on Communications, 1982, 30 (1), 82-90. Watson, R.W. "A Study of Group Decision Support System Use in Three and Four-Person Groups for a Preference Allocation Decision." Unpublished doctoral dissertation, University of Minnesota, 1987.
L Zigurs / Interaction Analysis in GDSS Research Wood, J. "Leading in Purposive Discussions: A Study of Adaptive Behavior." Communication Monographs, 1977, 44, 152-165. Zigurs, I. "The Effect of Computer Support on Influence Attempts and Patterns in Small Group Decision-Making."
241
Unpublished doctoral dissertation, University of Minnesota, 1987. Zigurs, I., Poole, M.S., and DeSanctis, G. "A Study of Influence in Computer-Mediated Group Decision Making," MIS Quarterly. 1988, 12(4), 625-644.
Interaction Analysis in GDSS Research: Description of an Experience and Some Recommendations * 1. Introduction
Ilze Z I G U R S College of Business, University of Colorado, Boulder, CO 80309, USA
An emerging body of research in group decision support systems (GDSS) provides evidence that computer technology impacts the quality of decision making in groups. Most GDSS research focuses on the effects of computer support on outcomes, such as decision quality, commitment, or consensus. This article discusses the analysis of group process, rather than outcomes, via the technique of interaction analysis. Interaction analysis provides a micro-level examination of group behavior by the use of coding schemes for verbal interaction. The application and extension of interaction analysis in GDSS research is reviewed and illustrated. Keywords: Group Decision Support System; Interaction Anal-
ysis.
ilze Zigurs is an Assistant Professor of Management Information Systems in the Management Science/Information Systems Division of the University of Colorado, Boulder. She holds a Ph.D. in management information systems from the University of Minnesota. Dr. Zigurs' primary research interest is in the use and impact of computer and communications technologies in the support of collaborative work. Her research has been published in the
T h e g r o w t h of a p p l i c a t i o n s of c o m p u t e r techn o l o g y in o r g a n i z a t i o n s includes an i n c r e a s i n g interest in c o m p u t e r - b a s e d s u p p o r t for decision m a k i n g in groups. A n e m e r g i n g b o d y of research in g r o u p d e c i s i o n s u p p o r t systems ( G D S S ) p r o vides evidence that c o m p u t e r t e c h n o l o g y can a n d d o e s i m p a c t the q u a l i t y o f g r o u p decision m a k i n g [ G a l l u p e (1985), Lewis (1982), T u r o f f a n d Hiltz (1982), Steeb a n d J o h n s t o n (1981), A p p l e g a t e (1986)], a l t h o u g h results are n o t c o n s i s t e n t [Joyner a n d T u n s t a l l (1970)]. M o s t G D S S research is orie n t e d t o w a r d e x a m i n i n g the effects of c o m p u t e r s u p p o r t on g r o u p o u t c o m e s , t y p i c a l l y d e c i s i o n quality or g r o u p consensus. This p a p e r discusses a t e c h n i q u e t h a t focusses n o t on o u t c o m e s b u t on g r o u p process. T h e t e c h n i q u e is i n t e r a c t i o n a n a l y sis, a n d the p u r p o s e of this p a p e r is to review and illustrate its use in G D S S research. T h e e x a m p l e is f r o m an e x p e r i m e n t that i n v e s t i g a t e d how c o m p u t e r s u p p o r t affects influence p a t t e r n s in groups, b u t the p r i m a r y focus here is on the i n t e r a c t i o n analysis t e c h n i q u e as a tool for G D S S researchers. T h e p a p e r p r o c e e d s with a discussion of the i m p o r t a n c e of a n a l y z i n g i n t e r a c t i o n , followed b y a review o f i n t e r a c t i o n analysis schemes from p r i o r literature. T h e b a c k g r o u n d for the illustrative exa m p l e is p r o v i d e d , a n d the G D S S used in the r e s e a r c h is d e s c r i b e d . D e t a i l s of the i n t e r a c t i o n a n a l y s i s s c h e m e are then p r e s e n t e d , i n c l u d i n g how the c o d i n g was done, w h a t issues arose a b o u t the coding, a n d w h a t d a t a resulted.
Management Information Systems Quarterly and in the Proceedings of the Hawaii International Conference on System Sciences.
* Partial funding for this research was provided by the NCR Corporation, the General Electric Corporation, the MIS Research Center at the University of Minnesota, and the Graduate School of the University of Minnesota. The author thanks Gerardine DeSanctis and Scott Poole for their helpful input to this work. North-Holland Decision Support Systems 5 (1989) 233-241
2. Why Analyze Interaction? R e s e a r c h has d e m o n s t r a t e d the existence Of e m p i r i c a l l y r e d u n d a n t p a t t e r n s of h u m a n interaction [Fisher, D r e c k s e l a n d W e r b e l (1979, p. 3)]. A c c e p t i n g t h a t h u m a n c o m m u n i c a t i o n is pat+ terned,
0167-9236/89/$3.50 © 1989, Elsevier Science Publishers B.V, (North-Holland)
234
L Ztgurs / Interaction Analysis in GDSS Research
... the issue has shifted to whether the analytical system used to observe the interaction reveals significant characteristics of human communication, and whether the system adds to our knowledge of how communication functions to develop and maintain social relationships. A GDSS is a social technology that impacts the pattern of interaction in a group [Poole and DeSanctis (1987)]. As such, the effects of computer support on that interaction process are of interest. With few exceptions [Applegate (1986), Turoff and Hiltz (1982)], GDSS researchers who have looked at group process have done so in terms of participation by counting the number of remarks made, or by self-report measures such as satisfaction [Gallupe (1985), Watson (1987)] or perceived group member behavior [Watson (1987)]. Very few studies have essayed an in-depth, micro-level analysis of group process. Researchers in group dynamics have developed and validated a variety of instruments for detailed interaction analysis, and it seems both prudent and efficient to investigate the application of those schemes in GDSS research where the goal is to understand the impact of technology on the interaction process.
3. A Selected Review of Systems for Interaction Analysis Nearly forty years ago, Bales (1950) developed the now-classic Interaction Process Analysis (IPA) s y s t e m - the first extensive attempt to observe group interaction directly and systematically. That system dominated group interaction research for decades [McGrath (1984)]. The IPA is based upon the theory that groups are continually faced with both task-oriented concerns (focussed on accomplishing the group's task) and social-emotional concerns (centered on maintaining inter-member relations). The IPA itself consists of twelve comprehensive and mutually exclusive categories, as shown in table 1. The categories are linked to Bales' theory of how group interaction develops, through the phases of orientation, evaluation, and control, with constant interplay of task and social-emotional concerns. McGrath (1984) calls the IPA a generic process observation system, with applications to a wide variety of groups. McGrath
Table 1 Categories of Bales' Interaction Process Analysis System. a Positive Social-Emotional/Expressive 1. Shows solidarity 2. Shows tension release 3. Agrees Active Task/Instrumental 4. Gives suggestion 5. Gives opinion 6. Gives orientation Passive Task/Instrumental 7. Asks for orientation 8. Asks for opinion 9. Asks for suggestion Negative Social-Emotional/Expressive 10. Disagrees 11. Shows tension 12. Shows antagonism a From McGrath (1984, p. 141).
notes, however, that data derived from the IPA is of little use in testing theories of interaction other than that of Bales, since Bales' categories and theory are closely linked. In the decades since Bales' pioneering work, a great variety of interaction analysis schemes have been devised. Fisher, Drecksel, and Werbel (1979) developed the Social Information Processing Analysis (SIPA) scheme, based on modern systems theory and intended for use in as wide a variety of social situations as possible. The premise is that information "is the central organizing element of a social system, and communication, or information exchange, is how parts organize into a system" (p. 9). In SIPA, a verbal act is coded in each of four dimensions: (1) source of information, (2) time orientation, (3) information assembly rules, and (4) equivocality reduction. Each dimension contains subcategories for coding, e.g., the dimension of information assembly rules is coded as either (1) generation of information, (2) examining information for selection, or (3) retaining information. Proposed applications of SIPA patterns include examination of leader behavior, normative functioning of social information processing, and the role of social conflict. While the Bales and Fisher coding systems were developed for broad research agendas, other systems were designed for more specific purposes. For example, several coding systems analyze leadership communication. The Carter, Haythorn,
L Zigurs / Interaction Analysis in GDSS Research
Meirowitz, and Lanzetta (1951) scheme codes leader interaction into 72 categories. Mortensen (1964) codes attempted leadership communication and the group's responses. Wood's (1977) system codes how leader comments are structured. Sims and Manz (1984) have a coding system of fourteen categories designed for dyads with a formal leader. Rogers and Farace (1975) code relational control in order to determine dominance in dyadic relationships. Poole and Hirokawa (1986) cite additional coding systems for the classification of orientation and processual statements [Gouran (1969)], process disrupting acts [Leathers (1969)], decision proposals [Fisher (1970)], and the tenor of working relationships [Poole (1983)]. This review of interaction coding systems is by no means exhaustive, but it does indicate the wide variety of types of verbal acts that have been of interest to coders. Interaction analysis schemes have been established as an important research tool for studying communication processes [Poole and Folger (1981)]. With the growing interest in the effects of computer support on the process of group decision making, the technique of interaction analysis has potential as a meaningful addition to the toolkit of the GDSS researcher. Very few studies in the GDSS area have used the interaction analysis method. Applegate (1986) designed a structured observation scheme based on Bales' IPA. Called the Electronic Brainstorming Interaction Analysis instrument, Applegate's version consisted of three categories: task-orlentation, technology orientation, and social-orientation. Within the technology-orientation category, three types of behaviors were coded: (1) entering comments on the system, (2) reading the screen display, and (3) waiting for the next screen display. It is of interest that Applegate made an explicit attempt to incorporate computer use acts into the interaction analysis scheme. Turoff and Hiltz (1982) used Bales' interaction process analysis coding system to analyze the communication differences between computerconferencing versus face-to-face groups. These researchers were looking for patterns of usage among Bales' categories to understand how the computer communication affected a group's process. More common among GDSS researchers who study group process is a variety of individually-devised means of assessing level of participation, usually through counting the number of comments
235
in a group. Beauclair (1987) studied process (in addition to outcomes) by measuring quality of interaction and proclivity to participate. Proclivity to participate was measured as a tally of the number of substantive remarks per member. Quality of interaction was measured by raters who scored each individual's behavior with respect to goal-orientation, relevance, amplification, activity, information use, and critical examination. Siegel, Dubrovsky, Kiesler, and McGuire (1986) measured both participation and a construct called communication efficiency, the latter consisting of time to consensus, number of remarks, percent of task-oriented remarks, and percent of decision proposals. Remarks were classified into the categories of: (1) task-oriented, (2) decision proposals, and (3) uninhibited behavior. Gallupe (1985) supplemented his measures of group outcomes with process measures of number of alternatives generated, amount of member participation (a count of comments), and member satisfaction. The coding of social interaction permits a more complex analysis of process than what has previously been done in most GDSS research. Several questions arise, however. How easily do existing schemes apply to computer-supported groups? What types of behaviors might one want to code? What are the costs and benefits of interaction analysis in GDSS research, and can the expense of such a method be justified? The next section addresses these questions by illustrating an experience with interaction analysis in the context of GDSS research.
4. An Example of Interaction Analysis in GDSS Research The author conducted a study in which interaction analysis was one of the primary means of data capture [Zigurs (1987)]. The objective of this research was to study the impact of computer support on influence behavior in small groups of decision makers. Influence behavior in groups was defined as individual verbal or other acts that attempted to affect or determine the course of group behavior. Influence processes can impact both group process gains and process losses by affecting the group's drive, its cohesion, its goal selection, and its goal attainment (Bass, 1981). Influence is thus an important variable of study
236
GDSS Technology Other Situational Variables
L Zigurs / Interaction Analysis in GDSS Research
--\ /
~
Group Interaction Process
~/
Group Outcomes
f
Fig. 1. A General Model of the Relationship of GDSS Technology to Other Group Variables [adapted from Poole and DeSanctis (1987)].
for GDSS research. The rationale and background to the research are described below, to set the context for the interaction analysis. A group decision support system provides a channel through which interaction takes place, i.e., an electronic communication channel. Other situational factors impact a group's interaction pattern as well, factors such as the task, the group's structure, and member characteristics. The process of group interaction results in outcomes such as group performance or member satisfaction. Fig. 1 is a simplified linear representation of these relationships. Influence behaviors occur through group interaction. More specifically, influence attempts are made through available channels of communication. These channels may be verbal, nonverbal, or written. The introduction of an electronic communication channel via a GDSS provides a new and different means by which members may attempt to influence one another. This study focussed on the interaction process itself, that is, the effect of GDSS technology on influence activity during group interaction, rather than off group outcomes. One important way that individuals attempt to influence each other in groups is through the use of procedural statements [Putnam (1981)]. Five specific categories of procedural statements were of interest here: (1) initiation behavior, which deals with setting or initiating agendas; (2) goal-oriented behavior, concerned with group goals or group jurisdiction; (3) integrative behavior, which summarizes and integrates contributions of others; (4) implementation behavior, representing an action orientation toward getting the task done; and (5) process behavior, dealing with procedural movement of the group. The interaction analysis technique captured group members' use of these categories of influence.
Resulting data allowed an examination of types and patterns of influence behavior.
4.1. Conduct of the Experiment and the GDSS Used The experimental manipulation was type of technological support, with two kinds of groups: (1) GDSS groups (with computer support), and (2) manual groups (with no computer support but an equivalent paper-and-pencil agenda). All groups met in a conference-room type of environment. The groups were assigned an intellective task [McGrath (1984)] that required them to evaluate a set of applicants to an international studies program. The research setting for the GDSS groups consisted of a specially-designed and equipped "decision lab". The room contained a conference table, four chairs with a private computer terminal next to each chair, and a large-screen monitor at the end of the table that acted as the public screen. A camera with a wide-angle lens was positioned unobtrusively in one corner of the room. A chair for the experimenter was positioned in the opposite corner, out of the way of group interaction. The role of the experimenter was strictly to help with questions about the use of the system; the meetings were not facilitated in the sense that the experimenter intervened in or directed the group interaction process. The subjects were novice users of the GDSS, but the training provided was sufficient for relatively error-free operation of the system. The GDSS used in this research was designed and developed by a project team at the University of Minnesota. The programs, collectively called SAMM (Software Aided Meeting Management), are written in the C programming language and run under the Unix operating system. The GDSS consists of two main programs, one running on the public screen and the other on each individual group member's terminal. The two programs communicate with one another through the message exchange facility of Unix. The private program, running at each individual group member's termi-
L Zigurs / Interaction Analysis in GDSS Research
237
AGENDA I. 2. 3. 4. 5. 6. 7. 8.
Define/View Problem a. E n t e r / V i e w Define/View Selection Criteria Define/View Alternatives Rate Alternatives Rank Alternatives V o t e or S t r a w p o l l on A l t e r n a t i v e s Define Decision Conclude Meeting
Comments
Fig. 2. Main Menu for SAMM (SoftwareAided Meeting Management).
nal, collects the keyboard input (messages) of each member and sends it to the public program. Depending upon the type of message, the public program may store the message data, display group information on the public screen, or transmit information to a group member. The private program initially presents to each user a menu of "generic" group agenda items. The choice of which features to include was based on the literature of groups and group effectiveness. For example, research has demonstrated a positive relationship between group effectiveness and attempts to analyze the problem [Hirokawa (1983)]. Thus, one option on the SAMM menu is problem definition, which focusses group attention on examining and agreeing on the problem being addressed in the meeting. Similarly, other features support generation and evaluation of alternative solutions to the problem. To use the system, each individual selects an agenda item and is taken to sub menus that execute appropriate functions for that agenda item. This menu-driven approach makes SAMM relatively quick to learn and easy to use. Fig. 2 reproduces the main agenda of the SAMM software. 4.2. The Interaction Analysis Scheme Working from audiotapes and written transcriptions of the group interaction, coders classified every verbal act of each group member into a ten-category system for coding interaction. The coding system was developed and validated by Putnam (1981) and revised for this research. Its purpose is to code procedural messages during ongoing group interaction. Procedural messages are statements that guide the work of the group,
including what the group is doing, where it is going, and what it should do. Procedural communication "occupies a substantial portion of group talk time, performs vital meta-message functions, and serves as indices of leadership emergence and decision-making processes" [Putnam (1981, p. 332)]. The Putnam Procedural Messages coding system was chosen over other schemes for several reasons. The objective of the interaction analysis was to capture the effect of a technological intervention on overall influence behaviors in relatively leaderless groups. Many of the previously cited coding systems apply to formal leadership situations, and would have required substantial revision to make them relevant. Other coding systems were eliminated as being too general, in that they might not capture the particular effects of a GDSS in groups. A GDSS is likely to have the most impact in how groups structure their work, thus procedural messages are particularly important to capture in such an analysis. Putnam's first five categories record five different types of procedural messages, while the latter five categories record different levels of continuing interaction, including one category for non taskrelated discussion. For this research, every verbal act was coded into one of Putnam's ten categories, but it is the procedural messages (the first five categories) that form the basis for influence measurement. Table 2 shows Putnam's ten categories, with examples of messages in each category. 1 One of the first decisions in interaction coding is to define the unit to be coded. This research 1 A more detailed description of the categories, including instructions for coders, is available from the author on request.
238
L Zigurs / Interaction Analysis in G D S S Research
Table 2 Putnam Coding Scheme for Procedural Messages in Group Interaction [as revised in Zigurs (1987)]. Catg 1 Initiation messages - requesting or suggesting deadlines, agendas, or lists of activities. "Let the meeting begin." "'Let's rank these options now." Catg 2 Goal-oriented messages - requesting or making statements about group goals or group jurisdiction. "We can only pick 3 applicants." "So our task in this meeting is..." Catg 3 Integrative messages - s u m m a r i z i n g and integrating contributions. "So we all agree that Mike should be ado mitted?" "I think we've been saying the same thing here..." Catg 4 Implementation messages - s u g g e s t i n g or requesting division of labor or implementation of a course of action in getting the task done. "Let me write our ideas on the flipchart." "I gave this applicant a score of 40." Catg 5 Process messages - requesting our suggesting procedural direction. "What should we do next?" "Has everyone voted on the last applicant?" Catg 6 Topic change messages - changing task-related topic of discussion by introducing an abstract label or general heading. "The criteria for deciding among these applicants should not include their gender." Catg 7 Topic continuation messages - continues task-related discussion initiated by a general heading or by an agenda category; simple confirmation or acknowledgement. "That makes sense to me." Catg 8 Detail change messages - c h a n g i n g task-related discussion by switching topics via jumping from specific detail to specific detail. "We've been talking about Dorothy's score on the self-concept test but I think the fact that she has already travelled abroad is more relevant." Catg 9 Non task messages - messages which digress from the group task to socio-emotional issues. "' How about those Twins?" Catg 10 Talkovers - Initiates an interruption or multiple conversations such that the content of the interaction is difficult to decipher.
coded the act, i.e., each c o n t i n u o u s u t t e r a n c e b y a n i n d i v i d u a l [Bales (1950)]. O t h e r o p t i o n s are the t u r n [Rogers a n d F a r a c e (1975)], a t h e m e [Larson (1968)], or the interact [Fisher (1970)]. A second decision is what subset of available data to code as a representative sample. Since i n t e r a c t i o n analysis requires a heavy time i n v e s t m e n t a n d c o n s i d e r a b l e
expense, researchers are generally forced to limit c o d i n g in some way. O n e o p t i o n is to code selected segments of i n t e r a c t i o n across all group meetings. Since group meetings typically differ in length a n d character of i n t e r a c t i o n , the o p t i o n used here was to choose a subset of groups a n d code the entire meeting. A total of thirty-two groups were coded. T w o coders worked together o n the verbal coding u n t i l they achieved reliability with each other, a n d t h e n each p e r s o n i n d e p e n d e n t l y coded app r o x i m a t e l y half each of the r e m a i n i n g transcripts. T h e t r a i n i n g process b e g a n with a series of meetings b e t w e e n the two coders, d u r i n g which they discussed the m e a n i n g of each p r o c e d u r a l category. A cycle t h e n e n s u e d i n which the coders worked separately o n a given section of a transcript, t h e n m e t to resolve differences, then coded the next section, m e t again, a n d so on, until reliability was achieved. I n t e r - r a t e r reliabilities ranged b e t w e e n 75% to 85%, a r a n g e that prior researchers c o n s i d e r to be acceptable. 2 P u t n a m ' s basic categories were retained, with m i n o r a d d i t i o n s of c o n t e x t sensitive items to several categories. F o r instance, category 5 consists of messages that request or suggest procedural direction. Messages a b o u t the use of the technology (whether c o m p u t e r or m a n u a l ) were i n c l u d e d i n that category. 4.3. C o d i n g f o r U s a g e o f t h e T e c h n o l o g y
A n i m p o r t a n t e x t e n s i o n of the P u t n a m scheme in this research was to code i n f l u e n t i a l uses of the technology p r o v i d e d to groups - c o m p u t e r technology i n the case of G D S S groups, a n d papera n d - p e n c i l devices in the case of m a n u a l groups. T h e e x p e c t a t i o n was that i n d i v i d u a l s would use the technology to try to i n f l u e n c e the group. A G D S S c a n be used to i n f l u e n c e a group b y changing the c u r r e n t display o n the p u b l i c screen, deleting a n item (a p r o b l e m , criterion, or alternative definition) from the system, or using the group c o m m u n i c a t i o n feature. T h e n u m b e r of times each
2 Berelson (1952, p. 172) surveyed thirty studies and reported a "range of correlation coefficients between 0.78 and 0.99, with a concentration at about 0.90, and a range of percentage agreements between 68% and 96% with a concentration over 90%." Thomdike (1950) and Lasswell et al. (1952) state that 75% to 85% agreement is sufficient to establish reliability when the error is not systematic.
I. Zigurs / Interaction Analysis in GDSS Research
group member undertook one of those three behaviors was counted via an analysis of the computer log files. In manual groups, influence behaviors that corresponded to the three that were measured in GDSS groups were counted through observation of the videotapes. The manual behaviors consisted of changing the currently displayed page of the flipchart, crossing out material on the flipchart, and writing comments on the flipchart. These influence attempts via use of the technology (both GDSS and manual) were combined with the verbal influence attempts, resulting in a total count of influence attempts by each group member. The interaction analysis resulted in counts of statements within each of the first five categories of Putnam's scheme (representing different types of verbal influence behavior), counts on total amount of talk, and counts of influence behavior via use of the technology, for each individual in a group. 3
5. Conclusions and Recommendations
Interaction analysis is an expensive and time-consuming research method. Videotapes must be transcribed to written transcripts, multiple coders must be trained to a reliable level, and the coding itself must be done and periodically crosschecked for consistency. The training and coding processes are intensive and the time investment heavy. It may take four to six weeks to attain reliability on a previously validated coding scheme such as the one used here. Issues of consistency, interpretation and validity associated with the use or development of systems for coding social interaction need to be considered as well [see Folger, Hewes and Poole (1984)]. The question is whether the resource investment required for interaction analysis is worthwhile for GDSS research. The use of Putnam's procedural messages coding scheme in this research contributed in several ways to an understanding of how computer support impacts small groups of decision makers. The capability to examine specific categories of verbal behavior, in addition to overall verbal participation, provided a richer ground for interpreting
3 See Zigurs, Poole. and DeSanctis (1988) for a discussion of the findings.
239
GDSS effects. For instance, the expectation that computer-supported groups would spend more time on processual concerns was borne out by the interaction analysis. The types of messages used by GDSS groups differed significantly from the types used by manual groups, showing that the technology affected the process by which groups reached a decision. This particular result was not a surprising one, but the interaction analysis method provided a means for empirical confirmation. Usage of the computer system itself adds a level of complexity to coding interaction analysis in that one generally wants to capture system usage in addition to verbal acts. The present research coded use of the computer system by recording instances where individuals directed or redirected group attention through the system. Extensions of the interaction analysis in this area might include a more detailed examination of the extent to which verbal versus electronic channels are used and whether different types of channels are used for different purposes or in different ways. Such research would contribute to realizing the full potential of a GDSS as a communication channel in interaction with other channels. This example used the interaction analysis method to look exclusively at influence attempts and patterns in groups. Other researchers might benefit from using the method in a number of ways. The interaction analysis technique has seen considerable use in group behavior research, and there exists a literature on patterns in "naturallyinteracting" groups (i.e., groups without technological support). These prior findings represent a baseline on which GDSS researchers can build in their own analyses. In addition, interaction analysis can reveal problems with which groups strugg l e - problems that might be alleviated through technological support. This method also reveals links between behaviors and outcomes, for instance through examination of patterns of verbal behavior and their relationship to decision quality. The author's opinion is that interaction analysis brings a meaningful new dimension to the GDSS researcher's toolbox, in spite of its high resource cost. Few other techniques provide a micro-level view of group process. A simple i n p u t - o u t p u t model of decision making, which treats the process itself as a black box, is not sufficient to explain the effects of computer support [Todd and Benbasat (1987)]. The ultimate objective of pro-
240
L Zigurs / Interaction Analysis in GDSS Research
viding technological support is to enhance the process and products of group decision making. Interaction analysis helps to open the "black box" of process, and thus has the potential to contribute significantly to our understanding of the impacts of computer support on groups.
References Applegate, L. "Idea Management in Organization Planning." Unpublished doctoral dissertation, University of Arizona, 1986. Bales, R.F., "A Set of Categories for the Analysis of Small Group Interaction." American Sociological Review, 1950, 15, 257-263. Bass, B.M. Stogdill's Handbook of Leadership. New York: The Free Press, 1981. Beauclair, R. "An Experimental Study of the Effects of GDSS Process Support Applications on Small Group Decision Making." Unpublished doctoral dissertation, Indiana University, 1987. Berelson, B. Content Analysis in Communication Research. Glencoe, Illinois: The Free Press, 1952. Carter, L., Haythorn, W., Meirowitz, B., and Lanzetta, J. "A Note on a New Technique of Interaction Recording." Journal of Abnormal and Social Psychology, 1951, 46, 258-260. Fisher, B.A. "Decision Emergence: Phases in Group Decisision-Making." Speech Monographs, 1970, 37, 53-66. Fisher, B.A., Drecksel, G.L. and Werbel, W. "Social Information Processing Analysis (SIPA): Coding Ongoing Human Communication." Small Group Behavior, 1979, 10, 3-21. Folger, J.P., Hewes, D.E., and Poole, M.S. "Coding Social Interaction." In Dervin, B. and Voight, M. (eds.). Progress in Communication Sciences. Vol. 4, 1984. Gallupe, R.B. "The Impact of Task Difficulty on the Use of a Group Decision Support System." Unpublished doctoral dissertation, University of Minnesota, 1985. Gouran, D.S. "Variables Related to Consensus in Group Discussions of Questions of Policy." Speech Monographs, 1969, 36, 387-391. Hirokawa, R. "Group Communication and Problem-Solving Effectiveness II: An Exploratory Investigation of Procedural Functions." Western Journal of Speech Communication, 1983, 47, 59-74. Joyner, R. and Tunstall, K. "Computer Augmented Organizational Problem Solving." Management Science, 1970, 17 (4), B-212-B-225. Larson, C.U. "Leadership Emergence and Attention Span: A Content Analysis of the Time-Devoted-to-Themes Discussed in the Task-oriented Small Group." Unpublished doctoral dissertation, University of Minnesota, 1968. Lasswell, H.D., Lerner, D., and the Sola Pool, I. The comparative study of symbols. Palo Alto: Stanford University Press, 1952. Leathers, D. "Process Disruption and Measurement in Small
Group Communication." Quaterly Journal of Speech, 1969, 55, 287-300. Lewis, L.F. "Facilitator: A Microcomputer Decision Support System for Small Groups." Unpublished doctoral dissertation, University of Louisville, 1982. McGrath, J.E. Groups: Interaction and Performance. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1984. Mortensen, C. "A Content Analysis of Leadership Communication in Small, Task-Oriented Discussion Groups." Unpublished master's thesis, University of Minnesota, 1964. Poole, M.S. "Decision Development in Small Groups II: A Study of Multiple Sequences in Decision Making." Communication Monographs, 1983, 50, 321-341. Poole, M.S. and DeSanctis, G. "Group Decision Making and Group Decision Support Systems: A 3-Year Plan for the GDSS Research Project," working paper, MISRC-WP-8802, MIS Research Center, University of Minnesota, 1987. Poole, M.S. and Folger, J. "A Method for Establishing the Representational Validity of Interaction Coding Systems: Do We See What They See?" Human Communication Research, 1981, 8, 26-42. Poole, M.S. and Hirokawa, R.Y. "Communication and Group Decision Making: A Critical Assessment." In Hirokawa, R.Y. and Poole, M.S. (eds.), Communication and Group Decision-Making. Beverly Hills: Sage Publications, 1986, 15-31. Putnam, L.L. "Procedural Messages and Small Group Work Climates: A Lag Sequential Analysis." In Burgoon, M. (ed.), Communication Yearbook 5. New Brunswick, N.J.: Transaction Books, 1981, 331-350. Rogers, L.E. and Farace, R.V. "Analysis of Relational Communication in Dyads: New Measurement Procedures." Human Communication Research, 1975, 1, 222-239. Siegel, J. Dubrovsky, V., Kiesler, S., and McGuire, T.W. "Group Processes in Computer-Mediated Communication." Organizational Behavior and Human Decision Processes, 1986, 37, 157-187. Sims, H.P., Jr. and Manz, C.C. "Observing Leader Verbal Behavior: Toward Reciprocal Determinism in Leadership Theory." Journal of Applied Psychology, 1984, 69 (2), 222-232. Steeb, R. and Johnston, S.C. "A Computer-Based Interactive System for Group Decision making." IEEE Transactions on Systems, Man, and Cybernetics, August 1981, SMC-11 (8), 5a4-552. Thorndike, R.L. "Reliability." In Lindquist, E.G. (ed.). Educational Measurement. Washington, D.C.: American Council on Education, 1950. Todd, P. and Benbasat, I. "Process Tracing Methods in Decision Support System Research: Exploring the Black Box," MIS Quarterly, 1987, 11 (4), 493-512. Turoff, M. and Hiltz, S. "Computer Support for Group Versus Individual Decisions." IEEE Transactions on Communications, 1982, 30 (1), 82-90. Watson, R.W. "A Study of Group Decision Support System Use in Three and Four-Person Groups for a Preference Allocation Decision." Unpublished doctoral dissertation, University of Minnesota, 1987.
L Zigurs / Interaction Analysis in GDSS Research Wood, J. "Leading in Purposive Discussions: A Study of Adaptive Behavior." Communication Monographs, 1977, 44, 152-165. Zigurs, I. "The Effect of Computer Support on Influence Attempts and Patterns in Small Group Decision-Making."
241
Unpublished doctoral dissertation, University of Minnesota, 1987. Zigurs, I., Poole, M.S., and DeSanctis, G. "A Study of Influence in Computer-Mediated Group Decision Making," MIS Quarterly. 1988, 12(4), 625-644.