- Email: [email protected]
Longitudinal organizational research in field settings
Longitudinal Organizational Research in Field Settings John M. Ivancevich Michael T. Matteson
University of Houston
theorists, and researchers are interested in studying organizational phenomena such as technological change, organizational development, and leadership effectiveness over a period of time. However, a review of the literature in the field of organizational behavior and management indicates that a limited number of empirically based studies rigorously investigate changes within organizations. This article looks at the salient characteristics and issues facing those engaged in longitudinal research. While much more theoretical and methodological work is needed to improve the rigor of this type of research, some researchers have forged ahead and have attempted to conduct longitudinal research in field settings. A select few of these studies are presented to illustrate what can be accomplished by longitudinal research and what needs to be improved in such research designs. In conclusion, the authors offer some suggestions that may serve to increase the number of longitudinal organizational research studies in the future. Managers,
The most direct way to study changes within organizational settings over time is to obtain a sample of change and actually watch it occurring. The method of observing changes over time is designated differently in various disciplines and fields: longitudinal method (psychology, education, and organizational behavior) historical method (anthropology and sociology), and time series (economics). In this article, the term longitudinal will be used to reflect the “long view” of studying change processes. The present article attempts to examine some of the salient characteristics and issues facing organizational researchers who are conducting longitudinal research or considering using longitudinal research designs. In addition, a few selected examples of longitudinal research in the organizational behavior and management field will be examined. Finally, some suggestions will be made to encourage more researchers to work at securing organizational sites to perform longitudinal research. An important aspect of organizational research is to study change in natural settings. Research theorists and methodologists Address correspondence to: John M. Ivancevich, College of Business Administration, University of Houston, Houston, Texas 77004. JOURNAL
OFBUSINESS
RESEARCH
0 Elsevier North-Holland, Inc., 1978
6 (1978),
181-201 0148-2963/78/0006-0181$01.75
181
182
John M. Ivancevich
and Michael
T. Matteson
[3, 51 suggest that questions of internal and external validity are of critical importance in this type of research. Internal validity relates to whether the experimenter (researcher) has controlled for the influence of extraneous variables that could serve as rival hypotheses for explaining the effects produced by the independent variable. External validity deals with the question of generalizability of results. There are a number of threats to both internal and external validity that are serious issues when studying behavior and performance changes in either laboratory or field settings. Several desiderata offered by researchers [ 51 to minimize these threats include: 1. Before and after measures should be taken from groups. 2. Time series data should be gathered to lessen the obtrusiveness of the instruments and reduce the effects of instrumentation and test-treatment interaction. 3. Quasi-experiments should include control or comparison should assign groups to groups. If possible, researchers experimental or comparison status randomly. 4. Cross-lagged analyses can be used if a deliberate experimental manipulation is not performed. This method is a passive quasi-experimental procedure. Each of these suggestions referred to as a longitudinal
can best be incorporated research design.
The Nature of Longitudinal
Research
in what is
Longitudinal studies are originally set apart from what are described as cross-sectional studies. A cross-sectional study involves the investigation of one or more variables as they all are at one single point in time. On the other hand, longitudinal studies imply measuring the same variables on several successive occasions over some period of time. Thus, the longitudinal aspect involves both a time dimension and a presumption that a researcher can measure changes in some characteristic over time. The characteristic may involve a skill such as typing or it could be a psychological characteristic such as the ability to set challenging task goals. Since time and repeated measures are important distinguishing features of longitudinal research, they need to be included in the definition of this type of study. In addition, since there are internal and external threats to the validity of measuring change, it is necessary to introduce the issues of time and repeated measures
Longitudinal
Organizational
183
Research
into any definition of longitudinal research. Considering previous interpretations, the time and repeated measure phenomena, and the Cook and Campbell [5] concern with internal and external validity, the following definition is suggested: Longitudinal organizational research consists of using techniques and methodologies that permit the study, analysis, and interpretation of changes that occur over a time period sufficiently long to assess meaningful change in the variables of interest, as well as facilitate researcher and managerial understanding about causality.
Two important issues illustrated in this attempt to define longitudinal research are the notions of a sufficiently long time period and causality. The determination of how long a time period is sufficiently long will vary with both the objectives of the research and the nature of the phenomena under investigation. If the study covers an adequate time for the particular situation and utilizes repeated measurement, then by using our suggested definition, it would qualify as a longitudinal study. Deciding whether or not to call a particular relationship between two variables A and B a “causal” relationship is an important decision that managers must make in organizational change programs. If the observed relationship is the result of a rigorous experiment where A invariably precedes and is followed by B, then there is reason to conclude that the relationship is causal. The assumption being made is that the manipulated or independent variable is the cau,se and the dependent variable whose change is observed is the effect. Indeed, one of the most efficient and time honored ways to approach the cause-effect dilemma is to use an experimental approach. However, when experimental manipulation is not possible or is not entirely controlled, it may still be possible to investigate cause-effect linkages by incorporating causal-correlational techniques such as cross-lag and path analyses. Current
Literature
Trends on Longitudinal
Research
Longitudinal research has typically been associated with researchers in academic institutions. Consequently, studies in academically oriented publications were consulted to determine the current state of the art with regard to research methods. Three prominent journals in the organizational behavior and management field were reviewed to assess the frequency with which longitudinal research
184
John M. huncevich
and Michael T. Matteson
is being conducted. The three journals were Administrative Science Quarterly, Academy of Management Journal, and Organizational Behavior and Human Performance. The journals from the years 1974 to 1976 were thoroughly reviewed. Each article in the journals was briefly analyzed and categorized as (1) theoretical or empirical, (2) inferring causality, (3) cross-sectional or longitudinal, and (4) laboratory or field study. A summary of this analysis is presented in Table 1. The data shown in Table 1 suggest that of these three leading academician-oriented journals, only the Administrative Science Quarterly has reported a significant number of longitudinal research studies. In addition, it appears that most of the authors of longitudinal studies are not specifically addressing the issue of causality. For example, although 36.4% of the pieces in ASQ in 1976 can be classified as longitudinal, only 6.0% of them focused on causality in the design or discussion of results sections. Thus, although each of these journals publishes primarily empirical research, there is little evidence that there is a growing interest in conducting longitudinal research that uses rigorous design.
Longitudinal
Research Design
The term “design” refers to the plan or strategy conceived to obtain an answer to a research question. A design helps specify such things as how to proceed in collecting data and how to analyze the data collected. Design also has as its purpose the controol of unwanted variation. Kerlinger [ 131 discussed three research design criteria that apply to organizational research. 1. Does the design answer the research questions? 2. Have extraneous variables been controlled (internal validity)? 3. Can the results of the study be applied to individuals other than those who participated in the study (external validity)? These three criteria represent the ideal. Naturally, the first criterion must be met by all research studies. However, the degree to which the second and third are met varies from study to study with longitudinal organizational research in field settings placing equal emphasis on internal and external validity. In research conducted in actual organizational settings, it is often difficult to adopt the true experimental design criteria that
Longitudinal
185
Organizational Research
Table 1: A Summary of Characteristics and Features Appearing in Three Leading Journals Characteristics and Features of Literature
of Articles
1974
1975
1976
30 72
42 83
40 59
56
54
67
76.7 72.2 93.1
69.0 78.3 86.0
87.5 98.3 91.2
13.1 3.8 59.3
10.0 12.3 34.7
6.0 16.0 56.5
17.4 13.4 7.4
50.0 15.4 10.2
36.4 13.8 9.7
ASQ
95.7
82.7 29.6
100.0 75.4 49.0
97.0
AMJ OBHP
Total number of articles and/or notes Administrative Science Quarterly (ASQ) Academy of Management Journal (AMJ) OrganizationalBehavior and Human Performance (OBHP) Percentage of empirical pieces (others referred to as theoretical)
ASQ AMJ OBHP Percentage of pieces infering causality
ASQ AMJ OBHP Percentage of pieces considered longitudinal
ASQ AMJ OBHP Percentage of pieces considered field studies
82.4 32.3
are recommended by Cook and Campbell [ 51. Furthermore, it is difficult to gain and maintain access to research populations in field settings. There are also ethical problems associated with withholding treatment from various groups and not informing individuals that they are involved in a study. In addition, few researchers are able to convince management representatives that participants, work groups, or departments should be randomly assigned to control versus experimental conditions. A manager or executive committee typically makes the decision about conditions and whether there will even be control groups [ 61. This realism imposed on the researcher often eliminates what is considered the “true” experimental design.
186
John M. Zvancevich and Michael T. Matteson
The phrase “quasi-experimental design” refers to the application of experimental techniques and processes that do not meet the full requirements of experimental control [2], yet still allow for the possibility of causal inferences being drawn. Such inferences can be made by rendering other rival interpretations of research findings implausible. Or in the case of cross-lagged panel methods, a passive quasi-experimental approach, the issue of whether A is a stronger cause of B than B is of A is examined. Cook and Campbell [ 51 diagram a time-series quasi-experimental design that can be used to trace changes over time. It is referred to as the interrupted time series in that it requires the researcher to take a series of measurements before and after the introduction of some treatment conditions. It can be presented as follows:
01 02 03 04
x
(Pretreatment)
O5
O6
O7
O8
(Posttreatment)
The periodic measurement (e.g., two, three, four, or eight as shown) of some group or individual is presented as Observation 1 through Observation 8 (0, to 0,). These measurements are taken before and after the experimental treatment (X) is applied. The primary source of weakness in using this design is its failure to control for the effects of history. Another problem is that if extraneous events occur at about the time of the experimental treatment, they could serve as a rival hypothesis for changes that are identified. The multiple time-series design is basically an extension of the interrupted time-series design. However, it has the advantage of eliminating the history effect by including a comparable group of subjects that does not receive the treatment conditions. Note that by introducing a control group, a greater degree of control is exercised over sources of rival hypotheses. Experimental Group
01
Control
01 02 03 (Pretreatment)
Group
02
0s
04
04
X
05
06
0,
08
05
06
07
OS
(Posttreatment)
The multiple time-series design requires the researcher to randomly assign subjects into groups. Often this is not acceptable to
Longitudinal
187
Organizational Research
an organization’s
management.
Therefore,
a nonequivalent control
group design can be used. Nonrandomized
Group
Or
0s
X
03
04
Nonrandomized
Group
O1 0,
X
0s
O4
Despite the weakness of the nonequivalent control group design, it is perhaps the most popular strategy for performing longitudinal research due in large part no doubt to the nature of the constraints inherent in organizational research. The use of lagged data for tracing changes over time was first suggested by Simon [20]. Currently, the technique referred to as cross-lag analysis has been increasingly used in organizational behavior studies. This longitudinal design utilizes what is described on two identical variables as “panel data,” that is, measurements are taken from the same unit of analysis at two or more points in time. If two data points and two measures are used, then six correlations are developed. These correlations are typically shown in a cross-lag panel diagram as presented in Fig. 1. The first two correlations are called synchronous (cross section or static) correlations (AIB1 ; AzB2). They show the relationships between two variables at data collection point time 1 and at data collection point time 2. The second set of correlations is called auto (stability) correlations (A, A2 ; Bi Ba). These correlations show the relationships among the same variable, measured at two different time points, t, and t2. The third set of variables is designated as the cross-lag correlations (A1 BP ; BrAa), which are the relationships between the different variables, measured at t, and t,. The proper application of cross-lag analyses can aid significantly in understanding the theory and utilization of changes being traced over a period of time. According to the logic of cross-lag analyses, causality can be evaluated by examining the cross-lag coefficients. If AIBs > B, Aa, then the inference can be made that variable A acts as a stimulus, causing a change in variable B. Conversely, if B1 AZ > A1 Bz, then causality can be reversed; the inference is made that variable B acts as a stimulus, causing a response in variable A. Cross-lag analysis does not permit the researcher to conclude that one variable is the cause of another variable. Rather, Cook and Campbell [5] aptly suggest that cross-
188
John M. Ivancevich Al
T
r3
and Michael
->
T. Matteson A2
r6
rl
r2
Bl
Time 1
Time 2
(t1)
(tz)
FIGURE 1: Cross-Lag Analysis Model.
lag analysis is a passive quasi-experimental design that proposes that “. . . the question at issue is whether A is a stronger cause of B than B is of A”. Like most approaches to studying organizational behavior phenomena, cross-lag analysis has some limitations. There has been little attention paid to the length of the time lag. In most cases time lags are arbitrarily selected and this certainly reduces the degree of confidence in the study’s rigor [ 2 11. Feldman [ 71 considers some limitations in an analysis of causalcorrelational techniques. He suggests that a number of studies using the cross-lag technique are susceptible to alternative interpretation because tests of incongruent influence were not properly conducted. Four competing hypotheses for interpreting cross-lag coefficients were identified by Rozelle and Campbell [ 181. These competing hypotheses are analyzed by Feldman [7]. An example of one such hypothesis would be that if A1 B, > AZ Br , either A could be causing changes in B in the congruent direction (increasing agreement between A and B scores) or B could be causing changes in the incongruent direction (decreasing agreement be-
Longitudinal
Organizational
Research
189
tween A and B scores). According to Feldman [7] the incongruent influence explanation should be tested when using cross-lag analysis. Despite the limitations of the cross-lag technique, it is designed for studying over time the pattern of causal predominance between variables that are not amenable to experimental manipulation. The cross-lag procedure is not a substitute for the more powerful true experiment. However, since true experiments are so difficult to arrange, it should be considered a feasible alternative. Many variants of these four longitudinal designs can be developed to trace changes over time. Of the different quasi-experimental design features that should be incorporated in a longitudinal strategy, securing base-line measures (observing behaviors in their freely occurring state), taking multiple measures, and including experimental and comparison groups appear to be the most important. Admittedly, other features should be incorporated if possible and a checklist focusing on validity problems should be consulted before conducting research. Since longitudinal research in field settings is rarely “pure,” the researcher needs to design and execute the best study possible under existing conditions. Reluctance on the part of researchers to conduct longitudinal research because of a breach of purity can severely and unnecessarily limit gaining organizational knowledge. Advantages
of Longitudinal
Organizational
Research
As Kimberly [ 141 has noted, advantages of longitudinal research include (1) potential to establish causality, (2) helping to minimize problems encountered when processes are inferred from crosssectional data, (3) facilitating the development of better models of organizational growth and change, (4) permitting one to consider contextual constructs in the research design, and (5) enhancing the effectiveness of various strategies for organizational intervention. One of the main features of the proposed definition of longitudinal organizational research is the emphasis on causality. Longitudinal research designs can minimize or avoid the empirical obstacles that confuse researchers about causal relationships and this is a significant advantage of this type of plan. Using a longitudinal research design involving the manipulation of independent variables does not automatically eliminate the potential third factor error wherein the researchers draw incorrect conclusions about causality. Clearly, this type of error still needs
190
John M. Ivancevich
and Michael
T. Matteson
to be guarded against even in longitudinal research. However, by careful and repeated observation of the phenomena over a significantly long time period the probability of capitalizing on chance relationships and consequently making false causality inferences is reduced. A major reason for using longitudinal designs is not that problems are eliminated but that cross-sectional data used to establish causal sequencing are inadequate. The debate involving the causal relation between performance and satisfaction [4, 10, 17, 221 emphasizes this advantage. It has been hypothesized that performance causes satisfaction, satisfaction causes performance, and that a circular linkage exists between performance and satisfaction. As long as cross-sectional data are used to test such relationships, the question of causal sequence remains unanswered and unanswerable for a particular situation. Most significant changes in organizational process come about rather slowly. Cross-sectional studies are not able to deal adequately with these dynamic changes because they focus on one point in time and are essentially a static snapshot of a set of variables. Managers are not usually interested in what happens at a particular point in time, but what has occurred over a period of time. Nor are sound theories generally built from such snapshots. The inferences made from longitudinally based findings are theoretically and intuitively more insightful than a static description of organizational dimensions [5]. For example, a manager might be interested in determining the relative influence of individual personality and organizational climate on job performance. One approach to this problem would be to regress performance on a personality measure such as authoritarianism or locus of control and on a climate measure such as responsibility, production, or participation. By using a longitudinal design it is possible to measure these variables over a number of time periods and acquire a more thorough interpretation of the dynamic situation. Because of its static nature, cross-sectional research contributes to the perpetuation of what Seiler [ 191 calls the single-cause habit or the tendency to think that the effects observed often have only single causes. Viewing variables at only one point in time obscures the processes taking place immediately before and after our observation, reinforcing the single-cause habit. The reality is, of course, that organizational processes are effects that are a function of multiple causation: a chain of interrelated separate causes combine
Longitudinal
Organizational
Research
191
to produce the effect being observed. Once again, a longitudinal design has a greater chance of capturing this causal chain. In cross-sectional research there is a tendency to make false causal inferences because of temporal sequencing of organizational events. As Likert [ 151 illustrates, the time lag between certain classes of causal and end-result variables may yield radically different inferences depending on when the research is done. For example, an observation may be made at a time of high production and extreme conflict between departments. Causal inferences made from the observation might be quite different from ones made from a longitudinal research study that tracked the production and conflict over a period of time. The longitudinal study, in effect, exposes the time-lag nature of cause-effect relationships, which is simply not possible in cross-sectional research. Longitudinal research is invaluable in studying the effects of training and development programs. Training in performance appraisal, goal setting, team building, and other areas of managerial emphasis involves spending billions annually [8] . Empirically evaluating the results of training is extremely important to organizations. Unless effective evaluations are conducted, managers are unable to improve training procedures, justify further training, or determine whether training objectives are being achieved. An example of an evaluation study which was able to provide management with scientifically based answers to important training issues was reported by Goodacre [9] . He reported a longitudinal study evaluating a supervisory training program at B. F. Goodrich Company. Eight hundred trainees were randomly placed into two groups-an experimental group and a control group. The control group did not show any significant changes, but the experimental group improved on posttraining performance back on the job. This longitudinal study using a quasi-experimental design did not eliminate all threats to validity but it enabled the researchers to track changes on the job after the training was completed. Again, this kind of monitoring is not possible using crosssectional data. Another advantage of longitudinal research is that it allows the researcher to stay in an organizational setting while variables are changing and while the impact of training or organizational development is taking place. Using such devices as diary recordings of critical incidents and events, managerial reports, and other
John M. Ivancevich and Michael T. Matteson
192
forms of notes taken over a period of time allows the researcher to witness and recall changes first hand. This experience can result in better interpretations of data by the researcher. In summary, based both on the literature and personal experiences in conducting field studies using a longitudinal strategy, it is our contention that the advantages of such an approach are: 1. 2. 3. 4. 5.
6. 7.
The issue of causality can be addressed [ 141. The probability of hidden third-factor error can be lessened. Studying dynamic processes is possible. Time lags between cause and effect can be identified, resulting in a better understanding of causal relationships. The investigation of the impact of training and organizational change and development is more scientific and therefore has greater value to a manager. The researcher is able to remain in a situation to capture the actual changes occurring. The researcher is able to remain in an organization long enough to determine whether any changes take place.
Some Problems
of Longitudinal
Organizational
Research
In organizations, various processes are changing over a period of time. Consequently, it seems that managers would want to study these processes so that better decisions can be made about managing the changes. However, there are a number of practical drawbacks to what is described as longitudinal organizational research. Perhaps the most obvious is the fact that what are considered final results of such research efforts often require extended time periods. Few managers have the patience or job security to wait for “final” results before taking some action. The same type of patience requirement applies to academic researchers. There are few universities willing to wait patiently for years while one of its young nontenured, and perhaps best trained researchers, collects longitudinal data. Another major problem involves sustaining a representative sample over the course of the study. At the outset, a researcher may have many participants, but shrinkage of the sample due to personnel changes, illness, death, transfer, termination, or a change in attitude about participating in the research may be severe. These events are usually outside the control of the researcher and need to be considered when developing a longitudinal study.
Longitudinal
Organizational Research
193
Changes in either the internal or external organizational environment may operate to the detriment of a well-conceived longitudinal research plan. Internally, management may change, the organization may be restructured, or significant changes in technology may occur. fixternally, markets may change, competition may increase or lessen, the economy may falter or pick-up, any of which may have an impact upon research going on inside the organization. The passage of time also has an impact upon the techniques used in a study. As knowledge develops, new theories emerge, and new techniques appear which present improved alternatives to the original plan. Outdated theories and archaic instruments cannot be justified merely because they were the ones used at the beginning. Thus, a pressing dilemma facing a researcher involves making decisions to change instruments or observation methods during a study. Determining what time periods to use in a longitudinal study is a difficult decision. There is a wide variety of “causal intervals” used in longitudinal research. For example, some changes in organizational practices are relatively short-run, such as improving goal setting skills [23] , while other changes such as implementing a company-wide management by objectives program may require years [ 121. There are other changes involving structural and interpersonal factors that take at least 5 years [ IS] . Therefore, repeated observations must correspond with the time of actual change if the process is to be monitored accurately. A previously stated advantage of longitudinal research may also act as a potential disadvantage to the unwary researcher. Earlier it was suggested that the hidden third factor would tend to be minimized in a longitudinal design as more information was gathered over time. While it is probably not possible to learn too much, it is possible for the researcher to become overwhelmed by so much data and/or the possibilities of multiple causal chains that the research objectives are never met. This is a difficulty that is not inherent in longitudinal research, but occurs because of inadequate conceptualization of the problem by the researcher.
Examples of Longitudinal
Organizational
Research
The manner in which researchers deal with invalidity, tion, comparison and experimental groups, frequency
randomizaof measure-
194
John M. Ivancevich
and Michael
T. Matteson
ment, and other relevant issues in longitudinal research can be illustrated by a sample of studies. Effectiveness of Human Relations Training Hand and Slocum [ 111 used a longitudinal design to evaluate the effectiveness of a managerial human relations training program. An experimental group and a control group of 42 line and staff managers were randomly selected from one hierarchical level in a specialty steel plant. These researchers used a pre- and posttest strategy to measure training program effectiveness. By testing the change in correlations between the behavioral criterion and performance, they infer causality that is, what knowledge was acquired in the training was associated or linked with improved on-the-job performance. The study’s major strengths are not the degree of control exercised by the researchers, but the fact that they used an extended observation cycle to identify positive “sleeper effects,” groups were randomly selected, and after training, behaviors were assessed back on the job. The experimental “purist” could argue with the time-lag selection but longitudinal research at this level of rigor and that illustrated when discussing Goodacre’s [9] study are needed in the area of organizational training and development. Organizational Commitment and Turnover Employee turnover is a costly problem that all organizations face in varying degrees. Much of the organizational research on turnover has involved collecting attitude data at the point of departure or shortly thereafter through the use of exit interviews or questionnaires. A longitudinal study of the relationship between organizational commitment and turnover of managerial trainees in a large merchandising company was reported by Porter et al. [ 16 I . The subjects of this study were assigned to retail stores for 9 to 12 months of training. Data were collected during the 9-12 months of training and during the individuals’ first 3-6 months on a regular job. The longitudinal analysis revealed that commitment scores for stayers differed from those scores of leavers. The differences appeared as early as the first week of employment, with leavers lacking a positive attitude. These differences were identified with the longitudinal design used by the researchers. The researchers by using a longitudinal design were able to illustrate to managers that some type of “alarm” system could be used so that steps could be initiated to retain some of the “leavers.” A one-shot cross-sectional design would unlikely have been as effective.
Longitudinal
Organizational
Research
195
Technological Change and Job Attitudes A change in technology from batch to mass production in a hospital dietary department was examined using a longitudinal design employing structured interview data [ 11. A technological change affecting the assembly and delivery of meals for all patients was introduced. The research design is essentially quasi-experimental with measures obtained at unequal time intervals for different individuals. There were four waves of interviews, two before the change and two after. Randomization could not be practiced rigidly because of work scheduling constraints. The longitudinal design allowed the researchers to trace changes in job attitudes after applying the new technology. Performance and Job Satisfaction Wanous [22] gathered job satisfaction and performance data from female telephone operators. Cross-lagged analyses were used to study the data. The researcher, by using a 2-month time lag, correlated measures of telephone operators’ intrinsic and extrinsic job satisfaction with a performance index. His analysis showed extrinsic satisfaction at t, correlated more highly with t2 performance than vice versa, and that t, performance correlated more highly with intrinsic satisfaction at t, than vice versa. The causal inferences drawn were that satisfaction with organizational-mediated goals influenced performance, and that good performance influenced satisfaction with task-mediated rewards. Of course, as the researcher indicates, the conclusions must be treated with caution. This is because only female participants were included and the job sample was limited (telephone operators only). However, the passive quasi-experimental cross-lag method enabled Wanous [22] to examine multiple measures of both satisfaction and performance at two time points. This is a positive step in the direction of empirically investigating the important performance-satisfaction causal linkage. The major characteristics of each of these attempts at lontitudinal research are summarized in Table 2. These four studies involving training, turnover and commitment, technological change, and performance-satisfaction issues are not offered as ideal examples of scientific research or of quasi-experimental designs. Instead they are presented because they attempt to cope with changes and time by incorporating a longitudinal design. The researchers of each study admit that they have not achieved anything resembling the “pure” experimental research design.
Technological change and job attitudes
Performance satisfaction
Billings, Klimoski, and Breaugh (1977)
Wanous
(1974)
Organizational commitment turnover
Porter, Crampon, and Smith (1976)
and
and
Effectiveness of human relations training
Hand and Slocum (1972)
Research
Female telephone operators
Dietary personnel in a hospital
Retail management trainees
Line and staff managers in steel company
Sample
Organizational
Organizational Behavior and Management Topic
of Longitudinal
Researchers
Table 2: Summary
Inferred
Discussed specifically
Discussed specifically
Discussed specifically
Causality
Studies
The use of multiple measures of both satisfaction and performance can be examined at two data points for newly hired employees
Technological change may not have the anticipated effect and this is why it should be studied over a period of time
The need to develop an early alert system so that valuable human resources can be salvaged before individuals quit an organization
The importance of examining the transfer of learning phenomenon on the job by evaluating performance
Insight Provided to the Field of Knowledge
$
$
Longitudinal
Organizational Research
197
Some Future Suggestions to Consider The survey results shown in Table 1 as well as the points raised throughout this article clearly suggest that there currently exists no ground swell movement to conduct more longitudinal organizational research. Certainly this is not due to the lack of statistical and mathematical models now available which enable researchers to analyze and interpret longitudinal data. If the initiative is not taken by researchers, many of the opportunities to conduct such research are likely to be missed. Therefore, some suggestions for the future are offered to those researchers interested in becoming involved in longitudinal research. The managers of organizations, like the students cited earlier, need to be better informed about what longitudinal organizational research is and how it can be used to improve managerial effectiveness. In order to educate managers, researchers must be able to demonstrate to management the utility of longitudinal research results. It is the researcher who must reach an agreement with management to intrude upon an organization and conduct a study over a period of time that might span several years. Since managers are, in many instances, moving from job to job, they are not always receptive to a study that takes time. Managers typically want results now and are not so concerned with the impact of a technological change 2 years in the future. It is top management, however, that is usually developing 5- and lo-year plans. Thus, the aspiring longitudinal researcher must seek out those top-level executives who have a long-run time horizon and convince them that longitudinal research can be integrated meaningfully and effectively into such planning activities. Gaining entree into an organization requires marketing expertise as well. Personal experiences indicate that there are some top-level executives who will listen to a researcher and even allow that person to conduct a longitudinal study. Patience and perseverence are two important attributes of researchers involved in longitudinal research. We have also found that an endless discourse of research terminology is one excellent way not to gain access into top-level executive suites and eventually the company. Executives are not and do not wish to be conversant about experimental groups, treatment conditions, internal and external invalidities, quasi-experimental designs, interrupted time series, and other jargon of the academic community. Speaking the executives’ language, using descriptive phrases, and being concise are the most successful strategies to use.
198
John M. Ivancevich
and Michael
T. Matteson
A review of many of the national professional association meetings and proceedings indicates that numerous symposia are presented covering such varied and important topics as the quality of work life, leadership research, behavior modification, comparative organizational research, job design applications, and goal setting. Conspicuously absent from most agendas is any presentation of the issues, current state of affairs, and problems of longitudinal research. The professional meeting is the place to begin illustrating to students, colleagues, and managers how vital longitudinal research is and what it can contribute to the field of knowledge. The graduate student is another important party in stimulating more longitudinal research. Currently, longitudinal research is almost unheard of for dissertation research. Students, not unreasonably, are not willing to wait the necessary length of time to get their data. Perhaps a reasonable alternative would be for dissertation advisors to develop a research program that would allow one student to begin a rigorous longitudinal study and have another student subsequently follow it up. Or, perhaps several students could “plug into” a longitudinal study, which would at least expose the students to participation in an ongoing “long view” research project. One possible reason for the limited number of longitudinal research studies is that it is generally easier to complete crosssectional studies and this often means more publications in less time. If evaluations of academic researchers consider “counting pages” as one method of judging research productivity, longitudinal research efforts will suffer. Department chairs, deans, academic vice presidents, and similar agents may wish to carefully examine the merits of longitudinal research and encourage individuals to plan and execute realistic longitudinal projects. With improved understanding of the role and potential significance of such projects could come merit review processes that reward these efforts as a function of legitimate progress made rather than solely on ultimate publication. The mathematical and statistical problems associated with longitudinal research may appear to be significant [ 141. However, the need to use various “uncommon” statistical procedures should not limit a researcher’s enthusiasm to conduct longitudinal research. Many excellent source books and examples are available. Popular statistical procedures of longitudinal research include causalcorrelational models, repeated measures analysis of variance, and
Longitudinal
Organizational Research
199
regression analysis, among others. Each of these techniques is discussed in detail in multivariate statistics books and various articles. Many researchers who lack confidence in their abilities in this area have access to colleagues who do have the needed expertise. Indeed, one potential advantage in undertaking longitudinal research projects is the collaboration that may be needed and that results in mutually rewarding symbiotic relationships for the research team. Another suggestion would be to encourage prominent journals to occasionally devote an entire issue to longitudinal organizational research. Journals have devoted issues to cascaded inferences, organizational development practice and theory, comparative management, bureaucratic organizations, and leadership. Why not longitudinal organizational research? Perhaps each of these other special issues could have been improved if more longitudinal research designs were included to present the area covered. By using an entire issue, researchers, students, and managers will be presented with longitudinal research ideas, suggested projects, and findings. This type of stimulation can result in other researchers attempting longitudinal research. In a similar vein, just as several journals have added “short notes” sections to their publication format, why not add a research methodology section in which one or more pieces would be published each issue? Perhaps researchers working on longitudinal projects may be encouraged to use such an outlet. Also worthy of consideration might be a refereed “longitudinal research abstract” section in which researchers could report in a brief paragraph current longitudinal projects as an information and stimulation device for other investigators. In writing this article, we have attempted to illustrate some characteristics, issues, examples, and needs of longitudinal organizational research. It should be obvious at this point that longitudinal research designs are not the solution to all problems facing managers and organizational researchers. The implementation of a longitudinal design in an organizational setting requires patience, creativity, and understanding. There is no easy and quick set of longitudinal research suggestions available. In fact, most of the suggestions and information presented in this article indicate that work, challenge, and problems lie ahead for those researchers initiating longitudinal research projects. The personal payoffs, however, can certainly be worth the costs that must be borne.
200
John
M. Ivancevich
and Michael
T. Matteson
References 1.
Billings, R. S., Klimoski, R. J., and Breaugh, J. A., The Impact of a Change in Technology on Job Characteristics: A Quasi-Experiment, Administrative Sci. Q. 22 (1977): 318-339.
2.
Campbell, D. T., Quasi-Experimental Design, in International Encyclopedia of the Social Sciences, 5, D. L. Gills, ed., Macmillan and Free Press, New York, 1968.
3.
Campbell, D. T., and Stanley, J. D., Experimental for Research, Rand McNally, Chicago, 1963.
4.
Cherrington, contingent formance,J.
5.
Cook, T. D., and Campbell, D. T., The Design and Conduct of Quasi-Experiments and True Experiments in Field Settings, M. Dunnette, ed., Rand McNally, Chicago, 1976, pp. 223-326.
6.
Evans, M. D., Opportunistic Organizational Research: Designs, Acad. Management.I. 18 (1975): 98-108.
7.
Feldman, J., Considerations in the Use of Causal-Correlational Applied Psychology, J. Appl. Psychol. 60 (1975): 663-670.
8.
Goldstein, Monterrey,
9.
Goodacre, D. M., Experimental Evaluation of Training, tion Industrial Relations 2 (1955): 143- 149.
and Quasi-Experimental
D. J., Reitz, H. J., and Scott, W. E., Effects of Contingent Reward on the Relationship Between Satisfaction and Appl. Psychol. 35 (1971): 531-536.
I., Training: Calif., 1974.
Program
Development
The
Role
Designs
and NonTask Per-
of Patch-Up
Techniques
and Evaluation.
in
Brooks/Cole,
J. Personnel AdministraBusiness Horizons
10.
Greene, (1972):
11.
Hand, H. H., and Slocum, J. W., Jr., A Longitudinal Study of the Effects of a Human Relations Training Program on Managerial Effectiveness, J. Appl. Psychol. 56 (1972): 412-417.
12.
Ivancevich, J. M., Changes in Performance in a Management gram, Administrative Sci. Q. 19 (1974): 563-574.
13.
Kerlinger, F. N., Foundations New York, 1973.
14.
Kimberly, J. R., Issues in the Design of Longitudinal Social. Methods Res. 4 (1976): 321-347.
1.5.
Likert,
16.
Porter, L. W., Crampon, W. J., and Smith, F. J., Organizational Commitment and Managerial Turnover: A Longitudinal Study, Organizational Behav. Hum. Perform. 15 (1976): 87-98.
17.
Porter, Richard
18.
Rozelle, R. M., and Campbell, D. T., More Plausible Rival Hypotheses in the Cross-Lagged Panel Correlation Technique, Psychol. Bull. 7 I (1969): 74-80.
C. N., The Satisfaction-Performance 31-41.
Controversy.
of Behavioral Research, Holt,
R., The Human Organization, McGraw-Hill,
L. W., and Lawler, D. Irwin, Homewood,
by Objectives
15
Pro-
Rinehart
& Winston,
Organizational
Research,
New York,
E. E., III, Managerial Attitudes Ill., 1968.
1967.
and Performance,
Longitudinal 19.
20.
Organizational
Research
201
Seiler, John A., Systems Analysis in Organizational Behavior, Richard D. Irwin and The Dorsey Press, Homewood, Ill., 1967. Simon, H. A., Spurious Correlations:
A Causal Interpretation,
J. Am. Statistical
Assoc. 49 (1954): 467-479.
21.
Sims, H. P., Jr., and Wilkerson, D. A., Time-Lags in Cross-Lag Correlation Studies: A Computer Simulation, Decision Sci. 8 (1977): 630-644.
22.
Wanous, J. P., Causal-Correlational Analysis of the Job Satisfaction formance Relationship, J. Appl. Psych& 59 (1974): 139- 144.
23.
Wexley, K. N., and Nemeroff, W. F., Effects of Positive Reinforcement and Goal Setting as Methods of Management Development, J. Appl. Psychol. 60 (1965): 446-450.
and Per-
University of Houston
theorists, and researchers are interested in studying organizational phenomena such as technological change, organizational development, and leadership effectiveness over a period of time. However, a review of the literature in the field of organizational behavior and management indicates that a limited number of empirically based studies rigorously investigate changes within organizations. This article looks at the salient characteristics and issues facing those engaged in longitudinal research. While much more theoretical and methodological work is needed to improve the rigor of this type of research, some researchers have forged ahead and have attempted to conduct longitudinal research in field settings. A select few of these studies are presented to illustrate what can be accomplished by longitudinal research and what needs to be improved in such research designs. In conclusion, the authors offer some suggestions that may serve to increase the number of longitudinal organizational research studies in the future. Managers,
The most direct way to study changes within organizational settings over time is to obtain a sample of change and actually watch it occurring. The method of observing changes over time is designated differently in various disciplines and fields: longitudinal method (psychology, education, and organizational behavior) historical method (anthropology and sociology), and time series (economics). In this article, the term longitudinal will be used to reflect the “long view” of studying change processes. The present article attempts to examine some of the salient characteristics and issues facing organizational researchers who are conducting longitudinal research or considering using longitudinal research designs. In addition, a few selected examples of longitudinal research in the organizational behavior and management field will be examined. Finally, some suggestions will be made to encourage more researchers to work at securing organizational sites to perform longitudinal research. An important aspect of organizational research is to study change in natural settings. Research theorists and methodologists Address correspondence to: John M. Ivancevich, College of Business Administration, University of Houston, Houston, Texas 77004. JOURNAL
OFBUSINESS
RESEARCH
0 Elsevier North-Holland, Inc., 1978
6 (1978),
181-201 0148-2963/78/0006-0181$01.75
181
182
John M. Ivancevich
and Michael
T. Matteson
[3, 51 suggest that questions of internal and external validity are of critical importance in this type of research. Internal validity relates to whether the experimenter (researcher) has controlled for the influence of extraneous variables that could serve as rival hypotheses for explaining the effects produced by the independent variable. External validity deals with the question of generalizability of results. There are a number of threats to both internal and external validity that are serious issues when studying behavior and performance changes in either laboratory or field settings. Several desiderata offered by researchers [ 51 to minimize these threats include: 1. Before and after measures should be taken from groups. 2. Time series data should be gathered to lessen the obtrusiveness of the instruments and reduce the effects of instrumentation and test-treatment interaction. 3. Quasi-experiments should include control or comparison should assign groups to groups. If possible, researchers experimental or comparison status randomly. 4. Cross-lagged analyses can be used if a deliberate experimental manipulation is not performed. This method is a passive quasi-experimental procedure. Each of these suggestions referred to as a longitudinal
can best be incorporated research design.
The Nature of Longitudinal
Research
in what is
Longitudinal studies are originally set apart from what are described as cross-sectional studies. A cross-sectional study involves the investigation of one or more variables as they all are at one single point in time. On the other hand, longitudinal studies imply measuring the same variables on several successive occasions over some period of time. Thus, the longitudinal aspect involves both a time dimension and a presumption that a researcher can measure changes in some characteristic over time. The characteristic may involve a skill such as typing or it could be a psychological characteristic such as the ability to set challenging task goals. Since time and repeated measures are important distinguishing features of longitudinal research, they need to be included in the definition of this type of study. In addition, since there are internal and external threats to the validity of measuring change, it is necessary to introduce the issues of time and repeated measures
Longitudinal
Organizational
183
Research
into any definition of longitudinal research. Considering previous interpretations, the time and repeated measure phenomena, and the Cook and Campbell [5] concern with internal and external validity, the following definition is suggested: Longitudinal organizational research consists of using techniques and methodologies that permit the study, analysis, and interpretation of changes that occur over a time period sufficiently long to assess meaningful change in the variables of interest, as well as facilitate researcher and managerial understanding about causality.
Two important issues illustrated in this attempt to define longitudinal research are the notions of a sufficiently long time period and causality. The determination of how long a time period is sufficiently long will vary with both the objectives of the research and the nature of the phenomena under investigation. If the study covers an adequate time for the particular situation and utilizes repeated measurement, then by using our suggested definition, it would qualify as a longitudinal study. Deciding whether or not to call a particular relationship between two variables A and B a “causal” relationship is an important decision that managers must make in organizational change programs. If the observed relationship is the result of a rigorous experiment where A invariably precedes and is followed by B, then there is reason to conclude that the relationship is causal. The assumption being made is that the manipulated or independent variable is the cau,se and the dependent variable whose change is observed is the effect. Indeed, one of the most efficient and time honored ways to approach the cause-effect dilemma is to use an experimental approach. However, when experimental manipulation is not possible or is not entirely controlled, it may still be possible to investigate cause-effect linkages by incorporating causal-correlational techniques such as cross-lag and path analyses. Current
Literature
Trends on Longitudinal
Research
Longitudinal research has typically been associated with researchers in academic institutions. Consequently, studies in academically oriented publications were consulted to determine the current state of the art with regard to research methods. Three prominent journals in the organizational behavior and management field were reviewed to assess the frequency with which longitudinal research
184
John M. huncevich
and Michael T. Matteson
is being conducted. The three journals were Administrative Science Quarterly, Academy of Management Journal, and Organizational Behavior and Human Performance. The journals from the years 1974 to 1976 were thoroughly reviewed. Each article in the journals was briefly analyzed and categorized as (1) theoretical or empirical, (2) inferring causality, (3) cross-sectional or longitudinal, and (4) laboratory or field study. A summary of this analysis is presented in Table 1. The data shown in Table 1 suggest that of these three leading academician-oriented journals, only the Administrative Science Quarterly has reported a significant number of longitudinal research studies. In addition, it appears that most of the authors of longitudinal studies are not specifically addressing the issue of causality. For example, although 36.4% of the pieces in ASQ in 1976 can be classified as longitudinal, only 6.0% of them focused on causality in the design or discussion of results sections. Thus, although each of these journals publishes primarily empirical research, there is little evidence that there is a growing interest in conducting longitudinal research that uses rigorous design.
Longitudinal
Research Design
The term “design” refers to the plan or strategy conceived to obtain an answer to a research question. A design helps specify such things as how to proceed in collecting data and how to analyze the data collected. Design also has as its purpose the controol of unwanted variation. Kerlinger [ 131 discussed three research design criteria that apply to organizational research. 1. Does the design answer the research questions? 2. Have extraneous variables been controlled (internal validity)? 3. Can the results of the study be applied to individuals other than those who participated in the study (external validity)? These three criteria represent the ideal. Naturally, the first criterion must be met by all research studies. However, the degree to which the second and third are met varies from study to study with longitudinal organizational research in field settings placing equal emphasis on internal and external validity. In research conducted in actual organizational settings, it is often difficult to adopt the true experimental design criteria that
Longitudinal
185
Organizational Research
Table 1: A Summary of Characteristics and Features Appearing in Three Leading Journals Characteristics and Features of Literature
of Articles
1974
1975
1976
30 72
42 83
40 59
56
54
67
76.7 72.2 93.1
69.0 78.3 86.0
87.5 98.3 91.2
13.1 3.8 59.3
10.0 12.3 34.7
6.0 16.0 56.5
17.4 13.4 7.4
50.0 15.4 10.2
36.4 13.8 9.7
ASQ
95.7
82.7 29.6
100.0 75.4 49.0
97.0
AMJ OBHP
Total number of articles and/or notes Administrative Science Quarterly (ASQ) Academy of Management Journal (AMJ) OrganizationalBehavior and Human Performance (OBHP) Percentage of empirical pieces (others referred to as theoretical)
ASQ AMJ OBHP Percentage of pieces infering causality
ASQ AMJ OBHP Percentage of pieces considered longitudinal
ASQ AMJ OBHP Percentage of pieces considered field studies
82.4 32.3
are recommended by Cook and Campbell [ 51. Furthermore, it is difficult to gain and maintain access to research populations in field settings. There are also ethical problems associated with withholding treatment from various groups and not informing individuals that they are involved in a study. In addition, few researchers are able to convince management representatives that participants, work groups, or departments should be randomly assigned to control versus experimental conditions. A manager or executive committee typically makes the decision about conditions and whether there will even be control groups [ 61. This realism imposed on the researcher often eliminates what is considered the “true” experimental design.
186
John M. Zvancevich and Michael T. Matteson
The phrase “quasi-experimental design” refers to the application of experimental techniques and processes that do not meet the full requirements of experimental control [2], yet still allow for the possibility of causal inferences being drawn. Such inferences can be made by rendering other rival interpretations of research findings implausible. Or in the case of cross-lagged panel methods, a passive quasi-experimental approach, the issue of whether A is a stronger cause of B than B is of A is examined. Cook and Campbell [ 51 diagram a time-series quasi-experimental design that can be used to trace changes over time. It is referred to as the interrupted time series in that it requires the researcher to take a series of measurements before and after the introduction of some treatment conditions. It can be presented as follows:
01 02 03 04
x
(Pretreatment)
O5
O6
O7
O8
(Posttreatment)
The periodic measurement (e.g., two, three, four, or eight as shown) of some group or individual is presented as Observation 1 through Observation 8 (0, to 0,). These measurements are taken before and after the experimental treatment (X) is applied. The primary source of weakness in using this design is its failure to control for the effects of history. Another problem is that if extraneous events occur at about the time of the experimental treatment, they could serve as a rival hypothesis for changes that are identified. The multiple time-series design is basically an extension of the interrupted time-series design. However, it has the advantage of eliminating the history effect by including a comparable group of subjects that does not receive the treatment conditions. Note that by introducing a control group, a greater degree of control is exercised over sources of rival hypotheses. Experimental Group
01
Control
01 02 03 (Pretreatment)
Group
02
0s
04
04
X
05
06
0,
08
05
06
07
OS
(Posttreatment)
The multiple time-series design requires the researcher to randomly assign subjects into groups. Often this is not acceptable to
Longitudinal
187
Organizational Research
an organization’s
management.
Therefore,
a nonequivalent control
group design can be used. Nonrandomized
Group
Or
0s
X
03
04
Nonrandomized
Group
O1 0,
X
0s
O4
Despite the weakness of the nonequivalent control group design, it is perhaps the most popular strategy for performing longitudinal research due in large part no doubt to the nature of the constraints inherent in organizational research. The use of lagged data for tracing changes over time was first suggested by Simon [20]. Currently, the technique referred to as cross-lag analysis has been increasingly used in organizational behavior studies. This longitudinal design utilizes what is described on two identical variables as “panel data,” that is, measurements are taken from the same unit of analysis at two or more points in time. If two data points and two measures are used, then six correlations are developed. These correlations are typically shown in a cross-lag panel diagram as presented in Fig. 1. The first two correlations are called synchronous (cross section or static) correlations (AIB1 ; AzB2). They show the relationships between two variables at data collection point time 1 and at data collection point time 2. The second set of correlations is called auto (stability) correlations (A, A2 ; Bi Ba). These correlations show the relationships among the same variable, measured at two different time points, t, and t2. The third set of variables is designated as the cross-lag correlations (A1 BP ; BrAa), which are the relationships between the different variables, measured at t, and t,. The proper application of cross-lag analyses can aid significantly in understanding the theory and utilization of changes being traced over a period of time. According to the logic of cross-lag analyses, causality can be evaluated by examining the cross-lag coefficients. If AIBs > B, Aa, then the inference can be made that variable A acts as a stimulus, causing a change in variable B. Conversely, if B1 AZ > A1 Bz, then causality can be reversed; the inference is made that variable B acts as a stimulus, causing a response in variable A. Cross-lag analysis does not permit the researcher to conclude that one variable is the cause of another variable. Rather, Cook and Campbell [5] aptly suggest that cross-
188
John M. Ivancevich Al
T
r3
and Michael
->
T. Matteson A2
r6
rl
r2
Bl
Time 1
Time 2
(t1)
(tz)
FIGURE 1: Cross-Lag Analysis Model.
lag analysis is a passive quasi-experimental design that proposes that “. . . the question at issue is whether A is a stronger cause of B than B is of A”. Like most approaches to studying organizational behavior phenomena, cross-lag analysis has some limitations. There has been little attention paid to the length of the time lag. In most cases time lags are arbitrarily selected and this certainly reduces the degree of confidence in the study’s rigor [ 2 11. Feldman [ 71 considers some limitations in an analysis of causalcorrelational techniques. He suggests that a number of studies using the cross-lag technique are susceptible to alternative interpretation because tests of incongruent influence were not properly conducted. Four competing hypotheses for interpreting cross-lag coefficients were identified by Rozelle and Campbell [ 181. These competing hypotheses are analyzed by Feldman [7]. An example of one such hypothesis would be that if A1 B, > AZ Br , either A could be causing changes in B in the congruent direction (increasing agreement between A and B scores) or B could be causing changes in the incongruent direction (decreasing agreement be-
Longitudinal
Organizational
Research
189
tween A and B scores). According to Feldman [7] the incongruent influence explanation should be tested when using cross-lag analysis. Despite the limitations of the cross-lag technique, it is designed for studying over time the pattern of causal predominance between variables that are not amenable to experimental manipulation. The cross-lag procedure is not a substitute for the more powerful true experiment. However, since true experiments are so difficult to arrange, it should be considered a feasible alternative. Many variants of these four longitudinal designs can be developed to trace changes over time. Of the different quasi-experimental design features that should be incorporated in a longitudinal strategy, securing base-line measures (observing behaviors in their freely occurring state), taking multiple measures, and including experimental and comparison groups appear to be the most important. Admittedly, other features should be incorporated if possible and a checklist focusing on validity problems should be consulted before conducting research. Since longitudinal research in field settings is rarely “pure,” the researcher needs to design and execute the best study possible under existing conditions. Reluctance on the part of researchers to conduct longitudinal research because of a breach of purity can severely and unnecessarily limit gaining organizational knowledge. Advantages
of Longitudinal
Organizational
Research
As Kimberly [ 141 has noted, advantages of longitudinal research include (1) potential to establish causality, (2) helping to minimize problems encountered when processes are inferred from crosssectional data, (3) facilitating the development of better models of organizational growth and change, (4) permitting one to consider contextual constructs in the research design, and (5) enhancing the effectiveness of various strategies for organizational intervention. One of the main features of the proposed definition of longitudinal organizational research is the emphasis on causality. Longitudinal research designs can minimize or avoid the empirical obstacles that confuse researchers about causal relationships and this is a significant advantage of this type of plan. Using a longitudinal research design involving the manipulation of independent variables does not automatically eliminate the potential third factor error wherein the researchers draw incorrect conclusions about causality. Clearly, this type of error still needs
190
John M. Ivancevich
and Michael
T. Matteson
to be guarded against even in longitudinal research. However, by careful and repeated observation of the phenomena over a significantly long time period the probability of capitalizing on chance relationships and consequently making false causality inferences is reduced. A major reason for using longitudinal designs is not that problems are eliminated but that cross-sectional data used to establish causal sequencing are inadequate. The debate involving the causal relation between performance and satisfaction [4, 10, 17, 221 emphasizes this advantage. It has been hypothesized that performance causes satisfaction, satisfaction causes performance, and that a circular linkage exists between performance and satisfaction. As long as cross-sectional data are used to test such relationships, the question of causal sequence remains unanswered and unanswerable for a particular situation. Most significant changes in organizational process come about rather slowly. Cross-sectional studies are not able to deal adequately with these dynamic changes because they focus on one point in time and are essentially a static snapshot of a set of variables. Managers are not usually interested in what happens at a particular point in time, but what has occurred over a period of time. Nor are sound theories generally built from such snapshots. The inferences made from longitudinally based findings are theoretically and intuitively more insightful than a static description of organizational dimensions [5]. For example, a manager might be interested in determining the relative influence of individual personality and organizational climate on job performance. One approach to this problem would be to regress performance on a personality measure such as authoritarianism or locus of control and on a climate measure such as responsibility, production, or participation. By using a longitudinal design it is possible to measure these variables over a number of time periods and acquire a more thorough interpretation of the dynamic situation. Because of its static nature, cross-sectional research contributes to the perpetuation of what Seiler [ 191 calls the single-cause habit or the tendency to think that the effects observed often have only single causes. Viewing variables at only one point in time obscures the processes taking place immediately before and after our observation, reinforcing the single-cause habit. The reality is, of course, that organizational processes are effects that are a function of multiple causation: a chain of interrelated separate causes combine
Longitudinal
Organizational
Research
191
to produce the effect being observed. Once again, a longitudinal design has a greater chance of capturing this causal chain. In cross-sectional research there is a tendency to make false causal inferences because of temporal sequencing of organizational events. As Likert [ 151 illustrates, the time lag between certain classes of causal and end-result variables may yield radically different inferences depending on when the research is done. For example, an observation may be made at a time of high production and extreme conflict between departments. Causal inferences made from the observation might be quite different from ones made from a longitudinal research study that tracked the production and conflict over a period of time. The longitudinal study, in effect, exposes the time-lag nature of cause-effect relationships, which is simply not possible in cross-sectional research. Longitudinal research is invaluable in studying the effects of training and development programs. Training in performance appraisal, goal setting, team building, and other areas of managerial emphasis involves spending billions annually [8] . Empirically evaluating the results of training is extremely important to organizations. Unless effective evaluations are conducted, managers are unable to improve training procedures, justify further training, or determine whether training objectives are being achieved. An example of an evaluation study which was able to provide management with scientifically based answers to important training issues was reported by Goodacre [9] . He reported a longitudinal study evaluating a supervisory training program at B. F. Goodrich Company. Eight hundred trainees were randomly placed into two groups-an experimental group and a control group. The control group did not show any significant changes, but the experimental group improved on posttraining performance back on the job. This longitudinal study using a quasi-experimental design did not eliminate all threats to validity but it enabled the researchers to track changes on the job after the training was completed. Again, this kind of monitoring is not possible using crosssectional data. Another advantage of longitudinal research is that it allows the researcher to stay in an organizational setting while variables are changing and while the impact of training or organizational development is taking place. Using such devices as diary recordings of critical incidents and events, managerial reports, and other
John M. Ivancevich and Michael T. Matteson
192
forms of notes taken over a period of time allows the researcher to witness and recall changes first hand. This experience can result in better interpretations of data by the researcher. In summary, based both on the literature and personal experiences in conducting field studies using a longitudinal strategy, it is our contention that the advantages of such an approach are: 1. 2. 3. 4. 5.
6. 7.
The issue of causality can be addressed [ 141. The probability of hidden third-factor error can be lessened. Studying dynamic processes is possible. Time lags between cause and effect can be identified, resulting in a better understanding of causal relationships. The investigation of the impact of training and organizational change and development is more scientific and therefore has greater value to a manager. The researcher is able to remain in a situation to capture the actual changes occurring. The researcher is able to remain in an organization long enough to determine whether any changes take place.
Some Problems
of Longitudinal
Organizational
Research
In organizations, various processes are changing over a period of time. Consequently, it seems that managers would want to study these processes so that better decisions can be made about managing the changes. However, there are a number of practical drawbacks to what is described as longitudinal organizational research. Perhaps the most obvious is the fact that what are considered final results of such research efforts often require extended time periods. Few managers have the patience or job security to wait for “final” results before taking some action. The same type of patience requirement applies to academic researchers. There are few universities willing to wait patiently for years while one of its young nontenured, and perhaps best trained researchers, collects longitudinal data. Another major problem involves sustaining a representative sample over the course of the study. At the outset, a researcher may have many participants, but shrinkage of the sample due to personnel changes, illness, death, transfer, termination, or a change in attitude about participating in the research may be severe. These events are usually outside the control of the researcher and need to be considered when developing a longitudinal study.
Longitudinal
Organizational Research
193
Changes in either the internal or external organizational environment may operate to the detriment of a well-conceived longitudinal research plan. Internally, management may change, the organization may be restructured, or significant changes in technology may occur. fixternally, markets may change, competition may increase or lessen, the economy may falter or pick-up, any of which may have an impact upon research going on inside the organization. The passage of time also has an impact upon the techniques used in a study. As knowledge develops, new theories emerge, and new techniques appear which present improved alternatives to the original plan. Outdated theories and archaic instruments cannot be justified merely because they were the ones used at the beginning. Thus, a pressing dilemma facing a researcher involves making decisions to change instruments or observation methods during a study. Determining what time periods to use in a longitudinal study is a difficult decision. There is a wide variety of “causal intervals” used in longitudinal research. For example, some changes in organizational practices are relatively short-run, such as improving goal setting skills [23] , while other changes such as implementing a company-wide management by objectives program may require years [ 121. There are other changes involving structural and interpersonal factors that take at least 5 years [ IS] . Therefore, repeated observations must correspond with the time of actual change if the process is to be monitored accurately. A previously stated advantage of longitudinal research may also act as a potential disadvantage to the unwary researcher. Earlier it was suggested that the hidden third factor would tend to be minimized in a longitudinal design as more information was gathered over time. While it is probably not possible to learn too much, it is possible for the researcher to become overwhelmed by so much data and/or the possibilities of multiple causal chains that the research objectives are never met. This is a difficulty that is not inherent in longitudinal research, but occurs because of inadequate conceptualization of the problem by the researcher.
Examples of Longitudinal
Organizational
Research
The manner in which researchers deal with invalidity, tion, comparison and experimental groups, frequency
randomizaof measure-
194
John M. Ivancevich
and Michael
T. Matteson
ment, and other relevant issues in longitudinal research can be illustrated by a sample of studies. Effectiveness of Human Relations Training Hand and Slocum [ 111 used a longitudinal design to evaluate the effectiveness of a managerial human relations training program. An experimental group and a control group of 42 line and staff managers were randomly selected from one hierarchical level in a specialty steel plant. These researchers used a pre- and posttest strategy to measure training program effectiveness. By testing the change in correlations between the behavioral criterion and performance, they infer causality that is, what knowledge was acquired in the training was associated or linked with improved on-the-job performance. The study’s major strengths are not the degree of control exercised by the researchers, but the fact that they used an extended observation cycle to identify positive “sleeper effects,” groups were randomly selected, and after training, behaviors were assessed back on the job. The experimental “purist” could argue with the time-lag selection but longitudinal research at this level of rigor and that illustrated when discussing Goodacre’s [9] study are needed in the area of organizational training and development. Organizational Commitment and Turnover Employee turnover is a costly problem that all organizations face in varying degrees. Much of the organizational research on turnover has involved collecting attitude data at the point of departure or shortly thereafter through the use of exit interviews or questionnaires. A longitudinal study of the relationship between organizational commitment and turnover of managerial trainees in a large merchandising company was reported by Porter et al. [ 16 I . The subjects of this study were assigned to retail stores for 9 to 12 months of training. Data were collected during the 9-12 months of training and during the individuals’ first 3-6 months on a regular job. The longitudinal analysis revealed that commitment scores for stayers differed from those scores of leavers. The differences appeared as early as the first week of employment, with leavers lacking a positive attitude. These differences were identified with the longitudinal design used by the researchers. The researchers by using a longitudinal design were able to illustrate to managers that some type of “alarm” system could be used so that steps could be initiated to retain some of the “leavers.” A one-shot cross-sectional design would unlikely have been as effective.
Longitudinal
Organizational
Research
195
Technological Change and Job Attitudes A change in technology from batch to mass production in a hospital dietary department was examined using a longitudinal design employing structured interview data [ 11. A technological change affecting the assembly and delivery of meals for all patients was introduced. The research design is essentially quasi-experimental with measures obtained at unequal time intervals for different individuals. There were four waves of interviews, two before the change and two after. Randomization could not be practiced rigidly because of work scheduling constraints. The longitudinal design allowed the researchers to trace changes in job attitudes after applying the new technology. Performance and Job Satisfaction Wanous [22] gathered job satisfaction and performance data from female telephone operators. Cross-lagged analyses were used to study the data. The researcher, by using a 2-month time lag, correlated measures of telephone operators’ intrinsic and extrinsic job satisfaction with a performance index. His analysis showed extrinsic satisfaction at t, correlated more highly with t2 performance than vice versa, and that t, performance correlated more highly with intrinsic satisfaction at t, than vice versa. The causal inferences drawn were that satisfaction with organizational-mediated goals influenced performance, and that good performance influenced satisfaction with task-mediated rewards. Of course, as the researcher indicates, the conclusions must be treated with caution. This is because only female participants were included and the job sample was limited (telephone operators only). However, the passive quasi-experimental cross-lag method enabled Wanous [22] to examine multiple measures of both satisfaction and performance at two time points. This is a positive step in the direction of empirically investigating the important performance-satisfaction causal linkage. The major characteristics of each of these attempts at lontitudinal research are summarized in Table 2. These four studies involving training, turnover and commitment, technological change, and performance-satisfaction issues are not offered as ideal examples of scientific research or of quasi-experimental designs. Instead they are presented because they attempt to cope with changes and time by incorporating a longitudinal design. The researchers of each study admit that they have not achieved anything resembling the “pure” experimental research design.
Technological change and job attitudes
Performance satisfaction
Billings, Klimoski, and Breaugh (1977)
Wanous
(1974)
Organizational commitment turnover
Porter, Crampon, and Smith (1976)
and
and
Effectiveness of human relations training
Hand and Slocum (1972)
Research
Female telephone operators
Dietary personnel in a hospital
Retail management trainees
Line and staff managers in steel company
Sample
Organizational
Organizational Behavior and Management Topic
of Longitudinal
Researchers
Table 2: Summary
Inferred
Discussed specifically
Discussed specifically
Discussed specifically
Causality
Studies
The use of multiple measures of both satisfaction and performance can be examined at two data points for newly hired employees
Technological change may not have the anticipated effect and this is why it should be studied over a period of time
The need to develop an early alert system so that valuable human resources can be salvaged before individuals quit an organization
The importance of examining the transfer of learning phenomenon on the job by evaluating performance
Insight Provided to the Field of Knowledge
$
$
Longitudinal
Organizational Research
197
Some Future Suggestions to Consider The survey results shown in Table 1 as well as the points raised throughout this article clearly suggest that there currently exists no ground swell movement to conduct more longitudinal organizational research. Certainly this is not due to the lack of statistical and mathematical models now available which enable researchers to analyze and interpret longitudinal data. If the initiative is not taken by researchers, many of the opportunities to conduct such research are likely to be missed. Therefore, some suggestions for the future are offered to those researchers interested in becoming involved in longitudinal research. The managers of organizations, like the students cited earlier, need to be better informed about what longitudinal organizational research is and how it can be used to improve managerial effectiveness. In order to educate managers, researchers must be able to demonstrate to management the utility of longitudinal research results. It is the researcher who must reach an agreement with management to intrude upon an organization and conduct a study over a period of time that might span several years. Since managers are, in many instances, moving from job to job, they are not always receptive to a study that takes time. Managers typically want results now and are not so concerned with the impact of a technological change 2 years in the future. It is top management, however, that is usually developing 5- and lo-year plans. Thus, the aspiring longitudinal researcher must seek out those top-level executives who have a long-run time horizon and convince them that longitudinal research can be integrated meaningfully and effectively into such planning activities. Gaining entree into an organization requires marketing expertise as well. Personal experiences indicate that there are some top-level executives who will listen to a researcher and even allow that person to conduct a longitudinal study. Patience and perseverence are two important attributes of researchers involved in longitudinal research. We have also found that an endless discourse of research terminology is one excellent way not to gain access into top-level executive suites and eventually the company. Executives are not and do not wish to be conversant about experimental groups, treatment conditions, internal and external invalidities, quasi-experimental designs, interrupted time series, and other jargon of the academic community. Speaking the executives’ language, using descriptive phrases, and being concise are the most successful strategies to use.
198
John M. Ivancevich
and Michael
T. Matteson
A review of many of the national professional association meetings and proceedings indicates that numerous symposia are presented covering such varied and important topics as the quality of work life, leadership research, behavior modification, comparative organizational research, job design applications, and goal setting. Conspicuously absent from most agendas is any presentation of the issues, current state of affairs, and problems of longitudinal research. The professional meeting is the place to begin illustrating to students, colleagues, and managers how vital longitudinal research is and what it can contribute to the field of knowledge. The graduate student is another important party in stimulating more longitudinal research. Currently, longitudinal research is almost unheard of for dissertation research. Students, not unreasonably, are not willing to wait the necessary length of time to get their data. Perhaps a reasonable alternative would be for dissertation advisors to develop a research program that would allow one student to begin a rigorous longitudinal study and have another student subsequently follow it up. Or, perhaps several students could “plug into” a longitudinal study, which would at least expose the students to participation in an ongoing “long view” research project. One possible reason for the limited number of longitudinal research studies is that it is generally easier to complete crosssectional studies and this often means more publications in less time. If evaluations of academic researchers consider “counting pages” as one method of judging research productivity, longitudinal research efforts will suffer. Department chairs, deans, academic vice presidents, and similar agents may wish to carefully examine the merits of longitudinal research and encourage individuals to plan and execute realistic longitudinal projects. With improved understanding of the role and potential significance of such projects could come merit review processes that reward these efforts as a function of legitimate progress made rather than solely on ultimate publication. The mathematical and statistical problems associated with longitudinal research may appear to be significant [ 141. However, the need to use various “uncommon” statistical procedures should not limit a researcher’s enthusiasm to conduct longitudinal research. Many excellent source books and examples are available. Popular statistical procedures of longitudinal research include causalcorrelational models, repeated measures analysis of variance, and
Longitudinal
Organizational Research
199
regression analysis, among others. Each of these techniques is discussed in detail in multivariate statistics books and various articles. Many researchers who lack confidence in their abilities in this area have access to colleagues who do have the needed expertise. Indeed, one potential advantage in undertaking longitudinal research projects is the collaboration that may be needed and that results in mutually rewarding symbiotic relationships for the research team. Another suggestion would be to encourage prominent journals to occasionally devote an entire issue to longitudinal organizational research. Journals have devoted issues to cascaded inferences, organizational development practice and theory, comparative management, bureaucratic organizations, and leadership. Why not longitudinal organizational research? Perhaps each of these other special issues could have been improved if more longitudinal research designs were included to present the area covered. By using an entire issue, researchers, students, and managers will be presented with longitudinal research ideas, suggested projects, and findings. This type of stimulation can result in other researchers attempting longitudinal research. In a similar vein, just as several journals have added “short notes” sections to their publication format, why not add a research methodology section in which one or more pieces would be published each issue? Perhaps researchers working on longitudinal projects may be encouraged to use such an outlet. Also worthy of consideration might be a refereed “longitudinal research abstract” section in which researchers could report in a brief paragraph current longitudinal projects as an information and stimulation device for other investigators. In writing this article, we have attempted to illustrate some characteristics, issues, examples, and needs of longitudinal organizational research. It should be obvious at this point that longitudinal research designs are not the solution to all problems facing managers and organizational researchers. The implementation of a longitudinal design in an organizational setting requires patience, creativity, and understanding. There is no easy and quick set of longitudinal research suggestions available. In fact, most of the suggestions and information presented in this article indicate that work, challenge, and problems lie ahead for those researchers initiating longitudinal research projects. The personal payoffs, however, can certainly be worth the costs that must be borne.
200
John
M. Ivancevich
and Michael
T. Matteson
References 1.
Billings, R. S., Klimoski, R. J., and Breaugh, J. A., The Impact of a Change in Technology on Job Characteristics: A Quasi-Experiment, Administrative Sci. Q. 22 (1977): 318-339.
2.
Campbell, D. T., Quasi-Experimental Design, in International Encyclopedia of the Social Sciences, 5, D. L. Gills, ed., Macmillan and Free Press, New York, 1968.
3.
Campbell, D. T., and Stanley, J. D., Experimental for Research, Rand McNally, Chicago, 1963.
4.
Cherrington, contingent formance,J.
5.
Cook, T. D., and Campbell, D. T., The Design and Conduct of Quasi-Experiments and True Experiments in Field Settings, M. Dunnette, ed., Rand McNally, Chicago, 1976, pp. 223-326.
6.
Evans, M. D., Opportunistic Organizational Research: Designs, Acad. Management.I. 18 (1975): 98-108.
7.
Feldman, J., Considerations in the Use of Causal-Correlational Applied Psychology, J. Appl. Psychol. 60 (1975): 663-670.
8.
Goldstein, Monterrey,
9.
Goodacre, D. M., Experimental Evaluation of Training, tion Industrial Relations 2 (1955): 143- 149.
and Quasi-Experimental
D. J., Reitz, H. J., and Scott, W. E., Effects of Contingent Reward on the Relationship Between Satisfaction and Appl. Psychol. 35 (1971): 531-536.
I., Training: Calif., 1974.
Program
Development
The
Role
Designs
and NonTask Per-
of Patch-Up
Techniques
and Evaluation.
in
Brooks/Cole,
J. Personnel AdministraBusiness Horizons
10.
Greene, (1972):
11.
Hand, H. H., and Slocum, J. W., Jr., A Longitudinal Study of the Effects of a Human Relations Training Program on Managerial Effectiveness, J. Appl. Psychol. 56 (1972): 412-417.
12.
Ivancevich, J. M., Changes in Performance in a Management gram, Administrative Sci. Q. 19 (1974): 563-574.
13.
Kerlinger, F. N., Foundations New York, 1973.
14.
Kimberly, J. R., Issues in the Design of Longitudinal Social. Methods Res. 4 (1976): 321-347.
1.5.
Likert,
16.
Porter, L. W., Crampon, W. J., and Smith, F. J., Organizational Commitment and Managerial Turnover: A Longitudinal Study, Organizational Behav. Hum. Perform. 15 (1976): 87-98.
17.
Porter, Richard
18.
Rozelle, R. M., and Campbell, D. T., More Plausible Rival Hypotheses in the Cross-Lagged Panel Correlation Technique, Psychol. Bull. 7 I (1969): 74-80.
C. N., The Satisfaction-Performance 31-41.
Controversy.
of Behavioral Research, Holt,
R., The Human Organization, McGraw-Hill,
L. W., and Lawler, D. Irwin, Homewood,
by Objectives
15
Pro-
Rinehart
& Winston,
Organizational
Research,
New York,
E. E., III, Managerial Attitudes Ill., 1968.
1967.
and Performance,
Longitudinal 19.
20.
Organizational
Research
201
Seiler, John A., Systems Analysis in Organizational Behavior, Richard D. Irwin and The Dorsey Press, Homewood, Ill., 1967. Simon, H. A., Spurious Correlations:
A Causal Interpretation,
J. Am. Statistical
Assoc. 49 (1954): 467-479.
21.
Sims, H. P., Jr., and Wilkerson, D. A., Time-Lags in Cross-Lag Correlation Studies: A Computer Simulation, Decision Sci. 8 (1977): 630-644.
22.
Wanous, J. P., Causal-Correlational Analysis of the Job Satisfaction formance Relationship, J. Appl. Psych& 59 (1974): 139- 144.
23.
Wexley, K. N., and Nemeroff, W. F., Effects of Positive Reinforcement and Goal Setting as Methods of Management Development, J. Appl. Psychol. 60 (1965): 446-450.
and Per-