Problems with detecting moderators in leadership research using moderated multiple regression

The Leadership Quarterly 14 (2003) 3 – 23

Problems with detecting moderators in leadership research using moderated multiple regression Jennifer R. Villa*, Jon P. Howell, Peter W. Dorfman, David L. Daniel Department of Management, New Mexico State University, Box 30001/3DJ, Las Cruces, NM 88003, USA Accepted 31 October 2002

Abstract A number of recent leadership studies have questioned whether situational variables are important moderators of leadership effectiveness. Pessimistic conclusions from these studies regarding situational modifiers challenge the foundations of path – goal and substitutes for leadership theories. However, analysis of this research reveals questionable methodological practices that cast doubt on the validity of these conclusions. This article discusses two methodological issues, elucidates specific flaws in methods used in recent leadership studies, and makes recommendations for the use of moderated multiple regression (MMR) in leadership studies. We argue that low power to detect moderators and inappropriate use of regression methods can account for the lack of confirmatory findings regarding moderators. Comparative analysis using a previously published data set provides strong support for major arguments presented in this article. We conclude that situational variables are important moderators of leadership effectiveness and are detectable using appropriate procedures. D 2002 Elsevier Science Inc. All rights reserved.

1. Introduction Practitioners and leadership researchers have long assumed that the effectiveness of leaders is dependent upon situational factors. These situational variables are referred to as moderators if they interact with a leader behavior to change the leader’s impact. Recent debate has focused on the prevalence of moderator variables and their interpretation. Conclusions from

* Corresponding author. Tel.: +1-505-646-1201; fax: +1-505-646-1372. E-mail addresses: [email protected] (J.R. Villa), [email protected] (J.P. Howell), [email protected] (P.W. Dorfman), [email protected] (D.L. Daniel). 1048-9843/02/$ – see front matter D 2002 Elsevier Science Inc. All rights reserved. PII: S 1 0 4 8 - 9 8 4 3 ( 0 2 ) 0 0 1 8 4 - 4

4

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

recent studies challenge the notion that situational variables are important moderators of the effect of leader behaviors on criteria (Podsakoff & MacKenzie, 1997; Podsakoff, MacKenzie, Aherne, & Bommer, 1995; Podsakoff, MacKenzie, & Fetter, 1993; Podsakoff, Niehoff, MacKenzie, & Williams, 1993). If these conclusions are correct, the validity of two major streams of leadership research—path–goal (House, 1971, 1996; House & Dessler, 1974; House & Mitchell, 1974) and substitutes for leadership theories (Howell, Dorfman, & Kerr, 1986; Kerr & Jermier, 1978)—must be questioned. Researchers in other areas of organizational behavior, human resources management, industrial/organizational psychology, and related disciplines have also noted difficulties in detecting the existence of hypothesized moderating effects (e.g., Aguinis, 1995; Aguinis & Stone-Romero, 1997; Dunlap & Kemery, 1988; McClelland & Judd, 1993). A number of factors have been identified across various literatures that affect the ability to detect moderators using moderated multiple regression (MMR). This article discusses these methodological issues, elucidates specific flaws in methods used in recent leadership studies, and makes recommendations for the use of MMR in leadership studies. We also provide data from previously published leadership studies to demonstrate our main points.

2. Moderated multiple regression MMR is the preferred statistical method for identifying moderator effects (interaction effects) when the predictor and the moderator are continuous variables or when the predictor is continuous and the moderator is categorical (Aiken & West, 1991; Cohen & Cohen, 1983; McClelland & Judd, 1993; McNeil, Newman, & Kelly, 1996; Stone-Romero, Alliger, & Aguinis, 1994). ANOVA can also be used for identifying interactions, but is more appropriately used for the analysis of planned experiments than for observational and survey data (Aiken & West, 1991). A moderated relationship is one in which a variable (Z) interacts with a predictor variable (X) to change the relationship between the predictor and an outcome (criterion) variable ( Y). MMR provides a straightforward method of testing whether the form of the relationship (represented graphically by the slope of the regression line) changes with the addition of a moderator (Podsakoff et al., 1995; Stone & Hollenbeck, 1989). Changes in form are clearly hypothesized in both the substitutes for leadership theory and path–goal theory. To test for a moderated relationship using MMR, an interaction term that is formed as the product of the independent variables, is added to a regression equation that first contains the variables of which the interaction term is comprised. For example, if we were interested in whether a variable Z moderates the relationship between some variable X and a criterion Y, we would express that interaction as XZ. In the regression equation, the interaction is carried by the XZ term as shown below. Y ¼ a þ b1 X þ b2 Z þ b3 XZ

ð1Þ

The main effects of the X and Z variables are linearly partialled from the XZ term by entering the X and Z variables into the regression equation along with the XZ term. Both the lower order

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

5

terms (X and Z) and the higher order term (XZ) are entered into the equation simultaneously to test the significance of each term. If the interaction term (b3) is significant, that is, if XZ contributes to the prediction of Y over and above the other predictor variables in the equation, a moderated relationship is indicated (Aiken & West, 1993; Cohen & Cohen, 1975, 1983). Once the beta coefficients have been estimated, it is possible to vary the moderator (Z) to produce a family of simple regression lines representing the effect of the variable X on criterion Y. If the interaction term, b3, is significant, the resulting family of simple regression lines will be nonparallel. Graphing a two-variable model when b3 is not significant would result in the family of simple regression lines being parallel. In this case (i.e., b3 is not significant), a reduced model (lower order terms only) may be investigated by comparing the two-variable model with single-variable models and with the intercept-only model. For the interested reader, a more thorough description of MMR has been provided by several authors (Aiken & West, 1991; Bobko, 1995; Cohen & Cohen, 1983; Jaccard, Turisi, & Wan, 1990; McNeil et al., 1996). 2.1. Commonly recognized problems in detecting moderators using MMR Researchers, particularly those doing field research, have frequently had difficulty finding hypothesized moderated relationships (Champoux & Peters, 1987; McClelland & Judd, 1993). Many of these problems are due to factors that decrease statistical power. With MMR, statistical power is the probability of detecting a moderator effect in a sample when a moderator effect of specified magnitude exists in the population (Aiken & West, 1991). Among those factors that have been found to decrease the power to detect interactions are small sample size (Cohen & Cohen, 1983; Cohen, 1988), unequal subgroup sizes (Aguinis, 1995; Hsu, 1993), range restriction (Aguinis, 1995; McClelland & Judd, 1993), measurement error in the predictor variables that make up the interaction term (Aiken & West, 1991; Dunlap & Kemery, 1988), scale coarseness (Aguinis, Bommer, & Pierce, 1996; Russell & Bobko, 1992), and predictor intercorrelation (Aguinis, 1995; Morris, Sherman, & Mansfield, 1986). The impact of the above factors on the power to detect interaction effects has been broadly recognized and most of these factors are being addressed by researchers who use MMR. However, a number of issues have surfaced in recent leadership studies and cast doubt on the appropriateness of the ways in which multiple regression procedures are being used. In the following sections, these issues are addressed using examples from two streams of research on path–goal theory and the substitutes for leadership model: the approaches used by Howell and his colleagues (Howell & Dorfman, 1981, 1986; Howell et al., 1986) and those used by Podsakoff and his colleagues (Farh, Podsakoff, & Cheng, 1987; Podsakoff, Mackenzie, & Bommer, 1996; Podsakoff, MacKenzie, et al., 1993; Podsakoff, Niehoff, et al., 1993; Podsakoff, Todor, Grover, & Huber, 1994), respectively. 2.2. Recent problems in detecting moderators in leadership studies A major portion of leadership research throughout the 1970s and early 1980s was conducted to identify moderator variables. Although numerous moderators were found,

6

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

some researchers have recently concluded that the results are little more than what might be found by chance (Podsakoff et al., 1996; Podsakoff, Niehoff, et al., 1993). Their conclusions, however, are problematic for at least two reasons. First, the results underlying their conclusions may be directly affected by the number of atheoretical tests for interaction effects that they conducted. Second, the inclusion of extraneous predictor variables in the regression equations reduces the likelihood of finding interaction effects that exist. Although our critique focuses on studies of leadership moderators, the theoretical and methodological problems addressed are germane to all research involving moderator variables. 2.2.1. Theoretical rationale and the number of tests Howell et al. (1986) appropriately required a theoretical reason for testing for the presence of a moderated relationship. For example, as path–goal theory suggests, one should expect that environmental stress would moderate the effect of supportive leadership on follower satisfaction. Kerlinger and Lee (2000, p. 29) support the theoretical requirement by suggesting, ‘‘Investigators who do not hypothesize relations in advance, in short, do not give the facts a chance to prove or disprove anything.’’ In addition to the theoretical rationale, Howell and colleagues required that the leader behavior have an effect on the criterion variable before a test for a moderator could be conducted. This condition was operationalized by requiring the leader behavior to have a significant effect when the criteria were regressed on the leader behavior alone, as represented by the following equation: Y ¼ a þ b1 LB:

ð2Þ

In hindsight, the requirement that the leader behavior show a main effect prior to the addition of the moderator variable may have been too restrictive. Although most types of moderated relationships will indicate a main effect for the leader behavior prior to the addition of the interaction term, a few will not. Technically, significant interactions may occur without the main effects of the predictor variables. In addition, interactions in which the leader behavior has opposite effects at high and low levels of the moderator may mask the main effect of the leader behavior prior to the addition of the interaction term (Aiken & West, 1991). Podsakoff et al. (1993, 1996) recognized the fact that moderators can exist without the presence of a leader behavior main effect and reflected this in their analysis. However, with one possible exception (Podsakoff & MacKenzie, 1997), they did not use a theoretical rationale before testing for moderated relationships. Instead, they tested every possible interaction by regressing each criterion against every leader behavior and situational variable they measured, whether there was a hypothesized theoretical relationship or not (Farh et al., 1987; Podsakoff et al., 1996; Podsakoff, MacKenzie, et al., 1993; Podsakoff, Niehoff, et al., 1993; Podsakoff et al., 1994). In a recent review of 73 empirical studies of the substitutes for leadership and path–goal theories, Podsakoff et al. (1995) expressed

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

7

disappointment when so few of the tests for moderators were found to be significant. They state: Quite frankly, we find that the lack of support for the moderating effects predicted by the path–goal and substitutes for leadership models both shocking and disappointing. (p. 465) By not allowing theory to guide their analysis, we believe these researchers have not conducted a fair test of the potential moderators. In a recent publication, Podsakoff and MacKenzie (1997) reported an analysis that limited their tests to the original leader behaviors, moderators, and criteria reported by Kerr and Jermier (1978). Even in this case, however, no theoretical hypotheses were offered which indicated why these leader behaviors or moderators should be important in the organizations that were sampled. Podsakoff, MacKenzie, et al. (1993) themselves recognize that such methods are questionable by stating: ‘‘There is no theoretical reason to expect that these particular moderating effects would exist in the first place.’’ They argue, however, that the lack of consistency in the moderator findings should allow such exploratory methods. We acknowledge that exploratory methods may be acceptable when building a theory, but they may obscure results when testing a theory. We argue that using non-theory-based tests greatly underestimates the presence of moderators where they should exist. For instance, in their 1995 review, Podsakoff et al. (p. 457) state that, ‘‘Of the 4303 tests1 of differences in form reported, 456 were significant.’’ This represents about 10.55% of significant findings for all moderator studies combined. They fail to inform the reader that of the 4303 tests they reported, 3476 are accounted for in the five articles by Podsakoff and colleagues (Farh et al., 1987; Podsakoff et al., 1996; Podsakoff, MacKenzie, et al., 1993; Podsakoff, Niehoff, et al., 1993; Podsakoff et al., 1994) in which they used a nontheoretical approach. Table 1 compares the results of theory-driven tests of form2 with the nontheoretical studies by Podsakoff and colleagues. These studies are listed in Appendix A. Although all of these studies were cited in the review by Podsakoff et al. (1995), comparing the non-theory-based and theory-driven approaches demonstrates interesting differences—21.11% (179) of the interactions tested were significant with the theory-driven studies, a much less disappointing number. Thus, the theory-driven approach has been considerably more successful at identifying important moderator variables as key elements of contingency leadership theories than indicated in that 1995 review. Considering the information in Table 1 reveals the questionable value of reporting the overall percentage of times all moderators were found to

1

The total number of tests of form discussed in Podsakoff et al. (1995) appears to be 4324, not 4303 as reported; 848 tests of form in 40 studies plus the 3476 from the five articles by Podsakoff et al. sum to 4324 not 4303. The list of studies is included in Podsakoff et al. (1995). 2 Podsakoff et al. (1995) separated the 73 empirical studies into tests of form (identified by changes in the slope of the regression line) and degree (identified by differences in correlation coefficients or by the spread of points about the regression line). Forty-five studies included tests of form.

8

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

Table 1 Comparison of Podsakoff and colleagues’ findings with those of 40 other empirical studiesa Empirical studies

Total tests

Significant interactions

Five recent articles by Podsakoff et al. Forty other path – goal and Substitutes articles that report form-type tests for moderators Total

3476

277

7.97

848

179

21.11

4324

456

10.55

a

Percentage of significant interactions

See Appendix A for a list of these studies.

be significant in numerous studies of different organizations. It would be more instructive for the field and for working managers to have researchers report the number or percentage of times a hypothesized moderator occurs across a number of samples. For example, Howell, Dorfman, Hibino, Lee, and Tate (2000) recently found perceived leader expertise to be a significant moderator of path–goal leader behaviors in 62% of the tests they conducted across organizational samples from five countries. Organizational formalization has also been found to be an important substitute for leadership in military organizations (Dorfman, Howell, Cotton, & Tate, 1992) and an important moderator of leader reward and punishment behaviors on various measures of satisfaction in other samples (Podsakoff, Dorfman, Howell, & Todor, 1986; Podsakoff, MacKenzie, et al., 1993; Podsakoff, Niehoff, et al., 1993; Podsakoff et al., 1994). Formalization also moderated the effect of a leader’s articulating a vision, fostering group goals, and intellectual stimulation in another recent study (Podsakoff et al., 1996). Reporting the frequency that a particular moderator is found to be significant would give us a better understanding of which situational variables are more critical for managers to be aware of and to possibly manipulate. 2.2.2. Multiple atheoretic independent variables Our next point concerns the inappropriateness of using numerous atheoretic predictors in each regression equation when testing for a particular moderator since multiple predictor variables often lead to collinearity problems. If the objective in performing regression is prediction, collinearities among predictors may have little impact on one’s success because the relative importance of each predictor is less important to the researcher than the ability of the model to predict. However, if the goal is to model a relationship or test a theory about the importance of a predictor variable (e.g., a leader behavior, a moderating factor, or their interaction), then multicollinearities can have a major impact on the results. The presence of multicollinearities causes the variances of the estimated regression coefficients, and hence the variance of the test statistics, to increase. This can occur to the point that predictors that should have strongly positive regression coefficients may produce negative coefficient estimates. The opposite may also occur, where a normally negative predictor may produce a positive coefficient (Mason, Gunst, & Webster, 1975). Thus, testing the

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

9

importance of a specific predictor variable involved in a multicollinearity can be misleading. This consequence makes sense when one considers that such tests are based on the amount of additional variability in the criterion that each predictor variable being tested accounts for after all other predictor variables in the model have accounted for variability in the criterion. This implies that, if there are important but related predictors in the model, often none of them will appear significant because each of the related predictors accounts for very little of the response variability after the other predictors have been considered (Cohen & Cohen, 1983). This is the nature of testing in regression, and the statistically sound implication is that collinearities must be substantially removed before any testing can be conducted. The ideal solution is to do additional sampling, adding observations with combinations of variables that will reduce/remove the collinearities, but this is a complicated issue and generally not possible in field research. Thus, the implication is that one must be careful in the process that is used to test theories using regression. It was noted earlier that when testing for interaction effects, Podsakoff and colleagues enter all the leader behaviors (4–7) and all the situational variables (13–24) from their surveys into a single regression equation, before testing for the interaction term of interest (Podsakoff, MacKenzie, et al., 1993; Podsakoff, Niehoff, et al., 1993; Podsakoff et al., 1994). This approach leads to the multicollinearity problems discussed above, including the decreased ability to detect important moderator effects. High levels of multicollinearity are evidenced by high variance inflation factors (VIFs), which are commonly used in regression as a diagnostic for multicollinearity. The VIF is a statistic, which indicates for a specific coefficient, the strength of the relationship between the independent variable and all other independent variables in a multiple regression model (Mendenhall & Sincich, 1992). When the variance of the regression coefficients gets large, the estimates of the coefficients can exhibit large swings from data set to data set (often switching sign). Hence, any coefficient estimates obtained from a data set where the predictor variables have substantial multicollinearity can scarcely be trusted. As Cohen and Cohen (1975, p. 160) state: ‘‘Having more variables when fewer are possible increases the risk of both finding things that are not so and failing to find things that are.’’ Podsakoff, MacKenzie, et al.’s (1993, p. 39) reason for their procedure is to ‘‘reduce the risk of finding the same relationship over and over with different leader behaviors that are correlated with each other.’’ Their rationale is to control for potential covariation among the leader behaviors and all the possible moderator variables (Podsakoff & MacKenzie, 1997). With this approach, however, they are more likely to commit Type II errors (that is, fail to find interactions that do exist). Cohen and Cohen (1983) indicate that when controlling for confounding variables, researchers should give careful consideration to their presumed causal priority and include only those variables that should logically precede the predictors of interest (see also McNemar, 1969). Controlling for all leader behaviors and moderator variables that were originally proposed in early theory building is an attempt to make a very general model fit all

10

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

organizational situations. Tosi and Kiker (1997) and Tosi and Banning (1998) have recently argued that the type of substitutes found will vary with the type of organization. In support of their argument, researchers recently reported a study in which they found significant differences among organizations in the level of certain substitute variables (Dionne, Yammarino, Atwater, & James, 2002). The notion of meso-theories implies the need to be more limited and specific in the domains of our predictions to allow for more accurate testing of causal and moderated relationships. Some of the research on leadership substitutes has used aggregated samples from numerous organizations of widely varying type (Podsakoff et al., 1996). Rather than trying to apply all measured leader behaviors and the entire set of originally proposed substitutes to all situations, we should take Cohen and Cohen’s advice and limit our research predictions to specific predictors and moderators that should be relevant to the organization under study. This should also produce results that are more valuable to practitioners. Recent leadership researchers recognized the importance of limiting the number of predictors within a single regression equation (Dionne et al., 2002). They purposefully selected a single leader behavior and a single situational variable in each regression equation to test for moderator effects. Of the moderator studies reported in Appendix A, not one of the non-Podsakoff articles used this data analytic procedure. It thus appears, that the practice of using numerous atheoretic leader behaviors and moderators as predictors in a single regression equation to test for interaction effects is not only questionable statistically, it does not conform with current practice by other leadership researchers.

3. Comparative analysis To demonstrate our major points, we took the suggestion of one reviewer and reanalyzed an existing data set. Portions of this data set were published in one of our earlier studies (Dorfman et al., 1992). We acknowledge that this data analysis can only serve as an illustration of analysis properties. Results of comparing different methods of data analysis using a single data set may vary widely if multicollinearity is present. The data were derived from a military sample of commissioned (CO) and noncommissioned officers (NCO) ranging in rank from colonels to staff sergeants. The original study compared the impacts of several leader behaviors and specific situational variables (independent variables) on the attitudes and performance (dependent variables) of COs and NCOs. There were 264 individuals in the sample; all were on active duty holding leadership positions in air defense artillery units in the western United States. Respondents completed a leadership survey questionnaire that asked them to describe their own leader’s behavior and several possible moderator variables (substitutes, neutralizers, or enhancers) that are described below. The leadership behaviors included in this survey were all believed to be potentially important in this military organization and included the following: directiveness—clarifying

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

11

performance expectations, assigning tasks, specifying methods to be used; supportiveness— showing a concerned and caring attitude to followers; participativeness—obtaining input from followers for important decisions; contingent rewards—providing social rewards to followers (e.g., compliments) for good performance; contingent punishment—providing aversive social consequences to followers for poor performance; charisma—inspiring and developing confidence among followers; representativeness—supplying, networking, and buffering followers from unreasonable demands. The scales for directiveness and supportiveness were derived from those developed by Schriesheim (1978) for path–goal theory testing. The reward and punishment scales were developed by Podsakoff and Skov (1980). The participativeness and charisma scales were modified from work by House (1984, personal communication) and Yukl (1982). The representativeness scale was modified from Yukl. All leader behavior scales resulted in coefficient alpha reliabilities of .87 or above. Kerr and Jermier’s (1978) original substitutes for leadership scales were used to measure the 13 dimensions they hypothesized to neutralize or substitute for leadership effects on followers. These scales included the following potential moderator variables: follower’s ability, experience, training, and knowledge; follower’s professional orientation; follower’s indifference to organizational rewards; unambiguous routine, methodologically invariant work tasks; tasks providing feedback concerning accomplishment; intrinsically satisfying tasks; organizational formalization; organizational inflexibility; advisory and staff support; closely knit cohesive work groups; organizational rewards not in the leader’s control; spatial distance between superior and subordinate; and follower’s need for independence. The coefficient alpha reliabilities for these scales ranged between .56 and .87. A few minor modifications were made to adapt their instrument to the military sample. Attitudes included in this survey were satisfaction with supervision and satisfaction with work, measured by the Minnesota Satisfaction Questionnaire (Weiss, Dawis, England, & Lofquist, 1967) and organizational commitment was assessed using the Porter and Smith (1970) questionnaire. Reliabilities for these scales were all above .82. For purposes of parsimony and comparability, a single performance measure for each CO and NCO was obtained by combining various measures of job performance found in each soldier’s performance evaluation report. The following are examples of 14 performance dimensions that were measured and combined: possesses capacity to acquire knowledge/grasp concepts, demonstrates appropriate knowledge and expertise in assigned tasks, and maintains appropriate level of physical fitness. These dimensions consistently appear in contemporary appraisals of military personnel (McHenry, Hough, Toquam, Hanson, & Ashworth, 1990). We analyzed their data to detect moderator variables using two approaches. First, we examined the moderating effects of all 13 substitute/neutralizer variables that were included in Kerr and Jermier’s (1978) model along with the seven leader behaviors that were surveyed in this sample. All the data were standardized to running the analyses using the Friedrich (1982) approach in which one first calculates z scores for the leader behavior and moderator variables and then forms their cross product. This approach to standardization

12

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

preserves the relationship between the raw and standardized solutions. Interaction terms were created for all 13 moderators combined with each of the seven leader behaviors, resulting in 13  7 = 91 interaction terms created for each criterion measure. Since we used four criterion measures, there were a total of 91  4 = 364 interactions created using this method. We next applied the approach utilized by Podsakoff and colleagues where all leader behaviors and all 13 substitute/neutralizer variables were entered into the regression equation before a single interaction term was entered and tested. Then we applied the regression approach we have recommended in this article where only a single leader behavior and a single situational variable were entered into the regression equation prior to entering and testing the interaction effect, which is composed of these two main effect variables. The purpose of this comparative analysis was to investigate the argument that our suggested analytical method will have greater ability to detect influential moderator variables than the method used by Podsakoff and colleagues. The approach utilized by Podsakoff resulted in somewhat high VIFs on many of the variables. If no two independent variables are correlated, then all the VIFs will be one. A VIF in the region of 3–4 or more suggests problems with multicollinearity (Montgomery & Peck, 1992). The regression models that included all leader behaviors and situational variables generally had several variables with VIFs between 3 and 4 for each regression (maximum VIF was 4.24). Using our suggested approach, VIFs were much lower (maximum 1.62). Table 2 compares the results of these two analytical approaches on the number of significant interactions found. For three of the four criterion measures, our suggested analytical approach yielded a higher percentage of significant interaction effects than the approach used by Podsakoff and colleagues. For all four criteria, the percentage of interactions was considerably greater than that which might be expected by chance. The only criterion that yielded a higher percentage of significant interactions with the Podsakoff and colleagues’ approach was satisfaction with supervision. Both analytical approaches yielded an impressive number of significant interactions for this criterion—27% and 23%, respectively. However, it is difficult to have confidence in the results found Table 2 Analysis of 13 original substitutes variables using alternative analytical approaches Criterion measure

Significant interactions * found utilizing approach used by Podsakoff and colleagues

Significant interactions * found utilizing approach suggested by this article (Villa et al.)

Organizational commitment Job performance Satisfaction with work Satisfaction with supervisor

8 (9%) 7 (8%) 12 (13%) 25 (27%)

13 16 18 21

* P < .05 for all interactions tested.

(14%) (18%) (20%) (23%)

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

13

under Podsakoff’s method due to problems with multicollinearity indicated by the high VIFs. We then conducted another analysis that differed in a single respect from the one just described. We included all seven leader behaviors that were surveyed and those substitute/ neutralizer variables that seemed particularly relevant to the organization being studied. The situational variables that we believed to be most relevant to this military organization were the existence of organizational formalization; closely knit cohesive work groups; follower’s ability, experience, training, and knowledge; and advisory staff groups. Each of these organizational and individual characteristics was emphasized in the military units studied. These units worked intensively with highly technical military hardware. Extensive formal policies and procedures governed the actions of these military personnel. Continuous training was conducted to increase follower abilities and job knowledge, cooperation, and cohesion was emphasized by officers, and staff training officers visited the units regularly to update their skills and acquaint them with new technology and operational procedures. This resulted in 4  7 = 28 interaction terms being tested for each criterion measure, for a total of 4  28 = 112 interactions tested for all four criteria. We applied the analytical approach of Podsakoff and colleagues as well as our suggested approach. The purpose of this second comparative analysis was to investigate a different argument. Recall that we argued that it was questionable to test for moderator effects that were not expected to be important in the organization being studied. We hypothesize that when one tests only those situational variables that are relevant to the organization, our approach should produce a higher percentage of significant interaction effects than when all the situational variables that were measured are tested. The results of this comparative analysis are shown in Table 3. Comparing the results shown in Table 3 with those in Table 2 shows considerable support for our argument. Using our suggested analytical approach, testing for moderator effects using only those situational variables thought to be relevant to this military organization produced sizable increases in the percentage of significant interactions obtained. This occurred for three of the four criterion measures studied. Taken together, the two comparative analyses shown in Tables 2 and 3 provide strong support for two of Table 3 Analysis of four relevant substitutes variables using alternative analytical approaches Criterion measure

Significant interactions * found utilizing approach used by Podsakoff and colleagues

Significant interactions * found utilizing approach suggested by this article (Villa et al.)

Organizational commitment Job performance Satisfaction with work Satisfaction with supervisor

7 2 3 7

9 8 7 6

* P < .05 for all interactions tested.

(25%) (7%) (11%) (25%)

(32%) (29%) (25%) (21%)

14

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

our major arguments presented in this article. First, our suggested analytical method (using fewer predictor variables in each regression) demonstrated more power to detect important moderators of leader behavior–criteria relationships than the method used by Podsakoff and colleagues. Second, when our analytical approach was used to test only those situational variables thought to be relevant to the organizational sample under study, a higher percentage of important moderators was identified than when a larger set of situational variables was tested. While the analysis of a single data set does not prove the superiority of our method, it together with reasoned statistical and logical argument certainly casts doubt on the wisdom of including all leader behaviors and potential moderators in the regression equation, and of conducting numerous atheoretic tests.

4. Discussion and recent theoretical developments Recall that the purpose of our article was to investigate the validity of assertions recently made concerning the impact of moderator variables in leadership research. After careful analysis, we found the conclusions of Podsakoff and colleagues are suspect. Much of the research on which these conclusions are based contains questionable methodological practices. Practices such as testing 910 relationships (whether or not they are hypothesized) or including 28 variables in a regression equation (whether or not there is a logical reason to assume they are related to the interaction being studied) result in errors that create bias against finding significant interactions. Moderators (identified by interactive regression models) are indeed important to our understanding of leadership effectiveness (Larson, Hunt, & Osborn, 1976). These situational variables also often have important main effects on workplace criteria (Podsakoff et al., 1995) even for multiple levels of analysis (Podsakoff & MacKenzie, 1995). Both the interactive and main effects of these variables increase our understanding of leader’s influence processes in organizations (Howell & Costley, 2001). Although this article has addressed methods issues in the study of leadership moderators, several recent theoretical developments deserve comment. In a recent issue of The Leadership Quarterly, Jermier and Kerr (1997) indicated there has been very little conceptual/theoretical work aimed at elaborating substitutes for leadership theory. We believe these writers have overlooked several developments. The addition of leadership enhancers (Howell et al., 1986) importantly complements the two constructs of neutralizers and substitutes originally proposed by Kerr and Jermier (1978). Including situational factors that increase a leader’s effect on followers expands the number of potential moderators and makes substitutes for leadership theory more meaningful for practicing managers. DeVries (1997) recently recommended a construct called need for leadership as a potential moderator of leadership behaviors. Vecchio (1990) has proposed that leader characteristics might be another source of factors that can impact leadership effects. Inclusion of leader characteristics should increase the predictive power of contingency

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

15

theories of leadership. House (1996) recently proposed several task, organizational, group, and follower characteristics as important leadership moderators in his revision of path–goal theory. Numerous other situational variables have also been suggested as moderators of leadership effects. These include self-managed work teams where employees rely on people other than their appointed leader to help organize their work and to obtain satisfaction from their organizational role (Jermier & Kerr, 1997); the McDonaldization of society, where complete specification of processes, and procedures leave little need for a leader’s direction Jermier & Kerr; organizational culture and values, where workers who share organizational values have little need for much supervision (Podsakoff & MacKenzie, 1997); subordinate cynicism, where followers’ suspicion and lack of trust make followers less responsive to leaders Podsakoff & MacKenzie; and leader’s rank and expertise, which add different types of power to a leader’s requests, enhancing the impacts on followers (Dorfman et al., 1992). Much has happened in organizations in the 20 plus years since the seminal articles on leadership substitutes and path–goal theory were published. We believe current researchers should consider these recent developments when selecting possible moderator variables for their studies. Future research should also reflect recent findings that show that many situational variables can have important main effects of their own on follower criterion variables (Podsakoff & MacKenzie, 1997), particularly at the individual level of analysis (Podsakoff & MacKenzie, 1995). Dionne et al. (2002) recently demonstrated that the regression approach suggested here is also appropriate for group level analysis. A new approach to within and between analysis uses multiple regression and the approach advocated in this article to test group- and individual-level effects (Schriesheim, 1995; Schriesheim, Castro, & Yammarino, 2000; Schriesheim, Cogliser, & Neider, 1995; Schriesheim, Neider, & Scandura, 1998).

5. Recommendations for moderator research The likelihood of identifying important moderator variables can be improved by recognizing and addressing problems that decrease statistical power. Large samples, balanced subgroups, reliable measures, and continuous (rather than interval) scales have been prescribed to deal with many of these problems (Aguinis, 1995; Aguinis et al., 1996; Aiken & West, 1991; McClelland & Judd, 1993; Russell & Bobko, 1992). To give credit to the recent leadership research cited above, the researchers have often avoided these problems. The sample sizes were large and diverse, and scale reliabilities cited in the articles were generally high. Yet, their results were disappointing. Our analysis leads us to the following recommendations. Researchers should look to the theory being tested and restrict their investigations to hypothesis testing for those variables relevant to specific theories and in specific situations. Testing every possible leader behavior and situational factor, when there is no theoretical reason to expect a significant interaction for all these variables, makes no sense and only

16

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

increases the chance of finding spurious relationships that may occur. This approach may also lead to not finding moderator effects when they are present. The one situation where it may be appropriate to test every possible moderator is when the study is conducted only for exploratory purposes to guide future research. However, as with any data exploration, one must test apparent relationships with a data set other than the one that initially suggested any possible relationships. Regarding the number of variables in a regression equation, Cohen (1990) noted, ‘‘less is more’’—more statistical test validity, more power, and more clarity in the meaning of results (p. 161). The centrality of other predictor variables to the leader behavior–criterion relationship being tested must be carefully considered. Refrain from including additional variables when their relevance to the relationship is minimal. Reporting the percentage of times all moderators were found to be significant is not often meaningful, particularly when testing every possible combination of leader behaviors, situational variables, and interactions. A more useful practice would be to determine the consistency of the findings by reporting the percentage of times that a particular situational variable moderated leader behaviors over a number of samples. Contrary to the pessimistic conclusions about leadership moderators by Podsakoff and MacKenzie (1995), the findings described in this article and in the recent meta-analysis of path–goal theory by Wofford and Liska (1993) show that important leadership moderators do exist, but we have much to learn about their effects.

Appendix A. Forty-five empirical studies of the path–goal and substitutes for leadership theories that report form-type tests for moderators Researchers

Leader behaviors

Potential moderators

Forty theory-based path – goal and substitutes articles Abdel-Halim (1981) 2 3 Arvey and Neel (1974) 1 1 Brief, Aldag, Russell, 1 1 and Rude (1981) Childers, Dubinsky, 2 15 and Skinner (1990) Cummins (1972) 6 2 Dawson, Messe, 2 2 and Phillips (1972) Dobbins and 2 1 Zaccaro (1986) Evans (1970) 2 3 (2 studies) Fry, Kerr, 2 2 and Lee (1986) Graen, Dansereau, 1 1 and Minami (1972) (two samples)

Criterion variables

Total tests

Significant interactions

2 8 5

12 8 5

6 5 1

1

21

5

2 3

20 3

5 1

5

10

5

?

60 60 4

3 8 3

14 14

1 2

2 14

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

17

Appendix A (continued) Researchers

Leader behaviors

Potential moderators

Forty theory-based path – goal and substitutes articles House, Filley, 2 1 and Gujarati (1971) (two samples) Howell and 4 6 Dorfman (1981) Howell and 4 4 Dorfman (1986) Katz (1977) 2 2 Keller (1989) 1 3 (two samples) Kohli (1989) 2 4 Kroll and 1 2 Pringle (1986) Lahat-Mandelbaum 2 3 and Kipnis (1973) Levanoni and 2 7 Knoop (1985) (five samples)

Mathieu (1990) McIntosh (1990) Mitchell, Smyster, and Weed (1975) Mossholder, Niebuhr, and Norris (1990) O’Reilly and Roberts (1978) Palmer (1974) Pitner and Charters (1978/1988) Podsakoff et al. (1986) (two samples) Podsakoff, Todor, and Schuler (1983) Runyon (1973) Schriesheim and DeNisi (1981) (two samples) Schriesheim (1980) Schriesheim and Schriesheim (1980)

Criterion variables

Total tests

Significant interactions

8

16 16

1 4

2

14

0

2

24

4

2 3 2 1

4 8 17 16 2

1 4 4 7 1

1

4

3

2

7 5 5 7 8 2 ? 1

2 1 1

2 3 1

1 2 1

28 28 28 28 14 4 ? 1

2

1

3

6

2

2

4

6

30

7

1 2

1 7

1 1

1 11

0 1

2

8

5

94 94

17 12

2

1

1

2

2

1 1

2 3

1 1

2 3 3

1 2 3

2 2

1 3

3 7

6 42

5 4

(continued on next page)

18

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

Appendix A (continued) Researchers

Leader behaviors

Potential moderators

Forty theory-based path – goal and substitutes articles Schuler (1976) 1 2 Sheridan and 3 2 Vredenburgh (1978) Sheridan and 2 4 Vredenburgh (1979) Sheridan, Vredenburgh, 7 1 and Abelson (1984) Skaret and 2 3 Bruning (1986) Valenzi and 2 1 Dessler (1978) Vecchio (1981) 2 4 Weed, Mitchell, 1 2 and Moffit (1976) Totals for 40 82 119 theory-based studies Percentage of significant interactions Five recent articles by Podsakoff and colleagues Farh et al. (1987) 4 17 Podsakoff et al. (1996) 6 13 Podsakoff, MacKenzie, 7 13 et al. (1993) Podsakoff, Niehoff, 7 13 et al. (1993) Podsakoff 4 24 et al. (1994) 28 80 Totals for 5 recent non-theory-based studies Percentage of significant interactions Total of 110 199 45 studies Percentage of significant interactions

Criterion variables

Total tests

Significant interactions

1 3

2 12

1 1

2

16

1

1

7

2

3

15

3

1

2

1

3 1

15 2

4 1

113

848

179 21.11

6 11 10

408 858 910

30 69 61

8

728

45

6

572

72

41

3476

277

7.97

154

4324

456 10.55

Source: Table 1 in Podsakoff et al. (1995).

References Abdel-Halim, A. A. (1981). Personality and task moderators of subordinate responses to perceived leader behavior. Human Relations, 34, 73 – 83.

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

19

Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21(6), 1141 – 1158. Aguinis, H., Bommer, W. H., & Pierce, C. A. (1996). Improving the estimation of moderating effects by using computer-administered questionnaires. Educational and Psychological Measurement, 56, 1045 – 1049. Aguinis, H., & Stone-Romero, E. F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82(1), 192 – 206. Aiken, L. S., & West, S. G. (1991). Multiple regression: testing and interpreting interactions. Newbury Park, CA: Sage. Aiken, L. S., & West, S. G. (1993). Multiple regression: testing and interpreting interactions. Newbury Park, CA: Sage. Arvey, R. D., & Neel, C. W. (1974). Moderating effects of employee expectancies on the relationship between leadership consideration and job performance of engineers. Journal of Vocational Behavior, 4, 213 – 222. Bobko, P. (1995). Correlation and regression: Principles and applications for industrial/organizational psychology and management. New York: McGraw-Hill. Brief, A. P., Aldag, R. J., Russell, C. J., & Rude, D. E. (1981). Leader behavior in a police organization revisited. Human Relations, 34, 1037 – 1051. Champoux, J. E., & Peters, W. S. (1987). Form, effect size, and power in moderated regression analysis. Journal of Organizational Psychology, 60, 243 – 255. Childers, T. L., Dubinsky, A. J., & Skinner, S. J. (1990). Leadership substitutes as moderators of sales supervisory behavior. Journal of Business Research, 21, 363 – 382. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Cohen, J. (1990, December). Things I have learned (so far). American Psychologist, 1304 – 1312. Cohen, J., & Cohen, P. (1975). Applied multiple regression/correlation analysis for the behavioral sciences. New York: Wiley. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Cummins, R. C. (1972). Leader – member relations as a moderator of the effects of leader behavior and attitudes. Personnel Psychology, 25, 660 – 665. Dawson, J. E., Messe, L. A., & Phillips, T. L. (1972). Effects of instructor – leader behavior on student performance. Journal of Applied Psychology, 56, 369 – 376. DeVries, R. E. (1997). Need for leadership: A solution to empirical problems in situational theories of leadership. Doctoral dissertation, Tilburg University, Tilburg, Netherlands. Dionne, S. D., Yammarino, F. J., Atwater, L. E., & James, L. R. (2002). Neutralizing substitutes for leadership theory: leadership effects and common source bias. Journal of Applied Psychology, 87(3), 454 – 464. Dobbins, G. R., & Zaccaro, S. J. (1986). The effects of group cohesion and leader behavior on subordinate satisfaction. Group & Organization Studies, 11, 203 – 219. Dorfman, P. W., Howell, J. P., Cotton, G. C. G., & Tate, U. (1992). Leadership within the ‘‘discontinuous hierarchy’’ structure of the military: are effective leader behaviors similar within and across command structures? In K. E. Clark, B. M. Clark, & D. P. Campbell (Eds.), Impact of leadership ( pp. 399 – 416). Greensboro, NC: Center for Creative Leadership. Dunlap, W. P., & Kemery, E. R. (1988). Effects of predictor intercorrelations and reliabilities on moderated multiple regression. Organizational Behavior and Human Decision Processes, 341, 248 – 258. Evans, M. G. (1970). The effects of supervisory behavior on the path – goal relationship. Organizational Behavior and Human Performance, 5, 277 – 298. Farh, J. L., Podsakoff, P. M., & Cheng, B. S. (1987). Culture-free leadership effectiveness versus moderators of leadership behavior: an extension and test of Kerr and Jermier’s ‘‘substitutes for leadership’’ model in Taiwan. Journal of International Business Studies, 18, 43 – 60. Friedrich, R. J. (1982). In defense of multiplicative terms in multiple regression equations. American Journal of Political Science, 26, 797 – 833.

20

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

Fry, L. W., Kerr, S., & Lee, C. (1986). Effects of different leader behavior under different levels of task interdependence. Human Relations, 39, 1067 – 1082. Graen, G., Dansereau, F., & Minami, T. (1972). Dysfunctional leadership styles. Organizational Behavior and Human Performance, 7, 216 – 236. House, R. J. (1971). A path goal theory of leader effectiveness. Administrative Science Quarterly, 16, 321 – 338. House, R. J. (1984). Personal communication. House, R. J. (1996). Path – goal theory of leadership: lessons, legacy and a reformulated theory. Leadership Quarterly, 7(3), 323 – 352. House, R. J., & Dessler, G. (1974). The path – goal theory of leadership: some post hoc and a priori tests. In J. Hunt, & L. Larson (Eds.), Contingency approaches to leadership ( pp. 29 – 55). Carbondale: Southern Illinois Press. House, R. J., Filley, A. C., & Gujarati, D. N. (1971). Leadership style, hierarchical influence, and the satisfaction of subordinate role expectations: a test of Likert’s influence proposition. Journal of Applied Psychology, 55, 422 – 432. House, R. J., & Mitchell, T. R. (1974). Path goal theory of leadership. Journal of Contemporary Business, 3, 81 – 97. Howell, J. P., & Costley, D. (2001). Understanding behavior for effective leadership. Upper Saddle River, NJ: Prentice-Hall. Howell, J. P., & Dorfman, P. W. (1981). Substitutes for leadership: test of a construct. Academy of Management Journal, 24(4), 714 – 728. Howell, J. P., & Dorfman, P. W. (1986). Leadership and substitutes for leadership among professional and nonprofessional workers. Journal of Applied Behavioral Science, 22(1), 29 – 46. Howell, J. P., Dorfman, P. W., Hibino, S., Lee, J. K., & Tate, U. (2000). Substitutes for leadership in Western and Asian countries. Working paper, Bureau of Business Research, New Mexico State University. Howell, J. P., Dorfman, P. W., & Kerr, S. (1986). Moderator variables in leadership research. Academy of Management Review, 11, 88 – 102. Hsu, L. M. (1993). Using Cohen’s tables to determine the maximum power attainable in two-sample tests when one sample is limited in size. Journal of Applied Psychology, 78, 303 – 305. Jaccard, J., Turrisi, R., & Wan, C. K. (1990). Interaction effects in multiple regression. Newbury Park, CA: Sage Publications. Jermier, J.M., & Kerr, S. (1997). Substitutes for leadership: their meaning and measurement—contextual recollections and current observations. Leadership Quarterly, 8 (2), 95 – 102. Katz, R. (1977). The influence of group conflict on leadership effectiveness. Organizational Behavior and Human Performance, 20, 265 – 286. Keller, R. T. (1989). A test of the path – goal theory of leadership with need for clarity as a moderator in research and development organizations. Journal of Applied Psychology, 74, 208 – 212. Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research (4th ed.). Fort Worth, TX: Harcourt Brace Jovanovich. Kerr, S., & Jermier, J. M. (1978). Substitutes for leadership: their meaning and measurement. Organizational Behavioral and Human Performance, 22, 375 – 403. Kohli, A. K. (1989). Effects of supervisory behavior. The role of individual differences among salespeople. Journal of Marketing, 53, 40 – 50. Kroll, M. J., & Pringle, C. D. (1986). Path – goal theory and the task design literature: a tenuous linkage. Akron Business and Economic Review, 17, 75 – 84. Lahat-Mandelbaum, B. S., & Kipnis, D. (1973). Leader behavior dimensions related to students’ evaluation of teaching effectiveness. Journal of Applied Psychology, 58, 250 – 253. Larson, L. L., Hunt, J. G., & Osborn, R. N. (1976). The great hi-hi leader behavior myth: a lesson from Occam’s razor. Academy of Management Journal, 19(4), 628 – 639. Levanoni, E., & Knoop, R. (1985). Does task structure moderate the relationship of leaders’ behavior and employees’ satisfaction? Psychological Reports, 57, 611 – 623. Mason, R. L., Gunst, R. F., & Webster, J. T. (1975). Regression analysis and problems of multicollinearity. Communications in Statistics, 4(3), 277 – 292.

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

21

Mathieu, J. E. (1990). A test of subordinates’ achievement and affiliation needs as moderators of leader path – goal relationships. Basic and Applied Social Psychology, 11(2), 179 – 189. McClelland, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114(2), 376 – 390. McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A. Validity results: the relationship between predictor and criterion domains. Personnel Psychology, 43, 335 – 354. McIntosh, N. J. (1990). Leader support and responses to work in U.S. nurses: a test of alternative theoretical perspectives. Work and Stress, 4, 139 – 154. McNeil, K. A., Newman, I., & Kelly, F. J. (1996). Testing research hypotheses with the general linear model. Carbondale, IL: Southern Illinois University Press. McNemar, Q. (1969). Psychological statistics (4th ed.). New York: Wiley. Mendenhall, W., & Sincich, T. (1992). A second course in statistics: Regression analysis (5th ed.). Upper Saddle River, NJ: Prentice-Hall. Mitchell, T., Smyser, C., & Weed, S. (1975). Locus of control: supervision and work satisfaction. Academy of Management Journal, 18, 623 – 631. Montgomery, D. C., & Peck, E. A. (1992). Introduction to linear regression analysis. New York: Wiley. Morris, J. H., Sherman, J. D., & Mansfield, E. R. (1986). Failures to detect moderating effects with ordinary least squares-moderated multiple regression: some reasons and a remedy. Psychological Bulletin, 99, 282 – 288. Mossholder, K. W., Niebuhr, R. E., & Norris, D. R. (1990). Effects of dyadic duration on the relationship between leader behavior perceptions and follower outcomes. Journal of Organizational Behavior, 11, 379 – 388. O’Reilly III, C. A., & Roberts, K. H. (1978). Supervisor influence and subordinate mobility aspirations as moderators of consideration and initiating structure. Journal of Applied Psychology, 63, 96 – 102. Palmer, W. J. (1974). Management effectiveness as a function of personality traits of the manager. Personnel Psychology, 27, 283 – 295. Pitner, N. J., & Charters Jr., W. W. (1978/1988). Principal influence on teacher commitment: substitutes for leadership. Educational Research Quarterly, 12, 25 – 36. Podsakoff, P. M., Dorfman, P., Howell, J. M., & Todor, W. D. (1986). On the nature of the moderators of leader reward and punishment behaviors: a cross cultural analysis. In R. N. Farmer (Ed.), Advances in international comparative management, vol. 1 ( pp. 95 – 138). Greenwich, CT: JAI Press. Podsakoff, P. M., & MacKenzie, S. B. (1995). An examination of substitutes for leadership within a levels of analysis framework. Leadership Quarterly, 6(3), 289 – 328. Podsakoff, P. M., & MacKenzie, S. B. (1997). Kerr and Jermier’s substitutes for leadership model: background, empirical assessment, and suggestions for future research. Leadership Quarterly, 8(2), 117 – 125. Podsakoff, P. M., MacKenzie, S. B., Aherne, M., & Bommer, W. H. (1995). Searching for a needle in a haystack: trying to identify the illusive moderators of leadership behavior. Journal of Management, 21(3), 422 – 470. Podsakoff, P. M., MacKenzie, S. B., & Bommer, W. H. (1996). Transformational leader behaviors and substitutes for leadership as determinants of employee, satisfaction, commitment, trust and organizational citizenship behaviors. Journal of Management, 2, 259 – 298. Podsakoff, P. M., MacKenzie, S. B., & Fetter, R. (1993). Substitutes and the management of professionals. Leadership Quarterly, 4(1), 1 – 44. Podsakoff, P. M., Niehoff, B. P., MacKenzie, S. B., & Williams, M. L. (1993). Do substitutes for leadership really substitute for leadership? An empirical examination of Kerr and Jermier’s situational leadership model. Organizational Behavior and Human Decision Processes, 5, 1 – 44. Podsakoff, P. M., & Skov, R. (1980). Leader reward and punishment behavior scales. Unpublished research, Indiana University, Bloomington, IN. Podsakoff, P. M., Todor, W. D., Grover, R. A., & Huber, V. L. (1994). Situational moderators of leader reward and punishment behavior: fact or fiction? Organizational Behavior and Human Performance, 34, 21 – 63. Podsakoff, P. M., Todor, W. D., & Schuler, R. S. (1983). Leader expertise as a moderator of the effects of instrument and supportive leader behaviors. Journal of Management, 9, 173 – 185.

22

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

Porter, L. W. & Smith, F. J. (1970). The etiology of organizational commitment. Unpublished manuscript, University of California-Irvine. Runyon, K. E. (1973). Some interactions between personality variables and management styles. Journal of Applied Psychology, 57, 288 – 294. Russell, C. J., & Bobko, P. (1992). Moderated regression analysis and Likert scales: too coarse for comfort. Journal of Applied Psychology, 77, 336 – 342. Schriesheim, C. A. (1978). Development validation and application of the new leader behavior and expectancy research instruments. Unpublished doctoral dissertation, Ohio State University, Columbus Ohio. Schriesheim, C. A. (1995). Multivariate and moderated within- and between-entity analysis (WABA) using hierarchical linear multiple regression. Leadership Quarterly, 6, 1 – 18. Schriesheim, C. A., Castro, S. L., & Yammarino, F. J. (2000). Investigating contingencies: an examination of the impact of span of supervision and upper controllingness on leader – member exchange using traditional and multivariate within- and between-entities analysis. Journal of Applied Psychology, 85(5), 659 – 677. Schriesheim, C. A., Cogliser, C. C., & Neider, L. L. (1995). Is it ‘‘trustworthy’’? A multiple levels of analysis reexamination of an Ohio State leadership study, with implications for further research. Leadership Quarterly, 6, 111 – 145. Schriesheim, C. A., & DeNisi, A. S. (1981). Task dimensions as moderators of the effects of instrumental leadership: a two sample replicated test of path – goal leadership theory. Journal of Applied Psychology, 66, 589 – 597. Schriesheim, C. A., Neider, L. L., & Scandura, T. A. (1998). Delegation and leader – member exchange: main effects, moderators and measurement issues. Academy of Management Journal, 41, 298 – 318. Schriesheim, J. F. (1980). The social context of leader subordinate relations on investigation of the effects of group cohesiveness. Journal of Applied Psychology, 65, 183 – 194. Schriesheim, J. F., & Schriesheim, C. A. (1980). A test of the path – goal theory of leadership and some suggested directions for future research. Personnel Psychology, 33, 349 – 370. Schuler, R. S. (1976). Participation with supervisor and subordinate authoritarianism: a path – goal theory reconciliation. Administrative Science Quarterly, 21, 320 – 323. Sheridan, J. E., & Vredenburgh, D. J. (1978). Usefulness of leader behavior and social power variables in predicting job tension, performance, and turnover of nursing employees. Journal of Applied Psychology, 63, 89 – 95. Sheridan, J. E., & Vredenburgh, D. J. (1979). Structural model of leadership influence in a hospital organization. Academy of Management Journal, 22, 6 – 21. Sheridan, J. E., Vredenburgh, D. J., & Abelson, M. A. (1984). Contextual model of leadership influence in hospital units. Academy of Management Journal, 27, 57 – 78. Skaret, D. J., & Bruning, N. S. (1986). Attitudes about the work group: an added moderator of the relationship between leader behavior and job satisfaction. Group & Organization Studies, 11, 254 – 279. Stone, E. F., & Hollenbeck, J. R. (1989). Some issues associated with the use of moderated regression. Organizational Behavior and Human Performance, 34, 195 – 213. Stone-Romero, E. F., Alliger, G. M., & Aguinis, H. (1994). Type II error problems in the use of moderated multiple regression for the detection of moderating effects of dichotomous variables. Journal of Management, 20(1), 167 – 178. Tosi, H. L., & Banning, K. (1998). A need to reconceptualize ‘‘substitutes for leadership’’. In F. Dansereau, & F. J. Yammarino (Eds.), Leadership: The multiple-level approaches ( pp. 271 – 276). Stamford, CT: JAI Press. Tosi, H. L., & Kiker, S. (1997). Commentary on ‘‘substitutes for leadership’’. Leadership Quarterly, 8(2), 109 – 112. Valenzi, E., & Dessler, G. (1978). Relationships of leader behavior, subordinate role ambiguity and subordinate job satisfaction. Academy of Management Journal, 21, 671 – 678. Vecchio, R. P. (1981). Situational and behavioral moderators of subordinate satisfaction with supervision. Human Relations, 34, 947 – 963. Vecchio, R. P. (1990). Theoretical and empirical examination of cognitive resource theory. Journal of Applied Psychology, 75(2), 141 – 147.

J.R. Villa et al. / The Leadership Quarterly 14 (2003) 3–23

23

Weed, S. E., Mitchell, T. R., & Moffitt, W. (1976). Leadership style, subordinate personality and task type as predictors of performance and satisfaction with supervision. Journal of Applied Psychology, 61, 58 – 66. Weiss, D. J., Dawis, R. V., England, G. W., & Lofquist, L. H. (1967). Manual for the Minnesota satisfaction questionnaire. Minneapolis, MN: University of Minnesota. Wofford, J. C., & Liska, L. Z. (1993). Path – goal theories of leadership: a meta-analysis. Journal of Management, 19(4), 857 – 876. Yukl, G. A. (1982). A behavioral approach to needs assessment for managers [Paper]. New York: Academy of Management.