Identifying and assessing tacit knowledge: understanding the practical intelligence of military leaders

The Leadership Quarterly 14 (2003) 117 – 140

Identifying and assessing tacit knowledge: understanding the practical intelligence of military leaders Jennifer Hedlunda,*, George B. Forsytheb, Joseph A. Horvathc, Wendy M. Williamsd, Scott Snooke, Robert J. Sternbergf a

Department of Criminology, Central Connecticut State University, 1615 Stanley Street, P.O. Box 4010, New Britain, CT 06050, USA b United States Military Academy, West Point, NY, USA c Millennium Pharmaceuticals, Cambridge, MA, USA d Cornell University, Ithaca, NY, USA e Harvard Business School, Boston, MA, USA f Yale University, New Haven, CT, USA Accepted 2 January 2003

Abstract Tacit knowledge (TK) is knowledge drawn from everyday experience that helps individuals to solve real-world, practical problems. This study applied a method for identifying and assessing TK to the domain of military leadership in order to understand why some leaders are more successful than others. Interviews were conducted with Army officers at three levels of leadership in order to identify the type of practical, experience-based knowledge that is not necessarily part of formal training or doctrine. Subsequently, the Tacit Knowledge for Military Leaders (TKML) inventory, consisting of a series of leadership scenarios, was developed to assess the amount of knowledge leaders possess. Three versions of the TKML were administered to a total of 562 leaders at the platoon, company, and battalion levels. At all three levels, TKML scores correlated with ratings of leadership effectiveness from either peers or superiors, and the scores explained variance in leadership effectiveness beyond a test of general verbal ability and a test of TK for managers. These results indicate that domain-specific TK can explain individual differences in leadership effectiveness and suggest that leadership development initiatives should include efforts to facilitate the acquisition of TK. D 2003 Elsevier Science Inc. All rights reserved.

* Corresponding author. 1048-9843/03/$ – see front matter D 2003 Elsevier Science Inc. All rights reserved. doi:10.1016/S1048-9843(03)00006-7

118

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

1. Introduction One of the defining characteristics of leadership is the ability to develop and implement appropriate responses to a variety of problem situations (Mumford, Zaccaro, Harding, Jacobs, & Fleishman, 2000). Leaders must solve an array of problems including resource allocation, interdepartmental coordination, interpersonal conflict, and subordinate morale to name a few. In order to effectively solve such problems, leaders must draw on a body of knowledge gained from formal education, advice from other leaders, and personal experience. Some of that knowledge may be in the form of explicit rules (e.g., ‘‘In order to maintain unit morale under adverse conditions, provide employees with extra recognition and rewards’’), which can easily be conveyed through formalized training and education. Some knowledge, however, cannot be reduced to a simple, formalized rule that can be readily conveyed to others. The most effective form of recognition and reward, for example, may depend on the nature of the work, what resources are available, and the personalities of the individuals the leader is trying to motivate. This type of knowledge can be characterized as tacit—it is knowledge, largely acquired through experience, about how to act in specific situations, but that is not readily articulated or widely shared. The term ‘‘tacit knowledge’’ (TK) has roots in works on the philosophy of science (Polanyi, 1966), ecological psychology (Neisser, 1976), and organizational behavior (Scho¨n, 1983). The concept stems from the idea, expressed in everyday phrases like ‘‘professional intuition’’ or ‘‘common sense,’’ that much of the knowledge that is relevant to competent performance is not openly expressed or readily stated. Research on expertise in a variety of domains supports the notion that much of the knowledge associated with successful performance is tacit. Expert knowledge tends to be procedural in nature and to operate outside of focal awareness (see Chi, Glaser, & Farr, 1988). It also reflects the structure of the situation more closely than it does the structure of formal, disciplinary knowledge (Groen & Patel, 1988). Sternberg (1988, 1997) proposed that TK is a factor of practical intelligence. It is a function of an individual’s practical ability to learn from and to solve everyday problems in order to adapt to, to select, and to shape one’s environment in the pursuit of one’s personal goals. Sternberg and his colleagues (Grigorenko, Gil, Jarvin, & Sternberg, 2000; Sternberg, Wagner, & Okagaki, 1993; Sternberg, Wagner, Williams, & Horvath, 1995; Wagner, 1987; Wagner, Sujan, Sujan, Rashotte, & Sternberg, 1999) have found that TK distinguishes among more and less successful performers in several domains, including academic psychology, sales, bank management, general college life, and clerical work. This study represents an extension of the TK research to the domain of military leadership. We sought to identify the TK of military leaders and to develop and validate a tool to assess the amount of TK leaders possess. We first summarize research on practical intelligence and TK, and then discuss the relevance of TK to leadership. Next, we describe the methodology used to identify the TK of military leaders, to develop inventories to assess the acquisition of TK, and to validate the inventories with samples of leaders at three levels of command. Finally, we present the results of our development and validation efforts, and discuss the implications of these findings for other performance domains and leadership development.

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

119

2. TK and practical intelligence In order to understand TK as an aspect of practical intelligence, it is first necessary to review the distinction between practical and general, or academic, intelligence. The recognition that intelligence is more than just ‘‘book smarts’’ dates back to Thorndike (1920) who distinguished between abstract, mechanical, and social forms of intelligence. He argued that the ability to understand others and act appropriately with others was distinct from the ability to understand and solve abstract reasoning problems. More than 50 years later, the concept of practical intelligence was introduced by Neisser (1976) who argued that the types of problems found in school were quite different than the types of problems found in realworld settings. The distinction between academic and practical problems was initially defined by Neisser and later elaborated by Wagner and Sternberg (1985). Academic problems tend to be already formulated by others, well-defined, and disembedded from ordinary experience. They also tend to have complete information, only one correct answer and only one method of obtaining the correct answer. Practical problems, in contrast to academic problems, tend to be poorly defined, unformulated or in need of reformulation, and lacking in information necessary for solution. They also have multiple ‘‘correct’’ solutions, each with liabilities as well as assets, and allow for multiple methods of developing a problem solution. Finally, practical problems tend to be more closely related to everyday experience and of more personal interest than academic problems. TK is associated with practical problem solving and thus reflects many of the defining characteristics of practical problems. 2.1. Defining characteristics of TK First, TK generally is acquired on one’s own with little support from other people or resources (e.g., formal training, written instruction). In other words, the individual must determine for him or herself what information is relevant and how to make sense of it. Second, TK is a form of procedural knowledge, which guides an individual’s action in particular situations or classes of situations (Anderson, 1983). TK, however, represents a subset of procedural knowledge that is not readily articulated in the form of explicit rules and procedures. Third, TK has practical value to the individual. This last feature stems from the first two; knowledge that is experience-based and action-oriented will likely be more instrumental to achieving one’s goals than knowledge that is based on someone else’s experience or does not specify action. For example, a leader may learn through formal training what methods of disciplinary action can be used for what purposes, but he or she may learn from experience what methods are most effective and in what particular situations they should be used (Sternberg et al., 1995, 2000; Sternberg & Horvath, 1999; Wagner & Sternberg, 1985). 2.2. Prior research on TK TK has been measured using a format similar to situational judgment tests (Chan & Schmitt, 1998; Hanson & Ramos, 1996; Motowidlo, Dunnette, & Carter, 1990; Weekley & Jones, 1997). Individuals are presented with a description of a problem relevant to the domain

120

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

of interest (e.g., a manager intervening in a dispute between two subordinates) followed by a set of options (i.e., strategies) for solving the problem (e.g., meet with the two subordinates individually to find out their perspective on the problem; hold a meeting with both subordinates and have them air their grievances). Like most practical problems, there may be multiple ‘‘correct’’ or ‘‘incorrect’’ responses, which may or may not be determined based on a set of explicit procedural rules. The individual’s rating of the appropriateness or quality of these options is used to gage his or her level of TK. The primary focus of research so far has been to establish evidence of the convergent, discriminant, and predictive validity of TK relative to experience, general cognitive ability, and domain-specific performance. Since one of the defining features of TK is its basis in everyday experience, we expect that TK generally will increase as one’s experience in the performance domain increases. Experience has been widely acknowledged as playing an important role in the acquisition of job-relevant knowledge (Borman, Hanson, Oppler, & Pulakos, 1993; Schmidt, Hunter, & Outerbridge, 1986). Consistent with this research, Wagner (1987) found differences in TK scores among business managers, business graduate students, and general undergraduates, with the managers exhibiting the highest scores. In addition, Wagner et al. (1999) found that scores on a TK test for salespeople correlated significantly with number of years of sales experience. A second aspect of TK is its relevance to practical problems. As such, TK inventories are viewed as measures of practical rather than abstract, academic intelligence, and thus should be distinct from conventional intelligence or ability tests. The most widely touted predictor of performance, general cognitive ability ( g) (e.g., Schmidt & Hunter, 1998), has been argued to reflect more closely the skills associated with abstract, academic tasks than those associated with practical tasks (Mayer, 2000; Sternberg, 2000). As such, scores on TK inventories are not expected to correlate highly with scores on tests of g. In studies involving undergraduate students, business executives, and salespersons, scores on TK inventories exhibited nonsignificant correlations with tests of verbal reasoning, with correlations ranging from .04 to .16 (Wagner, 1987; Wagner & Sternberg, 1985, 1990). Wagner and Sternberg (1990) further showed TK inventories to be distinct not only from measures of general intelligence but also from measures of personality and cognitive style. More importantly, they found that TK scores accounted for 32% of the variance in performance beyond scores on a traditional IQ test. Finally, TK is characterized as relevant to one’s personal goals. As such, it should be linked to positive outcomes in one’s job or education. In studies with general business managers, TK scores correlated between .2 and .4 with salary, years of management experience, and whether or not the manager worked for a company at the top of the Fortune 500 list (Wagner, 1987; Wagner & Sternberg, 1985). Wagner and Sternberg (1990) subsequently found a correlation of .61 between scores on a TK test for managers and performance on a managerial simulation for a group of business executives attending a Leadership Development Program at the Center for Creative Leadership. The relationship between TK and performance is not limited to the domain of management. In the field of academic psychology, TK scores correlated .3 to .4 with criterion measures such as citation rate, number of publications, and quality of department (Wagner, 1987; Wagner & Sternberg, 1985). Wagner et al. (1999) also found that the TK scores of salespeople correlated

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

121

.3 to .4 with criteria such as sales volume and number of sales awards received. Finally, Grigorenko et al. (2000) found that a test of common sense knowledge for the workplace (e.g., how to handle oneself in a job interview) predicted supervisor performance ratings in a sample of 230 workers ranging from custodians to teachers. Thus, there is fairly consistent evidence that TK correlates with successful performance in a variety of domains.

3. TK and leadership Leadership can be characterized as a complex performance domain requiring the ability to solve a variety of interpersonal and organizational problems (Mumford et al., 2000; Zaccaro, Mumford, Connelly, Marks, & Gilbert, 2000). Researchers traditionally have sought to understand this ability either in terms of what traits leaders possess (e.g., intelligence, need for achievement) or in terms of how leaders behave (e.g., showing consideration for others, clarifying roles and objectives). Many of these approaches have failed to consistently account for individual differences in leadership effectiveness. In response to these inconsistencies, Fiedler (1995) commented ‘‘it is very difficult to believe that intellectual abilities fail to contribute to such critical leadership functions as decision-making and coordinating and organizing work processes, or that leaders cannot learn from past events’’ (p. 6). Fiedler, along with other researchers, attributed these inconsistencies to more complex relationships among the variables. His cognitive-resources theory (Fiedler, 1995; Fiedler & Garcia, 1987), for example, suggests that situational conditions (e.g., stress) moderate the relationships between cognitive resources (e.g., intelligence, experience) and leader performance. An alternative explanation for these inconsistencies is that they represent limited conceptualizations of intelligence and experience as they relate to effective leadership. Effective leadership can be viewed as a form of developed expertise—it requires having a broad response repertoire from which to draw as well as the ability to use that knowledge to solve everyday problems. Other researchers have suggested that effective leaders possess well-organized, domain-specific knowledge structures that allow them to respond flexibly to a range of situations (Zaccaro, Gilbert, Thor, & Mumford, 1991). Thus, knowledge, as a manifestation of intelligence and experience, may serve as a more relevant factor in understanding leadership effectiveness. Unfortunately, there has been limited attention to what leaders know about how to lead, how they use that knowledge, and how they develop that knowledge (Bass, 1990; Hollander, 1985). Our aim was to identify what ‘‘lessons,’’ or TK, military leaders gain from their experiences and to determine the extent to which this knowledge contributes to effective leadership.

4. Methodology We organize the methodology around three main stages of our research. In the first stage, we collected examples of TK from leaders in order to understand the content and structure of

122

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

that knowledge, and to facilitate subsequent instrument development. In the second stage, we selected promising items of TK from the initial pool and developed a set of inventories to assess the acquisition of TK. In the third and final stage, we submitted the inventories to new samples of leaders for preliminary validation. 4.1. Identification of the TK of military leaders In order to identify the TK of military leaders, we interviewed 81 Army officers selected by their senior commanders to participate in the study. The officers represented each of three organizational leadership levels (30 platoon leaders, 32 company commanders, and 19 battalion commanders). The interviewers, following a protocol, asked participants to relate a story about a job-related experience from which they had learned something about leadership at their current organizational level. Members of the research team reviewed written summaries of each interview to evaluate if the knowledge was consistent with our definition of TK. The knowledge was designated as tacit if it (a) was based in a personal experience, (b) was not well supported by formal training or doctrine, and (c) expressed some form of action. Additionally, we only were interested in knowledge that pertained to interpersonal rather than technical aspects of leadership. Next, all the stories were rewritten into a standard format to facilitate subsequent use. Each item of TK was represented as a production system that consisted of a mapping between a set of antecedent conditions (‘‘If’’ statements), a set of consequent actions (‘‘Then’’ statements), and a brief explanation (‘‘Because’’ statement). Other logical operators (‘‘And,’’ ‘‘Or,’’ and ‘‘Else’’) were used to indicate conjunctive relationships, disjunctive relationships, and complex procedures, respectively. The following example illustrates this representation: If your company commanders have a strong desire to be successful and earn top block ratings, and if they also have a tendency to take on resource-intensive missions that exceed their capabilities, then require commanders to conduct resource assessments before they take on missions, because an accurate resource assessment should indicate whether or not the unit has the resources to handle the mission. We do not consider these procedural formalisms to be the TK per se, but rather ‘‘markers’’ for more complex, predominantly implicit mental representations. The coded items were reviewed by three senior military members of the research team and revised as necessary to ensure that they accurately reflected the meaning of the original stories. This process resulted in 173 unique items of TK, 42 at the platoon level, 64 at the company level, and 67 at the battalion level. In order to understand the content and structure of TK, a preliminary framework was developed for classifying the items. This framework was based on a general structure of TK as developed in previous research (Wagner, 1987) along with a review of various domainrelevant publications (e.g., field manuals, trade journals, doctrine, military memoirs). The general structure consisted of three categories: intrapersonal, interpersonal, and organizational

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

123

TK. Intrapersonal TK includes knowledge of how to manage tasks and manage one’s personal development. Interpersonal knowledge includes knowledge about how to influence and motivate subordinates, cooperate with others, and communicate effectively. Organizational TK includes knowledge of how to manage organizational change and protect the integrity of the organization. Using this framework as a guideline, military members of the research team were asked to sort the items of TK at each level into categories of their choosing. The categories were used to create a set of dissimilarity matrixes, which were subjected to cluster analyses. The resulting clusters were labeled by military members of the team based on the content of the items included in those clusters and are presented and summarized in the Results section. 4.2. Development of the TK inventories for military leaders We developed three separate inventories representing the unique content of the TK items that emerged at each leadership level. TK inventories consist of a series of brief problem descriptions accompanied by a set of 10 to 15 response options. Individuals are asked to rate the quality or appropriateness of each option. Due to the length of each question, we sought to limit each inventory to 15 and 20 questions each. Therefore, we conducted a series of analyses in order to identify items, from the original pool of 173, that would best discriminate between more and less experienced and effective officers at each level. The TK items were compiled into three separate surveys, one for each leadership level. The surveys were administered to two samples of military leaders, who were asked to rate the quality of each TK item. The first sample consisted of 791 Army officers who completed the surveys either as novices for the next level of command or as experienced leaders for the one they had just completed. Discriminant analyses were used to determine how well the items as a whole distinguished between experts and novices, and to identify individual items that best discriminated between these two groups. On the whole, there were differences in how experts and novices rated the items as indicated by significant canonical discriminant function at each level (battalion: R=.73, p < .01; company: R=.72, p < .01; platoon: R=.55, p < .01). For a second sample of 447 Army officers, we received, in addition to ratings of the TK items, ratings of each leader’s effectiveness from at least three sources (self, superior, and peer or subordinate). In this sample, correlational analyses were used to identify TK items that distinguished between more and less effective leaders. Based on the above analyses, along with the judgment of military experts, we selected TK items that were considered to best represent the construct of TK for military leaders. Each selected TK item was expanded into a scenario format that presented a brief description of a leadership problem along with a set of 5 to 15 possible options for handling the problem. These options were generated from the original interview summaries and input from military experts. A scenario plus response options represents a complete ‘‘question’’ in the TK inventory, with each inventory containing multiple questions. A sample question is shown in Fig. 1. The revised inventories, collectively referred to as the Tacit Knowledge for Military Leaders (TKML) inventories, were administered to new samples of officers for preliminary validation.

124

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

Fig. 1. Sample question from the TKML.

4.3. Preliminary validation of the TKML inventories We conducted a preliminary validation study on the three TKML inventories in order to evaluate the convergent validity of the TKML relative to leadership experience, discriminant validity in relation to measures of general verbal ability and TK for managers, and predictive validity relative to leadership effectiveness. We expected that leaders with more experience would possess more TK, and that leaders who possess greater TK would be more effective than those who possess less TK. We further hypothesized that TK scores would be unrelated to scores on a TK test for managers and a test of general verbal ability. Finally, we expected that TK scores would predict leadership effectiveness beyond scores on a TK test for managers and a test of general verbal ability. 4.3.1. Sample The sample was drawn from 44 battalions stationed at six posts around the United States. Battalion units were selected for participation by division, corp, or brigade staff. At an appointed time, the entire available officer chain-of-command for each battalion (approximately 25–30 officers) met at a central location, usually in their battalion conference room. We obtained complete data from 368 platoon leaders, 163 company commanders, and 31

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

125

battalion commanders. In addition, we obtained ratings of leadership effectiveness from the superior officers of the 31 battalion commanders (i.e., brigade commanders) who were not study participants. 4.3.2. Measures Leaders at each level completed their respective TKML inventory. The TKML inventories consisted of 16 questions at the platoon level, 20 questions at the company level, and 16 questions at the battalion level. The questions covered issues ranging from how to deal with soldier insubordination to how to ensure one’s vision is communicated accurately down the chain of command. Leaders also were asked to report the number of months they had served in their current position in order to assess the relationship between experience and TK. In addition to the TKML, leaders completed the Tacit Knowledge Inventory for Managers (TKIM; Wagner & Sternberg, 1991), which was designed to measure the experience-based knowledge of civilian managers. The TKIM has been validated in earlier research and has been found to be a significant predictor of managerial success (Sternberg et al., 1993; Wagner, 1987). We included a measure of TK for managers to show that despite any potential overlap in the knowledge of civilian managers and military leaders, the TKML represents domain-specific knowledge that is more relevant to understanding effective leadership than the TKIM. Next, a measure of verbal ability, the Concept Mastery Test (CMT; Terman, 1950), was included in order to establish the extent to which (a) the TKML measures more than verbal reasoning ability, and (b) the TKML can add to the prediction of leadership effectiveness beyond a traditional measure of general cognitive ability. The CMT, as administered here, consisted of two sections, synonym/antonym problems and verbal analogy problems. In order to assess the relationship of TK to leadership performance, we administered the Leadership Effectiveness Survey (LES), which consisted of single-item measures that solicited ratings of each leader’s interpersonal, task-oriented, and overall effectiveness using a seven-point scale. An example question from the LES is shown in Fig. 2. We attempted to obtain ratings from multiple sources since those sources can represent significant and meaningful sources of variation in perceptions of performance (e.g., Salam, Cox, & Sims, 1997; Tornow, 1993). We asked all officers in the battalion (immediate superiors, peers in the unit, and subordinate officers) to provide ratings. We were unable to obtain peer ratings of battalion commanders due to their limited contact with their cohorts. We also did not obtain subordinate ratings of platoon leaders due to the unavailability of noncommissioned officers to participate in the study. When feasible, we obtained multiple ratings of effectiveness from each source. For the purpose of analyses, we looked at the average performance rating a leader received on all three dimensions (overall, task, and interpersonal) from each rating source (e.g., subordinates, peers, supervisor). 4.3.3. Scoring The TKML inventories were scored by computing a profile match, or difference score, between participants’ ratings and an expert profile. Select groups of officers at each level

126

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

Fig. 2. Sample questions from LES.

served as the expert samples. Fifty-nine students attending the Army War College completed the TKML inventory for battalion commanders. Twenty-nine majors and lieutenant colonels attending the Pre-Command Course served as the expert sample for the TKML for company commander. Finally, 50 captains attending the Command and General Staff College provided responses to the TKML for platoon leaders. At each level, the average interrater reliability among experts was greater than .95. From their response data, we constructed expert profiles consisting of the mean and standard deviation of the experts’ ratings for each response option within a question. We scored responses to the TKML by computing a squared Euclidean distance of the incumbent’s responses from the expert means. However, before summing these distances, we performed a correction to account for the variability in expert ratings. That is, we did not want to penalize respondents for ratings that were far from the experts when the experts themselves varied in their ratings. Therefore, we divided the squared distances on each response option by the standard deviation among experts for that option. The TKIM was scored using a similar procedure as that for the TKML. The expert profile, however, consisted of the mean responses of 13 business executives from Fortune 500 firms (see Wagner, 1987). As with the TKML, the total score on the TKIM reflects the squared Euclidean distance between respondents’ ratings and the expert profile. Finally, the CMT was

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

127

scored using an answer key and the number of correct responses for the synonym/antonym and analogy problems was combined to create a total CMT score.

5. Results The purpose of our research was to identify the TK of military leaders at three levels of command and to develop and validate TK inventories for leaders at those levels. We first present results of our initial efforts to identify the content and structure of TK. Then we present results of our efforts to validate the inventories. 5.1. The content and structure of TK for military leaders Table 1 presents the categories of TK for military leaders that emerged from the sorting task and cluster analyses on the original set of 173 TK items. The values in each column represent the percentage of items that emerged in the category for the particular leadership level. A blank in the table indicates that no items emerged in the category at that level. We discuss below the meaning of these categories in reference to the role of leaders at each level of command. 5.1.1. Platoon leaders At the platoon level, the TK of leaders reflects their limited experience in Army leadership (typically 1–3 years), yet responsibility for directly supervising soldiers (approximately 25– 45 in number) who have relatively greater time in service. Key challenges for platoon leaders include motivating subordinates, establishing credibility, and managing oneself. Motivating subordinates represented the largest category of TK for platoon leaders (28%), and pertained to issues such as how to maintain consistency in subordinates’ lives, meet their basic needs, prevent boredom, build their confidence, recognize their limits, and provide them with support. The second largest category of TK for platoon leaders pertained to managing oneself (19%), and included how to manage stress, monitor performance, and seek feedback from others. Items pertaining to managing oneself also were more frequent at the platoon than company or battalion levels, indicating that such knowledge may be particularly important for leaders who have limited experience in the organization. Finally, unique to platoon leaders was TK about establishing credibility, which is consistent with their role of supervising subordinates with relatively more tenure. 5.1.2. Company commanders At the company level, leaders have more experience than at the platoon level and more position power, such as determining how missions will be accomplished. They lead larger organizations, typically 120 to 200 soldiers, and as a result have less direct contact with their subordinates. The TK of company commanders is distributed across several categories. Unique to company commanders is knowledge pertaining to directing and supervising subordinates, balancing mission and troops, and cooperating with others. Because company

128

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

Table 1 Categories of TK with proportion of items obtained by level Category

Communicating

Taking care of soldiers

Managing oneself Motivating subordinates

Establishing trust

Developing subordinates

Influencing the boss Protecting the organization Dealing with poor performers Managing organizational change Directing and supervising subordinates Balancing mission and troops Cooperating with others Establishing credibility

Description

Level Battalion (n = 67)

Company (n = 64)

Platoon (n = 42)

Imparting values/vision; communicating expectations; seeking information; correcting misperceptions Providing support; managing work loads; dealing with personal problems Managing stress and emotions; setting goals and seeking feedback Providing rewards/recognition; engaging subordinates in decision making; meeting subordinates basic needs Protecting solders; keeping soldiers informed; giving subordinates responsibility; being open and honest Providing opportunities to gain experience; counseling; identifying strengths and weaknesses Confronting superiors; taking initiative; seeking autonomy Exhibiting loyalty; encouraging trust; protecting subordinates from unreasonable demands Solving problems of insubordination and poor performance Using subordinates as change agents

15%

13%

13%

14%

12%

7%

11%

9%

19%

9%

14%

28%

7%

8%

7%

18%

6%

–

–

8%

14%

14%

–

–

7%

–

–

4%

–

–

Coordinating and organizing units; encouraging initiative; holding subordinates accountable Resolving conflicts between orders from above and subordinate needs Networking; developing cooperation and trust among peers Recognizing limitations; improving one’s knowledge and skills

–

16%

–

–

8%

–

–

6%

–

–

–

12%

commanders are responsible for managing a larger organization with less direct contact with their subordinates, they need to know how to coordinate, encourage cooperation, organize units, manage training assignments, encourage subordinates to take initiative, and hold subordinates accountable. Their knowledge also pertains to how to resolve conflicts between orders from above and the welfare of their soldiers, as well as how to network and

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

129

develop cooperation and trust among their peers. There is less emphasis on managing oneself and motivating subordinates relative to platoon leaders, but more so than battalion commanders. 5.1.3. Battalion commanders At the battalion level, leaders have considerable experience in the Army, having served 16 to 20 years as an officer and been through a highly competitive process to achieve their position. They command organizations of typically 500 to 700 soldiers, making it difficult to interact with subordinates face-to-face, but they have considerable power and discretion in discharging the legal authority of command. The unique TK of battalion commanders reflects a broader, system-level perspective and their greater discretion. It includes knowledge about protecting the organization, managing organizational change, and dealing with poor performers. For example, battalion commanders need to know how to exhibit loyalty, encourage trust, and protect subordinates from unreasonable external demands; how to use subordinates as change agents; and how to solve problems and decide when to relieve an officer of his or her duty. Battalion commanders exhibit more TK about developing subordinates, taking care of soldiers, and communicating than platoon leaders and company commanders, and the content of that knowledge is somewhat unique. At the battalion level, knowledge about developing subordinates includes knowing how to allow subordinates to solve their own problems, to provide them with opportunities to gain experience, to counsel them on mistakes, and to help them identify strengths and weaknesses. Taking care of soldiers at the battalion levels pertains to providing support, managing soldiers work load, making living quarters comfortable, and showing concern for their well-being. Communicating at the battalion level involves indirect methods and more systems-level emphasis, and includes TK about how to impart values, communicate one’s vision, and communicate expectations. The TK of battalion commanders also pertains to establishing trust, managing oneself, and motivating subordinates, but to a lesser extent than that of platoon leaders and company commanders. 5.2. Preliminary validation of the TKMLs Each TKML inventory consisted of a unique set of questions that reflected the knowledge relevant to the particular level of command. Therefore, separate validation analyses were conducted at each level. However, we present the results according to key relationships rather than by leadership level, and where appropriate, we highlight inconsistencies in relationships across levels. We first examined the reliability of the three TKML inventories and then performed correlational analyses to examine the discriminant, convergent, and predictive validity. Further, we followed up the correlational analyses with hierarchical regression analyses to assess the incremental validity of TK for military leaders relative to verbal ability and managerial TK in predicting leadership effectiveness. The means, standard deviations, and intercorrelations among all the variables are presented in Tables 2–4 for platoon leaders, company commanders, and battalion commanders, respectively.

130

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

Table 2 Means, standard deviations, and intercorrelations for platoon leaders 1. 2. 3. 4. 5. 6.

TKML TKIM Verbal ability Months in position Supervisor rating Peer rating

n

Min

Max

M

SD

353 348 343 344 430 435

0 0 17 0 1.00 2.00

376 340 131 43 7.00 7.00

300 239 79 7 4.96 4.72

52 57 25 6 1.10 .89

1

2 – .36** .08 .02 .17** .05

3

– .09 .02 .01 .06

– .02 .08 .04

4

5

6

– .06 .02

– .38**

–

Test scores and effectiveness ratings have been reverse coded as necessary so that higher values reflect better performance. ** p < .01.

5.2.1. Reliability of the TKML inventories TK inventories are different from traditional knowledge tests in that the item stems consist of poorly defined, complex problem situations (Legree, 1995). Most situational judgment problems are considered multidimensional in nature, drawing on a combination of knowledge, skills, and abilities (Chan & Schmitt, 1998). Across the inventory, the questions may measure diverse areas of knowledge, some of which may be acquired by the individual, some of which may not. The complexities of the TK inventory reduce the likelihood of obtaining the same levels of internal consistency as found for traditional knowledge and ability tests. However, it is important to determine the reliability of these particular measures relative to other situational judgment tests. Initially, we examined item–total correlations for each TKML to identify questions that exhibited poor ‘‘fit’’ with the rest of the inventory. We more closely examined questions that had item–total correlations less than .15, and removed one question at the platoon level, two questions at the company level, and three questions at the battalion level that were determined Table 3 Means, standard deviations, and intercorrelations for company commanders 1. 2. 3. 4.

TKML TKIM Verbal ability Months in position 5. Supervisor rating 6. Peer rating 7. Subordinate rating

n

Min Max

M

SD

163 159 156 156

0 0 9 0

861 216 85 9

132 52 27 6

1027 306 142 24

1

2

3

4

– .32** .18* .09

– .16* .02

– .03

5

6

7

–

116 1.00

7.00

5.02

1.15

.05

.13

.04

.04

–

157 3.00 140 1.67

6.67 7.00

4.80 4.80

.70 1.21

.18* .04

.05 .11

.15 .22*

.05 .33** – .09 .29** .35**

–

Test scores and effectiveness ratings have been reverse coded as necessary so that higher values reflect better performance. * p < .05. * * p < .01.

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

131

Table 4 Means, standard deviations, and intercorrelations for battalion commanders 1. 2. 3. 4. 5. 6.

TKML TKIM Verbal ability Months in position Supervisor rating Subordinate rating

n

Min

Max

M

SD

31 31 30 22 24 31

0 0 26 4 4.00 2.33

241 204 134 23 6.67 6.78

144 130 74 15 5.60 5.10

64 43 29 6 .67 .99

1

2 – .11 .02 .22 .46* .11

3 – .22 .02 .02 .29

4

– .40 .10 .23

5

– .21 .19

6

– .15

–

Test scores and effectiveness ratings have been reverse coded as necessary so that higher values reflect better performance. * p < .05.

to represent ‘‘common knowledge’’ (and thus not TK) or to be too narrow in focus (e.g., pertaining to a particular specialty like chemical weapons). We then computed coefficient alpha reliabilities on each inventory. The TKML for platoon leaders consisted of 15 questions (a=.69), the TKML for company commanders consisted of 18 questions (a=.76), and the TKML for battalion commanders consisted of 13 questions (a=.66). Although these reliabilities are below the desired levels for traditional knowledge and ability tests (>.8), they are consistent with those obtained for other situational judgment tests (.5 to .8; Legree, 1995). Given that our obtained reliabilities fall within this range, we consider the internal consistency of our instruments to be acceptable and chose to use the overall score on the TKML in the remaining analyses. 5.2.2. Relationships among predictors We first conducted correlational analyses to assess the extent of discriminant and convergent validity between the TKML and other predictors. TK for military leaders did not correlate significantly with verbal ability at the platoon or battalion levels, but the two scores were significantly correlated at the company level (r=.18, p < .05). Company commanders with higher CMT scores were more likely to obtain higher scores on the TKML. There were no significant relationships between experience, as indicated by months in the current position, and TKML scores. Interestingly, the direction of the correlation was positive only at the battalion level (r=.22, ns). Finally, TK for military leaders correlated significantly with managerial TK at the platoon (r=.36, p < .01) and company (r=.32, p < .01) levels. Platoon leaders and company commanders who scored higher on the TKML were more likely to score higher on the TKIM. At the battalion level, the correlation between TKML and TKIM scores was not significant (r=.11, ns). 5.2.3. Predictors of leadership effectiveness Scores on the TKML correlated with ratings of leadership effectiveness at all three levels. The types of relationships, however, varied across levels. Platoon leaders who scored higher on the TKML were rated as more effective by their supervisors (r=.17, p < .01). There was no significant relationship between TKML scores and peer ratings (r=.05, ns). At the company

132

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

level, a different relationship emerged. Company commanders who scored higher on the TKML were rated as more effective by their peers (r=.18, p < .05), but there was no significant relationship with supervisor ratings (r=.05, ns) or with subordinate ratings (r=.04, ns). Finally, at the battalion level, higher TKML scores were associated with higher ratings of effectiveness by supervisors (r = .46, p < .05). As with company commanders, there was no significant relationship between TKML scores and subordinate ratings (r = .11, ns). In regards to the other predictors, there were no significant correlations between managerial TK and effectiveness ratings at any level. There also were no significant relationships between experience and effectiveness ratings. Additionally, verbal ability generally failed to correlate with effectiveness ratings, with one exception. Company commanders who scored higher on the CMT actually received lower effectiveness ratings from their subordinates (r = .22, p < .05). 5.2.4. Incremental prediction of leadership effectiveness Although verbal ability and managerial TK generally did not correlate with effectiveness ratings, we conducted follow-up hierarchical regression analyses in order to assess the extent to which TK for military leaders accounts for unique variance in effectiveness ratings that is not accounted for by verbal ability or managerial TK. In each hierarchical regression, we entered scores on the CMT and the TKIM in the first step, followed by scores on the TKML in the second. Table 5 presents the results of the hierarchical regression analysis for all three levels of command.

Table 5 Incremental validity of TKML on effectiveness ratings Step

Subordinate ratings R

Platoon leaders 1. Verbal ability TKIM 2. TKML

NA

Company commanders 1. Verbal ability TKIM 2. TKML

n = 132

Battalion commanders 1. Verbal ability TKIM 2. TKML

n = 29

R2

Peer ratings

B

R

.34 .37

Supervisor ratings B

R

n = 275 .06 .07

.21* .25*

R2

.05* .02

.12 .03

.18 .27 .16

.16 .27* NA

B

n = 276 .00 .00

.04 .03 .04

.02 .05**

.18* .12 .24**

n = 149 .20* .14 .14

R2

.09 .19*

.01 .03**

.08 .08 .18**

.01 .02

.02 .17 .14

.01 .21*

.08 .06 .46*

n = 109 .12 .17 n = 22 .11 .47

Test scores and effectiveness ratings have been reverse coded as necessary so that higher values reflect better performance. * p < .05. * * p < .01.

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

133

In each case where the zero-order correlation was significant, TKML scores provided a significant increment in prediction in the second step of the regression, with the multiple R for the overall models ranging from .19 at the platoon level to .47 at the battalion level. At the platoon level, TKML scores accounted for 3% variance in supervisor ratings of effectiveness beyond CMT and TKIM scores. At the company level, TKML scores accounted for 5% incremental variance in peer ratings of effectiveness. And at the battalion level, TKML scores accounted for 21% of the variance beyond CMT and TKIM scores, although the significance of this effect is restricted by the small sample size involved (n = 22).

6. General discussion The purpose of this research was to identify the TK of military leaders at three leadership levels, to develop an instrument to measure the extent to which leaders possess such knowledge, and to assess the validity of this measure as a predictor of leadership effectiveness. We elicited numerous examples of TK from leaders in the form of stories and pieces of advice about leadership, and from those stories we identified a set of items that seemed to best represent the TK of experienced and effective leaders. These items were compiled into inventories, which were then administered to new samples of leaders at three organizational levels (platoon, company, and battalion) along with measures of general verbal ability, TK for civilian managers, and leadership effectiveness. 6.1. Relationship of TK to existing leadership frameworks Many of the categories of TK that emerged from our initial analyses overlap or encompass the types of skills or behaviors characterized by previous frameworks of leadership and management. For example, communicating, motivating, and developing subordinates; providing support; organizing units; and setting goals are represented in Yukl’s (1998) 14 managerial practices and Mintzberg’s (1980) 10 manager roles. Interestingly, the characteristics of transformational leaders (e.g., showing individualized consideration, inspiring subordinates to act, establishing a clear vision), as identified by Bass and Avolio (1993), are more closely aligned with the unique TK of battalion commanders, which includes communicating a vision, helping subordinates identify strengths and weaknesses, using subordinates as change agents, and showing concern for soldiers’ well-being. One of the categories of TK that was relevant to all three levels of command was ‘‘managing oneself.’’ This category includes behaviors that are consistent with theories of self-management and self-leadership (Manz & Sims, 1980; Markham & Markham, 1995), such as setting goals and taking initiative to accomplish them, regulating one’s thoughts and emotions, and providing self-reinforcement. Although there is a fair amount of overlap between our categories of TK and existing taxonomies of leadership found in the literature, the categories ‘‘establishing credibility’’ and ‘‘establishing trust’’ appear to be unique to military leaders. Issues of credibility are addressed by leadership approaches that focus on how leaders gain influence over their followers; in

134

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

other words, how they acquire power. For example, increasing one’s knowledge and skills is a source of expert power, while demonstrating respect is a source of referent power (French & Raven, 1959). Trust also is recognized as important aspects of leadership (e.g., Bass & Avolio, 1993), but the issues of establishing trust and establishing credibility have not emerged as major categories in existing taxonomies. The emergence of these two areas (establishing trust and establishing credibility) as prominent categories in our research may reflect some of the unique challenges faced by military leaders as compared to civilian leaders. 6.2. Relationship of TK to effective military leadership At all three leadership levels, we found some significant relationships between scores on the TKML inventories and ratings of leadership effectiveness. At the platoon and battalion levels, TKML scores correlated significantly with supervisor ratings of effectiveness. The finding that leaders with higher TKML scores were rated as more effective by their superiors may be explained, in part, by the method used to score the TKML. Responses to the TKML were scored in terms of their proximity to a group of expert raters, who were designated as experts primarily on the basis of ratings they received from their own superiors. In other words, the view of what is effective leadership as well as what is appropriate knowledge may be largely driven by perceptions from the next level in the chain of command. This finding raises concerns that our measures may be capturing ingratiation rather than some aspect of effective performance. Ingratiation, which characterizes behaviors designed to favorably influence another person’s perception of oneself, may be part of what it takes to be successful (Jones, 1963). Ultimately, it is the superior who the leader must account to and who makes decisions regarding the future of his or her military career. This issue may also serve to explain why TKML scores did not correlate with subordinate ratings. In other words, a response that may be viewed as effective by one’s superior may be viewed as ineffective or undesirable from a subordinate’s perspective. Future research is needed to further understand the relationships among TK, ingratiation, and performance. At the company level, TKML scores were not related to supervisor ratings but rather to peer ratings. This finding may reflect the unique challenges associated with leadership at the company level. The company commander is caught in the middle and must learn how to motivate and develop subordinates, cooperate with peers, and simultaneously perform as part of a larger complex organization (a battalion). Because of this unique situation, peers may be most in tune with whether or not their fellow company commanders are performing effectively. In addition to establishing a relationship between TKML and leadership effectiveness, we sought to determine the contribution of the TKML relative to other potential predictors of performance. We found that neither verbal ability nor managerial TK correlated significantly with ratings of leadership effectiveness. In other words, leaders who scored well on the CMT or the TKIM were not necessarily rated higher than those who scored poorly on these measures. The one exception was a significant negative relationship between verbal ability

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

135

and subordinate ratings of leadership effectiveness at the company level. One possible interpretation of this finding is that leaders who possess ‘‘book smarts’’ are not necessarily perceived favorably by their subordinates. Although verbal ability and managerial TK were not predictive of leadership effectiveness, these measures did exhibit some significant relationships with the TKML. Scores on the CMT exhibited a moderate, significant correlation with TKML scores at the company level. This finding deviates somewhat from previous studies that have found small, nonsignificant correlations between TK scores and scores on verbal ability tests (Wagner, 1987; Wagner & Sternberg, 1985, 1990). It is important to note that the magnitude of the correlation is still modest (.18) relative to the correlations traditionally found between tests of general cognitive ability ( g) and job knowledge (.31 to .46; Borman et al., 1993; Schmidt et al., 1986). It is equally important to note that the size of these correlations on the whole suggests little overlap in the abilities measured by the TKML and the CMT. Scores on the TKIM also correlated somewhat with TKML scores at the platoon and company levels (.32 and .36, respectively). The correlation between TKML and TKIM scores suggests some overlap, to be expected, in the knowledge of managers and leaders. Military leaders engage in similar activities to civilian managers (e.g., resource allocation, subordinate development) that allow them to recognize and respond to comparable situations presented in the context of civilian management. This correlation also may reflect an underlying ability (e.g., practical intelligence) that allows individuals to respond appropriately to situations in a variety of performance domains. The domain specificity of TK, however, is supported by the modest size of these correlations, as well as the results of our hierarchical regression analyses. TK for military leaders accounted for a significant increment in variance above and beyond both verbal ability and managerial TK. Olson-Buchanan et al. (1998) similarly found that a situational assessment of interpersonal skills accounted for a significant increment in prediction of job performance ratings over verbal and quantitative test scores. A finding contrary to our expectations was that experience failed to correlate significantly with either TKML scores or leadership effectiveness. Although prior research generally has found a positive relationship between experience and TK (Wagner, 1987; Wagner et al., 1999), other researchers suggest that individuals with similar tenure or similar job tasks can differ in the amount of knowledge gained (Ford, Quinones, Sego, & Speer-Sorra, 1992; McCall, Lombardo, & Morrison, 1988; Quinones, Ford, & Teachout, 1995). Furthermore, time-based measures of experience have been found to correlate poorly and even negatively with leadership performance (e.g., Bettin & Kennedy, 1990; Fiedler, 1992). Thus, while time is considered important to the development of knowledge and skill, it is not necessarily a good indicator of the amount of development that has occurred. In the military, this relationship may be further complicated by the rapid progression of leaders through the organization’s hierarchy, particularly at the levels we studied. Therefore, our failure to find a correlation between TKML scores and experience likely is attributable to a methodological limitation rather than the lack of a conceptual relationship. An alternative approach to assessing this relationship, which should be addressed in subsequent research, is to compare how leaders at different levels in the chain-of-command perform on the same inventory.

136

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

6.3. TK as an indicator of success In general, the TK approach represents a method of assessing what individuals, both leaders and nonleaders, have learned beyond formal training that contributes to their success. Rather than assuming that a certain amount of time on a job or holding certain positions will produce the requisite knowledge and skills to succeed, we can measure more directly what individuals have learned from their experiences. Some researchers have suggested using developmental challenges to measure the quality of one’s experience (McCauley, Ruderman, Ohlott, & Morrow, 1994; Tesluk & Jacobs, 1998). The Developmental Challenge Profile (DCP), for example, measures aspects of the job (e.g., unfamiliar responsibilities, difficult boss) that are considered important to on-the-job learning (McCauley et al., 1994). The TK approach represents an alternative that focuses directly on the knowledge that emerges from those challenges. The TK approach also may provide particular value in domains that rely heavily on interpersonal and problem-solving skills. It is more difficult to articulate in a set of formalized rules the proper way to negotiate a conflict than the appropriate procedures for cleaning a weapon. When knowledge is not openly conveyed and must be acquired on one’s own, the likelihood increases that some individuals will fail to effectively acquire the relevant knowledge. The learner has greater responsibility for deciding what information to attend to and how to interpret that information. It is reasonable to expect that, in such situations, individual differences in performance will be attributable more to differences in TK than differences in formal, explicit knowledge. Thus, measures of TK should be particularly useful in explaining individual differences in performance in domains where much of the learning occurs on the job. 6.4. TK and leader development Beyond identifying which leaders are likely to be more or less effective, the TK approach can be applied toward improving the process of leader development. It is widely acknowledged that on-the-job experiences play a key role in the development of important managerial knowledge and skills beyond formal training (Davies & Easterby-Smith, 1984; Keys & Wolfe, 1988; McCall et al., 1988). In the U.S. Army, for example, experience, in the form of operational assignments, is considered one of the three pillars of leader development, along with institutional training and self-development (Department of the Army, 1990). The challenge is to ensure that individuals are exposed to the right learning opportunities and that they learn effectively and efficiently from those experiences. The TKML inventories can be used to help leaders identify areas in which they may need further development and can stimulate their thinking about important leadership issues (e.g., how to manage one’s anger in front of subordinates). Alternatively, the inventories can be used to help leaders develop the skills to learn more effectively from their experiences. Certain knowledge-acquisition and problem-solving processes are proposed to underlie the acquisition of TK (Sternberg, 1988, 1997). Knowledge-acquisition processes involve

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

137

selective encoding (filtering relevant from irrelevant information), selective comparison (relating new information to existing knowledge), and selective combination (putting information together to form a meaningful picture). Problem-solving, or executive, processes include problem recognition, problem definition, allocation of resources, strategy formation, solution monitoring, and outcome evaluation. An in-depth exploration of a TK scenario could be used to demonstrate more and less effective use of these processes. For example, some of these processes are exhibited in a story told of how one leader realized that his boss was inadvertently influencing the behavior of all his subordinates. The leader observed that all the subordinates were staying at the office until the same time as the boss but that the boss was not engaging in work-related activities (selective encoding). He concluded that the subordinates felt obligated to stay and work as long as the boss did (selective combination). He determined that the boss was unaware of this matter (problem definition) and informed the boss of how his actions were influencing his subordinates (strategy formation). The boss promptly explained that he used the time to unwind before going home and thereafter the subordinates began to leave at a reasonable hour (solution monitoring). 6.5. Directions for future research Although the present study provides encouraging results regarding the validity of TK in the domain of military leadership, a number of issues need to be addressed through further research. First, the validity of the TKML inventory should be tested with new samples of leaders, particularly at the battalion level, where our sample size was limited. Second, future research is needed to assess the predictive validity of TK relative to additional variables like formal job knowledge and relative to performance at higher levels of command. For example, does TK at the platoon level predict effective performance at the company level? Third, TK should be assessed relative to more rigorous and comprehensive performance criteria, which may include measures of unit performance or career success, and more appropriate measures of experience, such as one’s level in the chain-of-command. Fourth, researchers might explore how leaders take into account different perspectives (e.g., subordinates and superiors) in determining what course of action is appropriate in a given situation. It is clear from our results that leaders are not evaluated the same by different stakeholders. Finally, the TK methodology could be applied to more technical and tactical aspects of leadership (e.g., how to assess a battle situation), as well as entirely new performance domains such as medical care, law enforcement, or teamwork. Many professions have long recognized the importance of real-world practice in the development of expertise (e.g., medical residencies and corporate internship programs). Researchers also have begun to attend more to the role of on-the-job learning in employee development. As jobs become increasingly complex and environments continue to change rapidly, individuals will need to rely increasingly on their ability to acquire new knowledge from experience and apply that knowledge effectively to new situations. Thus, the type of knowledge we have characterized as tacit is expected to be an increasingly important determinant of successful performance.

138

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

Acknowledgements The authors would like to thank Richard C. Bullis, Jeffrey A. McNally, Patrick J. Sweeney, Trueman Tremble, and John Wattendorf for their various contributions to this work, as well as the anonymous reviewers for their helpful comments on earlier drafts. The research presented in this paper was sponsored by contract MDA903-92-K from the United States Army Research Institute for the Behavioral Sciences. The views and opinions expressed herein are solely those of the authors and do not represent the official view of the United States Military Academy, the United States Army, or any other agency of the United States Government.

References Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press. Bass, B. M. (1990). Bass and Stogdill’s handbook of leadership: theory, research, and managerial applications. New York: The Free Press. Bass, B. M., & Avolio, B. J. (1993). Transformation leadership: a response to critiques. In M. M. Chemers, & R. Ayman (Eds.), Leadership theory and research (pp. 49 – 80). San Diego, CA: Academic Press. Bettin, P. J., & Kennedy, J. K. (1990). Leadership experience and leader performance: some empirical support at last. The Leadership Quarterly, 1, 219 – 228. Borman, W. C., Hanson, M. A., Oppler, S. H., & Pulakos, E. D. (1993). Role of supervisory experience in supervisory performance. Journal of Applied Psychology, 78, 443 – 449. Chan, D., & Schmitt, N. (1998). Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143 – 159. Chi, M. T. H., Glaser, R., & Farr, M. J. (1988). The nature of expertise. Hillsdale, NJ: Erlbaum. Davies, J., & Easterby-Smith, M. (1984). Learning and developing from managerial work experiences. Journal of Management Studies, 21, 169 – 183. Department of the Army. (1990). Military leadership (FM-22-100). Washington, DC: U.S. Government Printing Office. Fiedler, F. E. (1992). Time-based measures of leadership experience and organizational performance: a review of research and a preliminary model. The Leadership Quarterly, 2, 5 – 23. Fiedler, F. E. (1995). Cognitive resources and leadership performance. Applied Psychology: An International Review, 44, 5 – 28. Fiedler, F. E., & Garcia, J. E. (1987). New approaches to leadership: cognitive resources and organizational performance. New York: Wiley. Ford, J. K., Quinones, M., Sego, D. J., & Speer-Sorra, J. (1992). Factors affecting the opportunity to perform trained tasks on the job. Personnel Psychology, 45, 511 – 527. French, J., & Raven, B. H. (1959). The bases of social power. In D. Cartwright (Ed.), Studies of social power (pp. 150 – 167). Ann Arbor, MI: Institute for Social Research. Grigorenko, E. L., Gil, G., Jarvin, L., & Sternberg, R. J. (2000). Toward a validation of aspects of the theory of successful intelligence. Unpublished manuscript. Groen, G. J., & Patel, V. L. (1988). The relationship between comprehension and reasoning in medical expertise. In M. T. H. Chi, R. Glaser, & M. Farr (Eds.), The nature of expertise (pp. 287 – 310). Hillsdale, NJ: Erlbaum. Hanson, M. A., & Ramos, R. A. (1996). Situational judgment tests. In R. S. Barrett (Ed.), Fair employment strategies in human resource management (pp. 119 – 124). Westport, CT: Greenwood Publishing Group. Hollander, E. P. (1985). Leadership and power. In G. Lindzey, & E. Aronson (Eds.), Handbook of social psychology (pp. 485 – 537). New York: Random House.

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

139

Jones, E. E. (1963). Ingratiation: a social psychological analysis. New York: Appleton-Century-Crofts. Keys, B., & Wolfe, J. (1988). Management education and development: current issues and emerging trends. Journal of Management, 14, 205 – 229. Legree, P. J. (1995). Evidence for an oblique social intelligence factor established with a Likert-based testing procedure. Intelligence, 21, 247 – 266. Manz, C. C., & Sims Jr., H. P. (1980). Self-management as a substitute for leadership: a social learning theory perspective. Academy of Management Review, 5, 361 – 367. Markham, S. E., & Markham, I. S. (1995). Self-management and self-leadership reexamined: a levels-of-analysis perspective. The Leadership Quarterly, 6, 343 – 359. Mayer, R. E. (2000). Intelligence and education. In R. J. Sternberg (Ed.), Handbook of intelligence (pp. 519 – 533). New York, NY: Cambridge University Press. McCall, M. W., Lombardo, M. M., & Morrison, A. M. (1988). The lessons of experience: how successful executives develop on the job. Lexington, MA: Lexington Books. McCauley, C. D., Ruderman, M. N., Ohlott, P. J., & Morrow, J. E. (1994). Assessing the developmental components of managerial jobs. Journal of Applied Psychology, 79, 544 – 560. Mintzberg, H. (1980). The nature of managerial work. Englewood Cliffs, NJ: Prentice-Hall. Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: the low-fidelity simulation. Journal of Applied Psychology, 75, 640 – 647. Mumford, M. D., Zaccaro, S. J., Harding, F. D., Jacobs, T. O., & Fleishman, E. A. (2000). Leadership skills for a changing world: solving complex social problems. The Leadership Quarterly, 11, 11 – 35. Neisser, U. (1976). Cognition and reality. San Francisco: Freeman. Olson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P. A., & Donovan, M. A. (1998). Interactive video assessment of conflict resolution skills. Personnel Psychology, 51, 1 – 24. Polanyi, M. (1966). The tacit dimensions. Garden City, NY: Doubleday. Quinones, M. A., Ford, J. K., & Teachout, M. S. (1995). The relationship between work experience and job performance: a conceptual and meta-analytic review. Personnel Psychology, 48, 887 – 910. Salam, S., Cox, J. F., & Sims, H. P. (1997). In the eye of the beholder: how leadership relates to 360-degree performance ratings. Group and Organization Management, 22, 185 – 209. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262 – 274. Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986). The impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance. Journal of Applied Psychology, 71, 432 – 439. Scho¨n, D. A. (1983). The reflective practitioner: how professionals think in action. New York: Basic Books. Sternberg, R. J. (1988). The triarchic mind: a new theory of human intelligence. New York: Penguin Books. Sternberg, R. J. (1997). Successful intelligence. New York: Penguin Books. Sternberg, R. J. (2000). The concept of intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence (pp. 3 – 15). New York: Cambridge University Press. Sternberg, R. J., Forsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M., Snook, S., & Grigorenko, E. L. (2000). Practical intelligence in everyday life. New York: Cambridge University Press. Sternberg, R. J., & Horvath, J. A. (1999). Tacit knowledge in professional practice. Mahwah, NJ: Erlbaum. Sternberg, R. J., Wagner, R. K., & Okagaki, L. (1993). Practical intelligence: the nature and role of tacit knowledge in work and school. In H. W. Resse, & J. M. Puckett (Eds.), Advances in lifespan development (pp. 205 – 227). Hillsdale, NJ: Erlbaum. Sternberg, R. J., Wagner, R. K., Williams, W. M., & Horvath, J. A. (1995). Testing common sense. American Psychologist, 50, 912 – 927. Terman, L. M. (1950). Concept Mastery Test. New York: The Psychological Corporation. Tesluk, P. E., & Jacobs, R. R. (1998). Toward an integrated model of work experience. Personnel Psychology, 51, 321 – 355. Thorndike, E. L. (1920). Intelligence and its use. Harper’s Magazine, 140, 227 – 235.

140

J. Hedlund et al. / The Leadership Quarterly 14 (2003) 117–140

Tornow, W. W. (1993). Perceptions or reality: is multi-perspective measurement a means or an end? Human Resource Management, 32, 221 – 229. Wagner, R. K. (1987). Tacit knowledge in everyday intelligent behavior. Journal of Personality and Social Psychology, 52, 1236 – 1247. Wagner, R. K., & Sternberg, R. J. (1985). Practical intelligence in real-world pursuits: the role of tacit knowledge. Journal of Personality and Social Psychology, 48, 436 – 458. Wagner, R. K., & Sternberg, R. J. (1990). Street smarts. In K. E. Clark, & M. B. Clark (Eds.), Measures of leadership (pp. 493 – 504). West Orange, NJ: Leadership Library of America. Wagner, R. K., & Sternberg, R. J. (1991). Tacit Knowledge Inventory for Managers. San Antonio, TX: The Psychological Corporation. Wagner, R. K., Sujan, H., Sujan, M., Rashotte, C. A., & Sternberg, R. J. (1999). Tacit knowledge in sales. In R. J. Sternberg, & J. A. Horvath (Eds.), Tacit knowledge in professional practice (pp. 155 – 182). Mahwah, NJ: Erlbaum. Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25 – 49. Yukl, G. (1998). Leadership in organizations. (4th ed.). Upper Saddle River, NJ: Prentice-Hall. Zaccaro, S. J., Gilbert, J. A., Thor, K. K., & Mumford, M. D. (1991). Leadership and social intelligence: linking social perceptiveness and behavioral flexibility to leader effectiveness. Leadership Quarterly, 2, 317 – 342. Zaccaro, S. J., Mumford, M. D., Connelly, M. S., Marks, M. A., & Gilbert, J. A. (2000). Assessment of leader problem-solving capabilities. The Leadership Quarterly, 11, 37 – 64.