Actor analysis methods and their use for public policy analysts

European Journal of Operational Research 196 (2009) 808–818

Contents lists available at ScienceDirect

European Journal of Operational Research journal homepage: www.elsevier.com/locate/ejor

Interfaces with Other Disciplines

Actor analysis methods and their use for public policy analysts Leon M. Hermans *, Wil A.H. Thissen Delft University of Technology, Faculty of Technology, Policy and Management, P.O. Box 5015, 2600 GA, Delft, The Netherlands

a r t i c l e

i n f o

Article history: Received 7 March 2007 Accepted 28 March 2008 Available online 4 April 2008 Keywords: OR in societal problem analysis Actor analysis Stakeholder analysis Policy analysis Multi-actor systems

a b s t r a c t Public policy analysts use methods rooted in OR and systems analysis to support policy makers in their judgement. In doing so, most policy analysts recognize the value of a certain understanding of the role of actors in policy making processes. Different methods are available to aid such understanding and, although they all focus on actors, there are important differences between them. Insight into the range of available methods and their characteristics will thus help policy analysts to learn more about the potential and limitations involved in analyzing multi-actor processes. This article provides such an overview, based on the main requirements these methods should meet. This overview is used to discuss some of the implications for policy analysts who are interested in analyzing multi-actor processes, focusing specifically on trade-offs between analytic quality and practical usability. Ó 2008 Elsevier B.V. All rights reserved.

1. Introduction Public policy analysts have traditionally concentrated on defining the content of policy problems and on designing and evaluating alternative solutions in order to assist policy makers. The methodology applied has its main roots in operations research (OR) and applied systems analysis (Quade and Miser, 1985; Walker, 2000). While the methods have become more sophisticated and varied over time, there is dissatisfaction among policy analysts about the role and the use of their work: ‘‘If policy analysis is not used, why do we produce so much of it?” (Shulock, 1999). More or less in parallel to the development of public policy analysis as a discipline during the 1960s and early 1970s, policy scientists have been criticizing policy analysts for being too narrowly focused on means-end rationality, pointing out the key roles of other types of rationality (political, procedural) and other factors in policy making, such as power, personal relations, strategic behavior and strategic use of information (Lindblom, 1959; Scharpf, 1973; Wildavsky, 1979). In this perception, policy making is a social process of and between actors, rather than a rational effort to search for the optimal solution given a fixed problem definition. While this view of policy making does not argue that traditional analysis is irrelevant, it emphasizes its limitations and partly explains why the products of analytic efforts have been used to a limited extent only. In response to these challenges posed by the multi-actor character of policy making, policy analysts have come up with a variety of new approaches, including participatory and interactive styles of * Corresponding author. Tel.: +31 15 2785776; fax: +31 15 2786233. E-mail address: [email protected] (L.M. Hermans). 0377-2217/$ - see front matter Ó 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2008.03.040

policy analysis (Thissen and Twaalfhoven, 2001; Mayer et al., 2004). The latter often draw on soft OR methods for problem structuring and participatory analysis, which typically combine process and content dimensions in problem solving (e.g. Eden and Ackermann, 2001). Regardless of the preferred mode or style, the work of policy analysts could benefit from an analytical reflection on the actors that play a role in the policy making realm. Consequently, policy analysts are increasingly interested in, and make increasing use of, methods that help them to get a better understanding of multi-actor policy processes. These methods are the focus of this article and they will be referred to as actor analysis methods. The most popular methods for actor analysis are the methods that have come to be known as stakeholder analysis (MacArthur, 1997; Bryson, 2004), but there are several other methods available to policy analysts who seek to get a better understanding of multi-actor policy situations. These include methods for social network analysis (Kenis and Schneider, 1991; Scott, 2000), cognitive mapping (Axelrod, 1976; Bots et al., 2000), and conflict analysis (Fraser and Hipel, 1984; Fang et al., 1993). These methods employ different theoretic perspectives, focus on different aspects of multi-actor processes, and put different requirements on analysts’ expertise and time. Therefore, a meaningful use of actor analysis methods requires insight into the range of available methods, their characteristics and potential use for the work of policy analysts. This article provides an overview of methods for actor analysis to help practitioners reflect on the scope and focus of the methods they are using. Similarly, it should help academics and teachers in public policy analysis to see what parts of multi-actor processes they are covering with the methods in their curricula. We start

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

by outlining the requirements that actor analysis methods should meet in order to be useful for policy analysts. These requirements are then used as a basis for the identification of methods for actor analysis. Next, a discussion is provided of some of the analytic and practical considerations that influence the use of these methods. 2. Requirements for actor analysis methods for policy analysts 2.1. Actor analysis methods: Purpose and scope of the review Actor analysis methods are similar to other OR methods, in that they should describe specific activities, designed to achieve a clear and well-defined purpose (Mingers, 2003). In the case of actor analysis, the purpose in which we are interested is to provide public policy analysts with insight into the multi-actor process of policy making. This has two important implications for the scope of our review. First, it means that we limit ourselves to OR application domains in public policy, although our findings and discussions may also be of relevance to other domains that have an important dimension of multi-actor complexity. For instance, stakeholder analysis is rooted in the strategic management domain (Mitroff, 1983; Freeman, 1984), where it is used extensively (e.g. Johnson et al., 2005). Second, we focus on the insights that actor analysis methods can provide into multi-actor processes, not on the role they can play in facilitating these processes. This further specification of our focus triggers two strands of questions, which are used to derive specific requirements for actor analysis methods. First: What is the multi-actor process of policy making and how is it understood theoretically and conceptually? Second: What are the relevant requirements for methods that aim to provide insights for policy analysis? 2.2. The multi-actor context of policy making: Theoretical dimensions Many theories address the role of actors in policy making processes and no single theory can be selected a priori as the best way to describe and explain these processes1. Nevertheless, comparing different theoretical studies from policy science suggests some structures and mechanisms that characterize most multi-actor policy making processes. Several authors emphasize that public policies are generally generated within networks in which multiple actors are interrelated in a more or less systematic way (Kenis and Schneider, 1991; Rhodes and Marsh, 1992; Klijn, 1997; De Bruijn and Ten Heuvelhof, 2000). Looking only at policy networks, however, has a limited potential to explain policy changes if not complemented by an analysis at a lower level in terms of properties of the actors (Rhodes and Marsh, 1992, p. 196). At this actor level, most theories converge around three basic dimensions that help explain actor behavior: perceptions, values, and resources (Mitroff, 1983; Sabatier, 1988; Jobert, 1989; Scharpf, 1997)2.

1 See for instance Ostrom et al. (1994, p.49), the overview book edited by Sabatier (1999) and the related debate on theories of the policy process in the Journal of European Public Policy (Dudley et al., 2000). 2 Although the labels differ, these three dimensions are identified by all cited authors, among others. Mitroff (1983, 36): identifies purposes and motivations of a stakeholder, beliefs that a stakeholder has and the resources a stakeholder commands. The advocacy coalitions framework includes belief systems, consisting of normative and causal beliefs (cf. values and perceptions), and resources as the main internal forces that drive the behavior of coalitions of actors (Sabatier, 1988, 131–132). Jobert identifies three dimensions of policy making: cognitive, instrumental and normative (Jobert, 1989, 377). The actor centered institutionalism frameworks states that: ‘‘Actors are characterized by specific capabilities, specific perceptions, and specific preferences” (Scharpf, 1997, 43).

809

If, in a somewhat crude simplification, one takes the network level to be a fourth dimension, the object of an actor analysis thus can be characterized along the following dimensions: 1. Networks: ‘‘More or less stable patterns of social relations between interdependent actors, which take shape around policy problems and/or policy programmes” (Klijn, 1997, 30). In these networks, the institutional context and rules limit and structure the possible range of activities (Ostrom et al., 1994). 2. Perceptions: The image that actors have of the world around them, both of the other actors and networks, and of the substantive characteristics of a policy problem (Bots et al., 2000; Scharpf, 1997). Perceptions may also be labelled causal beliefs, cognitions or frames of reference. Perceptions here refer only to ‘neutral’ theories of how the world operates, and not to normative beliefs on what is good and desirable. The latter are discussed under the dimension of ‘‘values”. 3. Values: These provide the directions in which actors would like to move; they describe the internal motivations of actors. Related concepts such as ‘norms’, ‘interests’ and ‘purposes’ function on a more abstract level, whereas ‘objectives’, ‘goals’ and ‘targets’ express values in more specific terms. ‘Preferences’ and ‘positions’ translate values into a (relative) preference ordering over specific solutions or policy outcomes. Variables on this dimension are closely linked to actors’ perceptions (see also Sabatier, 1988, 131–133). 4. Resources: The practical means or instruments that actors have to realize their objectives. Resources are the ‘‘things over which they have control and in which they have some interest” (Coleman, 1990, 28). Resources enable actors to influence the world around them, including other actors, relations and rules in a network. As such, resources are closely related to power and influence (Thomson et al., 2003, 8).

2.3. Requirements for methods that aim to provide insights for policy analysis Policy analysts typically work in an environment where it is important to be problem and client-oriented, to be pragmatic in fitting methods to problems and in using the simplest methods that will do the job (cf. Wildavsky, 1979; Miser, 1985; Walker, 2000). Likewise, we are not looking for elaborate theories or complicated models that provide the most accurate and detailed descriptions of multi-actor processes. Rather, we are looking for methods or models that are usable in practice – that provide guidance for the representation of multi-actor processes, in a way that captures the crucial elements, but without making things unnecessary complex or difficult. Although the use of policy analysis in practice implies that it is an art and craft rather than an exact science, the knowledge generated and disclosed by the use of policy analytic methods should meet certain standards, to discriminate between guesswork and sound analysis. Here, scientific method provides the model (Quade and Miser, 1985; Walker, 2000).3 This means for instance that the results of an actor analysis should be transparent and accessible, so that its accuracy can be assessed by a large group of people. Also, knowledge that has been subjected to a scientific peer-review process provides a useful basis on which to build.

3 We realize that different perspectives and roles of policy analysis may place more or less importance on the scientific criteria to evaluate the quality of the insights provided by policy analysis (e.g Thissen and Twaalfhoven, 2001; Mayer et al., 2004). As our focus here is on the content rather than the process dimension of using actor analysis, the scientific model is highly relevant.

810

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

3. Overview and description of actor analysis methods 3.1. Identifying actor analysis methods The previous section outlines some of the requirements that actor analysis methods should meet. Actor analysis methods should: 1. Provide (comparative) insight into the characteristics of multiple actors involved; 2. Focus on one or more dimensions of multi-actor policy processes (networks, values, perceptions, resources); 3. Describe specific analysis activities, meaning that methods or past applications are described in sufficient detail to allow one to reconstruct their use; 4. Have proven use in practice for analyzing the role of actors in real world policy making; 5. Have been subjected to scientific scrutiny, illustrated by scientific peer reviewed publications of method development and/or use. In a review of (soft) OR and policy analysis literature, we identified eighteen methods that score satisfactorily on these requirements. We do not claim to be exhaustive in our review, but we can be confident that our sample is representative of the type of methods that are available, as our sample covers the four main dimensions of multi-actor processes and includes methods from different theoretic domains. Some methods that one might expect to find in our overview have been excluded. For instance, some of the problem structuring methods that aim to articulate perceptions or cognitive maps of actors. Although those methods are certainly appropriate for use in multi-actor contexts, their main aim is to facilitate joint problem formulation and problem solving, rather than to provide insight into the multi-actor processes and networks. In merging perceptions and seeking consensus, the basis for a comparative analysis of actor characteristics is lost. Likewise, preference elicitation methods are only considered to focus on multi-actor systems if they explicitly recognize and analyze the different preferences of different actor groups. Applications that lump all preferences into one aggregate preference, for instance to represent the preference of the general public, are thus excluded. Finally, composite methods are not included. For instance, the role-think approach in the

Journey Making methodology (Eden and Ackermann, 1998) extends existing stakeholder analysis methods and techniques with elements of hypergame analysis and network analysis. The latter methods are included in our overview, the composite approaches that combine them are not. The methods in the overview meet the basic requirements outlined above, but this does not mean that they are identical for each of these requirements. Learning more about how they meet the requirements will help to decide when (not) to use a particular method. Therefore, we have adopted a framework for OR methods developed by Mingers (2003) to further describe the selected actor analysis methods. This framework describes methods in terms of their assumed purpose and use (axiology), the main concepts and causal relations a method assumes to exist (ontology); and the forms of knowledge and knowledge creation a method uses and the types of models that are being used to represent information from various sources (epistemology). Tables 1–3 have been developed using this framework. Each describes a sub-set of the identified actor analysis methods, based on their main focus on one of the four dimensions: networks, values, perceptions, and resources. This does not mean that each method only covers one dimension and not others, but it reflects that most methods put more emphasis on one particular dimension. The overview in these tables can be used to enable the comparison of methods and to help policy analysts decide which methods best meets their specific needs and conditions. The next sections will illustrate the use of this overview to facilitate a comparative discussion, focusing on the analytic quality and practical usability of the methods. These aspects are highlighted here as they have been identified above as important requirements for policy analysts. 4. Reviewing the analytic quality of actor analysis methods 4.1. Specifying measures for analytic quality Three measures are applied to Tables 1–3 to compare the methods therein with regard to analytic quality. Following Sabatier (1999, 5–6), we distinguish three relevant characteristics that influence the analytic quality provided by a method. The first is scope, based on the idea that covering a larger part of the space that characterizes multi-actor processes is likely to result in increasing

Table 1 Methods focusing on network level and on values Method

Framing/ structuring of. . .

What it assumes to exist

Model representation

Networks Network analysis: policy making as process influenced by network structure Social network Relational Actors and relations Relational graphs with analysis (Kenis characteristics between them actors as nodes, and Schneider, of actor relations as ties 1991; Scott, networks 2000) Configuration Debates among Actors, truth Multi-dimensional analysis groups of definitions, rules and scaling to graphically (Termeer, 1993) actors, patterns of interaction present configurations changing over of actors time Values Preference elicitation: Policy making as choosing the prefered alternative AHP, multi-attribute Hierarchy in Actors, objectives and Hierarchies and assessment,. . . various preferences, matrices, representing (Saaty, 1990; attributes and alternatives featuring rankings of attributes Ananda, 2007; alternatives various attributes and alternatives etc.)

What it does

Necessary information

Sources of information

Represent network structure using graphmodels and statistical analysis of relations

Relational data: direction, frequency and intensity of relations between actors Historic and relational data on interactions between actors and truth definitions

Surveys and interviews, historical records

Actors’ views on alternatives, attributes and their ranking

Documents/ interviews for problem structuring; questionnaires for ranking by actors

Relate similarities in truth-definitions to frequency and intensity of interactions

Produce hierarchy of elements or type of utility model by ranking attributes & alternatives

Policy documents, interviews with actor representatives

Table 2 Methods focusing on actors’ resources Method

Framing/ structuring of. . .

What it assumes to exist

Stakeholder analysis: policy making as a process influenced by external stakeholders Stakeholder Stakeholder environment, Stakeholders with influence on project analysis to assess cooperative success and interest in its outcomes (Freeman, 1984; potential and threat of Bryson, 2004) obstruction

What it does

Necessary information

Sources of information

Tables and matrices for stakeholder classification and participation strategies

Collect and structure information on stakeholders, resulting in specific participation strategies for each group

Stakeholders, their interests and influence, range of other issues on checklists

Key-informants, documents and, sometimes, interviews with stakeholders Policy documents and interviews with actors or knowledgable persons Policy documents and interviews with actors or knowledgable persons Policy documents and interviews with actors or knowledgable persons Interviews with actors or key informants, actors’ documents Policy documents and interviews with actors or knowledgable persons Policy documents and interviews with actors or knowledgable persons

Conflict analysis: policy Analysis of Options (Howard, 1971, 1989)

making as a non-cooperative game Policy ‘game’, to identify Actors with different interests and conflict, control, areas for options to exert control over issues of bargaining interest

Analysis of options tables, representing actors, their options and preferences

Identify and structure actors, their options to exert control and preferences

Actors involved in issues, their options to influence outcomes and preferences

Metagame analysis (Howard, 1971; Fraser and Hipel, 1984)

Policy ‘game’, to identify stable outcomes and strategies for negotiation and coalition building

Actors with different options to exert control over issues of interest and preferences for certain outcomes (positions)

Game matrices and strategic maps, showing control and preferences of actors and stability of outcomes

Identify and structure actors, their options, feasible outcomes and preferences; assess stability of outcomes mathematically

Actors, their options, and ordinal preferences for all feasible outcomes

Graph Model for Conflict Resolution (Fang et al., 1993) Hypergame analysis (Bennet et al., 1989)

Policy ‘game’ and impacts of different risk management strategies

Actors with different options and different positions, in face of uncertainty

Game matrices and strategic maps, showing control and preferences of actors and stability of outcomes

Identify and structure actors, options, risk strategies and preferences; assess stability of outcomes mathematically

Actors, their options, risk strategies and ordinal preferences for all feasible outcomes

Policy ‘game’ and role of (mis)information and strategic surprise

Actors with different perceptions on each others interests, options, and feasible outcomes

Game matrices and strategic maps as perceived by each of involved actors

Identify and structure actors’ perception of conflict (similar to GMCR, for each actor)

Actors’ views on who is involved, their options, preferences, outcomes

Series of games and the dilemmas that trigger transformation of these games

Communicating characters with evolving preferences, cards, positions and fallbacks, facing dilemmas

Card tables, showing positions of actors and threatened future

Identify and structure actors and games (characters and episodes), and identify dilemmas

Actors involved in issues, their options to influence outcomes and preferences

Position of parties in conflicts, to predict actors’ behavior and outcomes of conflicts

Actors who, driven by expected utility and risk attitudes, challenge each others’ policy positions

Decision trees depicting challenges to actors’ positions and their expected utility from accepting or giving in

Assess and structure policy positions and power of actors, as well as interests and support from others

Interests, power and positions of parties in a conflict

Matrices showing interests and control; graphs showing inter actor dependencies

Assess actor dependencies, based on interests and control over issues

Interest in issues of actors; control over issues of actors; perceived dependencies

Interviews or interactive session with actors involved in issues of interest

Drama theory (& confrontation analysis) (Bennet et al., 2001) Expected utility model (see Thomson et al., 2003) Transactional analysis: Transactional process models (Coleman, 1990; Timmermans, 2004) Vote-exchange models (Stokman, 1994; Thomson et al., 2003) Dynamic access models (Stokman and Zeggelink, 1996)

policy making as political bargaining Potential for exchange of Actors with interests in issues and control between different (perceived) control over those issues actors in a policy process

Policy positions of actors, to predict shifts in positions and outcomes of collective decision making

Actors with different policy positions, salience and voting power regarding issues of interests, engaged in mutually beneficial exchange of policy positions

Lists of potential exchanges between pairs of actors

Assess influence of actors on specific decisions and predict outcomes through algebraic modelling of potential exchanges

Actors’ policy positions (preferences for alternatives); salience; capabilities to influence decision making

Interviewing key informants

Access relations and their consequences for decision outcomes

Networks of actors with preferences, resources and (perceived) access to decision makers who take part in final vote

Graphs showing access relations between actors; Outcome matrices based on different rules for actor behavior

Explain establishment and shifts in access relations, and assess influence of access relations on decision outcomes

Actors’ preferences, voting power, access to decision makers, actors’ rules and estimates of acceptance access requests

Interviews with experts or representatives of actors

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

Model representation

811

Documents, but preferably interviews or participatory working sessions Actors’ assumptions on facts, links and goals in relation to policy problem Causal maps of actors: their assumptions on main objectives, instruments, factors and causal relations Actors with individual beliefs about important factors, causal relations, policy instruments and objectives

Represent actors’ perception: assumptions about main factors, goals, instruments and causality

Interview sessions for semistructured interviews with the main actors Actors’ views on policy problems and assumed causality Coded causal maps of actors, consisting of factors and causal relations

Cognitive mapping: policy making as problem solving Comparing causal Perceptions of actors to see maps (Jenkins, commonalities and differences 1994) and to explain different strategies Dynamic Actor Perceptions of actors to enable Network Analysis comparitive analysis of (DANA) (Bots agreement, conflict, problem solving potential, etc. et al., 2000)

As above + groups of actors sharing a similar perspectives

Actors with beliefs about important factors and their causal relations

Construct (coded) cognitive maps for each actor

Sample of statements representative for policy debate and opinion of actors on statements

Policy documents, transcripts of meetings, news articles, interviews with parties in the debate As above + surveys with actors for opinions on statements Actors’ views on controversial issues

Narratives that express judgements on roles of actors; problems/injustice; instruments; etc. Q-sorts: actors’ opinions on representative set of statements A relatively open and transparent policy debate on a highly controversial issue.

Represent different positions in policy debate as narratives and compare those to construct meta-narrative Uses statistical factor analysis to correlate people’s views based on sample of statements

Reasoning used in policy debate Chains of arguments, consisting of ground, claim, warrant, backing, rebuttal, modality A relatively open and transparent policy debate, with a certain degree of logic

Apply an adapted version of formal logic to structure reasoning used in policy debates

Necessary information What it does Model representation What it assumes to exist Framing/ structuring of. . . Method

Discourse analysis: policy making as exchanging arguments Argumentative Different chains of reasoning analysis used in policy debate and (Toulmin, 1958; underlying values and Mitroff, 1983, assumptions Chapter 8), Narrative policy Opposing views of controversial analysis (Roe, problems and possible meta1994; Van Eeten, narratives to reformulate those 1999) problems Q-methodology Groups of actors with shared (McKeown and perspectives and their Thomas, 1988) underlying basis

Table 3 Methods focusing on actors’ perceptions

Policy documents, transcripts of meetings, interviews with parties in the debate

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

Sources of information

812

explanatory potential and decreases the risk that one overlooks crucial aspects. An indication of scope is obtained by assessing how many of the four main dimensions (networks, perceptions, values, resources) are covered by the concepts included in a method. A second relevant characteristic is specificity, based on the notion that scientific standards require analysts to ‘‘be clear enough to be proven wrong” (Sabatier, 1999, 5). This characteristic can be traced back to the demarcation criterion of falsifiability as formulated by Popper (1963, 5–6)4, If a method uses specific rather than abstract concepts, its concepts carry more detailed information, are easier to relate to empirical observation and require less interpretation by the analyst. This means that the analysis procedure and underlying assumptions will be easier to communicate to others and leave less room for implicit and potentially erroneous assumptions by the analyst. An approximate measure for the specificity of actor analysis methods is formulated by ranking the concepts and variables used in methods on an ordinal scale, going from more abstract to more specific variables. This can be done for each of the four dimensions of networks, values, resources and perceptions. For instance, when discussing resources, concepts such as ‘‘power” and ‘‘influence” are quite abstract; ‘‘control over an issue” is somewhat less abstract; ‘‘options and instruments to (partially) control an issue” is already quite specific; and ‘‘voting power to control formal decisions” is most specific. Details of this specificity assessment are contained in Appendix A. A third measure for analytic quality is logical interconnectedness, which, similar to specificity, can be related to the issue of falsifiability. If the logic through which a method reasons from basic concepts and propositions to conclusions is stronger and more forceful, there is, again, less need for interpretation and assumptions by the analysts. This is most clearly the case when methods use mathematical expressions to formalize the relations between concepts and their consequences. An approximate measure for this logical interconnectedness is used, based on a three-point scale that distinguishes methods in which variables are tightly connected in a mathematical model, methods in which variables are connected through the use of a predefined structure or conceptual model, and methods in which there is only a loose connection, suggested by the use of checklists, matrices and guidelines. 4.2. Comparing the potential of actor analysis methods for analytic soundness Fig. 1 graphically presents an overview of analytic characteristics of actor analysis methods, summarizing specificity, logical interconnectedness and scope. Details on this figure are provided in Table A.2 in Appendix A. Fig. 1 has an indicative function and should be interpreted as providing a rough comparison of methods relative to each other. It gives an indication of the analytic quality one can expect from using methods in their standard form, but does not say anything about the type of explanations offered, the input data required or the possibilities to use methods in non-standard ways. For instance, methods with less specificity still may produce analytically sound outcomes if the user makes an effort to explicate how fairly abstract variables were operationalized and related to empirical observations. Nevertheless, when analytic quality is the main concern, one would be served best by methods represented by a large circle at the top right part of the diagram.

4 Even though this criterion has been criticized as being too strict and narrow (e.g. Putnam, 1981), we believe that this criticism does not affect the usefulness of falsifiability as a criterion to assess the analytic quality of actor analysis methods. Falsifiability remains important to policy analysts, as ‘‘the essence of policy analysis is learning to recognize and correct errors” (Wildavsky, 1979, p. 389).

Most specific

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

813

Vote exch Multi-attr Soc netw an

Hyper game

Drama theory

Argument

GMCR Metagame Dyn acc DANA Q-meth

Anal of options

Exp utility

Specificity

Config

Narrative

Transact Comp. causal maps

Most abstract

Stakeh

(Size of bullets indicates number of dimensions covered)

Checklists only

Logical interconnectedness

Mathematical models

Fig. 1. Analytic characteristics of actor analysis methods

Three methods are located at the top right corner: social network analysis, multi-attribute assessments and the vote-exchange model. Two of these methods cover just one dimension: multiattribute assessments, which can be used to analyze actors’ values (based on preferences), and social network analysis, which can be used to analyze the structure of actor networks. Both methods are frequently used and can be expected to support an adequate degree of analytical rigour on the investigated aspects. Notwithstanding that this can provide useful insights, these methods lack explanatory power when it comes to interactions between actors, because they only deal with a small part of the actor characteristics. Social network analysis gives insight into possibilities and conditions for interactions, but no explanation of why and when interactions actually occur. Multi-attribute assessments give insight into preferences of actors, but no explanation of when and how preferences will result in actions and collective decisions. The vote-exchange model is also placed at the top-right corner of the diagram and covers more than one dimension, thus having a better potential to provide sound explanations. This model makes specific assumptions on several dimensions, which increases its analytic quality, but which also limits its applicability to a specific sub-set of policy making processes. The vote-exchange model assumes actors to be fully aware of their own and each other’s interests and capabilities regarding the issues to be decided, and to be willing and able to exchange positions on all the issues in a rational way. Furthermore, the model uses numerical descriptions of the variables, which requires a specific data collection procedure. So far, it has mainly been used to analyze processes in which decisions are made ultimately by voting (see Thomson et al., 2003).

4.3. Trade-off between analytic quality and practical usability? The discussion of the three most analytically sound methods suggests trade-offs between a method’s analytic quality and its usability in practice. For instance, a high degree of logical interconnectedness supports analytic quality, but only if the input variables are assessed accurately. Thus, if logical interconnectedness is combined with a large number of variables, analytic quality will come at the expense of ease of use. Similarly, a more specific method produces more specific insights that are easier to verify, thus contributing to analytic quality. At the same time, practical usability is negatively affected as the range of situations in which a method is applicable reduces and as data collection requirements increase. The trade-off between analytic quality and practical usability is also aptly illustrated by looking at the other, lower left, side of the diagram in Fig. 1. Here, we find stakeholder analysis, which indicates that the analytic quality of this method is wanting. In fact, the literature on stakeholder analysis acknowledges the analytical limitations stemming from ‘‘laundry lists” of concerns and issues (Mitroff, 1983, 46) and ‘‘provisional” assessments of levels of influence, support or opposition” (Varvasovszky and Brugha, 2000, 342). Despite these analytical weaknesses, stakeholder analysis is the most popular method for actor analysis used in practice. Apparently, the analytical weaknesses of stakeholder analysis are off-set by advantages in practical usability. These are related to its ability to accommodate gaps in input data, the fact that it does not require specific skills beyond a fair amount of common sense, and the fact that the presence of various loosely coupled items in checklists and tables allows analysts to selectively pick out those items that are most appropriate and easy to cover.

814

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

5. Considering the practical usability of methods What method is appropriate to policy analysts in practice depends on a range of characteristics, including analytic quality, but also the focus of a method, the dimensions and concepts that are covered, the position of actor analysis activities within the larger set of policy analytic activities, the time and skills available, etcetera. Essentially, the characteristics of an actor analysis method should match the needs and constraints that arise from the situation in which policy analysts find themselves. This section illustrates how the characteristics of actor analysis methods that are contained in Tables 1–3 and in Fig. 1 can be matched against some of the situation-specific needs and constraints that are likely to play a role in practice. The discussion in this section follows the practical guidelines that are available in various texts on stakeholder analysis (Mitroff, 1983; Grimble and Chan, 1995; ODA, 1995; MacArthur, 1997; Varvasovszky and Brugha, 2000). It is not our intention to repeat the guidelines provided in these texts. Rather, we will complement them with a discussion of issues that are relevant for the selection of an appropriate method for a given situation. This section draws on experiences obtained in applying actor analysis methods as part of policy analytic projects related to water resources management (Hermans, 2005). 5.1. Preparation: Identifying purpose and focus of analysis The logical first step in preparing an actor analysis is to establish the purpose of the analysis and the specific questions that the actor analysis should address. In some cases, this may lead directly to the identification of one or a few appropriate methods. In other cases, an actor analysis may seem useful, even though the exact purpose cannot be articulated yet. Here, it may be helpful to review the character and phase of the policy analysis process of which the actor analysis is to be a part, and to assess the knowledge that is already available on actors and actor networks. 5.1.1. Phase in the policy analysis process Several phases can be distinguished in policy analysis processes of which a gross distinction is made between problem analysis and solution analysis (Weimer and Vining, 1989, 183). Generally, information on the perceptions of actors is more likely to be useful in problem analysis activities in the earlier phases of policy analysis, while information on the means and willingness of actors to contribute to, or frustrate, the implementation of solutions are more likely in later phases of policy analysis. This suggests that policy analysis activities aimed at problem analysis are more usefully supported by a method which focuses on perceptions. A focus on resources will be more useful in the later stages where solutions are being analyzed and discussed. A focus on networks and values may be (in)appropriate in either stage, depending on the specific type of values or network characteristics investigated. Also, as one progresses in a policy analysis process, more information is likely to become available and some of the specific issues that should play a role in the policy analysis may be known, such as specific criteria and promising alternatives. This suggests that methods with a higher degree of specificity are probably more fruitfully applied in connection to more advanced policy processes and policy analytic activities. 5.1.2. Apparent disagreement, conflict or resource dependencies A broad preliminary exploration of the actor context, based on readily available information, is generally considered useful in preparing an actor analysis (Grimble and Chan, 1995; ODA, 1995; Brugha and Varvasovszky, 2000). Such a quick-and-dirty assessment

should cover each of the four identified dimensions: important actors and relations in the network, actors’ values, resources and perceptions. This may help to suggest a focus for further analysis. For instance, if there is a large disagreement in the perceptions of different actors, a focus on perceptions seems worthwhile. If resources are distributed over various actors in the network, a focus on resources becomes more interesting. If there is a large agreement on the main values, the specific objectives and their relative importance, actor analysis may not be so important after all, but it may be better to focus attention on bringing actors around the table to initiate discussions – although it should be noted that this will rarely be the case when complex policy issues are being addressed. Apparent consensus among all actors on all of the main issues may also indicate that not all relevant actors or issues have been identified yet. 5.1.3. Covering more dimensions of the actor context In some cases it is necessary to cover several, or all, dimensions. In these cases, one can look for methods that cover more dimensions, i.e. the bigger dots in Fig. 1, but one may also consider using a combination of methods with a different focus. For instance, argumentative analysis (focusing on perceptions) and analysis of options (focusing mainly on resources) have been usefully combined in the past (Horita, 2000; Hermans, 2005). In combining methods, it will be useful to identify methods that differ in focus but that are similar in terms of information needs and sources of information. If one ensures that the questionnaires used in surveys or interviews cover issues that are of importance for both methods, the additional effort required for using two methods is limited. Finally, it is speculated that a method that has a lower level of specificity and logical interconnectedness is easier to use in combination with other methods than highly specific and ‘connected’ methods. 5.2. Data collection for actor analysis 5.2.1. Matching data requirements with opportunities for data collection Gathering information on the actors involved in policy making processes is a crucial but difficult step in actor analysis. Useful written sources of information are generally rare (Roe, 1994; Axelrod, 1976), which means that analysts will have to invest a significant amount of time in interviews, surveys or thorough document searches. Here, getting access to actors and ensuring their collaboration poses additional challenges (Jenkins, 1994). Thus, it is critical to compare the data requirements of different methods with the expected possibilities and limitations for data collection. For example, Q-methodology requires the preparation of a structured survey with statements that represent the policy debate. This can work well if sufficient information is available to prepare the survey. Alternatively, dynamic actor network analysis often uses semi-structured interviews that can be held even if little information is available in advance, but that require access to actors for face-to-face communication. 5.2.2. Making the most of interactions with respondents Analysts have to focus their data collection efforts on the key aspects, to ensure they make optimal use of the time a respondent has available for them. Methods that are relatively specific and have a higher degree of logical interconnectedness support the analyst in this regard, by prescribing the information that is needed to construct specific actor analysis models. The guidance offered by less structured methods such as stakeholder analysis is generally limited to a large list of possible items that can be covered and the observation that it may well be impossible to cover all of them (see e.g. Mitroff, 1983; Grimble and Chan, 1995; ODA, 1995).

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

5.2.3. Dealing with communication barriers Data collection may be hindered by geographical, language or cultural barriers. As more of such barriers are present, the use of more logically interconnected and specific methods may offer useful support. When less communication is possible due to language, cultural or geographical barriers, the use of pre-structured data-collection formats, which can be discussed and prepared with knowledgeable informants if appropriate, may facilitate the analyst in his/ her effort to obtain data that fit the analysis requirements. On the other hand, methods that are less logically interconnected and/or less specific, such as analysis of options, dynamic actor network analysis and argumentative analysis, are usually better suited to deal with incomplete input data because they are more flexible in their use of mathematics or use no mathematics at all. As they are less strict in their data requirements, the risk of a mismatch between method and data availability is smaller with these methods. 5.3. Analysis and interpretation of results How the collected data are structured and analyzed depends primarily on the method used for the actor analysis and here the more logically interconnected methods will offer more support than the more loosely coupled ones. Subsequently, the analysis outcomes should be interpreted and translated into conclusions and recommendations that can be used to support policy analytic activities. Also here, methods with a strong internal logic can help to provide focus and to support the interpretation of results, but even they do not replace the analyst’s judgement in arriving at final conclusions and recommendations. In the end, the analyst has to interpret the analysis outcomes to see what insights are most meaningful and how they translate into ‘‘actionable” recommendations that can help to improve policy analysis processes.

6. Discussion Finding an appropriate balance between analytic quality and practical usability is among the most difficult issues when deciding on the use of actor analysis methods. The current dominance of stakeholder analysis illustrates that in many cases considerations of practical usability override concerns related to analytic quality. Notwithstanding this current practice, the previous sections suggest that in many instances it will be worthwhile and possible to upgrade the appreciation for analytic quality in actor analysis. However, arguing for more analytic quality, possibly at the expense of a certain degree of ease of use, may give rise to several objections, some of which are discussed in this section. 6.1. Cost-effectiveness Tightening the analytic standards for actor analysis, partially at the expense of ease of use, raises the question of cost-effectiveness of using more analytically demanding actor analysis methods: can one afford to do it? There has been some discussion on this in relation to Q-methodology, where it was questioned whether or not such an analysis yields any surprising insights beyond the ‘‘qualitative picture that would emerge directly from interviews” (Weimer, 1999, 429). Arguably, it is exactly this doubt that has given rise to dominance of stakeholder analysis in the field. Nevertheless, past applications indicate that more structured actor analysis methods do have the ability to yield new insights (for instance Van Eeten, 1999, 2001; Hermans, 2005). Thus, a counter-question is if investing some time and effort in a thorough actor analysis is out of place in a policy analysis process that aims to support decision making on costly policy measures, for which failure or delay in implementation may be even more costly. The question then

815

is if one can afford to let ease of use prevail over analytic rigour in actor analysis. 6.2. Limits to validity 6.2.1. An argument in favour of quick-and-dirty methods? Making an argument for more analytic quality in actor analysis will invariably trigger scepticism related to the validity of actor analysis outcomes. The existence of covert interests, ambiguous power structures and hidden agendas poses limits to validity, as is known from literature (ODA, 1995; Varvasovszky and Brugha, 2000; Hermans, 2005). Creating a safe environment for participating actors, good data collection skills and cross checking of information may lower the impact of hidden agendas, but analysts will never be able to draw out the complete picture of all hidden motivations and informal power structures. These inherent limits give many critical observers reason to doubt the use of any additional effort that is primarily targeted at increased analytic quality. However, our conclusion from these inherent limits to validity is not that any attempt at being logical, structured and thorough is useless when talking about capricious and unpredictable actors, but rather, that this gives all the more reason to ensure analytic quality. Actor analysis methods that are more specific and more structured, based on some clear assumptions and theories about multi-actor processes, offer a model to guide observations and discussions. These methods enable analysts to identify anomalies in the collected data, by fitting the data to the method’s logic and by comparing the observed policy making process with the expectations generated by the analysis. An analytically sound method provides observers of policy processes with a lens that enables them to pinpoint surprising and seemingly irrational phenomena, which helps them to learn more about the presence of hidden agendas and ambiguous power structures. 6.2.2. The role of actor analysis The use of analytically sound methods may help analysts to identify some of the factors and processes that were previously hidden, but not all of them. If one expects the actor analysis to provide a definitive picture of what the multi-actor policy process looks like, the existence of hidden agendas and ambiguous power structures remains a worrying factor. However, when actor analysis is used as a tool to better link the policy analysis process to the policy making process, these limitations are less worrying. Policy making processes generally feature a ‘‘front-stage” and a ‘‘backstage”, with the decisions being taken on the backstage and the rationalization of decisions being communicated to the public on the front-stage (De Bruijn and Ten Heuvelhof, 2000). Policy analysts need to be able to explicate the front-stage information and considerations and they need to include them in their analysis. They also need to develop a sense of what happens backstage, but in order to be effective, they often do better to leave most of this information backstage. Bringing backstage information, hidden agendas and covert interests to the public front-stage is more likely to disrupt than support sensitive policy processes (see also ODA, 1995). Actor analysis has a role in explicating aspects of policy making that were previously unsaid and ambiguous, but mainly to enable different parties to express their concerns and interests better, not to explicate interests or ambiguous power mechanisms that they do not want to spell out to the outside world. Some hidden agendas and ambiguous power structures are not supposed to be discussed in the public policy making debate and policy analysts are not expected to address them explicitly. In striving to support multi-actor policy making processes, policy analysts would do good to recognize their role in the process and restrain their inclination to determine unilaterally what information actors should share.

816

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

Of course, we have started our overview of actor analysis methods with a specific type of OR practitioner in mind: the policy analyst working to support policy making. This poses limits to the scope of our discussion, also when it comes to the limits to validity of actor analysis. These limits to validity will continue to be important for other types of policy analysts, who are not working to support ongoing policy making processes, but who are interested to use actor analysis for instance to support truthful reconstructions of past policy processes or to make predictions of others’ policy decisions for strategic purposes. 7. Conclusion One of the most fundamental challenges for policy analysts working to support policy making, is to improve the link between themselves and the policy making processes. In addressing this challenge, some understanding of the multi-actor characteristics

that complicate most policy making processes is indispensable. There are different methods that can help policy analysts to make sense of multi-actor policy processes, and there are certainly more methods to be considered than the few methods that are usually known to the individual analyst. The different focus of different methods makes them useful for different situations, as well as complementary. And although in many cases there seems to be a trade-off between analytic quality and practical usability, there are cases where methods with more analytic quality offer also clear practical advantages. Therefore, we provided a framework for the identification and description of actor analysis methods, to help policy analysts to make well-informed choices for methods that fit their needs. We do not claim to be exhaustive in the requirements that were identified for actor analysis methods, in the identified methods, or in the discussions of those methods. This would be neither possible nor meaningful, as the selection and use of actor analysis methods

Table A.1 Specificity of actor analysis methods Dimension

Most abstract

Abstract

Specific

Most specific

Network

Relations

Access relations

Frequency, intensity of interactions

Methods

(Stakeholder analysis)a

Dynamic access models

Configuration analysis

Frequency & intensity + direction, reciprocity & durability Social network analysis

Perceptions

Perceptions of groups of actors Perceptions of individual actors

Q-Sorts Perceived factors and causal relations between them Q-Methodology, Comparing causal maps, Configuration anal

Narratives, arguments, games Factors, goals, instruments, type of causality, preferences Narrative, Argumentative, DANA, Hypergame, Drama theory

Resources

Power, influence

Control over an issue

Methods

Stakeholder analysis

Transactional analysis, Expected utility

Options/instruments/cards to (partially) control an issue All conflict analysis methods rooted in Game Theory, DANA

Voting to control a specific type of formal decision Dynamic access, Vote exchange

Values

Normative beliefs

Interests, values

Goals, targets, objectives

Methods

Narrative analysis

Stakeholder analysis, Comparing causal maps, Transactional analysis

DANA, Analysis of options

Preferences (options, risks), positions (preferred outcomes) Dynamic access, Vote exchange, Q-method, Multi-attr, GMCR, Meta/ hypergame, drama theory

Methods

a Stakeholder analysis does not necessarily cover the relations between actors, but in some of its variants and applications these relations are part of the analysis, for instance when they provide an indirect source of power or influence.

Table A.2 Potential analytic quality of actor analysis methods Logical interconnectedness (checklist, conceptual model, mathematical model)

Specificity (most) Abstract – (most) Specific

Scope # dimensions covered

Coverage, main dimension: Networks, Values, Resources, Perceptions

Network focus Social network analysis Configuration analysis

M M

MS A/S

1 2

N N

P

Perceptions focus Argumentative analysis Narrative analysis Q-method DANA Comparing causal maps

C C M M M

S A S S A

1 2 2 3 2

P P P P P

V V R,V V

Value focus Multi-attribute assessments

M

MS

1

V

Resources focus Stakeholder analysis Analysis of options GMCR Hypergame analysis Metagame analysis Confront. analysis/drama theory Dynamic access Expected utility Transactional Vote-exchange

L C M C M C M M M M

(M)A S (M)S (M)S (M)S (M)S (M)S S A MS

2 2 2 3 2 2,5 3.5 2 2.5 2

R R R R R R R R R R

Coverage, other dimensions

V V V V, P V V (P) N,V (P) V V (P) V

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

by policy analysts requires at least as much soft skills and pragmatism as analytic and theoretic support. Nevertheless, we hope that this article can be helpful in broadening the view of policy analysts as to what methods could be considered in analyzing multi-actor policy processes, and in contributing some of the structure and overview that are necessary to enable a critical reflection on these methods, their uses and limitations. Not only will this help policy analysts in designing an actor analysis, it will also help them to assess the value of the insights gained through actor analysis, and to be more aware of blind spots that might need to be revisited in the remainder of their policy analysis processes. Appendix A This Appendix provides details underlying the assessment of the analytic quality of actor analysis methods, discussed in Section 4 of the paper. Table A.1 categorizes the concepts on each of the four identified theoretical dimensions according to their specificity, moving from more abstract to more specific concepts, together with the methods that use these concepts. For the dimension of ‘‘perceptions” two rows have been identified, to clarify that there is a general difference between methods that start from perceptions of individual actors, based on cognitive mapping, and methods that address perceptions of groups of actors in a discourse. Table A.2 provides insight into the assessment of the analytic quality of actor analysis methods. Logical interconnectedness has been assessed using a three point scale. Scores correspond to the following: M = mathematical model; C = conceptual model including (some) assumed causal relations between the concepts; L = checklists of concepts without much specifications on the relations between the concepts included in these lists. Specificity has been assessed using Table A.1, ranging from most abstract, via abstract and specific to most specific, and represents all the dimensions that are covered. Scope has been assessed by counting the number of dimensions covered (1–4). Dimensions in last two columns refer to the network (N); perceptions (P); objectives (O); or resources (R). The (P) in between brackets for some methods means that perceptions are only covered in certain specific applications (transactional process models) or in a limited way (dynamic access models; drama theory). The information in Table A.2 is graphically represented in Fig. 1 in the main text of the paper. References Ananda, J., 2007. Implementing participatory decision making in forest planning. Environmental Management 39, 534–544. Axelrod, R. (Ed.), 1976. Structure of Decision. The Cognitive Maps of Political Elites. Princeton University Press, Princeton, NJ. Bennet, P., Cropper, S., Huxham, C., 1989. Modelling interactive decisions: The hypergame focus. In: Rosenhead, J. (Ed.), Rational Analysis for a Problematic World. John Wiley & Sons, Chichester, England, pp. 283–314. Bennet, P., Bryant, J., Howard, N., 2001. Drama theory and confrontation analysis. In: Rosenhead, J., Mingers, J. (Eds.), Rational Analysis for a Problematic World Revisited. John Wiley & Sons, Chichester, England, pp. 225–248. Bots, P.W.G., Van Twist, M.J.W., Van Duin, J.H.R., 2000. Automatic pattern detection in stakeholder networks. In: Nunamaker, J.F., Sprague, R.H. (Eds.), Proceedings of the 33rd Hawaii International Conference on System Sciences. IEEE Press, Los Alamitos, CA. Brugha, R., Varvasovszky, Z., 2000. Stakeholder analysis: A review. Health Policy and Planning 15 (3), 239–246. Bryson, J.M., 2004. What to do when stakeholder matter. Stakeholder identification and analysis techniques. Public Management Review 6 (1), 21–53. Coleman, J.S., 1990. Foundations of Social Theory. The Belknap Press, Cambridge, MA. De Bruijn, J.A., Ten Heuvelhof, E.F., 2000. Networks and Decision Making. Lemma, Utrecht, The Netherlands. Dudley, G., Parsons, W., Radaelli, C.M., Sabatier, P., 2000. Symposium: Theories of the policy process. Journal of European Public Policy 7 (1), 122–140. Eden, C., Ackermann, F., 1998. Making strategy. The Journey of Strategic Management. Sage, London, England.

817

Eden, C., Ackermann, F., 2001. SODA – The principles. In: Rosenhead, J., Mingers, J. (Eds.), Rational Analysis for a Problematic World Revisited. John Wiley & Sons, Chichester, England, pp. 21–41. Fang, L., Hipel, K.W., Kilgour, D.M., 1993. Interactive Decision Making. The Graph Model for Conflict Resolution. John Wiley & Sons, New York. Fraser, N.M., Hipel, K.W., 1984. Conflict Analysis: Models and Resolutions. NorthHolland, New York. Freeman, R.E., 1984. Strategic management. A Stakeholder Approach. Pitman Publishing, Marshfield, MA. Grimble, R., Chan, M.-K., 1995. Stakeholder analysis for natural resource management in developing countries. Natural Resources Forum 19 (2), 113– 124. Hermans, L.M., 2005. Actor Analysis for Water Resources Management. Eburon, Delft, The Netherlands. Horita, M., 2000. Folding arguments: A method for representing conflicting views of a conflict. Group Decision and Negotiation 9 (1), 63–83. Howard, N., 1971. Paradoxes of Rationality: Theory of Metagames and Political Behavior. The MIT Press, Cambridge, MA. Howard, N., 1989. The manager as politician and general: The metagame approach to analysing cooperation and conflict. In: Rosenhead, J. (Ed.), Rational Analysis for a Problematic World. John Wiley & Sons, Chichester, England, pp. 239– 261. Jenkins, M., 1994. A methodology for creating and comparing strategic causal maps. Cranfield School of Management Working Paper Series, SWP 2/94, Cranfield University, Cranfield, England. Jobert, B., 1989. The normative frameworks of public policy. Political Studies 37, 376–386. Johnson, G., Scholes, K., Whittington, R., 2005. Exploring Corporate Strategy, seventh edition. Prentice-Hall Financial Times, Harlow, England. Kenis, P., Schneider, V., 1991. Policy networks and policy analysis: Scrutinizing a new analytical toolbox. In: Marin, B., Mayntz, R. (Eds.), Policy Networks. Empirical Evidence and Theoretical Considerations. Campus Verlag, Frankfurt am Main, Germany. Klijn, E.-H., 1997. Policy networks: An overview. In: Kickert, W.J.M., Klijn, E.-H., Koppenjan, J.F.M. (Eds.), Managing Complex Networks. Sage, London, England, pp. 14–34. Lindblom, C.E., 1959. The science of muddling through. Public Administration Review 19, 79–88. MacArthur, J., 1997. Stakeholder analysis in project planning: Origins, applications and refinements of the method. Project Appraisal 12 (4), 251–265. Mayer, I.S., Van Daalen, C.E., Bots, P.W.G., 2004. Perspectives on policy analyses: A framework for understanding and design. International Journal on Technology, Policy and Management 4 (2), 169–191. McKeown, B., Thomas, D., 1988. Q-Methodology. Sage, Newbury Park, CA. Mingers, J., 2003. A classification of the philosophical assumptions of management science methods. Journal of the Operational Research Society 54, 559– 570. Miser, H.J., 1985. The practice of systems analysis. In: Miser, H.J., Quade, E.S. (Eds.), Handbook of Systems Analysis: Overview of Uses, Procedures, Applications, and Practice. John Wiley & Sons, Chichester, England, pp. 281–326. Mitroff, I.I., 1983. Stakeholders of the organizational mind. Toward a New View of Organizational Policy Making. Jossey-Bass Publishers, San Francisco. ODA, 1995. Guidance note on how to do stakeholder analysis of aid projects and programmes. British Overseas Development Administration, Social Development Department, July 1995. Ostrom, E., Gardner, R., Walker, J., 1994. Rules, Games, and Common-Pool Resources. The University of Michigan Press, Ann Arbor. Popper, K.R., 1963. Conjectures and Refutations, Routledge Classics ed., 2002, The Growth of Scientific Knowledge, Routledge, London. Putnam, H., 1981. Reason. Truth and History. Cambridge University Press, Cambridge. Quade, E.S., Miser, H.J., 1985. The context, nature, and use of systems analysis. In: Miser, H.J., Quade, E.S. (Eds.), Handbook of Systems Analysis: Overview of Uses, Procedures, Applications, and Practice. John Wiley & Sons, Chichester, England, pp. 1–32. Rhodes, R.A.W., Marsh, D., 1992. New directions in the study of policy networks. European Journal of Political Research 21, 181–205. Roe, E., 1994. Narrative Policy Analysis: Theory and Practice. Duke University Press, Durham. Saaty, T.L., 1990. How to make a decision: The Analytic Hierarchy Process. European Journal of Operational Research 48, 9–26. Sabatier, P.A., 1988. An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences 21, 129–168. Sabatier, P.A., 1999. The need for better theories. In: Sabatier, P.A. (Ed.), Theories of the Policy Process. Theoretical Lenses on Public Policy. Westview Press, Boulder, CO, pp. 3–18. Scharpf, F.W., 1973. Planung als Politischer Prozess. Aufsätze zur Theorie der Planenden Demokratie. Suhrkamp Verlag, Frankfurt am Main. Scharpf, F.W., 1997. Games real actor play. Actor-Centered Institutionalism in Policy Research. Westview Press, Boulder, Colorado. Scott, J., 2000. Social Network Analysis. A Handbook, second ed. Sage, London. Shulock, N., 1999. The paradox of policy analysis: If it is not used, why do we produce so much of it. Journal of Policy Analysis and Management 18, 226– 244.

818

L.M. Hermans, W.A.H. Thissen / European Journal of Operational Research 196 (2009) 808–818

Stokman, F.N., 1994. Besluitvormingsmodellen binnen beleidsnetwerken (Decision making models in policy networks, in Dutch). In: Huberts, L.W.J.C., Kleinnijenhuis, J. (Eds.), Methoden van Invloedsanalyse. Boom, Amsterdam, The Netherlands. Stokman, F.N., Zeggelink, E., 1996. Is politics power or policy oriented? A comparative analysis of dynamic access models in policy networks. Journal of Mathematical Sociology 21 (1-2), 77–111. Termeer, C.J.A.M., 1993. Dynamiek en inertie rondom mestbeleid. Een studie naar veranderingsprocessen in het varkenshouderijnetwerk. (Dynamics and stagnation in manure policy. A study on the processes of change in the agricultural network). VUGA, The Hague, the Netherlands (in Dutch). Thissen, W.A.H., Twaalfhoven, P.G.J., 2001. Towards a conceptual structure for evaluating policy analytic activities. European Journal of Operational Research 129, 627–649. Thomson, R., Stokman, F., Torenvliet, R., 2003. Models of collective decision-making: Introduction. Rationality and Society 15 (1), 5–14. Timmermans, J.S., 2004. Purposive Interaction in Multi-Actor Decision Making. Eburon, Delft, The Netherlands.

Toulmin, S.E., 1958. The Uses of Argument. Cambridge University Press, Cambridge. Van Eeten, M.J.G., 1999. Dialogues of the Deaf. Defining New Agendas for Environmental Deadlocks. Eburon, Delft, The Netherlands. Van Eeten, M.J.G., 2001. Recasting intractable policy issues: The wider implications of The Netherlands civil aviation controversy. Journal of Policy Analysis and Management 20, 391–414. Varvasovszky, Z., Brugha, R., 2000. How to do (or not to do). A stakeholder analysis. Health Policy and Planning 15 (3), 338–345. Walker, W.E., 2000. Policy analysis: A systematic approach to supporting policy making in the public sector. Journal of Multi-Criteria Decision Analysis 9, 11– 27. Weimer, D.L., 1999. Q-method and the isms. Journal of Policy Analysis and Management 18, 426–429. Weimer, D.L., Vining, A.R., 1989. Policy Analysis: Concepts and Practice. PrenticeHall, Englewood Cliffs. Wildavsky, A., 1979. Speaking truth to power. The Art and Craft of Policy Analysis. Transaction Publishers, New Brunswick, NJ (third printing, 1992; first edition 1979).