Cognitive conflict without explicit conflict monitoring in a dynamical agent

Neural Networks 19 (2006) 1430–1436 www.elsevier.com/locate/neunet

2006 Special Issue

Cognitive conflict without explicit conflict monitoring in a dynamical agent Robert Ward a,∗ , Ronnie Ward b a Centre for Cognitive Neuroscience, University of Wales, Bangor, Bangor LL57 2AS, UK b Department of Computer Science, Texas A&M University, College Station, TX, USA

Received 9 August 2006; accepted 16 August 2006

Abstract We examine mechanisms for resolving cognitive conflict in an embodied, situated, and dynamic agent, developed through an evolutionary learning process. The agent was required to solve problems of response conflict in a dual-target “catching” task, focusing response on one of the targets while ignoring the other. Conflict in the agent was revealed at the behavioral level in terms of increased latencies to the second target. This behavioral interference was correlated to peak violations of the network’s stable state equation. At the level of the agent’s neural network, peak violations were also correlated to periods of disagreement in source inputs to the agent’s motor effectors. Despite observing conflict at these numerous levels, we did not find any explicit conflict monitoring mechanisms within the agent. We instead found evidence of a distributed conflict management system, characterized by competitive sources within the network. In contrast to the conflict monitoring hypothesis [Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652], this agent demonstrates that resolution of cognitive conflict does not require explicit conflict monitoring. We consider the implications of our results for the conflict monitoring hypothesis. c 2006 Elsevier Ltd. All rights reserved.

Keywords: Conflict monitoring; Anterior cingulate; Dynamical agents; Genetic algorithms; Artificial comparative psychology

Cognitive conflict occurs when neural pathways associated with different concurrent processes or representations interfere with one another (Botvinick, Braver, Barch, Carter, & Cohen, 2001). For example, consider a frog eyeing two flies, one to the left and one to the right. Each fly might activate a specific attacking response to the fly’s location. But if only one attack can be made at a time, the frog is in danger of some kind of incoherent response, like attacking a midpoint between the two targets. The frog therefore needs to manage the conflicting responses and focus on a single target. Conflict tasks have a long history of study in psychology, with the Stroop task being perhaps the best known, in which the identity of a colored word can interfere with color naming (Mari-Beffa, Estevez, & Danziger, 2000). How might conflict be managed? An influential proposal was developed by Botvinick et al. (2001), who suggested a two-staged “evaluate–regulate” approach to the control of conflict. Botvinick et al. hypothesized a top-down conflict monitoring system, which first detects conflict in underlying neural structures, and second, invokes control mechanisms to ∗ Corresponding author. Tel.: +44 1248 382211.

E-mail address: [email protected] (R. Ward). c 2006 Elsevier Ltd. All rights reserved. 0893-6080/$ - see front matter doi:10.1016/j.neunet.2006.08.003

regulate processing in a task-appropriate way. Botvinick et al. (2001) embedded this system within a number of discrete interactive models of conflict tasks, including the Stroop model of Cohen and Huston (1994). Conflict was measured by monitoring the Hopfield (1982) energy function in the response layer of the Stroop model. Energy increased during incongruent trials, suggesting that a potential monitoring mechanism could be activated by such a signal. Appropriate cognitive control could then be subsequently evoked. In the Botvinick et al. (2001) model, conflict monitoring was a localized function of a dedicated module. Botvinick et al. (2001) further speculated that in the human brain, the proposed conflict monitoring system could be localized to the Anterior Cingulate Cortex (ACC). In support of a relatively localized module for conflict monitoring, Botvinick et al. (2001) (see also Botvinick, Cohen, and Carter (2004)) considered a variety of evidence from brain imaging studies, showing ACC activation during conditions producing interference due to response conflict (Botvinick, Nystrom, Fissell, Carter, & Cohen, 1999; Carter et al., 2000; Casey et al., 2000). In this report we investigate other possible forms of conflict monitoring in a “minimally cognitive agent” (Beer,

R. Ward, R. Ward / Neural Networks 19 (2006) 1430–1436

2003). Beer (1996, 2003) suggests the use of an idealized “Visual Agent” (VA). Unlike connectionist approaches in which action is represented by the activation of an output node (e.g. Cohen, Dunbar, and McClelland (1990)), a VA is an Embodied, Situated and Dynamic (ESD) agent operating in continuous time. ESD agents stress what Clark (1999) calls “the unexpected intimacy between the brain, body, and world”. As model systems, they emphasize the contextually bound nature of solutions to cognitive problems, and allow a tractable analysis of the type of cognitive processing going on in more complex systems. Action in an ESD agent is significantly more sophisticated than in a disembodied network. Rather than activate a single node to represent action, ESD agents must use their effectors within the context of a perception–action loop, in which actions change perceptions, and perceptions guide action. Previous work has shown that sophisticated cognitive processes can occur even within a small network, including memory, selective attention (Slocum, Downey, & Beer, 2000), and the use of reactive inhibition (Houghton, Tipper, Weaver, & Shore, 1996) in the control of selective action (Ward & Ward, submitted for publication). We suggest that exploration of minimally cognitive agents must be valuable for psychology, as long as the agents are doing genuinely interesting tasks. Either (1) the agent will use mechanisms already described in the literature, allowing for a tractable computational analysis of those mechanisms, or (2) the agent will use some entirely novel approach to the problem, suggesting new approaches. We investigate conflict monitoring in a dual-task in which VA must select actions in the presence of stimuli suggesting conflicting responses. We used the dual-target task developed by Slocum et al. (2000), in which agents were constrained to run along the bottom of a 2D environment, moving left and right to catch two targets, T1 and T2, falling from the top of the environment. Let us briefly consider the cognitive demands of this task. First, we can see that catching a single target is not an interesting cognitive task: the sensors need only direct the motors towards the side with the greater stimulus input. In this way, the agent would track the “center of mass” of the perceptual input. However, things become much more complicated when we introduce a second target, and therefore potential response conflict: the agent can be pulled in opposite directions by the two targets. In terms of response conflict, we suggest this task has elements of both ‘undetermined responding’ and ‘response override’ (Botvinick et al., 2004). Multiple cognitive processes are required for success in this task. One of the two targets must be prioritized and selected for action. Responses must then be tied to the movement of the selected target, and insulated from the sensor activation of the other. After the first target is caught, a reallocation of processing is necessary: the second target, previously insulated from response mechanisms, must now be allowed to control them. Prioritization, selection for response control, and reconfiguration following a change of targets are all vital topics in current work on selective attention. In our analysis, we first demonstrate that our agent does indeed exhibit cognitive conflict. Periods of conflict are located using a stable-state equation, which the agent solves

1431

Fig. 1. Network layers and connections of the visual agent (VA). The top unfilled box indicates the seven-node sensor layer, which has no intra-layer connections. The middle box represents the eight-node hidden unit layer, and the lower box illustrates the two-node motor layer. The filled boxes indicate that each unit is connected to every other unit within the layer using bilaterally symmetric weights. Arrows between layers represent bilaterally symmetric connections.

during processing. These conflict periods are equated with disagreements in the source inputs to the agent’s control circuits. We then examine how the agent resolves this conflict. 1. Methods Agents were created with the same connection architecture used in the selective attention experiments of Slocum et al. (2000). Agent diameter was 30 units and target diameter was 22 units, and the environment was 400 units wide by 275 units high. The agent had 7 sensor rays of length 220 evenly spaced over a visual angle of π/6 degrees. External input magnitude varied from 0 to 10, inversely proportional to distance to an object. Seven sensor neurons (S1–S7) were connected bilaterally symmetric to eight hidden units (H1–H8) and two motor units (M1–M2). Units H1–H8 and M1–M2 were fully interconnected in a bilaterally symmetric, recurrent fashion. Units H1–H8 were also connected bilaterally symmetric to M1–M2, which in turn were recurrently connected back to H1–H8 in bilaterally symmetric fashion (see Fig. 1). Like Slocum et al. (2000) we searched for network parameters using a genetic algorithm, although in principle other unsupervised learning algorithms could also be used. In our case, the 102 network parameters were encoded for genetic algorithm search using GAlib (Wall, 1999). The agent was required to catch targets T1 and T2. After T1 impacted, it was removed from the environment so that it no longer triggered the sensors. We will call the trials used to evolve the agent the “training” trials (see Fig. 2). The experimental factors defining these trials were: (1) the side on which T1 appeared relative to the agent (Left or Right); (2) the position of T2 relative to T1 (Left or Right); (3) the spatial separations between T1 and T2 (24 units in Near; 48 in Far conditions); (4) the velocities of T1 and T2 (relative velocities 4 and 3 in Near; 5 and 2 in Far). This factorial design creates 16 trial types. For greater generalization, these 16 trials were each presented in three epochs, offset 8 units to the left, 8 to the right, and with 0 offset. After catching T1, the agent had to travel at most 55% of its maximum velocity to catch T2. In addition

1432

R. Ward, R. Ward / Neural Networks 19 (2006) 1430–1436

Conflict within VA Given that selective performance was generally very good, can we verify that there was conflict within VA, due to the presence of multiple targets? While VA is catching T1, does the presence of T2 produce conflict within VA about how to respond? We examined the peak violations of a stablestate equation the agent solves while catching targets. As VA catches a target, the network dynamics relax into an attractor state, so that as an optimization problem (see Appendix B for derivation), the agent is trying to solve: (y L − y R ) − [k(M L − M R ) + (I L − I R )] = 0,

(1)

where y L and y R represent the states values of the two motor neurons; M L and M R , their activation value; and I L and I R the sum of the other inputs to the motors (sensor and hidden inputs). From a goal perspective, VA should be least “happy” when its current position corresponds to a peak violation of Eq. (1). Peak violations are conflict periods

Fig. 2. The agent moves left and right underneath the falling targets (T1 and T2). T2 is out of VA’s field of view and the rightmost and the two leftmost and rightmost sensor rays detect nothing. The other rays intersect T1. An agent viewer and associated data files are available at http://www.psychology.bangor.ac.uk/ward.

to the Dual Target trials described above, a training epoch included Single Target trials, in which a single target appeared. Single trials were created by removing one target from the Dual set, so that there was a total of 144 trials, requiring the catching of 192 targets. The objective function used to evolve the agent minimized the average center-to-center miss distance (md) of the agent from each target at impact. 2. Results The highest performing agent (VA) had an average catch accuracy of 99.7% on the training trials, where catch accuracy reflects the degree of overlap between the target and agent, defined by Slocum et al. (2000) as: (VA radius + target radius − md)/(VA radius + target radius), with a minimum overlap of 0%. More important, how well did VA generalize to previously unseen trials using random target start positions and speeds? On 500 random Dual Target trials with T1 and T2 positions uniformly sampled from ranges [180, 212] and [132, 268], and speeds uniformly sampled from ranges [4, 5] and [2, 3], respectively, VA had an average T1 catch accuracy of 95.2%. VA never failed to catch T1, but on 36 trials, it did not catch T2. VA had an average T2 catch accuracy of 89%. Performance was close to perfect on the 1000 Single Target test trials corresponding to the 500 Dual test trials, averaging 99.8%. Overall performance on the combined 1500 trials was 95.9%. Adding a uniformly distributed noise from [−1, 1] with mean of zero to each sensor input at every timeslice did not substantially alter overall performance.

Here we describe both evidence for conflict, and also evidence that violations of the constraint Eq. (1) above mark these periods of conflict. We have elsewhere established a crucial behavioral marker for cognitive conflict in VA (Ward & Ward, submitted for publication), which we review here. We found that during the period leading up to T1 catch, VA demonstrated both an excitatory and compensating inhibitory reaction to T2. VA had a bias to move towards any activity within its sensor array, so, the more sensor activity from T2, the greater was T2 salience and its excitatory “pull”. Unopposed, this pull from T2 would result in poor catches of T1. In fact, VA generated an internal, inhibitory signal that opposed the pull from T2 salience: this meant that more salient T2s were inhibited more strongly than less salient ones. Although this inhibition was necessary to allow good processing of T1, it did exact a cost in later performance. We observed (Ward & Ward, submitted for publication) that after VA caught T1, it hesitated for a significant time before moving again to catch T2. This hesitation was an increasing function of T2 salience: after T1 catch, VA took longer to move towards more salient T2s than to less salient ones. That is, VA took longer to respond to the more strongly inhibited T2s. This pattern of “reactive inhibition” (Houghton & Tipper, 1994), in which inhibition is applied most strongly during the times of greatest conflict, is well documented in human behavioral experiments (e.g. Grison and Strayer (2001)). VA’s hesitation in reallocating from T1 to T2 is therefore a useful measure for us, as it reflects the internal response to conflict generated when VA must focus on T1 and prevent responses to T2. Given that conflict results in hesitation, we are now left with a straight-forward prediction. If violations of the constraint equation (1) result in conflict, then we should be able to observe changes in hesitation based on the violation. We tested this prediction as follows. We determined the time of peak constraint violation for a set of 1000 randomly generated trials.

R. Ward, R. Ward / Neural Networks 19 (2006) 1430–1436

Around this time of peak violation, we artificially removed any sensory activity due to T2. Four periods were tested: from the time of peak violation to 15 timeslices after the peak (peak to peak+15); from the 15 timeslices before the peak until the peak violation (peak − 15 to peak); from peak − 15 to peak + 15; and from peak + 15 to peak + 30. The results were clear. Hesitation for the baseline 1000 trials was 34.8 timeslices. Removing T2 sensory input after the peak of constraint violation had no effect on this hesitation. Hesitation was 35.2 in the peak to peak+15 condition, and 34.8 in the peak + 15 to peak + 30 condition, neither different from baseline, p > .1. However, removing T2 sensory input in the time immediately before peak violation significantly reduced hesitation, p < 0.00005, such that hesitation was 30.7 in the peak − 15 to peak, and 31.9 in the peak − 15 to peak + 15 conditions. This demonstrates that the time of peak constraint violation clearly relates to the hesitation period. Specifically, peak constraint violations mark periods of conflict within VA, reflected in the inhibition of T2 created during the hesitation period. Constraint violation and source conflict To better understand the operation of VA during periods of constraint violation, we now look in some detail at the peak constraint violations and the detailed processing of VA during a trial. Eq. (1) shows two sources: one from the motor units (the M source), consisting of their self- and cross-connections, k(M L − M R ), and another the input from the rest of the VA network (the I source), I L − I R . If one of these sources differentially activates one motor unit over the other, we will say that the source has a movement bias, and we can understand that source as acting to move the agent in a particular direction. In Fig. 3 we plot the bias of the M and I sources during a trial, as well as the distance between VA and T1, and the local peaks of constraint violation (labeled A, B, C, D; A represents the global peak). T1 starts just right of CVA at 203, speed 4.15, and T2 is Far left at 156, speed 2.2. Positive deflections represent a bias to move right, negative a leftward bias. Up until time 200 the sources hold the agent relatively immobile, as the targets fall closer. Shortly after this time, the I source biases VA to move right, underneath T1, slightly reducing their separation. Around time 300, the M source develops a rightward bias. However, this bias is opposed by an even stronger bias from the I source, pushing VA away from T1 (as evidenced by the increasing separation) and towards T2. The largest peak violation corresponds to (A), at time 368, when VA reaches its maximum separation from T1. Around time 400, VA appears to pause between the targets, evidenced by the flattening of the separation curve, and the relaxing of both sources. At time 433 another conflict (B) peaks as the I source signals a move right in opposition to the left signal from the motors. A smaller competition occurs around time 467 (C) as VA turns back toward T1, and the last significant competition (D) occurs at 492 as T2 passes out of VA’s sensor array while VA is in the final stages of catching T1. At source conflict (A), the resolution arose because of a change in a single hidden unit (no. 7), as its activation fell from

1433

Fig. 3. Conflict of Motors and Inputs (I ) sources within VA. The VA–T1 separation shows the distance between VA and T1. Positive bias indicates VA is to the right of T1. Both the Motors and Inputs sources are biasing VA movement direction. Positive-going lines on the Y -axis indicate a bias to move VA to the right; negative-going, to the left. Note that the two biases are consistently opposed throughout the trial. Local peaks of source disagreement are marked A–D, and coincide closely to violations of the optimization equation describing attractor states, as described in the text.

0.95 to 0.01. All other hidden unit activations were unchanged during the conflict period indicated at (A). However, at the other conflict points, it was not unit 7 that changed. During conflict (C), hidden unit 8 deactivated while the other units remained unchanged. During conflicts (B) and (D), there was no change in hidden unit activation. No part of the VA network was uniquely activated during all conflict points. This suggests some means of distributed conflict management, in which a variety of units participate in regulating conflicts within VA. This review highlights several important aspects of VA processing. Throughout the trial, M and I sources are in opposition. VA’s behavior at any moment in time is determined by the balance between the opposing response tendencies. When the disagreement between these opposing sources grows large, VA’s action changes. Local peaks of source disagreement correspond both to local peaks of constraint violation from Eq. (1), and to changes in VA’s direction of movement. Thus, VA illustrates multiple overlapping, but distinct, notions of conflict. First, conflict at the behavioral level of response selection: which of the two objects will control VA’s behavior at any given time? Specifically, which object will VA move towards? When it is impossible to generate a move towards both targets at the same time, any agent faces the problem of selecting a single response in an intelligent manner that allows the task to be completed. Conflict at this level in VA is realized in terms of interference, in this case measured as hesitation to respond to T2. A second notion of conflict is based on the “neural”, or perhaps more exactly, the “sub-behavioral” level: that is, on the competing, opposing sources that regulate and control VA’s behavior. For VA, the M and I sources were constantly opposed to each other. In one sense this opposition is a form of conflict, for as we have shown, the different sources had different biases about the best way for VA to move. However, inherent in this form of conflict is also the notion of regulation via opponent processes. In fact, when opposing sources are precisely balanced, the optimization Eq. (1) is best satisfied. Thus, disagreement among processing sources within

1434

R. Ward, R. Ward / Neural Networks 19 (2006) 1430–1436

Table 1 Internal structure of VA Groupings

H1–H4

H5–H8

M1

M2

S1–S4 (L) S4–S7 (R) H1–H4 (R) H5–H8 (L) M1 (R) M2 (L)

I E E I I E

E I I E E I

I E E I I I

E I I E I I

Shows the connectivity between different functional groupings within the VA network. I = inhibitory connections; E = excitatory.

an agent does not necessarily indicate that the agent is in an unstable state of “conflict” that needs resolution of some kind. This disagreement can also reflect the balanced operation of opposing responses tendencies that leads to an attractor state. Internal structure within VA We also examined the internal structure for evidence of explicit conflict monitoring. Instead we found a competitive internal structure, suggesting an architecture of “checks and balances”, consistent with the opposition of sources leading to stable states within VA. Consider Table 1, which shows the partitioning of units in VA, and the valence of intra- and inter-layer connections. In the leftmost column, the neural units termed Groupings (S, sensors; H, hidden units; M, motors) are given along with their associated movement bias (L, left; R, right). Entries in the table indicate the connection relationships (I, inhibitory; E, excitatory) between groupings. For simplicity, the middle sensor (S4) is grouped with both left/right sensor groups. To read the table, consider the first row. When sensors S1–S4 are activating, they tend to move the agent left by exciting H5–H8 and M2 whose activation also moves the agent left. However, S1–S4 inhibits H1–H4 and M1 because activation of these units tends to move the agent right. With only feed-forward links, the sensor groupings appear to have a “reactive” structure. That is, they excite or inhibit the hidden units and motors based only on their movement direction. However, the hidden unit groups have a “competitive” structure. The groupings are self-exciting and mutually inhibitory. Like the sensors, they also excite or inhibit the motors based on preferred movement direction. Most interesting is the motor activation effect, which inhibits both motors. This tends to stop movement! Moreover, motor activation inhibits the same-direction hidden-unit group, but excites the opposite-direction hidden-unit group. This interlayer competition seems strange, but it acts to turn the agent in the opposite direction. For example, in Fig. 3 around time 300 when the hidden units H5–H8 are acting to move the agent left by activating M2, M2 is activating to shut down H5–H8 and excite the opposite group H1–H4 to turn the agent around. Finally, we note that in the conflict monitoring model of Botvinick et al. (2001), the conflict monitoring module has reciprocal connections with the rest of the network: afferent connections to monitor network states, and efferent to exert appropriate control. As reported earlier, we did not see any

evidence of dedicated conflict monitoring units within VA’s hidden layer, nor in the competitive structure shown in Table 1. Yet it is the case that there are reciprocal connections between the motor and hidden layer of VA (see Fig. 1), so we investigated what effect these links serve, and whether there may be explicit conflict monitoring somehow in place here. If the back-connections from the motors to the hidden units are lesioned, T2 catch accuracy falls drastically, from 89% with these links active to 61.3% without them. These lesion data show that during T1 processing the agent gathers information to correctly process T2 after T1 catch, and that the motor feedback links play an important role in this function. However, there was no evidence that these links specifically supported resolution of response conflict between T1 and T2. T1 catch accuracy fell only very slightly, to 93.6% from 95.2%. Therefore, even without the back-connections, VA still manages T2 distraction to successfully select and catch T1. 3. Discussion We have shown that VA can address the demands of conflicting responses in a dual-target task, through a distributed system of control and the dynamics of competing activation sources within VA. We want to re-emphasize the interesting cognitive nature of the dual-target task, requiring selective action and reallocation between targets. Further, unlike previous connectionist models used to model Stroop and flanker tasks, VA is embedded within a genuine perception–action cycle, in which its actions affect its perceptions, and perceptions affect action. We are not aware of a model of perception–action within the psychological literature that has been developed with these qualities of VA. We began this detailed investigation of VA with the idea that there would necessarily be value in this form of “artificial comparative psychology”, that we would either find a tractable system that supports current ideas about conflict monitoring and resolution (e.g. Botvinick et al. (2001, 2004)), or we would find working, novel mechanisms that suggest alternative approaches to empirical and computational work. So where then do we stand? 3.1. Conflict monitoring does not require modules We submitted a generic network architecture to the genetic algorithm. There was no module for either conflict detection or regulation built in to the VA architecture. We allowed the computational properties of the catching task shape the solution, and we did not observe any set of units dedicated to monitoring or regulation. Instead we found a system that used competitive balancing of response biases, and distributed conflict management. This does not mean that other types of systems might use a dedicated module for conflict detection, but it does demonstrate that a model system operating within a genuine perception–action cycle, and facing real problems of response conflict (and solving them!) does not need this kind of dedicated module.

R. Ward, R. Ward / Neural Networks 19 (2006) 1430–1436

3.2. Conflict as disagreement and as imbalance In the behavioral literature, response conflict is discussed in terms of limitations on response systems to make conflicting actions: conflict arises when activated responses “disagree” about the action to be made (Eimer, Hommel, & Prinz, 1995). VA’s performance in the catching task is a good example of this kind of response conflict. The agent cannot move left and right simultaneously, and must choose which of the competing responses is to be made. However, VA also shows how conflict at the underlying “neural” level can be both related to, and dissociated from, conflict at the behavioral level. The inputs and motor sources of VA typically bias opposing responses (Fig. 3). These sources conflict in the sense that they bias opposing behaviors, but this does not necessarily result in conflict at the behavioral level. This is apparent in VA’s optimization equation: if the two sources are biasing opposite responses, but these biases are balanced, the agent is near an attractor state. This is true regardless of the magnitude of disagreement between the sources. What is crucial at this level is not disagreement per se, but the degree to which opposing, competitive sources are balanced or imbalanced. 3.3. The conflict monitoring hypothesis Earlier we reviewed the highly influential conflict monitoring hypothesis of Botvinick et al. (2001), in which the Anterior Cingulate Cortex (ACC) monitors for signs of response conflict throughout the brain, and responds to conflict by triggering strategic adjustments in cognitive control. A detailed review is not appropriate here (for a recent analysis see Botvinick et al. (2004)), and no doubt there are further investigations to come, but on the empirical side, agreement is good that the ACC is activated during conflict tasks, and that activation generally tracks the observed behavioral interference (Botvinick et al., 1999; van Veen, Cohen, Botvinick, Stenger, & Carter, 2001). ACC activation may not be limited solely to the effects of response conflict, since during response conflict tasks, ACC activation also appears increased by other cognitive demands, such as subgoaling (Badre & Wagner, 2004). The current controversy regarding the conflict monitoring hypothesis is therefore not an empirical dispute about whether the ACC is active during conflict, but instead revolves around the causal role of this ACC activation in the resolution of response conflict. Consistent with a non-causal role in response conflict, neuropsychological patients with damage to the ACC have been shown to have intact performance in response conflict tasks (Fellows & Farah, 2005). ACC activation might therefore represent a correlate of performance in response conflict tasks, such as autonomic arousal (Critchley et al., 2003), or effort (Mulert, Menzinger, Leicht, Pogarell, & Hegerl, 2005). Our results with VA, and its success in resolving cognitive conflict without explicit conflict monitoring, lead us to likewise question the causal link between ACC activation and the resolution of response conflict. However, recent work investigating temporal brain dynamics (Crottaz-Herbette & Menon, 2006) shows some interesting

1435

similarities to the patterns of competing sources we’ve seen in VA. Crottaz-Herbette and Menon found that, compared to frequently presented stimuli, infrequent “odd-ball” items activated a network involving the ACC and modality-specific sensory cortex. Since responses to infrequent items require the override of a more dominant response set, Crottaz-Herbette and Menon argue that this task is an example of a response conflict task. Recording of Evoked Response Potentials (ERP) showed latencies and topographies suggesting a dynamic and re-entrant interaction of signals from primary sensory cortices and higher cortex. There may be an important similarity here between the notion of dynamic interaction of signals in the Crottaz–Herbette model and VA. Rather than a specific “conflict monitor”, the ACC may therefore be one of many sites affording the type of conflict resolution we see within VA, competition between sources which are biasing different actions. This would include sensory sources, feedback from motor areas, and higher-level executive areas. This would mean that ACC activity would be related but not specific to, situations of high conflict, as well as high effort and errors. Appendix A The GAlib (Wall, 1999) parameters were set as follows. The algorithm used was GASteadyState with a population sizes from 25 to 300, replacement percentage of 50–75%, crossover probability of 96%, and mutation probability of 10%. The GARealGenome (real-valued vector genome) was used with allele sets for the 102 network parameters in the following ranges: weights [−10, 10], time constants [1, 2], sensor ray biases [−10, 10], hidden unit and motor biases [−5, 5], gains [1, 5] and motor gains were set to 1.0. The default random number generator in GAlib was used. Appendix B Recall that the agent’s final behavior is determined at the motor unit level by integrating the non-linear inputs from the sensor neurons, hidden units and the recurrent motor connections. Rewriting the non-linear CTRNN state equations given in Beer (1996) yields the following left and right (L , R) motor neuron state equations: τ · y L0 + y L = α · M L + β · M R + HL + SL τ · y 0R + y R = α · M R + β · M L + H R + S R .

(B1)

The time constant τ self-weight α and cross-weight β are assumed to be the same for both motor neurons; y L and y R are the state values and y L0 and y 0R are time-based derivatives. We define M L = σ (g(y L + θ )) and M R = σ (g(y R + θ )), where σ (X ) = 1+e1−X . The gain g and the bias term θ are the same for P P each motor neuron. HL = j w j L a j and H R = j w j Ra j are the summed, weighted inputs to each motor from the hidden units. SL and S R are similarly defined for the sensor neurons, where w j L and w j R are the weights from neuron j to L , R and a j = σ (g j (y j + θ j )) is the activation of neuron j feeding input to the motor neurons. Each a j has gain and bias terms g j and θ j .

1436

R. Ward, R. Ward / Neural Networks 19 (2006) 1430–1436

CVA calculates a horizontal move velocity for each Eulerintegration step defined as: V = (M L − M R )/c,

(B2)

where M L and M R are in the interval [0,1], and c is a constant. Beer (1996) selected c = 0.2 so that 1.0/c = 5.0 units per time step is the maximum velocity of the agent. V < 0 moves the agent left, V > 0 moves the agent right. If V = 0 the agent remains motionless and M L = M R . The CVA catch behavior is observed as a sequence of moves back and forth underneath a target T , using any difference in motor outputs to cause movement until T impacts. To isolate (M L − M R ) in (B2), subtract Eq. (B1) to get the following “difference” equation, where the time constant τ is assumed to be 1.0, and I L = HL + SL and I R = H R + S R : (y L0 − y 0R ) + (y L − y R ) = k(M L − M R ) + (I L − I R ).

(B3)

Eq. (B3) has the form of a single neuron state equation z = y L − y R with a self-connection weight k = (α − β) 6= 0 and an activation function whose output (M L − M R ) is in the interval [−1, 1]. The terms on the Right-Hand Side (RHS) govern the state update for z to determine a new output value for (M L − M R ). As CVA catches a target the network dynamics relax into a stable state, such that y L0 and y 0R in (B3) become zero. Thus, as an optimization problem, the agent moves to solve: (y L − y R ) − [k(M L − M R ) + I L − I R ] = 0.

References Badre, D., & Wagner, A. D. (2004). Selection, integration, and conflict monitoring; assessing the nature and generality of prefrontal cognitive control mechanisms. Neuron, 41(3), 473–487. Beer, R. D. (1996). Toward the evolution of dynamical neural networks for minimally cognitive behavior. In Paper presented at the From animals to animats 4: Proceedings of the fourth international conference on simulation of adaptive behavior. Beer, R. D. (2003). The dynamics of active categorical perception in an evolved model agent. Adaptive Behavior, 11, 209–243. Botvinick, M., Nystrom, L. E., Fissell, K., Carter, C. S., & Cohen, J. D. (1999). Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature, 402(6758), 179–181. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Science, 8(12), 539–546. Carter, C. S., Macdonald, A. M., Botvinick, M., Ross, L. L., Stenger, V. A., Noll, D., et al. (2000). Parsing executive processes: strategic vs. evaluative functions of the anterior cingulate cortex. Proceedings of the National

Academy of Sciences of the United States of America, 97(4), 1944–1948. Casey, B. J., Thomas, K. M., Welsh, T. F., Badgaiyan, R. D., Eccard, C. H., Jennings, J. R., et al. (2000). Dissociation of response conflict, attentional selection, and expectancy with functional magnetic resonance imaging. Proceedings of the National Academy of Sciences of the United States of America, 97(15), 8728–8733. Clark, A. (1999). An embodied cognitive science? Trends in Cognitive Science, 3(9), 345–351. Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97(3), 332–361. Cohen, J. D., & Huston, T. A. (1994). Progress in the use of interactive models for understanding attention and performance. In C. Umilta, & M. Moscovitch (Eds.), Attention and performance: XV. Conscious and nonconscious information processing (pp. 453–476). Cambridge, MA: MIT Press. Critchley, H. D., Mathias, C. J., Josephs, O., O’Doherty, J., Zanini, S., Dewar, B. K., et al. (2003). Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. Brain, 126(10), 2139–2152. Crottaz-Herbette, S., & Menon, V. (2006). Where and when the anterior cingulate cortex modulates attentional response: combined fMRI and ERP evidence. Journal of Cognitive Neuroscience, 18(5), 766–780. Eimer, M., Hommel, B., & Prinz, W. (1995). S–R compatibility and response selection. Acta Psychologia, 90, 301–303. Fellows, L. K., & Farah, M. J. (2005). Is anterior cingulate cortex necessary for cognitive control? Brain, 128(4), 788–796. Grison, S., & Strayer, D. L. (2001). Negative priming and perceptual fluency: more than what meets the eye. Perception & Psychophysics, 63(6), 1063–1071. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2558. Houghton, G., & Tipper, S. P. (1994). A model of inhibitory mechanisms in selective attention. In D. Dagenbach, & T. Carr (Eds.), Inhibitory mechanisms in attention, memory, and language. San Diego, CA: Academic Press. Houghton, G., Tipper, S. P., Weaver, B., & Shore, D. I. (1996). Inhibition and interference in selective attention: Some tests of a neural network model. Visual Cognition, 3, 119–164. Mari-Beffa, P., Estevez, A. F., & Danziger, S. (2000). Stroop interference and negative priming: problems with inferences from null results. Psychonomic Bulletin & Review, 7(3), 499–503. Mulert, C., Menzinger, E., Leicht, G., Pogarell, O., & Hegerl, U. (2005). Evidence for a close relationship between conscious effort and anterior cingulate cortex activity. International Journal of Psychophysiology, 56(1), 65–80. Slocum, A. C., Downey, D. C., & Beer, R. D. (2000). Further experiments in the evolution of minimally cognitive behavior: From perceiving affordances to selective attention. In Paper presented at the From animals to animats: Sixth international conference on simulation of adaptive behavior. van Veen, V., Cohen, J. D., Botvinick, M. M., Stenger, V. A., & Carter, C. S. (2001). Anterior cingulate cortex, conflict monitoring, and levels of processing. Neuroimage, 14(6), 1302–1308. Wall, M. (1999). GAlib, a C++ library of genetic algorithm components. http://lancet.mit.edu/ga/ Retrieved 17.02.2003. Ward, R., & Ward, R. Selective attention and control of action: Comparative psychology of an artificial, evolved agent and people (submitted for publication).