Choosing the speed of dynamic mental simulations

ARTICLE IN PRESS

Choosing the speed of dynamic mental simulations Alexis D.J. Makin1 University of Liverpool, Liverpool, United Kingdom Corresponding author: Tel.: +44-0151-7943909; Fax: +44-0151-7943909, e-mail address: [email protected]

1

Abstract The brain continuously maintains a current representation of its immediate surroundings. Perceptual representations are often updated when the world changes, e.g., when we notice an object move. However, we can also update representations internally, without incoming signals from the senses. In other words, we can run internal simulations of dynamic events. This ability is evident during mental object rotation. These uncontroversial observations lead to an obvious question that nevertheless remains to be answered: How does the brain control the speed of dynamic mental simulations? Is there a central rate controller or pacemaker module in the brain that can be temporarily coupled to sensory maps? We can refer to this as the common rate control theory. Alternatively, the primitive intelligence within each map could tune into the speed of recent changes and use this information to keep going after stimuli disappear. We can call this the separate rate control theory. Preliminary evidence from prediction motion experiments supports common rate control, although local predictive mechanisms may cover short gaps of <200 ms. The putative rate controller might turn out to be the same thing as the velocity store described in smooth pursuit eye movement models and/or the pacemaker module proposed in the cognitive timing literature. Indirect neuroimaging evidence suggests rate control is a function of the core timing system in the dorsal striatum.

Keywords Motion extrapolation, Time to contact, Prediction motion, Mental object rotation, Smooth pursuit eye movements, Internal clock

1 INTRODUCTION Pinker (1997) supported the representational–computational account of the brain in his popular book “How the mind works.” Representations are internal states in a computing system which systematically code features of the world and which can be read by other parts of the system. All mental processes involve the purposeful Progress in Brain Research, ISSN 0079-6123, http://dx.doi.org/10.1016/bs.pbr.2017.05.001 © 2017 Elsevier B.V. All rights reserved.

1

ARTICLE IN PRESS 2

Choosing the speed of dynamic mental simulations

manipulation of representations. This version of cognitive science also proposes that the brain is divided into functionally specialized subsystems, which are sometimes called cognitive modules or cognitive mechanisms. These are like little neural gadgets, which are part of our genetic endowment. Each module grows during development and serves an adaptive function. The philosopher Daniel Dennett approvingly quotes a summary of his ideas which appeared in an Italian magazine: “Sı`, abbiamo un’anima. Ma e` fatta di tanti piccoli robot (Yes, we have a soul, but it’s made of lots of tiny robots).” Pinker and Dennett would claim that cognitive scientists should discover the nature of the tiny robots and how they interact to produce intelligent behavior. We can estimate that there are about 100 Billion neurons in the brain. But how many discrete cognitive modules do we have? Are there 500? Are there a million? There is no textbook answer here because it trivially depends on how we define a cognitive module. While the word “atom” has a precise meaning in chemistry and the word “species” has a precise meaning in biology, words like “cognitive mechanism” and “cognitive module” have no equivalent precise meaning in psychology. In fact, there is deep theoretical disagreement about the extent and nature of modularity. Different propositions can be traced back to the philosophical dichotomy between empiricism and rationalism, and the same arguments resurface today in disagreements about behaviorism and connectionism (Gopnik, 2009; Mitchell et al., 2009). Maybe the brain is like a big toolbox of specialized gadgets, each purpose-built for a unique cognitive job? Alternatively, complex behavior could emerge from the self-organization of blank-slate neural networks exposed an orderly and structured world (see Kriegeskorte, 2015 for a review of recent developments). Many researchers agree there is at least some degree of modularity in the mature brain. For example, the brain certainly has discrete visual, language, and motor systems. Within these, special subsystems mediate color and motion perception, left arm and right arm control, language comprehension and production. Brain damage has specific rather than general effects. On the other hand, there is strong evidence local networks can be functionally combined into temporary neural ensembles. This allows for a combinatorial explosion of representational states: We can conceive of a blue-moving-car or an unfamiliar-high-pitched-red-parrot. These distributed representations may be the engrams of short and long-term memory (Fries et al., 2007; Fuster, 2009). Furthermore, even our most “innate” faculties are often shaped by the environment during critical windows of development (Pinker, 1994). Given this nuanced picture of the brain, it is unsurprising cognitive scientists often argue about modularity. When should two putative cognitive mechanisms be reconceptualized as just one? Conversely, when should we reconceptualize a putative single mechanism as made from fundamentally different parts? These are often the most fundamental debates in psychology. We can use models of human memory to illustrate here. Is there a difference between long-term and

ARTICLE IN PRESS 1 Introduction

short-term memory? Is there a meaningful difference between semantic, episodic, and procedural memory? Is there a difference between verbal and visual short-term memory? Is memory for sparrows cognitively different to our memory for starlings? The major distinctions surely carve nature at its joints, but it would be a profound mistake to propose a brand new memory system for everything that can be remembered. Modularity appears in emotion research too. The core emotion account states that there are multiple, discrete emotions, each associated with specific neural, endocrine, and cardiovascular reactions (e.g., amygdala activation, adrenaline release, tachycardia). Each emotion has a distinct facial expression (e.g., smiling, snarling, or frowning), and each promotes a stereotypical behavior (e.g., running, shouting, or fighting). Core emotions supposedly evolved because our ancestors were repeatedly faced with the same set of challenging physical and social scenarios (e.g., frustration, danger, opportunity). There may be 15 core human emotions: amusement, anger, contempt, contentment, disgust, embarrassment, excitement, fear, guilt, pride, relief, sadness, satisfaction, sensory pleasure, and shame (Ekman, 1999). These are supposedly distinct, brief, and automatic reactions to specific events, which are universal across human cultures (Sauter et al., 2010). The core emotion account can be contrasted with the alternative dimensional account. Human emotional experience varies in terms of arousal (low to high), valence (positive to negative), and action inclination (avoid to approach). The dimensional account holds that each emotional state is a point in that 3D space, and there are a vast number of continuous emotional experiences. Mauss and Robinson (2009) argued that there is slightly more evidence for the dimensional account. However, debates about modularity are clearly fundamental, and unresolved, in modern emotion research as well. Let us look at a final example. Animals can judge the duration of sensory events. It has been suggested that there is an “internal clock” in the brain which explains this ability. The scalar expectancy theory describes a clock with pacemaker, switch, accumulator and memory components (Gibbon et al., 1984; Wearden et al., 1998). The pacemaker emits ticks or pulses at regular intervals. When time is task relevant, a switch closes, and ticks are passed to the accumulator. The number of accumulated ticks is our raw representation of elapsed duration. The value in the accumulator can be contrasted with other values held in temporal reference memory. Despite much research, we still do not know whether the same internal clock is used for estimating duration in different modalities. For instance, is the same clock used to estimate the duration of sounds and lights? Some researchers favor the core timing theory and argue that there is a single, multimodal clock (Coull et al., 2011; van Rijn and Taatgen, 2008). Others argue that there are many separate timing systems in the brain (Burr et al., 2007; Johnston et al., 2006). Again, we see arguments about modularity are hard to resolve, and eminent researchers often take strongly opposing positions.

3

ARTICLE IN PRESS 4

Choosing the speed of dynamic mental simulations

It seems unlikely that all modularity arguments can be swept up with a single theoretical insight. Generally, most would agree that true unification is always desirable. For instance, it would be a satisfying breakthrough if we realized two forms of memory were really the same “thing”. However, premature and forced unification claims could be misleading. Therefore, best approach is for researchers to remain vigilant for ripe unification opportunities, but simultaneously open-minded about the possibility that our pet mechanism might not perform every function we assume it performs. In the following sections, I will review my recent attempts to characterize the cognitive mechanisms which control the speed of mental updating. I have tried to unify as much research as possible under the new “common rate control theory” (Makin and Bertamini, 2014; Makin and Chauhan, 2014). It is instructive to see the challenges faced by this unification project, the kind of experiments which provide evidence, and its potential for simplifying a diverse literature.

2 HOW DO WE UPDATE MENTAL SIMULATIONS AT THE RIGHT SPEED? The brain strives to maintain a current representation of its immediate environment. For example, we keep track of the position of moving vehicles and people when we walk about town. Footballers can track the position of the ball and other players. We always track our current internal physiological states and muscle positions. All our representations are continuously updated by new sensory input (e.g., Clark, 2013). I assume this is uncontroversial so far. The sensory parts of the brain are organized into maps. There is a retinotopic map of space in the primary visual cortex, where adjacent neurons have adjacent receptive fields. There are other representations of space in the parietal lobes. There are orientation maps in V1, there are color maps in V4, and there are tonotopic pitch maps in the auditory cortex (Op de Beeck et al., 2008). If a feature of the world changes smoothly and continuously, there will be smooth change in the neurons firing within the appropriate sensory map. When we say, “the brain continuously updates mental representations,” we often mean that the neurons coding each successive state reach maximum spike rate in turn. For illustration, imagine a static patch changing color from saturated yellow to saturated blue. In the external world there is smooth motion though color space (Blaser et al., 2000; Howard and Holcombe, 2008; Sheth et al., 2000). In color-sensitive V4, there is also a smooth shift: first cells sensitive to saturated yellow fire, then cells sensitive to less saturated yellow fire, then gray, then light blue, then saturated blue. The brain tirelessly tracks the changing world in this way. Interestingly, the brain can also run a continuous dynamic simulation of a changing world even in the absence of sensory input. Shepard and Metzler (1971) famously found that the time required to compare two static 3D structures

ARTICLE IN PRESS 2 How do we update mental simulations at the right speed?

increased linearly with their rotational offset. For example, two shapes separated by 45 degrees take less time to compare than two separated by 90 degrees. Shepard and Metzler (1971) claimed that participants were completing their task by mentally rotating one of the shapes until it overlapped with the other. Even if we hesitate when using terms like “imagination” and “mental imagery” (Pylyshyn, 2003), it is plausible that the brain moves through a sequence of representational states, each coding the object at a more advanced rotation angle than the last. The task in mental object rotation experiments is simply to mentally rotate as fast as possible. However, it is likely the speed of mental updating is a flexible internal parameter (de’Sperati, 2003). So, how do we control the speed of mental updating? Is there a single pacemaker, or “rate controller” in the brain, which can be temporarily coupled to different maps? Or is predictive local circuitry within each map sufficient for internal updating (Fig. 1)? We refer to these ideas as the common rate control theory (CRC) and separate rate control theory (SRC) (Makin and Bertamini, 2014; Makin and Chauhan, 2014). The CRC vs SRC distinction is a microcosm for the study of modularity in the mind. Both theories are fundamentally different, and both plausible a priori. Perhaps insights obtained here will have general implications? Unfortunately, it is difficult to conceive an experimental procedure which absolutely demands that the participant engage in rate controlled mental updating. Often laboratory-based tasks can be cracked with multiple strategies. Therefore, we cannot probe the CRC and SRC as directly and cleanly as we would like. However, one candidate experimental paradigm which can be used here is called the prediction motion task (Fig. 2). In the next section, I will briefly review the rather fragmented prediction motion literature and demonstrate how CRC and SRC might explain performance in these tasks.

Local predictive circuits Rate controller

+10+9+8+7…n

+10+9+8+7…n

FIG. 1 A cartoon of the common rate control theory is shown on the left, and cartoon of the separate rate control theory is shown on the right.

5

ARTICLE IN PRESS Choosing the speed of dynamic mental simulations

A Production tasks Occluder Moving object

Press here!

B Interruption paradigms

Did object reappear too early or too late?

C Production task performance metrics

5000 CTE (ms)

CTE = Completion time estimate VE = Variable error

N trials

Occluder

3000 2000

Num

1000

Pos 0

VE CTE

CTE

4000

0

Error VE (ms)

6

1000 2000 3000 4000 5000 Occlusion duration (ms)

1800 1600 1400 1200 1000 800 600 400 200 0

Num Pos 0

1000 2000 3000 4000 5000 Occlusion duration (ms)

FIG. 2 Prediction motion task response protocols and metrics. (A) Production tasks. Participants see a moving target travel behind an occluder and press when they think it would have reached the other side. (B) Interruption paradigms. Here the target disappears and reappears, and there is no actual occluding object. Reappearance may be too early, on-time, or too late, and participants discriminate reappearance error. (C) Performance metrics from a production task. On each trial, we can obtain completion time estimate (CTE, duration from occlusion onset to button press) and error (CTE—perfect CTE). If there are multiple trials in a condition, we can also obtain variable error (VE), which is the standard deviation of CTEs. CTE and VE increase with occlusion duration. We have often found that the slopes are similar in different kinds of feature space (Fig. 3). Example from Makin, A.D.J., Chauhan, T., 2014. Memory-guided tracking through physical space and feature space. J. Vis. 14 (13).

3 PREDICTION MOTION TASKS Prediction motion tasks are sometimes called “time-to-contact tasks,” “arrival time estimation tasks,” or “motion extrapolation tasks.” However, “prediction motion” is the best name because it avoids unwarranted theoretical assumptions about the

ARTICLE IN PRESS 3 Prediction motion tasks A

Position

Participant presses when they think target has arrived at the end of the track B

Number 10.00 7.60 5.20

Participant presses when they think the hidden counter has reached 0 C

Color

Participant presses when they think hidden color matches the background color

FIG. 3 Three kinds of prediction motion task. (A) In the standard “position task,” the participant sees a target move along a track, then disappear. The task is to press when it reaches the end of the track. (B) Number task. The counter rapidly decrements toward zero (smooth motion through number space, along a mental number line), then disappears. The task is to press when the hidden counter reaches zero. (C) Color task. The central disk moves smoothly through color space, becoming more and more like the background. The disk then disappears, and participants assume ongoing, hidden motion through color space. The task is to press when the hidden color matches the background.

7

ARTICLE IN PRESS 8

Choosing the speed of dynamic mental simulations

underlying mechanisms. There are two basic types of prediction motion tasks with different response protocols, called “production tasks” and “interruption paradigms” (Fig. 2A and B). In production tasks (Fig. 2A) participants see a moving target disappear behind an occluder and produce a button press response when they think it would reappear on the other side (Battaglini et al., 2015; DeLucia and Liddell, 1998; Makin et al., 2008; Peterken et al., 1991; Rosenbaum, 1975). People can press at approximately the correct time, so there is a linear relationship between completion time estimates (CTEs) and occlusion duration (Benguigui et al., 2004; Tresilian, 1995). There is also a linear relationship between the standard deviation of CTEs and occlusion duration (Makin and Bertamini, 2014; Makin and Chauhan, 2014). This metric is called variable error (VE). Fig. 2C shows CTE and VE metrics from production tasks. In interruption paradigms (Fig. 2B) participants see a moving target disappear and then reappear on the other side of the occluder. Reappearance error is varied on each trial, and participants discriminate whether the target reappeared too early or too late (Bennett and Benguigui, 2013; DeLucia and Liddell, 1998; Jonikaitis et al., 2009; Makin and Poliakoff, 2011; Makin et al., 2008). Early prediction motion studies simply measured the effects of target speed, distance, and duration parameters on performance (Gottsdanker, 1952; Slater-Hammel, 1955; Wiener, 1962) but were often silent about the underlying cognitive mechanisms. Conversely, many modern prediction motion studies are based on premature assumptions about the cognitive mechanisms that mediate performance. For example, it has been assumed that these tasks can straightforwardly probe mental imagery (Gilden et al., 1995), perception of looming motion (Vagnoni et al., 2012), or “purely perceptual” prediction (Roth et al., 2013). However, there are several candidate mechanisms which might mediate performance in prediction motion tasks. One key distinction is between the clocking and tracking strategies (DeLucia and Liddell, 1998; Tresilian, 1995). If people use the tracking strategy, they follow the visible moving target with the eyes or with the spotlight of visuospatial attention (c.f. Posner, 1980) and then continue to track as well as possible across the occluder. They press when the spatial attention arrives at the end of the occluder. If people use the alternative clocking strategy, they estimate time to contact just before occlusion (perhaps by picking up optic invariants; Gibson, 1979; Lee, 1976; Schiff and Detwiler, 1979; Schiff and Oldak, 1990), then use this temporal estimate to delay the motor response (Tresilian, 1995). Participants might use clocking in production tasks (Benguigui and Bennett, 2010), but the clocking strategy is not viable in interruption paradigms. Here it is not possible to estimate TTC at occlusion onset, and there is good evidence that people track the hidden target with their eyes if allowed to do so (Jonikaitis et al., 2009; Lyon and Waag, 1995). The tracking strategy is thus more versatile than clocking (DeLucia and Liddell, 1998). We certainly cannot ignore the clocking strategy in production tasks, but we must put it aside for now to return to the CRC vs SRC distinction. Let us assume that participants maintain a representation of occluded target position and update that representation at the appropriate speed. People can track visible targets very

ARTICLE IN PRESS 3 Prediction motion tasks

well with pursuit eye movements and can also track occluded targets approximately with a mixture of low gain pursuit and catch-up saccades (Bennett and Barnes, 2003; Lencer et al., 2004). Catch-up saccades are usually initiated when the eye falls behind the position of the occluded target. This demonstrates that the eyes are guided by a dynamic internal representation of target position (Bennett and Barnes, 2006a). Makin and Bertamini (2014) and Makin and Chauhan (2014) experimented with prediction motion tasks in feature space. Fig. 3 shows examples of position, number, and color tasks. In all cases, there was a dynamic process on the screen, which then disappeared. Participants assumed ongoing, hidden change, and pressed when the hidden process would be complete. For example, in the number task, a central counter counts down toward zero, then disappears midway. Participants assume that the hidden counter carries on counting down at the same speed, and press when they estimate it would have reached zero. In all tasks, the participants might update a mental representation to track the state of the hidden dynamic process. It could be that a common rate controller guides updating on all tasks, or it could be that separate rate controllers mediate updating in each. Makin and Bertamini (2014) and Makin and Chauhan (2014) both argued in favor of the common rate control hypothesis. They found that the slope of the occlusion duration vs CTE slopes was often similar in pairs of prediction motion tasks. Furthermore, the occlusion duration VE slopes were also similar (see Fig. 2C, for example from position and number tasks). This provides indirect evidence that the tasks are governed by a common rate control module, which always accumulates error at a similar rate (Lyon and Waag, 1995). However, there are certain exceptions to this rule (i.e., in Experiment 2 of Makin and Bertamini, 2014). We cannot claim slope analysis provides conclusive evidence for the CRC model. At present, the empirical evidence for the CRC is partial, and there is no clean way to distinguish between CRC and SRC hypotheses experimentally. Indeed, the SRC hypothesis also has some indirect support. For example, the trajectory network model states that daisy chaining between adjacent elementary motion detectors can explain perceptual phenomena like visual inertia (Anstis and Ramachandran, 1987). Watamaniuk and McKee (1995) argued that the predictive signal can keep going in the absence of sensory inputs and cover short occlusions. Indeed, Khoei et al. (2013) proposed that prediction motion is mediated by a finely structured set of diffuse mechanisms that can be implemented at the scale of a single cortical area. When a train of predictable sounds is suddenly broken by an omission or an unusual sound, the brain generates an event-related potential component called the mismatch negativity. The mismatch negativity can be recorded in unconscious participants and arguably showcases the “primitive intelligence” within each sensory system (Naatanen et al., 2007). It seems that all sensory mechanisms tune into temporal regularity in the input stream. The SRC simply states that this primitive intelligence with sensory maps is sufficient for all prediction motion performance.

9

ARTICLE IN PRESS 10

Choosing the speed of dynamic mental simulations

To reconcile evidence for CRC and SRC, I propose that the local mechanisms within each sensory map may cover very short occlusions of (<200 ms), while a central rate controller may be recruited for longer occlusions. For one thing, pursuit eye movement is relatively unaffected by occlusions of <200 ms, but breaks down after this, and approximate tracking is only maintained if volitional effort is applied (Pola and Wyatt, 1997). Second, fMRI studies show that additional brain areas, including the DLFPC and basal ganglia, are recruited during occluded tracking (Lencer et al., 2004; Nagel et al., 2006). In summary, we cannot be completely sure that participants reliably run dynamic simulations during prediction motion tasks (at least not when we use the production protocol). If participants do run such simulations, they are likely to rely on a common rate control module which kicks in after the first 200 ms of occlusion.

4 UNIFICATION BETWEEN COMMON RATE CONTROLLER AND IMAGERY If a common rate controller exists, we can move on to ask how it relates to other proposed mechanisms and modules. As described in the introduction, we want to avoid ontic bulge in cognitive science, so this is always a useful exercise. Several prediction motion studies assume that the task requires mental imagery or imaginary motion (Crespi et al., 2012; de’Sperati and Deubel, 2006; Jonikaitis et al., 2009; Gilden et al., 1995; Schnider et al., 1995). For instance, Huber and Krist (2004) found that fixation instructions did not affect responses on their prediction motion task and suggested that mental imagery mediates performance. Huber and Krist (2004) claim that eye movements are a useful way of measuring dynamic mental imagery when permitted, but that they are epiphenomenal, and do not causally contribute to imagery. It is possible that imagery may indeed occur during prediction motion tasks, although this is not a vivid experience. However, it is important to update imagery at the right speed. Once we propose a cognitive mechanism for extracting trajectory information and updating the film played in the mind’s eye, the imagery itself seems somewhat secondary. It is the director mechanism which is doing the work. Nevertheless, the CRC model is consistent with use of mental imagery—it could control the speed at which imagery is updated.

5 UNIFICATION BETWEEN RATE CONTROLLER AND VELOCITY MEMORY In earlier work (Makin and Poliakoff, 2011; Makin et al., 2008, 2009, 2012), I argued that prediction motion is entirely mediated by the smooth pursuit eye movement system as described in authoritative reviews like Barnes (2008) or Lisberger (2010). A rough outline of the smooth pursuit system is shown in Fig. 4. When light moves

ARTICLE IN PRESS 5 Unification between rate controller and velocity memory

Negative feedback

Target velocity

+

−

Retinal velocity

+

Motor drive +

Efference copy loop

+

−

Eye velocity

Fixation commands

Expectation Experience Velocity Memory

FIG. 4 Simplified diagram of the pursuit system.

across the retina, pursuit eye movements are triggered reflexively, so that retinal velocity signals are canceled (open-loop phase). However, visuomotor delay of 100 ms means that further, extraretinal systems are required for pursuit maintenance (close-loop phase). The efference copy loop achieves this by recycling motor commands. The efference copy loop also explains the sense of object motion during pursuit, and it allows perfect tracking across occlusions shorter than visuomotor delay (Bennett and Barnes, 2003). If upcoming motion is predictable, primates display anticipatory smooth pursuit eye movements before motion onset. Anticipatory pursuit is scaled to the expected velocity of the upcoming target. Barnes and Asselman (1991) explained this by proposing a velocity memory system, which can store previously seen velocities (Barnes et al., 1997; Chakraborti et al., 2002; Poliakoff et al., 2005). The velocity store in the pursuit system could also guide approximate tracking moving objects after the first 100–200 ms of occlusion (Bennett and Barnes, 2003, 2006a,b; de Xivry et al., 2006). Makin and Poliakoff (2011) noted that the pursuit system can explain performance in both production tasks and interruption paradigms (Fig. 2A and B). After all, the apparatus for tracking visible and occluded targets is present (and well understood at mechanistic and neural levels). Furthermore, according to the influential premotor theory of attention, covert shifts of spatial attention and oculomotor control are tightly coupled (Rizzolatti et al., 1987). The smooth pursuit system can mediate performance during fixation as well. Despite the plausibility of these ideas, Makin and Poliakoff (2011) were probably wrong. The pursuit system cannot explain prediction motion performance in feature

11

ARTICLE IN PRESS 12

Choosing the speed of dynamic mental simulations

space (as in Fig. 3B and C). However, we can replace velocity store with a common rate control system and leave the rest of the Makin and Poliakoff (2011) account untouched. This is not a radical step: Bennett and Barnes (2006a,b) argued that the brain updates a representation of target position during occlusion. It could be that the common rate controller not only controls the speed of position updating but also controls the speed of all other internal simulations as well. This is not considered in the pursuit literature, but neither is it inconsistent with pursuit models. In fact, Barnes and Marsden (2002) entertained the possibility that velocity memory guides hand movements as well as eye movements, so it is not a radical jump to reconceptualize velocity memory as a rate controller with a broader function than previously envisaged (Makin and Chauhan, 2014).

6 UNIFICATION BETWEEN RATE CONTROLLER AND PACEMAKER–ACCUMULATOR CLOCK Next we can ask whether the rate controller is in fact identical to other putative “pacemakers” in the brain. As mentioned earlier, many researchers believe duration estimates are made by an internal pacemaker–accumulator clock (Buhusi and Meck, 2005; Gibbon, 1977; Gibbon et al., 1984; Jones et al., 2011; van Rijn et al., 2014; Wearden et al., 1998). If we want to estimate the duration of an event, a switch closes, and ticks are transferred from the pacemaker to the accumulator. The number of ticks in the accumulator is proportional to elapsed time. The final value in the accumulator is a representation of duration. In this model, the pacemaker only has one target module—it can only be coupled to the accumulator (Fig. 5A). Nevertheless, it is possible that the pacemaker has other targets as well. Perhaps the pacemaker is more like a central rate controller that can be functionally coupled to many different sensory maps to guide the speed of mental updating? (Fig. 5B). Several eminent timing researchers take the pacemaker–accumulator clock theory seriously, and further propose that there is a core timing system in the brain, located in the dorsal striatum of the basal ganglia and supplementary premotor area (Coull et al., 2011). Neuroimaging research has found additional basal ganglia and DLPFC activations during occluded object tracking (Lencer et al., 2004). We can speculate that the rate controller is a function of the dorsal striatum, and the DLPFC plays an executive role, temporarily connecting it to different cortical maps (Fig. 5B). This neuroanatomical hypothesis is supported by evidence that the pacemaker speed is dopamine sensitive and can be increased by amphetamine (dopamine agonist) and decreased by haloperidol (dopamine antagonist) (Buhusi and Meck, 2005; Meck, 1983); however, there is some doubt about the dopamine sensitivity story in humans, because the effect of Parkinson’s disease on timing is controversial (Pastor et al., 1992; Wearden et al., 2008). Future research could measure the effect of dopamine agonists and antagonists on prediction motion performance.

ARTICLE IN PRESS 7 Rate control in synchronization–continuation paradigms

A

Start/stop signals

Clock Pacemaker

Switch

Accumulator

Memory

Comparison

Pacemaker–accumulator clock model

B Local predictive mechanisms (e.g., trajectory networks)

Executive control (DLPFC)

Spatial location Number

Sensory map Z

Rate controller (Basal ganglia)

Colour

Empty-full

Memory

Accumulator (SMA)

Sensory map Y

Decision

Sensory map X The common rate control model

FIG. 5 Can the pacemaker component of the internal clock be theoretically unified with the common rate controller? (A) Standard diagram of the pacemaker–accumulator clock. (B) Speculative reworking, where the pacemaker can be flexibly coupled to many cortical targets.

7 RATE CONTROL IN SYNCHRONIZATION–CONTINUATION PARADIGMS Synchronization–continuation paradigms have two phases. In the synchronization phase, participants produce a rhythmic motor response, such as finger taping, in time to a metronome. After the metronome is switched off, they continue tapping at the same frequency. Variability in motor response time can be divided into central and peripheral sources (Wing and Kristofferson, 1973). It is likely that an internal timing mechanism paces the tapping during continuation (Wing, 2002). If the ideas in the previous section are correct, the same internal timing mechanisms control updating during the occlusion phase of prediction motion tasks (Fig. 5B).

13

ARTICLE IN PRESS 14

Choosing the speed of dynamic mental simulations

8 CONCLUSION As cognitive science grows, researchers propose new mechanisms and modules. It is essential that this proliferation is constrained in a principled way. One approach is to seek theoretical unification where possible. There will be occasions when a neurocognitive mechanism which controls Task A turns out to be the very same neurocognitive mechanism that controls Task B. The potential for theoretical unification might sometimes be unrecognized by researchers using superficially different language in their own paradigms. Here I describe the CRC account, which explains a range of judgements and behaviors. It is possible that a single rate controller, or pacemaker, guides performance on different prediction motion tasks, in both physical space and feature space. The same mechanism works for production tasks and interruption paradigms. Furthermore, the same rate control system could mediate synchronization–continuation. The rate controller could be related to the velocity store in smooth pursuit models, or the pacemaker in internal clock models. On the positive side, it seems that we can make theoretical headway with this program of unification. On the negative side, it is difficult to conclusively validate the CRC with new empirical evidence. Even sticking to prediction motion, it is difficult to demonstrate that the same rate control mechanism is recruited in all the prediction motion tasks shown in Fig. 3. Early evidence points in that direction, but I would not claim it is conclusive. A major problem for prediction motion research is that there is no pure task, which cannot be attacked with other strategies. Timing has received much more attention than prediction motion, so we might expect more certainty here. However, nobody has conclusively demonstrated the existence of a supramodal timing system. After reviewing fMRI work, Coull et al. (2011) concluded that “results are suggestive of a centralized, context independent, supramodal timer localized in the dorsal striatum of the [Basal Ganglia]” (p. 7). However, the influential striatal beats frequency model proposes that cortical oscillations in any network can approximate the role of the pacemaker (Buhusi and Meck, 2005). Others have explicitly argued against supramodal timing (Johnston et al., 2006). It is instructive that we still do not know for sure whether there is a single neural clock in the brain. Indeed, various intermediate theories are possible here as well: Perhaps there are many systems capable of mediating the pacemaker role, but just one accumulator? Often the ontological status of a proposed cognitive module is problematic. For example, the velocity store in smooth pursuit models is perhaps just a convenient placeholder for “the brain’s ability to store velocity information.” We can think of it as a discrete information processing unit and speculate about its neural basis, but sometimes these steps might be misleading. This chapter records an attempt to see how far we can proceed with a program of collapsing superficially disparate cognitive mechanisms. This is intriguing and seems to break new ground. However, it is difficult to make solid progress based

ARTICLE IN PRESS References

on empirical evidence. To some readers, the implications of this limitation for cognitive science might be more interesting the than existence (or nonexistence) of a common rate controller.

REFERENCES Anstis, S., Ramachandran, V.S., 1987. Visual inertia in apparent motion. Vision Res. 27 (5), 755–764. Barnes, G.R., 2008. Cognitive processes involved in smooth pursuit eye movements. Brain Cogn. 68 (3), 309–326. Barnes, G.R., Asselman, P., 1991. The mechanism of prediction in human smooth-pursuit eye movements. J. Physiol. Lond. 439, 439–461. Barnes, G.R., Marsden, J.F., 2002. Anticipatory control of hand and eye movements in humans during oculo-manual tracking. J. Physiol. Lond. 539 (1), 317–330. Barnes, G.R., Grealy, M., Collins, S., 1997. Volitional control of anticipatory ocular smooth pursuit after viewing, but not pursuing, a moving target: evidence for a re-afferent velocity store. Exp. Brain Res. 116 (3), 445–455. Battaglini, L., et al., 2015. Probing the involvement of the earliest levels of cortical processing in motion extrapolation with rapid forms of visual motion priming and adaptation. Atten. Percept. Psychophys. 77 (2), 603–612. Benguigui, N., Bennett, S.J., 2010. Ocular pursuit and the estimation of time-to-contact with accelerating objects in prediction motion are controlled independently based on first-order estimates. Exp. Brain Res. 202 (2), 327–339. Benguigui, N., Broderick, M., Ripoll, H., 2004. Age differences in estimating arrival-time. Neurosci. Lett. 369 (3), 197–202. Bennett, S.J., Barnes, G.R., 2003. Human ocular pursuit during the transient disappearance of a visual target. J. Neurophysiol. 90 (4), 2504–2520. Bennett, S.J., Barnes, G.R., 2006a. Combined smooth and saccadic ocular pursuit during the transient occlusion of a moving visual object. Exp. Brain Res. 168 (3), 313–321. Bennett, S.J., Barnes, G.R., 2006b. Smooth ocular pursuit during the transient disappearance of an accelerating visual target: the role of reflexive and voluntary control. Exp. Brain Res. 175 (1), 1–10. Bennett, S.J., Benguigui, N., 2013. Is acceleration used for ocular pursuit and spatial estimation during prediction motion? PLoS One 8 (5), e63382. Blaser, E., Pylyshyn, Z.W., Holcombe, A.O., 2000. Tracking an object through feature space. Nature 408 (6809), 196–199. Buhusi, C.V., Meck, W.H., 2005. What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6 (10), 755–765. Burr, D., Tozzi, A., Morrone, M.C., 2007. Neural mechanisms for timing visual events are spatially selective in real-world coordinates. Nat. Neurosci. 10 (4), 423–425. Chakraborti, S.R., Barnes, G.R., Collins, C.J.S., 2002. Factors affecting the longevity of a short-term velocity store for predictive oculomotor tracking. Exp. Brain Res. 144 (2), 152–158. Clark, A., 2013. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sci. 36 (3), 181–204.

15

ARTICLE IN PRESS 16

Choosing the speed of dynamic mental simulations

Coull, J.T., Cheng, R.K., Meck, W.H., 2011. Neuroanatomical and neurochemical substrates of timing. Neuropsychopharmacology 36 (1), 3–25. Crespi, S., et al., 2012. Spotting expertise in the eyes: Billiards knowledge as revealed by gaze shifts in a dynamic visual prediction task. J. Vis. 12 (11), 1–19. de Xivry, J.J.O., et al., 2006. Evidence for synergy between saccades and smooth pursuit during transient target disappearance. J. Neurophysiol. 95 (1), 418–427. DeLucia, P.R., Liddell, G.W., 1998. Cognitive motion extrapolation and cognitive clocking in prediction motion tasks. J. Exp. Psychol. Hum. Percept. Perform. 24 (3), 901–914. de’Sperati, C., 2003. Precise oculomotor correlates of visuospatial mental rotation and circular motion imagery. J. Cogn. Neurosci. 15 (8), 1244–1259. de’Sperati, C., Deubel, H., 2006. Mental extrapolation of motion modulates responsiveness to visual stimuli. Vision Res. 46 (16), 2593–2601. Ekman, P., 1999. Facial expressions. In: Dalgleish, T., Power, T. (Eds.), Handbook of Cognition and Emotion. Wiley, New York, pp. 301–320. Fries, P., Nikolic, D., Singer, W., 2007. The gamma cycle. Trends Neurosci. 30 (7), 309–316. Fuster, J.M., 2009. Cortex and memory: emergence of a new paradigm. J. Cogn. Neurosci. 21 (11), 2047–2072. Gibbon, J., 1977. Scalar expectancy-theory and Webers law in animal timing. Psychol. Rev. 84 (3), 279–325. Gibbon, J., Church, R.M., Meck, W.H., 1984. Scalar timing in memory. Ann. N. Y. Acad. Sci. 423, 52–77. Gibson, J.J., 1979. The Ecological Approach to Visual Perception. Lawrence Earlbaum Associates, Hillsdale. Gilden, D., Blake, R., Hurst, G., 1995. Neural adaptation of imaginary visual-motion. Cogn. Psychol. 28 (1), 1–16. Gopnik, A., 2009. Rational constructivism: a new way to bridge rationalism and empiricism. Behav. Brain Sci. 32 (2), 208. Gottsdanker, R.M., 1952. The accuracy of prediction motion. J. Exp. Psychol. 43 (1), 26–36. Howard, C.J., Holcombe, A.O., 2008. Tracking the changing features of multiple objects: progressively poorer perceptual precision and progressively greater perceptual lag. Vision Res. 48 (9), 1164–1180. Huber, S., Krist, H., 2004. When is the ball going to hit the ground? Duration estimates, eye movements, and mental imagery of object motion. J. Exp. Psychol. Hum. Percept. Perform. 30 (3), 431–444. Johnston, A., Arnold, D.H., Nishida, S., 2006. Spatially localized distortions of event time. Curr. Biol. 16 (5), 472–479. Jones, L.A., Allely, C.S., Wearden, J.H., 2011. Click trains and the rate of information processing: does speeding up subjective time make other psychological processes run faster? Q. J. Exp. Psychol. 64 (2), 363–380. Jonikaitis, D., Deubel, H., de’Sperati, C., 2009. Time gaps in mental imagery introduced by competing saccadic tasks. Vision Res. 49 (17), 2164–2175. Khoei, M.A., Masson, G.S., Perrinet, L.U., 2013. Motion-based prediction explains the role of tracking in motion extrapolation. J. Physiol. Paris 107 (5), 409–420. Kriegeskorte, N., 2015. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annu. Rev. Vis. Sci. 1 (1), 417–446. Lee, D.N., 1976. Theory of visual control of braking based on information about time-tocollision. Perception 5 (4), 437–459.

ARTICLE IN PRESS References

Lencer, R., et al., 2004. Cortical mechanisms of smooth pursuit eye movements with target blanking. An fMRI study. Eur. J. Neurosci. 19 (5), 1430–1436. Lisberger, S.G., 2010. Visual guidance of smooth-pursuit eye movements: sensation, action, and what happens in between. Neuron 66 (4), 477–491. Lyon, D.R., Waag, W.L., 1995. Time-course of visual extrapolation accuracy. Acta Psychol. (Amst) 89 (3), 239–260. Makin, A.D.J., Bertamini, M., 2014. Do different types of dynamic extrapolation rely on the same mechanism? J. Exp. Psychol. Hum. Percept. Perform. 40 (4), 1566–1579. Makin, A.D.J., Chauhan, T., 2014. Memory-guided tracking through physical space and feature space. J. Vis. 14 (13), 1–15. Makin, A.D.J., Poliakoff, E., 2011. Do common systems control eye movements and motion extrapolation? Q. J. Exp. Psychol. 64 (7), 1327–1343. Makin, A.D.J., et al., 2008. The effect of previously viewed velocities on motion extrapolation. Vision Res. 48 (18), 1884–1893. Makin, A.D.J., Poliakoff, E., El-Deredy, W., 2009. Tracking visible and occluded targets: changes in event related potentials during motion extrapolation. Neuropsychologia 47 (4), 1128–1137. Makin, A.D.J., et al., 2012. Covert tracking: a combined ERP and fixational eye movement study. PLoS One 7 (6), e38479. Mauss, I., Robinson, M., 2009. Measures of emotion: a review. Cognit. Emot. 23 (2), 209–237. Meck, W.H., 1983. Selective adjustment of the speed of internal clock and memory processes. J. Exp. Psychol. Anim. Behav. Process. 9 (2), 171–201. Mitchell, C.J., De Houwer, J., Lovibond, P.F., 2009. The propositional nature of human associative learning. Behav. Brain Sci. 32 (2), 183. Naatanen, R., et al., 2007. The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clin. Neurophysiol. 118 (12), 2544–2590. Nagel, M., et al., 2006. Parametric modulation of cortical activation during smooth pursuit with and without target blanking. An fMRI study. Neuroimage 29 (4), 1319–1325. Op de Beeck, H.P., Haushofer, J., Kanwisher, N.G., 2008. Interpreting fMRI data: maps, modules and dimensions. Nat. Rev. Neurosci. 9 (2), 123–135. Pastor, M.A., et al., 1992. Time estimation and reproduction is abnormal in Parkinson’s disease. Brain 115, 211–225. Peterken, C., Brown, B., Bowman, K., 1991. Predicting the future position of a moving target. Perception 20 (1), 5–16. Pinker, S., 1994. The Language Instinct. HarperCollins, New York. Pinker, S., 1997. How the Mind Works. W.W. Norton & Company, New York. Pola, J., Wyatt, H.J., 1997. Offset dynamics of human smooth pursuit eye movements: effects of target presence and subject attention. Vision Res. 37 (18), 2579–2595. Poliakoff, E., Collins, C.J.S., Barnes, G.R., 2005. Attention and selection for predictive smooth pursuit eye movements. Cogn. Brain Res. 25 (3), 688–700. Posner, M.I., 1980. Orienting of attention. Q. J. Exp. Psychol. 32, 3–25. Pylyshyn, Z.W., 2003. Mental imagery: in search of a theory. Behav. Brain Sci. 25 (2), 157–238. Rizzolatti, G., et al., 1987. Reorienting attention across the horizontal and vertical meridians— evidence in favour of a premotor theory of attention. Neuropsychologia 25 (1A), 31–40. Rosenbaum, D.A., 1975. Perception and extrapolation of velocity and acceleration. J. Exp. Psychol. Hum. Percept. Perform. 1 (4), 395–403.

17

ARTICLE IN PRESS 18

Choosing the speed of dynamic mental simulations

Roth, M.J., Synofzik, M., Lindner, A., 2013. The cerebellum optimizes perceptual predictions about external sensory events. Curr. Biol. 23 (10), 930–935. Sauter, D.A., et al., 2010. Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. Proc. Natl. Acad. Sci. U.S.A. 107 (6), 2408–2412. Schiff, W., Detwiler, M.L., 1979. Information used in judging impending collision. Perception 8 (6), 647–658. Schiff, W., Oldak, R., 1990. Accuracy of judging time to arrival: effects of modality, trajectory, and gender. J. Exp. Psychol. Hum. Percept. Perform. 16 (2), 303–316. Schnider, A., Gutbrod, K., Hess, C.W., 1995. Motion imagery in Parkinsons-disease. Brain 118, 485–493. Shepard, R.N., Metzler, J., 1971. Mental rotation of 3-dimensional objects. Science 171 (3972), 701–703. Sheth, B.R., Nijhawan, R., Shimojo, S., 2000. Changing objects lead briefly flashed ones. Nat. Neurosci. 3 (5), 489–495. Slater-Hammel, A.T., 1955. Estimation of movement as a function of the distance of movement perception and target distance. Percept. Mot. Skills 5, 201–204. Tresilian, J.R., 1995. Perceptual and cognitive-processes in time-to-contact estimation— analysis of prediction-motion and relative judgment tasks. Percept. Psychophys. 57 (2), 231–245. Vagnoni, E., Lourenco, S.F., Longo, M.R., 2012. Threat modulates perception of looming visual stimuli. Curr. Biol. 22 (19), 826–827. van Rijn, H., Taatgen, N.A., 2008. Timing of multiple overlapping intervals: how many clocks do we have? Acta Psychol. (Amst) 129 (3), 365–375. van Rijn, H., Gu, B., Meck, W.H., 2014. Dedicated clock/timing circuit theories of time perception and timed performance. In: Merchant, H., de Lafuente, V. (Eds.), Neurobiology of Interval Timing. Springer-Verlag, New York. Watamaniuk, S.N.J., McKee, S.P., 1995. Seeing motion behind occluders. Nature 377 (6551), 729–730. Wearden, J.H., et al., 1998. Why “sounds are judged longer than lights”: application of a model of the internal clock in humans. Q. J. Exp. Psychol. B Comp. Physiol. Psychol. 51 (2), 97–120. Wearden, J.H., et al., 2008. Stimulus timing by people with Parkinson’s disease. Brain Cogn. 67 (3), 264–279. Wiener, E.L., 1962. Motion prediction as a function of target speed and duration of presentation. J. Appl. Psychol. 46 (6), 420–424. Wing, A.M., 2002. Voluntary timing and brain function: an information processing approach. Brain Cogn. 48 (1), 7–30. Wing, A.M., Kristofferson, A.B., 1973. Response delays and timing of discrete motor responses. Percept. Psychophys. 14 (1), 5–12.