Journal of Applied Research in Memory and Cognition 6 (2017) 14–19 Contents lists available at ScienceDirect
Journal of Applied Research in Memory and Cognition journal homepage: www.elsevier.com/locate/jarmac
A (Meta-) Cognitive Perspective on the Functional-Cognitive Perspective: The Applied Value of Behaviorally Oriented Cognitive Research and Theory夽 Morris Goldsmith ∗ University of Haifa, Israel Keywords: Functional-cognitive, Memory, Metamemory, Strategic control, Basic research, Applied research
“There is nothing as practical as a good theory.” This influential quote from Kurt Lewin, repeated hundreds of times in the social sciences literature (McCain, 2015), is one way of reconciling what has sometimes been seen as an inherent tension between the goals of basic and applied research. It has generally been taken to emphasize the value of basic theory-oriented research in guiding the conceptualization, study, and solution of applied problems, while similarly acknowledging the contribution of applied research to the development and refinement of basic scientific theory. Within the field of psychology, the mutually beneficial relationship and potential cross-fertilization between basic and applied research, as well as the problems involved, have frequently been discussed (e.g., Payne & Conrad, 1997). In their target article, De Houwer, Hughes, and BarnesHolmes address the relationship between application and theory from a fresh perspective: a proposed functional-cognitive framework designed to resolve a different divide within psychological science—between the functionally oriented and cognitively oriented approaches. Their article has several stated goals, one of which is to increase communication and cross-fertilization between cognitive and functional research within the domain of applied psychology. A more far-reaching goal is to promote “communication at the functional level so that applied psychology can evolve into a more integrated but still diverse discipline that could be referred to as ‘psychological engineering”’ (p. 2). This goal derives from the key assertion that “all applied psychology is ultimately directed at the functional level
Author Note Please note that this paper was handled by the current editorial team of JARMAC. ∗ Correspondence concerning this article should be addressed to Morris Goldsmith, Department of Psychology and Institute of Information 夽
of explanation. . .[entailing that] all applied researchers can communicate in functional terms” (p. 2). I commend the authors on a very thoughtful and systematic analysis that I believe contains many valuable points, particularly those pertaining to the need for greater care in conceptualizing and using theoretical and operational terms in ways that are appropriate for one’s chosen level of explanation (cognitive, neurophysiological, or functional), and the desirability of going beyond research that is “effect-centric,” by paying greater attention to the commonalities between cognitive and behavioral phenomena that are often studied in separate paradigms and literatures, or by choosing research questions and paradigms that address more general phenomena in the first place. Nevertheless, in this commentary I take issue with some aspects of the authors’ argument, specifically those leading to the conclusion that the goals of applied research are best served by a heavy preoccupation with functional explanation and the translation of applied research findings into a common functional language. To anticipate, my argument is that the twin goals of applied research emphasized in the target article, to predict and influence behavior, are best served by theories and explanations that are general and behaviorally oriented, regardless of whether these make use of hypothetical cognitive mechanisms or abstract functional constructs. Moreover, in many domains of behavior that are of applied interest, the derivation of general functional descriptions or explanations of environment–behavior relations, detached from cognitive theory, does not appear to be a realistic goal. As a case in point, I will present and analyze
Processing and Decision Making, University of Haifa, Haifa, Israel. Contact:
[email protected]
FUNCTIONAL-COGNITIVE PERSPECTIVE
some of my own research in the area of human memory and metamemory. From the outset, let me clarify that I identify myself as a cognitive psychologist who does basic, theory-oriented memory research, with much of that research being inspired by reallife memory phenomena that are relevant to applied contexts, such as eyewitness testimony and scholastic testing. As a cognitive psychologist, I am greatly interested in uncovering and understanding mental mechanisms, not only “for its own sake” (i.e., with the goal of understanding the human mind), but also because I believe that this is a very powerful way of advancing our understanding of the determinants of human behavior. This is not merely to say that I am interested in understanding both the mind and behavior (i.e., that I have both cognitive and functional research goals). There is a much stronger statement here, by which these goals are tightly interrelated—the belief that it is necessary to describe and understand the workings of the human mind (cognitive mechanisms) in order to fully describe and understand the determinants of human behavior. That is, I hold a strong, somewhat axiomatic conviction that it is simply not possible to develop a complete and comprehensive functional description/explanation of human behavior (at least within my areas of interest) that avoids the use of mediating mental mechanisms. By historical accounts (e.g., Greenwood, 1999; Miller, 2003), this developing conviction was one of the reasons for the advent of the “cognitive revolution” in the first place. In light of the above, I was left a bit unsettled by some aspects of the analysis and ensuing proposal put forward in the target article. First, although I can understand the heuristic value of an analysis by which the primary goal of functional research is to explain behavior in terms of events in the environment whereas the primary goal of cognitive research is to explain the impact of the environment on behavior in terms of mediating cognitive mechanisms, the implications that are drawn from this difference are not straightforward. The authors state that “there are valid reasons for adopting each type of goal. Functional researchers focus on environment–behavior relations because it allows them to predict-and-influence behavior” (p. 3). Is this meant to imply that the inclusion of mediating cognitive mechanisms precludes the use of cognitive theory to predict and influence behavior? The authors appear to be targeting primarily the type of cognitive research in which the behavioral responses that are observed (e.g., key pressing) and the aspects of the environment that are manipulated (e.g., subliminal vs. supraliminal presentation) are of no intrinsic interest. Yet, there is also a great deal of cognitive research in which the observed and manipulated variables are of intrinsic interest, as representative of a class of behaviors and environmental conditions that are no less integral to the research than are the cognitive mechanisms hypothesized to mediate them (e.g., Cohen & Conway, 2007). It is not clear how such “behaviorally oriented” (cf. “ecological”; Neisser, 1985) cognitive research fits into the authors’ proposed taxonomy within the functional-cognitive framework. Can such research, including the cognitive explanations of
15
behavior and empirical generalizations1 that it provides, directly serve the applied goals of prediction and influence, or to be of applied value, do the cognitive explanations need to be translated, through functional analysis, into functional explanations comprised of abstract functional principles and terms? A highly related question is whether one can, in fact, derive abstract functional explanations that do equal justice to the complexity of the empirical data. I would like to share some reflections that I had in relating these questions to my own behaviorally oriented cognitive research (using this term to characterize the research is one of the products of these reflections), conducted primarily in collaboration with Asher Koriat, on the strategic regulation of memory reporting. In doing so, to aid the unfamiliar reader, I will summarize that work in some detail. The initial impetus for the work was a conceptual analysis (Koriat & Goldsmith, 1994) of a seemingly inconsistent pattern of results pointed to by Neisser (1988), between traditional laboratory research and naturalistic (e.g., eyewitness memory) research, with regard to the quantity and accuracy of the information reported from memory under various testing conditions: In contrast to the well-established superiority of recognition testing over recall testing in traditional laboratory research, the established wisdom from eyewitness memory research is that recall testing yields more accurate memory responses than recognition testing. Our analysis led to the identification of several key factors that are commonly confounded in comparing results between the two research contexts: test format (recall vs. recognition), report option (free vs. forced reporting), and the memory property of interest (accuracy vs. quantity). By orthogonally manipulating-measuring these three variables (Koriat & Goldsmith, 1994), we were able to show that whereas test format had a substantial effect on memory quantity performance (the input-bound proportion of items correctly recalled or recognized), with recognition testing superior to recall testing, it had no effect on memory accuracy performance (the output-bound proportion of reported/selected answers that are correct). Instead, the critical factor affecting memory accuracy was report option (free report superior to forced): Under forced-report conditions (e.g., forced recall or forced-choice recognition) in which rememberers are forced to provide a substantive response for each and every input item/question, the percentage of errors that they report simply reflects their overall level of memory or knowledge. By contrast, under free-report conditions (e.g., free recall or free multiple-choice recognition), in which people are allowed to report (select) only the answers that they believe are actually correct, they can thereby screen out many of the erroneous responses that would otherwise have been provided under forced report. In this case, the percentage of
1 At some points in the target article, the term “functional” is used in a very weak sense, to refer to any description of empirically observed environment–behavior relations, even those that do not utilize general functional principles or terms. In this trivial sense, all empirical research in psychology, including cognitive research, involves some minimal level of “functional” description.
FUNCTIONAL-COGNITIVE PERSPECTIVE
errors that they report reflects the effectiveness of the screening process, as well as overall level of memory. Had our research goals been purely functional, that might have been the end of the story. However, it struck us that the option of free report is a fundamental aspect of memory reporting in real-life settings (somewhat less so in traditional laboratory settings), yet at that time very little was known about how rememberers utilize the option of free report in regulating their memory performance. Thus, as a framework to guide the study of free memory reporting, we subsequently put forward a theoretical model of how metacognitive monitoring and control processes are used by rememberers to regulate their memory performance under free-report conditions (Koriat & Goldsmith, 1996b): In responding to a memory query, the memory retrieval process is accompanied by a metacognitive monitoring mechanism that is used to assess the correctness of retrieved candidate answers, and a control mechanism that determines whether or not to volunteer the (subjectively) best accessible candidate answer by setting a report criterion on the monitoring output (subjective confidence). The answer is volunteered if subjective confidence in its correctness passes the criterion; otherwise it is withheld. The criterion is set on the basis of implicit or explicit payoffs, that is, the perceived gain for providing correct information relative to the cost of providing wrong information. According to the model, when given the opportunity to do so (i.e., the option of free report), rememberers should generally be able to enhance the accuracy of the information that they report by screening out answers that are relatively likely to be wrong. Such enhancement, however, is subject to a quantityaccuracy tradeoff: In general, raising the report criterion should result in fewer reported answers, a higher proportion of which are correct (increased accuracy), but assuming that the monitoring is not perfect, some potentially correct answers will be mistakenly withheld, thereby decreasing memory quantity performance. Because of this trade-off, the strategic regulation of memory performance requires rememberers to weigh the relative payoffs for accuracy and quantity in reaching an appropriate criterion setting. Although the model is rather simple, its implications for the determinants of free-report memory performance are not. Based on the model, the joint levels of achieved free-report memory accuracy and quantity performance can be shown to depend on four contributing and potentially interacting factors: (1) memory retrieval (retention)—the accessibility of the target information at the time of retrieval (the proportion of retrieved best-candidate answers that are correct); (2) monitoring effectiveness—the extent to which one’s subjective confidence actually discriminates between correct and incorrect best-candidate answers; (3) control policy (report criterion)—the confidence level above which a best-candidate answer is reported, and below which it is withheld; (4) control sensitivity—the extent to which the reporting or withholding of best-candidate answers is based on the monitoring output. Let me illustrate some of the potential complexity in the expected and observed patterns of results (for reviews, see Goldsmith, 2016; Goldsmith & Koriat, 2007; and for a similar model yielding similar results, see Higham, 2007): (1) In
16
general, the ability to retrieve greater amounts of correct information (measured under forced-report conditions) increases both the quantity and the accuracy of the information reported under free-report conditions. (2) Holding memory retrieval constant, accuracy increases and quantity decreases as the report criterion is raised. (3) Crucially, however, both the accuracy gains and the quantity costs of selective reporting (for a given joint level of retrieval and report criterion) depend on monitoring effectiveness: Accuracy gains increase and quantity costs decrease (in some cases, with no tradeoff at all) as monitoring improves. (4) Finally, all of the above is contingent on a high level of control sensitivity. When the basis of the selective reporting becomes less tightly coupled to confidence, for example when taking psychoactive medication (Massin-Krauss, Bacon, & Danion, 2002), in old age (Pansky, Goldsmith, Koriat, & Pearlman-Avnion, 2009), or in certain clinical populations (Hebscher, Barkan-Abramski, Goldsmith, Aharon-Peretz, & Gilboa, 2015; Koren, Seidman, Goldsmith, & Harvey, 2006), the contribution of selective reporting to memory performance becomes more erratic, thereby tending to lower memory accuracy. The preceding discussion relates to the strategic regulation of memory reporting by volunteering or withholding individual items of information. Using similar mechanisms and principles, the theoretical framework was subsequently extended to cover another means of strategic report regulation—controlling the grain size (i.e., the precision or coarseness) of the information reported from memory (Goldsmith, Koriat, & Weinberg-Eliezer, 2002). By reporting information at a coarser grain size (e.g., “sometime in the early evening” instead of “6:30”; “it was a dark color” rather than “navy blue”), rememberers can increase the likelihood that the information is correct, but here too this comes at the price of providing a less informative answer (i.e., an accuracy-informativeness tradeoff). The similarity of the principles and mechanisms involved in the use of report option and grain size to regulate one’s reporting eventually led to the development of an integrated model of strategic reporting that specifies the mechanisms and considerations that guide the choice of whether to provide a relatively precise or relatively coarse grained answer, or whether to withhold the answer entirely (Ackerman & Goldsmith, 2008; Goldsmith & Koriat, 2007). Essentially, the model holds that people strive to provide to provide the most precise-informative answer that they can, as long as they are sufficiently confident that it is correct. There may be situations, however, in which one’s memory is too poor to allow one to produce an answer that is both reasonably informative (with respect to a “minimum-informativeness” criterion, based on norms of communication and the specific social context) and sufficiently likely to be correct. In such cases, one will tend either to withhold the answer (respond “don’t know”) or provide an informative but low confidence answer, rather than provide an overly coarse answer that violates social communication norms (and would thereby be perceived as “ridiculous”). I have gone into some detail in the preceding description of our work and how it developed because I think that these details help convey both the critical role of abstractive
FUNCTIONAL-COGNITIVE PERSPECTIVE
conceptual-theoretical analysis in guiding the research, and the difficult challenges that would be involved in trying to derive a purely functional abstractive-analytic description or explanation of the retrieval and reporting of correct and false information from memory in terms of environmental factors (past and present) alone. In terms of the theoretical model, such an explanation would essentially need to account for the environmental factors that determine the amount and accuracy of the information that is accessible in memory (based on information regarding the conditions of encoding and retrieval, as well as all intervening events),2 the effectiveness of the rememberer in monitoring the correctness of that information, his or her subjective perceptions of the relative importance of competing goals of accuracy and quantity (informativeness) in specific personal and social contexts (as well as subjective perception of social norms regarding minimum levels of informativeness), and the tightness of the coupling between subjective confidence and the report control decision (control sensitivity). It would also have to account for the manner in which all of these interact to determine whether or not a solicited piece of information is retrieved, and (if retrieved), whether or not it is reported, and (if reported) at what level of precision or coarseness it is reported, and (ultimately), whether the reported information is correct or not. This is not to mention the need to consider the role of individual and population differences in all of these factors. The challenges that such a topic poses for functional explanation may be especially tough because of the intrinsically subjective and “self-directed” (Nelson & Narens, 1994) nature of the metamemory processes that interact with more basic memory processes in determining one’s overt (and covert) memory performance. In this regard, Nelson and Narens (1994, p. 23) quoted Skinner (1974, p. 209) as acknowledging that there is “a useful connection between feelings and behavior. It would be foolish to rule out the knowledge a person has of his current condition or the uses to which it may be put.” Yet, they noted that in his theorizing, Skinner treated self-reflective consciousness (cf. monitoring) only as a response, and did not allow it to have any causal role in controlling external behavior. By contrast, Nelson and Narens (1994) asserted that assigning a causal role to metacognitive processing is crucial both for achieving a coherent conception of cognition and for understanding the control of behavior in naturalistic settings (see also Koriat & Goldsmith, 1996a). So, where does this leave us with regard to the question of whether and how cognitive research and theory can be used to predict and influence behavior in applied contexts? The common approach to bridging the gap between cognitive research and applied contexts is to use the cognitive theory in applied research to derive predictions of meaningful environment–behavior relationships that are relevant to specific
2
Note that the individual and potentially interacting metacognitive contributions to memory performance emphasized in our framework are in addition to all of the complex factors involving the manner and conditions of encoding, storage, and retrieval (and encoding-retrieval interactions), that have been the target of most memory research to date.
17
applied contexts (Herrmann, 1998; see, e.g., Hollins & Weber, 2016, for work relating to the strategic regulation of memory reporting). These predictions can then be examined and potentially verified in experimental situations that are representative of the larger class of applied situations that constitutes the target of the research. If not, analysis of the reasons for the contrary findings can be used to further develop and refine the theory. Once verified, the relevant empirical generalizations, together with all of the necessary caveats, can be incorporated into various “products,” such as journal articles, guidelines and recommendations, and ultimately, the design and development of specific tools and interventions. Although this approach may seem quite similar to the one proposed in the target article, the crucial point of disagreement, as I see it, concerns the nature of the language in which the relevant empirical generalizations are expressed and organized. In my view, the relevant empirical generalizations and caveats cannot be completely divorced from the cognitive/metacognitive theory that bred them (at least when described in journal articles), because many of these, particularly those involving higher order interactions (moderator variables) may only be coherent when understood in terms of the underlying mechanisms that are responsible for them. Thus, any attempt to organize and describe the input-output “function” relating environmental inputs to behavioral outputs in purely functional terms will prove to be difficult, if not impossible. Admittedly, whereas it is relatively easy to demonstrate that something is possible, there is no simple way to demonstrate that something is impossible, so any assertion with regard to the inherent inadequacy of functional description with respect to a particular domain must be, to some extent, a judgment call. I should also emphasize that this pessimistic assertion does not detract in any way from the (proven) utility of functional research and explanation in domains of behavior that fall within its “range of convenience” (Kelly, 1955). However, in behavioral domains that lie outside of this range, I believe that there is presently no better alternative than a broad, behavior-oriented cognitive theory for providing a coherent and continually refined description3 and explanation of complex and growing sets of environment–behavior relations (those that are presently known and those that have yet to be discovered). Despite this fundamental point of disagreement, a strong point of agreement between our views concerns the need for more abstractive-conceptual analysis and generalization in cognitive research. Thus, for example, in the work described above, it was relatively straightforward to take the metacognitive framework for the strategic regulation of memory reporting, developed
3
Like some functional descriptions (those that make use of general theoretical constructs), the descriptions of environment-behavior relations based on cognitive theory can also be generative, in the sense that if the model is sufficiently well specified, one can generate the predicted behavioral outputs for specific sets of inputs “on the fly”. In such cases, one may conceive of the model as describing the function (in the mathematical sense) that relates environmental inputs to behavioral outputs, though this description becomes an explanation only when the model itself is described in abstract theoretical terms (for a related discussion, see Goldsmith, 1998).
FUNCTIONAL-COGNITIVE PERSPECTIVE
originally with regard to the decision whether to report or withhold specific items of information (Koriat & Goldsmith, 1996b), adapt it to cover the essentially similar processes involved in deciding at what level of precision or coarseness to report the information (Goldsmith et al., 2002), and then integrate these two types of regulation into a unified model (Ackerman & Goldsmith, 2008; Goldsmith & Koriat, 2007). It is also rather simple to adapt and apply the framework to a wide range of different memory topics, such as eyewitness testimony, psychometric and scholastic testing, developmental changes in memory performance, and so forth (for reviews, see Goldsmith & Koriat, 2007; Goldsmith, Pansky, & Koriat, 2014; Hollins & Weber, 2016). On a more local scale, the same principles and metacognitive mechanisms proposed to underlie the choice of how precise or coarse an answer to report from memory (control of grain size) have been found to generalize to the choice of how to respond on multiple-choice tests that allow the selection of more than one response alternative for each item (the “plurality” option; Higham, 2013; Luna, Higham, & MartínLuengo, 2011). Finally, issues involving the option of free report (“don’t-know” option), changes in report criterion, and the principle that responses that pass the report criterion (those that are freely reported) are generally more accurate than those that don’t (those that are withheld), have been receiving much attention recently in discussions concerning the optimal ways of conducting (and evaluating) eyewitness lineup procedures (e.g., Gronlund, Wixted, & Mickes, 2014; Hollins & Weber, 2016; Wells, 2014). In sum, I believe that while the functional-cognitive framework and accompanying analyses presented in the target article may provide a useful bridge between some types of experimental cognitive research and the goals of predicting and influencing behavior in applied settings, I believe that it has less to offer with respect to cognitive research that is behaviorally oriented and analytic-abstractive in the first place. Much of the credit for the large amount of such research being conducted today in the field of human memory (and metamemory) goes to the pioneers and organizers of the “practical aspects of memory” movement and conference series (Gruneberg, Morris, & Sykes, 1978, 1988; Herrmann, Hertzog, McEvoy, Hertel, & Johnson, 1996), which eventually gave birth to the Society for Applied Research on Memory and Cognition and its official journal, in which this discussion is now being published.
Conflict of Interest Statement The author declares no conflict of interest.
References Ackerman, R., & Goldsmith, M. (2008). Control over grain size in memory reporting – With and without satisfying knowledge. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 1224–1245. Cohen, G., & Conway, M. A. (Eds.). (2007). Memory in the real world. New York: Psychology Press.
18
Goldsmith, M. (1998). Connectionist modeling and theorizing: Who does the explaining and how? Commentary on Green on connectionist-explanation. Psycoloquy, 9(18) Goldsmith, M. (2016). Metacognitive quality-control processes in memory retrieval and reporting. In J. Dunlosky, & S. K. Tauber (Eds.), The Oxford handbook of metamemory (pp. 357–385). Oxford: Oxford University Press. Koriat, A., & Goldsmith, M. (1994). Memory in naturalistic and laboratory contexts: Distinguishing the accuracy-oriented and quantity-oriented approaches to memory assessment. Journal of Experimental Psychology: General, 123, 297–316. Goldsmith, M., & Koriat, A. (2007). The strategic regulation of memory accuracy and informativeness. In A. Benjamin, & B. Ross (Eds.), Psychology of learning and motivation, Vol. 48: Memory use as skilled cognition (pp. 1–60). San Diego, CA: Elsevier. Goldsmith, M., Koriat, A., & Weinberg-Eliezer, A. (2002). Strategic regulation of grain size memory reporting. Journal of Experimental Psychology: General, 131, 73–95. Goldsmith, M., Pansky, A., & Koriat, A. (2014). Metacognitive control of memory reporting. In T. Perfect, & D. S. Lindsay (Eds.), Sage handbook of applied memory (pp. 481–500). London: Sage Publications. Greenwood, J. D. (1999). Understanding the “cognitive revolution” in psychology. Journal of the History of the Behavioral Sciences, 35, 1–22. Gronlund, S. D., Wixted, J. T., & Mickes, L. (2014). Evaluating eyewitness identification procedures using receiver operating characteristic analysis. Current Directions in Psychological Science, 23, 3–10. Gruneberg, M. M., Morris, P. E., & Sykes, R. N. (Eds.). (1978). Practical aspects of memory. London: Academic Press. Gruneberg, M. M., Morris, P. E., & Sykes, R. N. (Eds.). (1988). Practical aspects of memory. Chichester: Wiley. Hebscher, M., Barkan-Abramski, M., Goldsmith, M., Aharon-Peretz, J., & Gilboa, A. (2015). Memory, decision-making, and the ventromedial prefrontal cortex (vmPFC): The roles of subcallosal and posterior orbitofrontal cortices in monitoring and control processes. Cerebral Cortex, http://dx.doi.org/10.1093/cercor/bhv220, pii:bhv220 Herrmann, D. J. (1998). The relationship between basic research and applied research in memory and cognition. Autobiographical memory: Theoretical and applied perspectives, 13–28. Herrmann, D. J., Hertzog, C., McEvoy, C., Hertel, P., & Johnson, M. K. (Eds.). (1996). Basic and applied memory research: Volume 1: Theory in context; Volume 2: Practical applications. Mahwah, NJ: Erlbaum. Higham, P. A. (2007). No special K! A signal detection framework for the strategic regulation of memory accuracy. Journal of Experimental Psychology: General, 136, 1–22. Higham, P. A. (2013). Regulating accuracy on university tests with the plurality option. Learning and Instruction, 24, 26–36. Hollins, T. J., & Weber, N. (2016). Monitoring and regulation of accuracy in eyewitness memory: Time to get some control. In J. Dunlosky, & S. K. Tauber (Eds.), The Oxford handbook of metamemory (pp. 171–196). Oxford: Oxford University Press. Kelly, G. A. (1955). . The psychology of personal constructs (Vol. 2) Norton. Koren, D., Seidman, L. J., Goldsmith, M., & Harvey, P. D. (2006). Real-world cognitive—And metacognitive—Dysfunction in schizophrenia: A new approach for measuring (and remediating) more “right stuff”. Schizophrenia Bulletin, 32, 310–326.
FUNCTIONAL-COGNITIVE PERSPECTIVE
Koriat, A., & Goldsmith, M. (1996a). Memory metaphors and the real-life/laboratory controversy: Correspondence versus storehouse conceptions of memory. Behavioral and Brain Sciences, 19, 167–228. Koriat, A., & Goldsmith, M. (1996b). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103, 490–517. Luna, K., Higham, P. A., & Martín-Luengo, B. (2011). Regulation of memory accuracy with multiple answers: The plurality option. Journal of Experimental Psychology: Applied, 17, 148–158. Massin-Krauss, M., Bacon, E., & Danion, J.-M. (2002). Effects of the benzodiazepine lorazepam on monitoring and control processes in semantic memory. Consciousness and Cognition, 11, 123–137. McCain, K. W. (2015). “Nothing as practical as a good theory” Does Lewin’s Maxim still have salience in the applied social sciences? Proceedings of the Association for Information Science and Technology, 52, 1–4. Miller, G. A. (2003). The cognitive revolution: A historical perspective. Trends in Cognitive Sciences, 7, 141–144. Neisser, U. (1985). The role of theory in the ecological study of memory: Comment on Bruce. Journal of Experimental Psychology: General, 272–276.
19
Neisser, U. (1988). Time present and time past. In M. M. Gruneberg, P. Morris, & R. Sykes (Eds.), Practical aspects of memory: Current research and issues (2) (pp. 545–560). Chichester, England: Wiley. Nelson, T. O., & Narens, L. (1994). Why investigate metacognition. In J. Metcalfe, & A. P. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 1–25). Cambridge, MA: The MIT Press. Pansky, A., Goldsmith, M., Koriat, A., & Pearlman-Avnion, S. (2009). Memory accuracy in old age: Cognitive, metacognitive, and neurocognitive determinants. European Journal of Cognitive Psychology, 21, 303–329. Payne, D. G., & Conrad, F. G. (Eds.). (1997). Intersections in basic and applied memory research. Erlbaum: Mahwah, NJ. Skinner, B. F. (1974). About behaviourism. London: Jonathan Cape. Wells, G. L. (2014). Eyewitness identification probative value, criterion shifts, and policy regarding the sequential lineup. Current Directions in Psychological Science, 23, 11–16.
Received 18 November 2016; accepted 21 November 2016