TSUNAMI: Simultaneous understanding, answering, and memory interaction for questions

COGNITIVE

SCIENCE

18,

51-85

(1594)

TSUNAMI:SimultaneousUnderstcihding, Answering,and Memory Interactionfor Questions SCOTTP.ROBERTSON IBM Thomas J. Watson Research Center

Question processing involves parsing, memory retrieval, question categorization, initiation of appropriate answer-retrieval heuristics, answer formulation, and output. Computational and psychological models have traditlonally treated these processes as separate, sequentlol, Independent, and in pursuit of a single answer type at a time. Here this vlew is challenged and the implications of a theory in which question processes operate simultaneously on multiple question interpretations are explored. A highly Interactive model Is described In which an expectation-driven parser generates multiple question candidates, including partially-specified candidates. Question candidates act as constraints for a matcher which activates memory items. An answer retrieval process examines question candidates and the active portions of memory in an attempt to generate answer candidates. Answer candidates are examined by an output process that derives the final answer. These processes run simultaneously and interact. Three experlments on human question answering are also described which provide evidence that working memory load during question reading is affected by processes related to answer retrieval.

1. INTRODUCTION The processing of questions requires parsing, matching of antecedent concepts in memory, answer candidate retrieval, answer selection, and output. Researchers in this area have concentrated on the processes involved in determining and applying retrieval heuristics for various question types or the pragmatic constraints on answers (Dyer, 1983; Graesser, 1981; Graesser & Clark, 1985; Lehnert, 1977, 1978; Lehnert, Dyer, Johnson, Yang, & Harley, 1983; Singer, 1982, 1986). Largely as a simplifying assumption, these researchers have proceeded as if question parsing is an independent Correspondence and requests for reprints should be sent to Scott P. Robertson, IBM, Thomas J. Watson Research Center, Box 704, Yorktown Heights, NY 10598. 51

52

ROBERTSON

processthat comesbeforeany question-relatedretrievalor answerformulation processes. In this article an alternativeview of humanquestionanswering is proposedin which manyprocesses operatesimultaneouslywith parsing. In the first section,questionansweringtheory is briefly reviewed.In the secondsection,a new,parallel architecturefor questionunderstandingand answerretrievalis described.In the third section,experimentalevidenceis, presentedin favor of a parallelview of questionunderstandingandanswering. In theconclusion,connectionistandhybrid architecturesarecontrasted with the modelpresentedhereandimplicationsof a parallelview for existing question-answering theoriesare discussed. 1.1 Comprehension

and Retrieval

Questioncomprehensioninvolvestwo main components.Onecomponentis the location of memory elementsthat are specifiedby the questionpresupposition,or what Clark & Clark (1977)referredto asthe giveninformation in the question.For example,the question“Why did Petedrive to the store?” presupposesthat Pete drove to the store.A cooperativequestion askerwill only askthis questionof someonewho can reasonablybe expected to sharethepresupposition.Researchers assumethat the first stepin question answeringis extractionof the questionpresupposition.Lehnert(1977,1978) referredto the extractedpresuppositionas thequestion concept. Thequestion conceptis matchedagainstitemsin the questionanswerer’smemoryto find the referenceconcept.Oncefound, the referenceconceptservesasa source nodefor the more directedretrievalthat comesin responseto the question type. The secondcomponentof questionansweringis determinedby the question word and differs dependingon what is beingasked.Comprehensionof the questionword leadsto applicationof rulesspecificto the questiontype. “Why,” for example, signals a causal antecedent or goal questionand appropriaterules for searchingmemory for suchinformation are applied. “When,” on the otherhand,signalsa time questionandinvolvesthe activation and applicationof a different setof rules.Lehnert(1977,1978)referred to this stageof questionprocessingas question categorization. As an exampleof the main componentsof questionprocessing,consider 41-43 below. Ql. Whydid Jim go to the store? Q2.Whendid Jim go to the store? 43. Whydid thetreefall down? Ql and 42 sharethe samepresupposition(that Jim droveto the store),and thereforewill requireactivationof the samesourcenodein memory.According to most question answeringresearchers,question processingwill be essentiallythe samefor Ql and Q2 in the initial comprehensionstep that

QUESTION

PROCESSING

53

involvesparsing,generationof the questionconcept,and activation of the referenceconceptin memory. After this stepthe questionwords, W& and When, will triggerdivergentretrieval.processes assearchbeginsfor reasons or causalantecedentsin the caseof Ql and times in the caseof Q2. Thereis an interactionbetweenthequestionword andthe presupposition of a questionasa comparisonof questionsQl and 43 reveals.Becausethe presuppositionof Ql is the action of an animateactor, the why-question may be answeredby a goal or motivating state(e.g., “He wantedto buy bread” or “He washungry”). Becausethe presuppositionof 43, that a tree fell down, is an eventinvolving a nonanimateobject, the why-question‘is more likely to be answeredby a prior, causallyrelatedeventor state(e.g., “A carhit it” or “It hadbeendeadfor 20years”).Thus retrievalrulesmust utilize information gainedfrom both the questionword and characteristics of the questionpresupposition. Questionansweringbecomesevenmore interestingin situationswhere the answerinformation is not explicitly presentin memory or when many’ connectinginferencesarerequired.For example,eventhoughit is anunmotivatedevent,“Becausea treefell on Jim’s kitchen” is a reasonableanswerto Ql if oneassumesthat Jim had a plan that was blockedby this eventand decidedto achievehis goalanotherway. Similarly, eventhoughit is a psychologicalstate,“Becausethewoodsmanalwayshatedit” is a reasonableanswer to 43 if onemakesthe connectinginferencethat the woodsmancut the tree down. In suchsituationsknowledge-based inferencesand strategicproblem solving may be required. Retrieval heuristicsassociatedwith particular combinationsof questionwords and presuppositiontypes,and the generation of explanatoryinferencesin order to generatereasonableanswers,are topics that have beenexploredin great detail (Lehnert, 1978;Graesser& Franklin, 1990). Singer(1982,1984a,1984b,1986)has examinedquestionansweringin somedepth,proposinga seriesof modelsfor questioncomprehensionand memory retrieval. Singer’s VAIL model assumesthat there is an initial “sentenceencoding” stagethat correspondscloselyto encodingprocesses described in prior psycholinguistic research(Clark& Haviland,1977;Kintsch, 1974).This stageis followed by processes that extractthe giveninformation in the questionanddeterminewhetherthe questionis a verification (yes/no) or other (wh-)type of question.Input to the retrievalsystemis assumedto be a parsedrepresentationof the question,and reactiontime predictions undervariousquestionconditionsweremadebasedon processeshypothesizedto follow parsing.Singer’sreactiontime data wereconsistentwith his predictions,yet subjectsin thesestudieswerepresentedwith whole questions andreactiontimesweremeasuredfrom the onsetof questionpresentation to the beginningof answergeneration.Thus questioncomprehension times and answerformulation times werecombinedin Singer’sdependent

54

ROBERTSON

measuressuchthat thereis no way to judge if the processes Singerhypothesizedareaffectinganswerretrievaltime, asSingersuggests,parsingtime, as a parallel model might suggest,or both. Graesserand his colleagueshavebeenconcernedfor sometime with the interactionof questionprocesses with differenttypesof knowledgestructures and with the selectionof an answerfrom a set of candidates(Graesser& Clark, 1985;Graesser & Franklin, 1990;Graesser 8cGoodman,1984,Graesser BEMurachver,1984).Graesserhasstresseddescriptionof the symbolicproceduresthat areappliedin different knowledgecontexts.He points out that question-answeringresearchershave assumedthat retrieval goeson in a single,coherentknowledgerepresentation.In Graesser& Murachver(1984) and the QUEST model of Graesser& Franklin (1990)this work hasshifted toward an understandingof how diverseknowledgesourcesmight be utilized in answeringquestions. Graesser’smodelshavealsoassumedan initial questionparsingstepthat yieldsa questionconcept.Relevantgenericandepisodicknowledgestructures are identified from the questionconceptand other contextualfactors, and retrievaltakesplacewithin theseknowledgestructures.As in othermodels, question categorizationoperatesto accessrules relevant to a particular questiontype.Rulesassociated with the questiontypearethenappliedwithin the activatedknowledgestructuresandmay identify many candidateanswer concepts.A convergence mechanismnarrowsthecandidateconceptswithin the activatedknowledgestructuresand appliesconstraintsto further determine “good” answers.Finally, pragmaticconsiderationsarebrought into play to determinean appropriateanswer. Graesser’smodel explainssubjects’ratings of answerquality and many memory phenomenarevealedby questionanswering.The QUEST model has interactivecomponents,and Graesser,Lang, & Roberts (1991)have recentlystudiedthetime courseof questionansweringin the QUEST framework. However,the model is still discussedas if it operateson conceptual input derivedafter parsing. Graesser’smechanismswould be good candidatesfor examinationin the contextof parallelcomprehensionandanswering models. 1.2 Siultaneous

Question Understanding

and Answering

As we haveseen,modelsof questionprocessinghaveusuallybeenpresented as a seriesof stages.Each processrequiredfor questionprocessing-presuppositionextraction,sourcenodeactivation,rule activation,rule application, search,answerpruning,andinference-hasbeenconsideredseparately. It is interestingthat modelsof questionansweringhavenot exploredhighly interactivemechanismsfor questionparsingand answerretrievalsincethis hasbeena lively debatein theliteratureon word andsentence comprehension for sometime (Altmamr 8cSteedman,1988;Anderson,1976,1983;Anderson & Bower, 1973;Collins & Loftus, 1975;Gorrell, 1989;Just & Carpenter,

QUESTION

PROCESSING

55

1980, 1987; Kintsch, 1988; Marslen-Wilson, 1989; Marslen-Wilson &Tyler, 1980; McClelland & Kawamoto, 1989; McClelland & Rumelhart, 1981; Miikkulainen & Dyer, 1991; Perfetti, 1990; Rumelhart &McClelland, 1981, 1982; St. John & McClelland, 1990; Taraban & McClelland, 1990; Waltz & Pollack, 1985). In fact, debate in the sentence processing literature now centers less on whether structural and semantic processing interact, and more on how strong the interaction is (Altmann, Garnham & Dennis, 1992; Altmann & Steedman, 1988) or how autonomous the two processes are (Perriera & Clifton, 1986; Perfetti, 1990; Taraban & McClelland, 1990). Just and Carpenter’s “immediacy hypothesis” (1987) proposes that syntactic and semantic knowledge is constantly being applied to the input during sentence comprehension. In their view, multiple interpretations of a sentence’s meaning are generated during reading and each new word adds constraints to the interpretation of a sentence. Kintsch’s (1988) “construction-integration” model of sentence processing proposes that, in an initial “construction” phase of interpretation, a semantic network is generated to represent the concepts conveyed in a sentence. The construction phase is followed by an “integration” phase during which nodes are assigned weights according to constraints derived from the context. In light of the fact that models of sentence comprehension routinely employ interacting syntactic and semantic interpreters, it is time to update question answering models by adding an interacting answer retrieval component. Data and models are present in the literature that suggest parallelism in simple query or query-like situations. Anderson (1974) provided evidence for spreading activation in a semantic network during sentence verification, a task closely related to verification question answering. Shastri (1988, 1989) presented a model in which fact queries, including recognition and property inheritance questions, are answered using a weighted semantic network with parallel, spreading activation. A query is represented in Shastri’s model as a node network that affects activation levels in a semantic network in a manner that is specific to each query’s content. “Response nodes” are activated by application of the query to the knowledge base, and the response node with the strongest weight becomes the answer to the question. Interestingly, even in this context Shastri describes question answering as a “three step process” involving 1. query handling to set up activation in the memory network, 2. spreading activation, and 3. competition among response nodes. Miikkulainen and Dyer (1991) have incorporated a question answering module into a connectionist comprehension model that operates on a distributed representation, but the model deals only with yes/no questions. Most work to date on parallelism in question answering has dealt with recognition, likelihood, or verification queries in the context of connectionist

56

ROBERTSON

models.Thesequeriesarerelativelyeasilyinterpretedand matchedto facts in the knowledgebase.In thecontextof a modular, nonconnectionistmodel designedto dealwith a wider rangeof questiontypes,a strict serialview of question answeringhas only recently been challengedby a parallel view (Robertson,Ullman, & Mehta, 1992;Robertson8cWeber,1990;Robertson, Weber,Ullman, & Mehta, 1993).In the following sectionswe elaboratethe model, providea walkthrough,andbring togetherdata from severalexperimentsthat supportits basicstructure. 2. A NEW QUESTION-ANSWERING

ARCHITECTURE

In this sectionan architectureis proposedwhich will supportquestioncomprehensionin a framework that allowssimultaneousquestionparsing,activationandapplicationof retrievalheuristics,answerformulationandoutput. The model is referredto as “TSUNAMI’,” for “Theory of Simultaneous UNderstandingAnsweringand,MemoryInteraction,” andit differs dramatically from existingproposalsfor symbol-based question-answering architectures(i.e., Dyer, Graesser,Lehnert,Singer).At this point the model is offeredasa broadarchitecturefor supportinghighly interactiveapplicationof the mechanismsalreadyidentified by questionansweringresearchers. It remainsto be seen’howthe natureof thesemechanismswill change,and what new mechanismsmight be necessary,whenimplementedin the TSUNAMI framework. 2.1 Structure of the Model

The TSUNAMI model is depictedin Figure 1. In the Figure,processesare indicatedby ovalsandmemoriesareindicatedby squares.Thereareseveral separatememorycomponentsin themodel, howeverthereis no strongclaim that they areseparatefrom eachother. Rather,the different memorystores are really claims about different typesof data with which the variousprocessesareconcerned.Arrows pointing to a memoryfrom a processindicate that the processgeneratesdata of that type and can modify data of that type. Arrows pointingto a processfrom a memoryindicatethat the process usesinformation of that type. Processesand memoriesmay interactin dif‘ferentwaysand on different problems(e.g.,the parsermay bebuilding two parsingstructuresat once,or a retrieval processmay be utilizing several structuresin episodicmemoryat the sametime). Thereareno directlinkages betweenprocesses,rather, in the spirit of “blackboard” models(Errnan, Hayes-Roth,Lesser,& Reddy, 1980;Hayes-Roth,1985),the behaviorof eachprocessdependson the stateof the memoryor memorieswith which it I A ‘Ymumi” is a tidal wavethat is causedby an earthquakeunder the ocean.

QUESTION

PROCESSING

57

QUESTION Question

Semantic Knowledge

I

Answer candidates \ 0

A00

A+

o 0%

1 ANSWER Figure radically

1. The TSUNAMI model of question wlth current serial models.

answering.

The parallel

architecture

contrasts

interacts.Also, like blackboardmodels,itemsstoredin a particularmemory may be inspectedand alteredby severalprocessesoperatingon it at once. Thus, thereis no formal “flow of control” to the model. Instead,the processingbehavioris a function of the input, the statesof the memories,and the speedof the processors. Figure 1 distinguishesbetweensemanticand episodicknowledgeand is predicatedon a situation in which questionsare askedabout a particular event(e.g.,a story or a personalmemory)that would be represented in episodic memory.Two specialpurposeworking memory componentsare also

58

ROBERTSON

isolated. One memory stores question candidates, or possible interpretations of the question. The other stores answer candidates, or potential answers to the question candidates. The candidates are represented as propositions and may be partially specified. The question candidate memory and answer candidate memory are the only knowledge structures that take output from processes in the model. The influences of processes on these memories might be to add propositions, update proposition contents, or delete propositions. While in the model we take the position that the question and answer buffers are separate memories, this is not a strong architectural claim. In fact, they may all be part of a single working memory with data elements tagged in some way that discriminates questions from answers. Either view is consistent with the model as presented and with the experimental data in section 3. The parser takes input from the question and utilizes grammatical, case, and pragmatic information in semantic memory to begin producing various propositional representations which are then stored in the question candidate buffer. The position taken by the TSUNAMI model is much like that of Just and Carpenter (1987) in that each word provides information about possible interpretations of the sentence. Multiple arrows from the parser into the question candidate buffer suggest that the parser can produce many candidates in response to the input at any given time. As words come into the parser, prior candidate structures may be updated or disconfirmed and deleted by the parser. As soon as there is any information in the question candidate buffer, a matcher begins comparing question candidate structures with activated information in episodic memory. The goal of the matcher is to find the question presupposition which will ultimately form the source node from which retrieval begins. The matching process occurs in parallel for several candidates as indicated by multiple arrows from both the question buffer and episodic memory into the matcher. It is reasonable to conceptualize the matcher in a manner similar to the connectionist fact retrieval mechanisms described by Shastri (1988, 1989) or Miikkulainen and Dyer (1991). Evidence presented in section 3.2 suggests that the speed of the matcher is influenced by the number of candidates and the number of matches. When the matcher finds a proposition in episodic memory that corresponds to a question candidate, then this is identified as a likely source node for answer retrieval processes. It is proposed that a presupposition match in episodic memory raises the activation level of this and related information and makes it more available to future analysis by any other processes that utilize episodic memory. This influence of the matcher on episodic memory is indicated by an arrow from the matcher into episodic memory. The presupposition matching process may also influence the contents of the question buffer. If there are multiple question candidates, then the iden-

QUESTION

PROCESSING

59

tification of a matching presupposition for one will lessen the influence of the others in further processing. Partial propositions in the question candidate memory can be matched to complete propositions in memory, thereby updating the question candidate list to include expectations. Influences of the matcher on question candidates are indicated by an arrow from the matcher into the question candidate memory. As soon as there is information in the question candidate buffer that might suggest a question or question type, the answer retrieval process can begin. This mechanism examines question candidates, attempts to determine question categories appropriate to the various candidates, and begins applying retrieval heuristics in episodic memory. The answer retrieval mechanism utilizes question answering rules stored in semantic memory and other, relevant general knowledge. This process operates simultaneously with the parser and the matcher, but is dependent on the contents of the question candidate memory and the activation states of nodes in episodic memory. The answer retrieval mechanism operates to highlight relevant portions of episodic memory. If one of the question candidates is a goal-orientation question, for example, then the answer retrieval mechanism may activate goal hierarchies in the episodic trace. On the other hand, if a causal-antecedent question candidate is present in the question buffer, then causal sequences may become more active. Both types of questions may be present in the buffer, especially early in question.parsing, in which case the retrieval mechanism will activate both types of knowledge structures and initiate appropriate retrieval heuristics in each. The heuristics utilized by the answer retrieval process are those already identified by question-answering researchers for various question types, however, their implementation may need to be augmented to deal with differential weights of information in episodic memory, likelihoods assigned to different question candidates, constraints imposed by incoming information, and potential interactions among different heuristics. The outputs of the answer retrieval process are candidate answers to questions that the system finds consistent with the input and memory and any given time. Potential answers produced by the answer retrieval process are stored in the answer candidate byffer as propositions. These propositions may also be partially specified, and are subject to subsequent modification and deletion by the operation of the answer retrieval process. For example, if a question candidate that spawned a retrieval process is later disconfirmed, then the answer candidate built by that process will be deleted by the answer retrieval process. The answer candidates are examined by an output generation process which also utilizes grammatical, case, pragmatic, and relevant general knowledge in semantic memory. This process can influence the answer candidate set. For example, if pragmatic concerns dictate that an answer is inappropriate, then the output generation process will delete it from the answer

60

ROBERTSON

An Examole John

woke

John John

was hungry. wanted some

John

drove

Mary

saw John

Storv

up early.

food.

to the store. at the store.

Mary wanted to say hi to John. She waved at him. John

bought

oatmeal.

John dropped the oatmeal outside because the parking lot was slick. It spilled John Later

in the trunk.

put it back he cooked

and ate it.

in the container. It,

TABLE 1 and Associated

Propositional

Analysis

Pl. (ACT wake-up, John) P2. (TIME early, Pl) P3 (STATE hungry, John) P4. (ACT eat, John, food) P5. (REL reason, P4, P3) P6. (ACT drive, John, store) P7. (REL reason, P6, P4) PB. (ACT see, Mary, John) P9. (STATE locatlon, PB, store) PlO. (ACT greet, Mary, John) Pll. (ACT wave-at, Mary, John) P12. (REL reason, Pll , P10) P13. (ACT buy, John, oatmeal) P14. (REL reason, P13, P4) P15. (EVENT drop, John, oatmeal) P16. (STATE location, P15, outside) P17. (STATE slick, parking lot) PlB. (REL cause, P15, P17) P19. (EVENT spill, oatmeal, trunk) P20. (REL cause, P19, Pls) P21. (ACT put, John, oatmeal, container) P22. (ACT cook, John oatmeal) P23. (REL reason, P22, P4) P24. (ACT eat, John, oatmeal) P25. (REL reason, P24, P4)

buffer. Finally, whenonecandidateremainsin the answerbuffer and all of the questionhasbeeninput to the parser,the final answeris formulated. It is reasonableto assumethat theoutput generationmechanismwill not commit to a final interpretationuntil all of the input hasbeenprocessedsincea final phraseon a questioncan changeits focus, and hencethe appropriate answer,tremendously. 2.2 Walkthrough In this sectiona traditional approachto questionprocessingis contrasted with TSUNAMI in an examplewalkthrough. The comparisonis meantto illustrate generalpoints and so many details are necessarilyglossed.The walkthrough,however,shouldgivea clearpictureof how drasticallydifferent previousserial models and the TSUNAh41model are. For the walkthrough the story in the left-hand portion of Table 1 will be used. Following Kintsch and other researchersin text comprehension,I claim that thelist of 25 propositions(designatedasPl-P25) in the right-handportion of Table 1 capturessomethingof the meaningof the explicit detailsin the text. In this list, eachpropositionhasa typeindicatorin capitallettersin

QUESTION

PROCESSING

61

the first position. Type indicators are important for questionanswering sincethe determinationof questioncategoriesoften dependson the type.of propositionbeingqueriedandsincesearchis oftendirectedalongnodesof a particular type. In this examplethereare node types for states(STATE), goal-directedactions(ACT), and parts of causalsequences (EVENT). The nodetype REL indicatesa proposition linking two other propositionsby virtue of some relation. The predicateof eachproposition is containedin the secondargument.As in most propositionalanalysesof storiesthe predicatesbasicallycorrespondto verbsand modifiers. The remainingslots of the propositionscontain argumentsto the predicates,for exampleactors and objects.The predicateof an REL nodeis the nameof a relation. In the examplethere are reasonpredicatesfor intentional relationshipsbetween goalsand actions/subgoals,and causepredicatesbetweencausallylinked events.The final two argumentsto REL nodesarethe identificationsof the linked nodes. I make no strongclaims about the representationalsystemor notation, but useit only for explicationin this section.Note that the propositionlist in Table 1 containsimplied linking propositionsabout goal relations(P5, P7, P14, P23, and P25 for John, and P12 for Mary) and causalrelations (P20). Considerableresearchon story comprehensionhas suggestedthat suchinferencesareconstructedduringreading(Long, Golding, & Graesser, 1992;Seifert,Robertson,& Black, 1985)andthat theyprovideconnectivity amongpropositionsin the internal representationwhich is useful in recall and reasoningabout characteractions (Graesser& Clark, 1985;Kintsch, 1974).Also, intentional actions and non-intentionaleventsare explicitly distinguishedby the notation, as Graesser(1981,1985)might haveit, but I leavethe possibility openthat this distinction might be madein a different manner. 2.2.1 Traditional

Serial Models

In the contextof this story, considerthe question“Why did John drive to the store?,”andthe derivationof a reasonableanswer,“To getsomefood.” In serialmodelsof questionanswering,the presuppositionof the question would first be derivedfrom a completeparseof the input sentencewhich we might representasthe proposition44: 44. (ACT drive, John, store)

A questioncategorizationmechanismwould nextdeterminethat the “Why” word suggestseithera causalantecedentquestioncategoryor a goalorientation questioncategory.Examinationby this mechanismof the questionpresuppositionrevealsthe facts that “drive” is a motivated action and that “John” is an animateactor, eitherof which would dictatethat the question

62

ROBERTSON

category is goal orientation. After question categorization, further analysis of the pragmatics of the situation or language conventions helps to focus the query and might even alter the question categorization. For example, if the question answerer knows that John lives next door to the store, then the question is about why he drove rather than walking. If the question answerer knows that Mary always does the shopping, then the question is about why John was the one who went to the store. In this case, context offers no alternative foci so the emphasis is assumed to be on John’s goal. The question concept and question category are passed to a process that will locate the question concept in memory and apply appropriate retrieval heuristics. Thus, the next step is to match 44 against the propositional representation in Table 1, thereby activating P6 as a source node for retrieval. Retrieval rules specify that propositions related to a source node via a reaSOn predicate will be possible answers to a goal orientation question. Further, additional candidates can be generated by following a sequence of reason predicates up a goal hierarchy. In our example, retrieval rules would attempt to find a proposition matching the following retrieval pattern: (REL reason, P6, ?P), where ?P is a variable. between P6 and another, sition bound to ?P would question pattern, binding anism to P4:

The pattern describes an explicit reason relation as yet unknown proposition. If found, the propobe an answer candidate. P7 in Table 1 matches the ?P to P4 and therefore leading the retrieval mech-

P7. (REL reason, P6, P4), matches the retrieval pattern. P4. (ACT eat, John, food), is an answer candidate and a new source node. Thus, “John wanted to eat some food” is a potential answer. The retrieval mechanism can continue to follow the goal hierarchy, searching for a reason predicate involving P4. The retrieval pattern for such a search is as follows: (REL reason, P4, ?P). PS matches this specification and leads to P3: PS. (REL reason, P4, P3), matches the retrieval pattern. P3. (STATE hungry, John), is an answer candidate and a new source node. Thus, “John, was hungry” is another potential answer. There are’no”further reason links (i.e., there is no match to (REL reason, P3, ?P)), and so two answer candidates are activated. Many considerations about the appropriateness, or “goodness,” of the answers may now be applied in order to choose one (Graesser & Clark, 1985; Graesser & Franklin, 1990). When one answer is finally determined, an output mechanism formulates the linguistic response.

QUESTION

-~

“Why

Time

lneut

t1

Why

did John

drive

15

TABLE 2 of TSUNAMI’s Processing of the Question to the Store?” In the Context of the Story

Walkthrough Did John Drive

Questlon Condldotes CI~I.(REL

reason,

C2n.(REL

cause,

?Pa,

in Table

Memory Nodes ?Pb)

?Pc, ?Pd)

(no change) Cl a.(REL C3a.(ACT

63

PROCESSING

Answer Candidates

PS P7, P14, P23, P25 P12 P18 P20 (no change)

reason, C3n, ?Pb) ?a, John, tobiectl)

P5-P4 P7-P6, P14-P13, P25-P24

1

P3 P4 PlO P17 P15 (no change)

P23-P22,

P3 P4

C2s.(REL cause, C4ti, ?Pd) Gin.(EVENT ?e, John, 7object2)

Pl&P15

P17

Cl 14.(REL reason, C~M.(ACT drive,

P7-P6

P4, P3

P7-P6 P5-P4

P4 P3

Clts.(REL C31s.(ACT C%.(ACT

C3t4,

John,

?Pb) ?ob/ectl)

reason, C3ts, CUB) drlve, John, store) eat, John, food)

Note that in analysisof answerretrievalonly a few of the conceptsin the storywereactivated.Specifically,P6, the sourcenode,qd P3, P4, P5, and P7. A questionansweringsystemthat waits until a completerepresentation of the questionis obtainedbeforeretrievalbeginsmaximally constrainsthe searchprocess. 2.2.2 The TSUNAMI

Model

Now let us considerhow TSUNAMI would handlethe samequestion,“Why did John drive to the store?” In many waysthe decisionrules andretrieval heuristicsare similar, but retrieval beginsimmediately basedon partial understandings of the questionasit is parsed.Becauseweassumethatretrieval andparsingoccur together,the walkthroughwill be consideredin a wordby-wordmanner. In the TSUNAMI example,we will needto specifythe statesof the questionand answerbuffers, the activity in story memory, and the operationof someprocessesas eachword is input. Table 2 showsthis activity at variousstagesin parsing.Table2 is organizedby time slices,indicatedastl-t5, which showthe stateof the questionansweringprocessafter eachword is processed.The six columnsof Table2 showthe time slice,the input word, the contentsof the questionbuffer, the activememorynodes,

64

ROBERTSON

and the contentsof the answerbuffer. Examination of Table 1 and Table 2 togetherwill be helpful in following the discussion. It is proposedthat the parsermay placepatternsin the questionbuffer which are consistentwith likely interpretationsof the input at any point. This view is consistentwith the “immediacyhypothesis’of JustandCarpenter (1980,1987)and contrastswith approachesthat allow semanticprocesses to operateonly whenthe syntacticprocessoris not occupied,for example,at clauseboundaries.By allowingeachword to add constraintsto the question interpretation,the retrievalprocesscannarrow its searchearlier. The cost is to initiate fruitlesssearchesearly on, but we provideevidencein section3.1 that this cost is outweighedby the headstartof the retrieval processthat ultimately “wins.” The matcherand answerretrievalprocesses will beginto useinformation in the questionbuffer assoonasit appears.First the questionword “Why” is inputto the parser,as the input columnof Table2 showsat time t 1. Already, thereare two possiblepatternsthat might be placedin the questionbuffer. Thesequestioncandidatesaredesignatedas Cl and C2. Sincethe form of the questioncandidatepropositionswill change,Cl and C2 are further specifiedwith the subscripttl to indicatethe time: Cltl. (REL reason, ?Pa, ?Pb) C2tl. (REL cause,?Pc, ?Pd)

The nodedesignationsappearat time tl in the question candidates column of Table2. Both questioncandidatesarepropositionsthat specifya relationship betweentwo nodes.Cltr specifiesa reason relationwhile C2trspecifies a causal relation.Thesepatternscorrespondrespectivelyto the goalorientation andcausalantecedentinterpretationsof “Why” questions.The symbols ?Pa,?Pb, ?Pc,and ?Pdstandfor the propositionsthat areconnectedby the reason and cause relations.The symbolsare precededby questionmarks to indicate that they are variableswhich have not yet beenspecifiedby the input or matchedto any memory items. In general,in responseto a questionword, all of thequestion-typepatterns that are highly associatedwith the questionword will be generatedby the parserand usedby the matcherandretrievalprocess.Nonetheless,it is possiblefor the matcherto begina comparisonwith memory,makingall possible matchesto the variables.Cltl andC&I matchall reasonandcausepredicates that are activein memory. Sucha comparisonrevealsthat Cltr matchessix propositionsin memory, namely PS, P7, P12, P14, P23, and P25. C2ti matchestwo propositions,namelyP18 and P20. If P18 and P20 werenot presentin the story (e.g.,if the oatmealdroppingincidentwereeliminated) thenit would be possibleto rule out the causalantecedentinterpretationof the questionin this casebeforeseeinganymore of the question!Onceruled out, the matcherwould removeC2ti from memory and this patternwould play no more role in processing.

QUESTION

PROCESSING

65

Operation of the matcher has resulted in increased activation of eight nodes among the 25 nodes which form the story representation in episodic memory. This increased activation makes those nodes more likely to be used by retrieval processes, much like the activation level of nodes in Anderson’s ACT* model (Anderson, 1983) affect the speed of production rule matching. It is possible for the answer retrieval mechanism to begin examining these nodes at this stage. In the representational scheme in Table 1, the superordinate member of a pair of nodes related by a reason link can be found in the last slot of a reason proposition. Thus, the answer retrieval mechanism will tag all matches to ?Pb among the active memory nodes and treat them as potential answers. Matches to ?Pb for each of the six activated reason nodes are as follows: the P5 node matches ?Pb to P3 (“John was hungry”), the nodes P7, P14, P23, and P25 all match ?Pb to P4 (“John wanted some food”), and the node P12 matches ?Pb to PlO (“Mary wanted to say hi to John”), The matched nodes appear in the answer candidate column of Table 2 at time tl . There is an interesting empirical issue here concerning whether P4 should be a “stronger” answer candidate since it is pointed to by several propositions. A priming study could be designed to examine such an issue. Causal antecedents can be found in the last slot of propositions with cause predicates. These match to ?Pd in C2t1. The answer retrieval mechanism will tag all matches to ?Pd among the active memory nodes and treat them as potential answers. Matches to ?Pd for the two activated cause nodes are as follows: the Pl8 node matches ?Pd to P17 (“The parking lot was slick”), and the P20 node matches ?Pd to P15 (“John dropped the oatmeal”). These nodes also appear in the answer candidate column of Table 2. From this example we see that it possible to identify two potential question types and generate five unique answer candidates even at the first word of the question in this example. The ability of the question answering system to move this far on the first word would depend on available resources. It seems unreasonable to assume that this much information could be held in memory at once or that this much processing could go on in the brief time it takes to read a familiar word like “why.” However, the system should be expected to move as far as possible at each step (Just & Carpenter, 1987). The values of various memory and processing parameters will be left to future empirical investigation. At t2 “did” is input to the matcher. While the parser performs a syntactic analysis of this word, it plays no role in updating the question or answer buffers since it contributes no new constraints on the interpretation of the question. In the time it takes to read the word, however, processes that were set in motion during reading of the first word but which were not finished by the time the second word appeared could be continuing. At t3 “John” is input to the parser. John could be a subject, as in “Why did John hit a tree?,” or an object, as in “Why did John get hit by a tree?” If we make the assumption that the parser follows the path of least resistance,

66

ROBERTSON

building the simplest structural representationof the sentence(Frazier, 1987),then “John” is treatedasthe likely subject.In this case,the proposition beingqueriedis likely to be an actionwith Johnasthe actoror an event with John as the subject.To representtheseconceptsthe parseraddstwo new candidatesto the questionbuffer at time t3, C3t3and C43: C3ts.(ACT ?a,John,l?objectl) C&s.(EVENT‘?e,John,?object2), and updatesClts and C2t3 to refer to the new propositions: Chs. @ELreason,C30,?Pb) C2t3.(RELcause,C4t3, ?Pd). The updatedcontentsof the questionbuffer are indicatedin the question candidatecolumn in Table 2 at time t3. The matcherrespondsimmediatelyby activatingall pairs of propositions in episodicmemorythat canbe matchedto the newconstraintsof the question buffer. The new constraint on the actor slot of C3tsresultsin the elimination of P12 as an item activatedby the matcher and PlO (“Mary wantedto sayhi to John”) as an answercandidatesincethe actor in PlO is Mary. For the same reason,,P19fails to match C&s and is eliminated, resulting in the removal of P15 (“John droppedthe oatmeal”) from the answercandidatebuffer. Remaining memory items that match the constraintsof the questionbuffer at time t3 arethe action pairsPS-P4,P7-P6, P14-P13,P23-P22,and P25-P24.Also remainingis the eventpair P18-P15. Answercandidatesthat remainat time t3 andP3 (“John washungry”), P4 (“John wantedsomefood”), and P17 (“The parking lot wasslick”). At time t4, “drive” entersthe parserandC3t4is updatedin the question buffer to reflect identification of the action: Clu. (RELreason,C3t4,?Pb) C2t4.(ACT drive,John,?objectl) Up to this point there has still beena reasonableexpectationof the event question“Why did John drop the oatmeal?,” represented by C& and C2ts. However,this interpretationcan not be reconciledwith the verb “drive,” andsoit is droppedfrom the questionbuffer. This deletionresultsin deactivation of the P18-P15nodepair in memory and the removalof P17 (“The parkinglot wasslick”) from the answerbuffer. At time t4 a comparisonof thecontentsof the questionbuffer with story memoryshowsonly oneway that the constraintsof the questionbuffer can be satisfied.Specifically,C3t4matchesP6 (“John droveto the store”) and Clt4matchesits parentnode, P7. The matchercan now reducethe activation level of all the other pairs of propositionsand give all resourcesto P6 and P7. The match of C3t4to P6 unambiguouslybinds the missingobject

QUESTION

PROCESSING

67

slot (?objectl) to “store.” Also the match of Clu to P7 unambiguously binds to ?Pb slot of Clt4to P4. Since?Pb in Clt4is boundto P4, this is addedto the questionbuffer at t5 as CSu and Clts is updatedto reflect the embeddedrelationship. These changes,and the matched?objectl slot in C3t4,arereflectedin the question candidatecolumn of Table2 at time t5: Clts. (RELreason, C3t5, CSts) C3t5.(Actdrive,John,store) C5ts.(ACT eat,John,foot) The answerretrievalmechanismrespondsto the currentconfigurationof the questionbuffer by eliminatinganswercandidatesthat can not bederived from P6 by reasonlinks. P4 remainsas an answercandidatebecauseit is relatedto P6 by virtue of the direct matchin P7. P3 remainsasanswercandidatebecauseit is relatedto P6 indirectly through P4. The parser continuesto receiveinput from the question, but unless future interpretationscontradictthe expectationgeneratedby the matcher therewill be no more influenceon the questionbuffer. If the questionbuffer doesnot change,thenthematcherwill not operatebeyondthis point and this freesresourcesfor other processes. As soonaspropositionsappearin the answercandidatebuffer their appropriatenesscanbe evaluatedby the outputgenerationmechanism.As with the otherprocesses,the operationof this mechanismis affectedby the number andstrengthof answercandidates,the resourcessharedwith otherprocesses, the complexityof the pragmaticsof the interaction, and the complexityof the answercandidates’contents.The number of answercandidatesvaried from five at time tl, to two at time t5. If resourceswereavailable,the output generationmechanismcould be evaluatingall five answercandidatesat time tl, applying “goodnessof answer” criteria to all of them. It is reasonableto assume,however,that this mechanismmight bedelayeduntil a small numberof candidatesare beingconsidered. The two answercandidatesidentifiedby time t5 remainviablecandidates afterthe completequestionhasbeenreadand,indeed,eitheris a reasonable answerto the question.Perhapsthe questionanswererwould decidethat P3 is better than P4 accordingto some obviousnesscriterion since “buying food” is a more obvious,default goal than beinghungry. Sucha criterion could be appliedas early as t4 in our examplesincethe final answercandidatesarepresentat this time and other processesand memory resources havebegunto requirelesscapacity.The final answercould be ready,then, just after readingthe verb in our example. The TSUNAMI model is expectation-drivenat everystep.The parseris predictingpossibleinterpretationsof the input and the answerretrieval mechanismis predictingpossibleanswercategories.The matcherwill often

68

ROBERTSON

find severalnodesin episodicmemory that are consistentwith partially specifiedquestioncandidates,andthusproduce“guesses”at questioncompletions. The output generationmechanismmay be ready to give several answersat any given time. Becauseany processcan operateat any time, control is exertedthroughthe contentsof the memoriesandbuffers. This is a radical departurefrom previousquestion-answering modelsin which control is exertedthrough the serialnature of the architectures. The walkthrough should be consideredas an example of possible TSUNAMI behavior.The actualbehaviorof the model will vary depending on many factors, for examplethe speedof input to the parser,the form of the question being input, the intentions of the question answerer,the memoryload from thespecificcontentsof theepisodictracebeingexamined. In the next sectionwemanipulatesomeof thesefactorsand makedifferential predictionsaboutthe behaviorof a model like TSUNAMI in contrastto serial modelsof questionprocessing. 3. EXPERIMENTS

In this sectionseveralexperimentsarediscussedwhich provideevidencethat answer-relatedprocessesare operatingduring questioncomprehension.In all of the experimentssubjectsreadand answeredvisually presentedquestions about short stories.The experimentsutilize severalconvergentmeasures of mental workload, or the demandplacedon cognitiveresourcesby on-goingprocesses andthe extentof activememory. Thereis an assumption in the TSUNAMI model (and most cognitivemodels)that the variousprocessescompetefor resourcessothat the more thereis for any oneprocessto do, the slowerall processes will operate.This demandshouldbeapparentin slowedreadingtimesor slowedreactiontimesto secondarytaskslike lexical decision. In the experimentsto follow, increasedworkload is shownduring parsing whenthe questiontypeis known vs. unknown, whenthe answermay be uniquely identified early vs. later, and whenthe subjects’intent is to provide an answervs. a paraphrase.In addition, in most casesthe addedworkload during parsingis accompaniedby a savingsin answeringtime at the end of the question.This suggeststhat the workload is in fact associated with answerretrieval.Finally, in all of theexperimentsthe workloadeffects differ with questiontype, againsuggestingthat theyarerelatedto questionspecificretrieval processes. 3.1 Position of the Query Component

of Questions

In a seriesof experimentsreportedin detail in Robertson8cWeber(1990) and Robertson,Weber,Ullman, & Mehta (1993)the position of the question componentrelativeto the presuppositionof a questionwasmanipulated

QUESTION

PROCESSING

69

and effectson readingand answeringtimes weremeasured.Considerthe following questionsQ5 and46 in which the questionword appearseitherat the beginningor end of the main part of the question. QS. For what reason did Pete go to the store? 46. Pete went to the store for what reason?

WhenansweringQS, the readerhas early knowledgeof the questiontype and so, if the human questionansweringsystemis designedfor it, answer retrievalheuristicsmight be activatedand evenappliedduring readingof thepresupposition.In 46, however,the questiontypeis unknownuntil the lastword. Thus Q5 forcesa stagedprocessin which sourcenodeactivation precedesdirectedretrieval. If comprehensionandretrievalprocesses arestagedduring normal question answering,then thereshouldbe no differencein the readingand answer retrievaltimes betweenQ5 and 46 sinceboth would require sourcenode activation followed by searchrule application and retrieval. If retrieval beginsearlyin Q5, however,thenreadingtime for the wordsmight increase relativeto 46 sinceretrieval and comprehensionresourcesarecompeting. Also, thetime to generatean answerafter readingthe last word of QSmight be fasterrelativeto 46 sinceretrievalhas a headstart in the former case. Two experimentswereconductedin this paradigm.In eachexperiment, subjectsread a sequenceof short storiespresentedon a computerscreen. Eachstory was followed by a questionwhich waspresentedword-by-word at thesubjects’normal readingratein responseto self-pacedkeypresses that revealedeach new word. Subjectsansweredverification questions,goal questions,and time questions. In the first experiment,the questionswerepresentedasa singlequestion word and an assertion,and the order of thesetwo questionelementswas varied.In the question-first(Q-first)condition,subjectssawquestionsof the form “Question-word?+ Presupposition,”and in the question-last(Q-last) conditionsubjectssawquestionsof the form or “Presupposition+ Questionword,” where the question words were either “True?,” “Why?,” or “When?,” andthe presuppositionwasan activesentence,that is, Noun1+ Verb+ Preposition+ Determiner+ Noun2. For example, a subject might haveseen“Why? John went to the store.” or “John went to the store. Why?” In the secondexperiment,the questionwords were replacedby phrases.In the question-phrase-first(Qp-first) condition subjects saw “Isn’t it truethat. . . ,” “What wasthe reasonthat. . . ,” and “At what time did. . . ,” before the questionpresuppositions.In the question-phrase-last (QJ’-last)conditionsubjectssaw$‘. . .isn’t it true?,” “. . .for whatreason?,” and “ ... .at what time?)’ after the questionpresuppositions.Subjectsread the questionsoneword at a time at their own paceby pressinga key to see eachnewword. Their readingtimes weremeasured.

70

ROBERTSON TABLE 3 the Three Questlon Types and Two QuestionIn the Query-Position Experiments. Reading for the Words in Bold Type. Answering Times from the Words in Underlined Bold Type.

Example Stimuli for Position Conditions Times Were Collected Were Colculated Example

Verification

Items

Q-first QP-first

True? Pete Isn’t It true

drove to the store? that Pete drove to the

Q-last

QP-last

Pete Pete

to the to the

Example

Reason

Q-first QP-first

Why? What

Pete drove to the store? was the reason that Pete

Q-last QP-last

Pete Pete

drove drove

Example

Time

Q-first QP-first

When? Pete drove What was the time

to the store? that Pete drove

Q-last QP-last

Pete Pete

store, store

drove drove

rtore, store

store?

True? isn’t it true?

Items

to the to the

store, store

drove

Why? for what

to the

store?

reason?

Items

drove drove

to the to the

When? at what

to the

rtore?

time?

Dependentmeasuresin all of the experimentswereword-by-wordreading times for the presuppositionsand “answer time.” Answer time was measuredas the time to read the questionword or parts of the question phraseplus time to readthe last word of the presupposition.Table 3 shows the sourceof readingand answeringtime dependentmeasures.Note that the answeringtime alwaysincludestime to read the sametwo words plus time beforepressingthe responsekey “when the answercomesto mind.” Figures2 and 3 show the mean word-by-wordquestionpresupposition readingtimesfor all threequestiontypesin the two experimentsrespectively. In all casesthe overall time to readwas slowerwhen the questionword or questionphrasecamefirst. The overall meantimes combinedacrosswords in the conditionswherethe questiontype wasknown and not known were 379msvs. 359ms,F(1,23)=9.90, pc .Ol in the first experimentand 5OOms vs. 39Oms,F(1,44)=44.89,pc .OOlin the secondexperiment.The cost of knowingthe questiontype wasmeasuredby dividing the differencebetween the conditions,an estimateof the absolutecostper word, by the meantime takenwhenthe questiontypewasnot known, an estimateof absolutereading time per word that doesnot include the cost. The costswere6% and 28% per word in the two experimentsrespectively.

Question Verif.

N

V

Type

Reason

P

D

Time

I

I

I

I

I

I

I

I

N

V

P

D

N

V

P

D

Word Figure 2. Mean word-by-word reading times for three question types word appeared first or last (adapted from Robertson, Weber, Ullman,

Question Verif.

N

V

when the question 8 Mehta, 1993).

Type

Reason

P

D

NVPDNVPD

Word Figure phrase

3. Mean appeared

word-by-word first or last

reading (adapted

times for three from Robertson,

question Weber,

types when the question Ullman, & Mehta, i993).

71

72

ROBERTSON TABLE Mean Answering Two Question-Position

4

Time (ms) for the Three Question Conditions In the Query-Position Experiment

Question Posltion

Question Verification

Types and Exaeriments

1

Type Reason

Time

Mean

Q-first Q-last

2131 2790

2560 3174

2930 3186

2543 3050

Mean

2461

2867

3W2

2797

Type Reason

Time

Mean

Exaeriment Question Position

Question Verification

2

QP-first QP-last

2253 2423

2989 3595

3147 3955

2796 3324

Mean

2338

3292

3551

3061

Table 4 showsthe meananswerretrieval times for verification, reason, and time questionsin the two experiments.As predictedby the parallel model, answerretrieval wasfasterwhenthe questionword or phrasecame first, Knowledgeof the questiontype gavesubjectsa 507msretrievaladvantage,F(1,23)= 10.90,pc .Ol in Experiment1 and a 528msretrievaladvantage,F(1,41)=6.73, p-c .05 in Experiment 2. Despitelarge differencesin absoluteansweringtimes acrossthe two experiments,the savingsin time enabledby knowledgeof the questiontype was measuredby dividing the differencebetweenthe two conditions,an estimateof absolutesavings,by the meantime taken whenthe questiontype wasnot known, an estimateof absolutetime that doesnot includethe savings.The savingswereastonishingly consistentin the two experiments,17% and 16% respectively. In the first experiment,the comprehension costfor thepresuppositionwas small relativeto the savingsat retrievaltime. In the secondexperiment,the cost seemsout of proportion to the savings.However,becausethe cost is extractedon relatively short readingtimes whereasthe savingsis achieved on the longer retrieval times, in the end there was an absolutesavingsin total time to readandcomprehenda questionin all caseswhenthe question type wasknown. The overall costsover the four words of the presuppositions were8Omsand444msin the two experiments,whereasthesavingswere 507msand 528ms.Theseyield cost/savingsratios of .16 and .84 respectively, all impressiveadvantagesfor simultaneousquestioncomprehension and answerretrieval.

QUESTION

PROCESSING

73

Theseexperimentsprovided strongsupport for the view that processes relatedto answerretrieval operateduring questioncomprehension.These processes extracta coston readingtime (sometimeshigh)but offer a savings at retrieval time. The TSUNAMI model is consistentwith the data in the aboveexperiments.When the questionwords or phrasescame last, the answerretrievalcomponentof the model couldnot operateduringreading. Only the matcherand the parseroperatedduring this time. Also, nothing would havebeenstoredin the answercandidatebuffer during readingand no use would have beenmade of questionansweringrules. In contrast, whenthe questionword or phrasecamefirst, the answerretrievalprocessor could operate.This would increasethe contentsof the questioncandidate buffer and the answercandidatebuffer. With datain the answercandidate buffer the output preparationprocessorwould beginto operate.Assuming limited resources,the simultaneousoperationof theseprocesses would take a toll on the parserand increasereadingtimes. In addition, sincethe processesof the answerretrieval mechanismdependon the contentsof the questioncandidatebuffer, questionsrequiring more complexretrievalwill increasereadingtimesmore. This effectwasdemonstratedin Experiment2, whereverification questionswerereadfasterthan reasonquestions,which in turn werereadfaster than time questions. Regardingansweringtimes, the conditionsin which the questionwords or phraseswerelast (Q/QP-last) allowed the questioncandidatebuffer to containa singleitem by the time the questioncomponentwasreached.At this time, althoughthe answerretrievalmechanismhad only oneinput from the questionbuffer it would beginsearchin episodicmemory from scratch andgenerateseveralanswercandidates.Thesecandidateswereexaminedat the sametime by the output preparationprocessor.Thus in theseconditions,whenthe last word wasreadthe answerand outputprocesses operated simultaneouslyandon a considerableamountof data. In contrast,whenthe, questionwordsor phrasescamefirst, many answercandidateshad already beengeneratedand eliminatedby the time the parserreachedthe last word< of the question.Thus, at the endof a questionin theseconditionstherewas lessleft to be doneby the answerand outputprocessors and fewerdataitems to process.This explainsthe faster answeringtimes in theseconditions. 3.2 Number of Answers 3.2.1 Introduction

Questionansweringis a processthat requiresconsiderableinteractionwith knowledgestructuresrelevantto the question.Graesser& Murachver(1984) & Graesser& Franklin (1990)havepointedout that onestepin the question answeringprocessis theidentification of relevantknowledgesources.In the

74

ROBERTSON

TSUNAMI framework the identification of knowledgesourcesoccursduring comprehension.As the walkthroughin section2.2.2 demonstrated,increasinglyspecificportions of a knowledgestructureareactivatedasmore andmore of a questionis beingread.If this is true, thenchangesin the content of knowledgestructureswill affect questionprocessing.If theseeffects are apparentduring readingof the question,then question-relatedknowledgestructureactivation will be shownto be occurringduring parsing. The courseof questionprocessingunder different knowledgestructure contextsmay be studiedby varying the numberof potential answersthere areto a questionat different points duringcomprehension.As an example, considerthe following story in which John drivesto the storethreetimes for threedifferent purposeswhile Mary drivesto the storeonce: John drove to the store to buy bread on Monday. John drove to the store to buy milk on Tuesday. John drove to the store to buy cheese on Wednesday. Mary drove to the store to buy eggs on Thursday.

In the context of the abovestory, considerthe following questions: Ql. Why did John drive to the store on Tuesday? Qg. Why did Mary drive to the store on Thursday?

In answering47 it is impossibleto identify theexactsourcenodein memory that corresponds to the questionpresuppositionuntil theendof the question. In Q8, however,it is possibleto identify the uniquesourcenodein memory asearly at the subjectnoun, “Mary.” In a questionansweringarchitecture with parallelism,like TSUNAMI, answerretrieval heuristicsshould begin earlierwhen readingQS than 47. In strict serialmodels,however,source node activation and application of retrieval heuristicsare alwaysdelayed until after questionparsing,and thus no effects of the uniquenessof the sourcenodeconceptshouldbe observedduring readingof a question. It is again assumedin this experimentthat retrieval processescompete with parsingprocessesfor cognitiveresources.If this is true, thenthe operation of retrieval processesshould be evident in increasedreading times. Thus the predictionin the aboveexampleis that 48 would be read more slowly than 47. The slowing of reading, however,shouldbe accompanied by a speedieransweringtime sincethe retrieval processwas finding an answer.It is important to showthat the readingcostis accompaniedby an answeringsavingsin order to claim that the effectsaredue to retrieval. Note that the readingspeedpredictionis at oddswith the “fan effect” literature (Anderson,1974,1976,1983;Anderson& Bower, 1973).When complexanswerretrievalis not considered,for exampleif 47 and Q8 were simple verification questions,then47 shouldbe readmore slowly than QS becausetherearemorenodesrelatedto 47 thanQS. Spreadingactivationin

QUESTION

PROCESSING

75

the episodicnetwork derivedfrom the four examplesentenceswould be greaterwhenthe conceptsin 47 wereactivated,andthis additionalresource allocationwould slow readingof 47. Thus, the prediction in this study is counterto the fan effect andmay evenhaveto work againsta fan effectto be noticeable. 3.2.2 Method

Subjects.Twenty-eightRutgersUniversity undergraduatesparticipated for credit in Introductory Psychologycourses.Each subjectservedin all conditions.Subjectsspentabout 40-50 minutesin the experiment. Materials. Sixteenstorieslike the previousJohn and Mary story were constructedfor the experiment.Each story consistedof four actionsperformed at four different times for four different purposes.In eachstory thereweretwo characters.When the storieswerepresented,one character was associatedwith three actions while the secondwas associatedwith a singleaction. For eachstory, one action was chosenas the query action, aboutwhich a questionwould beasked.The actorassociatedwith the query actionwasvariedacrosssubjectsso that for somesubjectsthe queryaction was performedby the unique characterand for other subjectsthe query actionwasperformedby the characterwho did severalthings. Design and Procedure. Eachsubjectreadthe sixteenstoriesand answered two questionsabout eachone. The entire text of eachstory waspresented on a computer screenand subjectsspent as long as they liked readingit. Whenthey werefinishedthey presseda responsekey. At this time a prompt (plus sign), marking where the questionwould begin, appearedon the screen.Each subsequentkeypressrevealeda word of the question. The wordsappearedside-by-sidein their normal positions,but only one word wasvisible at a time (eachpreviousword disappeared).Reasonquestions werein the form “Why did NOUN1 VERB PREP1 DET NOUN2 PREP2 TIME?“, for example“Why did John drive to the store on Tuesday?” Time questionswereof the form “When did NOUN1 VERB PREP1 DET NOUN2 AUX VERB2 NOUN3?“, for example“When did John drive to the storeto buy cheese?”Subjectswereinstructedthat on the last word of the questionsthey should pressthe responsekey “as soon as an answer comesto mind.” The readingtimes for eachword wererecordedalthough analyseswereperformedonly on the wordsthat the two questiontypeshad in common, specificallyNOUN1 VERB PREP1 DET NOUNZ. Subjectswerefirst askedthe reasonor time questionabout eachstory. Eachsubjectwasaskedreasonquestionsabout eightstoriesand time questions abouteightstories.For eachsubject,half of the questionswereabout

76

ROBERTSON

the action performeduniquelyby one actor (uniqueaction) and half were aboutoneof the threeactionsperformedby the otheractor (non-uniqueaction). The question type condition (reason/time)and action uniqueness condition(unique/non-unique) werecompletelycrossedandthe presentation orderof variouscombinationswasrandom.Acrosssubjects,thestorieswere rotatedthroughall conditions. Sinceit waspossibleto answerthe time andreasonquestionswithout paying attentionto theactorsin the story,thereason/timequestionwasfollowed by a “who” questionabouteachstory. The antecedentfor the who question was chosenrandomlybetweenthe uniqueaction and one of the nonunique actions.The who questionguaranteedthat subjectswould paycloseattention to the actors.Readingtimes werenot collectedfor thesequestions. 3.2.3 Results and Discussion Figure 4 showsthe word-by-wordreadingtimes for the reason-questions and time-questionsin the single-propositionand multiple-propositionconditions,As predicted,thetime to readthe questionswasgreaterin theunique action condition relative to the nonuniqueaction condition (495ms/word vs. 464ms/word),F(1,27)=7.50, p-c .OS.The cost of fmding a unique sourcenode and beginningretrieval was calculatedby dividing the differencebetweenthe conditions,an estimateof the absolutecost per word, by the meanreadingtime per word in the nonuniqueaction condition, an estimate of readingtime which doesnot includethe cost. The cost in reading time was7% per word, in line with the costsfound in the prior experiment. Although the time to read eachword varied, F(4,108)=6.13, p-e .05, the action uniquenesseffect did not interactwith words.Therewasno effectof the questiontype and no other interactions. Table 5 showsthe answertimes for the reason-questions and time-questions in the uniqueand nonuniqueaction conditions.The answertime was much fasterin the uniqueaction condition relativeto the nonuniqueaction condition (2449msvs. 3554ms),F(1,27)= 19.11,p< .OOl.The savingsof finding a unique sourcenode and beginningretrieval was calculatedby dividing the differencebetweenthe conditions,an estimateof the absolute savingsin answertime, by the meananswertime in the nonuniqueaction condition, an estimateof answertime which doesnot includethe savings. By identifying a sourcenode early, subjectsachieveda savingsof 31% in the-iransweringtime. In contrast to other experimentsin our lab, reason questionswere answeredmore slowly overall than time questions(3256msvs. 2746ms), F(1,27)=9.39,p< .05. Therewasno interaction. Subjectssavedan averageof 1105msin their answeringtimes whenthey wereableto identify a uniquesourcenodeearly. The averagecost to their readingtimeswas3lms/word asmeasuredacrossfive words.Assumingthis

E =

3 5

400

500

600

4. Mean

and

Figure

ward-by-ward non-unique answer

Questions

unique

Reason

reading times far to the questions.

Word

I

V

I

Nl

Time

two

question

P

I

D

I

questions

types

1

when

N2

there

was

-

a

Number Y

of

Non-Unique

Unique

Answers

78

ROBERTSON

Mean

TABLE 5 Answer Times (ms) far Reason and Time Questions in Unique and NonUnique Action Conditions Action

Condition

Unique

Nanunique

Mean

Reason Time

2870 2027

3642 ,a465

3256 2746

Mean

2449

3554

3001

Question

Type

costwas extractedon all eightwords in the questions,which is an overestimate of the cost, the overall cost to begin retrieval during parsing was 248ms.Thus, a liberal estimateof the cost/benefitratio of parallelretrieval and parsingin this experimentis 22%. A critical prediction of the TSUNAMI model, or any other parallel model, is that the contentof episodicmemory will influencethe courseof questioncomprehension.In this experimentwe demonstratedthat when early convergenceon a sourcenodeis possible,readingtimes increaseand answeringtimes decrease.This effect was achieveddespitethe possibility that fan effectswereworking againstthe hypothesis.Theseresultsprovide strongevidencethat retrievalbeginsduring reading. It is interestingto considerthe implications for more complexretrieval contextssuchasthosetreatedby Graesser8cMurachver(1984)andGraesser BEClark (1985).Accordingto theseresearchers, it is important for the question answererto determinerelevantknowledgesourcesin orderto constrain retrieval.Different questiontypesseemsto be associatedwith specifictypes of knowledgestructures,for examplegoal orientation questionsare associatedwith goalhierarchies,causalantecedentquestionsareassociatedwith causalchains,time questionsare associatedwith temporal knowledge(if it is encodedin a specialknowledgestructure),etc. The TSUNAMI architecture allows for the early identification of thesecomponentsof a knowledge structureand hypothesizesthat candidatesourcenodeswithin thesestructuresaregraduallynarrowedas eachword of the questionaddsconstraints to its interpretation.Oncea small enoughset of sourcenodesis identified, retrievalcan begin. 3.3 Comprehension

Instructions

3.3.1 Introduction

The TSUNAMI model hasa specialanswerretrievalprocessthat operates separately,but in parallel, with the languageparser.Presumablythis pro: cessoperatesonly when subjectsknow that they will be answeringa question. In the aboveexperimentsevidencefor parallel processesin question

QUESTION

PROCESSING

79

answeringwas gatheredby showing workload effects on the parser. However,it is important to showthat theseworkload effectsarespecialto questionanswering,andat thesametime to showthat subjectshavecontrol over the operationof the answerretrievalprocessor. The connectionof the workload effectsto questionprocessinghasbeen demonstratedso far by showingworkload effects that are linked to the questiontypes.In the query-positionexperiment(section3.1),time, reason, andverificationquestionswereansweredat different speeds,and in thesecond of thoseexperimentstheywerealsoreadat different speeds.In the experiment manipulating number of answers(section3.2) the reasonand time questionswere also answeredat different speeds.It is difficult to imagine an explanation for theseeffects that doesn’t involve questionspecificprocessing.However,in this experimentsubjects’intentionsasthey read questionswere manipulated. Specifically, sometimessubjectswere askedto readand answergoal and time questionsabout stories,andsometimes subjectswereaskedto read and paraphrasegoal and time questions about stories. In the latter case,the subjectsdid not haveto answerthe questions.In anotherconditionsubjectswereaskedto readand paraphrase goal and time questionswhich had no story context and could not be answered. The well-knownreadingtime effect relatedto the questiontype servesas a testof parallelprocesses in this experiment.Subjectsshouldshowa reading time differencebetweengoal-and time-questionsonly whentheyintend to answerthe questions.This result would be consistentwith the view that subjectshave control over the answerretrieval processand with the view that effectsspecificto the quesionansweringprocessareinfluencingparsing times. If subjectsin the paraphrasecondition show a question-typeeffect, then the answerretrieval processmay be automatic and not under the voluntary control of subjects. 3.3.2 Method Subjects. Twenty-sixRutgersUniversity undergraduates participatedin this study for credit in Introductory Psychologycourses.Subjectsspent about40-50minutesin the experiment. Materials. Forty-eightshort (5-7 line) storieswerewritten. In eachstory a characterwent to somelocation, and this action was queriedin subsequentquestions.All of the storiescontainedan explicit goal and time for the queriedaction. A reasonquestionanda time questionwerepreparedfor eachstory. The reasonquestionsread “Why did go to the < LOCATION> 7,” whereasthetime questionsread“When did go to the ?”

80

ROBERTSON

Design and Procedure. Therewerethreeinstructionconditionsin the experiment:no story paraphrase, story paraphrase, and story answer. In the no story paraphrase conditionsubjectsread questions(self-pacedone word at a time) and weretold to come up with a paraphrase.When they had reachedthe last word of the questionthey wereto pressthe responsekey “when the meaning of the questionwas understood.” They then wrote down their paraphrase.After this they pressedthe responsekey and sawa computergeneratedquestionandwereaskedto judge if it “meant the same thing” as the question. The latter task was intended to reinforce the paraphraseinstruction. Subjectsworked through eight why-questionsand eight when-questionsrandomly intermixed in a block. In the story paraphrase condition subjectsread a paragraph-longstory which was then followed by a question.The questionwaspresentedin the samemanneras the no story paraphrasecondition and subjectswereinstructedto comeup with a paraphrasein the sameway. Eachquestionwasfollowedby a “means the samething” judgementandtherewereagaineightwhy- andeightwhenquestionsrandomly intermixedin a block. Finally, in the story answer condition the subjectsreceivedstories followed by questionsas in the story paraphrasecondition, but this time for instruction was to come up with answersto the questionsandpresstheresponsekey “when an answercomes to mind.” Similarly, they wereaskedto judge if the secondquestion“has the sameanswer” asthe first. Storieswererandomlyassignedto conditions and rotated through conditions acrosssubjects.Instruction block orders werecounterbalanced. 3.3.3 Resullts and Discussion

Readingtimes weresummedoverall wordsin the questionsexceptthe final word and then divided by the number of syllablesin the words. This providesa per-syllablemeasurethat controlsfor differing word lengths.Reading time for the last word includestime for memory retrievalplus answer/ paraphraseformulation, and is therefore not included in the analysis designedto assesson-line processes. Table 6 showsthe meanreadingtimes per syllablefor reasonand time questionsreadunderthe threeinstructionconditions.Overall,time questions werereadmore slowlythan reasonquesions,F(1,25)= 7.44,pc .05, andthe instructionmanipulationalso affectedreadingtimes,F(2,50)= 3.33,pc .05. A significant interaction betweenthese factors, however, supports the hypothesisthat the question-typeeffect is presentonly underthe answering instruction F(2,50)= 3.24,p< .05. Time questionswere read at a 39ms/ syllableslower rate than reasonquestionsunder the answeringcondition, but the differencewasonly 1lms/syllable and 3ms/syllablein the no-storyand story- paraphraseconditionsrespectively. There was a difference in the answeringtime for time questionsvs. reasonquestions(1039msvs. 1173ms) which wasnot presentin eitherparaphrasecondition (1055msfor time-questionsvs. 1OOlmsfor reason-ques-

QUESTION

Mean

TABLE 6 Reading Times (ms/syllable) for Reason In the Three Comprehension Instruction Comprehension

Question Tme

No Story Paraphrase

81

PROCESSING

and Time Questions Conditions

tnstructlon

story Paraphrase

story Answer

Mean

Reason Time

390 401

367 370

332 371

363 381

Mean

395

368

351

372

tions in the no-storyparaphrasecondition, and 1530msfor time-questions vs. 1186msfor reason-questions in the story-paraphrase condition). Howeverthis interactionwas not statisticallyreliable. The resultscanbe interpretedin supportof the hypothesisthat questionrelatedretrievalprocesses arebeingactivatedduringreading.Suchprocesses arenot relatedto comprehensionalonesincecomprehensionis requiredin both the paraphraseand answeringconditions. Two possibletypes of processesmight explain the result. Sincewhenquestionshave more potential answersthan why-questions(Graesser& Murachver,1984),theremay be more retrievalroutesactiveduring normal processingof a when-questionthan a why-question.With more retrievalinformationactivein shorttermmemory,normalreadingis slowed(MacDonald, Just, BECarpenter,1992).Alternatively, therecould be more rules or more complexsetsof rules readiedfor answeringa time questionthan a reason question.On this view retrievaldoesnot begin, but the more cumbersome rule setburdensprocessingof the wordsin when-questions relativeto whyquestions. 3.4 Summary of Experimental Results

Three experimentswere describedwhich provide evidencefor different aspectsof theTSUNAMI framework.In the first, knowledgeof the question typeincreasedreadingtime for questionsbut decreased answeringtime. The resultssupportthe view that answerretrievalbeginsduring parsing.In the secondexperimentthe stateof episodicmemory affectedquestionprocessing. When a singlesourcenode could be identified in episodicmemory as the questionpresupposition,questionreadingtime increasedand answer retrievaltime decreasedwhen comparedto casesin which the sourcenode could not be uniquelyidentified until late in the question.This effect appearedearly in the questionstrings suggestingthat retrievalprocessesare not waiting for a full sentenceto be input beforebeginning.In both experiments,the cost in time for combiningparsingand retrievalwaswell worth the savingsat answeringtime.

82

ROBERTSON

A third experimentexaminedwhetherretrieval effectson parsingtimes werespecificto questionanswering.A discrepancyin readingtimesbetween reasonquestionsand time questionswas presentonly when subjectswere instructedto answerthe questionsandnot whentheyintendedonly to paraphrasethe questions.The results reinforce the hypothesisthat answerretrieval processesaffect questionparsingtimes. The resultsalso suggest that the retrieval componentsof questionprocessingare not automatic. 4. CONCLUSION

The literatureon questionansweringhasalwaysassumedserialoperationof the manyprocesses necessary to understanda questionandfind andgenerate an answer.Recentconnectionistmodelsof questionansweringhavebegun to challengethis view with parallel architectures(Miikkulainen & Dyer, 1991;Shastri, 1988,1989;St. John & McClelland, 1990),but thesemodels handle simple questiontypes that require little more than a match/nomatchheuristicto generateanswers.In somecases(Shastri,1988)they even preservea serialflavor by incorporatingseparateand orderednetwork processesfor query formulation, knowledgebaseaccess,and responsegeneration. They also typically operate0n.ahighly pre-encodedrepresentationof the query-implying at leastthat considerableparsinghasbeendonebefore retrieval begins. The experimentswere not designedto distinguish connectionistfrom symbol-basedmodelsof questionanswering,and they can not do so. The resultswereobtainedin the contextof questiontasksthat currentconnectionist modelsdo not perform. In particular, connectionistmodelsdo not exist for answeringcomplexgoal andtime questionsin the contextof narratives, andconnectionistmodelsdo not mix parsingwith questionanswering. Modular, symbol-basedmodelsof thesecomplextaskshavebeenproposed, however,and so this article focuseson augmentingthe architectureunderlying thesemodelsso that parsingand retrieval can be intermixed. The experimentalresultsare not inconsistentwith connectionistmodels as far as they go. In fact, the matcherin TSUNAMI might bestbe implementedasa connectionistnetworkwith local nodesmuchlike Shastri(1988, 1989)proposes.Sincethe goal of the matcheris to locatethe questionpresuppositionin memory, the verification or recognitionprocessin connectionist question-answering modelsmapswell to.this aspectof TSUNAMI. Somepart of the increasedreadingtimes observedwhen questionparsing and retrievalwereoperatingin parallel might be dueto simple presupposition matching,howeverthebulk of the time costis surelydueto the application of retrievalheuristicsfor complexreasonandtime searches.Matching is only the start of a highly interactivesetof processes whosenatureshould be the topic of future research.Someresearchershave proposedhybrid

QUESTION

PROCESSING

83

models of sentence processing that combine aspects of connectionism with symbol-based mechanisms (Kintsch, 1988; Mannes & Doane, 1991; Moisl, 1992), and this is a fascinating direction for question answering models as well. The TSUNAMI model is a proposed architecture that supports parallelism in question comprehenson and answer retrieval in the tradition of modular, symbol-based processing. Viewing question comprehension and answer retrieval as parallel processes has consequences for many aspects of current models in this tradition. The nature and implementation of question categorization and answer retrieval heuristics might be significantly different when considered in the context of a parallel architecture. Many new problems arise once this view is adopted. For example, the issue of interaction among parsing and answering components becomes important. The problem of determining which components of knowledge structures to search becomes more complex as the constraints are apparent earlier in parsing and must be applied in a dynamic manner. If partially specified question candidates can start retrieval processes, a problem arises concerning how they might be tracked and stopped if the question candidate is later disconfiimed. In future research new models of question processing will certainly arise as a parallel framework is adopted. 5. REFERENCES Altmann, G., Garnham, A., & Dennis, Y. (1992). Avoiding the garden path: Eye movements in context. Journal of Memory and Language, 31, 685-712. Altmann, G., & Steedman, M. (1988). Interaction with context during human sentence processing. Cog&ion, 30, 191-238. Anderson, J.R. (1976). Language, memory, and thought. Hiisdale, NJ: Erlbaum. Anderson, J.R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press. Anderson, J.R., & Bower, G.H. (1973). Human associative memory. Washington: Winston & Sons. Anderson, J.R. (1974). Verbatim and propositional representation of sentences in immediate and long term memory. Journal of Verbal Learning and Verbal Behavior, 13, 149-162. Clark, H.H., & Clark, E.V. (1977). Psychology and language. New York: Harcourt Brace Jovanovich. Clark, H.H., & Haviland, S.E. (1977). Comprehension and the given-new contract. In R.O. Freedle (Ed.), Discourse production and comprehension. Norwood, NJ: Ablex. Collins, A.M., & Loftus, E.F. (1975). A spreading activation theory of semantic processing. Psychological Review, 82, 407-428. Dyer, M.G. (1983). In-depth understanding: A computer model of integrated processing for narrative comprehension. Cambridge, MA: MIT Press. Erman, L.P., Hayes-Roth, F., Lesser, V.R., & Reddy, D.R. (1980). The Hearsay-11 speech understanding system: Integrating knowledge to resolve uncertainty. Computing Surveys, 12, 213-253. Ferriera, F., & Clifton, C. (1986). The independence of syntactic processing. Journal of Memory and Language, 25, 348-368.

a4

ROBERTSON

Fodor, J. (1988). On modularity in syntactic processing. Journal of Psycholinguistic Reseurch, 17, 125-168. Frazier, L. (1987). Theories of sentence processing. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing. Cambridge, MA: MIT Press. Gorrell, P. (1989). Establishing the loci of serial and parallel effects in syntactic processing. Psycholinguistic Research, IS, 61-74. Graesser, A.C. (1981). Prose comprehension beyond the word. New York: Springer Verlag. Graesser, A.C., & Clark, L.F. (1985). Structures andprocedures of implicit knowledge. Norwood, NJ: Ablex. Graesser, A.C., & Franklin, S.P. (1990). QUEST: A cognitive model of question answering. Discourse Processes, 13, 279-304. Graesser, A.C., & Goodman, SM. (1984). Implicit knowledge, question answering, and the representation of expository text. In B. Britton & J.B. Black (Eds.), Understanding expository text. Hillsdale, NJ: Erlbaum. Graesser, A.C., Lang, K.L., & Roberts, R.M. (1991). Question answering in the context of stories. Journal of Experimental Psychology, 120, 254-277. Graesser, A.C., & Murachver, T. (1984). Symbolic procedures of question answering. In A.C. Graesser & J.B. Black (Eds.), The psychology of questions. Hillsdale, NJ: Erlbaum. Hayes-Roth, B. (1985). A blackboard architecture for control. Artificial Intelligence, 26, 251-321, Just, M.A., & Carpenter, P.A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329-354. Just, M.A., & Carpenter, P.A. (1987). The psychology of reading and langauge comprehension. Newton, MA: Allyn & Bacon. Kintsch, W. (1974). The representation of meaning in memory. Hillsdale, NJ: Erlbaum. Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review, 95, 163-182. Kintsch, W., &van Dijk, T. (1978). Toward a model of discourse comprehension and production. Psychological Review, 85, 363-394. Lehnert, W.G. (1977). Human and computational question answering. Cognitive Science, 1, 47-73. Lehnert, W.G. (1978). The process of question answering. Hillsdale, NJ: Erlbaum. Lehnert, W.G., Dyer, M.G., Johnson, P.N., Yang, C.J., & Harley, S. (1983). BORIS: An experiment in in-depth understanding of narratives. Artificial Intelligence, 20, 15-62. Long, D.L., Golding, J.M., & Graesser, AC. (1992). A test of the on-line status of goalrelated influences. Journal of Memory and Language, 31. 634647. MacDonald, M., Just, M.A., & Carpenter, P.A. (1992). Working memory constraints on the processing of syntactic ambiguity. Cognitive Psychology, 24, 56-98. Mannes, S.M., & Doane, S.M. (1991). A hybrid model of script generation: Or getting the best from both worlds. Connection Science, 3, 61-87. Marslen-Wilson, W. (1989). Lexical representation and process. Cambridge, MA: Bradford Books. Marslen-Wilson, W., &Tyler, L.K. (1980). The temporal structure of natural language understanding. Cognition, 8, l-72. McClelland, J.L., & Kawamoto, A.H. (1989). Mechanisms of sentence processing: Assigning roles to constituents. In J. McClelland & D. Rumelhart (Eds.), Puralleldistributedprocessing: Explorations in the microstructure of cognition. Volume 2: Psychological and biological models. Cambridge, MA: MIT Press. McClelland, J.L., & Rumelhart, D.E. (1981). An interactive activation model of context effects in letter perception: Part 1, An account of basic findings. Psychological Review, 88, 375-497.

QUESTION

PROCESSING

85

Miikkulainen, R., & Dyer, M.G. (1991). Natural language processing with modular PDP networks and distributed lexicon. Cognitive Science, IS, 343-400. Moisl, H. (1992). Connectionist finite state natural language processing. Connection Science, 4, 67-91. Perfetti, C.A. (1990). The cooperative language processors: Semantic influences in an autonomous syntax. In D.A. Balota, G.B. Flores d’Arcais, & K. Rayner (Eds.), Comprehension processes in reuding. Hillsdale, NJ: Erlbaum. Rayner, K., & Pollatsek, A. (1989). Thepsychology of reading. Englewood Cliffs, NJ: Prentice Hall. Robertson, S.P., & Weber, K. (1990). Parallel processes during question answering. Proceedings of the 12th Meeting of the Cognitive Science Society. Hillsdale, NJ: Erlbaum, 542449. Robertson, S.P., Ullman, J.D., & Mehta, A. (1992). Siiultaneous question comprehension and answer retrieval. Proceedings of the 14th Annual Meeting of the Cognitive Science Society. Hillsdale, NJ: Erlbaum, 474-479. Robertson, S.P., Weber, K., Ullman, J.D., & Mehta, A. (1993). Parallel question parsing and memory retrieval. Journal of Memory and Longuuge, 32, 155-168. Rumelhart, D.E., & McClelland, J.L. (1981). Interactive processing through spreading activation. In A.M. Lesgold & C.A. Perfetti (Eds.), Interactive processes in reuding. Hifflsdale, NJ: Erlbaum. Rumelhart, D.E., & McClelland, J.L. (1982). An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. Psychological Review, 89, 60-94. Seifert, C.M., Roberson, S.P., & Black, J.B. (1985). Apes of inferences generated during reading. Journal of Memory and Language, 24, 405-422. Shastri, L. (1988). A connectionist approach to knowledge representation and limited inference. Cognitive Science, 12, 331-392. Shastri, L. (1989). Default reasoning in semantic networks: A formalization of recognition and inheritance. Artificror Intelligence, 39, 283-355. Singer, M. (1982). Answering questions from text: A process model. In A. Flammer & W. Kintsch (Eds.), Discourse processing. Amsterdam: North Holland. Singer, M. (1984a). Toward a model of question answering: Yes-no questions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 285-297. Singer, M. (1984b). Mental processes of question answering. In A.C. Graesser & J.B. Black (Eds.), The psychology of questions. Hillsdale, NJ: Erlbaum. Singer, M. (1986). Answering wh- questions about sentences and text. Journal of Memory and Language, 25, 238-254. Smgley, M., &Anderson, J. (1989). The transfer of cognitiveskill. Cambridge, MA: Harvard. St. John, M.F., & McClelland, J.L. (1990). Learrdng and applying contextual constraints in sentence comprehension. Artificial Intelligence, 46, 217-258. Taraban, R., h McClelland, J.L. (1990). Parsing and comprehension: A multiple constraint view. In D.A. Balota, G.B Flores d’Arcais, & K. Rayner (Eds.), Comprehension processesin reading. Hillsdale, NJ: Erlbaum. Waltz, D.L., & Pollack, J.B. (1985). Massively parallel parsing: A strongly interactive model of natural language interpretation. Cognitive Science, 9, 51-74.