- Email: [email protected]
Metaknowledge for time and reliability
Reliability Engineering and System Safety 36 (1992) 199-206
Metaknowledge for time and reliability C. Valot & R. Amalberti Ddpartement d'Ergonomie Adrospatiale, Centre d'Etudes et de Recherches de Mddecine Adrospatiale, Base adrienne du CEV, 91228 Brdtigny Cedex, France
The task analyses of rapid process control highlight the significant place of knowledge about time and reliability in operators' activities. This knowledge is strongly involved in the adjustment .of activity to the environment and to a knowledge about the operator's own capacities: a form of metaknowledge. The elicitation of that metaknowledge shows the importance of strategies oriented to the cognitive resource management. Some outcomes are discussed: the place of metaknowledge in cognitive modelling; the usefulness to integrate it in decision support systems designing such as pilot's intent recognition. A model of metaknowledge is proposed in the frame of combat aircraft pilots.
1 INTRODUCTION
demands of the situation. This finality can lead to acts not optimal for the system but adequate with regard to the available cognitive means. This functional dimension of adjustment is a determining factor in developing aids, for example, for an airplane pilot. Whether it is for intent recognition 1 or to improve the operator's familiarity with the functioning of associated automatisms like the electronic co-pilot (developed simultaneously in the USA and in France), these 'human-like' heuristics are useful. They allow a more developed understanding of cognitive activities through a better definition of objectives desired by the operator.
The process controls characterised by an intense temporal pressure are interesting for the study of the dynamic aspects of cognitive activity. The operator must make decisions in a limited given time; his decision being often irreversible; in a partially uncertain environment; where the diversity factor is important. For instance, a pilot flying at 900 km/h and 200 m high, without visibility is rapidly confronted by his own cognitive limits. This is, even in peace time, an omnipresent risk. This type of continual process presents numerous demands for the pilot. He must therefore develop control strategies which are compromises between the demands of the systems and his own means, knowledge and know-how. How does he do it? The activity analyses show the importance of two types of specific knowledge allowing:
2 TEMPORAL PRESSURE MANAGEMENT The temporal dimension of knowledge has been studied in several ways. The study of the past events in mental organisation2 or the temporal evolution of logical relationships. 3 These approaches are not directly based upon a process control activity. In that specific field, studies show various forms of time in process control: 4'5 time of the process, of the
--the management of his own activity in relation to temporal pressure; --the management of reliability and confidence placed in the system, other operators or himself, Those types of knowledge are not directly related to the system and objectives one must attain, they have another 'finality': the management, by the operator, of his own cognitive resources depending on the
clock, subjective duration of events, temporal overlapping of events. The time studies in operator's activities are essentially centred on the difficulties the operator faces: synchronisation of events or the management of duration. We would highlight here another temporal dimension: the interaction between the temporal
ReliabilityEngineeringandSystemSafety0951-8320/92/$05.00 © 1992 Elsevier Science Publishers Ltd, England. 199
200
C. Valet, R. Amalberti
processing of a machine and the operator's own cognitive capacity to manage the events he initiates or controls, In concrete terms for instance, the operator will avoid the overlapping of processes requiring from him a high level of attention. To do this he will take into account his own cognitive limits, For a better understanding of these particular aspects of temporal pressure management, we analysed the activity of combat pilots undergoing a mission with high temporal pressure (low level flight without visibility at a high speed). In a recent study, we analysed eight pilots carrying out a classical mission in a single seat fighter with the same temporal, tactical and cartographical information. The aim being to attain a specific geographical point at a specific moment in time. We analysed their activity during flight preparation and the performance of the flight, this being in a simulator with cabin movement and external visualisation. Each pilot's flight preparation was very detailed, often being longer than the flight itself (1 h of preparation for a 45 min flight), We saw that all the pilots precisely planned the time factors of the mission, the different steps, the flight path and the expected values for a certain number of key parameters (fuel, distance, chronology, count down time...). The precision criteria required for this mission were less than 20 s in time and less than 20 m on the position. Starting from the time over target, the pilot calculated the departure time within a second. During the actual flight we saw that none of the pilots kept to his own planning. They either took off within 2 min of the stated time or didn't follow a large percentage of the flight they had calculated or prepared, In spite of these facts, the majority completed the mission within the required time and place. They completed their mission correctly but using adjustment rules from their own experience. In the pilots' opinion, the mission was correctly completed. They t D
T
O
I
V
~
E
~
~.r~p
sizes of the actual temporal bracket
plannedp a t h
~ actualpath Fig. 1. Comparison between planned and actual mission,
achieved their aim using the given criteria. But an aid (such as an intent recognition) which would have detected the deviations from the planned flight path would have given continuous warnings: deviation from the flight path, from the time for example... Such an aid would have been excessively off putting and restricting with regard to their numerous adjustments. Figure 1 shows an example of comparison between planned and actual mission. The preparation and flight analyses give numerous explanations of the observed deviations. 2.1 Flight
preparation
Various types of planning are elaborated by pilots. --Quantitative planning. It translates the mission order into a real flight and integrates temporal, tactical and meteorological constraints to give a quantitative flight description: way points, passage times, fuel consumption. They are used as references by the pilot. --Qualitative planning. It concerns two kinds of events: the foreseeable ones and the potential ones. The foreseeable events have not a precise spatial or temporal place, but the pilot knows that they will happen: it is the case for many system checks. The potential events have generally an external source: meteo, hostile fighter, ground to air defences. They all belong to the mission universe but the pilot is unable to predict their occurrence for a given flight. But all those events are major potential problems. They can't be integrated into quantitative planning. The pilot faces such a temporal load during the flight that it is quite impossible to really solve a complex problem. He is able to make decisions but in a preplanned context. 'What would I do if...'. With the present planes it is possible to make adjustments but a new complete inboard planning can't be considered. So, the pilot has to preplan his responses to a wide range of potential events. The rapidity of the process constrains the cognitive resource allocation. It is not possible to consider a new problem space and at the same time to follow the short term control of the flight path and the long term control of the mission.
2.2 Situation prepl.ning The more the pilot is experienced, the wider is the preplanned response range. This well-known fact is essential in flying under such conditions. An expert pilot will consider more detailed problems and develop more different potential responses. This supposes that effective cognitive representations can be hypothetically induced in order to obtain preplanned responses. The schema variety and the
Metaknowledge for time and reliability depth of knowledge allow one to link up hypothetical problem spaces and solution ranges, In that constrained situation, a specific type of knowledge has a specific place. A pilot says for instance 'After that way point, I know that I will only be able to control the" flight p a t h . . . ' . It is to be inferred that only short term control can be achieved, due to the step load and the pilot's cognitive resources. The main consequences are that, during that period, a failure cannot be dealt with, all the control will be flight path dedicated. The planned precision is only an indication but not an effective rule. To be solved, all those kinds of events would require too high a level of attention incapable of being supplied by the pilot. Consequently the pilot manages checkings, actions, level of controls according to his own representations of his resources and the cognitive demands of the situations: 'I check fuel 30 s after a way point because I know it is a relatively quiet moment, never before...'. His past experience shows him that it is not useful to maintain a high level of precision during the whole flight. It is more functional to manage tolerable gaps, integrating at the same time, cognitive resources, potential events and a sufficient precision: ' . . . a gap of 30 s is easily recoverable, it is better to begin with 2 min to spare, which are easily lost, rather than to ask for a take off at the right time and have to wait for a landing airplane. This delay is more difficult to recover...', For numerous situations, a pilot develops a bracket management of accuracy, To be quite efficient, that bracket management supposes that the size of the brackets is proportional to the pilot's depth of knowledge about the situation coherence. The depth of knowledge is not only judged for a brief period (some seconds). It is equally endowed with a life duration and a knowledge about the systems' most likely evolution. Those temporal dimensions are strongly proportional to the working out of a bracket of possibilities tolerating a sufficient effort for an acceptable precision,
3 RELIABILITY MECHANICS Reliability is equally a special case in process control studies. It is omnipresent in the pilots' talk though it is rarely evoked in cognitive modelling. Nevertheless the mistrust cost of a system or of another operator have major effects: restricted uses of devices, vicarious strategies are expensive in time and attention, Reliability is important to characterise the man-
201
machine relationship6'7 or the self-knowledge of an operator. 8 The reliability functional dimension is particularly interesting: it shows that reliability is at one and the same time a risk and a necessity: - - a necessity: ' . . . I cannot achieve that without a minimum of reliability...'; - - a risk: ' . . . to rely on a system is partly suicidal...'. The word reliability used in those two sentences refers to different fields.
3.1 A minimum of reliability The minimum of reliability conveys that a pilot is not able to continuously consult the displays, to detect and prevent a wrong tendency in parameters or to estimate a suspicious value. This is not compatible with the mission achievement or with the available treatment times. The pilot is compelled to consult the displays in a sequential way to leave time to insert another activity. But this supposes that the period of consultation is optimised according, on the one hand, to the risk not to consult a display and, on the other hand, to the surplus time (on hand) dedicated to general control of the mission, for instance. 'I never spend more than 30 s without looking inside the cockpit...' puts that optimisation into words. The compromise between different activities is based upon various factors being weighed up: • The life-span of information: The correct value of a parameter, in a given time, doesn't mean that 10 s later, it will always be coherent. The rapid evolution of an airplane incites unstable parameters. Consequently, the pilot has to predict and to distinguish between the effects of the evolution and the early beginnings of a breakdown. This short information life-span is a major distrust factor. • Knowledge allowing the prediction of an evolution: The pilot is in possession of a variable amount of knowledge about the evolution of a parameter. The possibility of good technical and temporal prediction of a value evolution is a good way to spare time to consult another value. • The accumulated experience about breakdown likelihood: The technical reliability of a system is equally acquired. Past incidents and the squadron experience allow the weighing up of the breakdown risk of each system. The functionality of that kind of reliability aims at sparing time and resources for more long term activities. Reliability is the only way not to have to use short term treatments. At such a high level of temporal pressure, the pilot has to partially rely on
202
C. Valot, R. Amalberti
the systems, not to have to look at the displays for a given duration, 3.2 But not an unreserving trust Reliability is a compromise and so a source of risk. Too long an interval between consultations leaves the possibility of an ignored evolution. The fast amplification of consequences and the difficulty to recover the chronological evolution of events are the main risks. It is the suicidal aspect of reliability, That kind of restricted reliability expresses a distrust as much of the system as of the operator himself. His own behaviour is the object of his own doubts. His past mistakes are memorised and the pilot develops strategies to avoid or to detect a wrong insertion of data or readings of parameters. The main strategies are based upon comparison with other data from other sources or the evaluation of coherence of system evolutions, But the distrust is a heavy consumer of time and cognitive activity. As a matter of fact a distrusted system obliges the operator to develop many roundabout strategies. The operator treats in parallel a first level control to achieve the mission and a second level control to reduce the distrust. This second level is generally not foreseen in the design of the system; the pilot has to look for indications to verify the consistency of evolutions. That treatment is not so precise. Such a restricted way has a cumulative effect: the comparison must be carried out with first level control. In the stressed periods this overlapping reduces the quality of control, 3.3 Working out of reliability Some particular mechanisms are used to work out reliability (or distrust),
3.3.1 Information comparisons This is a question of data sources. A number of dials
that whole chain: 'the spot on the radar display is where I expected, so navigation is good, the radar is working well and the computer too'. Some seconds are sufficient to verify (with various levels of accuracy) the state of the chain components. 4 WHAT KIND OF KNOWLEDGE? Owing to their content and their place in the control of processing, reliability and time, knowledge is typically a metaknowledge from two points of view. In AI, MK is mainly considered as a way to control and filter a knowledge base, to initiate reasonings and strategies (bootstrapping) 9 using expert knowledge. Anyway, another common characteristic of MK in IA sets is their nature: they are ill-defined knowledge (declarative or procedural), possibly (but not systematically) regarding human capabilities and limitations (here is a discrepancy from the classical AI definition of MK), which function remains to weigh the procedural knowledge of a specific domain in a large range of situations. This knowledge about weighing is gathered from numerous forms and origins: it is strategic knowledge for Gruber l° elicited from expert or from programmers' knowledge about system capacities and reliability. In all cases, the weighing is based upon factors such as: cost, time, reliability, danger, etc. 11 The concept of metaknowledge clearly appeared in the psychological literature two decades ago. In the early 1970s, the concept was introduced in the field of child development in order to explain how the child was able to adapt to new situations and to control actions. 12 Papers from these above mentioned authors are related to a specific form of MK called metacognition: 'What we know about our own cognitive abilities'. It is a knowledge about cognitive tools: their abilities, limitations, using strategies and prediction about performances they allow, etc. This metaknowledge has a strong influence over chosen strategies. The self experience weighs up a large number of the operator's choices. Other aspects of metaknowledge are interesting: mental models are not only fed with objective and
display similar information (for instance artificial horizon is displayed under four different forms simultaneously and the speed is displayed four times under three different units). The comparison is that much more useful because the information is obtained through various chains of computation (air speed with external probe or computed ground speed, etc). In addition to that, the dials feed a large polysemy (up to ten different meanings are, for instance, taken out by a pilot from a speed dial). The redundancies of indications detected by the pilot are an etficient way to verify the coherence of a process. 8
accurate notions. Some rough notions and beliefs are involved in numerous models formed upon the 'world'.~3'14 This feature is equally present in the mental models we form about ourselves. The opinion of our own merit is then a kind of metaknowledge. Last but not least, metaknowledge allows the compromise between the available time, the control accuracy and the management of mental resources.
3.3.2 Hierarchical checking
But it is a double edged mechanism:
The reading of a value is representative of a chain of computations. One reading gives information about
• on the one hand, it is functionally useful to spare a pilot's cognitive resources;
Metaknowledge for time and reliability • but on the other hand, a new finality appears in the pilot's strategies: to run the resource/time/accuracy compromise and to decide in that context. This fluctuating adjustment is a source of risk. 5 M E T A K N O W L E D G E AND ACTIVITIES Task analysis shows the significant place of metaknowledge in the understanding of a pilot's activities, How is it possible to involve metaknowledge in mental models and what could be the use of that kind of knowledge? 5.1 Metaknowledge and knowledge classes Numerous authors have proposed knowledge classifications. They are often designed according to a knowledge functionality centred on the system. The analysis is system goal oriented and formed on job analysis techniques. Wirstad 15 proposed 'a plant related knowledge terminology'. In such an analysis, the regulation and the compromise allowed by the metaknowledge are not easily placed, An example is given by Bainbridge. ~6 For this author, the main classifications are: 'A definition of the product and the sequence of transformation. The process limits. The process functions and behaviours, The plant components. The present state of the process', 5.2 Metaknowledge and decision models Metaknowledge is strongly associated with decision making. But classically the means-ends analysis is considered from the technical ends point of view. The metaknowledge expresses another type of ends: the operator's ends. The resource management and the cognitive 'economy' are not clearly explained in the decision-making models. But they are not far behind, Let us consider, by way of example, the well known Rasmussen's decision-ladder. 17 The nearest concept of metaknowledge in that ladder is in the Rule Based Behaviours (RBB). They are defined as a short cut stemming from operator's experience. They aim at saving time, avoiding long and fragile mental activities. Rule based behaviour is connected with patterns of situations, they avoid reasoning or active understanding to carry out the right action, From our point of view metaknowledge is required to construct the RBB. The knowledge of the pilot's functioning is a significant part of the development of those rules. The pilot's activity analysis shows that a part of the effective action is system ended and resource management ended, The Rasmussen's ladder should be completed by
203
the possiblity of weighing up the choices made according to a set of rules with criteria such as: --is the cognitive cost of the chosen solution compatible with foreseeable activities? --what is our self-judgement of our own level of capacity for a chosen solution? The above weighing up is equally interesting to understand the operators' mistakes. A lack of attention or a shortened procedure could result from a strong or an inadequate weighing up due to cognitive economy 'logic' or time saved in a loaded step of activity. Cognitive resource management knowiedge leads to a distributed view of knowledge. Knowledge associated with problem-solving has to be interactively considered with reliability, time, coherence and self-knowledge. 5.3 Metaknowledge and performance models The analysis in terms of 'model of competence' or 'model of performance' allows equally some comparison with reliability and time metaknowledge. 18'19 The model of competence aims at. the understanding of the task difficulties: what are the cognitive problems the operator faces to achieve his job? The empirical development of the models of performance is more comparable to metaknowledge. Those models describe the mechanisms by which the operators carry out their task. In that way, Roth & Woods ~9 develop an approach to describe the minimisation strategies for information needs or sharing activity strategies. Those two kinds of strategies aim at the interaction management between man and the machine cornponents of the task. Those models are explicitly designed to escape from the language of the application. This is a necessary way to integrate the operator's resource management with the problem space.
6 L U X U R Y O R NECESSITY? 6.1 Luxury? This absence of the taking into consideration of metaknowledge in a large part of significant models and knowledge elicitations suggests its relative uselessness. Consequently, that kind of knowledge would be a luxury. It seems therefore quite possible to design models and develop systems without taking into account metaknowledge. But what kind of models are they? Models of cognitive demands or models issued from job analysis are useful and necessary but they don't describe enough the operator's cognitive activity
204
C. V a l o t , R . A r n a l b e r t i
about information processing resource management: • A goal oriented analysis emphasises the elicitation of technical and functional knowledge. This creates an internal logic inherited from the process. It is then possible to develop a general and system goal oriented model of activity. But it is not possible to confront these general models with the detail of the actual operator's activities. They are not designed for that goal. • Pooling knowledge of numerous experts leaves on one side the metaknowledge. Its content is essentially individual even if every one uses metaknowledge. It is difficult to generalise such knowledge and to integrate it into problem solving or process control.
have the same roots because the coherence is inherent in their acquired knowledge. A last remark about the place of metaknowledge in models and systems: strong relations exist between metaknowledge and planning, choice strategies according to resource management. The rules used by a person to choose from several answers involves the effect of his personal possibilities and know-how. Metaknowledge is a hidden dimension of individual strategies. To avoid or forget it will not prevent metaknowledge existing and to have a place in the regulation of activities. Therefore it seems more useful to analyse it in order to design well adapted systems. 7 MODELLING M E T A K N O W L E D G E
6.2 Necessity! Nevertheless, a detailed description of an operator's effective activity is necessary to develop, for instance, specific aids such as an electronic pilot's associate. In that case, it is useful and necessary to describe functional knowledge and metaknowledge. This is a necessity for several reasons: • The understanding of the temporal knowledge of an operator and especially the management of his activities according to the temporal horizons supposes the description of the adjustments between Short Term for action control (15s of flight path for a low level flight) and Long Term for checking mission coherence. The space problem is dynamically modified for the application of schemas. Metaknowledge contributes to that modification. A good carrying out of the schemas depends not only on their availability and their content but also on the quality of their temporal adjustment to the events, • Reliability knowledge is more dedicated to short and middle term management. They contribute to a detailed adjustment to the reality. This knowledge is essential to adjust the schema achievement according to the permanent events. Short delays or uncertainties are smoothed out in this way. Reliability is equally essential to allow time management. This time is afterwards dedicated to the long term (planning) or middle term (schema running) activities. • Time and reliability metaknowledge are a pilot's coherence tools. For that reason their description is essential in understanding a pilot's major concern: to keep a good level of coherence according to the events, m question arises to elicit those coherent tools: they are strongly tied to the whole pilot's knowledge. It is not possible to describe metaknowledge without the associated knowledge. They
A model of MK in process control could be seen as a three layer model; each of the layers corresponding to a different form in which the same MK entity could be expressed by pilots (Fig. 2). Level 1 is basic, MK is a pilot's every day experience. In each situation, the pilot experiences relations between planned activities and achievement quality. From this comparison with actual capacities, empirical and individual rules arise. They deal with numerous aspects of human activities: memory capacities, stress effects, available strategies and their efficiency, personal reliability, etc. This knowledge is typically individual according to events, past experience and training. Its evolvement is permanent. But some common features appear between pilots: they are the results of human cognitive tool capacities. Level 2 is the direct expression of psychological needs in terms of resource management. It is made of general heuristics whose goals are to keep mental processes at acceptable boundaries in order both to execute actions and to release time for additional activities. As a consequence, this level is independent of the domain and would be relatively common between operators. Only the content of this level is varying according to the pilot's experience. It is of natural consequence that MK from this level is rarely HUMAN: Level1Metacognitionandeveryday experience: " r m not able to conu'ol 3 parameters simultaneously..."
Level2 General resourcemanagementprimitives: - Avoid situations in which responses are poorly
mastered - Encourage situations in which responses are efficient - Hold on normal procedure as far as possible - Preserve m a x of freetime
DOMAIN: Level 3 Domain specific procedures as known & applied by the pilot
Context & situationdecision rules
SYSTEM:
Handbook offlightcontrolmanagementprocedures
Systemmanagementrules Fig. 2. A three layer model for representing metaknowledge.
Metaknowledge for time and reliability
205
Principle of economy : (i) prefer well mastered procedures even when they are not considered as the best ones. (ii) maintain at best the seriality of procedures, Time management heuristics : (i) when unexpected or not well understood situation occur, give priority to converge to a short term stable position in order to release safe time for long term reasoning (ii) Manage immediate situation in order to be capable to apply at least one valuable answer to the most possible events which could occur in the future. (iii) when time pressure increases, prefer action to reasoning (ref heuristic n ° 1) Principle of control : (i) check expected result of action (ii) check periodically parameters which are not concerned by the on-going schemata
Fig. 3. Some of the general MK heuristics.
elicited during activity analysis because it is not directly connected with activity. It has just a historical connection with activity. Only a specific focus regarding resource management can inform the scientist of its nature, Level 3 reflects very specific domain procedures as known and applied by the subject. They are the concrete result of knowledge integration acquired on system management, decision rules in abnormal situations, and MK rules from resource management, Therefore, at this level, the concept of MK remains fully intertwined with the other knowledge representation levels (domain specific decision rules, system management procedures, etc.). Standard techniques of operators' interviews generally conduct to this level (techniques such as autofacing the videotape or eliciting knowledge with goal directed perspectives) and it is probably the main reason why proper MK is not evidenced in most of operators' activity analysis, A human's proper mental constraints and system using proper constraints are listed at the top and bottom of the figure. Context and decision rules are generated by experience and operational instructions, Inversely compared to the previously mentioned limitations (from human and systems), they depend on the nature of the task. The resulting procedural knowledge representations developed by an operator for achieving tasks are represented by the procedures of the specific domain, At this level (which can be understood as a goal directed level), it is not MK which is represented but the result of MK regarding the procedures, It results from the model of MK described above that there is a layer independent of the domain specific application. This layer is made of general resource management heuristics which weigh the domain specific procedures. At the present time, we have isolated a non-exhaustic list of these heuristics (Fig. 3).
CONCLUSION Time, reliability, and other things (risk taking, etc.) metaknowledge is a real necessity to precisely understand the pilot's activity. Many activities are only understandable at that second level frame. But the study of metaknowledge is not only a better way to describe an operator's activities but also an original context to design intelligent aids. Every pilot uses metaknowledge but its content is quite different according to past experiences, previous airplanes, etc. Its use is essential in evaluating and keeping the system coherence. That is a major problem due to the difficulty in gathering consistent data. The next step expected for inboard aids is specifically to help the pilot to manage coherence. From our point of view, metaknowledge affords a good way to solve a critical problem: how to give coherence to pooled expert knowledge? We choose to design an expert knowledge based elicitation. That is to say: to gather not only the expert knowledge but equally his coherence tools to implement it in a model of a pilot's cognitive functioning. That kind of model includes the expert's style of mission performing. All the pilots know the characteristics of that style because they know the way the chosen expert flies. They would be able to immediately understand and to apply information given by such an aid. That would not be the better way to perform the mission but a good one, that is to say an understandable one, a predictable and a reliable one. From a support system designer's point of view, a question remains: is it useful to take into account metaknowledge about individual capacities in systems? Two forms of response exist: • This metaknowledge analysis is interesting for a better understanding of human cognitive activities,
206
C. Valot, R. Amalberti
it expresses the 'human logic' of strategies, selections and actions, • This metaknowledge could offer some possibilities to easy support systems design and m a n - m a c h i n e interaction. We are, at the present time, implementing metaknowledge on our computerised cognitive model of a pilot's cognitive activity 2°'z~ to evaluate metaknowledge usefulness.
REFERENCES 1. Shalin, V. L., Perschbacher, D. L. & Jamar, P . G . , Intent recognition: an emerging technology. Proceedings of Human Machine Interaction & Artificial Intelligence in Aeronautics and Space. CERT/ONERA Toulouse, France, 28-30 September, 1988. pp. 139-49. 2. Hajnicz, E., Absolute dates and relatives dates in an inferential system on temporal dependencies between events. Int. J. Man-Machine Studies, 30 (1989) 537-49. 3. Loui, R. P., Response to Hank & McDermott: Temporal evolution of beliefs and beliefs about temporal evolution. Cognitive Sci., 11 (1987) 283-97. 4. Volta, G., Time and decision. In Intelligent Decision Support in Process Environment, ed. E. Hollnagel, G. Mancini & D. D. Woods, NATO ASI Series, Springer, Berlin, 1986. 5. Decortis, F., Dimension temporelle de l'activit6 cognitive lors des d6marrages de syst~mes complexes. Le Travail Humain, 51(2) (1988) 125-38. 6. Muir, B. M., Trust between humans and machines, and the design of decision aids. Int. J. Man-Machine Studies, 27 (1987)527-39. 7. Valot, C., Paradoxes de la confiance dans les syst~mes d'aide. Congr~s ERGO-IA, AFCET-SELF, Biarritz, Sept., 1988, pp. 33-47. 8. Valot, C. & Amalberti, R., Les redondances dans le traitement de donnEes. Le Travail Humain, 52(2) (1989) 155-74.
9. Pitrat, J., Le futur de l'intelligence artificielle. Revue d'lntelligence Artificielle, 1(4) (1987) 69-88. 10. Gruber, T. R., Acquiring expert knowledge from experts. Int. J. Man-Machine Studies, 29 (1988) 579-97. 11. Hayes-Roth, F., Waterman, D. A. & Lenat, D. B., Building Expert Systems. Addison Wesley, Reading, MA, 1983. 12. Flavell, J. & Weilman, H., Metamemory. In Perspectives on the Development of Memory and Cognition, ed. R. V. Kail & J. W. Hagen. John Wiley, New York, 1977. 13. Norman, D. A., Some observations on mental models. In Mental Models, ed. D. Gentner & A. L. Stevens. Lawrence Erlbaum, Hillsdale, 1975. 14. Pedersen, S. A., Modelling the background of knowledge and belief. In Tasks, Errors and Mental Models, ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Taylor & Francis, London, 1988. 15. Wirstad, J., On knowledge structures for process operators. In Tasks, Errors and Mental Models ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Taylor & Francis, London, 1988. 16. Bainbridge, L., Types of representation. In Tasks, Errors and Mental Models, ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Taylor & Francis, London, 1988. 17. Goodstein, L. P. & Rasmussen, J., Representation of process state structure and control. Le Travail Humain, 51(1) (1988) 19-37. 18. Woods, D. D. & Hollnagel, E., Mapping cognitive demands: complex problem-solving worlds. Int. J. Man-Machine Studies, 26 (1987) 257-75. 19. Roth, E. & Woods, D. D., Aiding human performance, I Cognitive analysis. Le Travail Humain, 1"51(1) (1988) 39-64. 20. Amalberti, R., Valot, C. & Menu, J. P., Etude pr61iminaire dans le cadre d'aide intelligente au pilotage. Rapport CERMA 87-31, Paris, 1987. 21. Amalberti, R., Bataille, M., Deblon, F., Guengant, A., Paignay, J. M., Valot, C. & Menu, J. P., DEveloppement d'aides intelligentes au pilotage. Rapport CERMA 89-09, Paris, 1989.
Metaknowledge for time and reliability C. Valot & R. Amalberti Ddpartement d'Ergonomie Adrospatiale, Centre d'Etudes et de Recherches de Mddecine Adrospatiale, Base adrienne du CEV, 91228 Brdtigny Cedex, France
The task analyses of rapid process control highlight the significant place of knowledge about time and reliability in operators' activities. This knowledge is strongly involved in the adjustment .of activity to the environment and to a knowledge about the operator's own capacities: a form of metaknowledge. The elicitation of that metaknowledge shows the importance of strategies oriented to the cognitive resource management. Some outcomes are discussed: the place of metaknowledge in cognitive modelling; the usefulness to integrate it in decision support systems designing such as pilot's intent recognition. A model of metaknowledge is proposed in the frame of combat aircraft pilots.
1 INTRODUCTION
demands of the situation. This finality can lead to acts not optimal for the system but adequate with regard to the available cognitive means. This functional dimension of adjustment is a determining factor in developing aids, for example, for an airplane pilot. Whether it is for intent recognition 1 or to improve the operator's familiarity with the functioning of associated automatisms like the electronic co-pilot (developed simultaneously in the USA and in France), these 'human-like' heuristics are useful. They allow a more developed understanding of cognitive activities through a better definition of objectives desired by the operator.
The process controls characterised by an intense temporal pressure are interesting for the study of the dynamic aspects of cognitive activity. The operator must make decisions in a limited given time; his decision being often irreversible; in a partially uncertain environment; where the diversity factor is important. For instance, a pilot flying at 900 km/h and 200 m high, without visibility is rapidly confronted by his own cognitive limits. This is, even in peace time, an omnipresent risk. This type of continual process presents numerous demands for the pilot. He must therefore develop control strategies which are compromises between the demands of the systems and his own means, knowledge and know-how. How does he do it? The activity analyses show the importance of two types of specific knowledge allowing:
2 TEMPORAL PRESSURE MANAGEMENT The temporal dimension of knowledge has been studied in several ways. The study of the past events in mental organisation2 or the temporal evolution of logical relationships. 3 These approaches are not directly based upon a process control activity. In that specific field, studies show various forms of time in process control: 4'5 time of the process, of the
--the management of his own activity in relation to temporal pressure; --the management of reliability and confidence placed in the system, other operators or himself, Those types of knowledge are not directly related to the system and objectives one must attain, they have another 'finality': the management, by the operator, of his own cognitive resources depending on the
clock, subjective duration of events, temporal overlapping of events. The time studies in operator's activities are essentially centred on the difficulties the operator faces: synchronisation of events or the management of duration. We would highlight here another temporal dimension: the interaction between the temporal
ReliabilityEngineeringandSystemSafety0951-8320/92/$05.00 © 1992 Elsevier Science Publishers Ltd, England. 199
200
C. Valet, R. Amalberti
processing of a machine and the operator's own cognitive capacity to manage the events he initiates or controls, In concrete terms for instance, the operator will avoid the overlapping of processes requiring from him a high level of attention. To do this he will take into account his own cognitive limits, For a better understanding of these particular aspects of temporal pressure management, we analysed the activity of combat pilots undergoing a mission with high temporal pressure (low level flight without visibility at a high speed). In a recent study, we analysed eight pilots carrying out a classical mission in a single seat fighter with the same temporal, tactical and cartographical information. The aim being to attain a specific geographical point at a specific moment in time. We analysed their activity during flight preparation and the performance of the flight, this being in a simulator with cabin movement and external visualisation. Each pilot's flight preparation was very detailed, often being longer than the flight itself (1 h of preparation for a 45 min flight), We saw that all the pilots precisely planned the time factors of the mission, the different steps, the flight path and the expected values for a certain number of key parameters (fuel, distance, chronology, count down time...). The precision criteria required for this mission were less than 20 s in time and less than 20 m on the position. Starting from the time over target, the pilot calculated the departure time within a second. During the actual flight we saw that none of the pilots kept to his own planning. They either took off within 2 min of the stated time or didn't follow a large percentage of the flight they had calculated or prepared, In spite of these facts, the majority completed the mission within the required time and place. They completed their mission correctly but using adjustment rules from their own experience. In the pilots' opinion, the mission was correctly completed. They t D
T
O
I
V
~
E
~
~.r~p
sizes of the actual temporal bracket
plannedp a t h
~ actualpath Fig. 1. Comparison between planned and actual mission,
achieved their aim using the given criteria. But an aid (such as an intent recognition) which would have detected the deviations from the planned flight path would have given continuous warnings: deviation from the flight path, from the time for example... Such an aid would have been excessively off putting and restricting with regard to their numerous adjustments. Figure 1 shows an example of comparison between planned and actual mission. The preparation and flight analyses give numerous explanations of the observed deviations. 2.1 Flight
preparation
Various types of planning are elaborated by pilots. --Quantitative planning. It translates the mission order into a real flight and integrates temporal, tactical and meteorological constraints to give a quantitative flight description: way points, passage times, fuel consumption. They are used as references by the pilot. --Qualitative planning. It concerns two kinds of events: the foreseeable ones and the potential ones. The foreseeable events have not a precise spatial or temporal place, but the pilot knows that they will happen: it is the case for many system checks. The potential events have generally an external source: meteo, hostile fighter, ground to air defences. They all belong to the mission universe but the pilot is unable to predict their occurrence for a given flight. But all those events are major potential problems. They can't be integrated into quantitative planning. The pilot faces such a temporal load during the flight that it is quite impossible to really solve a complex problem. He is able to make decisions but in a preplanned context. 'What would I do if...'. With the present planes it is possible to make adjustments but a new complete inboard planning can't be considered. So, the pilot has to preplan his responses to a wide range of potential events. The rapidity of the process constrains the cognitive resource allocation. It is not possible to consider a new problem space and at the same time to follow the short term control of the flight path and the long term control of the mission.
2.2 Situation prepl.ning The more the pilot is experienced, the wider is the preplanned response range. This well-known fact is essential in flying under such conditions. An expert pilot will consider more detailed problems and develop more different potential responses. This supposes that effective cognitive representations can be hypothetically induced in order to obtain preplanned responses. The schema variety and the
Metaknowledge for time and reliability depth of knowledge allow one to link up hypothetical problem spaces and solution ranges, In that constrained situation, a specific type of knowledge has a specific place. A pilot says for instance 'After that way point, I know that I will only be able to control the" flight p a t h . . . ' . It is to be inferred that only short term control can be achieved, due to the step load and the pilot's cognitive resources. The main consequences are that, during that period, a failure cannot be dealt with, all the control will be flight path dedicated. The planned precision is only an indication but not an effective rule. To be solved, all those kinds of events would require too high a level of attention incapable of being supplied by the pilot. Consequently the pilot manages checkings, actions, level of controls according to his own representations of his resources and the cognitive demands of the situations: 'I check fuel 30 s after a way point because I know it is a relatively quiet moment, never before...'. His past experience shows him that it is not useful to maintain a high level of precision during the whole flight. It is more functional to manage tolerable gaps, integrating at the same time, cognitive resources, potential events and a sufficient precision: ' . . . a gap of 30 s is easily recoverable, it is better to begin with 2 min to spare, which are easily lost, rather than to ask for a take off at the right time and have to wait for a landing airplane. This delay is more difficult to recover...', For numerous situations, a pilot develops a bracket management of accuracy, To be quite efficient, that bracket management supposes that the size of the brackets is proportional to the pilot's depth of knowledge about the situation coherence. The depth of knowledge is not only judged for a brief period (some seconds). It is equally endowed with a life duration and a knowledge about the systems' most likely evolution. Those temporal dimensions are strongly proportional to the working out of a bracket of possibilities tolerating a sufficient effort for an acceptable precision,
3 RELIABILITY MECHANICS Reliability is equally a special case in process control studies. It is omnipresent in the pilots' talk though it is rarely evoked in cognitive modelling. Nevertheless the mistrust cost of a system or of another operator have major effects: restricted uses of devices, vicarious strategies are expensive in time and attention, Reliability is important to characterise the man-
201
machine relationship6'7 or the self-knowledge of an operator. 8 The reliability functional dimension is particularly interesting: it shows that reliability is at one and the same time a risk and a necessity: - - a necessity: ' . . . I cannot achieve that without a minimum of reliability...'; - - a risk: ' . . . to rely on a system is partly suicidal...'. The word reliability used in those two sentences refers to different fields.
3.1 A minimum of reliability The minimum of reliability conveys that a pilot is not able to continuously consult the displays, to detect and prevent a wrong tendency in parameters or to estimate a suspicious value. This is not compatible with the mission achievement or with the available treatment times. The pilot is compelled to consult the displays in a sequential way to leave time to insert another activity. But this supposes that the period of consultation is optimised according, on the one hand, to the risk not to consult a display and, on the other hand, to the surplus time (on hand) dedicated to general control of the mission, for instance. 'I never spend more than 30 s without looking inside the cockpit...' puts that optimisation into words. The compromise between different activities is based upon various factors being weighed up: • The life-span of information: The correct value of a parameter, in a given time, doesn't mean that 10 s later, it will always be coherent. The rapid evolution of an airplane incites unstable parameters. Consequently, the pilot has to predict and to distinguish between the effects of the evolution and the early beginnings of a breakdown. This short information life-span is a major distrust factor. • Knowledge allowing the prediction of an evolution: The pilot is in possession of a variable amount of knowledge about the evolution of a parameter. The possibility of good technical and temporal prediction of a value evolution is a good way to spare time to consult another value. • The accumulated experience about breakdown likelihood: The technical reliability of a system is equally acquired. Past incidents and the squadron experience allow the weighing up of the breakdown risk of each system. The functionality of that kind of reliability aims at sparing time and resources for more long term activities. Reliability is the only way not to have to use short term treatments. At such a high level of temporal pressure, the pilot has to partially rely on
202
C. Valot, R. Amalberti
the systems, not to have to look at the displays for a given duration, 3.2 But not an unreserving trust Reliability is a compromise and so a source of risk. Too long an interval between consultations leaves the possibility of an ignored evolution. The fast amplification of consequences and the difficulty to recover the chronological evolution of events are the main risks. It is the suicidal aspect of reliability, That kind of restricted reliability expresses a distrust as much of the system as of the operator himself. His own behaviour is the object of his own doubts. His past mistakes are memorised and the pilot develops strategies to avoid or to detect a wrong insertion of data or readings of parameters. The main strategies are based upon comparison with other data from other sources or the evaluation of coherence of system evolutions, But the distrust is a heavy consumer of time and cognitive activity. As a matter of fact a distrusted system obliges the operator to develop many roundabout strategies. The operator treats in parallel a first level control to achieve the mission and a second level control to reduce the distrust. This second level is generally not foreseen in the design of the system; the pilot has to look for indications to verify the consistency of evolutions. That treatment is not so precise. Such a restricted way has a cumulative effect: the comparison must be carried out with first level control. In the stressed periods this overlapping reduces the quality of control, 3.3 Working out of reliability Some particular mechanisms are used to work out reliability (or distrust),
3.3.1 Information comparisons This is a question of data sources. A number of dials
that whole chain: 'the spot on the radar display is where I expected, so navigation is good, the radar is working well and the computer too'. Some seconds are sufficient to verify (with various levels of accuracy) the state of the chain components. 4 WHAT KIND OF KNOWLEDGE? Owing to their content and their place in the control of processing, reliability and time, knowledge is typically a metaknowledge from two points of view. In AI, MK is mainly considered as a way to control and filter a knowledge base, to initiate reasonings and strategies (bootstrapping) 9 using expert knowledge. Anyway, another common characteristic of MK in IA sets is their nature: they are ill-defined knowledge (declarative or procedural), possibly (but not systematically) regarding human capabilities and limitations (here is a discrepancy from the classical AI definition of MK), which function remains to weigh the procedural knowledge of a specific domain in a large range of situations. This knowledge about weighing is gathered from numerous forms and origins: it is strategic knowledge for Gruber l° elicited from expert or from programmers' knowledge about system capacities and reliability. In all cases, the weighing is based upon factors such as: cost, time, reliability, danger, etc. 11 The concept of metaknowledge clearly appeared in the psychological literature two decades ago. In the early 1970s, the concept was introduced in the field of child development in order to explain how the child was able to adapt to new situations and to control actions. 12 Papers from these above mentioned authors are related to a specific form of MK called metacognition: 'What we know about our own cognitive abilities'. It is a knowledge about cognitive tools: their abilities, limitations, using strategies and prediction about performances they allow, etc. This metaknowledge has a strong influence over chosen strategies. The self experience weighs up a large number of the operator's choices. Other aspects of metaknowledge are interesting: mental models are not only fed with objective and
display similar information (for instance artificial horizon is displayed under four different forms simultaneously and the speed is displayed four times under three different units). The comparison is that much more useful because the information is obtained through various chains of computation (air speed with external probe or computed ground speed, etc). In addition to that, the dials feed a large polysemy (up to ten different meanings are, for instance, taken out by a pilot from a speed dial). The redundancies of indications detected by the pilot are an etficient way to verify the coherence of a process. 8
accurate notions. Some rough notions and beliefs are involved in numerous models formed upon the 'world'.~3'14 This feature is equally present in the mental models we form about ourselves. The opinion of our own merit is then a kind of metaknowledge. Last but not least, metaknowledge allows the compromise between the available time, the control accuracy and the management of mental resources.
3.3.2 Hierarchical checking
But it is a double edged mechanism:
The reading of a value is representative of a chain of computations. One reading gives information about
• on the one hand, it is functionally useful to spare a pilot's cognitive resources;
Metaknowledge for time and reliability • but on the other hand, a new finality appears in the pilot's strategies: to run the resource/time/accuracy compromise and to decide in that context. This fluctuating adjustment is a source of risk. 5 M E T A K N O W L E D G E AND ACTIVITIES Task analysis shows the significant place of metaknowledge in the understanding of a pilot's activities, How is it possible to involve metaknowledge in mental models and what could be the use of that kind of knowledge? 5.1 Metaknowledge and knowledge classes Numerous authors have proposed knowledge classifications. They are often designed according to a knowledge functionality centred on the system. The analysis is system goal oriented and formed on job analysis techniques. Wirstad 15 proposed 'a plant related knowledge terminology'. In such an analysis, the regulation and the compromise allowed by the metaknowledge are not easily placed, An example is given by Bainbridge. ~6 For this author, the main classifications are: 'A definition of the product and the sequence of transformation. The process limits. The process functions and behaviours, The plant components. The present state of the process', 5.2 Metaknowledge and decision models Metaknowledge is strongly associated with decision making. But classically the means-ends analysis is considered from the technical ends point of view. The metaknowledge expresses another type of ends: the operator's ends. The resource management and the cognitive 'economy' are not clearly explained in the decision-making models. But they are not far behind, Let us consider, by way of example, the well known Rasmussen's decision-ladder. 17 The nearest concept of metaknowledge in that ladder is in the Rule Based Behaviours (RBB). They are defined as a short cut stemming from operator's experience. They aim at saving time, avoiding long and fragile mental activities. Rule based behaviour is connected with patterns of situations, they avoid reasoning or active understanding to carry out the right action, From our point of view metaknowledge is required to construct the RBB. The knowledge of the pilot's functioning is a significant part of the development of those rules. The pilot's activity analysis shows that a part of the effective action is system ended and resource management ended, The Rasmussen's ladder should be completed by
203
the possiblity of weighing up the choices made according to a set of rules with criteria such as: --is the cognitive cost of the chosen solution compatible with foreseeable activities? --what is our self-judgement of our own level of capacity for a chosen solution? The above weighing up is equally interesting to understand the operators' mistakes. A lack of attention or a shortened procedure could result from a strong or an inadequate weighing up due to cognitive economy 'logic' or time saved in a loaded step of activity. Cognitive resource management knowiedge leads to a distributed view of knowledge. Knowledge associated with problem-solving has to be interactively considered with reliability, time, coherence and self-knowledge. 5.3 Metaknowledge and performance models The analysis in terms of 'model of competence' or 'model of performance' allows equally some comparison with reliability and time metaknowledge. 18'19 The model of competence aims at. the understanding of the task difficulties: what are the cognitive problems the operator faces to achieve his job? The empirical development of the models of performance is more comparable to metaknowledge. Those models describe the mechanisms by which the operators carry out their task. In that way, Roth & Woods ~9 develop an approach to describe the minimisation strategies for information needs or sharing activity strategies. Those two kinds of strategies aim at the interaction management between man and the machine cornponents of the task. Those models are explicitly designed to escape from the language of the application. This is a necessary way to integrate the operator's resource management with the problem space.
6 L U X U R Y O R NECESSITY? 6.1 Luxury? This absence of the taking into consideration of metaknowledge in a large part of significant models and knowledge elicitations suggests its relative uselessness. Consequently, that kind of knowledge would be a luxury. It seems therefore quite possible to design models and develop systems without taking into account metaknowledge. But what kind of models are they? Models of cognitive demands or models issued from job analysis are useful and necessary but they don't describe enough the operator's cognitive activity
204
C. V a l o t , R . A r n a l b e r t i
about information processing resource management: • A goal oriented analysis emphasises the elicitation of technical and functional knowledge. This creates an internal logic inherited from the process. It is then possible to develop a general and system goal oriented model of activity. But it is not possible to confront these general models with the detail of the actual operator's activities. They are not designed for that goal. • Pooling knowledge of numerous experts leaves on one side the metaknowledge. Its content is essentially individual even if every one uses metaknowledge. It is difficult to generalise such knowledge and to integrate it into problem solving or process control.
have the same roots because the coherence is inherent in their acquired knowledge. A last remark about the place of metaknowledge in models and systems: strong relations exist between metaknowledge and planning, choice strategies according to resource management. The rules used by a person to choose from several answers involves the effect of his personal possibilities and know-how. Metaknowledge is a hidden dimension of individual strategies. To avoid or forget it will not prevent metaknowledge existing and to have a place in the regulation of activities. Therefore it seems more useful to analyse it in order to design well adapted systems. 7 MODELLING M E T A K N O W L E D G E
6.2 Necessity! Nevertheless, a detailed description of an operator's effective activity is necessary to develop, for instance, specific aids such as an electronic pilot's associate. In that case, it is useful and necessary to describe functional knowledge and metaknowledge. This is a necessity for several reasons: • The understanding of the temporal knowledge of an operator and especially the management of his activities according to the temporal horizons supposes the description of the adjustments between Short Term for action control (15s of flight path for a low level flight) and Long Term for checking mission coherence. The space problem is dynamically modified for the application of schemas. Metaknowledge contributes to that modification. A good carrying out of the schemas depends not only on their availability and their content but also on the quality of their temporal adjustment to the events, • Reliability knowledge is more dedicated to short and middle term management. They contribute to a detailed adjustment to the reality. This knowledge is essential to adjust the schema achievement according to the permanent events. Short delays or uncertainties are smoothed out in this way. Reliability is equally essential to allow time management. This time is afterwards dedicated to the long term (planning) or middle term (schema running) activities. • Time and reliability metaknowledge are a pilot's coherence tools. For that reason their description is essential in understanding a pilot's major concern: to keep a good level of coherence according to the events, m question arises to elicit those coherent tools: they are strongly tied to the whole pilot's knowledge. It is not possible to describe metaknowledge without the associated knowledge. They
A model of MK in process control could be seen as a three layer model; each of the layers corresponding to a different form in which the same MK entity could be expressed by pilots (Fig. 2). Level 1 is basic, MK is a pilot's every day experience. In each situation, the pilot experiences relations between planned activities and achievement quality. From this comparison with actual capacities, empirical and individual rules arise. They deal with numerous aspects of human activities: memory capacities, stress effects, available strategies and their efficiency, personal reliability, etc. This knowledge is typically individual according to events, past experience and training. Its evolvement is permanent. But some common features appear between pilots: they are the results of human cognitive tool capacities. Level 2 is the direct expression of psychological needs in terms of resource management. It is made of general heuristics whose goals are to keep mental processes at acceptable boundaries in order both to execute actions and to release time for additional activities. As a consequence, this level is independent of the domain and would be relatively common between operators. Only the content of this level is varying according to the pilot's experience. It is of natural consequence that MK from this level is rarely HUMAN: Level1Metacognitionandeveryday experience: " r m not able to conu'ol 3 parameters simultaneously..."
Level2 General resourcemanagementprimitives: - Avoid situations in which responses are poorly
mastered - Encourage situations in which responses are efficient - Hold on normal procedure as far as possible - Preserve m a x of freetime
DOMAIN: Level 3 Domain specific procedures as known & applied by the pilot
Context & situationdecision rules
SYSTEM:
Handbook offlightcontrolmanagementprocedures
Systemmanagementrules Fig. 2. A three layer model for representing metaknowledge.
Metaknowledge for time and reliability
205
Principle of economy : (i) prefer well mastered procedures even when they are not considered as the best ones. (ii) maintain at best the seriality of procedures, Time management heuristics : (i) when unexpected or not well understood situation occur, give priority to converge to a short term stable position in order to release safe time for long term reasoning (ii) Manage immediate situation in order to be capable to apply at least one valuable answer to the most possible events which could occur in the future. (iii) when time pressure increases, prefer action to reasoning (ref heuristic n ° 1) Principle of control : (i) check expected result of action (ii) check periodically parameters which are not concerned by the on-going schemata
Fig. 3. Some of the general MK heuristics.
elicited during activity analysis because it is not directly connected with activity. It has just a historical connection with activity. Only a specific focus regarding resource management can inform the scientist of its nature, Level 3 reflects very specific domain procedures as known and applied by the subject. They are the concrete result of knowledge integration acquired on system management, decision rules in abnormal situations, and MK rules from resource management, Therefore, at this level, the concept of MK remains fully intertwined with the other knowledge representation levels (domain specific decision rules, system management procedures, etc.). Standard techniques of operators' interviews generally conduct to this level (techniques such as autofacing the videotape or eliciting knowledge with goal directed perspectives) and it is probably the main reason why proper MK is not evidenced in most of operators' activity analysis, A human's proper mental constraints and system using proper constraints are listed at the top and bottom of the figure. Context and decision rules are generated by experience and operational instructions, Inversely compared to the previously mentioned limitations (from human and systems), they depend on the nature of the task. The resulting procedural knowledge representations developed by an operator for achieving tasks are represented by the procedures of the specific domain, At this level (which can be understood as a goal directed level), it is not MK which is represented but the result of MK regarding the procedures, It results from the model of MK described above that there is a layer independent of the domain specific application. This layer is made of general resource management heuristics which weigh the domain specific procedures. At the present time, we have isolated a non-exhaustic list of these heuristics (Fig. 3).
CONCLUSION Time, reliability, and other things (risk taking, etc.) metaknowledge is a real necessity to precisely understand the pilot's activity. Many activities are only understandable at that second level frame. But the study of metaknowledge is not only a better way to describe an operator's activities but also an original context to design intelligent aids. Every pilot uses metaknowledge but its content is quite different according to past experiences, previous airplanes, etc. Its use is essential in evaluating and keeping the system coherence. That is a major problem due to the difficulty in gathering consistent data. The next step expected for inboard aids is specifically to help the pilot to manage coherence. From our point of view, metaknowledge affords a good way to solve a critical problem: how to give coherence to pooled expert knowledge? We choose to design an expert knowledge based elicitation. That is to say: to gather not only the expert knowledge but equally his coherence tools to implement it in a model of a pilot's cognitive functioning. That kind of model includes the expert's style of mission performing. All the pilots know the characteristics of that style because they know the way the chosen expert flies. They would be able to immediately understand and to apply information given by such an aid. That would not be the better way to perform the mission but a good one, that is to say an understandable one, a predictable and a reliable one. From a support system designer's point of view, a question remains: is it useful to take into account metaknowledge about individual capacities in systems? Two forms of response exist: • This metaknowledge analysis is interesting for a better understanding of human cognitive activities,
206
C. Valot, R. Amalberti
it expresses the 'human logic' of strategies, selections and actions, • This metaknowledge could offer some possibilities to easy support systems design and m a n - m a c h i n e interaction. We are, at the present time, implementing metaknowledge on our computerised cognitive model of a pilot's cognitive activity 2°'z~ to evaluate metaknowledge usefulness.
REFERENCES 1. Shalin, V. L., Perschbacher, D. L. & Jamar, P . G . , Intent recognition: an emerging technology. Proceedings of Human Machine Interaction & Artificial Intelligence in Aeronautics and Space. CERT/ONERA Toulouse, France, 28-30 September, 1988. pp. 139-49. 2. Hajnicz, E., Absolute dates and relatives dates in an inferential system on temporal dependencies between events. Int. J. Man-Machine Studies, 30 (1989) 537-49. 3. Loui, R. P., Response to Hank & McDermott: Temporal evolution of beliefs and beliefs about temporal evolution. Cognitive Sci., 11 (1987) 283-97. 4. Volta, G., Time and decision. In Intelligent Decision Support in Process Environment, ed. E. Hollnagel, G. Mancini & D. D. Woods, NATO ASI Series, Springer, Berlin, 1986. 5. Decortis, F., Dimension temporelle de l'activit6 cognitive lors des d6marrages de syst~mes complexes. Le Travail Humain, 51(2) (1988) 125-38. 6. Muir, B. M., Trust between humans and machines, and the design of decision aids. Int. J. Man-Machine Studies, 27 (1987)527-39. 7. Valot, C., Paradoxes de la confiance dans les syst~mes d'aide. Congr~s ERGO-IA, AFCET-SELF, Biarritz, Sept., 1988, pp. 33-47. 8. Valot, C. & Amalberti, R., Les redondances dans le traitement de donnEes. Le Travail Humain, 52(2) (1989) 155-74.
9. Pitrat, J., Le futur de l'intelligence artificielle. Revue d'lntelligence Artificielle, 1(4) (1987) 69-88. 10. Gruber, T. R., Acquiring expert knowledge from experts. Int. J. Man-Machine Studies, 29 (1988) 579-97. 11. Hayes-Roth, F., Waterman, D. A. & Lenat, D. B., Building Expert Systems. Addison Wesley, Reading, MA, 1983. 12. Flavell, J. & Weilman, H., Metamemory. In Perspectives on the Development of Memory and Cognition, ed. R. V. Kail & J. W. Hagen. John Wiley, New York, 1977. 13. Norman, D. A., Some observations on mental models. In Mental Models, ed. D. Gentner & A. L. Stevens. Lawrence Erlbaum, Hillsdale, 1975. 14. Pedersen, S. A., Modelling the background of knowledge and belief. In Tasks, Errors and Mental Models, ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Taylor & Francis, London, 1988. 15. Wirstad, J., On knowledge structures for process operators. In Tasks, Errors and Mental Models ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Taylor & Francis, London, 1988. 16. Bainbridge, L., Types of representation. In Tasks, Errors and Mental Models, ed. L. P. Goodstein, H. B. Andersen & S. E. Olsen. Taylor & Francis, London, 1988. 17. Goodstein, L. P. & Rasmussen, J., Representation of process state structure and control. Le Travail Humain, 51(1) (1988) 19-37. 18. Woods, D. D. & Hollnagel, E., Mapping cognitive demands: complex problem-solving worlds. Int. J. Man-Machine Studies, 26 (1987) 257-75. 19. Roth, E. & Woods, D. D., Aiding human performance, I Cognitive analysis. Le Travail Humain, 1"51(1) (1988) 39-64. 20. Amalberti, R., Valot, C. & Menu, J. P., Etude pr61iminaire dans le cadre d'aide intelligente au pilotage. Rapport CERMA 87-31, Paris, 1987. 21. Amalberti, R., Bataille, M., Deblon, F., Guengant, A., Paignay, J. M., Valot, C. & Menu, J. P., DEveloppement d'aides intelligentes au pilotage. Rapport CERMA 89-09, Paris, 1989.