A formal model of creative decision making

Robotics & Computer.Integrated Manufacturing, Vol. 8, No. 1, pp. 53-65, 1991

0736-5845/91 $3.00 + 0.00 © 1991 Pergamon Press plc

Printed in Great Britain

•

Paper A FORMAL M O D E L OF CREATIVE DECISION MAKING STEVEN H. K I M Laboratory for Manufacturing and Productivity, Massachussetts Institute of Technology, Cambridge, MA 02139, U.S.A.

Real-world tasks are often complex, uncertain, and constraine& The complexity arises from the need to consider numerous factors that are of varying degrees of relevance to the problem at hand. The uncertainty springs from imperfect information concerning, the state of the world, the repertoire of feas/ble alternatives, and the consequences of each action. The constraints are attributable to time, money, and computational resources as well as individual tastes and societal values. Despite the rich nature of practical tasks, previons work in decision making--whether in engineering, statistics, management, or economics--has focused solely on partial aspects of the problem. This state of affairs is reflected in the nomenclature, which involves categories such as "constrained optimization" or "decisions under uncertainty". If real-world tasks are to be addressed in a coherent fashion, it is imperative to develop a systematic framework providing an integrated view. The framework may then serve as the foundation for a general theory of decision making that can capture the full richness of realistic problems. This paper explores how these goals might he achieved. Algebraic and stochastic models of innovative decision making are presented. This is followed by an examination of idea generation in product design. Finally, suggestions are made for extending the work along both theoretical and empirical lines.

INTRODUCTION Over the past few decades, studies of problem solving have tended to focus on simple tasks. A representative example is that of parlor games such as checkers and chess. For such domains, abstract models often involved no more than the identification of a static set of states and predetermined operators for moving from one state to another. Unfortunately, real world problems tend to be more complex. This is especially true of innovative realms such as creative design or strategic planning. In this milieu, the decision problem is characterized by the following attributes: 9'2s'29 •

•

•

•

•

•

The problem may not be clearly identifiable. This happens in the realm of business, for instance, when a product begins to lose market share for no apparent reason. The functional requirements may not be obvious. What should be the speed, range, and capacity of the next-generation aircraft? The constraints may be nebulous. How much capital should be invested in advertising to introduce a new product line? Knowledge of the current state of the decision maker, as well as his information about the environment, might be imperfect. The imperfection

might result from inaccurate, incomplete, or even contradictory information. These difficulties may be due to inadequate resources available for collecting information; or to the stochastic nature of the problem domain, as in quantum dynamics; or to deliberate misinformation supplied by adversaries, as in a military context. The courses of action or operations available to the decision maker may be unclear. For instance, is it feasible to change the antitrust laws to permit greater collaboration among companies in an industry? The impact of alternative actions may be unknown. If, say, soya beans are substituted for animal fat in a food product, will the customers respond positively or negatively?

These attributes often characterize difficult problems. Unfortunately, most problems of consequence are difficult. Our task is to better identify the nature of difficult problems, model their modes of resolution, and explore how they may be supported by intelligent tools.

Dijficult problems and creative solutions Challenging problems are those that defy a ready solution. Creativity is often associated with novelty. In fact, some individuals would offer novelty as a synonym for creativity. The argument has been made, however, that novelty is a necessary but not sufficient attribute for creative endeavors, e8"29

Acknowledoement--This work was supported in part by the National Science Foundation through Grant No. DMC8817261. 53

54

Robotics & Computer-Integrated Manufacturing • Volume 8, Number 1, 1991

One could easily write a computer program to create patterns on paper, each one differing from all the rest. An example is one that draws triangles and squares: the i th program draws a triangle in every i da position starting from the first, and squares in all other locations. The first figure will consist only of triangles; the second will alternate between squares and triangles; and so on. Each figure differs from the rest and is novel, but few people would regard the drawings or the program itself as creative. Creativity, in fact, has been defined as purposive novelty. It is reflected in the solution to a task whose resolution is nonobvious. This perspective is characterized in the following definitions: 29

•

•

A difficult problem is one that admits no obvious solution, and no method for seeking such a solution. A creative solution is a resolution to a difficult problem.

These definitions are consistent with the everyday perspectives of creativity. For instance, the goal of an artist is to express his sentiment on canvas, a nonobvious task; the mission of a scientific inquiry is to discover hitherto unknown principles or facts. CONCEPTUAL MODEL The conceptual framework for intelligent action is given in Fig. 1. The key element in the figure is an agent or intelligent system. The agent is driven by a need to fulfill a goal or purpose (Kim, 3° Ch. 1). The goal may be externally imposed, as with a robot, or internally generated, as in a human. In order to fulfill its goal, the agent must acquire knowledge of the world. This knowledge comprises the condition of the environment as well as the internal state of the agent itself. Some of the knowledge is inherent in the system as defined; an example is the fear of height in an animal, or procedures for analyz-

ing stocks in a financial program. Other knowledge is acquired through the course of operation, such as the creation of tools or information on current stock prices. This collective store of knowledge is used to make a decision. As mentioned previously, the decision is usually made under conditions of uncertainty. Finally, the decision must then be implemented through some action or operation which changes the state of the world. Since the decision as well as the action involve probabilistic elements, the resulting state will be only partially foreseeable: the change in state may be conducive to the attainment of the overall objective, or indifferent, or even detrimental. For this reason, the action must be monitored in terms of its impact on the world. Then the new state must be evaluated to determine whether the goal has been reached. If so, the current cycle of actions terminates; if not, a new cycle begins once more.

FORMAL MODEL The quest for a decision theory of creative problem solving is itself a challenging task. As with most research efforts, a theory of creative decisions will surely evolve over time rather than appear one serendipitous morning. The goal of this paper is to propose an initial approach that will serve to guide future efforts in this direction. A starting point for a creativity theory should build on existing fields that relate to problem solving under uncertainty. Three such disciplines having a quantitative perspective are decision theory, information theory, and stochastic optimal control, as depicted in Fig. 2. The general decision problem under uncertainty may be formalized in terms of states, operators, and probabilities. The components of the problem are as f o l l o w s : 9,13,25,27

Mathematical tools

@

ptimaJ

/

\

Referent models

Fig. 1. A general model of purposive behavior. An agent strives toward its goals by drawing on its knowledge of the world, making a decision, and taking appropriate action. The world encompasses both the agent and its environment.

Fife 2. Overall framework for a theory of creative problem solving. Routine or easy problems may be viewed as limiting cases of creative problems in the spectrum of difficulty.

Formal model ofcreativedecision making • S. H. KIM •

A system state space S of possible states of a

system. W h e n the state space is discrete and finite, it may be associated with the enumerated set

{sl, s2 ..... s.}. •

•

•

•

•

• •

•

An environment state space E of possible states of the environment. When these states can be quantized or discretized, and are finite as well, then the environmental space is given by {El, E 2 . . . . . Era}. A world space W defined by the pair (E, S). In other words, the world W is characterized by the condition of the environment and the system. The set of operators D which map elements of the world space to each other, but in unreliable (probabilistic) fashion. In other words, the mapping fl: W-~ W is associated with a probability function. The set of actions A which have some physical effect on the world. The actions constitute a subset of the repertoire of operators. The probability function P which defines the likelihood of transitions of W based on each operator in f~ and action in A. More specifically, the probability vector p~j indicates the likelihood of transformation from si when tojef~ is applied. For instance, if W = {w 1, w2, w3} then the probability vector P2¢ = (0.1, 0.9,0) denotes that the likelihood of moving from s 2 to w 1, w 2, and w a are 0.1, 0.9 and 0, respectively, when operator COg is invoked. The uncertain initial state, wo, a particular element of the world space W. The goal set G, a particular element of 2s, the power set of S. For instance, G = {Ws, ws} would indicate that either of w s or w a are acceptable final states of the decision process. The threshold 0, defining the lower bound on the level of confidence that the goal has been achieved. The value of 0 is a real number in the interval

[0, 13. The input Q defines the impact of the environment E on the system. The input may be physical as in the collapse of a floor, or merely informational, as in a visual panorama. An operator to is a computational activity that transforms the system from one state to another. An operator is elementary if it cannot be decomposed further, and is complex if it consists of two or more operators. The act of applying an operator is called an operation. The nature of an elementary operation is defined by the granularity at a particular level of reasoning. To illustrate, an elementary operation at the strategic level may be "Determine the rate of return for a hydrogen-powered automobile"; it may be "Calculate reliability" at the operational level; or "Multiply X and Y" at the processor level. An action a is a physical activity which causes a change in the state of the world. The action is called internal if it is not directly discernible to an outside observer; examples lie in "Activate temperature sen-

55

sors" or "Increase pressure in the hydraulic reservoir". An external action, in contrast, is discernible to an observer. An example lies in "Pick up the book". An external action is also called a response. As with operations, each action may be elementary or complex. An action must be guided in some way: it must be invoked, monitored, and terminated. Since these involve informational processes, each action can be associated with one or more operators. As a result, the set of actions may be regarded as a subset of the collection of operators.

Decision making process A strategy or plan T is a sequence of operations. In general, it is a stochastic rather than deterministic process. Instead of a predetermined structure W(t + 1) obtained from w(t) and Q(t), T assigns probabilities (P1, P2 .... ) to the permissible set of structures (wl, w2 .... ) and chooses the next structure. Let P be the set of permissible probability distributions over W. Then T first selects P(t + 1) from P, followed by a selection of w(t + 1) from w. For the deterministic case, P(t + 1) assigns the single value 1 to a particular structure w in W. In practice, the determination of w(t + 1) from w(t) is effected by an operator from the set fl = {co: W - ) P}. The adaptive plan is then given by the mapping T": Y. x W - ) f~. Given an operator tot selected at time t: COt= T"(Q(t), W(t)) the resulting distribution over w is

P(t + 1) = tot(w(t)). A key requirement in using self-learning systems is to determine effective plans under differing environmental conditions. Let T' be the set of plans to be compared for use in the set of possible environments E. For a specific environment E in E, the input Q to the plan must contain some measure of the efficacy of T in fulfilling the goal G. In other words, a component of Q(t) must contain the payoff ~(W(t)), given by the mapping W--)Reals, which measures the degree to which the functional requirements are met in the environment E. In the special case where the plan acts only on a knowledge of the reward, Q(t) = ~E(w(t)). Plans that utilize information in addition to reward should perform at least as well as those which use only payoff. To compare the efficiency of adaptive plans against each other, we must define some criterion k. Constructing such a criterion is a nontrivial task. For example, consider two plans: one performs superbly in some environments but poorly in others, while the other performs reasonably well in all cases. In this case, the selection of one plan over another should be guided by the particular application. The relationships among these parameters are depicted in Fig. 3.

56

Robotics & Computer-Integrated Manufacturing • Volume 8, Number 1, 1991 Functional requirements

F

System S Strategic knowledge

Kernel systemS k I

,_T

'11

I Information I

IACtJOb

Response

Fig. 3. Structure of the abstract framework for intelligent agents.

The decision making problem under uncertainty may be defined as the quintuple consisting of the preceding objects: ~ = ( W, Wo, f~, G, 0). As discussed previously, each component of the problem might be defined only implicitly. For instance, the world space W may be infinite rather than finite; or the uncertainty values p~j may be unknown; or the set of operators may evolve over time; or the confidence threshold 0 may be unknown a priori. Despite these practical limitations, the formal model provides a rigorous terminology and a basis for the systematic investigation of problem solving. The task of the decision maker is to apply a sequence of unreliable operators {COx,o92.... } beginning with the initial state Wo and ending with some final state w,. The first transition is defined by col(wo) = wl; the second by c02(wl)= co2(col(w0)); and so on. Each element wi of W has associated with it a probability function P which indicates the chance of the elements being in the goal set G. The uncertain problem is solved when an uncertain state w, is reached whose total probability exceeds the threshold 0 of attaining G. In other words, P(w n ~ G) > 0. Let fl = {cox,CO2.... } be the set of operators available to the agent. Then there is a transition function Pi defined for each cos; that is, Pi = P(coi). When the transitions are linear, we may represent the change from a world at step k to the next by a matrix multiplication: W k + l,i = P i W k .

Suppose the system has N states and environment M states. Since W - (S, E) r, the transition function P~

will be a square matrix of order (N + M). Further, if there are L operators, the complete transition function will be defined by P = I-P1, P2 . . . . . PL]: a 3-dimensional matrix of order (N + M). b (N + M). The uncertain problem involves the transformation of uncertain current states into some desired end state through a series of unreliable transformations. Since the transformations are nondeterministic, the system might be in any of a number of states. At any given time, the states having positive probabilities may be called a zone. Each uncertain zone refers to a set of possible worlds which may obtain, as in modal logic. In general, the decision maker has imperfect knowledge of the probabilities of being in various states within a zone. The values of probabilities used by the agent are only best estimates based on the available evidence. The decision problem under certainty can easily be seen as a special case of the uncertain problem. More specifically, the deterministic problem occurs when each zone a consists of a single state of W with probability 1; and the threshold 0 has the value 1.

Optimizing vs. satisficing

The economist speaks of the optimizing decision maker. For example, a woman with a certain amount of money will select a basket of goods that maximizes her happiness or utility. The management scientist, on the other hand, speaks of the satisficing agent. 2'52 In deciding whether or not to introduce a new product, a manager faces many constraints: the window of opportunity may close in 6 months; he may have only 3 subordinates to

Formal modelof creativedecisionmaking • S. H. KIM assist him; his budget for market research may be $1 million; and so on. In reality, the notions of optimization and satisfaction are not mutually exclusive. More specifically, satisfaction may be viewed as constrained optimization. In other words, the role of a satisficing agent is to generate an optimal solution subject to all the resource constraints, whether in terms of time, money, computational capacity, or other factors. As a result, the problems addressed by the economist and management scientist may be regarded under the unifying theme of optimal decision making under constraints.

A cybernetic theory of knowledge In this paper, we define knowledge as that quantity which increases the likelihood of attaining a set of objectives. In a world riddled with uncertainty and unpredictability, no knowledge can guarantee success. Hence the best that knowledge can do is to help assure the attainment of current or future goals.* •

Knowledge refers to an awareness of the world that can be used by an agent to attain its goals.

The awareness may be declarative or procedural. The declarative knowledge refers to the state of the world, whether in terms of the environment or the system itself. Examples of such knowledge are "Gold is a good conductor of electricity" or "High market share tends to yield high profit margins". Procedural knowledge deals with specifications for attaining welldefined goals; an example is "Insert key into ignition, then twist it" to start an automobile. We may define information as a subset of knowledge, and data as a precursor of information. More specifically, we introduce the following definitions:

• •

Information is a type of knowledge pertaining to the state of the world. Data refer to elementary quantities that can be converted into information. This conversion is accomplished by using other items of knowledge.

Examples of information lie in the statement "The ambient temperature is 16.5 degrees Celsius" or "The fuel tank is half full". These statements in turn contain data such as the number "16.5", the temperature scale "Celsius", or the fuel level "half". An item of data such as "16.5" can be used in conjunction with higher-level knowledge to determine, for instance, that it refers to the temperature on the Celsius scale rather than the atomic weight of oxygen. In turn, the information "The ambient tem-

* The cyberneticviewof knowledgehas also beencalledgeneralized informationtheory(see Haglerand Kim,19 Kim,25 Kim,Ch. 1 and Appendix E3°). This is in contrast to the classical information theory,whichconcernsitselfwiththe syntacticreceptionofsymbols transmitted through a communicationchannel"9 or with purely statistical correlations among variables;5 neither of these latter viewsfocuseson the semanticimport of the conveyedsymbols.

57

perature is 16.5 degrees Celsius" might be used with declarative knowledge such as "The comfort zone for bioengineered algae lies between 20 and 27 degrees Celsius" and the procedural item, "Turn on the heaters if the ambient temperature falls below the comfort zone". Such declarative and procedural assertions may well comprise the knowledge base for the objective of maintaining a viable hydroponics plant.

KNOWLEDGE-BASED TOOLS Knowledge-based systems may be used to support high-level decision making. These tasks are often characterized by uncertainties in both the input data and the relative efficacy of alternative actions. An example in the business domain is a decision support system for evaluating corporate revenues as a function of sales volume minus marketing costs; the system compares actual values to expected figures, identifies discrepancies, and helps to diagnose the underlying source of problems. 44 Even in such highlevel tasks, computerized systems can augment the capability and efficiency of individual decision makers.

Quality vs. efficiency Studies indicate that efficiency can be greatly enhanced with only a small loss in the quality of results. This phenomenon has been demonstrated, for example, in the design of production lines: an evaluation of only 1% of the potential space of 4096 solutions can lead to an expected quality lying within 10% of the optimal; or an evaluation of 10% to reach 1% of the optimal with a better-than-even chance. 53's4 The improvement in performance is obtained in conjunction with an order of magnitude increase in productivity: from about 13 days to manually create a single design for a small-to-medium gear production line, to less than 1 day to explore dozens of alternatives with the help of the knowledge system. Another study involved project scheduling for the construction of a highway. In a simulation study of 1000 trials, a decision-theoretic approach relying on only a handful of alternatives at each stage closely approximated the optimal solution obtained through the more exhaustive techniques of mathematical programming; in fact, the loss in quality seemed marginal. 6 These examples highlight the fact that knowledge-based systems can enhance both the quality and productivity of decision making, and that even partial searches of alternative solutions can yield results that approach the global optimum.

APPLICATION OF THE FRAMEWORK TO DEPTH-BREADTH SEARCH

The utility of the framework for creative problem solving may be examined through a number of case studies. One such application is the role of breadthfirst vs. depth-first search.

58

Robotics & Computer-Integrated Manufacturing • Volume 8, Number 1, 1991 Table 1. Parameters of the formal model, interpreted in terms of the breadth-depth search

problem Symbol

Object

Interpretation

S

System state

E G f~

Environment state Goal state Operator set

P

Transition probability

Understanding of the locations of nuggets; defined by a probability function f(x) in the search space x Location of nuggets Possession of nuggets Exploration or excavation in x, y, and z directions Probability of moving from one state to another

Interpretation in terms of the formal model The parameters of the formal model must be interpreted according to the domain of inquiry. The interpretation for the breadth-vs.-depth tradeoffproblem is given in Table 1. In this context, S represents internal knowledge of the location of the nuggets; alternative states represent worlds in which the agent has greater or lesser knowledge of the likely locations of nuggets. The goal state G is reached when a nugget of acceptable quality is assimilated. The environment states E represent the actual locations of nuggets and their respective qualities. In a physical context such as prospecting for gold ore, the environment E will tend to change over time: earth will be removed, and so will the nuggets. In an abstract situation such as scientific inquiry, the environment will remain unmoved; only the agent's understanding of the conceptual environs will change. The operator set Q represents exploration in particular directions--say in x, y, or z. The effect of an operator on the state of the agent or the environment is moderated by the transition function P, which defines the likelihood of moving from one state to another. As explained before, the environment may change due to the action of the agent in a physical context, but not the abstract; when no change is indicated, the corresponding transition probability is 0. Breadth-first vs. depth-first search When approaching a new territory or intellectual domain, should the problem solver use a breadth-first or depth-first strategy? The appropriate strategy will depend on his prior knowledge of the expected distribution of results. Figure 4 depicts a field of inquiry; the x and y axes represent horizontal directions, and z the vertical depth. The nuggets or desired results are assumed to be strewn around the x-y field and to lie at various depths z. The best approach will depend on the expected distribution of nuggets. Suppose, for instance, that the nuggets are likely to exist in one region (xo, Yo, z) of the x-y plane; the depth z is not known beforehand. This situation is depicted in the uniform density fl(x, y, z) of Fig. 5. In this case, it would make sense to perform purely a depth-first search at (Xo, Yo).

In contrast, f2(x, y, z) denotes a low density at location (x, y). If the value of f2 is no higher at (x, y) than at any other spot, then a depth-first search at the location would be counterindicated. In fact, if the density f(x, y, z) is approximately uniform over the entire field and at all depths (up to some finite limit Zm=,), then a breadth-first search would be appropriate. The density fa(x, y, z) in the same figure depicts the case where the results cluster at depths between z2 and z3, at the location (x, y). In a similar way, f4(x, y, z) indicates a high likelihood of concentration at depths

/

Breadth-first

x

DepthFirst

•

Fig. 4. Search strategy vs. expected results. The best app r o a c h - b r e a d t h - f i r s t , depth-first, or other search--depends on the expected distribution of nuggets.

Probability density f(x, y, z)

1 z3 - z2

f4 1 z4-zl

f~ I.

..... f2.. . . . . . . . . . .

I

[ Zl

........ 1 z2

z3

z4

Fig. 5. Two probability densities for the distribution of nuggets along the z axis. F o r instance, fl(x, y, z) is the uniform density on the interval [Zl, z4]. The density f l would suggest a depth-first search, and f2 a breadth-first strategy. (Each fi must integrate to 1 over all values of x, y, and z.)

Formal model of creative decision making •

between z2 and z 3, although not quite so promising as f3. If the values of f3 and f4 are higher here than at other locations in the 3-dimensional field, then a depth-first strategy would again be appropriate, Implicit in the choice of depth-vs.-breadth strategies is the assumption that the search process entails some cost. The cost may be primarily monetary, as in mining for diamonds; or temporal, as in searching through a conceptual space. Suppose that there is a fixed rate of cost Co for each unit of activity, whether removing a cubic meter of dirt or examining an elementary region of intellectual space. Without weakening the argument, we can assume unit cost; that is, Co = 1. We assume that the goal is to find a nugget, or a solution of satisfactory quality. Then how does the shape of the probability density affect the optimal search strategy? Without loss of generality, assume that breadth-first and depth-first searches occur as shown in Fig. 6. In other words, breadth-first involves changing the x variable most rapidly, followed by y, then by z; similarly, depth-first involves changing the z variable most rapidly, followed by x then by y. In this figure, the value of each variable has been normalized so that each of x, y, and z varies from a low of 0 to a high of 1. Now, what is the cumulative cost of evaluation when the search is at an arbitrary point Q = Q(x, y, z)? The cost is proportional to the volume of space examined by each method. The volume VBdue to breadth-first examination is VB = XmaxYmaxZ = Z.

The second equation results from the normalization of variables; in other words, Xm~x-- Ymax~- Zmax------I. In a similar way, VD =

XmaxYZmax ~

y.

Let f(x, y, z) denote the probability density of finding an acceptable solution. Then the correspond-

/

1

// I I I I ; I I I I I

1-

I I I I ; I I I I I

I I I I ; I I I I I

t

t

~ :

x

B1

B2

B3

59

S. H . KIM 1

0.5

x

J J

J M

"~--- h

y ~~I~L

0.5

S ° fl P

f°

z Fig. 7. Let fl be the uniform density on the "horizontal" block having comers A-B-C-D-E-F-G-H; and f2 the "vertical" block defined by J-K-L-M-P-Q-G-H. A depth-first strategy is indicated for ./'1; a breadth-first search would have yielded no nuggets in the first half of the excavation process. Since the "location" of the f2 block is not k0own a priori, a breath-first inquiry would determine its location and a subsequent deep foray uncover the nuggets.

ing probability of success due to breadth-first is

PB =

fof0fl

f ( x , y, z) dx dy dz

and that for depth-first is

PD =

fofofo

f(x, y, z) dx dz dy.

Which strategy is superior? The answer obviously depends on the nature of the density f(x, y, z) as well as the importance of the cost c. Some simple density functions are shown in Fig. 7. In the figure, fl(x, y, z) is the uniform density defined by the "horizontal" block in the bottom half of the search space; f2(x, y, z) is given by the "vertical" slab occupying the eastern half of the space. For fl, depth-first would be superior to breadthfirst; in fact, a breadth-first approach would have uncovered half the search space without exposing any nugget. On the other hand, f2 is more amenable to a depth-first search if the location of the slab along the x axis is known beforehand; otherwise a breadth-first search would have revealed the position of the block and could be followed by an informed depth-wise examination.

• Q(x, y, z)

z

Fig. 6. Directions of search for breadth-first vs. depth-first search. For breadth-first, the BI sweep is followed by B2, then by B3, etc. until Y.,z = 1 is reached; then the search begins anew at the next higher value of z. For depth-first search, the D~ sweep is followed by D z, then D3, etc., until x ~ = 1 is reached; then the process is repeated at the next higher value of y.

Fusion of breadth and depth strateoies Consider the task of developing a science of manufacturing systems. Such an effort must draw on previous disciplines such as systems theory, control engineering, information theory, thermodynamics, microeconomics, operations research, automata theory, and many other fields. How is an investigator to approach this task? Should she select one discipline in particul a r - s a y , information theory--and explore its relevance to manufacturing science to the exclusion of all other disciplines? Or should she conduct a breadth-

60

Robotics & Computer-Integrated Manufacturing • Volume 8, Number 1, 1991

wise search, exploring all plausible disciplines in parallel? In practice, neither approach by itself is reasonable. A depth-wise search--say information theory, followed by an exhaustive examination of control engineering, and so on--will present several difficulties:

1. Partial relevance. The results of a particular field such as information theory will likely have only a partial bearing on the emerging theory of manufacturing. For instance, information is a key ingredient of production operations, but a factory is much more than just an information process. 2. lnfeasibility. Most disciplines continue to evolve over the years. How is a researcher to explore the full length and breadth of information theory? This would be a ceaseless task. 3. Lack of synergism. To pursue only one referent discipline is to lose out on the synergism that can accrue by the juxtaposition of two or more fields. For instance, the fusion of thermodynamic principles with statistical reasoning has resulted in the field of statistical thermodynamics. At the opposite end of the spectrum, a purely breadth-wise search has its own shortcomings. These are as follows.

1. Differential relevance. For most realistic problems, certain areas or disciplines are more likely to be fruitful than others. To pursue all leads with equal intensity is to ignore this prior knowledge. 2. Limited resources. A parallel search of all potentially relevant fields will probably overtax the resources available to the researcher or project. Morever, most of the referent areas or disciplines continue to evolve, making even more unrealistic the task of covering each field. 3. Superficial comprehension. Due to limited resources, any understanding of the referent domain is likely to be tenuous, incomplete, or biased. Often some knowledge can be more dangerous than no knowledge at all. It is clear that for most difficult problems, neither a depth-wise nor breadth-wise search for the solution is realistic. A more workable strategy which is practiced by design or accident is the fusion of the two approaches. This approach has been called the Method of Directed Refinement: 29 0. Ascertain the nature of the problem and the criteria for sucess. 1. Perform a breadth-first search of potential solutions, taking into account all the knowledge available so far. If a satisfactory solution is found, then stop. Otherwise, after a reasonable investment in effort, go to Step 2. 2. Select the most promising direction identified by Step 1. Perform a depth-first search of this direction. If the solution is found, or if the cost of further effort

would exceed the expected benefits, stop. Otherwise, after a reasonable investment of effort, go to Step 1. What constitutes a "reasonable amount of effort'" at each stage? The answer obviously depends on the nature of the task, as does the specification of the problem objectives and criteria for evaluating solutions. STOCHASTIC MODELS OF IDEA GENERATION This section explores probabilistic models of idea generation during a problem-solving endeavor. The models are appficable to creative problem solving in diverse domains. For concreteness, however, we will often invoke the application of product design. The design process involves the fulfillment of functional requirements through some artifact. The composition and structure of the engineered device requires the generation of ideas. These ideas may be mundane or innovative, depending on the nature of the task and the specifications. Given the differential utility among ideas, they may be ordered relative to each other. The ordering defines, at least implicitly, some measure of utility or quality. Further, given limitations to our cognitive abilities, ideas take time to generate. In general, ideas of greater utility seem to take longer to germinate than those of lesser value. Based on the differential utilities of ideas and their periods of incubation, it is plausible to develop models of creative problem solving. One such domain pertains to that of engineering design. The following subsections present several stochastic models of the design process.

Quality of ideas Suppose that the likelihood of generating an idea is constant over time, and independent of the quantity generated in the preceding periods. More specifically, assume that 2 is the likelihood of obtaining exactly one idea within a small time interval, and that the probability of multiple ideas is negligible. Then the interarrival time T between ideas assumes an exponential density with parameter 2: f(T) =

/20e-aT;: for T > 0 for T < 0.

The ideas are generated with a mean time and standard deviation of 1/2. Now assume that the quality of an idea is constant over the interval [0, q]. In other words, the quality Q of a particular idea takes a uniform density on that interval:

/1/q: for 0 < Q < q f(Q) = \ 0 : otherwise. Suppose that the required quality of an idea is q,. Then no ideas of sufficient merit will emerge if q,

Formal model of creative decision making • S. H. KIM

exceeds q. If q, is less than q, then the likelihood that an idea will be acceptable is (q - q,)/q. What happens if the quality takes an exponential form with parameter r/rather than a uniform density? The chance that an idea is adequate is again given by the probability that the quality Q exceeds q,. This is the area under the curve to the right of q, in Fig. 8. In this case, the probability as a function of q, is e x p ( - r/q,). What is the chance of obtaining precisely k adequate candidates among n ideas? This event takes a binomial density wi.th parameter p, the probability that an idea is successful. In other words:

P(K = k) = "Ckpk(1 - -

p)n-k for 0 <

of any or all ideas which might be generated in D days?" Another example is " H o w long will it take to develop enough ideas to solve this problem?" As it happens, the expected value and standard deviation for a random sum of independent random variables are related in relatively simple ways to the properties of the underlying variables. We first state the results as a theorem.

Theorem. Suppose X 1, X 2. . . . . are independent random variables having a common density function. Let S = X1 + X2 + ... be the sum of a random number N of the random variables Xi. Then: (a) E(S) = E(N)E(X) (b) Var(S) = E(N) Var(X) + [E(X)] 2 Var(N)

K _ n

where "Ck is the binomial coefficient given by n!/ k ! (n - k)! As a special case, we note that the probability of getting at least one adequate idea among n is given by P(K > 1) = 1 - P(K = 0) = 1 - (1 - p)". If the quality of an idea happens to take a uniform density, we have already seen that the parameter of the binomial density is p = ( q - q,)/q. In a similar way, the binomial parameter for an exponential quality density is p = exp(-~/q,).

Cumulative characteristics of multiple ideas In generating ideas to resolve a problem, their cumulative characteristics are often of interest. This is particularly the case since the solution to a problem rarely consists of a single idea but rather a collection of such. 2s'29 Examples of cumulative characteristics are the total quality of a sequence of ideas, or the total time for their generation. For a fixed number of independent variables, the expectation of the sum is simply the sum of the expectations; and similarly for the variance of the sum. More specifically, let S = X1 + X2 + ... + X , be the sum of n random variables X~, each having m e a n / z and variance 02. Then the expectation of the sum is E ( S ) = n/z, while the corresponding variance is Var(S) = na 2. This kind of analysis corresponds to the query, for instance, of " W h a t is the expected total quality of n ideas?" The more interesting case involves the random sum of random variables: here the number of variables is allowed to vary. This type of analysis might correspond to the query "What is the expected total quality

f ( Q ) = ~ l e ~10

: Q>O : otherwise

where E denotes the expectation and Var the variance of the respective arguments; and all parameters are assumed to exist. The proof is available elsewhere, a'al To illustrate, let Q = QI + Q2... QN be the total quality of a sequence of ideas having quality Qi. Suppose that the mean and variance of N are given by /~N and tr2, while the Q~ are exponential with parameter ~/. Then the expected value of the cumulative quality is E(Q) = I~N/rl while the corresponding variance is Var(Q) = (/~N + tr2)/r/2" The total time to generate a set of ideas of quality Q is familiar. For instance, if the interarrival time between ideas is exponential with parameter/A then the total time T has the mean E(T) = #N//~ and variance Var(T) = (/~s + tr2)/# 2. Thus far we have concentrated on some characteristics of the idea-generation process. What can we say about optimal strategies in resource allocation? We now turn to decision rules for the amount of effort to be dedicated during a problem-solving process.

Optimal length of search We consider the impact of the requisite quality of an idea on the duration of search. Our scenario involves a designer who generates one idea during each time period. The quality Q, of the nth idea has a fixed density f which is assumed to be known to the designer. Each period of the idea-generation process entails a cost c, which may include factors such as the salary of the designer, the cost of equipment, or other aspects. In this context the designer is not looking for an idea of a specific quality, but rather seeks to maximize the value of the best idea, minus the operating cost. The longer he searches, the higher is the expected utility of the design activity; but so is the cost of generating it. More specifically, the overall worth of the idea-generation process is W, = max{Q1, Q2 . . . . . Q,} - nc.

0

qr

quality Q

Fig. 8. Probability that the quality of an idea exceeds q, given an exponential density.

61

(1)

The designer's task is one of optimal stopping: at which point should he stop and select the best of the ideas generated up to that time?

62

Robotics & Computer-lntegrated Manufacturing • Volume 8, Number 1, 1991

1

//

a density function f, and IV, be a sequence of returns as defined in Eq. (1). Moreover, let W* be the overall value of the reward sequence defined as the unique solution of the relation:

f

max (W', O)

c =

0

Quality Q W'

Fig. 9. The cumulative distribution F(Q) of the quality Q, and the function max(W*, Q).

An optimal stopping rule is guaranteed to exist when the expected value of FV~ is bounded and the variable tends toward infinite cost in the long run. This key result depends on the following lemma. Lemma. Let Q1, Qz . . . . . be a sequence of random

variables having a common density function f . We define the following quantities, where c is a positive real number: W, = max{Q1, Q2 . . . . . Q,,} - nc W* = sup, I4I,. Then the following conditional results apply: (a) If the expectation of f exists, then lim W~ - ~ as n --* oo, with probability 1. (b) If the variance of f is finite, then E(I W ' l ) <<90.

The lemma is proved in DeGroot. 7 Note that the lemma does not require the independence of the qualities Q~. Result (a) states that when the mean of f exists, I/V, tends to decrease without bound as n increases. This is entirely plausible in our application, since ideas are not likely to be worth the cost of an unbounded amount of effort to generate them. According to the second result, part (b) of the lemma, a finite variance on the Q~ensures a finite mean for the overall worth W*. These two results are required for the following theorem. Theorem. Let W1, W2. . . . . be a sequence of variables

from a density function g. Further, assume the following conditions: (a) lim IV, --. - oo as n ~ oo, with probability 1. (b) E(I sup, W, I) < oo. Then an optimal stopping rule exists. The proof is given in DeGroot. 7 To summarize, if the mean and variance of the Q~ exist, and the worth W, drops without bound, then an optimal stopping rule exists. The next theorem defines the structure of such an optimal stopping rule. Theorem. Let QI, Q2 . . . . . be a sequence of indepen-

dent, indentically distributed random variables with

(2)

(Q - W * ) f ( Q ) dQ.

Now if E(Q~) < ~ for each n = 1, 2. . . . . then there exists an optimal stopping rule T which maximizes E(W0 and takes the following form: stop if Q, > w* and continue if Q, < W*. The proof is available in DeGroot. 7 According to the theorem, the optimal stopping rule specifies that the first Q, matching or exceeding W* should be accepted, where W* is obtained as the solution to Eq. (2). The threshold W* may be called the reservation value. What is the basis for Eq. (2)? Let us take the first quality Q1 from a sequence of independent, identically distributed random variables Qt, Q2 . . . . . If the optimal stopping rule in the above theorem is followed, the expected return is E (max {W*, Q1}) - c. In other words, the expected worth of the strategy is the higher of the observed value QI or the overall worth W*, less the unit cost c of operation. But this is precisely the overall worth of the strategy at each step: W* = E max (W*, Q1) -

c.

(3)

Let us consider the expectation term in the preceding equation. It is equivalent to the following (see Fig.

9): E max (W*, QI) = w*

fo"dF(Q) + f. Q dF(Q)

where F(Q) is the cumulative distribution for the density f ( Q ) . Next we add and subtract the quantity W*S~,.dF(Q) to the right side of the above equation, and combine some integrals. The result is E max (W*, Q.1) = W *

ioodF(Q) + f foo

= w* +

(Q - w*) dF(Q)

(Q - w * ) d F ( Q )

since the first integral yields the value 1. Substituting this result into Eq. (3) yields Eq. (2) as given in the optimal stopping theorem. What can we say about the properties of Eq. (2)? The integral in the equation has W* as a parameter. We may therefore view the integral as some function R with the argument W*: R(W*) =

I.

(Q - w * ) f ( Q ) dQ.

The function R has the following properties. 39'4° n ( w * ) --, E ( Q 0 as W* --, 0;

R(W*)~O

as W* ~ ~ ;

dR(W*)/d W* = - [ 1 - F(W*)]; and d 2 R ( W * ) / d W .2 = f ( W * ) >_ O.

Formal model of creative decision making • S. H. KIM ~)

E(Q)

' W*

Prospectivo quality IV'

Fig. 10. The return R as a function of the prospective quality W*. According to the optimal stopping rule, W* should be chosen so that R(W*) = c. This value of W* is the reservation level.

The return R is therefore nonnegative, strictly decreasing, and convex as depicted in Fig. 10. The optimal stopping rule calls for a prospective quality W* such that R ( W * ) = c. When the cost of search c > 0 is high, the reservation level W* is low; conversely, a decline in c results in a higher value of W*.

The interpretation for our application is as follows. The value W* is selected so that it just matches the marginal cost c. If the prospective quality W* were higher than the cost c, then the search should have continued; if W* were less than c, then the search should have stopped earlier. This is another illustration of the equality of marginal returns prevalent in economic theory: the optimal point is one where expected benefits and costs match precisely. An interesting point to note in this example is that global optimization results from a decision rule which is, in some sense, local. The designer decides whether or not to continue his quest simply by evaluating the quality of the current idea and comparing it against the cost of generating one more idea. So far we have been comparing the quality of an idea--an abstract concept--with the cost of generating it. In most, if not all, practical settings, the quality of ideas can be associated with economic value. At times this occurs implicitly; for instance, when a government decides to allocate so many million dollars to search for extraterrestrial life. In more down-to-earth settings, the valuation of ideas can often be accomplished explicitly. We now turn to this topic. VALUATION OF THE QUALITY OF IDEAS As discussed previously, the differential quality of alternative ideas results in a preference ordering, which in turn implies a preference metric of some kind. In general, it is difficult to associate some monetary value to the quality of an idea; for instance, what is the worth of the knowledge that the galaxies outside our local cluster are receding from us? For this reason, the suggestion has been made that an arbitrary quantity called the qual be the unit of quality for ideas in general. 29 This unit of measure is

63

similar to the util in utility theory, but is free of connotations of financial worth. In most practical settings, however, the valuation of ideas is less difficult. One of these is the product development context. A systematic approach to product development includes some analysis, however informal, of the cost of development, the potential market, and plausible financial returns. The financial returns will, of course, depend on a variety of factors such as the cost of capital, efficiency of production, and the elasticity of market demand in relation to the price of the product. One result of the analysis is an indication of the worth of the project to the company. The analysis is the derivation, at least in preliminary form, of the prospective price p and the production cost c. Any of these parameters--the worth of the product to the organization, or its production cost c, or its price p--gives some measure of the total worth of the ideas incorporated in the design. The ideas that underlie the design may then be assigned some worth as a fraction of the total design. These assessments may then be translated into dollar values based on the financial worth of the product. To illustrate, suppose that a new hydrogen-powered car is to be developed and sold at $10,000. Then the worth of the idea(s) behind the new engine will lie somewhere between $0 and $10,000. The precise value will depend on the relative worth of other components. For instance, the transmission is another important component in the car, but its worth may be considered to be only one tenth that of the engine. By proceeding in this way, the entire stock of components in the product, and consequently the worth of the ideas behind them, can be assigned a financial value. In fact, this is the standard procedure in the field of value analysis or value engineering. The valuation of ideas in financial terms therefore presents no major difficulties in most practical contexts. Consequently, the mathematical models presented in the previous section may be interpreted in realistic terms rather than as purely abstract concepts. Further, the association of monetary value with ideas paves the way for empirical investigation. We will return to this topic at the end of the next section. CONCLUSION AND FUTURE WORK Knowledge-based systems can be used in various ways to enhance both the effectiveness and efficiency of decision making. One way to enhance the effectiveness is to incorporate knowledge which compensates for human error. An area where people consistently perform poorly is in the task of making probabilistic judgments. In considering the tosses of a fair coin, for instance, people tend to view the pattern H-T-H-T-T-H to be more probable than either H-H-H-T-T-T (which appears nonrandom) or H-H-T-H-H-H (which seems to refute the assumption of a "fair" coin), z4'56 Since stochastic activities permeate problem solving under

64

Robotics & Computer-IntegratedManufacturing • Volume8, Number 1, 1991

imperfect information, a software module having knowledge of probabilistic reasoning could be a major factor in enhancing decision making. Another area of creative decision making lies in product design, which calls for the cooperation of multiple experts. The effectiveness of a design depends on the proper utilization of all relevant information. The information may relate to different facets of the design, such as the integration of mechanical, electronic, and software aspects. It also depends on the integration of post-design knowledge such as production processes; for instance, the use of a press fit to assemble two components could reduce concentrated stresses caused by the use of screws and thereby improve product longevity. In markets subject to increasing levels of turbulence, the pace of product development can also affect the viability of the product in the marketplace. A product that appears too late in the market might never win enough share to result in a profitable business. This limitation in current practice may be overcome by providing an integrated knowledge base incorporating the diverse islands of data, procedures, and heuristics in an organization. The marshalling of relevant information is hindered by different perceptions of a product: the mechanical engineer views it differently from a software engineer, who in turn regards it differently from the production staff, and so on. An integrated knowledge base must be able to serve each type of user according to his own special needs. In other words, the knowledge base must be general in the sense of serving as a unifying foundation for the information needs of diverse individuals in the organization. But it must also be tailorable to the requirements of specific users. A reasonable way to attain both objectives is to provide two layers: a foundation and an interface. The foundation contains the knowledge in a uniform structure, while the interface utilizes the foundation to provide each user with a specialized perspective. A current effort at our laboratory is the development of a general architecture to support concurrent design. The goal of the project is to develop a methodology and software prototype to accommodate concurrent engineering. A set of mathematical models is also being developed to formalize the approach and provide a theoretical evaluation. The theoretical analysis can be used to evaluate the prototype and to determine the implications for further analysis and development. The architecture incorporates the concept of desiffn filters, a set of programs for evaluating the manufacturability of designs. Candidate designs can be evaluated against such filters to provide suggestions for refinement during the subsequent iteration of the design. The architecture is being tested against particular case studies, such as the design and fabrication of printed circuit boards or of food products. The proto-

type is under development in close collaboration with potential end users in the industrial sector. Another direction for future work lies in modeling creativity as a drive for increased structure or order. The uncertainty of a system, as measured by information-theoretic metrics, tends to decrease over time as a system operates in fulfillment of its goals, s7 Conversely, this may be viewed as an increase in structure, a phenomenon generally observed in intelligent systems. In a similar way, we may view creative problem solving in terms of an agent that seeks to reduce the uncertainty between the current state and some desired future state. This modeling process may draw on previous work in information theory and its application to intelligent systems. 2'5'2s'29-31 Another extension for the future is the refinement of the quantitative models presented in this paper. For instance, what happens if the interarrival time between ideas is not exponential but takes some other distribution? In a similar way, what happens if the quality of an idea depends, at least in part, on the number and nature of other ideas generated in the past? And what are the implications of differing densities for distinct types of ideas--say, structural vs. procedural aspects of a design? The investigation of the basic model as well as its refinements may also be performed in conjunction with empirical studies. For example, psychologists may work in concert with commercial development teams as they proceed from initial conception to product development and marketing. Such studies would serve to enhance the preliminary models, calibrate model parameters, and motivate further extensions to the theory.

REFERENCES 1. Ashby, W. R.: Introduction to Cybernetics. London, Chapman and Hall, 1956. 2. Boettcher, K. L., Levis, A. H.: Modeling the interacting decisionmaker with bounded rationality. IEEE Trans. Systems, Man Cybernetics SMC-12(3): 334-344, May 1982. 3. Burks, A. W.: Essays on Cellular Automata. Urbana, University of Illinois, 1970. 4. Chandrasekaran, B., Shen, D. W. C.: On expediencyand convergence in variable-structure automata. IEEE Trans. Systems Science Cybernetics, SSC-4(I): 52-60, March 1968. 5. Conant, R. C.: Laws of information which govern systems. IEEE Trans. Systems, Man Cybernetics SMC6(4): 240-255, April 1976. 6. Davis, W. J., West, J.: An integrated approach to stochastic decision making: a project scheduling example. IEEE Trans. Systems, Man, Cybernetics 17(2): 199-209, 1987. 7. De Groot, M. H.: Optimal Statistical Decisions. New York, McGraw-Hill, pp. 347-352, 1970. 8. Drake, A. W.: Fundamentals of Applied Probability Theory. New York, McGraw-Hill, 1967. 9. Farley, A. M.: A probabilistic model for uncertain problem solving. IEEE Trans. Systems, Man Cybernetics 13(4): 568-579, 1983.

Formal model of creative decision making * S. H. KXM 10. Farmer, D., Toffoli, T., Wolfram, S.: Cellular Automata: Proceedings of an Interdisciplinary Workshop, Los Alarags, NM, March 1983. New York, North-Holland, 1984. 11. Ferguson, T. S.: Mathematical Statistics: A Decision Theoretic Approach. New York, Academic Press, 1967. 12. Fu, K. S.: Sequential Methods in Pattern Recognition and Machine Learning. New York, Academic Press, 1968. 13. Genesereth, M. R., Nilsson, N. J.: Logical Foundations of Artificial Intelligence, oh. 13. Los Altos, M. Kaufmann, 1987. 14. Ginzberg, A.: Algebraic Theory of Automata. New York, Academic Press, 1968. 15. Glorioso, R. M., Osario, F. C. C.: Engineering Intelligent Systems. Bedford, Digital Press, 1980. 16. Goldberg, D. E.: Computer-aided gas pipeline operation using genetic algorithms and rule learning. Ph.D. thesis, University of Michigan, 1983. 17. Guiasu, S.: Information Theory and Its Applications. New York, McGraw-Hill, 1977. 18. Hafez, W. A.: Autonomous planning under uncertainty: planning models. Prec. Int. Syrup. on Intelligent Control. Albany, New York, Sept. 1989, pp. 188-193. 19. Hagler, C., Kim, S. H.: Information and its effect on the performance of a robotic assembly process. Prec. Symp. on Intelligent and Integrated Manufacturing: Analysis and Synthesis. ASME Winter Annual Meeting, Boston, Dec. 1987, pp. 349-356. 20. Holland, J. H.: Adaptation in Natural and Artificial Systems. Ann Arbor, University of Michigan Press, 1975. 21. Holland, J. H.: Escaping brittleness: the possibilities of general-pupose learning algorithms applied to parallel rule-based systems. In Michalski, R. S. et al. (Eds), Vol. II, pp. 593-624. Los Altos, M. Kaufmann, 1986. 22. Holland, J. H., Reitman, J. S.: Cognitive systems based on adaptive algorithms. In Pattern-directed Inference Systems, Waterman, D. A., Hayes-Roth, F. (Eds), pp. 313-329. New York Academic Press, 1978. 23. Hoperoft, J. E., Ullman, J. [).: Introduction to Automata Theory, Languages, and Computation. Reading, Addison-Wesley, 1979. 24. Kahneman, D., Tversky, A.: Subjective probability: a judgement of representativeness. Cognitive Psych. 3: 430-454, 1972. 25. Kim, S. H.: Mathematical foundations of manufacturing science: theory and implications. Ph.D. thesis, M.I.T., May 1985. 26. Kim, S. H.: A mathematical framework for intelligent manufacturing systems. Proc. of Symposium on Integrated and Intelligent Manufacturing Systems, ASME, Anaheim, CA, Dec., 1986, pp. 1-8. 27. Kim, S. H.: An automata-theoretic framework for intelligent systems. Robotics Computer-lnteg. Mfg. 5(1): 43-51, 1988. 28. Kim, S. H.: Difficult problems and creative solutions. lnt. J. Computer Applications Technol. 2(3): 171-185, 1989. 29. Kim, S. H.: Essence of Creativity: A Guide to Tackling Difficult Problems. New York, Oxford University Press, 1990. 30. Kim, S. H.: Designing Intelligence: A Framework for Smart Systems. New York, Oxford University Press, 1990. 31. Kim, S. H.: Statistics and Decisions. Unpublished manuscript, M.I.T., Cambridge, MA, 1990. 32. Kim, S. H., Suh, N. P.: Mathematical foundations for manufacturing. J. Engineering Industry 109(3): 213-218, 1987. 33. Kohonen, T.: Self-Organization and Associative Memory. New York, Springer, 1984. 34. Kompass, E. J., Williams, T. J. (Eds): Learnino Systems

65

and Pattern Recognition. Barrington, Technical Publications, 1983. 35. Koomen, C.: The entropy of design: a study on the meaning of creativity. IEEE Trans. Systems, Man Cybernetics, 15(1): 16-30, Jan. 1985. 36. Lasdon, L. S., Optimization Theory for Large Systems. New York, Macmillan, 1970. 37. Lenat, D.: EURISKO: a program that learns new heuristics and design concepts: the nature of heuristics, III: program design and results. Artificial Intelligence 21(2): 61-98, 1983. 38. Lewis, H. R., Papadimitriou, C. H.: Elements of the Theory of Computation. Englewood Cliffs, PrenticeHall, 1981. 39. Lippman, S. A., McCall, J. J. (Eds): Studies in the Economics of Search. New York, North-Holland, 1979. 40. Malliaris, A. G., Brock, W. A.: Stochastic Methods in Economics and Finance, pp. 48-51. New York, NorthHolland, 1982. 41. Mendel, J. M., Fu, K. S. (Eds): Adaptive Learning "and Pattern Recognition Systems. New York, Academic Press, 1970. 42. Mesarovic, M. D., Takahara, Y.: General Systems Theory: Mathematical Foundations. New York Academic Press, 1975. 43. Minsky, M. L.: Computation: Finite and Infinite Machines. Englewood Cliffs, Prentice-Hall, 1967. 44. Mohammed, N. H., Courtney, F. Jr., Paradice, D. B.: A prototype DSS for structuring and diagnosing managerial problems. IEEE Trans. Systems, Man, Cybernetics 18(6): 899-907, 1988. 45. Narendra, K. S., Thathachar, M. A. L.: Learning automata--a survey. IEEE Trans. Systems, Man, Cybernetics 323-334, July 1974. 46. Parthasarathy, S., Kim, S. H.: Formal models of manufacturing systems. Technical Report, Laboratory for Manufacturing and Productivity, M.I.T., Cambridge, MA, 1987. 47. Pearl, J.: Probabilistic Reasoning in Intelligent Systems. San Mateo, M. Kaufmann, 1988. 48. Sanderson, A. C.: Parts entropy methods for robotic assembly system design. Prec. Conf. Robotics and Automation, Atlanta, GA, March 1984, pp. 600-608. 49. Shannon, C. E.: The mathematical theory of communication. Bell System Tech. J. 27: 379-423, 623-656, 1948. 50. Shannon, C. E., Weaver, W.: The Mathematical Theory of Communication. Urbana, University of Illinois, 1949. 51. Shapiro, I. J., Narendra, K. S.: Use of stochastic automata for parameter self-optimization with multimodal performance criteria. I EEE Trans. Systems Science Cybernetics, SSC-5(4): 352-360, Oct. 1969. 52. Simon, H. A.: Administrative Behavior. New York Macmillan, 1947. 53. Tokawa, T.: A knowledge-based approach to factory configuration. S. M. Thesis, M.I.T., Cambridge, MA, May 1989. 54. Tokawa, T., Kim, S. H.: Quality and design automation: effectiveness vs. completeness of search. Technical Report, Laboratory for Manufacturing and Productivity, M.I.T., Cambridge, MA, 1989. 55. Tsypkin, Y. Z.: Adaptation and Learning in Automatic Systems, trans, by Nikolic, Z. J. New York, Academic Press, 1971. 56. Tversky, A., Kahneman, D.: Judgment under uncertainty: heuristics and biases. Science, 185 (4152): 1124-1131, 1974. 57. Valavanis, K. P., Saridis, N.: Information-theoretic modelling of intelligent robotic systems. IEEE Trans. Systems, Man Cybernetics, 18(6): 852-872, Nov./Dec. 1988. 58. von Neumann, J.: Theory of Self-Reproducing Automata, Burks, A. W. (Ed.). Urbana, University of Illinois, 1966.