TÆMS agents: enabling dynamic distributed supply chain management

Electronic Commerce Research and Applications 2 (2003) 114–132 www.elsevier.com / locate / ecra

TÆMS agents: enabling dynamic distributed supply chain management Tom Wagner*, Valerie Guralnik, John Phelps Honeywell Laboratories, 3660 Technology Drive, MN65 -2600, Minneapolis, MN 55418, USA Received 30 October 2002; received in revised form 20 February 2003; accepted 28 February 2003

Abstract Some dynamic supply chain problems are instances of a class of distributed optimization problems that TÆMS and other intelligent agents were made to address. In this paper we define a discrete distributed dynamic supply chain management problem and specify how TÆMS agents, equipped with new coordination mechanisms, automate and manage the supply chain. The agents increase the level of flexibility in the chain and enable members of the supply chain to be more responsive through producer / consumer negotiation and reasoning about manufacturing availability, raw material requirements, and shipping time requirements. Planning / scheduling and coordination research enables the agents to perform this level of automation on-line, responding to change as it happens in the environment, rather than relying on precomputed solutions or reasoning via abstract flow characterizations.  2003 Elsevier B.V. All rights reserved. Keywords: Agent-mediated electronic commerce; Dynamic supply chain management; Coordination; TÆMS agents

1. Introduction In general, the intelligent agent paradigm lends itself to distributed supply chain management because members of a typical supply chain are loosely coupled interacting systems, i.e., raw material suppliers, shippers, manufacturers, can all be seen as agents. More strongly, agent technologies that perform resource and temporal coordination across heterogeneous agents, without assumptions of global knowledge, are directly applicable to certain classes of distributed supply chain management problems *Corresponding author. Tel.: 11-612-951-7313; fax: 11-612951-7438. E-mail address: [email protected] (T. Wagner).

because such technologies can perform decision making, order and material flow, and production scheduling to meet deadlines / resource limitations. If such technologies operate online, in real time, the agents can manage dynamic distributed supply chain problems in which frequent and timely adjustments to the flow of goods and production schedules are needed in order to leverage conditions in the marketplace or to take advantage of opportunities as they arise. Honeywell Laboratories, part of Honeywell International, which is one of the five largest supply chain management (SCM) providers 1 and is included in the Dow Jones Industrial Average, is examining

1

Per an independent market survey.

1567-4223 / 03 / $ – see front matter  2003 Elsevier B.V. All rights reserved. doi:10.1016 / S1567-4223(03)00014-0

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

this class of agent technologies for dynamic distributed supply chain management. The business motivation for this work is discussed in greater detail later. In this paper we present the use of TÆMS [7,13] agent technologies, equipped with new coordination mechanisms, on a sanitized version of a dynamic supply chain problem in the discrete manufacturing domain. In this application the agents work to manage the production schedule and material flow of a small build-to-order production line. The focus of this work is not on agent-based bidding schemes or price determination but is instead on reasoning about actual orders, production schedules, and material flow as discrete orders. This research also does not make strong assumptions about statistical characteristics of demand or product orders—the focus is not on setting up a steady-state manufacturing schedule. The example application used in this paper is that of manufacturing goods for the outdoor recreational market sector—specifically backpacks and sleeping bags. The actors in our example will include retailers and a fictitious manufacturer called ‘‘MountainMan’’. Any resemblance to actual retailers or manufacturers in this sector or other market sectors is purely coincidental. The example itself is based on a real world situation, though the work presented here is simulated and the problem specification should be viewed as an educated assessment of a real world situation. It is important to emphasize that the focus of this work is on the distributed optimization problem that occurs when agents have multiple interacting activities and the activities have individual deadlines and individual resource constraints—it is not on negotiation protocols or market mechanisms. Even if we could build a single centralized representation 2 of the task coordination space, the hybrid planning / scheduling (with interacting time limits, interacting resource constraints, and utility interactions) problem is intractable for any but toy problems. In this setting, provably optimal solutions generally require exhaustive search. Now consider what happens when 2

In the commercial setting centralized models are not encouraged because it would require releasing competitive data to a third party for planning and optimization.

115

the problem space is distributed and put into a dynamic setting. It is this space in which we operate. Complete discussion is beyond the scope of the paper—the point is to understand that a large class of supply chain management problems fall into this very difficult problem space that has more in common with distributed scheduling than it does with market mechanisms or research whose focus is on communication protocols. Note that such other technologies are relevant in this space, for example to determine the prices for goods which may interplay with the issue of temporal negotiation or to structure the negotiation protocols themselves, but that our focus is on the distributed scheduling aspects. In this paper we examine the production scheduling problem of a small volume manufacturing line that produces build-to-order goods for private label customers. The line is managed by an agent that interacts with other agents that represent retailers. The retailer agents, raw material supplier agents, etc., all potentially need to solve the same class of problem as the manufacturing agent—we focus primarily on the small volume production line for simplicity and clarity. As shown in Fig. 1 the small volume production line is an off-shoot from the primary production line of MountainMan. Generally with private label production a large order is produced via the primary production line and subsequent smaller orders, which are caused by unanticipated demand or customized variants of the goods, are routed to the smaller production line. The small volume production line supports small private label jobs, such as orders generated by unanticipated sales, that would interrupt the flow of the primary production line if scheduled for that line. This is particularly true for custom variants or restocking orders, in part because such orders are small but still require different fabric, fasteners, etc., but also because such orders generally have short term deadlines. In other words, when a retailer requests a small volume they generally want it in a few days rather than the period of weeks for which the primary production line is geared. This type of production has different requirements than the standard main stream production operations. While minimizing raw and finished goods inventories is important for all manufacturing processes, this is particularly true for private label goods as the customer base is very

116

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

Fig. 1. Primary and the build-to-order production lines.

limited (generally just one) and often portions of the raw materials are specific to the private label purchase. Additional requirements for the line include ending the day with the ‘‘tables empty’’ (no work-inprogress), maintaining zero inventory on finished goods, and building products to order only. In our implementation, agents are situated at each of the involved sites. For example, one agent is located at MountainMan and one agent is located at each retailer, shown in Fig. 6. The MountainMan agent’s job is to interact with the retailer agents and to determine production schedules for the MountainMan small volume production line accordingly. Note that the overall supply chain, shown in Fig. 2, involves raw material flows, shippers, and distribution centers. In this paper we focus on the problem of coordinating production to meet dynamic demand at the retailers. The general flow of events across the agents is that when inventory levels fall below the specified threshold at a retailer, the retailer’s agent places orders with MountainMan for replacement goods. The MountainMan agent reasons about the new order and how it relates to currently scheduled production in terms of temporal constraints and

overall value. It can then negotiate over delivery time with the retailer agent in order to optimize MountainMan’s mix of goods. The use of agents for this example is what enables the retailers and MountainMan to coordinate dynamically and automatically to optimize their activities.3 Fig. 3 shows a screen snapshot of the MountainMan production line and the JJBoom store front at which customers can purchase goods. The two task structures shown are of (center screen) the global task structure (or portions of it) for all agents as seen by the simulation environment, and (to the right) the task structure of the MountainMan agent. Note the 3

The issue of what criteria over which to optimize is deliberately unspecified. If we assume that MountainMan and the retailers have no direct relationship then MountainMan’s goal is to optimize over its own local criteria. In general this is simply to maximize profit but other secondary items might be to keep the line fully utilized or to use particular machines on a regular basis. For this example the exact optimization criterion is unimportant because all of these aforementioned issues can be mapped into the TÆMS quality attribute over which the MountainMan agent optimizes. Subsequent sections contain more information on TÆMS and TÆMS agents.

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

117

Fig. 2. MountainMan’s overall supply chain.

Fig. 3. Screen image of the production line, hypothetical global task structure, MountainMan’s task structure, and the JJBoom retailer.

118

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

difference between the two. The busy central representation is the problem that is being solved and optimized by the individual agents, each of which can only see and reason with a small portion of the total space. In Section 2 we provide a high-level view of TÆMS agents and TÆMS technologies. In Section 3 we return to the details of the application and illustrate the use of TÆMS agents for managing MountainMan’s dynamic supply chain problem. We then discuss chains of interactions and identify selected related work, discuss limitations, experimental plans, and future work.

2. TÆMS agents for dynamic supply chain management We use the expression TÆMS agents to describe our agent technology because the cornerstone of our approach is a modeling language called TÆMS (Task Analysis Environment Modeling and Simulation) [8,13]. TÆMS is a way to represent the activities of a problem solving agent—it is notable in that it explicitly represents alternative different ways to carry out tasks, it represents interactions between activities, it specifies resource use properties, and it quantifies all of these via discrete probability distributions in terms of quality, cost, and duration. The end result is a language for representing activities that is expressive and has proven useful for many different domains including the BIG information gathering agent [14], the Intelligent Home project (IHome) [12], the DARPA ANTS real-time agent sensor network for vehicle tracking [10,20], distributed hospital patient scheduling [7], agent diagnosis [9], and others like distributed collaborative design, process control, and agents for travel planning. Fig. 4 shows a TÆMS task structure like that used by the MountainMan agent in this application. The structure is a hierarchical decomposition of a top level goal which is simply to Produce. The top level goal, or task, has two subtasks which are to Make Bags and Make Back Packs. Each of these tasks are decomposed into subtasks and finally into primitive actions. Note that most of these are omitted from the figure for clarity. The details are shown for the Make BEI Jasper pack task—it consists of

Fig. 4. Sample TÆMS task structure for a manufacturing agent.

four primitive actions that are picking zippers and fasteners, cutting the webbing, cutting the fabric, and sewing the pack. The inter-dependence of these tasks is modeled in TÆMS using an enables non-localeffect. The dotted edges (enablements) from tasks like cutting and picking to the sewing tasks indicate that these tasks must be performed first and be performed successfully for the sewing task to be performed. Note that all of the primitive actions (leaf nodes) also have Q (quality), C (cost), and D (duration) discrete probability distributions associated with them. For simplicity in this paper we do not use uncertainty and all values will have a density of 100%. Picking the zippers thus takes 10 min (0.16 h) and generates a quality of one. Cutting the webbing has a similar allocation while cutting the fabric takes longer (0.5 h) and produces a higher quality (four). Sewing the packs takes over an hour and also produces a higher quality (four). The sum() function under the Make BEI Japser task is called a quality-accumulation-function or qaf. It describes how quality (akin to utility) generated at the leaf nodes relates to the performance of the

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

parent node. In this case we sum the resultant qualities of the subtasks—note that the cutting of the fabric and the sewing operations dominate how well the bags are made in this application. Quality is a deliberately abstract concept into which other attributes may be mapped. In this paper we will assume that quality is a function of the amount of profit generated by the production of a given good. In the sample task structure there is also an element of choice—this is a strong part of the TÆMS construct and important for dynamic supply chains. The Make Back Packs task, for example, has two subtasks joined under the sum() qaf. In this case the MountainMan agent may perform either subtask or it may perform both depending on what activities it has time for and their respective values. The explicit representation of choice—a choice that is quantified by those discrete probability distributions attached to the leaf nodes—is how TÆMS agents make context dependent decisions. In supply chain applications this is how the MountainMan agent sees that it has a choice of which products to produce and when. By establishing a domain independent language (TÆMS) for representing agent activity, we have been able to design and build a core set of agent construction components and reuse them on a variety of different applications (mentioned above, e.g. Refs. [7,10,12,14,20]). TÆMS agents are created by bundling our reusable technologies with a domain specific component, generally called a domain problem solver, that is responsible for knowing and encapsulating the details of a particular application domain. In the BIG information gathering agent, for instance, the domain problem solver is a blackboard planner that knows how to model software products, build models of products from raw text data, and compare / recommend products to purchase. In another application the domain problem solver may be a process plan or a legacy database application. In each of these cases we abstract away from the details of the domain by creating a custom mapping function that translates the internals of the domain problem solver into TÆMS task structures that are then operated on by the rest of the system. For this paper it is sufficient to know that TÆMS agents have components for scheduling and coordination that enable them to: (1) reason about what

119

they should be doing and when, (2) reason about the relative value of activities, (3) reason about temporal and resource constraints, and (4) reason about interactions between activities being carried out by different agents. A high-level view of a TÆMS agent is shown in Fig. 5; everything except for the domain problem solver is reusable code. Note that each module is a research topic in its own right. The agent scheduler is the Design-to-Criteria [16,21,23] scheduler and the coordination module is derived from GPGP [7]. Other modules, for example learning, can be added to this architecture in a similar plug and play fashion. In the supply chain application there are two types of domain problem solvers, those that manage the retailers’ inventories and the MountainMan agent that manages MountainMan’s production. The retailer problem solvers are of similar construction. Their function is to monitor purchasing activities and check inventory levels when purchases are made. If a good falls below a specified threshold, they reorder and negotiate with the MountainManagent to determine delivery times / dates. The MountainMandomain problem solver is different an instead reasons about MountainMan’s production. It creates new candidate runs for new orders as they come in and removes jobs from the list of candidates if orders are cancelled.

Fig. 5. A single TÆMS-based agent ready to coordinate its activities with other agents.

120

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

3. Dynamic supply chain example As mentioned in this example each retailer has a TÆMS agent that manages its local interests and orders products when appropriate. MountainMan also has an agent that interacts with the retailer agents, responds to order requests, negotiates delivery times, and manages MountainMan’s production.4 In this paper we focus on a subset of the supply problem and do not address interacting directly with shippers or raw material suppliers. Work involving chains of interactions is discussed later. The agent network is shown in Fig. 6. This example has specific properties, requirements, and assumptions

4

The actual computation about which items to produce and when is performed by the DTC [16,21,23] TÆMS agent scheduler.

that frame the problem. A subset of the more notable ones are: • Production is a single shift and scheduled from 8 a.m. to 4 p.m. However, the supply management agents operate both day and night and adjust production schedules as necessary. • All goods are shipped over night via an express carrier. • Raw materials are always sufficient to support production. • Orders may arrive at any time day or night. • Retailers order goods in lots. • No WIP (work in progress) is left on the tables at the end of the day. • Orders are not interruptible once they have begun. • The TÆMS quality associated with production

Fig. 6. Each company has its own agent that manages its local interests.

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

• •

• • •

tasks is a function of the margins produced by different products. Production activities will be modeled as primitive actions in TÆMS at the grain size of Make Product X.5 All customers are equally valuable. If this were not the case, it too could be mapped into TÆMS quality associated with the production tasks. When orders arrive they have a desired delivery date / deadline (that is specified by the retailer agents). Production specifics: sleeping bag lots require four production hours, backpacks require two hours per lot. All TÆMS distributions are 100% certain (single valued functions).

To illustrate, let the current simulated world time be 10 a.m. MountainMan’s TÆMS task structure, which describes MountainMan’s current production options and requirements at 10 a.m., is shown in Fig. 7. MountainMan’s current schedule is also shown in the figure. In the task structure MountainMan has 5

This is sufficient for scheduling and selection in this example. The focus is on the interaction across agents—the more detailed MountainMan job shop scheduling problem is addressable with the TÆMS technology and other well-defined techniques.

Fig. 7. MountainMan’s TÆMS task structure and associated schedule at 10 a.m.

121

two orders—one for JJBoom 3 Season sleeping bags and one for BEI Jasper backpacks. (A different TÆMS task is associated with each order.) The JJBoom lot has a higher expected quality because the margins are better on the JJBoom product than they are with the BEI Jasper. However, the JJBoom sleeping bags take longer to produce than the BEI Jasper backpacks. If the MountainMan agent were optimizing over the quality / duration ratio rather than maximizing quality, and the two orders were mutually exclusive, the agent would choose the BEI Jasper run over the JJBoom sleeping bag run. In this case as both the orders can be satisfied and the agent is maximizing total quality both production runs are scheduled and both orders are set to be filled. MountainMan is currently two hours into the JJBoom sleeping bag production run and is planning to produce BEI Jasper backpacks after the sleeping bag run (at 2 p.m.). At 12 p.m., when MountainMan is two hours into the JJBoom sleeping bag production run, a new order arrives. It is for the very high profit KMS Sequoia backpacks. The arrival of the new order causes the domain problem solver for the MountainMan agent to produce a new candidate production task, Make KMS Sequoia, associated with the order as shown in Fig. 8. The new order has a desired delivery date for the subsequent day. Because the production schedule for today is full, the MountainMan agent must either negotiate with the KMS retailer that placed the order or find some other way to produce the desired goods. Note that at this time (12 p.m.) the BEI Jasper packs are scheduled for production at 2 p.m. Thus the MountainMan agent actually has four possible choices: (1) it can reject the KMS Sequoia order because production is full for the current day, (2) it can reject the existing BEI Jasper order and do the KMS Sequoia run instead, (3) it can negotiate with the KMS retailer agent to obtain a delivery deadline that it can meet more easily, (4) it can negotiate with the BEI retailer agent to obtain a later delivery date for that order. In this case we assume that KMS orders are generally non-negotiable and the MountainMan agent considers rescheduling accordingly. Upon consideration the agent itself detects that the KMS

122

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

Fig. 8. MountainMan’s modified TÆMS task structure reflects the new order.

Sequoia order can be filled and that it should bump the BEI Jasper because the KMS Sequoia production run is more profitable. (This analysis is performed by the TÆMS Design-to-Criteria agent scheduler.) In making this determination the agent could reason about the cost of decommitment for the existing order and compare said cost to the higher value associated with the new order. In this example decommitment cost is not used. Given the higher value of the new order, the agent reschedules as shown in Fig. 9. When the agent reschedules it sends the BEI retailer agent a decommitment message that indicates the BEI order will not be filled as expected. Note that the JJBoom run that is currently in production proceeds uninterrupted. If runs were interruptible (they are not, see the assumptions above) the agent would consider aborting the current run and could even evaluate taking the run off the line at some cost (overhead) and putting it back on during a future time when the line was idle or constraints more relaxed.

Fig. 9. MountainMan’s production schedule modified to leverage the new order.

In response to the notice that MountainMan will not fulfill its order the BEI retailer agent examines its own local TÆMS task structure (not shown) and because there are no other orders that are competing for financial resources (shelf space could be considered here also) it re-issues its order with a later delivery date. The MountainMan agent reschedules again, as shown in Fig. 10, and decides to: (1) complete the JJBoom run (as it should given the requirements above), (2) then do the KMS Sequoia production run, (3) and then tomorrow (at 8 a.m.) to do the BEI Jasper run. This small example illustrates an important class of capabilities for dynamically managing a supply chain. First it shows autonomously making a quantified choice between candidate activities as the situation changes. On a given day there could be many such events and many such exchanges. Automating some or all of this process enables the aggregate system to optimize continuously to improve efficiency, lower costs, maximize profit, or whatever the objective criteria is appropriate. This example also illustrates how intelligent agent technology that incorporates temporal reasoning maps to supply chain problems where deadlines (delivery times) and other related constraints are present. The example also identifies many areas where application specific sophistication can be added. For instance, the agents could engage in a complex negotiation process to determine appropriate delivery times or could predetermine a price for decommitment (failing to fulfill an order after a guarantee has been given) that would be considered by the MountainMan agent before a decommitment action was taken. Note that the key properties of the supply chain problem space represented here is that control is distributed and the situation is dynamic as the orders are driven by actual consumer demand and not by estimates that are computed a priori. Other supply chain problems that map directly to this space include automatically changing production schedules

Fig. 10. MountainMan’s production schedule is revised again to support the KMS order and the revised BEI order.

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

to take advantage of spot market materials where the suppliers are represented by agents or shifting activities at both the manufacturer and the retailer to adapt to changes in shipping times or even a shipper going on strike. Another mapping is automatically modifying production to take advantage of changes in the marketplace as communicated by other agents, for example new customers, a change in product mix, a change in the product design itself, etc. In general, by adding dynamic control to the problem space the entire supply chain becomes more flexible and potentially more efficient. Note also that the use of agents at all of the involved parties is what produces the increase in flexibility—because the agents automatically negotiate over time, and potentially quality and costs, and because they communicate and convey information as it happens, they converge on an optimization across the network of interested parties. With respect to control of actual business processes, particularly when large dollar figures are involved, the agents can fill a support / advisory role and still leave the ultimate decision making capabilities with a trusted human.

4. Chains of enablement In the previous example we discussed a coordination episode between retailer agents and a manufacturing agent. Consider if the same problem is expanded to include raw and intermediate material

123

suppliers who ship to the manufacturer in response to the order placed by the retailer. In a conventional setting, most if not all of the suppliers will maintain an inventory of raw materials and will simply ship as needed from the inventory (scheduling production to generate more inventory as needed). However, in our problem space the raw material suppliers are running with a set of requirements akin to those of MountainMan in the previous example—everything is build to order (or the inventory levels are so small as to have the same effect). The implications of this are not obvious. If everything that is produced along an entire chain is custom (for only one possible customer) and no inventories are maintained, it means that if the retailer cancels the order while it is in the process of being filled, those who have produced goods for said order lose money. Fig. 11 sketches the situation. Each production activity only has value if the retailer completes its purchase of the finished good. For coordination the implications are pronounced. GPGP coordination is, in general, a peer-to-peer coordination technology that operates through pairwise dialogs. (The rationale for this is beyond the scope of the paper.) If a pairwise process is used to coordinate over any chain (not just those that have the very unusual property just described) it may take many iterations back and forth across the chain to resolve all of the temporal interactions and converge on a solution that works throughout the chain. If one adds the characteristic that each activity only has

Fig. 11. Chains of interactions where tasks only have value if the chain is completed.

124

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

value if the entire chain is performed from start to finish, the agent must conceptually wait until the entire chain has solidified (or converged) before beginning production. This ‘‘global’’ commitment is very different than a conventional GPGP style commitment—it has more in common with a commitment to a particular course of action regardless of the actual temporal bindings as described in Ref. [4]. To address the chains of interactions, where tasks along the supply chain only have value if the retailer purchases the product, we have created a new distributed coordination algorithm that uses commitment value and decommitment cost to achieve global coherence across the chain. The pseudo-code details of part of the algorithm are presented later (Fig. 16), however the conceptual framework is straightforward. Consider the task structure shown in Fig. 12. Agent A’s task A1 enables agent B’s task B1, which enables agent C’s task C1, which enables agent D’s task D1. Think of agent D as being a manufacturer who sells to end users and of agents A–C as being a chain of raw material suppliers. In order for agent D to perform D1, agent A must perform A1 and all the other intermediate tasks must also be executed by their respective agents. What is unique to the dynamic supply chain problem, particularly if zero inventories are maintained and the goods in question are private label or custom, is that if the sub-chain from A1 to C1 is performed but agent D decides not to perform D1, the goods produced along the sub-chain do not have any value to agent C and thus C will incur a loss. By the same token, if agent A decides not to perform A1, agents B and C will be unable to produce the goods desired by agent D and they may incur a decommitment penalty (D will also be unable to meet production plans / schedules). Recall that the focus of this work is on the temporal distributed coordination class of issues. Technologies like auc-

tions or market mechanisms for determining price can provide the quantitative values used during coordination. To enable the agents to commit to a course of action without being at undue risk for loss due to another agent not adhering to the course of action, coordination must conceptually commit all the agents to a given course of action at one moment in time. Along with this commitment must come some specified value for decommitting or breaking the pledge and some specified value for satisfying the commitment and producing goods as promised. Because the agents are distributed, obtaining the required commitment and agreement must be performed in a distributed fashion. The general approach is for each agent to reserve a spot in production for the desired good while the chain is being negotiated. For example, agent D would reserve a future spot for D1 in its production schedule but not begin work on D1 until the coordination session is complete (a brief period in terms of wall clock time). This step of the process is shown in Fig. 13. The reservation process is part of the activity carried out in the triangular box labeled DP D1—this is the first decision point for agent D. During this period, agent D decides when it needs to produce D1 to fill its needs and then reserves that slot in its production schedule. Associated with the reservation spot is a model of the value or utility that it will obtain for performing D1. This value is necessary— it enables agent D to reason about other opportunities that may arise while it is negotiating the formation of the chain of commitments with the other agents. After reserving the slot for D1, agent D then determines when it needs goods from agent C and communicates this need / order to agent C. In Fig. 14 the process continues. Agent C receives D’s request and determines when it can perform C1. If it cannot perform C1 to meet D’s needs, agent C

Fig. 12. Generic TÆMS chain of inter-dependent tasks.

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

125

Fig. 13. Coordination over the chain—step 1.

informs agent D of this and the two may then renegotiate or agent D can unreserve the slot allocated to D1 (this part of the protocol / reasoning process is not shown in the figure). Assuming agent C can schedule C1 to meet the needs of agent D, i.e., produce the desired good by the desired time, agent C will reserve a slot for C1 but not begin C1. It will then determine when it needs the inputs to C1, including the output of B1, and communicate these needs to agent B as an order with an associated deadline. Agent B then performs a similar process, communicates an order to agent A, and then the algorithm enters a new phase. At this point in time, the agents all have reserved times for their respective tasks and associated values with them. During the next phase, shown in Fig. 15, the agents send back commitment messages. For example, agent A informs B that it is committed to performing A1 and informs B of when the desired

goods will be available for use in B1. When the commitment message chain reaches agent D, it then back propagates a confirmation message and moves D1 from reserved status to performable status. The other agents follow suit upon receipt of their confirmation messages. This process is depicted algorithmically in Fig. 16. The algorithm is presented from the perspective of agent B and the top-level function, CommitmentRequestReceived(), is performed when agent B receives a commitment request from agent C to perform B1 (so that C can perform C1). Support functions appear toward the lower half of the figure.6 The code contains partial variable 6

The code presented implies a straight-line execution model. In practice, the agent does not block pending communications and the response handlers are called when appropriate messages arrive. This enables the agents to perform multiple negotiation sessions and to be responsive to change in the environment.

Fig. 14. Coordination over the chain—step 2.

126

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

Fig. 15. Coordination over the chain—all steps.

bindings, e.g., fromAgent C, to simplify disambiguation. The general flow is that when agent B receives a commitment request from C, it first checks to see if B1 is enabled by another non-local activity, A1 (denoting that B1 needs raw material from another agent in order to be performed). If so, agent B first attempts to schedule B1 to see if its constraints are feasible given B’s current schedule. Assuming that B1 is feasible, agent B will then attempt to obtain a commitment from agent A to obtain the necessary raw material. If / when it does so, it sends a corresponding commitment to agent C for the performance of B1. If agent B is unable to obtain a commitment from agent A, it must remove its temporary ‘‘what-if’’ scheduling of B1 and send a message to agent C indicating that a commitment will not be forthcoming. The algorithm illustrates the interplay of commitment offering and scheduling

feasibility at a granularity finer than that of the preceding text. The algorithm does not, however, contain code pertaining to the negotiation between agents over temporal constraints (e.g., delivery deadlines) or task values. The former is generally handled by having the constrained agent send the requester a no-commitment message with a suggestion of a time frame that would be acceptable. The requester then has to evaluate locally whether or not the new time frame is acceptable. If so, the algorithm presented in Fig. 16 can be restarted with different temporal bindings. Value-based negotiation is not currently implemented in our framework though it could follow a conventional propose–evaluate–counter model. For clarity, the algorithm abstracts another important detail. The statement scheduleTaems(B.whatIfTAEMS) encapsu-

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

127

Fig. 16. Partial sketch of the algorithm performed by agent B when a request is sent from agent C to agent B to obtain a commitment for task B1.

128

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

lates the process of: (1) determining if B1’s temporal constraints are feasible for agent B, and (2) determining if it is worthwhile to perform B1 at all. This latter point comes into play if agent B has another good whose temporal constraints conflict with B1, i.e., if both it and B1 cannot be produced. The point also comes into play if we consider opportunity cost in our computation. To better illustrate the role of opportunity cost and decommitment cost, consider what happens if, in the previous example, agent C receives a new opportunity and considers decommitting. Assume that the new opportunity is task C2 and that C2 is mutually exclusive with C1 due to temporal constraints. Assume that this opportunity arrives after C has committed to agent D to perform C1. In this situation, agent C must compare the value of performing C2 to the value of performing C1. Since agent C has committed to the chain including C1, it must consider several factors when assessing the value of C1, namely: (1) its quality or utility, which is tied to economic value, (2) its cost, and (3) the penalty for decommitting or decommitment cost. When assessing the value of C2 (before a commitment is offered), agent C can simply consider its quality and cost. In our application, the cost for decommitment is a function of the qualities and costs of the methods that come before C1 in the supply chain—in this case a function of methods A1 and B1. This is because agent C’s decommitment results in agents A and B producing goods that do not have market value outside of the context of C1. There is also a role for decommitment cost or a penalty being paid to agent D for D1’s opportunity cost—in this case it could be a function of D1’s quality minus its cost, i.e., some percentage of a model of agent D’s profit for D1. We are currently evaluating fair but more simple models of decommitment cost for this application. Tying all values to economics and using market mechanisms to set prices, or assuming that prices reflected the market place, would simplify the computation. The focus of this work is more on supporting decommitment cost in the control reasoning and local scheduling of the agents. While this example abstracts the process of determining decommitment cost it illustrates the concept. Hereto unmentioned is the opportunity cost of reserving a time slot for a candidate task while the

coordination chain is being negotiated. Currently, this is not considered by the agents—at issue is whether agent D should reserve a slot for some task, e.g. D9, if D9 is not very profitable given D’s task history or general margins. Reserving a slot for D9 means that while D is negotiating over the chain it cannot reason appropriately about new tasks that arrive (which may be of higher value) if said tasks have temporal constraints that interact with D9. This problem only occurs while a chain is being propagated. Once it is confirmed, D can reason about decommitting from D9 and respond to new opportunities. This has not been addressed because the coordination episode is generally regarded as taking on the order of seconds. However, if the coordination process involved human operators so that the time window could expand to days, the framework would have to incorporate the value of that reserved time slot and trade that off against new opportunities.

5. Abstract formalization Supply chain problems have many forms. In this paper, we discuss a specialized version appropriate for discrete supply chains that have particular characteristics, e.g. zero inventory, build to order, and those combined with production scheduling. From an abstract view, one formalization of the problem is as a 6-tuple (G, Q, S, D, C, V ), where • G is a set of agents, • Q is a set of tasks, • S is a mapping from tasks in Q to utility values, • D is a mapping from tasks in Q to deadlines, • C is a mapping from tasks in Q to agents that may perform those tasks in G, and • V is a partial ordering on Q. The problem is to attempt to choose and assign each task, t i [ Q, to an agent, a j [ G, such that the partial ordering specified in V, the task deadlines, d i [ D, and the assignment mapping, assign(t i ) [ C, are satisfied and the utility, u i [ S, associated with the associated tasks is maximized. This is an optimization problem (not a satisfaction problem) thus different tasks have different values and it is entirely

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

possible to have unsatisfiable temporal constraints in a given problem instance. Note also the role of utility—agents in this supply chain choose between candidate orders based on utility while considering temporal constraints. Hidden by this abstraction is the concept that utilities are dependent on the partial orderings V. In the previous section we discussed chains of enablement and the characteristic that in a build to order supply chain the involved parties need the transaction to happen start-to-finish in order to avoid loss in the middle of the chain.

6. Related work Huhns et al. [11] implemented several methods to automate the construction of agent-based supply chains by translating UML diagrams and Business Object Documents (BODs) into state machines that model the conversations necessary for supply chain management and that can be then wrapped in agents. (Fig. 7 in their paper shows their process well.) Their work differs in that they are not solving the problem of how agents decide which requests to fulfill in a collaborative setting beyond the protocol level. In our work there is a strong element of quantified and temporal decision making or choice that is lacking from theirs. Note, however, that some of their ideas could be used to frame the communication process of our work. The MASCOT [17] architecture for dynamic supply chain coordination uses blackboard agents to create and manage supply chains. MASCOT differs from the work here in its support of mixed-initiative functionalities that enable different human users at different levels in a supply problem to manipulate planning / scheduling activities. MASCOT also differs in that it generates planning / scheduling options through a bid mechanism, i.e., considering different suppliers. In our work, constraints are resolved by (aka ‘‘options are generated through’’) coordination, scheduling, and negotiation. While our work focuses on temporal interactions between activities that span agents, that aspect of MASCOT is unclear, though the blackboard agents include a sophisticated scheduling module. Leveled Commitment Contracting [18] is a commitment instrument for capitalizing on the possi-

129

bilities provided by probabilistically known future events. Instead of conditioning a contract on the occurrence of future events, as is the case with contingency contracting, a decommitment penalty is built into the contract that allows unilateral decommitting. We implement a form of implicit leveled commitment contracting—but decommitment cost is precisely the quality gain on a commitment request, ex ante, and the quality posted on a satisfied commitment, ex post. More relevant, however, is that though the gain for commitment satisfaction is monotonically increasing, we still need a synchronization procedure to prevent thrashing across the ordering chain via end-to-end decision time guarantees. Shen et al. [19] detail a general, domain independent, collaborative agent system architecture which incorporates standard agent services such as ontology, yellow pages, and centralized local coordination managers as well as the the notion of a dynamic cooperation domain abstraction for groups of cooperating agents. While this work identifies the importance of cooperation, it does not describe or implement quantified choice / coordination technologies. Zeng and Sycara [24] define a model that can be evaluated to identify efficient combinations of supply-chain activities. The model consists of and / or task decomposition trees. Parts of a supply chain are represented by these nodes. These models are translated into problems for which decisions can be made following inventory theory models. Said models do not appear to consider commit / decommit problems, or the explicit modeling of one task’s properties versus another. It is thus unclear how flexible the system is—certainly it does not leverage quantified choice or selection. Barbuceanu [1] gives a representation for tasks and constraints on the execution of tasks (behaviors) called a goal network. Obligations and interdictions in his framework roughly correspond to commitments (part of the TÆMS coordination process) or the facilitates / hinders interactions in TÆMS (not shown in Section 2 but related to TÆMS enablement). One agent’s authority over another is required to set an obligation. The author also describes a way of reasoning about the representation, which is branch and bound, to find the right commitment for goals which optimize the utility of the task network.

130

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

Collins et al. [5] describe a MultiAgent NEgotion Testbed (MAGNET) which implements collaboration via an auction model. Agents which require services request them via a task network that includes task descriptions and time constraints. Provider agents then send back bids with the tasks that they are willing to undertake, when they can do them, and at what price. A bid manager fills in a requesting agent’s task structure with an appropriate schedule from the bids by using either an integer programming or a simulated annealing evaluator. Their framework is not as rich as TÆMS, and they are not dealing with commit / decommit issues though they are considering temporal constraints and a choice mechanism—features we consider important. Davidsson and Wernstedt [6] model just-in-time distribution networks and evaluate cooperative MAS for management of the networks. In this work the focus is on high-level formal modeling of the problem space and the implementation of the models in a simulation environment. The work does not focus on the discrete, temporally situated, and detailed dynamic supply chain and production scheduling problem presented in this paper. In general our work differs in the combination of factors it addresses. These tie back to TÆMS’ rich feature set, for instance: complex process representation, explicit quantified modeling of task and process interactions, explicit quantified existence of alternative ways to perform tasks and quantified choice functions, combined with various hard and soft temporal and resource constraints. Coupled with this is the explicit effort of all of the TÆMS technologies to support dynamic adaptation to situations as they evolve. Thus both the agent scheduling and coordination technologies are designed not to rely on a priori or off-line computations and are designed specifically to always evaluate options from a qualified perspective.

7. Limitations, experimental plans, and future work This paper describes TÆMS agent technology and shows its use on an implemented supply chain management problem. The technology used here currently has a few limitations—some of which are

being addressed and some of which are larger issues. One limitation is that we do not coordinate over resources. The chains of interactions described above and easily envisioned by raw-to-manufacturer-to-retailer chains are not actually chains from task-to-task but are chains from task-to-resource-to-task. In other words, MountainMan produces a good that is consumed by the retailer. If we modeled and coordinated over that good, rather than using task interactions, the system would automatically handle situations in which MountainMan had a requested good in inventory or lacked a raw material needed for production. Currently this functionality is partly implemented in the domain problem solvers of the retailer agents and the MountainMan domain problem solver could be extended to provide this functionality as well, e.g. if goods in inventory, ship and charge (do not make new production task). Resource coordination of a different nature has been done before in TÆMS [10,12] and the explicit representation of the resources potentially introduces an additional level of flexibility and simplifies the construction of the domain problem solvers. The larger issue that may not be obvious is that when control is decentralized in this fashion, and the problem decomposition itself is not structured but instead evolves, this type of distributed optimization is not always guaranteed to be optimal. When can it fail? When the problem spaces get large it is occasionally difficult for the Design-to-Criteria agent scheduler to produce an optimal solution. (The general case of the problem it solves is not computable—it uses approximation techniques to make the space tractable and operate on-line in soft real time.) Another case in which it may not be optimal is when the constraints fall in a particular way—the partial and distributed views held by the agents may not always contain enough information for them to fully optimize. The generalization of this coordinated decision problem has been shown to be exponential (NEXP–complete) [2,15,3] which is why in practice implementations that operate in real time do not guarantee optimality. With this potential for suboptimality mentioned, it is also important to recognize that most human-centered business processes today are far from optimal. In this work we are seeking to improve efficiency and reduce costs— making gains over the approaches currently used.

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

In most research there is always room for more rigorous empirical verification and this work is no exception. Depending on the level and thrust of corporate interest, we may develop an optimal, exhaustive, centralized controller and compare the new protocol for chains of enablement against the centralized oracle. Note that the purpose of this comparison is not to motivate a distributed approach—the need for distribution is inherent in the privacy and autonomy issues—but to present a yardstick against which the approximate TÆMS agent solution can be compared. On an unrelated DARPA team coordination project, GPGP-derived coordination technologies have compared favorably to an optimal centralized oracle [22] though the results depend on the general character of the problem sets and the tightness of the constraints.

Acknowledgements We would like to acknowledge Professor Victor Lesser of the University of Massachusetts for his collaboration in bringing portions of the TÆMS technologies to Honeywell Laboratories. We would also like to thank the other researchers who have contributed to TÆMS.

References [1] M. Barbucceanu, A negotiation shell, in: Proceedings of the 3rd International Conference on Autonomous Agents, 1999, pp. 348–349. [2] D.S. Bernstein, S. Zilberstein, N. Immerman, The complexity of decentralized control of MDPs, in: The Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI), 2000, pp. 32–37. [3] C. Boutilier, Sequential optimality and coordination in multiagent systems, in: IJCAI, 1999, pp. 478–485. [4] C. Castelfranchi, Commitments: from individual intentions to groups and organizations, in: Proceedings of the First International Conference on Multi-Agent Systems (ICMAS95), 1995, pp. 41–48. [5] J. Collins, M. Gini, Exploring decision processes in multiagent automated contracting, in: Proceedings of the 5th International Conference on Autonomous Agents, 2001, pp. 81–82. [6] P. Davidsson, F. Wernstedt, Characterization and evaluation of just-in-time production and distribution, in: Working Notes of the AAMAS 2002 Workshop on MAS Applications

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

131

and Problem Spaces, 2001, to appear in an edited collection in 2003. K. Decker, J. Li, Coordinated hospital patient scheduling, in: Proceedings of the Third International Conference on MultiAgent Systems (ICMAS98), 1998, pp. 104–111. K.S. Decker, Environment centered analysis and design of coordination mechanisms, PhD thesis, University of Massachusetts, 1995. B. Horling, B. Benyo, V. Lesser, Using self-diagnosis to adapt organizational structures, in: Proceedings of the Fifth International Conference on Autonomous Agents (Agents2001), 2001. B. Horling, R. Vincent, R. Mailler, J. Shen, R. Becker, K. Rawlins, V. Lesser, Distributed sensor network for real-time tracking, in: Proceedings of Autonomous Agent 2001, 2001. M.N. Huhns, L.M. Stephens, N. Ivezic, Automating supply chain management, in: Proceedings of the 6th International Conference on Autonomous Agents, 2002, to appear. V. Lesser, M. Atighetchi, B. Horling, B. Benyo, A. Raja, R. Vincent, T. Wagner, P. Xuan, S.X.Q. Zhang, A multi-agent system for intelligent environment control, in: Proceedings of the Third International Conference on Autonomous Agents (Agents99), 1999. V. Lesser, B. Horling et al., The TÆMS whitepaper / evolving specification, http: / / mas.cs.umass.edu / research / taems / white. V. Lesser, B. Horling, F. Klassner, A. Raja, T. Wagner, S.X.Q. Zhang, BIG: an agent for resource-bounded information gathering and decision making, Artificial Intelligence 118 (1 / 2) (2000) 197–244. D. Pynadath, M. Tambe, Multiagent teamwork: analyzing the optimality and complexity of key theories and models, in: 1st International Conference on Autonomous Agents and MultiAgent Systems, 2002, pp. 873–880. A. Raja, V. Lesser, T. Wagner, Toward robust agent control in open environments, in: Proceedings of the Fourth International Conference on Autonomous Agents (Agents2000), 2000. N.M. Sadeh, D.W. Hildum, D. Kjenstad, A. Tseng, MASCOT: an agent based architecture for dynamic supply chain creation and coordination in the Internet economy, Production Planning and Control 12 (3) (2001). T. Sandholm, V. Lesser, Leveled commitment contracting: a backtracking instrument for multiagent systems, in: AI Magazine, AAAI Press, 2002, pp. 89–100. W. Shen, M. Ulieru, D.H. Norrie, R. Kremer, Implementing the Internet enabled supply chain through a collaborative agent system, in: Proceedings of the 1999 Autonomous Agents Workshop on Agent Based Decision Support for Managing the Internet Enabled Supply Chain, 1999. R. Vincent, B. Horling, V. Lesser, T. Wagner, Implementing soft real-time agent control, in: Proceedings of Autonomous Agents (Agents-2001), 2001. T. Wagner, A. Garvey, V. Lesser, Criteria-directed heuristic task scheduling, International Journal of Approximate Reasoning, Special Issue on Scheduling 19 (1 / 2) (1998) 91–118, a version also available as UMASS CS TR-97-59.

132

T. Wagner et al. / Electronic Commerce Research and Applications 2 (2003) 114–132

[22] T. Wagner, V. Guralnik, J. Phelps, A key-based coordination algorithm for dynamic readiness and repair service coordination (submitted for publication). [23] T. Wagner, V. Lesser, Design-to-criteria scheduling: real-time agent control, in: T. Wagner, Rana (Eds.), Infrastructure for Agents, Multi-Agent Systems, and Scalable Multi-Agent Systems, LNCS. Springer, 2001. Also appears in the 2000 AAAI Spring Symposium on Real-Time Systems and a

version is available as University of Massachusetts Computer Science Technical Report TR-99-58. [24] D.D. Zeng, K. Sycara, Agent facilitated real-time flexible supply chain structuring, in: Proceedings of the 1999 Autonomous Agents Workshop on Agent Based Decision Support for Managing the Internet Enabled Supply Chain, 1999, pp. 21–28.