- Email: [email protected]
A multi-agent supply network control framework
ARTICLE IN PRESS
Control Engineering Practice 15 (2007) 1394–1402 www.elsevier.com/locate/conengprac
A multi-agent supply network control framework Bart Saint Germain, Paul Valckenaers, Paul Verstraete, Hadeli, Hendrik Van Brussel K.U.Leuven-PMA, Celestijnenlaan 300B, B-3001 Heverlee, Belgium Received 11 March 2005; accepted 11 December 2006 Available online 9 February 2007
Abstract This paper presents a bottom-up solution for the supply network coordination problem. This manuscript treats the coordination as an ongoing task and takes disturbances and exceptions as business as usual. A manufacturing control and coordination mechanism based on PROSA and ants [Hadeli et al. (2003). Self-organising in multi-agent coordination and control using stigmergy. ESOA] is used as base for the overall coordination of the supply network. Reputation serves to enforce the social behaviour among the production entities. The interactions of the control mechanism with the reputation models are given. r 2007 Elsevier Ltd. All rights reserved. Keywords: Supply networks; Multi-agent; Trust; Reputation; Ant coordination
1. Introduction
Today, manufacturing industries are aware that designing the right kind of flexibility into their forthcoming production systems is vital to their future competitiveness. Companies have tried to optimize their own internal processes over a long period of time. Nowadays companies do not operate anymore in an isolated world. To achieve further optimization in their production system, industrial units have to look beyond the ‘‘walls’’ of their company (Sauter & Parunak, 1999). Supply network coordination is an essential requirement in current industries. Coordination is managing dependencies between activities (Malone & Crowson, 1994). Shared resources, producer/consumer relationships, simultaneous constraints, and tasks/subtasks can be identified as dependencies between activities in the supply network. Coordination, between entities of the supply network while offering and using services, exists on different levels (Rice & Ronchi, 2002):
markets: non-repetitive transactions (e.g.: e-commerce); alliances: long term contracts, repetitive transactions; Corresponding author. Tel.: +32 016322954; fax: +32 016322987.
E-mail address: [email protected] (B. Saint Germain). 0967-0661/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.conengprac.2006.12.003
partnerships: run the business with other part owners (partners); and vertical integration: running the business as owner (organization).
This paper discusses a multi-agent coordination and control approach to operate the external and internal logistics in multi-site/multi-organization topologies. The coordination discussed in this paper applies to long term relations and happens in a repetitive way. A fully centralized control is not applicable for this kind of coordination. Only long term relations are considered. As a consequence, the control system is not exposed to open ‘‘violent’’ environment (markets). This does not mean that all partners can be trusted in all situations. Exceptional behaviour should either be foreseen or captured and taken into account for later interactions. Past experience reveals that manufacturing control cannot be implemented through ‘one shot scheduling solutions’. On the shop floor, schedules have acquired a reputation for rapidly becoming invalid because of the frequent changes and disturbances. This effect becomes even bigger in supply networks (Lee, Padmanabhan, & Whang, 1997) and requires until now some ‘‘brute-force’’ solutions (i.e. huge buffers, low product-type variability). Consequently, the multi-site manufacturing control
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
presented in this paper, treats changes and disturbances as business-as-usual. It considers manufacturing as a nonstop ‘going concern’. This paper first defines the vision on the problem, supply network control. A bottom-up design is taken as a starting point. Next, the paper describes how a supply network can be controlled starting from factory control and coordination mechanism (Hadeli et al., 2003). The supply network description will show that social behaviour of all entities involved is required. In supply networks this assumption is not fulfilled and requires some extra mechanisms. In the final section, a trust framework is described answering this requirements. 2. Supply network control definition This section presents a definition of supply network control. Control and especially supply network control can be understood in many ways. Following a bottom-up approach, supply network control is defined by the nature of a supply network and the problems it brings about. The paper uses the insight that a production entity cannot be regarded as a black box. Instead a production entity will be defined as an autonomous and specific unit in the overall supply network. Supply networks are specified as multilevel networks of entities (Lejeune & Yakova, 2005). Supply network control will be defined as a non-stop going concern with as main concern the productivity and responsiveness of the total supply chain. This global productivity has to be understood with respect to the social nature of the problem.
1395
a single owner, and therefore the network control cannot impose a single goal and/or strategy. The entities will have varying degrees of autonomy, which the network control has to account for. 2.1.2. Specificity Specificity reflects the uniqueness of an entity. With respect of the technology and strategic decisions enabling to provide services, it is very difficult to model the different entities in general. Demand variation, finite capacity, disturbances and policy changing in the technological process makes forecasting and planning of production a unfeasible/inaccurate task without having a deeper insight into the entity. Therefore, an entity cannot be adequately modelled by a black box comprising a processing unit with input and output buffers. This is a novel approach that contrasts with more economic and control theoretical approaches where a lot of assumptions are made towards the logic inside the entities: fixed lead-times, no disturbances, etc. This observation—entities are specific—builds on the research team’s experience with manufacturing control. When a control system operates in a competitive environment, overconstraining its options by oversimplifying the world model is likely to be detrimental for system performance. And, unlike in administrative systems, it is not economically feasible to reengineer the production systems to fit the model of the control system. Hence, production entities cannot be reflected in the control system by a single generic model. 2.2. Supply network definition
2.1. Production entity definition Production entities are units in a production environment that deliver services. Services as they are carried out by entities have to be understood in the broad sense. Transport services, producing components and repairing machines are only a few examples of possible services. Services are described by means of their capabilities and constraints. Truck transport services are described by the transport capability and by constraints like maximum height/width and weight of the pieces that trucks can carry. Entities have typically two properties that characterize themselves: autonomy and specificity (Sauter & Parunak, 1999), explained in the following paragraphs.
A supply network is defined as a multi-level network of production entities collectively responsible for procurement, manufacturing and distribution activities associated with one or more families of related products. Entities can be aggregated or may cooperate with each other. (1) Cooperation expresses the ability to use services of the connected entity. (2) Aggregation models the organizational hierarchy at one point in time from the supply network. Organizations are divided in several plants, plants comprise a couple of divisions, etc. Both aggregations and cooperations connections can be subject to changes. New partners are coming into the supply network, organizations restructure, outsourcing, etc. 2.3. Supply network control definition
2.1.1. Autonomy The autonomy stands for the ability to decide how, when and where to deliver services together with which services to provide to their environment. These local decisions are not always without consequences. In particular, entities should behave in a socially acceptable manner. This first observation above emerges out of the essential difference between factory control and the supply network control. Production entities in a supply network do not all belong to
Supply network control copes with the logistics in a supply network as defined in the previous section. Supply network control handles the external factory logistics while interacting with internal factory logistics (Lejeune & Yakova, 2005; Sauter & Parunak, 1999). Logistics management includes not only decision taking but also and not less important, information handling, monitoring and forecasting (Valckenaers & Van Brussel, 2005b).
ARTICLE IN PRESS 1396
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
Controlling supply networks comprises a variety of objectives. The first objective is to cope with the going concern in supply chain control. Treating the control system as a static solution will fail in the dynamic and uncertain environment of the supply network. Disturbances on production level are propagated in both horizontally and vertically in the supply network, and the fluctuation on the uncertainty becomes even bigger (Lee et al., 1997). The second and probably the most important objective of the control system is the productivity over the overall supply network. There should be a tendency to pareto-optimal solutions where win–win situations—out of the pareto-optimal set—over different entities are preferred when taking a local decision. Internal versus external logistic interactions happen with respect for the autonomy and specificity of each of the entities belonging to the supply network. Specificity makes central coordination and control very hard and not desirable due to the crucial knowledge inside the entity. This knowledge is very hard to transfer in a general way and has to be inevitably handled by the entity itself. 2.4. Discussion In this paper a definition for supply chain control is given starting from a bottom-up design principle. Entities from supply networks differ from resources in a factory in two ways. Entities are autonomous and specific. These differences and the dynamics of the problem make central approaches and approaches where assumptions on internal entity logic are made hard to maintain. Supply networks are not defined as a single graph but as a multi-level network that represents the structure of the entities at one point in time. Supply network control is defined as a going concern (robust against changes and disturbances) with as main target maximizing the productivity and responsiveness of the total supply network. 3. Supply network coordination and control The supply network control application is based on a already developed multi-agent factory control framework (Valckenaers & Van Brussel, 2005b). This framework is build on the PROSA architecture and is inspired by ant colony behaviour. Instead of an one shot planning system like MRP, MRPII (Verstraete, Valckenaers, Germain, Van Brussel, & Hadeli, 2006), this framework reacts on changes and disturbances in its environment. Besides the flexibility and the robustness planning, the control system is build in a modular way which enables the easy transition from one application towards another. PROSA was developed for manufacturing control applications; nonetheless this section shows how this architecture can be adapted towards a supply network architecture. Next, insights on ants which can help to coordinate the supply network are described.
3.1. Supply network architecture The reference architecture described in this paper is based on the holonic paradigm developed by Koestler (1989). Koestler introduced the word holon as a composition of the word ‘‘holos’’ ð¼ wholeÞ an the post-fix-on in proton ð¼ partÞ. The two main properties of a holon are autonomy and cooperation. The autonomy provides stability and robustness in the changing world. The cooperation on the other hand gives the holon a place in the bigger system. In the framework the holarchy is implemented by an agent community. Each holon is represented as an agent and manages the organizational structure of the holons. 3.1.1. PROSA Starting from the holonic concept a reference architecture was developed for production systems, called PROSA (Wyns, 1999). Manufacturing control, as a conceptual design, is divided in a number of sub-systems, cooperating with each other, and across the different types of holons, a holarchy is formed. The structure of this architecture is built around three basic holons: product, order and resource holons.
Product holons provide the process knowledge of a product (recipe). This means that a product holon can identify all possible subsequent production steps for an order in a certain state. The implementation and representation of such a recipe is internal to the product holon. The order holon manages the physical product instance being produced, the product state model, and all logistic information processing related to the job. It is responsible for performing the assigned work correctly and on time. Each resource holon corresponds to a physical part in the manufacturing system. A resource holon corresponding to a machine provides the knowledge of the physical machine or part; it knows the supported operations, the production capacity, queuing strategy, etc.
A main advantage of this architecture is that it reflects reality (physical things, order, machines, product types) and no functions (e.g.: average lead time). No unstable constraints are introduced until this point e.g.: no ‘‘bill of materials’’ is used (Valckenaers & Van Brussel, 2005a). 3.1.2. Supply network architecture extensions As Koestler (1989) stated, systems can consist out of several smaller entities and some systems belong to bigger aggregated systems without having an upfront-defined hierarchy. As described in the previous paragraph, PROSA is a holarchy over the different holon types. The extension towards supply network control will employ the same mechanism and a holarchy will formed inside each PROSA
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
holon type. The following overview indicates what this extension means for every separate PROSA holon. Product holons are not longer stand-alone sub-systems. Whenever you take a snapshot at a point in time, the structure of different product holons will form a hierarchy. E.g. suppose there is a product holon that corresponds to a specific car model. Such a holon provides the knowledge to manufacture this car type to the other holons in the control and coordination mechanism. As in real life, this product holon is not able to cope with all complexity. In fact, the product holon for that car type consists out of several subholons each of them describing their respective subsystem, the wheels, the engine, the seats, etc. Holons can delegate tasks to their respective sub-holons while taking the responsibility to coordinate them, informing about changes, aligning, etc. A resource holon can consist out of smaller resource holons each of them responsible for a part of the total, initial problem. In this way the supply network is modelled reflecting the reality (see Section 2). All machines, operators, raw materials, etc. are composed into a production facility. By composing production facilities and logistics between the facilities, organizations are modelled. Aggregations, like an ‘‘organization resource holon’’, serve as an other level of abstraction. Service users do not necessary have to cope with the sub-holons. Order holons can separate their concerns and delegate sub-problems to smaller order holons, responsible for that sub-problem. For instance, an order holon, responsible for a large transportation task, may spawn several new order holons responsible for smaller and more manageable tasks (like a single transport segment covered by a truck), which the original order holon coordinates. 3.2. Supply network coordination This section gives a short overview on how a coordination and control mechanism is implemented on top of the PROSA architecture. The control system described uses the insights of ant coordination techniques. These insights rise some interesting properties in manufacturing coordination and control. First a brief overview is given about the approach used in manufacturing control (more information about the used approach can be found in Valckenaers & Van Brussel, 2005a). Next some extensions towards supply network control are discussed and links to the trust and reputation framework are indicated. 3.2.1. Ant coordination and control Ant coordination and control is based on the insights from the behaviour of ants searching for food. The main advantage of such an ant-based design is that individual agents/holons are not exposed to the complexity and dynamics of the global situation. None of the agents needs a mental map of the environment nor do these agents communicate amongst themselves about the environment.
1397
Also, the communication is carried out in a very robust way by means of an evaporate-and-refresh mechanism. Pheromones are put by the ants which evaporate after some time. In order to keep the information available, ants need to refresh the pheromones. There are however two requirements for the ant coordination mechanism to be effective: (1) there must be a large number of ants in the system, and (2) it must be cheap and easy to lose some of the ants. In practice, this means that the ants in the network control system must be virtual entities, and cannot be strongly connected to the presence and location of physical entities in the supply network. E.g. an ant cannot represent a car as it is not cheap to loose an ant and it is not affordable to require many ants to get to a solution. Instead of representing a ‘‘car’’, the ants in the framework represent either the exploration and the intention tasks of the order holon. Ants representing the order holon behaviours fulfill the two requirements of the ant coordination mechanism. In the following sections, the exploration and intention behaviour is explained in more detail. 3.2.2. Exploring The exploration services investigate possible routes and processing sequences for the product instances in a factory. The basic exploration service implement the following scheme: (1) Each order agent creates exploring ants at a regular frequency. (2) Each exploring ant virtually executes a possible scenario on behalf of its order agent. A scenario is more than only a set of resources to be visited. Possibilities to join, make batches, execution slots for the different operations, etc. are part of the total solution space. (3) At the end, the explorer ant will report the explored solution to the order agent. Based on the results of the exploring ants, the order agent keeps a set of candidate routes and is refreshed regularly. If the information gets out dated, it evaporates. 3.2.3. Intention propagation An intention reflects to some extent the future behaviour of a holon. The intention emerges out of the set of possible, explored, candidate solutions. One of the possible candidate solutions is regarded as the best and will be executed, with high probability, in the near future. Next the holon triggers an intention ant responsible for informing the different service providers about the intention of its holon. This intention is posted on the resources in the form of pheromones, it will evaporate if the intention is not refreshed regularly. Based on these intention service providers can build a dynamic load forecast, which can be used by explorer ants to make their performance estimate reliable.
ARTICLE IN PRESS 1398
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
3.2.4. Confidentiality/filtering of information The main difference for the ant topology with factory control lies in the holarchical structure of the supply network. Some resource holons represent a set of production entities. These aggregated holons have not only horizontal connections, e.g. suppliers and clients, but also vertical connections to their sub-holons, e.g. divisions. Ants cannot always access the vertical connections due to two reasons: (1) some information is confidential for the outside world and (2) it is not always desirable to have all information, it would only trigger information overload. The aggregated holon tackles this problem by hiding the connections and encapsulating the needed information from the underlying production facilities.
tions as much as possible. Although a situation is not immediately in line with their own goal, entities should tend to an improvement of the global supply network control performance. This behaviour should happen in a distributed environment. A centralized control is not desired or not applicable (see autonomous and specific). Trust and reputation systems claim to serve as key elements in making the network behave in a social way. Any unsocial behaviour will be punished in terms of a reputation drawback. Reputation as it is, builds upon and affects entity behaviours by means of all communication channels and interactions.
3.3. Discussion
In this section trust and related concepts are defined. The definitions are meant to give a clear view on trust and reputation. Next to the definition of trust and reputation, confidence is explained. The framework, described later, operates based on the confidence of particular explored solutions. Trust: Trust is a generalized expectancy held by an individual or group that the word, promise, verbal, or written statement of another individual or group can be relied on Carter, Bitting, and Ghorbani (2002) and Devinney and Pillutla (1998). In the supply network control system, order agents rely on resource and product agents while sending intention ants and explorer ants. At the same time, resource agents rely on the accuracy of the intentions send by the order agents. Confidence: Confidence can be described as the act of confiding, trusting, or putting faith in. The main difference between trust and confidence is that trust involves choosing alternatives whilst confidence does not (Luhmann, 2000). Confidence can be described as habitual expectation, e.g. ‘‘I am habitually confident that my milkman will deliver milk to the doorstep tomorrow’’. Trustworthiness: Trustworthiness is seen as a property of another agent while trust involves the temporal ‘relationship’ between truster and trustee. As trustworthiness is private to the other agent only an estimation of its trustworthiness can be measured (Abdul-Rahman & Hailes, 2000). Reputation: Reputation is an expectation about an agent’s behaviour based on the information about or observations of his past actions. Reputation can be seen as a reflection of the trustworthiness (Abdul-Rahman & Hailes, 2000). Note that trust and reputation are based on the expectation held by an entity. These expectations are seen as a fundamental issue in order to make decisions based on trust.
The supply network control is build upon an architecture, providing a coherent foundation for the design of the control system. Upon this stable layer a coordination and control system is implemented using ant-based techniques. Disturbances are business as usual, the explorers will discover the disturbance. This disturbance will be notified to the order holon as consequence. Next the order agent can decide to switch from intention if the disturbance is big enough. Still there are some questions left. The behaviour of the different holons should be social. Indeed, holons must adhere sufficiently to declared intentions if the forecasting mechanism needs to be effective. In factory control this is a relative easy task, you can hardcode the social behaviour of every holon and force them to use this behaviour. In a supply chain an additional mechanism is required. The key to that additional mechanism is trust. The trust mechanism is addressed in the following section. 4. Trust and reputation for supply networks This section gives both an overview on the challenges and a design of a possible framework to make use of trust and reputation systems in supply networks. The framework description consists of the various building blocks. Building blocks serve to connect trust mechanisms to the supply network control system. A concrete example of how the framework works together with the supply network control system is given in the last subsection. Trust between entities is, in the scope of the paper, considered as trusting in long term relationships. This means that entities cannot join or leave the supply network at regular base. In many cases entities can fall back on a large interaction history while placing a new request. 4.1. Enabling social behaviour in supply networks The previous section showed that in order to scale the coordination mechanism from factory control to supply network control a social behaviour is desired (Section 2). Socially behaving entities try to create ‘‘win–win’’ situa-
4.2. Trust and reputation definitions
4.3. Trust properties This section describes some properties related to trust. The descriptions are mainly meant to explain some
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
deficiencies in the current research which are fundamental in order to apply current techniques in the context of supply network control. In this section two specific properties are described: the context and the time dependency. 4.3.1. Context Context is one of the most difficult issues related to trust and reputation systems. A small difference in the context implies that previous observations are not valid/useful any more in the new situation. In this section a description of the context is given and the mainstream schema of the current approaches is explained. Problems with or enhancements to this schema are also noted. Context specification: Reputation and trust are highly dependent on the context in which the interactions take place. As a consequence, judging or adapting reputation profiles requires context information. Context in general can be divided into two major categories, observable context and social norms, which are explained below. Observable context consists out of parameters which can be observed immediately by the interaction. Examples of possible parameters determining the context are manifold. In the following list some examples are given:
Service type: Some organizations can be trusted for delivering one type of service compared to other types (outside their core business). Entity role: Entities like suppliers, clients, etc. can act according to different roles. E.g.: entities usually have a client and provider role. In some cases, an entity behaviour can differ depending on its role. In everyday-life, persons can have a working role and a husband role. It is clear that the behaviour in these roles, working or husband role, is completely different. As a consequence, reputation profiles are not exclusive to one entity. Entities can have many different reputation profiles, each one of them attached to a role. Behaviour compensation: The expected behaviour of an entity is not always the one actually executed but in many cases there is an acceptable alternative (for all parties) chosen. This compensation by executing an alternative has less negative impact on the reputation profile. E.g.: a customer not keeping its declared intention but executing an equally or even better alternative should not lead towards a decreased reputation level. Commitment: Commitment is a value indicating the willingness/ability to keep an intention. This level of commitment is important to indicate to the environment whether or not the intention is accurate, independent of the other context parameters. Time: Supply network entities seldom have a time invariant behaviour. Sometimes, a systematic repetitive behaviour can be observed, which can influence the trustworthiness of the respective entity. E.g.: the confidence of getting very tasty strawberries in the winter from the supermarket will be very low.
1399
Together with the observed parameters, social norms define the context in which interactions happen. The observed context provides a clear statement of what is expected with respect to each context parameters (Ramchurn, Sierra, Godo, & Jennings, 2004). However, the ‘‘social’’ setting in which the interaction is taking place may also give rise to expectations which are not explicitly stated in the interaction itself. For example, a buyer agent from country A might expect seller agent from country B to deliver goods nicely wrapped in gift paper. This clause may not have been specified in the contract as it is a norm in the client’s group. Current approaches: In many current approaches (March, 1994; Schillo, Funk, & Rovatsos, 2000; Zacharia, 2002) context information is neglected or not used at all. Current approaches using context information apply the following schema: (1) Identify possible entities (entities capable of performing desired service). (2) For all entities, identify context. In most cases the context consist out of the contract parameters. (3) Locate similar (with respect to the context) situations in history. If the context is not present in the history, a context generalization or specialization is performed. (4) Calculate/retrieve trust level for all entities. (5) Sort and select the entity having the best trust level. Note that in this schema trust levels are attached to context instances instead of to the entity. The major difficulty is generalizing or specializing the context in order to use history information. Generalization is in some cases possible but as consequence there is a need for a long bootstrap phase. Many interactions are needed in the history to judge a new interaction in a slightly different context. The framework described in the paper gives a basic schema to overcome this gap. With the proposed system, reputation can be associated to an entity. In this way generalization and specialization becomes easier. 4.3.2. Time dependency Trust and reputation are subject to evolution. Information is not staying valid all the time, or does not have the same impact over time. This time dependency is expressed in two ways namely: the evaporation of trust information and the continuous evaluation of trust levels. Evaporation: Evaporation or forgetting trust information is important to make a trust system stable. Evaporation is the mechanism where information will become obsolete after a certain amount of time. Decisions are based on information which is mostly subject to a lot of uncertainties. Consider a decision to decrease the reputation level of a service provider due to a misunderstanding or an unknown norm (e.g. see 4.3.1). This shortcoming of the service provider could indicate a possible bad service provider but on the other hand this could only be temporary (e.g. the service provider learned from the
ARTICLE IN PRESS 1400
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
earlier shortcoming). In such cases it is very important to evaporate the information. Evaporation goes hand in hand with refreshing trust information if it is still valid. Note that evaporation does not only apply to external trust information (witness reputation) but also to internal trust information. Continuous evaluation: Entities live in a dynamic world where not all information is available at one point in time. Still, entities need to make decisions, e.g.: assigning a request for a service to a service provider. In some cases, entities do not have all necessary information available. Current trust mechanisms (March, 1994; Schillo et al., 2000; Zacharia, 2002) follow a schema where entities can be punished by making decisions early. The reputation level is calculated only once. So once a decision is made, the level holds till execution, even if the confidence (at the side of the trustee) of the solution would change. In order to give the opportunity to interact between the decision and the final execution, trust systems should provide an incremental evaluation and reevaluation of reputation levels. 4.4. Trust framework description This section describes an approach to deal with the problems in the trust systems indicated above. The following paragraph first gives a short overview of this section. The framework starts from an interaction which results in a promise made by the trustee. The truster builds an expectation based on the promise made. In contrast to current systems, the confidence of the specific interaction is determined by the expectation instead of by the promise. Also evaluation and the consequence of updating the reputation level is based on the difference between the expectation and the outcome. All components related to this framework are described in more detail in the following subsections. 4.4.1. Promises Entities are interacting with other entities in order to achieve their goals. An interaction looks in general as follows (Huynh, Jennings, & Shadbolt, 2004): (1) ‘‘entity a’’ creates a request, (2) ‘‘entity a’’ forwards his request to another agent, ‘‘entity b’’, (3) ‘‘entity b’’ interprets the request, (4) ‘‘entity b’’ builds a implicit promise and answers to ‘‘entity a’’. The promise made by ‘‘entity b’’ has no value like a contract but in a social environment this promise should be a statement with a certain level of commitment. Important to note is that ‘‘entity b’’ builds the promise with respect to the context in which the interaction happens. A promise could be specified in less detail if the entities are interacting
on a regular base (e.g. the punctual postman coming every day does not need to tell you when he will arrive at your house). 4.4.2. Expectations Promises made by the service provider imply some explicit expectations of the service user. The service user expects that the delivery of a product will not be delayed seriously. Based on this expectations further decisions are made. In the framework expectations are build by the ‘‘expectation builder’’. This component is a key part in the trust framework and is described in more detail in the following paragraph. 4.4.3. Expectation building The expectation builder is a component creating expectations from promises made from the trustee. The expectation builder can be implemented in many ways from very sophisticated to very simple behaviours, from simple rules to learning mechanisms. One possible rule could be to soften promised delivery times by observing the late deliveries in the history. In other words, a expectation builder can learn out of the history of interactions. In the case where the builder is not able to give a useful expectation he can perform two things:
indicate a bad expectation confidence (the reputation level serves as a fall back mechanism for the expectation builder, see further); and forwards the doubt towards the trustee. The truster will challenge the trustee to give more information about his promise. Suppose that, I buy a chocolate box in an online shop to give it as a present to my girlfriend. On the website there is a ‘‘special wish’’ input field where I indicate that the box should be send in wrapping paper. The confirmation of my order noted the delivery day but did not state how the box will be packed. In this situation my expectation builder could ask for more detail in order to verify my special wish.
4.4.4. Evaluation Interactions relating to their respective promises are subject to the context. As indicated in 4.3.1 context is one of the major difficulties to judge entities based on promises. In contrast to interactions and promises, expectations are less dependent on the context. The expectation is build with respect to the context. e.g.: expectations are normalized against the context. As discussed earlier, trust is subjective. So any representation of trust is also subjective. A level or value of trust only exists as a subjective belief rather than an objective property of some entity or relationship between two entities (Abdul-Rahman & Hailes, 2000). This subjectiveness of trust justifies the use of the confidence of expectations instead of promises. As the expectation is closer to the final outcome (normalized), evaluation and decision taking will be easier.
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
4.4.5. Reputation Interacting entities which have a ‘‘good relationship’’ will be able to create better expectation from each other compared to interacting with new partners. Ones entity’s reputation gives an idea on how good the entity can interact with the other entity. In other words it indicates how good one entity can build expectation from the promises made by the other entity. An expectation may or may not be realistic depending on:
lacking information and misinterpretations; and information from entities trying to cheat the truster.
The ability of ‘‘agent x’’ to create good expectations from ‘‘agent y’’ is a much less context dependent statement. For this reason, traditional reputation mechanisms (Cachon & Netessine, 2004; Sierra & Debenham, 2005) are well suitable and applicable to implement this building block. 4.4.6. Trust and decisions Trust and reputation systems do not deliver added value if they are used independent of the total system. The original goal of introducing trust systems is to grant advantages to more socially behaving entities. Of course, the risks (chances of risk) should be weighted against the advantages of a certain service provider. To do so, a trust system needs to be coupled with an entity’s decision model. The trust framework presented in this article interconnects the trust framework with the decision model in two ways:
the system interprets a promise by using history information and by creating an expectation (normalized with respect to the context); and the system gives an indication about the confidence of the created expectation.
In this way the decision modules are able to calculate the performance of the expectation and weigh it by the confidence level. This weighted performance gives a clear indication about the service provider’s promise. 4.4.7. Interaction outcome Interactions are leading to outcomes which can be stored into the history. The history of the outcomes (compared with the expectation) is forwarded to both the expectation builder and the entity’s reputation. Outcomes can be either final service execution outcomes or intermediate promise renewals or rejections. 4.5. Trust framework example This section describes the trust framework for the interaction of a simple explorer ant (see 3.2.2) with a particular resource agent. In fact the originating order agent (to where the explorer belongs) should trust the resource agent. As explained previously in this paper,
1401
explorer ants create solutions and assess the performance of this solution. Solutions are not order sequences but are composed out of a set of operations performed on resources in a particular time slot. Time slots can have multiple implementations, from ‘simple start and end times’ to ‘start and end times enriched with slack time indications’, ‘distributions on the start and end times’, etc. In general explorer ants give information about the required service and get a time slot back from the resource agent. Explorer ants build this solution by interacting with resource agents. In factory control, these interactions happen in a ‘‘safe environment’’: all entity goals are aligned. In supply networks this property is no longer true. Moreover, it is not always possible to give adequate information, e.g. MRP systems cannot give precise deliver times. Supply network control systems need to address and compensate the missing, inadequate information. Solutions cannot be regarded as ‘‘correct’’ information per se. In order to illustrate how the trust framework deals with the lack of information or with cheating entities, the simplest interaction possible is considered. The explorer ant indicates the time on which it arrives at the resource agent together with the operation needed. As consequence the resource agent gives a time slot, start and end time, back. The resource agent, which gives a time slot back to the explorer ant, is considered giving a promise to the ant. As every entity in the system, the ant representing the order agent, can build an expectation out of a promise (in this situation an execution time slot). More precisely, the expectation builder of the order agent will enrich or transform the given time slot. The sample expectation, build by the expectation builder, looks as follows: a start and endtime and a possible slack time related to the given end time are included. At the execution of the operation, the order agent is validating whether the expectation was accurate or not. The reputation of the resource will be high if the execution of the operation is always between the start and endtime plus possible slack time. Next to the adaptation of the reputation profile the order agent could update the expectation builder by increasing or decreasing the slack time for that specific entity. The history can be implemented as a blackboard, where outcomes of interactions are pheromones. In this way old information will be regarded outdated without explicit management (it evaporates). As previous explored solutions will be refreshed from time to time intermediate steps are considered and are used incrementally. The start and end time given by the resource while refreshing a solution is now considered as output. This output will be evaluated with respect to the declared promise (from the same resource) in the previous explorer phase. In the beginning a resource agent still can change without affecting its reputation profile seriously. The closer the execution time is approaching, the more its reputation
ARTICLE IN PRESS 1402
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
level will decrease when it fails to uphold its promise/ expectation. 5. Conclusion This paper presents a bottom-up view of the supply network control problem. Starting from a manufacturing control system based on PROSA and ants, useful insights on several common problems in supply networks are given. Disturbances, the changing environment and other uncertainties are treated as business as usual. Extending the used manufacturing control is done in several steps: (1) extending the PROSA architecture, (2) adapting the ant based coordination mechanism, and (3) adding reputation models to enforce social behaviour. References Abdul-Rahman, A., & Hailes, S. (2000). Supporting trust in virtual communities. In Hawaii international conference on system sciences (Vol. 33). Hawaii: Maui. Cachon, G., & Netessine, S. (2004). Game theory in supply chain analysis. Handbook of quantitative supply chain analysis: Modeling in the eBusiness Era. Carter, J., Bitting, E., & Ghorbani, A. A. (2002). Reputation formalization within information sharing multiagent architectures. Computational Intelligence, 45–64. Devinney, R. B. T., & Pillutla, M. (1998). A formal model of trust based on outcomes. Academy of Management Review, 459–472. Hadeli, Valckenaers, P., Zamfirescu, C., Van Brussel, H., Saint Germain, B., Holvoet, T., et al. (2003). Self-organising in multi-agent coordination and control using stigmergy. ESOA. Huynh, T. D., Jennings, N. R., & Shadbolt, N. (2004). Fire: An integrated trust and reputation model for open multi-agent systems. In Proceedings of 16th European conference on artificial intelligence (pp. 18–22). Valencia, Spain. Koestler, A. (1989). The ghost in the machine. Arkana Books.
Lee, L., Padmanabhan, V., & Whang, S. (1997). The bullwhip effect in supply chains. Sloan Management Review, 93–102. Lejeune, M. A., & Yakova, N. (2005). On characterizing the 4 c’s in supply chain management. Journal of Operations Management, 81–100. Luhmann, N. (2000). Familiarity, confidence, trust: Problems and alternatives. In Trust: Making and breaking cooperative relations (electronic ed.) (pp. 94–107). Department of Sociology, University of Oxford. Malone, T. W., & Crowson, K. (1994). The interdisciplinary study of coordination. ACM Computing Surveys, 87–119. March, S. (1994). Formalizing trust as a computational concept. Ph.D. thesis, University of Stirling. Ramchurn, S. D., Sierra, C., Godo, L., & Jennings, N. R. (2004). Devising a trust model for multi-agent interactions using confidence and reputation. International Journal of Applied Artificial Intelligence, 18(9–10), 833–852. Rice, J. B., & Ronchi Jr., S. (2002). Strategic partnerships: Collaboration, alliances and the coordination spectrum. Logistics Solutions 22(1), 22–26. Sauter, J. A., & Parunak, H. V. D. (1999). Ants in the supply chain, workshop on agent-based decision support for managing the internetenabled supply chain, pp. 1–9. Schillo, M., Funk, P., & Rovatsos, M. (2000). Using trust for detecting deceitful agents in artificial societies. Applied Artificial Intelligence (Special issue on trust, deception and fraud in agents societies), 825–848. Sierra, C., & Debenham, J. (2005). An information-based model for trust (pp. 25–29). AAMAS Utrecht University. Valckenaers, P., & Van Brussel, H. (2005a). Fundamental insights into holonic systems design. Applications of Holonic and Multi-Agent Systems, 3593, 11–22. Valckenaers, P., & Van Brussel, H. (2005b). Holonic manufacturing execution systems. CIRP Annals, 427–432. Verstraete, P., Valckenaers, P., Germain, B. S., Van Brussel, H., & Hadeli, H. (2006). Integration of planning systems and an agent-oriented mes. International Journal for Information Technology and Management, 159–174. Wyns, J. (1999). Reference architecture for holonic manufacturing systems. Ph.D. thesis, K.U.Leuven. Zacharia, G. (2002). Collaborative reputation mechanisms for online communities. Ph.D. thesis, Massachusetts Institute of Technology.
Control Engineering Practice 15 (2007) 1394–1402 www.elsevier.com/locate/conengprac
A multi-agent supply network control framework Bart Saint Germain, Paul Valckenaers, Paul Verstraete, Hadeli, Hendrik Van Brussel K.U.Leuven-PMA, Celestijnenlaan 300B, B-3001 Heverlee, Belgium Received 11 March 2005; accepted 11 December 2006 Available online 9 February 2007
Abstract This paper presents a bottom-up solution for the supply network coordination problem. This manuscript treats the coordination as an ongoing task and takes disturbances and exceptions as business as usual. A manufacturing control and coordination mechanism based on PROSA and ants [Hadeli et al. (2003). Self-organising in multi-agent coordination and control using stigmergy. ESOA] is used as base for the overall coordination of the supply network. Reputation serves to enforce the social behaviour among the production entities. The interactions of the control mechanism with the reputation models are given. r 2007 Elsevier Ltd. All rights reserved. Keywords: Supply networks; Multi-agent; Trust; Reputation; Ant coordination
1. Introduction
Today, manufacturing industries are aware that designing the right kind of flexibility into their forthcoming production systems is vital to their future competitiveness. Companies have tried to optimize their own internal processes over a long period of time. Nowadays companies do not operate anymore in an isolated world. To achieve further optimization in their production system, industrial units have to look beyond the ‘‘walls’’ of their company (Sauter & Parunak, 1999). Supply network coordination is an essential requirement in current industries. Coordination is managing dependencies between activities (Malone & Crowson, 1994). Shared resources, producer/consumer relationships, simultaneous constraints, and tasks/subtasks can be identified as dependencies between activities in the supply network. Coordination, between entities of the supply network while offering and using services, exists on different levels (Rice & Ronchi, 2002):
markets: non-repetitive transactions (e.g.: e-commerce); alliances: long term contracts, repetitive transactions; Corresponding author. Tel.: +32 016322954; fax: +32 016322987.
E-mail address: [email protected] (B. Saint Germain). 0967-0661/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.conengprac.2006.12.003
partnerships: run the business with other part owners (partners); and vertical integration: running the business as owner (organization).
This paper discusses a multi-agent coordination and control approach to operate the external and internal logistics in multi-site/multi-organization topologies. The coordination discussed in this paper applies to long term relations and happens in a repetitive way. A fully centralized control is not applicable for this kind of coordination. Only long term relations are considered. As a consequence, the control system is not exposed to open ‘‘violent’’ environment (markets). This does not mean that all partners can be trusted in all situations. Exceptional behaviour should either be foreseen or captured and taken into account for later interactions. Past experience reveals that manufacturing control cannot be implemented through ‘one shot scheduling solutions’. On the shop floor, schedules have acquired a reputation for rapidly becoming invalid because of the frequent changes and disturbances. This effect becomes even bigger in supply networks (Lee, Padmanabhan, & Whang, 1997) and requires until now some ‘‘brute-force’’ solutions (i.e. huge buffers, low product-type variability). Consequently, the multi-site manufacturing control
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
presented in this paper, treats changes and disturbances as business-as-usual. It considers manufacturing as a nonstop ‘going concern’. This paper first defines the vision on the problem, supply network control. A bottom-up design is taken as a starting point. Next, the paper describes how a supply network can be controlled starting from factory control and coordination mechanism (Hadeli et al., 2003). The supply network description will show that social behaviour of all entities involved is required. In supply networks this assumption is not fulfilled and requires some extra mechanisms. In the final section, a trust framework is described answering this requirements. 2. Supply network control definition This section presents a definition of supply network control. Control and especially supply network control can be understood in many ways. Following a bottom-up approach, supply network control is defined by the nature of a supply network and the problems it brings about. The paper uses the insight that a production entity cannot be regarded as a black box. Instead a production entity will be defined as an autonomous and specific unit in the overall supply network. Supply networks are specified as multilevel networks of entities (Lejeune & Yakova, 2005). Supply network control will be defined as a non-stop going concern with as main concern the productivity and responsiveness of the total supply chain. This global productivity has to be understood with respect to the social nature of the problem.
1395
a single owner, and therefore the network control cannot impose a single goal and/or strategy. The entities will have varying degrees of autonomy, which the network control has to account for. 2.1.2. Specificity Specificity reflects the uniqueness of an entity. With respect of the technology and strategic decisions enabling to provide services, it is very difficult to model the different entities in general. Demand variation, finite capacity, disturbances and policy changing in the technological process makes forecasting and planning of production a unfeasible/inaccurate task without having a deeper insight into the entity. Therefore, an entity cannot be adequately modelled by a black box comprising a processing unit with input and output buffers. This is a novel approach that contrasts with more economic and control theoretical approaches where a lot of assumptions are made towards the logic inside the entities: fixed lead-times, no disturbances, etc. This observation—entities are specific—builds on the research team’s experience with manufacturing control. When a control system operates in a competitive environment, overconstraining its options by oversimplifying the world model is likely to be detrimental for system performance. And, unlike in administrative systems, it is not economically feasible to reengineer the production systems to fit the model of the control system. Hence, production entities cannot be reflected in the control system by a single generic model. 2.2. Supply network definition
2.1. Production entity definition Production entities are units in a production environment that deliver services. Services as they are carried out by entities have to be understood in the broad sense. Transport services, producing components and repairing machines are only a few examples of possible services. Services are described by means of their capabilities and constraints. Truck transport services are described by the transport capability and by constraints like maximum height/width and weight of the pieces that trucks can carry. Entities have typically two properties that characterize themselves: autonomy and specificity (Sauter & Parunak, 1999), explained in the following paragraphs.
A supply network is defined as a multi-level network of production entities collectively responsible for procurement, manufacturing and distribution activities associated with one or more families of related products. Entities can be aggregated or may cooperate with each other. (1) Cooperation expresses the ability to use services of the connected entity. (2) Aggregation models the organizational hierarchy at one point in time from the supply network. Organizations are divided in several plants, plants comprise a couple of divisions, etc. Both aggregations and cooperations connections can be subject to changes. New partners are coming into the supply network, organizations restructure, outsourcing, etc. 2.3. Supply network control definition
2.1.1. Autonomy The autonomy stands for the ability to decide how, when and where to deliver services together with which services to provide to their environment. These local decisions are not always without consequences. In particular, entities should behave in a socially acceptable manner. This first observation above emerges out of the essential difference between factory control and the supply network control. Production entities in a supply network do not all belong to
Supply network control copes with the logistics in a supply network as defined in the previous section. Supply network control handles the external factory logistics while interacting with internal factory logistics (Lejeune & Yakova, 2005; Sauter & Parunak, 1999). Logistics management includes not only decision taking but also and not less important, information handling, monitoring and forecasting (Valckenaers & Van Brussel, 2005b).
ARTICLE IN PRESS 1396
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
Controlling supply networks comprises a variety of objectives. The first objective is to cope with the going concern in supply chain control. Treating the control system as a static solution will fail in the dynamic and uncertain environment of the supply network. Disturbances on production level are propagated in both horizontally and vertically in the supply network, and the fluctuation on the uncertainty becomes even bigger (Lee et al., 1997). The second and probably the most important objective of the control system is the productivity over the overall supply network. There should be a tendency to pareto-optimal solutions where win–win situations—out of the pareto-optimal set—over different entities are preferred when taking a local decision. Internal versus external logistic interactions happen with respect for the autonomy and specificity of each of the entities belonging to the supply network. Specificity makes central coordination and control very hard and not desirable due to the crucial knowledge inside the entity. This knowledge is very hard to transfer in a general way and has to be inevitably handled by the entity itself. 2.4. Discussion In this paper a definition for supply chain control is given starting from a bottom-up design principle. Entities from supply networks differ from resources in a factory in two ways. Entities are autonomous and specific. These differences and the dynamics of the problem make central approaches and approaches where assumptions on internal entity logic are made hard to maintain. Supply networks are not defined as a single graph but as a multi-level network that represents the structure of the entities at one point in time. Supply network control is defined as a going concern (robust against changes and disturbances) with as main target maximizing the productivity and responsiveness of the total supply network. 3. Supply network coordination and control The supply network control application is based on a already developed multi-agent factory control framework (Valckenaers & Van Brussel, 2005b). This framework is build on the PROSA architecture and is inspired by ant colony behaviour. Instead of an one shot planning system like MRP, MRPII (Verstraete, Valckenaers, Germain, Van Brussel, & Hadeli, 2006), this framework reacts on changes and disturbances in its environment. Besides the flexibility and the robustness planning, the control system is build in a modular way which enables the easy transition from one application towards another. PROSA was developed for manufacturing control applications; nonetheless this section shows how this architecture can be adapted towards a supply network architecture. Next, insights on ants which can help to coordinate the supply network are described.
3.1. Supply network architecture The reference architecture described in this paper is based on the holonic paradigm developed by Koestler (1989). Koestler introduced the word holon as a composition of the word ‘‘holos’’ ð¼ wholeÞ an the post-fix-on in proton ð¼ partÞ. The two main properties of a holon are autonomy and cooperation. The autonomy provides stability and robustness in the changing world. The cooperation on the other hand gives the holon a place in the bigger system. In the framework the holarchy is implemented by an agent community. Each holon is represented as an agent and manages the organizational structure of the holons. 3.1.1. PROSA Starting from the holonic concept a reference architecture was developed for production systems, called PROSA (Wyns, 1999). Manufacturing control, as a conceptual design, is divided in a number of sub-systems, cooperating with each other, and across the different types of holons, a holarchy is formed. The structure of this architecture is built around three basic holons: product, order and resource holons.
Product holons provide the process knowledge of a product (recipe). This means that a product holon can identify all possible subsequent production steps for an order in a certain state. The implementation and representation of such a recipe is internal to the product holon. The order holon manages the physical product instance being produced, the product state model, and all logistic information processing related to the job. It is responsible for performing the assigned work correctly and on time. Each resource holon corresponds to a physical part in the manufacturing system. A resource holon corresponding to a machine provides the knowledge of the physical machine or part; it knows the supported operations, the production capacity, queuing strategy, etc.
A main advantage of this architecture is that it reflects reality (physical things, order, machines, product types) and no functions (e.g.: average lead time). No unstable constraints are introduced until this point e.g.: no ‘‘bill of materials’’ is used (Valckenaers & Van Brussel, 2005a). 3.1.2. Supply network architecture extensions As Koestler (1989) stated, systems can consist out of several smaller entities and some systems belong to bigger aggregated systems without having an upfront-defined hierarchy. As described in the previous paragraph, PROSA is a holarchy over the different holon types. The extension towards supply network control will employ the same mechanism and a holarchy will formed inside each PROSA
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
holon type. The following overview indicates what this extension means for every separate PROSA holon. Product holons are not longer stand-alone sub-systems. Whenever you take a snapshot at a point in time, the structure of different product holons will form a hierarchy. E.g. suppose there is a product holon that corresponds to a specific car model. Such a holon provides the knowledge to manufacture this car type to the other holons in the control and coordination mechanism. As in real life, this product holon is not able to cope with all complexity. In fact, the product holon for that car type consists out of several subholons each of them describing their respective subsystem, the wheels, the engine, the seats, etc. Holons can delegate tasks to their respective sub-holons while taking the responsibility to coordinate them, informing about changes, aligning, etc. A resource holon can consist out of smaller resource holons each of them responsible for a part of the total, initial problem. In this way the supply network is modelled reflecting the reality (see Section 2). All machines, operators, raw materials, etc. are composed into a production facility. By composing production facilities and logistics between the facilities, organizations are modelled. Aggregations, like an ‘‘organization resource holon’’, serve as an other level of abstraction. Service users do not necessary have to cope with the sub-holons. Order holons can separate their concerns and delegate sub-problems to smaller order holons, responsible for that sub-problem. For instance, an order holon, responsible for a large transportation task, may spawn several new order holons responsible for smaller and more manageable tasks (like a single transport segment covered by a truck), which the original order holon coordinates. 3.2. Supply network coordination This section gives a short overview on how a coordination and control mechanism is implemented on top of the PROSA architecture. The control system described uses the insights of ant coordination techniques. These insights rise some interesting properties in manufacturing coordination and control. First a brief overview is given about the approach used in manufacturing control (more information about the used approach can be found in Valckenaers & Van Brussel, 2005a). Next some extensions towards supply network control are discussed and links to the trust and reputation framework are indicated. 3.2.1. Ant coordination and control Ant coordination and control is based on the insights from the behaviour of ants searching for food. The main advantage of such an ant-based design is that individual agents/holons are not exposed to the complexity and dynamics of the global situation. None of the agents needs a mental map of the environment nor do these agents communicate amongst themselves about the environment.
1397
Also, the communication is carried out in a very robust way by means of an evaporate-and-refresh mechanism. Pheromones are put by the ants which evaporate after some time. In order to keep the information available, ants need to refresh the pheromones. There are however two requirements for the ant coordination mechanism to be effective: (1) there must be a large number of ants in the system, and (2) it must be cheap and easy to lose some of the ants. In practice, this means that the ants in the network control system must be virtual entities, and cannot be strongly connected to the presence and location of physical entities in the supply network. E.g. an ant cannot represent a car as it is not cheap to loose an ant and it is not affordable to require many ants to get to a solution. Instead of representing a ‘‘car’’, the ants in the framework represent either the exploration and the intention tasks of the order holon. Ants representing the order holon behaviours fulfill the two requirements of the ant coordination mechanism. In the following sections, the exploration and intention behaviour is explained in more detail. 3.2.2. Exploring The exploration services investigate possible routes and processing sequences for the product instances in a factory. The basic exploration service implement the following scheme: (1) Each order agent creates exploring ants at a regular frequency. (2) Each exploring ant virtually executes a possible scenario on behalf of its order agent. A scenario is more than only a set of resources to be visited. Possibilities to join, make batches, execution slots for the different operations, etc. are part of the total solution space. (3) At the end, the explorer ant will report the explored solution to the order agent. Based on the results of the exploring ants, the order agent keeps a set of candidate routes and is refreshed regularly. If the information gets out dated, it evaporates. 3.2.3. Intention propagation An intention reflects to some extent the future behaviour of a holon. The intention emerges out of the set of possible, explored, candidate solutions. One of the possible candidate solutions is regarded as the best and will be executed, with high probability, in the near future. Next the holon triggers an intention ant responsible for informing the different service providers about the intention of its holon. This intention is posted on the resources in the form of pheromones, it will evaporate if the intention is not refreshed regularly. Based on these intention service providers can build a dynamic load forecast, which can be used by explorer ants to make their performance estimate reliable.
ARTICLE IN PRESS 1398
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
3.2.4. Confidentiality/filtering of information The main difference for the ant topology with factory control lies in the holarchical structure of the supply network. Some resource holons represent a set of production entities. These aggregated holons have not only horizontal connections, e.g. suppliers and clients, but also vertical connections to their sub-holons, e.g. divisions. Ants cannot always access the vertical connections due to two reasons: (1) some information is confidential for the outside world and (2) it is not always desirable to have all information, it would only trigger information overload. The aggregated holon tackles this problem by hiding the connections and encapsulating the needed information from the underlying production facilities.
tions as much as possible. Although a situation is not immediately in line with their own goal, entities should tend to an improvement of the global supply network control performance. This behaviour should happen in a distributed environment. A centralized control is not desired or not applicable (see autonomous and specific). Trust and reputation systems claim to serve as key elements in making the network behave in a social way. Any unsocial behaviour will be punished in terms of a reputation drawback. Reputation as it is, builds upon and affects entity behaviours by means of all communication channels and interactions.
3.3. Discussion
In this section trust and related concepts are defined. The definitions are meant to give a clear view on trust and reputation. Next to the definition of trust and reputation, confidence is explained. The framework, described later, operates based on the confidence of particular explored solutions. Trust: Trust is a generalized expectancy held by an individual or group that the word, promise, verbal, or written statement of another individual or group can be relied on Carter, Bitting, and Ghorbani (2002) and Devinney and Pillutla (1998). In the supply network control system, order agents rely on resource and product agents while sending intention ants and explorer ants. At the same time, resource agents rely on the accuracy of the intentions send by the order agents. Confidence: Confidence can be described as the act of confiding, trusting, or putting faith in. The main difference between trust and confidence is that trust involves choosing alternatives whilst confidence does not (Luhmann, 2000). Confidence can be described as habitual expectation, e.g. ‘‘I am habitually confident that my milkman will deliver milk to the doorstep tomorrow’’. Trustworthiness: Trustworthiness is seen as a property of another agent while trust involves the temporal ‘relationship’ between truster and trustee. As trustworthiness is private to the other agent only an estimation of its trustworthiness can be measured (Abdul-Rahman & Hailes, 2000). Reputation: Reputation is an expectation about an agent’s behaviour based on the information about or observations of his past actions. Reputation can be seen as a reflection of the trustworthiness (Abdul-Rahman & Hailes, 2000). Note that trust and reputation are based on the expectation held by an entity. These expectations are seen as a fundamental issue in order to make decisions based on trust.
The supply network control is build upon an architecture, providing a coherent foundation for the design of the control system. Upon this stable layer a coordination and control system is implemented using ant-based techniques. Disturbances are business as usual, the explorers will discover the disturbance. This disturbance will be notified to the order holon as consequence. Next the order agent can decide to switch from intention if the disturbance is big enough. Still there are some questions left. The behaviour of the different holons should be social. Indeed, holons must adhere sufficiently to declared intentions if the forecasting mechanism needs to be effective. In factory control this is a relative easy task, you can hardcode the social behaviour of every holon and force them to use this behaviour. In a supply chain an additional mechanism is required. The key to that additional mechanism is trust. The trust mechanism is addressed in the following section. 4. Trust and reputation for supply networks This section gives both an overview on the challenges and a design of a possible framework to make use of trust and reputation systems in supply networks. The framework description consists of the various building blocks. Building blocks serve to connect trust mechanisms to the supply network control system. A concrete example of how the framework works together with the supply network control system is given in the last subsection. Trust between entities is, in the scope of the paper, considered as trusting in long term relationships. This means that entities cannot join or leave the supply network at regular base. In many cases entities can fall back on a large interaction history while placing a new request. 4.1. Enabling social behaviour in supply networks The previous section showed that in order to scale the coordination mechanism from factory control to supply network control a social behaviour is desired (Section 2). Socially behaving entities try to create ‘‘win–win’’ situa-
4.2. Trust and reputation definitions
4.3. Trust properties This section describes some properties related to trust. The descriptions are mainly meant to explain some
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
deficiencies in the current research which are fundamental in order to apply current techniques in the context of supply network control. In this section two specific properties are described: the context and the time dependency. 4.3.1. Context Context is one of the most difficult issues related to trust and reputation systems. A small difference in the context implies that previous observations are not valid/useful any more in the new situation. In this section a description of the context is given and the mainstream schema of the current approaches is explained. Problems with or enhancements to this schema are also noted. Context specification: Reputation and trust are highly dependent on the context in which the interactions take place. As a consequence, judging or adapting reputation profiles requires context information. Context in general can be divided into two major categories, observable context and social norms, which are explained below. Observable context consists out of parameters which can be observed immediately by the interaction. Examples of possible parameters determining the context are manifold. In the following list some examples are given:
Service type: Some organizations can be trusted for delivering one type of service compared to other types (outside their core business). Entity role: Entities like suppliers, clients, etc. can act according to different roles. E.g.: entities usually have a client and provider role. In some cases, an entity behaviour can differ depending on its role. In everyday-life, persons can have a working role and a husband role. It is clear that the behaviour in these roles, working or husband role, is completely different. As a consequence, reputation profiles are not exclusive to one entity. Entities can have many different reputation profiles, each one of them attached to a role. Behaviour compensation: The expected behaviour of an entity is not always the one actually executed but in many cases there is an acceptable alternative (for all parties) chosen. This compensation by executing an alternative has less negative impact on the reputation profile. E.g.: a customer not keeping its declared intention but executing an equally or even better alternative should not lead towards a decreased reputation level. Commitment: Commitment is a value indicating the willingness/ability to keep an intention. This level of commitment is important to indicate to the environment whether or not the intention is accurate, independent of the other context parameters. Time: Supply network entities seldom have a time invariant behaviour. Sometimes, a systematic repetitive behaviour can be observed, which can influence the trustworthiness of the respective entity. E.g.: the confidence of getting very tasty strawberries in the winter from the supermarket will be very low.
1399
Together with the observed parameters, social norms define the context in which interactions happen. The observed context provides a clear statement of what is expected with respect to each context parameters (Ramchurn, Sierra, Godo, & Jennings, 2004). However, the ‘‘social’’ setting in which the interaction is taking place may also give rise to expectations which are not explicitly stated in the interaction itself. For example, a buyer agent from country A might expect seller agent from country B to deliver goods nicely wrapped in gift paper. This clause may not have been specified in the contract as it is a norm in the client’s group. Current approaches: In many current approaches (March, 1994; Schillo, Funk, & Rovatsos, 2000; Zacharia, 2002) context information is neglected or not used at all. Current approaches using context information apply the following schema: (1) Identify possible entities (entities capable of performing desired service). (2) For all entities, identify context. In most cases the context consist out of the contract parameters. (3) Locate similar (with respect to the context) situations in history. If the context is not present in the history, a context generalization or specialization is performed. (4) Calculate/retrieve trust level for all entities. (5) Sort and select the entity having the best trust level. Note that in this schema trust levels are attached to context instances instead of to the entity. The major difficulty is generalizing or specializing the context in order to use history information. Generalization is in some cases possible but as consequence there is a need for a long bootstrap phase. Many interactions are needed in the history to judge a new interaction in a slightly different context. The framework described in the paper gives a basic schema to overcome this gap. With the proposed system, reputation can be associated to an entity. In this way generalization and specialization becomes easier. 4.3.2. Time dependency Trust and reputation are subject to evolution. Information is not staying valid all the time, or does not have the same impact over time. This time dependency is expressed in two ways namely: the evaporation of trust information and the continuous evaluation of trust levels. Evaporation: Evaporation or forgetting trust information is important to make a trust system stable. Evaporation is the mechanism where information will become obsolete after a certain amount of time. Decisions are based on information which is mostly subject to a lot of uncertainties. Consider a decision to decrease the reputation level of a service provider due to a misunderstanding or an unknown norm (e.g. see 4.3.1). This shortcoming of the service provider could indicate a possible bad service provider but on the other hand this could only be temporary (e.g. the service provider learned from the
ARTICLE IN PRESS 1400
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
earlier shortcoming). In such cases it is very important to evaporate the information. Evaporation goes hand in hand with refreshing trust information if it is still valid. Note that evaporation does not only apply to external trust information (witness reputation) but also to internal trust information. Continuous evaluation: Entities live in a dynamic world where not all information is available at one point in time. Still, entities need to make decisions, e.g.: assigning a request for a service to a service provider. In some cases, entities do not have all necessary information available. Current trust mechanisms (March, 1994; Schillo et al., 2000; Zacharia, 2002) follow a schema where entities can be punished by making decisions early. The reputation level is calculated only once. So once a decision is made, the level holds till execution, even if the confidence (at the side of the trustee) of the solution would change. In order to give the opportunity to interact between the decision and the final execution, trust systems should provide an incremental evaluation and reevaluation of reputation levels. 4.4. Trust framework description This section describes an approach to deal with the problems in the trust systems indicated above. The following paragraph first gives a short overview of this section. The framework starts from an interaction which results in a promise made by the trustee. The truster builds an expectation based on the promise made. In contrast to current systems, the confidence of the specific interaction is determined by the expectation instead of by the promise. Also evaluation and the consequence of updating the reputation level is based on the difference between the expectation and the outcome. All components related to this framework are described in more detail in the following subsections. 4.4.1. Promises Entities are interacting with other entities in order to achieve their goals. An interaction looks in general as follows (Huynh, Jennings, & Shadbolt, 2004): (1) ‘‘entity a’’ creates a request, (2) ‘‘entity a’’ forwards his request to another agent, ‘‘entity b’’, (3) ‘‘entity b’’ interprets the request, (4) ‘‘entity b’’ builds a implicit promise and answers to ‘‘entity a’’. The promise made by ‘‘entity b’’ has no value like a contract but in a social environment this promise should be a statement with a certain level of commitment. Important to note is that ‘‘entity b’’ builds the promise with respect to the context in which the interaction happens. A promise could be specified in less detail if the entities are interacting
on a regular base (e.g. the punctual postman coming every day does not need to tell you when he will arrive at your house). 4.4.2. Expectations Promises made by the service provider imply some explicit expectations of the service user. The service user expects that the delivery of a product will not be delayed seriously. Based on this expectations further decisions are made. In the framework expectations are build by the ‘‘expectation builder’’. This component is a key part in the trust framework and is described in more detail in the following paragraph. 4.4.3. Expectation building The expectation builder is a component creating expectations from promises made from the trustee. The expectation builder can be implemented in many ways from very sophisticated to very simple behaviours, from simple rules to learning mechanisms. One possible rule could be to soften promised delivery times by observing the late deliveries in the history. In other words, a expectation builder can learn out of the history of interactions. In the case where the builder is not able to give a useful expectation he can perform two things:
indicate a bad expectation confidence (the reputation level serves as a fall back mechanism for the expectation builder, see further); and forwards the doubt towards the trustee. The truster will challenge the trustee to give more information about his promise. Suppose that, I buy a chocolate box in an online shop to give it as a present to my girlfriend. On the website there is a ‘‘special wish’’ input field where I indicate that the box should be send in wrapping paper. The confirmation of my order noted the delivery day but did not state how the box will be packed. In this situation my expectation builder could ask for more detail in order to verify my special wish.
4.4.4. Evaluation Interactions relating to their respective promises are subject to the context. As indicated in 4.3.1 context is one of the major difficulties to judge entities based on promises. In contrast to interactions and promises, expectations are less dependent on the context. The expectation is build with respect to the context. e.g.: expectations are normalized against the context. As discussed earlier, trust is subjective. So any representation of trust is also subjective. A level or value of trust only exists as a subjective belief rather than an objective property of some entity or relationship between two entities (Abdul-Rahman & Hailes, 2000). This subjectiveness of trust justifies the use of the confidence of expectations instead of promises. As the expectation is closer to the final outcome (normalized), evaluation and decision taking will be easier.
ARTICLE IN PRESS B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
4.4.5. Reputation Interacting entities which have a ‘‘good relationship’’ will be able to create better expectation from each other compared to interacting with new partners. Ones entity’s reputation gives an idea on how good the entity can interact with the other entity. In other words it indicates how good one entity can build expectation from the promises made by the other entity. An expectation may or may not be realistic depending on:
lacking information and misinterpretations; and information from entities trying to cheat the truster.
The ability of ‘‘agent x’’ to create good expectations from ‘‘agent y’’ is a much less context dependent statement. For this reason, traditional reputation mechanisms (Cachon & Netessine, 2004; Sierra & Debenham, 2005) are well suitable and applicable to implement this building block. 4.4.6. Trust and decisions Trust and reputation systems do not deliver added value if they are used independent of the total system. The original goal of introducing trust systems is to grant advantages to more socially behaving entities. Of course, the risks (chances of risk) should be weighted against the advantages of a certain service provider. To do so, a trust system needs to be coupled with an entity’s decision model. The trust framework presented in this article interconnects the trust framework with the decision model in two ways:
the system interprets a promise by using history information and by creating an expectation (normalized with respect to the context); and the system gives an indication about the confidence of the created expectation.
In this way the decision modules are able to calculate the performance of the expectation and weigh it by the confidence level. This weighted performance gives a clear indication about the service provider’s promise. 4.4.7. Interaction outcome Interactions are leading to outcomes which can be stored into the history. The history of the outcomes (compared with the expectation) is forwarded to both the expectation builder and the entity’s reputation. Outcomes can be either final service execution outcomes or intermediate promise renewals or rejections. 4.5. Trust framework example This section describes the trust framework for the interaction of a simple explorer ant (see 3.2.2) with a particular resource agent. In fact the originating order agent (to where the explorer belongs) should trust the resource agent. As explained previously in this paper,
1401
explorer ants create solutions and assess the performance of this solution. Solutions are not order sequences but are composed out of a set of operations performed on resources in a particular time slot. Time slots can have multiple implementations, from ‘simple start and end times’ to ‘start and end times enriched with slack time indications’, ‘distributions on the start and end times’, etc. In general explorer ants give information about the required service and get a time slot back from the resource agent. Explorer ants build this solution by interacting with resource agents. In factory control, these interactions happen in a ‘‘safe environment’’: all entity goals are aligned. In supply networks this property is no longer true. Moreover, it is not always possible to give adequate information, e.g. MRP systems cannot give precise deliver times. Supply network control systems need to address and compensate the missing, inadequate information. Solutions cannot be regarded as ‘‘correct’’ information per se. In order to illustrate how the trust framework deals with the lack of information or with cheating entities, the simplest interaction possible is considered. The explorer ant indicates the time on which it arrives at the resource agent together with the operation needed. As consequence the resource agent gives a time slot, start and end time, back. The resource agent, which gives a time slot back to the explorer ant, is considered giving a promise to the ant. As every entity in the system, the ant representing the order agent, can build an expectation out of a promise (in this situation an execution time slot). More precisely, the expectation builder of the order agent will enrich or transform the given time slot. The sample expectation, build by the expectation builder, looks as follows: a start and endtime and a possible slack time related to the given end time are included. At the execution of the operation, the order agent is validating whether the expectation was accurate or not. The reputation of the resource will be high if the execution of the operation is always between the start and endtime plus possible slack time. Next to the adaptation of the reputation profile the order agent could update the expectation builder by increasing or decreasing the slack time for that specific entity. The history can be implemented as a blackboard, where outcomes of interactions are pheromones. In this way old information will be regarded outdated without explicit management (it evaporates). As previous explored solutions will be refreshed from time to time intermediate steps are considered and are used incrementally. The start and end time given by the resource while refreshing a solution is now considered as output. This output will be evaluated with respect to the declared promise (from the same resource) in the previous explorer phase. In the beginning a resource agent still can change without affecting its reputation profile seriously. The closer the execution time is approaching, the more its reputation
ARTICLE IN PRESS 1402
B. Saint Germain et al. / Control Engineering Practice 15 (2007) 1394–1402
level will decrease when it fails to uphold its promise/ expectation. 5. Conclusion This paper presents a bottom-up view of the supply network control problem. Starting from a manufacturing control system based on PROSA and ants, useful insights on several common problems in supply networks are given. Disturbances, the changing environment and other uncertainties are treated as business as usual. Extending the used manufacturing control is done in several steps: (1) extending the PROSA architecture, (2) adapting the ant based coordination mechanism, and (3) adding reputation models to enforce social behaviour. References Abdul-Rahman, A., & Hailes, S. (2000). Supporting trust in virtual communities. In Hawaii international conference on system sciences (Vol. 33). Hawaii: Maui. Cachon, G., & Netessine, S. (2004). Game theory in supply chain analysis. Handbook of quantitative supply chain analysis: Modeling in the eBusiness Era. Carter, J., Bitting, E., & Ghorbani, A. A. (2002). Reputation formalization within information sharing multiagent architectures. Computational Intelligence, 45–64. Devinney, R. B. T., & Pillutla, M. (1998). A formal model of trust based on outcomes. Academy of Management Review, 459–472. Hadeli, Valckenaers, P., Zamfirescu, C., Van Brussel, H., Saint Germain, B., Holvoet, T., et al. (2003). Self-organising in multi-agent coordination and control using stigmergy. ESOA. Huynh, T. D., Jennings, N. R., & Shadbolt, N. (2004). Fire: An integrated trust and reputation model for open multi-agent systems. In Proceedings of 16th European conference on artificial intelligence (pp. 18–22). Valencia, Spain. Koestler, A. (1989). The ghost in the machine. Arkana Books.
Lee, L., Padmanabhan, V., & Whang, S. (1997). The bullwhip effect in supply chains. Sloan Management Review, 93–102. Lejeune, M. A., & Yakova, N. (2005). On characterizing the 4 c’s in supply chain management. Journal of Operations Management, 81–100. Luhmann, N. (2000). Familiarity, confidence, trust: Problems and alternatives. In Trust: Making and breaking cooperative relations (electronic ed.) (pp. 94–107). Department of Sociology, University of Oxford. Malone, T. W., & Crowson, K. (1994). The interdisciplinary study of coordination. ACM Computing Surveys, 87–119. March, S. (1994). Formalizing trust as a computational concept. Ph.D. thesis, University of Stirling. Ramchurn, S. D., Sierra, C., Godo, L., & Jennings, N. R. (2004). Devising a trust model for multi-agent interactions using confidence and reputation. International Journal of Applied Artificial Intelligence, 18(9–10), 833–852. Rice, J. B., & Ronchi Jr., S. (2002). Strategic partnerships: Collaboration, alliances and the coordination spectrum. Logistics Solutions 22(1), 22–26. Sauter, J. A., & Parunak, H. V. D. (1999). Ants in the supply chain, workshop on agent-based decision support for managing the internetenabled supply chain, pp. 1–9. Schillo, M., Funk, P., & Rovatsos, M. (2000). Using trust for detecting deceitful agents in artificial societies. Applied Artificial Intelligence (Special issue on trust, deception and fraud in agents societies), 825–848. Sierra, C., & Debenham, J. (2005). An information-based model for trust (pp. 25–29). AAMAS Utrecht University. Valckenaers, P., & Van Brussel, H. (2005a). Fundamental insights into holonic systems design. Applications of Holonic and Multi-Agent Systems, 3593, 11–22. Valckenaers, P., & Van Brussel, H. (2005b). Holonic manufacturing execution systems. CIRP Annals, 427–432. Verstraete, P., Valckenaers, P., Germain, B. S., Van Brussel, H., & Hadeli, H. (2006). Integration of planning systems and an agent-oriented mes. International Journal for Information Technology and Management, 159–174. Wyns, J. (1999). Reference architecture for holonic manufacturing systems. Ph.D. thesis, K.U.Leuven. Zacharia, G. (2002). Collaborative reputation mechanisms for online communities. Ph.D. thesis, Massachusetts Institute of Technology.