Incorporating GA-Based Optimization into a Multi-Agent Architecture for FMS Scheduling

Incorporating GA-Based Optimization into a Multi-Agent Architecture for FMS Scheduling Iman Badr* and Peter Göhner* *Institute of Industrial Automation and Software Engineering, Stuttgart, Germany (e-mail: {iman.badr, peter.goehner}@ ias.uni-stuttgart.de) Abstract: Scheduling for flexible manufacturing systems (FMS) poses the challenge of optimizing the generated schedule while exhibiting flexibility to environmental dynamics. While the agent-based paradigm has been shown to tackle the inherent complexity of the problem and exhibit the required flexibility, it hinders the global optimization due to its decentralized structure. In this paper, genetic algorithms (GA) are incorporated into an agent-based scheduling architecture to overcome this shortcoming of agents and to achieve the required combination of flexibility with efficiency. Based on GA, individual agents optimize the schedule from their local view. Through the cooperation among the individual agents, a near-optimal schedule under consideration of the different optimization objectives is attained. Test results prove that the proposed integration yields near-optimal schedules with low computational complexity. Keywords: flexible manufacturing systems, FMS, agent-based manufacturing, genetic algorithms, GA. 1. INTRODUCTION The need to react to fluctuations and versatility in market demand and face competition threats has led to the increasing trend to produce a wide variety of product types in small batches. The advent of advanced technology like computerized numerically controlled (CNC) machines, automatic guided vehicles and robot arms has motivated the realization of automated factories or flexible manufacturing systems (FMS). FMS cater for bringing the efficiency of mass production to small-to-medium sized batch production with high product diversity. This objective necessitates flexible scheduling approaches which utilize the available resources efficiently and at the same time react to the dynamics of the environment flexibly. The agent-based paradigm has been advocated for exhibiting the required scheduling flexibility for FMS (Leitao, 2008 and Shen, 2002b). The underlying principle of agent-based scheduling is to decompose the problem into self-contained entities and to represent these entities by autonomous agents. Schedules are generated based on negotiations among these autonomous entities. The autonomy and interactivity of agents provide very good support for tackling the inherent complexity of the scheduling problem and its interrelationship with the various decision-making manufacturing tasks. The main shortcoming of the agentbased paradigm with regard to scheduling concerns the optimization of the generated schedule from a global perspective. For agents to achieve the required efficiency of the generated schedule, they have to accommodate for more sophisticated scheduling and optimization techniques (Shen, 2002b).

Genetic algorithms (GA) have been successfully applied to the manufacturing scheduling problem. They are distinguished from other meta-heuristic search methods, such as tabu search and simulated annealing, by their resistance to premature convergence to local optima (Lin et. al. 1997). One of the early studies combining GA with agent-based scheduling is reported in (Cardon et. al., 2000). The objective was to optimize an existing schedule. This is done by representing a Gantt-chart, capturing a manufacturing schedule, by agents. GA were employed to evolve agents that yield a more efficient schedule. In (Shen, 2002a), agents are combined with GA with the objective of schedule generation. However, GA are not incorporated into agents but rather applied at a centralized level to complement the flexibility of agents. In other words, agents are used whenever a flexible reaction is necessary and GA are used when optimization is required. The paper addresses the incorporation of GA into an agentbased scheduling architecture for FMS. This agent-based architecture is proposed in a previous work. The objective is to achieve the combination of flexibility and efficiency required for FMS. The rest of the paper is organized as follows. Section 2 defines the scheduling problem for FMS. Section 3 illustrates the proposed agent-based modelling of the problem. Section 4 embarks on the details of the evolution-based scheduling generation process. Section 5 provides evaluation results. Section 6 concludes with a brief summary and future work. 2. PROBLEM DESCRIPTION Scheduling for flexible manufacturing systems is usually described as an extension to the conventional job shop scheduling problem. The conventional job shop scheduling

problem has been extensively studied in the last decades because of its well known complexity as one of the most difficult combinatorial optimization problems. The problem is stated as follows. A set of jobs are to be executed on a set of machines according to a prescribed order. The objective is to minimize the makespan, i.e. the time required for all jobs to finish execution. In this problem, the machines required for every job are predefined. Therefore, the problem can be regarded as the sequencing of jobs on each machine. The FMS scheduling problem is even more complex than the standard job shop scheduling problem. The added complexity stems basically from the flexibility of machines in performing operations. Each job is to be allocated to a predefined set of operations rather than machines. Since every operation can typically be performed by more than one machine at any point in time, the allocation of machines to jobs for a certain operation has to be solved as well. This allocation implies the planning of dividing job quantities into lots to be distributed on machines for parallel execution. The FMS scheduling problem is further influenced by the flexibility of the supported part types. These part types exist in a wide spectrum and are thus grouped into part families based on their similarities in physical dimensions and their predefined operations. Machines switching from a part type to another have to incur setup time which is classified into minor and major, depending on whether the consecutive parts belong to different types of the same family or to different families respectively. While sequencing jobs on a certain machine, the incurred setup should be minimized. Conventionally, the job shop scheduling problem aims at minimizing the makespan. However, when it comes to FMS different optimization objectives have to be taken into consideration. First, the employed machines have to be fully utilized. This objective is significant for FMS because of their relatively high investment in resources. In addition, since the customer satisfaction plays a significant role in FMS, the makespan of jobs to be delivered to the individual customers should be taken into consideration. In general, the generated schedule has to be optimized by maximizing the resource utilization and at the same time minimizing the makespan of jobs, by taking the collective job constrains into consideration.

allocated to it. To enhance the generated schedule from a global point of view, schedule alternatives are delivered to the job and job group levels for evaluation. This evaluation is propagated to the lower layers and the schedule alternative with the highest evaluation is eventually selected and confirmed.

3. AGENT-BASED MODELLING

The entire schedule generation process is depicted in Fig. 2. As illustrated in the figure, this process consists of the following four steps.

The proposed agent-based scheduling approach employs a four layer architecture modelling physical and logical entities involved in the scheduling problem as agents. These layers correspond to resources, operations provided by these resources, jobs, and groups of jobs with common constraints. Regardless of the underlying FMS configuration, scheduling is performed through the interaction among these four types of agents. For a more detailed discussion of this agent-based modelling and its gained advantages, the interested reader is referred to (Badr, 2008) and (Badr and Göhner, 2009b). Fig. 1 captures the agent-based architecture. Schedules are generated and updated through the cooperation among operation agents and their machine agents. While operation agents work on optimizing the allocation sub-problem, every machine agent caters for optimizing the sequence of jobs

G1

Job group agents

Schedule evaluation

Job agents

J1

Operation agents

J2

J3 Allocation

Op 1

Op 2

Resource agents

Sequencing

M1

M2

M3

Schedule generation

M4

Fig. 1. Proposed agent-based FMS scheduling architecture Two models of the job arrival should be considered when scheduling for FMS: the sequential and the simultaneous. In the former model, jobs are incorporated in the schedule one by one. This implies the reduction of the sequencing subproblem into the simple insertion of the incoming job in the location corresponding to its priority. For this problem, no search is required for solving the sequencing sub-problem and simple heuristics were shown to lead to a near-optimal schedule (Badr, 2008). 4. AGENT-BASED EVOLUTIONARY SCHEDULING In case of the arrival of a set of jobs that have to be considered simultaneously when generating the schedule, GA are applied for searching for a near-optimal solution both at the operation and at the machine levels. First, the cooperative scheduling process along the different levels of the agentbased hierarchy is explained. Second, the incorporation of GA in the scheduling process is focused on. 4.1 Overview on the cooperative scheduling

1. Initialization The process is initiated by order agents, which correspond to groups of jobs that have to be delivered for a specific customer. Every order agent creates a job agent for every job. Every job agent is responsible for finding an allocation for itself. It sends a call for proposal for agents representing the operations in its work plan. 2. Cooperative evolution Upon collecting allocation requests, operation agents initialize the schedule generation process by deciding on the chromosome length l and the number of machines n to be considered for allocation. This is calculated from the number

of job requests and a predefined parameter, as will be explained in the next section. The operation agent looks up machines currently supporting the operation in concern and sends requests for proposals to the corresponding n machine agents. {Job group agent} Instantiate job agent

{Job agent}

{Operation agent}

Contact operations

Collect requests

1.Intitialization

Initiate evolution

2. Cooperative evolution

Population initialization Local evolution

Collect and decode

Collect and decode

Decode and validate {Valid chromosome} True

False Diversify

Evaluate

Evaluate {Termination}

{Machine agent}

Population initialization

Local evolution

3.Validation and evaluation Adapt GA parameters

False

True Propose save schedule

save schedule

Schedule

4. Confirmation save schedule

Fig. 2. Illustration of the schedule generation process Machine agents receive from their operation agent the jobs to be scheduled and the calculated chromosome length. Every machine agent searches for the best sequence of l jobs. The search for an optimal sequence is carried out by running GA at the local machine level for a number of predefined generations. Having reached the termination condition at the machine level, each machine agent delivers the best c chromosomes to the operation agent along with the average fitness of the current population. The results of the local search are delivered to the operation agent, which searches for the optimal allocation out of the gathered sequencing alternatives at the different machines. The size of the solution space amounts to cn. Since this number grows exponentially by increasing the number of the selected chromosomes or the number of machines considered for allocation, an exhaustive exploration of the entire set of solutions is not possible and GA is applied. Once the termination condition is met, the best allocations are selected to be validated in the next step. 3. Validation and evaluation Schedule alternatives combined by operation agents are the result of the local optimization at the individual machines. Therefore, the generation of a feasible solution from the operation perspective is not guaranteed. That is why the schedule alternatives generated at the operation level have to be checked for validity. The validity check applied at this stage concerns the inclusion of every job by at least one machine. If during the search no valid chromosome could be found, the operation agent works on diversifying the populations of a subset of the machine agents. Diversifying the population of the selected machine agents takes place by increasing the mutation rate and pursuing the execution of the local evolution process. This implies the investigation of unexplored areas of the solution space and regaining missing

jobs. Machines requested to pursue their exploration of the search space are those having the worst average fitness. The percentage of the selected machines is directly proportional to the percentage of the missing jobs. Having reached the termination condition, these machine agents deliver their best c chromosomes to their operation agent. The operation agent pursues its search for valid chromosomes as before. Upon generating valid chromosomes, operation agents take a decision on whether to stop the search process or to go on searching for better schedules. Continuing the search process takes place by requesting the entire set of the n machine agents to pursue their local optimization processes. This is performed by iterating back to the cooperative evolution step. As illustrated in Fig. 2, the evolution is pursued at the machine level without reinitializing a new population. The last population generated so far is rather used to generate the new offspring. This cooperative evolution continues until the termination condition at the operation level is satisfied. The termination condition can depend on the quality of the current generation. This is measured by comparing the quality of the best or the average fitness of the current population with the one from the previous valid population. In case no improvement is achieved, the process can terminate by considering the previous population. However, this may lead to a local optimum. Therefore, a predefined number of generations is considered by the termination condition to improve the probability of reaching a near-optimal solution. Having reached the termination condition of the cooperative evolution, operation agents have to select the best chromosome to be applied for scheduling. To accommodate for the involved perspectives and converge to a near-optimal schedule from the global point of view, candidate schedules are raised to the upper layers of the hierarchy for evaluation. First, job agents collect operation offers in the form of offered start and end times. Second, these offers are combined together to form schedule alternatives out of the job perspective. Third, the overall start and end times of the job in concern are calculated for every schedule alternative. These schedule alternatives are propagated to the associated order agent. Likewise, order agents collect the offered start and end times of their job agents and investigate possible schedule alternatives by combining job offers. Schedule alternatives at the order level are evaluated based on the makespan of the collection of jobs belonging to this order. The best c schedules alternatives are then considered and their constituents are evaluated based on voting. In other words, every occurrence of a job offer in the best c schedule alternatives is rewarded by increasing its vote. Eventually, these votes reflect the relative preferences of the order agent to the job offers. These votes are then used for evaluating the operation offers at the job level. Every job agent evaluates its schedule alternatives combined from the operation offers. The evaluation at the job level takes both the local evaluation criteria and the order votes into consideration. The local evaluation criteria include the estimated time to finish the job, or the job makespan, in addition to the total waiting time. Considering the waiting time leads to schedules with reduced work in progress. The incorporation of these criteria in the

evaluation is governed by predefined weights which are configured during the planning phase. Operation offers are accordingly voted for and propagated back to the operation agents. 4. Confirmation At this step, the votes are collected from all jobs involved in the schedule at the operation level. For every schedule alternative at the operation level, the votes from the different jobs are added up. The schedule alternative having the highest vote is selected. This schedule alternative is deemed to result in the best satisfaction of the evaluation criteria along the four levels of the hierarchy. Accordingly, the schedule is optimized from a global perspective. Confirming the schedule is carried out by sending a command to the involved machine agents informing them with the index of the selected chromosome in the proposed chromosomes. Similarly, the job agents are informed with their start and end times at the operation in concern. Job agents inform in their turn their order agents with the selected schedule. In this way, the schedule is saved locally from the different perspectives. This decentralization of schedule helps enhancing the reactivity while reacting to the environmental dynamics, as shown in (Badr and Göhner, 2009a). 4.2 GA-based evolution In this section, the previously mentioned cooperative evolution step is focused on. Details concerning the initialization, the chromosome representation and the fitness function are discussed. 1. Evolution initialization The evolution initialization is carried out at the operation level. The objective is to derive the number of machines to be considered for the allocation and the number of job lots to be allocated to each machine. This is calculated as follows. K=J*r n= N l= ceil (K/n) While (l < L and n>1) begin n= n-1 l= ceil (K/n) end

Fig. 3. Initialization algorithm First, the total number of job lots K is calculated by multiplying the number of jobs J by a predefined parallelization rate r. For instance, if the number of jobs amounts to 20 and the parallelization rate r is set to 2 the total number of job lots would be 40. The parallelization rate in this case corresponds to the simultaneous allocation of every job to two machines for the parallel execution of the operation in concern. Second, the number of machines n, to be considered for allocation, and the associated chromosome length l are

derived by the iterative procedure, listed in Fig. 3. Initially, the total number of machines N is considered for allocation. The chromosome length at each machine l is calculated by rounding up the result of dividing K by N. In case the resulting chromosome length is below a predefined minimum chromosome length L, the number of machines is decremented by 1 and the l is calculated again. This procedure iterates until l reaches L or the number of machines reaches 1. To illustrate, consider the above example and assume the availability of 20 machines. Allocating the 40 job lots to the 20 available machines would result in every machine having 2 lots. This implies a chromosome length of 2 at each machine which is relatively small and can hinder the application of standard genetic operators like the two-point crossover. In this example, if we suppose a minimum length of 4, only 10 machines would suffice to reach the required parallelization rate. However, a higher machine utilization can be attained by configuring the parallelization rate. It is worth noting that in case n is less than the total number of machines, the contacted machines are selected at random. This makes sense due to the assumption that all machines are initially unloaded. Therefore, it would make no difference to favour one machine on the other among the machines performing a certain operation. 2. Encoding Chromosomes at the machine level are represented as permutations of job identifiers. In Fig. 4, the encoding at both the machine and the operation levels is illustrated. The decoded schedules corresponding to the chromosomes with dashed contours are captured in the figure. At each machine agent, the initial population is generated based on the random selection of l job identifiers out of the total number of J given jobs. The only constraint in the encoding is that no duplicates are allowed in the job identifiers in a given chromosome. This is to avoid duplicate execution of the same job on the same machine for the same operation. Generating feasible chromosomes, whose encoding satisfies this constraint, is ensured based on a repair method. This repair method is invoked after the generation of every population to traverse the new offspring and to adjust the infeasible chromosomes by randomly selecting a non-occurring job identifier in place of the repeated identifiers. Operation agents

Op1

M1 2 3 M2 2

12 1 11 1

1 1

M3 3 2

3 1

t0 t1 t2 t3 t4 t5

Machine agents M1

M2

M3

2 3 1

132

3 2 1

3 2 1

213

1 2 3

2 3

1

2

1

3

3 2

Part family 1 Part family 2

1

t0 t1 t2 t3 t4 t0 t1 t2 t3 t4 t5 t0 t1 t2 t3 t4

Fig. 4. Illustrating chromosome encoding

Setup time

3. Fitness function In order to converge to a near-optimal solution from the global perspective, the fitness function accommodates for the optimization criteria of both the sequencing and the allocation sub-problems. The optimization criterion of the sequencing problem is the minimization of the setup time associated with a given sequence. This criterion is accounted for by the fitness function of the GA at the machine level as follows.

f m (c m ) = ∑ s ( j )

(1)

j

Where cm is a machine chromosome and s(j) is the setup time required for a job j in the chromosome cm. As in (1), the fitness of a machine chromosome is calculated by adding up the setup required for every job in the sequence of jobs encoded by this chromosome. This setup time is affected by the arrangement of jobs in the chromosome. As illustrated in Fig. 4, different sequences of the same set of jobs are associated with different setup times. This depends on whether succeeding jobs refer to the same part type and part family or not. The objective at the machine level is to minimize the function calculated in (1).

f o (co ) = w j ⋅ stdv ( Lo ) + wr ⋅ Max ( Fm (co )) Lo = {l j o | j ∈ J , l j o =

Q( j ) } n jo

(2) (3)

Where: wj, wr stdv(x) Lo Fm(Co) Q(j) njo

Predefined weights Standard deviation of set x Set of lot sizes on operation o Set of fitness values of the machine chromosomes constituting the operation chromosome Co Quantity of job j Number of machines allocated for job j on operation o

On the other hand, the fitness function at the operation level aims at optimizing the allocation of machines to the jobs with respect to the operation in concern. Due to its intermediary nature, the allocation of machines to jobs has to account for machine-related criteria and job-related criteria. Therefore, the fitness of operation chromosomes is calculated as a weighted sum of two terms, as in (2). The first term corresponds to the job-related criteria and measures the fairness of job allocation on the machines. This fairness is estimated based on the standard deviation of the lot sizes of all jobs allocated on the machines of a given operation. The

second term is an estimate of the machine utilization which is again measured in terms of the setup time. However, at the operation level, the maximum setup time incurred by all the machines is regarded as an indicator of the utilization of the allocated machines. The objective at the operation level is to minimize fo(co), calculated in (2). 5. APPLICATION AND EVALUATION RESULTS The proposed algorithm has been tested on data from an IBM test line. This test line represents a huge FMS consisting of thirty one testers grouped into four tester families. A total of ten different card types grouped into four families are to be tested on one or more testers according to their predetermined plan. The problem is characterized by major and minor setup times. Setup times are significant compared to execution time which is fixed and amounts to 10 sec. The objective is to minimize the makespan (Wittrock, 1990). The problem has been modelled by instantiating 31 agents representing the testers. The four tester families are modelled at the operation layers by 4 agents managing the corresponding tester agents. The quantities and part types for a typical manufacturing day, reported in (Wittrock, 1990) have been used to constitute sets of 50, 70, and 100 jobs. For test purposes, the whole set of jobs were considered to belong to the same group because of the lack of benchmarking data with respect to the makespan of job groups. Different test experiments with various parameters have been conducted. The parameters corresponding to the results depicted in Fig. 5 and Fig. 6 were selected as follows. The parallelization factor r is selected to be 2.5. The relative weights wj and wr are set to 0.25 and 0.75 respectively. The population size is set at the machine level to 10 and at the operation level to 50. The crossover and mutation rates are 0.8 and 0.02 respectively, at both levels. The two-point crossover is used at the operation level and the one-point crossover at the machine level. The mutation rate at machines, requested by their operation agents to diversify their local population, is adapted by increasing it with a value amounting to 25% of the old mutation rate. 600

Makespan (minutes)

The encoding of chromosomes at the operation level takes the form of indices to the offered machine chromosomes. As illustrated in Fig. 4, a chromosome at operation level with the value {1,2,1} encodes the schedule representing the combination of the first chromosome from M1, the second chromosome from M2 and the first chromosome from M3, in the c offered chromosomes. This encoding ensures a 1-1 mapping of chromosomes to feasible schedules. Therefore, no validity checks are required.

500 50 Jobs 400

70 Jobs 100 Jobs

300

200 1

3 5

7

9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 Number of hierarchical generations

Fig. 5. Illustration of the effect of changing the number of jobs on the resulting makespan Fig. 5 captures the effect of changing the data set on the resulting makespan. The makespan of optimizing the three different jobs is depicted along 50 hierarchical generations. A hierarchical generation refers to the execution of the local

evolution at the machine level followed by the execution of the local evolution at the operation level. Generations that resulted in invalid schedules were simply ignored and not counted. While the first dataset with 50 jobs resulted in an optimal schedule after 33 generations, the other two sets resulted in schedules with a deviation from the optimal makespan ranging from 7% to 13% in the span of the 50 generations. Evidently, increasing the solution space most probably leads to delaying the convergence. However, the computational complexity is low enough to allow for a larger number of generations. On average, one hierarchical generation was found to take around 0.3 seconds on a 2 Duo 2.0 GHz processor. 600

Makespan (minutes)

500

5 Generations 400

7 Generations 11 Generations

delegated to the operation agents, sequencing is assigned to the machine agents. This decomposition enables the incorporation of optimization in the existing agent-based architecture. Extending agents with optimization capabilities allows for the combination of flexibility with efficiency. The optimization is performed iteratively through the cooperation among the concerned agents to ensure the convergence to a near-optimal schedule from a global perspective. The different optimization objectives are encapsulated into the local views of agents and are thus taken into consideration. Test results show that this cooperative GA-based optimization results in globally optimized schedules in small computational times, which ensures the desired reactivity. Work is ongoing on benchmarking with other datasets with further optimization objectives. A concept for a more sophisticated termination condition that can be automated is under investigation. This concept should allow the agents to decide on the schedule to be applied. This decision should not be based on a predefined number of generations but rather on the relative efficiency of the schedule as well as on the realtime requirements.

300

REFERENCES 200 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 Number of hierarchical generations

Fig. 6. Illustration of the effect of varying the number of local generations on the resulting makespan Fig. 6 illustrates the results of an experiment in which the effect of increasing the number of generations of the local optimizations was investigated. Initially, increasing the number of local generations was found to expedite the convergence. With a local optimization of 5 generations a near-optimal schedule could be generated at the 27th generation. By raising the number of local generations to 7, a schedule with almost an optimal makespan could be generated at the 7th generation. However, with 11 generations, the performance goes down. This shows that cooperation among agents is necessary to attain a global optimization. 6. CONCLUSIONS Scheduling for FMS has been associated with the challenge of combining efficiency with flexibility. The agent-based paradigm has a proven potential in dealing with the complexity and dynamics of the problem. However, this paradigm has found very limited application in practice. The wide adoption of agents in FMS scheduling is hindered by the lack of support to schedule optimization in agent-based decentralized structures. This work contributes in overcoming this problem by the incorporation of genetic algorithms into an agent-based architecture, which has been proposed in a previous work. The scheduling problem is decomposed into two subproblems: allocation and sequencing. While allocation is

Badr, I. (2008). An agent-based scheduling framework for flexible manufacturing systems. International Journal of Computer, Information, and Systems Science, and Engineering (IJCISSE): 2 (2) Spring, 123-129. Badr, I. and Göhner, P. (2009a). An agent-based approach for automating the disturbance handling for flexible manufacturing systems. In Proceedings of the 14th IEEE International Conference on Emerging Technologies and Factory Automation, ETFA09, Mallorca, Spain. Badr, I. and Göhner, P. (2009b). An agent-Based Approach for scheduling under consideration of the influencing factors in FMS“. In Proceedings of the 35th Annual Conference of the IEEE Industrial Electronics Society, (IECON-09), Porto, Portugal. Cardon, A., Galinho, T., and Vacher, J. (2000). Genetic algorithms using multi-objectives in a multi-agent system. Robotics and Autonomous Systems, 33, 179190. Leitao, P. (2008). Agent-based distributed manufacturing control: A state-of-the-art survey. Engineering Applications of Artificial Intelligence, 22, 979-991. Lin, S., Goodman, E. D. and Punch, W. F. (1997). Investigating parallel genetic algorithms on job shop scheduling problems. In Proceedings of the 6th International Conference on Evolutionary Programming VI. 1213, 383 – 393. Shen, W. (2002a). Genetic algorithms in agent-based manufacturing scheduling systems. Integrated Computer-Aided Engineering, 9, 207-217. Shen, W. (2002b). Distributed Manufacturing Scheduling Using Intelligent Agents. IEEE Intelligent Systems, 17(1): 88-94. Wittrock, R. J. (1990). Scheduling parallel machines with major and minor setup times. The International Journal of Flexible Manufacturing Systems, 2: 329-341.