- Email: [email protected]
HYBRID SYSTEMS MODELING FOR GAS TRANSMISSION NETWORK
IFAC MCPL 2007 The 4th International Federation of Automatic Control Conference on Management and Control of Production and Logistics September 27-30, Sibiu - Romania
HYBRID SYSTEMS MODELING FOR GAS TRANSMISSION NETWORK ¹Amir Noori, ²Mohammad Bagher Menhaj, ³Masoud Shafiee Amirkabir University of technology, Electrical Engineering Departme, Iran-Tehran ¹[email protected], ²[email protected], ³[email protected] Abstract: Expansion of gas transmission networks and advancement in related automation systems has made these networks more complex. Study of these networks in an analytical approch, can help us in their design, management and optimization. We use a hybrid model based on hybrid automata to model the network. By using abstraction methods, a finite state automaton can be achieved which is used for intelligent control of the network. Due to lack of a decision model, Q-learning method is selected for decisionmaking purpose in which the proposed hybrid model makes our environment. Copyright © 2007 IFAC Keywords: Hybrid systems, Discrete Event Systems, Intelligent Control, Machine Learning
1. INTRODUCTION
restricted, for example in timed and rectangular hybrid systems or the discrete dynamics must be restricted, as is the case for o-minimal hybrid systems (Koutsoukos, et al., 2000). Thus, in largescale and hybrid systems, we often investigate a simpler model, which is derived from original system. In first method, we approximate the continuous behavior of system by a finite discrete behavior. So, we yield a unified model in discrete form. However, this approximation requires being safe, meaning that a controller must be guaranteed to provide same logical specifications for both the underlying concrete system and the abstract model. Besides the safety problem, we are interest in solving this problem in an optimal way. Moreover, this matter highlights the need of flexible techniques for approximation, which can provide different grades of approximation. By such approximator, that preserves many important behaviors of system in abstraction procedure, makes it possible doing analysis in a simpler level of complexity and extend results to the concrete system. However, in our design the abstracted model of the underlying hybrid system is a Discrete Event Systems (DES) and needs using DES approaches and its extensions. In the late 1980s, Ramadge and Wonham (Ramadge, et al., 1989) proposed the Supervisory Control Theory (SCT), which applies feedback theory to DESs that is modeled by automata. The supervisory control theory is a method for automatically synthesizing supervisors that
Large-scale systems such as Power Transmission Networks, Oil and Gas Transmission Networks and Transportation systems have critical rule in all downstream industries. These systems grow rapidly and their management and analysis need more attentions, especially, in a systematic way. Today, those efficient Analysis and control is the matter of new researches (Chapman, et al., 2002). In the Oil and Gas, we have different prominent features that make problem more complex. Large delays in response to either smooth or rapid change of variables and dependency to time and space make analysis difficult. Optimal operation of these networks widely considered in literatures (Alato, 2005; Osiadacz, et al.,1994; Carter, 1998; Kelling, 2000), that reduces losses and saves money. The main assumption of these works is that the system widely maintains its liveness and safe operation, which needs more considerations. However, Such Systems belong to hybrid systems and a general class of hybrid systems can be represented using hybrid automaton. Hybrid automaton are an extension of state automaton, which includes continuous and discrete dynamics. For complexity reason, abstraction methods usually utilize to reduce the underlying problem to a simpler one. These approaches make a system more abstract in a way that it preserves properties being analyzed while hiding details that are of no interest. In abstraction, either the continuous dynamics must be 389
restrict the behavior of a plant such that the given specifications are fulfilled as much as possible (for further reading, see (Ramadge, et al., 1989)). Today, this approach is common in discrete event system modeling and analysis and many researches are employing this framework. But, RW framework is based on automata and language theory (and symbolic computation), which is far from linear system theory principles. Moreover, for analysis of these types of problems, which often meet in lower level analysis of hybrid systems many approaches have been proposed (Cohen, et al., 1989; Murata, 1989; Harel, 1987; Ho, 1989). In linear system theory, we have many powerful tools for analysis of dynamical system, which can reuse in discrete event systems but most of mentioned approach does not have strong connection to classical system theory. In 1981, the researchers in INRIA started working on what we now know as Max-algebra; a system theoretic approach for discrete event systems. Although, this approach is well suited in some applications (Doustmohammadi, 1993; Kamen, 1993) but after twenty-six years, it is not as general as linear system theory. Recently, fuzzy discrete event systems (FDES) are introduced which convert symbolic computation of DES to a mathematical framework. In this paper, we present a hybrid model of gas transmission network. This network can model using hybrid automaton. Hybrid automaton is a suitable tool for modeling large-scale systems. Moreover, in contrast with petri-nets and its extensions, this framework brings decidability property, which in complex systems is an important subject. A problem is said to be decidable if there exists an algorithm to solve that problem and otherwise, is said to be undecidable. However, utilization of this feature needs (or perhaps, are easier) converting the underlying hybrid system to a unified discrete event model. Then, we can decide about important properties of this discrete event system and extend these results to the concrete one. Today, the analysis of complex automation systems becomes more important. However, dependencies between infrastructures such as power transmission network, oil and gas transmission network and telecommunication network make this issue prominent. In reality, we have some events that are not observable to operator, because of the lack of sensors or detection means, too high cost of detection, or difficulty in information transmission. Thus, some behavior of the underlying system is not observable to user. In this paper, we first model some important components of gas transmission network using hybrid automaton. This model for a part of Iranian Gas Transmission Network is also implemented in Simulink™ and Stateflow™ . Then, we present an abstraction method named Fuzzy abstraction which yields a variable structure automaton. Based on this model, we proposed a reinforcement learning method for tuning variable transition of a supervisory
controller. However, the membership degree of this variables are as our control variables. In a decision support system, which can be used in supervisory control systems, these transitions yield using if-then rules (and some other fusion rules). The major advantage of our model is its interactivity and reconfigurability, which makes it a suitable framework for diagnosing and management purpose. Also, based on this model, analysis and systematic design of control system is possible. The rest of the paper is organized as follows. In section 2, we present a brief introduction to Natural Gas Transmission Network, followed by section 3 that models this network based on hybrid automaton. In section 4, This network is then implemented in SIMULINK TM /Stateflow TM which, provides a suitable test bed for further researches. Section 5 is devoted to present control system design. Section 6 concludes the paper. 2. NATURAL GAS TRANSMISSION NETWORK Natural gas pipeline networks, or pipeline systems, are used to transport gas from sources to consumers over long distances. The distance from a source to a consumer may be thousands of kilometers. The gas in the pipeline is pressurized in order to maintain a pressure difference necessary for moving the gas. Compressors are used to pressurize the gas. These are needed at regular intervals along the pipelineusually every 50 to 150 kilometers, since the gas pressure decreases rapidly due to frictional losses. The most elementary components of a pipeline system are pipeline segments and compressors. Other components of pipeline system are valves (discrete or continuously operating) and gas storages. Longdistance transmission pipelines operate under high pressures and gas users at the off-takes use pressure reduction stations to adapt the gas pressure to their needs (Alato, 2005). However, in this paper we only investigate the following components: Pipeline Segments, Compressors (compressor Stations), Line Break Valves and other type of valves. Modeling of these components is done based on hybrid automaton. A major advantage of this modeling approach is its user friendliness that can be easily used and generalized for other relevant applications. In pipeline systems, compressor stations are the most complex parts of the model. Fig.1, illustrates connections between components in a compressor station. Compressor Connection Valve Line1
Pipeline
Line2
Line Break Valve Line3
Bypass Valve
Fig.1 Components of a compressor station In the following, we discuss modeling of network components. 390
Where k12 is a constant and z12 is the average compressibility of the gas in the segment. If the elevation of the pipeline segment is essential, then the following expression yields:
3. HYBRID MODELING OF COMPONENTS In this section, we provide a typical hybrid model for each component of gas transmission network. As mentioned above, we have pipeline segments, compressor stations and various types of valves as major components of gas transmission network. The following definition is based on (Johansson, et al., 1999; Lygeros, et al., 1999).
p12 − p 22 −
k12 g (h2 − h1 ) p122 = k12 z12 q12 q12 z12T12
0.8539
(2) Where h2 is the elevation of the end of the pipeline segment above some basic level h1 is the elevation of the beginning of the segment above some basic level T12 is the average gas temperature of the segment
Definition 1 (Hybrid Automaton). A hybrid automaton H is a 6-tuple H = (Q, X, X0 , f, D, R) , Where Q is a finite set of discrete variables; X is a finite set of continuous variables with
X 0 ⊆ Q × X is a set of initial states; f : Q × X → TX is a vector field; D ⊆ Q × X is the domain of H; R : Q × X → P (Q × X ) is a reset relation.
The average pressure p12 in the segment is calculated using the expression:
p12 =
We refer to (q, x ) ∈ Q × X as the state of H.
p p 2⎛ ⎜⎜ p1 + p 2 − 1 2 3⎝ p1 + p 2
⎞ ⎟⎟ ⎠
(3)
In the leak mode, we assume the pressure of nodes decreases exponentially. In the break mode, pressures assumed to be zero. a hybrid model of a pipe segment is shown in fig.2. Fig.3 shows implementation of a pipe segment in Stateflow TM .
The MathWorks modeling and simulation tools, Simulink™/Stateflow™ facilitate the design of complex control systems. In the following, we also describe modeling of these components in Simulink™ environment. 3.1 Pipeline Segments Pipelines are used to transport various kinds of mass, mainly crude oil and natural gas; however, different oil products are also transported. Gas pipelines are also used for nitrogen, oxygen, pressurized air and gases used by the petrochemical industry such as ethane, propane and others. In a typical segment of a pipeline, in addition to the mass flow rate, a pressure variable is defined at every node, which based on the analysis assumption (steady state or transient) and for each segment; one of these variables can be computed based on two others. Roughly speaking, in hybrid automaton, we have a discrete set (i.e., modes) in which continuous changes of variables take place. We will remain in a mode until the state of H belongs to its invariant set (i.e., domain). A transition between modes occurs by means of events and/or conditions. We may also consider a set/reset operaion in transitions. Typical discrete modes of a pipeline segment are Safe, Break and Leak. The Safe mode indicates the safe and desired operation of a segment. In this mode, the pressure of one end of the pipeline computes based on the flow rate and other end node pressure, or in a same way, the flow rate can be computed based on pressures. However, steady-state behavior of compressible mass in a pipe is described by this well-known equation (Alato, 2005):
p12 − p 22 = k12 z12 q12 q12
0.8539
Fig.2 Hybrid model of a pipe segment
Fig.3 A typical implementation of pipeline segment In this figure, some significant events are shown which may be observable or non-observable. For example, the pipeline segment goes to the break state because of the occurrence of over-pressure or break events. Continuous variables such as pressures and flow rates compute by invoking function Calc_Pressure_LineSeg_x_x that is carried out in a
(1)
391
subsystem in SIMULINK TM environment. In Fig.4, this subsystem is shown.
Fig.4 pressure calculation subsystem
Fig.5 hybrid model of Compressor
In transient condition, based on the identification methods and acquired data from simulation of network in Simone TM , we obtained a transfer function for pipe segments with an acceptable error, which is useful for optimization and control purpose. But, these linear models of components are very sensitive to component’s parameters and individually should be computed for each component of network. Therefore, in this paper, we will not discuss this subject.
As we can see in equation (4), the output pressure of compressor has a nonlinear algebraic relation with the input pressure. To represent such behavior for compressor, we use a subsystem that invokes through a function call in Stateflow TM . This subsystem is shown in Fig.6.
3.2 Compressors The active components of a pipeline system are compressors, which add potential energy to the gas flow by increasing the pressure. The adiabatic head Ha is the energy content increase (kJ/kg) of the gas when it flows through a compressor (Sandler, et al., 1987):
z RT Ha = s s γ Mw
⎡⎛ P ⎢⎜⎜ d ⎢⎣⎝ PS
γ ⎤ ⎞ ⎟⎟ − 1⎥ ⎥⎦ ⎠
Fig.6 implementation of compressor (by selction input for pressure or turn command)
(4)
3.3
Where Pd is the discharge (outlet) pressure of the
Discrete valves are used to block sections of pipelines and selecting paths for the gas to flow through it. In Fig.1, discrete valves are used to connect or disconnect pipes belonging to the same parallel pipeline system. Also, several valves at the compressor station are used to select which units are used for the compression and how to distribute the gas between the three outgoing parallel pipes. In fig.7 a hybrid model of a typical valve are shown. Major modes of a valve are: Open, Close and Fail. By means of these modes and corresponding events, we can express strings which are origin of an crisis in the nework.
compressor PS is the suction (inlet) pressure of the compressor
R is the universal gas constant z S is the compressibility at compressor suction conditions TS is the gas temperature at the suction point
M w is the molecular weight of the gas
γ
is defined as:
γ =
Valves
ka − 1 ka
Where ka is the molar specific heat ratio of the gas assuming adiabatic compression. In an adiabatic compression process no heat is transferred into or out from the compressor. In Fig.5, a hybrid automaton is shown which represent compressor behaviour. Fig.7 hybrid model of a valve Fig.8, Shows the stateflow block of a typical Valve, which implements these operating modes. 392
However, the fail state rarely uses but it is useful for diagnosing and management purpose.
Fig.9 A T-junction block In this figures, we have five mode for T-junction; Direct, End, Up, Down and MIX. Transition between these modes occurs with events which come from neighbour valves. Moreover, we can consider effect which take place in these points such as Route Preference.
Fig.8 implementaion of a Valve The other types of valves (such as Line Break Valves_LBVs,…) are similar to this figure and have such states. The important event of a typical LBV is pressure imbalance, which causes a transition from open state to close state. Moreover, by means of initial state, we can model two well-known types of valves : normally-open and normally-close.
5. INTELLIGENT CONTROL OF GAS TRANSMISSION NETWORK In this section, we will briefly present precedure for designing an intelligent control system based on the proposed model. Due to complex nature of large networks, analysis and controller design procedures is usually done in an abstracted level with fewer complexity. In other words, we convert a problem in uniform (rather discrete and less continuous) framework. Before, discussing these abstaction methods in details, we propose a structure for control system.
4. MODELING OF GAS TRANSMISSION NETWORK 4.1 Network modeling Modular Design of Network has many advantages. In other words, reduced design and modification time is main advantage of a modular approch. Based on the components modeled in previous section, we have modeled a part of Iranian Gas Transmission Network (IGTN). However, a complex system is defined as a system with many degree of freedom that coupled to each others. In network modeling, we must consider interaction between components. For example, “when content of a pipe segment is empty?” and other questions which should be considered. We have done these subjects by Routing Subsystems, which determines “flow in junction and branch points, status of pipe segments based on status of valves, compressors, and other pipe segments”, and all such situations. These subjects are essential when we diagnosis the network. We note that these interactions can guide us finding source of fault and help to diagnosis the network. Fig.9, shows a stateflow block that are used to represent a T-junction which are frequently found in gas transmission network.
5.1 Structure of Control System Complex control systems consist of many states, modes, parameters. Operator interacts with these systems in an abstract level (using Graphical User Interface) and only has control over a small subset of discrete dynamic of the concrete system. However, presenting large information to user makes decisionmaking more complex and time-consuming and reflexing little knowledge of system’s behaviours makes it impossible to take a good decision. Therefore, we extract different amount of information for two purposes, one for Automated Control System and another for designing a GUI. Fig.10 shows a typical abstraction of a complex control system. User Interface
.
(Abstract Discrete Behavior)
Abstracted model (Finite Disrete Behavior)
Plant
.
(Hybrid model)
Fig.10 different levels of a complex automation system 393
However, defining these levels depend on plant and desired operation features. In small-scale plants, the second level is unnecessary and user can simply interact with the concrete system. In Fig.10, it is obvious that the abstraction process reduces the knowledge and decision-making takes place under incomplete knowledge. However, in large-scale systems (such as gas transmission network) interactions between operator and the underlying process are very complex and needs a systematic approach for analysing the complex behaviours. In this paper, we concentrate on the level two and propose an approach for designing an automated control system (ACS). We can present this level by Discrete Event Systems (DESs) (or Fuzzy DES). Designing controller (which is named supervisor) for such systems widely considered in literatures and has many application. One way for designing an ACS is by using Expert systems and Decision Support Systems (DSSs). However, some authors proposed intelligent approach, which effectively handles uncertainties in knowledge and model (Nokhbeh, et al., 2006). This task is accomplished using fuzzy variables, which generally has gaussian membership function. However, implementations of such systems are difficult and are not scalable. In the following, we present a new approch, which are based on a hybrid model of network and deals with model and knowledge uncertainties in a computational framework.
Fig.11 Fuzzy abstraction of hybrid automaton Alternatively, we can formulate problem in different manner to solve it. The supervisor gets positive reinforcement for maintaining pressure near the desired value, negative reinforcement for large deviations from this value. Then, supervisor can learn the best strategy, to operate network by providing different setpoints for station and producers, and trying to maximize the reinforcement, it receives. This strategy is easy to implement and scalable also. This way of solving problem is know as RL (Sutton, et al., 1998). In the follownig, we discuss a learning method for tunning variables of supervisor automaton. 5.3 Reinforcement Learning Machine learning is study of algorithms, which improve performance with experience (Mitchell,1997). Such an algorithm is known as learner and procedure of improving performance with experience is known as training experience. RL is learning from interaction with environment. The learner and decision-maker is called the agent. The thing it interacts with, is called the environment. These interact continually, the agent selecting actions and the environment responding to those actions and presenting new situations (known as states) and reward associated with them to the agent, as shown in Fig.12. There are two flavors of RL problem based on the fact, that the agent is aware of the environmental model or not. These are known as model-based and model-free reinforcement learning. Q-learning is an example of model-free RL and pritorized sweeping is a model-based RL. In modelbased approaches, we learn a model and based on the estimated model make a decision. In model-free RL, no effort are used to estimate a model.
5.2 Fuzzy Abstraction of Hybrid model Reduction or abstraction techniques replace a complex system with a more abstract one, and partly soften the state explosion problem. There are two common method for this purpose; predicate abstraction which divide state space into several distinct regions, and projection which simply remove some less important variables. By considering uncertainties, which naturally exists in the concrete model, we extend first approach by dividing state-space into fuzzy subsets. The resulting model is a Fuzzy DES that can be represented in matrix form. In fig.11, State diagram of a fuzzy automaton is shown. Transistions in this diagram are carried out using a matrix which represent possibility of each transition. In supervisory control system, this variables are as our control variable. However, transitions of such systems are somewhat vague, and can be represented by a variable structure automaton. A variable structure automaton is an automaton that their transitions can tune. Thus, using these abstracted model, one can design a supervisor based on if-then rules. In other words, one way to solve this problem is to manually code rules for controling subsystems and their interactions. Nevertheless, as previously mentioned, this approach is complex to implement and not easy to scale.
Agent
State(Si)
Action(Ai)
Reward(Ri)
Environment Fig.12 RL model 394
a11 : Open_valve_11 a12 : Open_valve_13
Watkins' Q-learning, is a model-free method which is very easy to implement . Q-learning works by estimating the value of state-action pair, Q(s; a). The value Q(s; a), known as Q-value, is defined to be the expected sum of future reinforcement (penalty) obtained by taking action a from state s and following an optimal policy thereafter. Once these values have been learned, the optimal action from any state is the one with the lowest Q-value.The estimation of Q-values can be done on the basis of experience using following learning rule (Sutton, et al., 1998):
We also used a simple cost function for this purpose, which give all needed information about these two states :
Q( s, a ) = Q( s, a ) + α ( X + min (Q( s′, a′) − Q( s, a ))) a′∈A
(5) where, s0 is new state after taking action a on state s and X is reinforcement observed. X = PathCost , if transition is safe X = R , if transition is unsafe α is known as learning rate. A part of IGTN is shown in fig.13 which is used for explainig the implementation of intelligent control system.
CostFun = (press_1_1–500)^2 + (press_1_2– 500)^2 + (press_1_3–500)^2 + (Flwrate_1_1 – 120)^2 + (Flwrate_1_2–120)^2 + (Flwrate_1_3–120)^2 (6) Finally, after the training the supervisor, Action 3 in state1 and action1 in state4 is selected as best actions with minimum q-values. Fig.15, shows that the action with lowest Cost function, is selected more than others and fig.16 represent the convergence of Q-values. However, in this case, there are one Optimal Policy as mentioned above, First, Open_valve_7, then Open_valve_5.
Fig.15 Histogram of Selected Cost functions in Training period. Fig. 13 a part of a Iranian gas transmission network Based on the Q-learnig algorithm and defining a cost function of network states, we trained the supervisor to learn best action (see fig.14).
Fig. 16 Estimation of Q-value 6. CONCLUSION Simulation is our major tool for analysis of Largescale and Complex systems. In oil and gas transmission, commercial packages or toolboxes give little information about dynamic behavior of components and often are not interactive (for instance, we cannot simply simulate failure modes of network). However, SIMULINK TM /Stateflow TM is very powerful and rich for complex system analysis. Therefore, in first place, we developed a comprehensive network model, which can utilize for further researches. This model is also helpful for dispatchers, which should make a decision under incomplete knowledge. Moreover, based on this model, we derived an abstract model and applied reinforcement learning to tune the supervisor’s transitions degrees of the associated automaton.
Fig.14 Policies which exsit in above example Actions are listed below: a1 : Open_valve_2 a2 : Open_valve_3 a3 : Open_valve_7(bypass) a4 : Open_valve_7 a5 : Open_valve_9 a6 : Open_valve_11 a7 : Open_valve_5 a8 : Open_valve_7 a9 : Open_valve_11 a10 : Open_valve_5 395
7. REFERENCES
Conference on Decision and Control, Phoenix, AZ. H. J. Sandler and E. T. Luckiewicz, , (1987) “Practical Process Engineering”, McGraw-Hill, 638 pages. H. Nokhbeh, M. B. Menhaj, (2006) “fuzzy decision support system for crisis management in gas transmission network”, Proceedings of the 15th International Iranian Conference on Electrical Engineering. N.R. Jennings, P. Faratin, A.R. Lomuscio, S. Parsons, C. Sierra, M. Wooldridge, (2001) “Automated negotiation: Prospects, methods and challenges”, Internat. J. Group Decision Negotiation 10 (2) 199–215. T. M. Mitchell, (1997) “Machine Learning”. McGraw Hill. R. S. Sutton and A. G. Barto, (1998) “Reinforcement Learning: An Introduction”. London, England: The MIT Press. F. Lin, H. Ying, (2002) “Modeling and Control of Fuzzy Discrete Event Systems” IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 32, NO. 4 C.G. Cassandras and S. Lafortune, (1999) “Introduction to Discrete Event Systems”, Kluwer Academic Publishers. “MATLAB&SIMULINK user’s guide”, The MathWorks, Inc (2005). A. Noori, M. B. Menhaj, (2007) “Fuzzy abstraction of hybrid systems” , will appear in : Proceedings of International conference on application of Computational Intelligence for Measurement Systems and Applications.
K. S. Chapman, M. Abbaspour, (june,2002) “Virtual Pipeline System Testbed to Optimize the U.S. Natural Gas Transmission Pipeline System”, Technology Status Assessment Report, The Department of Energy, Strategic Center for Natural Gas, Kansas State University. H. Alato, (2005) “Real-time Receding Horizon Optimisation of Gas Pipeline Networks” PHD dissertation, Helsinki University of Technology. available on-line : http://lib.tkk.fi/Diss/2005/isbn9512276593 J. Osiadacz, S. Swierczwski, (1994) “Optimal Control of Gas Transportation Systems”. The 3rd IEEE Conference on Control Applications, Yhe University of Strathclyde, Glasgow. R. G. Carter, “Pipeline Optimisation: (1998) “Dynamic Programming after 30 years”, Pipeline Simulation Interest Group, Annual Conference ,Denver, Colorado, USA,www.psig.org/papers Kelling C., K. Reith and E. Sekirnjak (2000), “A Practical Approach to Transient Optimisation for Gas Networks”, Pipeline Simulation Interest Group, Annual Conference , Savannah, Georgia, USA, www.psig.org/papers X. D. Koutsoukos, P. J. Antsaklis, J. A. Stiver, AND M. D. Lemmon, (2000) “Supervisory Control of Hybrid Systems”, Proceedings of the IEEE, VOL. 88, NO. 7 P.J.G. Ramadge andW.M.Wonham. (1989) “The control of discrete event systems”. IEEE Proceedings: Special issue on Discrete Event Systems, 77(1):81–97,. G. Cohen, P. Moller, J.-P. Quadrat and M. Viot, (1989) “Algebraic Tools for the Performance Evaluation of Discrete Event Systems”, Proc. IEEE, Vol 77, No 1, T. Murata, (1989) “Petri nets: Properties, analysis and applications”, Proceedings of the IEEE. Vol 77 No. 4. D. Harel, (1987) “Statecharts: A Visual Formalism for Complex Systems”, Science of Computer Programming, vol. 8. Y.-C. Ho, (1989) “Special Issue on the Dynamics of Discrete Event Systems”, Proceedings of the IEEE. Doustmohammadi, (1993) “Modeling of Discrete Event Dynamic Systems based on Petri Nets and Minimax Algebra Approach”, PhD thesis proposal, Georgia Inst. Tech., Atlanta. E. W. Kamen, (1993) “An Equation-Based Approach to the Control of Discrete Even Systems with Applications to Manufacturing”, Proc. Int’l Conf. on Control Theory & its Applications, Jerusalem, Israel. K. H. Johansson, M. Egerstedt, J. Lygeros, and S. Sastry. (1999)”On the regularization of Zeno hybrid automata.” Systems & Control Letters, 38:141-150. J. Lygeros, K. H. Johansson, S. Sastry, and M. Egerstedt. (1999) “On the existence of executions of hybrid automata.” In IEEE 396
HYBRID SYSTEMS MODELING FOR GAS TRANSMISSION NETWORK ¹Amir Noori, ²Mohammad Bagher Menhaj, ³Masoud Shafiee Amirkabir University of technology, Electrical Engineering Departme, Iran-Tehran ¹[email protected], ²[email protected], ³[email protected] Abstract: Expansion of gas transmission networks and advancement in related automation systems has made these networks more complex. Study of these networks in an analytical approch, can help us in their design, management and optimization. We use a hybrid model based on hybrid automata to model the network. By using abstraction methods, a finite state automaton can be achieved which is used for intelligent control of the network. Due to lack of a decision model, Q-learning method is selected for decisionmaking purpose in which the proposed hybrid model makes our environment. Copyright © 2007 IFAC Keywords: Hybrid systems, Discrete Event Systems, Intelligent Control, Machine Learning
1. INTRODUCTION
restricted, for example in timed and rectangular hybrid systems or the discrete dynamics must be restricted, as is the case for o-minimal hybrid systems (Koutsoukos, et al., 2000). Thus, in largescale and hybrid systems, we often investigate a simpler model, which is derived from original system. In first method, we approximate the continuous behavior of system by a finite discrete behavior. So, we yield a unified model in discrete form. However, this approximation requires being safe, meaning that a controller must be guaranteed to provide same logical specifications for both the underlying concrete system and the abstract model. Besides the safety problem, we are interest in solving this problem in an optimal way. Moreover, this matter highlights the need of flexible techniques for approximation, which can provide different grades of approximation. By such approximator, that preserves many important behaviors of system in abstraction procedure, makes it possible doing analysis in a simpler level of complexity and extend results to the concrete system. However, in our design the abstracted model of the underlying hybrid system is a Discrete Event Systems (DES) and needs using DES approaches and its extensions. In the late 1980s, Ramadge and Wonham (Ramadge, et al., 1989) proposed the Supervisory Control Theory (SCT), which applies feedback theory to DESs that is modeled by automata. The supervisory control theory is a method for automatically synthesizing supervisors that
Large-scale systems such as Power Transmission Networks, Oil and Gas Transmission Networks and Transportation systems have critical rule in all downstream industries. These systems grow rapidly and their management and analysis need more attentions, especially, in a systematic way. Today, those efficient Analysis and control is the matter of new researches (Chapman, et al., 2002). In the Oil and Gas, we have different prominent features that make problem more complex. Large delays in response to either smooth or rapid change of variables and dependency to time and space make analysis difficult. Optimal operation of these networks widely considered in literatures (Alato, 2005; Osiadacz, et al.,1994; Carter, 1998; Kelling, 2000), that reduces losses and saves money. The main assumption of these works is that the system widely maintains its liveness and safe operation, which needs more considerations. However, Such Systems belong to hybrid systems and a general class of hybrid systems can be represented using hybrid automaton. Hybrid automaton are an extension of state automaton, which includes continuous and discrete dynamics. For complexity reason, abstraction methods usually utilize to reduce the underlying problem to a simpler one. These approaches make a system more abstract in a way that it preserves properties being analyzed while hiding details that are of no interest. In abstraction, either the continuous dynamics must be 389
restrict the behavior of a plant such that the given specifications are fulfilled as much as possible (for further reading, see (Ramadge, et al., 1989)). Today, this approach is common in discrete event system modeling and analysis and many researches are employing this framework. But, RW framework is based on automata and language theory (and symbolic computation), which is far from linear system theory principles. Moreover, for analysis of these types of problems, which often meet in lower level analysis of hybrid systems many approaches have been proposed (Cohen, et al., 1989; Murata, 1989; Harel, 1987; Ho, 1989). In linear system theory, we have many powerful tools for analysis of dynamical system, which can reuse in discrete event systems but most of mentioned approach does not have strong connection to classical system theory. In 1981, the researchers in INRIA started working on what we now know as Max-algebra; a system theoretic approach for discrete event systems. Although, this approach is well suited in some applications (Doustmohammadi, 1993; Kamen, 1993) but after twenty-six years, it is not as general as linear system theory. Recently, fuzzy discrete event systems (FDES) are introduced which convert symbolic computation of DES to a mathematical framework. In this paper, we present a hybrid model of gas transmission network. This network can model using hybrid automaton. Hybrid automaton is a suitable tool for modeling large-scale systems. Moreover, in contrast with petri-nets and its extensions, this framework brings decidability property, which in complex systems is an important subject. A problem is said to be decidable if there exists an algorithm to solve that problem and otherwise, is said to be undecidable. However, utilization of this feature needs (or perhaps, are easier) converting the underlying hybrid system to a unified discrete event model. Then, we can decide about important properties of this discrete event system and extend these results to the concrete one. Today, the analysis of complex automation systems becomes more important. However, dependencies between infrastructures such as power transmission network, oil and gas transmission network and telecommunication network make this issue prominent. In reality, we have some events that are not observable to operator, because of the lack of sensors or detection means, too high cost of detection, or difficulty in information transmission. Thus, some behavior of the underlying system is not observable to user. In this paper, we first model some important components of gas transmission network using hybrid automaton. This model for a part of Iranian Gas Transmission Network is also implemented in Simulink™ and Stateflow™ . Then, we present an abstraction method named Fuzzy abstraction which yields a variable structure automaton. Based on this model, we proposed a reinforcement learning method for tuning variable transition of a supervisory
controller. However, the membership degree of this variables are as our control variables. In a decision support system, which can be used in supervisory control systems, these transitions yield using if-then rules (and some other fusion rules). The major advantage of our model is its interactivity and reconfigurability, which makes it a suitable framework for diagnosing and management purpose. Also, based on this model, analysis and systematic design of control system is possible. The rest of the paper is organized as follows. In section 2, we present a brief introduction to Natural Gas Transmission Network, followed by section 3 that models this network based on hybrid automaton. In section 4, This network is then implemented in SIMULINK TM /Stateflow TM which, provides a suitable test bed for further researches. Section 5 is devoted to present control system design. Section 6 concludes the paper. 2. NATURAL GAS TRANSMISSION NETWORK Natural gas pipeline networks, or pipeline systems, are used to transport gas from sources to consumers over long distances. The distance from a source to a consumer may be thousands of kilometers. The gas in the pipeline is pressurized in order to maintain a pressure difference necessary for moving the gas. Compressors are used to pressurize the gas. These are needed at regular intervals along the pipelineusually every 50 to 150 kilometers, since the gas pressure decreases rapidly due to frictional losses. The most elementary components of a pipeline system are pipeline segments and compressors. Other components of pipeline system are valves (discrete or continuously operating) and gas storages. Longdistance transmission pipelines operate under high pressures and gas users at the off-takes use pressure reduction stations to adapt the gas pressure to their needs (Alato, 2005). However, in this paper we only investigate the following components: Pipeline Segments, Compressors (compressor Stations), Line Break Valves and other type of valves. Modeling of these components is done based on hybrid automaton. A major advantage of this modeling approach is its user friendliness that can be easily used and generalized for other relevant applications. In pipeline systems, compressor stations are the most complex parts of the model. Fig.1, illustrates connections between components in a compressor station. Compressor Connection Valve Line1
Pipeline
Line2
Line Break Valve Line3
Bypass Valve
Fig.1 Components of a compressor station In the following, we discuss modeling of network components. 390
Where k12 is a constant and z12 is the average compressibility of the gas in the segment. If the elevation of the pipeline segment is essential, then the following expression yields:
3. HYBRID MODELING OF COMPONENTS In this section, we provide a typical hybrid model for each component of gas transmission network. As mentioned above, we have pipeline segments, compressor stations and various types of valves as major components of gas transmission network. The following definition is based on (Johansson, et al., 1999; Lygeros, et al., 1999).
p12 − p 22 −
k12 g (h2 − h1 ) p122 = k12 z12 q12 q12 z12T12
0.8539
(2) Where h2 is the elevation of the end of the pipeline segment above some basic level h1 is the elevation of the beginning of the segment above some basic level T12 is the average gas temperature of the segment
Definition 1 (Hybrid Automaton). A hybrid automaton H is a 6-tuple H = (Q, X, X0 , f, D, R) , Where Q is a finite set of discrete variables; X is a finite set of continuous variables with
X 0 ⊆ Q × X is a set of initial states; f : Q × X → TX is a vector field; D ⊆ Q × X is the domain of H; R : Q × X → P (Q × X ) is a reset relation.
The average pressure p12 in the segment is calculated using the expression:
p12 =
We refer to (q, x ) ∈ Q × X as the state of H.
p p 2⎛ ⎜⎜ p1 + p 2 − 1 2 3⎝ p1 + p 2
⎞ ⎟⎟ ⎠
(3)
In the leak mode, we assume the pressure of nodes decreases exponentially. In the break mode, pressures assumed to be zero. a hybrid model of a pipe segment is shown in fig.2. Fig.3 shows implementation of a pipe segment in Stateflow TM .
The MathWorks modeling and simulation tools, Simulink™/Stateflow™ facilitate the design of complex control systems. In the following, we also describe modeling of these components in Simulink™ environment. 3.1 Pipeline Segments Pipelines are used to transport various kinds of mass, mainly crude oil and natural gas; however, different oil products are also transported. Gas pipelines are also used for nitrogen, oxygen, pressurized air and gases used by the petrochemical industry such as ethane, propane and others. In a typical segment of a pipeline, in addition to the mass flow rate, a pressure variable is defined at every node, which based on the analysis assumption (steady state or transient) and for each segment; one of these variables can be computed based on two others. Roughly speaking, in hybrid automaton, we have a discrete set (i.e., modes) in which continuous changes of variables take place. We will remain in a mode until the state of H belongs to its invariant set (i.e., domain). A transition between modes occurs by means of events and/or conditions. We may also consider a set/reset operaion in transitions. Typical discrete modes of a pipeline segment are Safe, Break and Leak. The Safe mode indicates the safe and desired operation of a segment. In this mode, the pressure of one end of the pipeline computes based on the flow rate and other end node pressure, or in a same way, the flow rate can be computed based on pressures. However, steady-state behavior of compressible mass in a pipe is described by this well-known equation (Alato, 2005):
p12 − p 22 = k12 z12 q12 q12
0.8539
Fig.2 Hybrid model of a pipe segment
Fig.3 A typical implementation of pipeline segment In this figure, some significant events are shown which may be observable or non-observable. For example, the pipeline segment goes to the break state because of the occurrence of over-pressure or break events. Continuous variables such as pressures and flow rates compute by invoking function Calc_Pressure_LineSeg_x_x that is carried out in a
(1)
391
subsystem in SIMULINK TM environment. In Fig.4, this subsystem is shown.
Fig.4 pressure calculation subsystem
Fig.5 hybrid model of Compressor
In transient condition, based on the identification methods and acquired data from simulation of network in Simone TM , we obtained a transfer function for pipe segments with an acceptable error, which is useful for optimization and control purpose. But, these linear models of components are very sensitive to component’s parameters and individually should be computed for each component of network. Therefore, in this paper, we will not discuss this subject.
As we can see in equation (4), the output pressure of compressor has a nonlinear algebraic relation with the input pressure. To represent such behavior for compressor, we use a subsystem that invokes through a function call in Stateflow TM . This subsystem is shown in Fig.6.
3.2 Compressors The active components of a pipeline system are compressors, which add potential energy to the gas flow by increasing the pressure. The adiabatic head Ha is the energy content increase (kJ/kg) of the gas when it flows through a compressor (Sandler, et al., 1987):
z RT Ha = s s γ Mw
⎡⎛ P ⎢⎜⎜ d ⎢⎣⎝ PS
γ ⎤ ⎞ ⎟⎟ − 1⎥ ⎥⎦ ⎠
Fig.6 implementation of compressor (by selction input for pressure or turn command)
(4)
3.3
Where Pd is the discharge (outlet) pressure of the
Discrete valves are used to block sections of pipelines and selecting paths for the gas to flow through it. In Fig.1, discrete valves are used to connect or disconnect pipes belonging to the same parallel pipeline system. Also, several valves at the compressor station are used to select which units are used for the compression and how to distribute the gas between the three outgoing parallel pipes. In fig.7 a hybrid model of a typical valve are shown. Major modes of a valve are: Open, Close and Fail. By means of these modes and corresponding events, we can express strings which are origin of an crisis in the nework.
compressor PS is the suction (inlet) pressure of the compressor
R is the universal gas constant z S is the compressibility at compressor suction conditions TS is the gas temperature at the suction point
M w is the molecular weight of the gas
γ
is defined as:
γ =
Valves
ka − 1 ka
Where ka is the molar specific heat ratio of the gas assuming adiabatic compression. In an adiabatic compression process no heat is transferred into or out from the compressor. In Fig.5, a hybrid automaton is shown which represent compressor behaviour. Fig.7 hybrid model of a valve Fig.8, Shows the stateflow block of a typical Valve, which implements these operating modes. 392
However, the fail state rarely uses but it is useful for diagnosing and management purpose.
Fig.9 A T-junction block In this figures, we have five mode for T-junction; Direct, End, Up, Down and MIX. Transition between these modes occurs with events which come from neighbour valves. Moreover, we can consider effect which take place in these points such as Route Preference.
Fig.8 implementaion of a Valve The other types of valves (such as Line Break Valves_LBVs,…) are similar to this figure and have such states. The important event of a typical LBV is pressure imbalance, which causes a transition from open state to close state. Moreover, by means of initial state, we can model two well-known types of valves : normally-open and normally-close.
5. INTELLIGENT CONTROL OF GAS TRANSMISSION NETWORK In this section, we will briefly present precedure for designing an intelligent control system based on the proposed model. Due to complex nature of large networks, analysis and controller design procedures is usually done in an abstracted level with fewer complexity. In other words, we convert a problem in uniform (rather discrete and less continuous) framework. Before, discussing these abstaction methods in details, we propose a structure for control system.
4. MODELING OF GAS TRANSMISSION NETWORK 4.1 Network modeling Modular Design of Network has many advantages. In other words, reduced design and modification time is main advantage of a modular approch. Based on the components modeled in previous section, we have modeled a part of Iranian Gas Transmission Network (IGTN). However, a complex system is defined as a system with many degree of freedom that coupled to each others. In network modeling, we must consider interaction between components. For example, “when content of a pipe segment is empty?” and other questions which should be considered. We have done these subjects by Routing Subsystems, which determines “flow in junction and branch points, status of pipe segments based on status of valves, compressors, and other pipe segments”, and all such situations. These subjects are essential when we diagnosis the network. We note that these interactions can guide us finding source of fault and help to diagnosis the network. Fig.9, shows a stateflow block that are used to represent a T-junction which are frequently found in gas transmission network.
5.1 Structure of Control System Complex control systems consist of many states, modes, parameters. Operator interacts with these systems in an abstract level (using Graphical User Interface) and only has control over a small subset of discrete dynamic of the concrete system. However, presenting large information to user makes decisionmaking more complex and time-consuming and reflexing little knowledge of system’s behaviours makes it impossible to take a good decision. Therefore, we extract different amount of information for two purposes, one for Automated Control System and another for designing a GUI. Fig.10 shows a typical abstraction of a complex control system. User Interface
.
(Abstract Discrete Behavior)
Abstracted model (Finite Disrete Behavior)
Plant
.
(Hybrid model)
Fig.10 different levels of a complex automation system 393
However, defining these levels depend on plant and desired operation features. In small-scale plants, the second level is unnecessary and user can simply interact with the concrete system. In Fig.10, it is obvious that the abstraction process reduces the knowledge and decision-making takes place under incomplete knowledge. However, in large-scale systems (such as gas transmission network) interactions between operator and the underlying process are very complex and needs a systematic approach for analysing the complex behaviours. In this paper, we concentrate on the level two and propose an approach for designing an automated control system (ACS). We can present this level by Discrete Event Systems (DESs) (or Fuzzy DES). Designing controller (which is named supervisor) for such systems widely considered in literatures and has many application. One way for designing an ACS is by using Expert systems and Decision Support Systems (DSSs). However, some authors proposed intelligent approach, which effectively handles uncertainties in knowledge and model (Nokhbeh, et al., 2006). This task is accomplished using fuzzy variables, which generally has gaussian membership function. However, implementations of such systems are difficult and are not scalable. In the following, we present a new approch, which are based on a hybrid model of network and deals with model and knowledge uncertainties in a computational framework.
Fig.11 Fuzzy abstraction of hybrid automaton Alternatively, we can formulate problem in different manner to solve it. The supervisor gets positive reinforcement for maintaining pressure near the desired value, negative reinforcement for large deviations from this value. Then, supervisor can learn the best strategy, to operate network by providing different setpoints for station and producers, and trying to maximize the reinforcement, it receives. This strategy is easy to implement and scalable also. This way of solving problem is know as RL (Sutton, et al., 1998). In the follownig, we discuss a learning method for tunning variables of supervisor automaton. 5.3 Reinforcement Learning Machine learning is study of algorithms, which improve performance with experience (Mitchell,1997). Such an algorithm is known as learner and procedure of improving performance with experience is known as training experience. RL is learning from interaction with environment. The learner and decision-maker is called the agent. The thing it interacts with, is called the environment. These interact continually, the agent selecting actions and the environment responding to those actions and presenting new situations (known as states) and reward associated with them to the agent, as shown in Fig.12. There are two flavors of RL problem based on the fact, that the agent is aware of the environmental model or not. These are known as model-based and model-free reinforcement learning. Q-learning is an example of model-free RL and pritorized sweeping is a model-based RL. In modelbased approaches, we learn a model and based on the estimated model make a decision. In model-free RL, no effort are used to estimate a model.
5.2 Fuzzy Abstraction of Hybrid model Reduction or abstraction techniques replace a complex system with a more abstract one, and partly soften the state explosion problem. There are two common method for this purpose; predicate abstraction which divide state space into several distinct regions, and projection which simply remove some less important variables. By considering uncertainties, which naturally exists in the concrete model, we extend first approach by dividing state-space into fuzzy subsets. The resulting model is a Fuzzy DES that can be represented in matrix form. In fig.11, State diagram of a fuzzy automaton is shown. Transistions in this diagram are carried out using a matrix which represent possibility of each transition. In supervisory control system, this variables are as our control variable. However, transitions of such systems are somewhat vague, and can be represented by a variable structure automaton. A variable structure automaton is an automaton that their transitions can tune. Thus, using these abstracted model, one can design a supervisor based on if-then rules. In other words, one way to solve this problem is to manually code rules for controling subsystems and their interactions. Nevertheless, as previously mentioned, this approach is complex to implement and not easy to scale.
Agent
State(Si)
Action(Ai)
Reward(Ri)
Environment Fig.12 RL model 394
a11 : Open_valve_11 a12 : Open_valve_13
Watkins' Q-learning, is a model-free method which is very easy to implement . Q-learning works by estimating the value of state-action pair, Q(s; a). The value Q(s; a), known as Q-value, is defined to be the expected sum of future reinforcement (penalty) obtained by taking action a from state s and following an optimal policy thereafter. Once these values have been learned, the optimal action from any state is the one with the lowest Q-value.The estimation of Q-values can be done on the basis of experience using following learning rule (Sutton, et al., 1998):
We also used a simple cost function for this purpose, which give all needed information about these two states :
Q( s, a ) = Q( s, a ) + α ( X + min (Q( s′, a′) − Q( s, a ))) a′∈A
(5) where, s0 is new state after taking action a on state s and X is reinforcement observed. X = PathCost , if transition is safe X = R , if transition is unsafe α is known as learning rate. A part of IGTN is shown in fig.13 which is used for explainig the implementation of intelligent control system.
CostFun = (press_1_1–500)^2 + (press_1_2– 500)^2 + (press_1_3–500)^2 + (Flwrate_1_1 – 120)^2 + (Flwrate_1_2–120)^2 + (Flwrate_1_3–120)^2 (6) Finally, after the training the supervisor, Action 3 in state1 and action1 in state4 is selected as best actions with minimum q-values. Fig.15, shows that the action with lowest Cost function, is selected more than others and fig.16 represent the convergence of Q-values. However, in this case, there are one Optimal Policy as mentioned above, First, Open_valve_7, then Open_valve_5.
Fig.15 Histogram of Selected Cost functions in Training period. Fig. 13 a part of a Iranian gas transmission network Based on the Q-learnig algorithm and defining a cost function of network states, we trained the supervisor to learn best action (see fig.14).
Fig. 16 Estimation of Q-value 6. CONCLUSION Simulation is our major tool for analysis of Largescale and Complex systems. In oil and gas transmission, commercial packages or toolboxes give little information about dynamic behavior of components and often are not interactive (for instance, we cannot simply simulate failure modes of network). However, SIMULINK TM /Stateflow TM is very powerful and rich for complex system analysis. Therefore, in first place, we developed a comprehensive network model, which can utilize for further researches. This model is also helpful for dispatchers, which should make a decision under incomplete knowledge. Moreover, based on this model, we derived an abstract model and applied reinforcement learning to tune the supervisor’s transitions degrees of the associated automaton.
Fig.14 Policies which exsit in above example Actions are listed below: a1 : Open_valve_2 a2 : Open_valve_3 a3 : Open_valve_7(bypass) a4 : Open_valve_7 a5 : Open_valve_9 a6 : Open_valve_11 a7 : Open_valve_5 a8 : Open_valve_7 a9 : Open_valve_11 a10 : Open_valve_5 395
7. REFERENCES
Conference on Decision and Control, Phoenix, AZ. H. J. Sandler and E. T. Luckiewicz, , (1987) “Practical Process Engineering”, McGraw-Hill, 638 pages. H. Nokhbeh, M. B. Menhaj, (2006) “fuzzy decision support system for crisis management in gas transmission network”, Proceedings of the 15th International Iranian Conference on Electrical Engineering. N.R. Jennings, P. Faratin, A.R. Lomuscio, S. Parsons, C. Sierra, M. Wooldridge, (2001) “Automated negotiation: Prospects, methods and challenges”, Internat. J. Group Decision Negotiation 10 (2) 199–215. T. M. Mitchell, (1997) “Machine Learning”. McGraw Hill. R. S. Sutton and A. G. Barto, (1998) “Reinforcement Learning: An Introduction”. London, England: The MIT Press. F. Lin, H. Ying, (2002) “Modeling and Control of Fuzzy Discrete Event Systems” IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 32, NO. 4 C.G. Cassandras and S. Lafortune, (1999) “Introduction to Discrete Event Systems”, Kluwer Academic Publishers. “MATLAB&SIMULINK user’s guide”, The MathWorks, Inc (2005). A. Noori, M. B. Menhaj, (2007) “Fuzzy abstraction of hybrid systems” , will appear in : Proceedings of International conference on application of Computational Intelligence for Measurement Systems and Applications.
K. S. Chapman, M. Abbaspour, (june,2002) “Virtual Pipeline System Testbed to Optimize the U.S. Natural Gas Transmission Pipeline System”, Technology Status Assessment Report, The Department of Energy, Strategic Center for Natural Gas, Kansas State University. H. Alato, (2005) “Real-time Receding Horizon Optimisation of Gas Pipeline Networks” PHD dissertation, Helsinki University of Technology. available on-line : http://lib.tkk.fi/Diss/2005/isbn9512276593 J. Osiadacz, S. Swierczwski, (1994) “Optimal Control of Gas Transportation Systems”. The 3rd IEEE Conference on Control Applications, Yhe University of Strathclyde, Glasgow. R. G. Carter, “Pipeline Optimisation: (1998) “Dynamic Programming after 30 years”, Pipeline Simulation Interest Group, Annual Conference ,Denver, Colorado, USA,www.psig.org/papers Kelling C., K. Reith and E. Sekirnjak (2000), “A Practical Approach to Transient Optimisation for Gas Networks”, Pipeline Simulation Interest Group, Annual Conference , Savannah, Georgia, USA, www.psig.org/papers X. D. Koutsoukos, P. J. Antsaklis, J. A. Stiver, AND M. D. Lemmon, (2000) “Supervisory Control of Hybrid Systems”, Proceedings of the IEEE, VOL. 88, NO. 7 P.J.G. Ramadge andW.M.Wonham. (1989) “The control of discrete event systems”. IEEE Proceedings: Special issue on Discrete Event Systems, 77(1):81–97,. G. Cohen, P. Moller, J.-P. Quadrat and M. Viot, (1989) “Algebraic Tools for the Performance Evaluation of Discrete Event Systems”, Proc. IEEE, Vol 77, No 1, T. Murata, (1989) “Petri nets: Properties, analysis and applications”, Proceedings of the IEEE. Vol 77 No. 4. D. Harel, (1987) “Statecharts: A Visual Formalism for Complex Systems”, Science of Computer Programming, vol. 8. Y.-C. Ho, (1989) “Special Issue on the Dynamics of Discrete Event Systems”, Proceedings of the IEEE. Doustmohammadi, (1993) “Modeling of Discrete Event Dynamic Systems based on Petri Nets and Minimax Algebra Approach”, PhD thesis proposal, Georgia Inst. Tech., Atlanta. E. W. Kamen, (1993) “An Equation-Based Approach to the Control of Discrete Even Systems with Applications to Manufacturing”, Proc. Int’l Conf. on Control Theory & its Applications, Jerusalem, Israel. K. H. Johansson, M. Egerstedt, J. Lygeros, and S. Sastry. (1999)”On the regularization of Zeno hybrid automata.” Systems & Control Letters, 38:141-150. J. Lygeros, K. H. Johansson, S. Sastry, and M. Egerstedt. (1999) “On the existence of executions of hybrid automata.” In IEEE 396