- Email: [email protected]
It takes knowledge to apply neural networks for control
ISA Transactions 32 (1993) 235-240 Elsevier
235
It takes knowledge to apply neural networks for control Malcolm C. Beaverstock Automation Technology International, Mobile, AL 36693, USA
Until very recently, process control applications of neural networks have been limited to theoretical studies and fairly small experimental projects on somewhat simple processes. Early problems identified with using these systems in an on-line environment included appropriate handling of raw process data, extending the knowledge base of the network to include more broad operating conditions, transforming the network results into useful controller setpoints, and providing for graceful degradation of the controller functions. This paper describes a practical approach for implementing artificially intelligent process control functions based on a unique combination of rule-based expert systems and neural network technology. The system has been successfully applied to a complex pulp and paper process and new applications are currently under development for other industries.
Introduction
Artificial intelligence technology promised advances in control applications by capturing the power of the human brain. The successful application of this technology requires an understanding of the type of knowledge needed to make the system work in a real-world environment. Expert systems utilize a person's insight of process operation to create a model that interprets operating conditions and takes control action. Knowledge of discrete, causal relationships, based on experienced observations, is required from operators and others working with process equipment. That knowledge is imbedded in the human mind and has to be extracted and formatted into rules for use. Neural nets also establish a model of a process operation. However, the neural net creates a continuous model by extracting knowledge directly from the historical operating data of the target unit. Human knowledge requirements involve only a general understanding of causal rela-
Correspondence to: Dr. Malcolm Beaverstock, Vice President, Automation Technology International, 755 Lakeside Drive West, Mobile, AL 36693, USA.
tionships along with basic engineering fundamentals of dynamics and process operations. Because neural nets can be used to create software sensors, there is an increased interest in their use for control systems.
Neural net structure
The learning process for neural nets is based on a mathematical simulation of the biological neuron characteristics in the human brain as
/,,-~
11. WI 1 ,, ~
It1" wH 1
(~
Proces~s
Predcited
Inputs
Outputs
Input N o d e s (I)
Hidden Nodes (H)
i n = / (process input n) where [ : linear sca~efrom 0-I over process input operating range
Hm = g (Zin * Win) where g: non linear eigmo~l scale from 0-I
Wi n : Weighting associated with Inpul node n
WHn ~ Weighting ~ a t e d with Inpu~ node n
0019-0578/93/$06.00 © 1993 - Elsevier Science Publishers B.V. All rights reserved
Output
Fig. 1. Neural net topology.
No(lee (O) On = g C~H m " W Hm) where g: non linear sigrnoid scale from 0-1
236
M.C. Beaverstock / Neural networks for control
shown in Fig. 1. A set of inputs is applied to each neuron, with each input coming either from a process measurement or another neuron. Each input is scaled and then multiplied by a weighting factor. The weighted inputs are then summed and scaled through a nonlinear sigmoid function. The complete neural network can contain many layers of interconnected neurons although the threelayer architecture of Fig. 1 is predominately used in control. While the mathematics appear simplistic for a single neuron, the total network of interconnected equations is highly complex and nonlinear. The example discussed in this paper consisted of 27 inputs, 30 hidden nodes, and 3 outputs. The neural network describing the inp u t - o u t p u t relationships therefore contained 33 equations with 600 weight factors. The intelligence of the network is embedded in the weighting factors. It is the training function that establishes these weighting factors. During training, the network takes the input values and propagates them forward through the layers of neurons to create output values. These are then compared with the correct historical value for the output. Based on the amount of error, a correction mechanism then starts and propagates in the reverse direction to establish new weighting values. The procedure continues until an error criterion is satisfied. To investigate neural net capability as well as the practical considerations involved in using this technology, an operating brown stock washer in a
paper mill was selected as a target for neural net based control. Since this original project, other applications in the pulp and paper, refining, and polymerization industries have been investigated with similar success.
Neural net control example
The brown stock washer (Fig. 2) uses multiple vacuum drums to remove dissolved organic and soluble inorganic materials present in a pulp slurry. Its operation leverages the financial performance of many other areas of the mill. Its control must balance production requirements, water usage, and washing effectiveness. Despite its strategic position, control of this unit remains an art form. The control objective is to maintain mat conditions leaving the last washing stage. These conditions are specified as the mat consistency (the unit mass rate leaving the washer, the mat density (weight per square feet), and the soda loss (pounds of sodium per ton of pulp). Consistency and density reflect production through the washer, while soda loss indicates washing efficiency. Soda loss can not be measured on-line and laboratory samples are used to monitor operation. Attempts using conductivity meters and multi-variable regression analysis have been tried for control with little success over a wide washer operating range.
Wash Water Pulp from Digesters
Pulp to Bleaching
DeMNy • Id~ Cons~tency • 8oda Loss
Fig. 2. Brown stock washer.
M.C. Beaverstock / Neural networksfor control 26
• Soda Loss, #~
,~ndsrd Control
237
Making neural nets work
~.j,s 25
i
ChemicalSavin~ s S150-20~0(~I~i" ,~j,s
24 23
/
NuWe~)'" Control
~,jJ'
22 /
$1o2 Milli0rffy t
21 20
9OO
7OO
Production,t/day
Fig. 3. Operational results. Using historical information, a neural net was trained to learn the relationship:
[Density] Consistency Soda loss
=f
D r u m speeds Shower flows Dillutionflows Temperatures Pressures Stock flow
This relationship then was used to control the washer over a wide range of mill conditions. The results, shown in Fig. 3, document improved washer efficiency over the operating range of the equipment. This resulted in reduced variability, decreased chemical loss, and the possibility of increased production. Depending on operating requirements, savings in the range of $150,000 to $2,000,000 per year were estimated. These results show that the neural net technology works and works well. The experience also provided many lessons for the practical installation of neural nets.
This installation, as well as subsequent projects, has established a methodology for neural net projects. Project activities can be divided into various phases as shown in Table 1. Neural net applications do require an understanding of their applicability and theory. The level of neural net familiarity is easily within the grasp of process engineers who have access to adequate software support. Process and control knowledge is a definite requirement and the time and effort required for an application is dependent on project experience. Engineering
The first phase of a neural net project is typical of any engineering effort. The work starts with a review of the process, current instrumentation, and current control strategy. The first decision is to select the desired output of the neural network. It must be something that can be measured in some way and related to the selected inputs. If more than one variable is selected, they should not be closely correlated. Selection of inputs can occur once the outputs are established. Neural nets accept any number of inputs. Process engineering and experience is key at this point. Understanding process dynamics is also important. When the trajectory of an input may be important, then values at various times should be selected (e.g. flow rate at time t, t - 1, t - 2, etc.). Efficiency in the learning phase will also be improved if redundancy and correlated inputs are eliminated. For example, a flow
Table 1 Neural network project steps Step
Activities
Engineering Study,Select inputs, Select outputs, Determine network architecture, Develop control strategy, Establish base line Learning
Collectsamples, Train network, Test network, Analyze results, Optimize network architecture
Predicting
Open-loop tests, Analyze results
Controlling
Closed-loop control, Monitor results
Auditing
Document system, Establish benefits
238
M.C. Beaverstock / Neural networks for control " Mat Density Mat Consistency
P
r
o
c
e
s
s
~ - Soda Loss
Process inputs
Fig. 4. Network configuration.
rate measure and flow valve positions both represent the same flow input. For the washer project, mat consistency, mat density, and soda loss were selected as output variables. Originally, a single network predicted all three, but increased accuracy was achieved by predicting consistency and density from one network and then using those outputs along with other process measurements to predict soda loss. The first network design used 47 inputs, including trended values of stock flow to the unit. After further review the number of inputs was consolidated to 27. The network diagram is shown in Fig. 4.
Learning Once a network architecture is established, sets of data can be collected for training. Understanding process dynamics, keeping track of sample times, and obtaining valid measurements are critical elements for training. The input values must be taken at the time in the process that they
Soda Loee W ton
impacted the output values. Because of this training requirement, neural nets cannot be applied to new construction until operating data is established. A set of output values, with their related input values is called a training set. Training sets may be collected manually or through an existing control system. While it is not necessary to perturb the operating equipment, the data should represent the entire operating range of the process unit. Where operations vary widely, or various grades are involved, separate neural networks, each with their own training sets, should be established. Training sets should always be divided into two groups. The network is trained with one group while the second is used to validate the results. For the brown stock washer a total of 140 training sets were collected--57 were used to train the network and the remainder were used for testing.
Prediction The trained network now is capable of predicting output variables on line, usually in an openloop configuration. The results of the washer open-loop predictions are shown in Fig. 5. This soda loss prediction ability exceeded expectations and appeared to surpass results from using standard correlation procedures. The fact that the network was trained using historical data collected eight months earlier emphasized the robustness of the technology.
S O O A LOSS C O M P A R I S O N --~--
110.00
Lab
00.00 70.00
S0.00, 30.00
10.00
I 5
I 9
I 13
I 17
I 21
I 25
I I I 20 33 37 Sample Number
I 41
I 48
I 49
Fig. 5. Open-loop predictions (four-week test period).
I 53
I 57
I 61
I 65
M.C. Beaverstock / Neural networks for control
239
Control
Neural net shells
The trained neural net was used as part of a control strategy. In the case of the pulp washer, the network predictions of density and consistency were used to provide a dilution factor control strategy. The operator established a dilution factor set point and the neural net based control adjusted the third washer shower flow set point. The soda loss prediction was used to adjust dilution flows to the washer vats based on a movingzone statistical control approach that continued to drive the washer to maximum washing efficiency. Control results showing reduced soda loss and 45% reduction in control variation are shown in Fig. 6. The neural net control demonstrated excellent response over a wide region of control and exhibited graceful degradation when faced with multipie input failures. At a time when 20% of the inputs were incorrect, the prediction error only changed from 5% to 12%.
There are a number of neural network shells available in the market that operate on personal computers or workstations. These include NeuroShell (Ward Systems Group), BrainMaker (C.S.S), ANSim (SAIC), and GENESIS (Neural Systems, Inc.). This list is certainly not complete and continues to change. Shells differ from neural net application environments (e.g. N u W e b - Automation Technology) in that they do not include file management and computer interface resources. The shells only provide the algorithms for training and thereby create networks that must be imbedded in control systems. In most cases, the user is responsible for writing code for collecting and validating data for training, presenting training sets to the shell, and formatting output files. Additionally, before the trained network can be used in a control system, the user must create drivers to collect and send data to a control system, develop file transfer routines to format I / O data, and compile all related code into an application to manage the operation of the network. Once written and put into operation, the custom software must be well documented and maintained to reduce life cycle costs. During the development of a neural net application there are other critical tasks. Since these are normally off-line tasks, they are not part of
Software support requirements While good process and control knowledge are required for such neural net applications, the true potential for success will be determined by the software used to implement the system.
Soda Loss, #h T
0
_
on Neural Net
Noon Time of Day, Hr. Fig. 6. Closed-loop results.
24
M.C. Beaverstock / Neural networks for control
240
those that rank RMS values in various ways as well as providing sensitivity analysis statistics over all the inputs.
System development environment
I°,..,.-I
Control System
~
~
Process
Fig. 7. Neural environment.
commercial shell software. These operations deal with determining the validity of the test data, checking the relationship established by the network, and documenting the network. Statistical analysis is used to help select valid training sets from a large group of test data. The selection process is critical to the application success and should never be simplified. The analysis must check for even representation over the application operating range and for duplicate inputs, outputs, or both. Inconsistent data sets (e.g. same input levels with different outputs, or values attributed to bad measurements) must be identified and eliminated. Once a network is trained, it is first tested against the samples selected for testing purposes. The resulting information must be analyzed to optimize the training process. Statistics can again help the engineer. The normal statistic is a calculation of the root mean square error (RMS) of all the predictions against the correct values. Useful reports that can aid in the training process are
A complete development environment for neural networks would include a standard approach for control system interface drivers, file translators, network training capability, historization file management, and i n p u t / o u t p u t validation as shown in Fig. 7. The personal computer based system allows for the simultaneous execution and bumpless transfer of networks. It also includes a wide range of software tools for statistically analyzing training and test sets. Additionally, the approach incorporates the diagnostic ability of expert system rules to monitor and manage input and output values.
Conclusion
Neural networks work and can be successfully applied to process unit operations to achieve financial improvement. They have proven to be robust when used with control systems. Most importantly, they can be implemented using knowledge that focuses more on the process operation and control than the neural net technology. While this puts neural net projects within the reach of most engineers, implementation speed and efficiency, which brings faster realization of benefits, is directly proportional to the application team experience and the software environment.
235
It takes knowledge to apply neural networks for control Malcolm C. Beaverstock Automation Technology International, Mobile, AL 36693, USA
Until very recently, process control applications of neural networks have been limited to theoretical studies and fairly small experimental projects on somewhat simple processes. Early problems identified with using these systems in an on-line environment included appropriate handling of raw process data, extending the knowledge base of the network to include more broad operating conditions, transforming the network results into useful controller setpoints, and providing for graceful degradation of the controller functions. This paper describes a practical approach for implementing artificially intelligent process control functions based on a unique combination of rule-based expert systems and neural network technology. The system has been successfully applied to a complex pulp and paper process and new applications are currently under development for other industries.
Introduction
Artificial intelligence technology promised advances in control applications by capturing the power of the human brain. The successful application of this technology requires an understanding of the type of knowledge needed to make the system work in a real-world environment. Expert systems utilize a person's insight of process operation to create a model that interprets operating conditions and takes control action. Knowledge of discrete, causal relationships, based on experienced observations, is required from operators and others working with process equipment. That knowledge is imbedded in the human mind and has to be extracted and formatted into rules for use. Neural nets also establish a model of a process operation. However, the neural net creates a continuous model by extracting knowledge directly from the historical operating data of the target unit. Human knowledge requirements involve only a general understanding of causal rela-
Correspondence to: Dr. Malcolm Beaverstock, Vice President, Automation Technology International, 755 Lakeside Drive West, Mobile, AL 36693, USA.
tionships along with basic engineering fundamentals of dynamics and process operations. Because neural nets can be used to create software sensors, there is an increased interest in their use for control systems.
Neural net structure
The learning process for neural nets is based on a mathematical simulation of the biological neuron characteristics in the human brain as
/,,-~
11. WI 1 ,, ~
It1" wH 1
(~
Proces~s
Predcited
Inputs
Outputs
Input N o d e s (I)
Hidden Nodes (H)
i n = / (process input n) where [ : linear sca~efrom 0-I over process input operating range
Hm = g (Zin * Win) where g: non linear eigmo~l scale from 0-I
Wi n : Weighting associated with Inpul node n
WHn ~ Weighting ~ a t e d with Inpu~ node n
0019-0578/93/$06.00 © 1993 - Elsevier Science Publishers B.V. All rights reserved
Output
Fig. 1. Neural net topology.
No(lee (O) On = g C~H m " W Hm) where g: non linear sigrnoid scale from 0-1
236
M.C. Beaverstock / Neural networks for control
shown in Fig. 1. A set of inputs is applied to each neuron, with each input coming either from a process measurement or another neuron. Each input is scaled and then multiplied by a weighting factor. The weighted inputs are then summed and scaled through a nonlinear sigmoid function. The complete neural network can contain many layers of interconnected neurons although the threelayer architecture of Fig. 1 is predominately used in control. While the mathematics appear simplistic for a single neuron, the total network of interconnected equations is highly complex and nonlinear. The example discussed in this paper consisted of 27 inputs, 30 hidden nodes, and 3 outputs. The neural network describing the inp u t - o u t p u t relationships therefore contained 33 equations with 600 weight factors. The intelligence of the network is embedded in the weighting factors. It is the training function that establishes these weighting factors. During training, the network takes the input values and propagates them forward through the layers of neurons to create output values. These are then compared with the correct historical value for the output. Based on the amount of error, a correction mechanism then starts and propagates in the reverse direction to establish new weighting values. The procedure continues until an error criterion is satisfied. To investigate neural net capability as well as the practical considerations involved in using this technology, an operating brown stock washer in a
paper mill was selected as a target for neural net based control. Since this original project, other applications in the pulp and paper, refining, and polymerization industries have been investigated with similar success.
Neural net control example
The brown stock washer (Fig. 2) uses multiple vacuum drums to remove dissolved organic and soluble inorganic materials present in a pulp slurry. Its operation leverages the financial performance of many other areas of the mill. Its control must balance production requirements, water usage, and washing effectiveness. Despite its strategic position, control of this unit remains an art form. The control objective is to maintain mat conditions leaving the last washing stage. These conditions are specified as the mat consistency (the unit mass rate leaving the washer, the mat density (weight per square feet), and the soda loss (pounds of sodium per ton of pulp). Consistency and density reflect production through the washer, while soda loss indicates washing efficiency. Soda loss can not be measured on-line and laboratory samples are used to monitor operation. Attempts using conductivity meters and multi-variable regression analysis have been tried for control with little success over a wide washer operating range.
Wash Water Pulp from Digesters
Pulp to Bleaching
DeMNy • Id~ Cons~tency • 8oda Loss
Fig. 2. Brown stock washer.
M.C. Beaverstock / Neural networksfor control 26
• Soda Loss, #~
,~ndsrd Control
237
Making neural nets work
~.j,s 25
i
ChemicalSavin~ s S150-20~0(~I~i" ,~j,s
24 23
/
NuWe~)'" Control
~,jJ'
22 /
$1o2 Milli0rffy t
21 20
9OO
7OO
Production,t/day
Fig. 3. Operational results. Using historical information, a neural net was trained to learn the relationship:
[Density] Consistency Soda loss
=f
D r u m speeds Shower flows Dillutionflows Temperatures Pressures Stock flow
This relationship then was used to control the washer over a wide range of mill conditions. The results, shown in Fig. 3, document improved washer efficiency over the operating range of the equipment. This resulted in reduced variability, decreased chemical loss, and the possibility of increased production. Depending on operating requirements, savings in the range of $150,000 to $2,000,000 per year were estimated. These results show that the neural net technology works and works well. The experience also provided many lessons for the practical installation of neural nets.
This installation, as well as subsequent projects, has established a methodology for neural net projects. Project activities can be divided into various phases as shown in Table 1. Neural net applications do require an understanding of their applicability and theory. The level of neural net familiarity is easily within the grasp of process engineers who have access to adequate software support. Process and control knowledge is a definite requirement and the time and effort required for an application is dependent on project experience. Engineering
The first phase of a neural net project is typical of any engineering effort. The work starts with a review of the process, current instrumentation, and current control strategy. The first decision is to select the desired output of the neural network. It must be something that can be measured in some way and related to the selected inputs. If more than one variable is selected, they should not be closely correlated. Selection of inputs can occur once the outputs are established. Neural nets accept any number of inputs. Process engineering and experience is key at this point. Understanding process dynamics is also important. When the trajectory of an input may be important, then values at various times should be selected (e.g. flow rate at time t, t - 1, t - 2, etc.). Efficiency in the learning phase will also be improved if redundancy and correlated inputs are eliminated. For example, a flow
Table 1 Neural network project steps Step
Activities
Engineering Study,Select inputs, Select outputs, Determine network architecture, Develop control strategy, Establish base line Learning
Collectsamples, Train network, Test network, Analyze results, Optimize network architecture
Predicting
Open-loop tests, Analyze results
Controlling
Closed-loop control, Monitor results
Auditing
Document system, Establish benefits
238
M.C. Beaverstock / Neural networks for control " Mat Density Mat Consistency
P
r
o
c
e
s
s
~ - Soda Loss
Process inputs
Fig. 4. Network configuration.
rate measure and flow valve positions both represent the same flow input. For the washer project, mat consistency, mat density, and soda loss were selected as output variables. Originally, a single network predicted all three, but increased accuracy was achieved by predicting consistency and density from one network and then using those outputs along with other process measurements to predict soda loss. The first network design used 47 inputs, including trended values of stock flow to the unit. After further review the number of inputs was consolidated to 27. The network diagram is shown in Fig. 4.
Learning Once a network architecture is established, sets of data can be collected for training. Understanding process dynamics, keeping track of sample times, and obtaining valid measurements are critical elements for training. The input values must be taken at the time in the process that they
Soda Loee W ton
impacted the output values. Because of this training requirement, neural nets cannot be applied to new construction until operating data is established. A set of output values, with their related input values is called a training set. Training sets may be collected manually or through an existing control system. While it is not necessary to perturb the operating equipment, the data should represent the entire operating range of the process unit. Where operations vary widely, or various grades are involved, separate neural networks, each with their own training sets, should be established. Training sets should always be divided into two groups. The network is trained with one group while the second is used to validate the results. For the brown stock washer a total of 140 training sets were collected--57 were used to train the network and the remainder were used for testing.
Prediction The trained network now is capable of predicting output variables on line, usually in an openloop configuration. The results of the washer open-loop predictions are shown in Fig. 5. This soda loss prediction ability exceeded expectations and appeared to surpass results from using standard correlation procedures. The fact that the network was trained using historical data collected eight months earlier emphasized the robustness of the technology.
S O O A LOSS C O M P A R I S O N --~--
110.00
Lab
00.00 70.00
S0.00, 30.00
10.00
I 5
I 9
I 13
I 17
I 21
I 25
I I I 20 33 37 Sample Number
I 41
I 48
I 49
Fig. 5. Open-loop predictions (four-week test period).
I 53
I 57
I 61
I 65
M.C. Beaverstock / Neural networks for control
239
Control
Neural net shells
The trained neural net was used as part of a control strategy. In the case of the pulp washer, the network predictions of density and consistency were used to provide a dilution factor control strategy. The operator established a dilution factor set point and the neural net based control adjusted the third washer shower flow set point. The soda loss prediction was used to adjust dilution flows to the washer vats based on a movingzone statistical control approach that continued to drive the washer to maximum washing efficiency. Control results showing reduced soda loss and 45% reduction in control variation are shown in Fig. 6. The neural net control demonstrated excellent response over a wide region of control and exhibited graceful degradation when faced with multipie input failures. At a time when 20% of the inputs were incorrect, the prediction error only changed from 5% to 12%.
There are a number of neural network shells available in the market that operate on personal computers or workstations. These include NeuroShell (Ward Systems Group), BrainMaker (C.S.S), ANSim (SAIC), and GENESIS (Neural Systems, Inc.). This list is certainly not complete and continues to change. Shells differ from neural net application environments (e.g. N u W e b - Automation Technology) in that they do not include file management and computer interface resources. The shells only provide the algorithms for training and thereby create networks that must be imbedded in control systems. In most cases, the user is responsible for writing code for collecting and validating data for training, presenting training sets to the shell, and formatting output files. Additionally, before the trained network can be used in a control system, the user must create drivers to collect and send data to a control system, develop file transfer routines to format I / O data, and compile all related code into an application to manage the operation of the network. Once written and put into operation, the custom software must be well documented and maintained to reduce life cycle costs. During the development of a neural net application there are other critical tasks. Since these are normally off-line tasks, they are not part of
Software support requirements While good process and control knowledge are required for such neural net applications, the true potential for success will be determined by the software used to implement the system.
Soda Loss, #h T
0
_
on Neural Net
Noon Time of Day, Hr. Fig. 6. Closed-loop results.
24
M.C. Beaverstock / Neural networks for control
240
those that rank RMS values in various ways as well as providing sensitivity analysis statistics over all the inputs.
System development environment
I°,..,.-I
Control System
~
~
Process
Fig. 7. Neural environment.
commercial shell software. These operations deal with determining the validity of the test data, checking the relationship established by the network, and documenting the network. Statistical analysis is used to help select valid training sets from a large group of test data. The selection process is critical to the application success and should never be simplified. The analysis must check for even representation over the application operating range and for duplicate inputs, outputs, or both. Inconsistent data sets (e.g. same input levels with different outputs, or values attributed to bad measurements) must be identified and eliminated. Once a network is trained, it is first tested against the samples selected for testing purposes. The resulting information must be analyzed to optimize the training process. Statistics can again help the engineer. The normal statistic is a calculation of the root mean square error (RMS) of all the predictions against the correct values. Useful reports that can aid in the training process are
A complete development environment for neural networks would include a standard approach for control system interface drivers, file translators, network training capability, historization file management, and i n p u t / o u t p u t validation as shown in Fig. 7. The personal computer based system allows for the simultaneous execution and bumpless transfer of networks. It also includes a wide range of software tools for statistically analyzing training and test sets. Additionally, the approach incorporates the diagnostic ability of expert system rules to monitor and manage input and output values.
Conclusion
Neural networks work and can be successfully applied to process unit operations to achieve financial improvement. They have proven to be robust when used with control systems. Most importantly, they can be implemented using knowledge that focuses more on the process operation and control than the neural net technology. While this puts neural net projects within the reach of most engineers, implementation speed and efficiency, which brings faster realization of benefits, is directly proportional to the application team experience and the software environment.