Structural health monitoring system based on multi-agent coordination and fusion for large structure

Advances in Engineering Software 86 (2015) 1–12

Contents lists available at ScienceDirect

Advances in Engineering Software journal homepage: www.elsevier.com/locate/advengsoft

Structural health monitoring system based on multi-agent coordination and fusion for large structure Dong Liang a,⇑, Shenfang Yuan b a

Department of Aeronautics, Xiamen University, South Siming Road 422#, Xiamen 361005, People’s Republic of China The State Key Lab of Mechanics and Control of Mechanical Structures, Nanjing University of Aeronautics and Astronautics, Yu Dao Street 29#, Nanjing 210016, People’s Republic of China b

a r t i c l e

i n f o

Article history: Received 31 March 2014 Received in revised form 10 February 2015 Accepted 15 March 2015

Keywords: Structural health monitoring Damage identification Multi-agent Fusion Coordination Strain distribution Joint failure Large structure

a b s t r a c t In practical applications of structural health monitoring technology, a large number of distributed sensors are usually adopted to monitor the big dimension structures and different kinds of damage. The monitored structures are usually divided into different sub-structures and monitored by different sensor sets. Under this situation, how to manage the distributed sensor set and fuse different methods to obtain a fast and accurate evaluation result is an important problem to be addressed deeply. In the paper, a multiagent fusion and coordination system is presented to deal with the damage identification for the strain distribution and joint failure in the large structure. Firstly, the monitoring system is adopted to distributedly monitor two kinds of damages, and it self-judges whether the static load happens in the monitored sub-region, and focuses on the static load on the sub-region boundary to obtain the sensor network information with blackboard model. Then, the improved contract net protocol is used to dynamically distribute the damage evaluation module for monitoring two kinds of damage uninterruptedly. Lastly, a reliable assessment for the whole structure is given by combing various heterogeneous classifiers strengths with voting-based fusion. The proposed multi-agent system is illustrated through a large aerospace aluminum plate structure experiment. The result shows that the method can significantly improve the monitoring performance for the large-scale structure. Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction In recent years, there has been an increasing awareness of structural health monitoring (SHM) of the large infrastructures including aircraft structure, long-span bridges and high-rise buildings, since structure damage resulting from loading, joint failure and, etc. may cause the tremendous disaster [1,2]. However, for the actual large-scale structure monitoring, there are a number of sensors and various ones, which are distributed and dispersed, and the various evaluation methods, which are appropriate for different situations. For the application, the integration of a wide range of sensors and different evaluation methods must be done. Using the SHM technology on the large complicated practical structures, it is a critical problem that how to effectively manage the distributed sensor network, and coordinate and fusion different evaluation methods. Now multi-agent system (MAS) in artificial intelligence (AI) has been a natural model for developing a largescale, complex, distributed system, which is loosely coupled and heterogeneous. It is an effective way for solving large-scale ⇑ Corresponding author. http://dx.doi.org/10.1016/j.advengsoft.2015.03.008 0965-9978/Ó 2015 Elsevier Ltd. All rights reserved.

distributed problem [3]. Hence, multi-agent technology can be used to solve the SHM problem for the large structure. At present, the MAS is researched in fault diagnosis and health monitoring, and there are some reports on MAS, related to mechanical fault diagnosis and health monitoring, etc. In fault diagnosis domain, Stephen et al. worked at the research and the application of multi-agent power system condition monitoring [4]. NASA’s Lyell et al. analyzed and designed the International Space Station’s electric power system health monitoring using agent-based methods, and gave the simulation [5]. For fault detection and identification in chemical processes, Ng and Srinivasan utilized MAS to fusion different FDI methods for eliminating the results conflict and improving the diagnosis performance [6]. Their research focused on the construction of a single monitoring agent, the design and the functional verification of MAS for their fields. In SHM domain, it is being in the initial stages of research on the MAS technology. Since 2002 NASA’s Science and Technology Information Program (STI) was a research on non-life aerial vehicles using self-agent theory, and the work focused on the coordinate research of MAS [7]. Esterline et al. put forward the MAS

2

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

model for the vehicle health management system, and their research was the distributed problem-solving based on contract net protocol [8]. Subsequently, Esterline et al. improved the above work and implemented MAS using JADE platform [9]. Yuan et al. initially proposed the distributed SHM system based wireless sensors and MAS which gave the agent’s basic definition and implementation [10]. Then, Zhao et al. presented a complete design method for SHM based on MAS (SHM MAS) including ontology design, distributed database realization, facilitator design, and introduced the validation work of the case study in a large aviation aluminum plate in detail [11,12]. On the basis of Zhao’s work, Sun et al. put forward the case-based reasoning and rule-based reasoning to coordinate obtaining the damage evaluation method for the accurate identification [13]. Bosse and Lechleiter present a hybrid data processing approach of on-line sensing and off-line inverse problem soluting for SHM systems by using self-organizing mobile multi-agent systems (MAS) [14]. In our past work [11–13], SHM MAS is also used to monitor the static distribution and joint failure on the structure. There, a strain sensor is an agent, and they are self-organized to monitor the strain load. Nevertheless, the system is complicated and every sensor agent has less autonomous. Meanwhile, the agent social ability is not studied intensively in the resource distribution for monitoring two types of damage uninterruptedly, and in the fusion of kinds of evaluation methods for the damage identification accuracy. Hence, this paper presents a MAS fusion and coordination framework for the damage accurately and real-time identification in the large structure, and gives the system implementation and experimental verification. The organization of this paper is as follows: Section 2 provides the multi-agent coordination framework for the SHM. In Section 3, the proposed multi-agent system for the static distribution and joint failure monitoring is described in a case evaluation for damage identification. Then, Section 4 gives a discussion and a brief comment.

(5)

(6)

(7)

(8) (9)

damages assessment. Thus this is convenient for the agent to search the services and resources to achieve interaction and collaboration. District monitor agent (DMA) manages several sensing agents, and obtains the important (valid) damage data in local area. Central coordination agent (CCR) is responsible for the coordination among subsystems, such as conflict solving, time synchronization, resource distribution, and negotiation strategy. Central information fusion agent (CIFA) is in charge of fusing the damage information from different subareas to give a global estimation of the whole structure. User interface agent (UIA) provides information to the user and accepts the user’s instruction. In every subsystem, sharing information management agent (SIMA) is designed. It is a distributed database, and it allows the agent to publish its identity (ID) and address in order that other agents can easily find it, and it is beneficial to exchange the information between different DEAs in the same or different subsystems. Meanwhile, it saves and provides the DEA’s parameter.

The data monitoring layer includes (1) and (5) agents. The data interpretation and damage diagnostic layer separately corresponds to (2) and (3) agents. The information fusion layer is realized by (4), (6), (7), (8) and (9) agents. The whole structure’s damage evaluation work can be realized through agent’s coordination and cooperation, in which consists of the coordination between the adjacent sensors and the cooperation of the DEAs between subsystems, and the one of the agents between different layers of a subsystems for the distributed problem studied by the paper. The SHM architecture based on MAS presented is shown in Fig. 1. The detail design of this architecture has been consideration in our work [11].

2. Development of SHM based on MAS 3. Evaluation of SHM based on MAS Considering a typical SHM system, a large-scale structure can be divided into several subareas monitored. The sensors are placed in or on the structure to acquire the data on the structural status parameters. The appropriate signal or information processing methods are adopted to analyze and extract the damage-sensitive features from the sensing data. The corresponding damage evaluation methods can obtain the structure health status using the key feature. Thus, according to the three typical parts, the data monitoring layer, data interpretation and damage diagnostic layer can be defined to form SHM MAS. Considering a large scale structure divided into some subareas, an information layer is needed to collaborate and fuse the damage information from the local subareas and provide the whole information to the user. Hence, according to the various functional components of the large SHM system, and considering the management and the coordination function, the agent in the system can be divided into 9 categories as follows: (1) Sensing agent (SA) is to monitor the structure specific parameters. (2) Signal processing agent (SPA) extracts the structural key features from the sensing signal. (3) Damage evaluation agent (DEA) assesses the structural state in the light of structural features. (4) In every subsystem, facilitator (agent) provides ‘‘yellow pages’’ services to every kind of agent. It is in charge of registering every agent services, such as feature extraction,

3.1. System setup In order to verify the effectiveness of multi-agent system, in this paper, a large aerospace aluminum plate structure is studied as the experimental object. In Fig. 2 it gives the flat structure and the sensor distribution diagram, and the photo of the structure and the monitoring system, as shown in Fig. 3. The plate structural material is the aviation-specific hard aluminum LY12, whose basic dimensions and thickness are 120 cm  200 cm  0.25 cm. Around the structure there are 64 M6-bolts which are used to fix the plate with bracket, and the bolt space is 10 cm. The bracket is put on the ground vertically, supporting the aluminum structure. The structure is divided into eight sub-regions, each of which is 49 cm  45 cm except its edge. PZT (Lead Zirconate Titanate) sensors are laid on the vertices of each sub-region. In the 3th sub-region, FBG sensors are arranged, and in other sub-regions, the strain gauge sensors are arranged. In addition, each sub-area is divided into nine regional units. SHM MAS has been developed by Zhao et al. [11]. In the paper, the presented improved multi-agent system is described in detail for the strain distribution and joint failure monitoring as follows. It should demonstrate the following advantages: the system can manage different sensor networks to monitor two kinds of damage, and focus on the static load on its boundary, and choose the idle damage evaluation method in different subsystems to improve the efficiency, and fusion different evaluation methods to

3

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

User Interface Agent

Central Information Fusion Agent

Central Coordination Agent

DEA

DEA

DEA

Subsystem1

Facilitator1

Facilitator2

Facilitator N SPA

SPA DMA

SPA

DMA

DMA

SIMA

SIMA

SA1

Subsystem N

SIMA

SAN

SA2

Large structure Subarea1 Sub u area1

Subarea2 Sub u area2

SubareaN Sub u areaN a

Fig. 1. Developed MAS for SHM.

FBG

Subarea2

Subarea6

Strain

Subarea4

Subarea7

PZT SA

Subarea8

Bolt

Fig. 2. System setup.

accurately estimate the strain distribution change and joint failure position. 3.2. Theory Strain distribution change and joint failure are two types of damage in SHM, which can be identified by the pattern recognition technology. Their principle is as follow:

The static load localization is based on the strain distribution change monitored by the strain sensor network [2]. The sensor outputs of the monitored substructure form a pattern to represent its strain distribution. Once a concentrated load applies on different positions of the structure, the strain distribution changes correspondingly. As a result, the output mode of the sensors in the monitored subarea also changes. The pattern recognition method can be used to classify the different patterns to decide

4

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

pattern recognition. The neural network is often applied to identify the loosening bolt.

PZT

Wireless Sensor Node

Strain Gauge

FBG

Fig. 3. The photo of the structure and the monitoring system.

different positions of the load. At present, the minimum-distance classification has been adopted for the static load localization. If the distance between the monitored strain pattern and a reference strain pattern in database is minimum, the load position of the reference strain pattern is deemed to be the one of the monitored pattern. The active SHM method is generally adopted to monitor the joint failure induced by bolt loosening [2]. The method is based on the structural vibration response, and use the piezoelectric sensor as the actuator or the sensor. Its actuator–sensor scheme includes the single-actuator multi-sensor and cycle-actuator multi-sensor, as shown in Fig. 4 [12]. In the first scheme, the fixed driver PZT sensor is arranged on the structure to stimulate the sensors surrounding the structure at the same time. Since the power of the actuator is finite, and numerous structure joint bolts and pattern overlapping exist, the second scheme is presented for the large structure. The PZT sensor around the boundary acts as actuator in turn. Each time the signals of two adjacent PZT sensors (left and right, or upper and lower) are sampled. For instance, in Fig. 4b, when PZT sensor 1 acts as the actuator, the signals of PZT sensor 2 and 5 as the sensors are sampled. Generally, a sine wave is excited by the PZT actuator to the structure at a frequency, under which the vibration response of the structure is sensitive to the bolt loosening. The experiment proves the sensor signal varies before and after the bolt loosening [12]. Hence, the peak changes of the sensor signals are extracted to form the feature mode for

3.3. Multi-agent system The entire hardware schematic diagram for SHM MAS is shown in Fig. 5. The entire system includes three parts for the large aluminum plate monitoring. The PZT scanning system is used in the active monitoring for bolt loosening. In order to monitor the static load, the FBG sensor signal is acquired by Micro Optics si425 Swept Laser Interrogator and the strain gauges sensor signal are got by the wireless sensor developed independently by the laboratory. These systems communicate with the central computer by the Ethernet, the Serial Port and the wireless transmission based on ZigBee protocol. In the multi-agent system, except SA implemented by the measurement hardware and software, other agents are implemented by the software. The software is programmed with LabVIEW 8.5 and MATLAB R2006a in the industry control computer. In the software, there are data monitoring layer agents, data interpretation layer agents, damage diagnostic layer agents and information layer agents. Each of the layers contains a number of agents performing different functions. The social ability and cooperation between the agents leads to the final damage estimation. The functionality and major composition principles of the agent within each layer are described below. (1) Data monitoring layer agents Strain sensing agent (Strain SA): The static strain change generated by the concentrated load applied to the plate is measured by the sensing agent. Generally, the strain gauge and the Fiber Optic are adopted to measure the strain distribution. The strain gauge sensor is the most usual sensor to monitor the strain, and it has cheap price and stable performance. In order to reduce the information amount of the sensor network, the wireless sensor is employed to improve the system speed, weight and complexity [10,15]. The strain gauge is connected to the wireless sensor node through the bridge circuit and condition circuit, as shown in Fig. 6. The Fiber Bragg Grating (FBG) sensor is also a usual strain sensor. Its advantage is resistant to electromagnetic interference, light weight, multiplexing and few output wires. Its signal is passed to the interrogator, which provides rapid, accurate measurements for hundred of Fiber Optic sensors, as shown in Fig. 7. Here, the sensing agent includes Strain Gauge SA and Fiber Optic SA. Stain Gauge SA consists of four

PZT sensor1

2

3

1

2

3

4

5

6

4

5

6

7

8

7

8

(a) Single-actuator multi-sensor

9

(b) Cycle-actuator multi-sensor

Fig. 4. The active monitoring method for bolt loosening.

9

5

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

Central Compuer

Demodulation equipment 1 n Coupler

Swept Laser Module Wave Reference

Router

Serial Port FBG sensor

ZigBee

Ethernet

Detector Module Coupler Detector

Low Speed Signal Processing

Micro processor

Fig. 7. The interrogator structure of the Fiber Bragg Grating sensor.

PZT Scanning System

Micro Optics si425 Swept Laser Interrogator

Wireless Sensor

Power Amplifier

I/O board

Relay board

Fig. 5. The schematic diagram for the system integration.

sensors in a subarea, a wireless sensor node, its self-monitoring and acquirement program implemented by LabVIEW. Fiber Optic SA includes four FBG sensors in Subarea 3, the interrogator and LabVIEW software. For the wireless sensor, the relationship between the output voltage change DV and the strain parameter variation De is as follows: DV ¼ K 1 K 2 V De=4. K 1 is the sensitivity coefficient of the strain gauge, K 1 ¼ 2. K 2 is the amplify gain of the node, K 2 ¼ 151:5. V is the bridge supply voltage provided, V ¼ 3:3v . The sample frequency of the node is 1 Hz. For the FBG sensor, the relationship between the central wavelength Dk of its output and the strain change De is as follows: Dk=De ¼ 1:2 nm=1000le. The sample frequency of its interrogator is 250 Hz. PZT sensing agent (PZT SA): The agent is responsible for monitoring the joint failure using the cycle-actuator multi-sensor method. In the assessment experiment, PZT SA consists of twelve PZT sensors around the boundary, the PZT scanning system, as shown in Fig. 8, and LabVIEW software. The system’s sample frequency is 10 MHz. The active monitoring for bolt loosening can be finished by the multi-channel piezoelectric scanning system, as shown in Fig. 9 [16]. The system integrates the waveform generator module, data acquisition module, charge amplifier module, digital I/O module, multi-channel scanning switch board and power amplifier. It can interrogate the large numbers of actuator–sensor channel automatically and efficiently. District monitor agent (DMA): The agent is in charge of managing Strain SAs and PZT SAs. Each DMA supervises four neighboring Stain SAs, which includes Fiber Optic SA. It uses the blackboard model implemented by the LabVIEW Shared Variable to organize the strain sensor network, and focuses the region in which the concentrated load happens, and gets the effective strain data. PZT SA is managed by one DMA. (2) Data interpretation layer agents The layer agents have the LabVIEW work threads, and the communication protocol interaction thread, and their tasks are to extract the signal feature.

Strain gauge

Bridge circuit

Condition circuit

Wireless sensor node

Base Station

Computer

Power Fig. 6. The connection of the strain gauge and the wireless sensor node.

Integrated Scanning System

Waveform Generation board

Data Acquisition board

Charge Amplifier

Fig. 8. The photo of the PZT scanning system and its components.

Static load signal processing agent (SL SPA): The agent uses the moving average method for the signal of Strain SA. Bolt signal processing agent (B SPA): It extracts the peak value of the response from PZT SA. (3) Damage diagnostic layer agents The layer agents also consist of the work thread integrating with the MATLAB function, and the communication protocol interaction thread. They begin the process of turning the data into information that is of greater use to the user. In addition, the agent in the layer uses the advanced intelligent system techniques, coupled with codified knowledge problems and offers a prognosis. The key aspect of the improved SHM MAS is that it will support more than the interpretation technique. Hence, five pattern classification agents are utilized to identify the strain load position and the loosening bolt. The utilized classifier agents are described as follows: Support vector machine damage evaluation agent (SVM DEA): The agent can implement the good recognition rate derived from a few training samples using SVM algorithm, which is based on statistical learning theory [17]. Kernel function is a key parameter for SVM, which includes linear, polynomial, Gaussian RBF and sigmoid. C4.5 damage evaluation agent (C4.5 DEA): The agent implements ‘‘If-Then’’ rules derived from the training data set using the C4.5 algorithm [18]. These rules are used to classify the ‘‘unseen’’ data. k nearest neighbor damage evaluation agent (k-NN DEA): The classifier is very simple and effective [19]. The k nearest neighbors of the unidentified test pattern is searched within a hyper-sphere of predefined radius in order to determine its true class, which is the most class in the k samples. If only one nearest neighbor is detected, k-NN is the minimum-distance classification.

6

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

power amplifier

Actuator piezoelectric sensor arrays Sensor

Multi-channel integrated device

switch charge amplifier

waveform generator digital I/O

Computer

data acquisition

Fig. 9. The principle structure of the active monitoring system.

Improved iterative scaling damage evaluation agent (IIS DEA): IIS is one of the major algorithms for finding the optimal parameters for the conditional exponential model [20]. Its underlying idea is: by approximating the log-likelihood function of the conditional exponential model as some kind of ‘simple’ auxiliary function, it is able to decouple the correlation between the parameters and search for the maximum point along many directions simultaneously. By carrying out this procedure iteratively, the approximated optimal point found over the ‘simplified’ function is guaranteed to converge to the true optimal point due to the convexity of the objective function. Gaussian mixture model damage evaluation agent (GMM DEA): The classifier is based on Gaussian component functions [21]. The linear combination of Gaussian functions is capable of representing a large class of the sample distribution. In principle, it is a compromise between the performance and the complexity. Gaussian mixture has remarkable capability to model the irregular data. (4) Information layer agents The layer agents also integrate with the work thread and the communication protocol interaction thread. They are responsible for fusing and coordinating the bottom layer agent. Facilitator agent (FA): Every subsystem should host a default FA. It manages the agent’s ID and service name built with Labview cluster array. Sharing information management agent (SIMA): A default SIMA should host in every subsystem, and manages all the agent’s IDs and addresses in the subsystem. The agent stores the agent’s ID and address built with Labview cluster array, which is set be Shared Variable, and is able to start up and stop other agents in its subsystem. Central information fusion agent (CIFA): The agent fuses the results of five classifier agents to give the strain load and the bolt loosening estimation with the multi-classifier decision fusion method. User interface agent (UIA): The agent provides the information to the user and accepts the user’s instruction. The UIA is realized with LabVIEW control. Central coordination agent (CCA): The agent is responsible for coordinating different subsystems. The agent management in SHM MAS for the strain load and bolt loosening monitoring consists of three components depicted as yellow-page, white-page and life-cycle service. SIMA is responsible for the white-page and life-cycle service, maintaining a directory of agent identifier (AID) and agent state. AID makes up of agent ID and address. The agent states used are ‘‘start’’, ‘‘suspend’’, ‘‘resume’’ and ‘‘stop’’. When each agent stars up, it registers with SIMA. FA then requests the agents’ addresses from SIMA and queries each about what services are available. In the study, standard agent conventions have been used. This means that all inter-agent communications are handled using just a few types of message, examples of which are ‘‘Subscribe’’, ‘‘Query-ref’’, ‘‘Inform’’, ‘‘Propose’’, ‘‘Publish’’, ‘‘Confirm’’, ‘‘Call for

propose’’. Sending messages or sharing the blackboard is employed for inter-agent communication in SHM MAS. SHM MAS ontology is based on the content we presented [11,12]. It contains the concept, the data attribute and the object attribute. The concept Sensor data with its data attribute is defined as follows: Sensor data [subarea ID, PZT sensor ID, sensor, static or dynamic, valid or invalid, data length, data, timestamp]. Its object attribute tells us that SPA is suitable for Sensor data. For the concept Sensor data feature, its data attribute is Sensor data feature [Feature ID, data length, data, timestamp], and its object attribute is that DEA is suitable for the feature. MAS comprises a number of agents, which is to solve the complex problem not finished by a single agent, and is the coordination network among agents. The coordination mechanism is the key issue in the MAS research. In our SHM MAS, multi-agent coordination technology is utilized to manage sensor networks, fuse various information and coordinate various algorithm modules, and improve the whole system’s accurate and effective assessment. The detail coordination principle is given as follow, which consists of the sensors blackboard supervision and contract net coordination evaluation. 3.3.1. Sensors blackboard supervision In MAS, the blackboard is the system’s shared database, with which the agent can exchange data, information and knowledge. It is divided into a number of information layers, which correspond to the intermediate representation of the problem, used to record raw data, intermediate results and final conclusions, so it is various solutions set of problems. Each agent monitors the state of the blackboard, and seeks the chance to solve the problems. Once the agent finds that the blackboard information can support it to further solve the problem, it begins to solve the problem, and the results are recorded in the blackboard. The new additional information is good for the other agent to continually solve. Repeat this process until the problem is completely resolved [22]. In SHM for the large structure in Fig. 2, eight Strain SAs and one PZT SA are needed. The ith subarea is sensed by the ith Strain SA, and Strain SA 3 is FBG SA. Strain SA 1, 2, 5 and 6 are supervised by the 1st DMA. Strain SA 3, 4, 7, 8 and only one PZT SA are managed by the 2st DMA. Because the joint failure is a cumulative damage, PZT SA uses the PZT sensor as actuator to excite the structure each 1 min, and acquires the signal from the PZT sensors on the plate’s border. However, Strain SAs are required to real-time monitor the structure strain distribution. Once DMA as subscriber gets the data from Strain SA or PZT SA as publisher, the agent can inquire FA to search SPA’s address according to the damage type. A Strain SA is in charge of monitoring a subarea. But for Strain SA, there is a blind area not monitored between two subareas sensed by their defined SA, as shown in Fig. 2. Hence, the blackboard model is presented to supervise the blind area by DMA. The self-monitoring program of Strain SA is used to determine whether the static load is in the region or other regions. For an agent, if the sum of the absolute values of its four sensors changes exceeds a threshold, it deems that the load is in its sub-region or near it. If it is bigger than 0 and less than the threshold, the load

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

7

Fig. 10. SPA invites SVM DEA in different subsystems for bid.

Fig. 11. The initiator work flow.

is deemed to be in other subarea, else no load. The threshold parameter is set by the finite element modeling analysis and experiment. The blackboard model of DMA is replaced with Shared Variables which is used to storage the subarea damage state ‘‘damage near the subarea’’ or ‘‘damage in the other subarea’’ and data for the

Strain SA. When a Strain SA detects the strain sum crossing a threshold, it sends the load event status and data to its blackboard unit in DMA. The Shared Variable is set to be logic 1, and it indicates the load happens near this subarea of Strain SA. If there is no load event or the load being in other regions, its Shared Variable is logic 0.

8

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

Bolt 19

Bolt 23 PZT 9 S31

PZT 3 Bolt 9

Bolt 27 PZT 15

Bolt 33

S32

Subarea 3

S33

S34 Bolt 39

Strain1 Strain3 Bolt 5 Subarea 7 Bolt 2

Bolt 43

Strain2 Strain4 PZT 7

PZT 1

Bolt 54

Bolt 63

PZT 13

Bolt 50

Fig. 12. The strain distribution subareas and loosening bolts monitored.

UIA CIFA Subsystem i SVM DEA

C4. 5 DEA

kNN DEA

IIS DEA

GMM DEA CCRA B SPA

SL SPA FA

DMA Strain SA Large structure

PZT SA Subarea 2

Subarea 1

…… Subarea 8

Fig. 13. MAS coordination and fusion framework in Subsystem i.

Subarea 7

Subarea 3

4

2

8

9

10

13

16 Strain Gauge

6

18

The load focused on

FBG

Fig. 14. The static load localization on the boundary between two subareas.

Once DMA finds that a Strain SA’s state in the blackboard is 1, then it could inquire whether the status of its neighboring Strain SA in the blackboard is 1, and if so, the load occurs near the border between two Strain SAs (including FBG SA).

Then DMA can organize the sensor network, and focus the damage and extract the relative damage data. If the status of its neighboring agent is 0, the load is deemed to be close to its subarea.

9

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

3

10

100% 70% 35% 0%

2 1

Vlotage (V)

Voltage (V)

5

0

0 -1

-5

-2 -10

0

0.5

1

1.5

Time (s)

-3 0

2 x 10

0.5

-5

1

1.5

Time (s)

2 x 10

-5

Fig. 15. Excitation wave and signals recorded at different level of tightening condition.

3.3.2. Contract net coordination evaluation The yellow pages service provided by FA is also exploited to allow the agents in the higher layer to dynamically discover the available DEA agents in time. Once the DMA as publisher has the damage data and the AID, its the data and AID are respectively passed to SL SPA or B SPA as subscriber. At the end, the static strain change or the dynamic strain peak is get. After SPA gets data, the contract net coordination [23] is employed by the SPA to propose the five classifier agents for identifying the staitc load position or the bolt loosening. The whole work flow is as follows.

If SPA receives MNB proposals in TC, or m (5 6 m 6 MNB) proposals until TC, it refuses to accept the bid. The agent can choose the first five different types of DEAs in sequence and awards them, and sends the signal feature, the damage type and the subarea number to them. If SPA receives n (0 6 n 6 4) until TC, it do not also accept the bid. The agent deems some types of DEAs are busy, chooses these bidder and begins to invite each types of DEAs for bid next time. (4) Complete the task and inform of the result: Once DEA as the participant has completed the task, it respectively sends the strain load or bolt loosening location to SPA. The five kinds of DEAs’ results are published to CIFA by SPA. SPA’s work flow is given in Fig. 11. The above improved contract net coordination is introduced, and its advantage is that the initiator need not broadcast all the agents for bid, and only idle agents can bid, and MNB and TC are restricted to reduce the communication amount and improve the efficiency of the invitation and bid. If the damage occurs, SPA waits and orderly accepts the results of five DEAs, and sends the results and the subarea number to CIFA. 7Due to integrate different pattern classification evaluation

-0.555

1546.82

-0.56

1546.81

Central wavelength (nm)

Voltage (V)

(1) Invite for bid: SPA publishes the damage localization service to FA, and wishes the agent to search and match the service description. Then FA informs SPA of the online DEAs’ IDs. SPA can find everyone capable of providing the service with their IDs, and solicit m (1 6 m 6 maximum number of bid(MNB)) proposals from these DEAs by issuing a ‘‘Call for propose’’, and send own ID, address, MNB and time to completion (TC) to them. The tender may most avoid the large number of bid and reduce the amount of information. The agent sequence diagram in Fig. 10 provides an example of this for SPA. (2) Bidding: DEA receives the invitation, and determines whether to bid according to own status. If the agent is busy, it refuses to bid. Otherwise, it sends a bid which contains the own address to SPA with the initiator’s address in the invitation.

(3) Award: SPA does not receive the proposals whenever the number of bid is MNB, or the time is TC. This may largely reduce the computation amount. The design process is as follows.

-0.565 -0.57 -0.575

0

10

20

30

40

50

1546.8 1546.79 1546.78 1546.77

0

5

10

15

20

time (s)

time (s)

(a) Strain 1

(b) S33

Fig. 16. The structural quasi static signal in the 70 N load.

25

30

35

10

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

Table 1 Strain distribution of Subarea 7 of plate structure applied on by 70 N load. Load location no. Strain (le)

1 348

2 204

3 290

4 138

5 54

6 21

7 10

8 68

9 155

10 153

Load location no. Strain (le)

12 51

13 28

14 9

15 302

16 167

17 321

18 103

19 50

20 34

21 14

methods of two kinds of damage and improve the system identification performance, multi-classifier decision fusion with majority voting is introduced for multi-agent fusion, where a decision is made according to the maximum value of the voting index. The agent fuses the results with majority voting, as shown in Eq. (1). Then it notifies the damage type and the final value to UIA. M X d ¼ max Y ij i2½1;N

! ð1Þ

j¼1

where N is the total number of the class, M is the total number of the classifier, Y ij 2 f0; 1g is the decision made by the jth classifier in relation to the ith class, and d is the result of fusion decision. 3.4. Experiment validation In order to verify the effectiveness of the multi-agent system, the strain change in Subarea 3 and 7 and the joint failure near the boundary are considered in Fig. 12. In SHM MAS, the coordination and fusion framework in Subsystem i is given in Fig. 13, and Strain Gauge SA 7, Fiber Optic SA and PZT SA are started up in the data monitoring layer and 5  2 classifier DEAs are also started up in the damage diagnostic layer for two subsystems. In the experiment, the 70 N load is applied to change the strain distribution in the plate. In Fig. 14, twenty-one possible load positions are numbered in two subareas. Hence, there are twenty-one classes for the strain change. The length of the moving average is 70 and 500 for the strain gauge and FBG sensor. For Strain SA, the strain changes of the four strain sensors are combined to be a feature vector with four dimensions. For each position, six samples are measured to train parameters of the classifiers. For the joint failure, the loosening of Bolt 2, 5, 9, 19, 23, 27, 33, 39, 43, 50, 54 and 63 are studied, as shown in Fig. 12. In the experiment, and tests are conducted with healthy and damage configuration which includes the completely and partially loose state of 12 bolts in different locations around the structure, and each time only one bolt is loosening. In order to quantitatively measure the loosen degrees of bolt, the tightening condition S is introduced and defined as

S¼

Ts  100½% T0

ð2Þ

where T s is axial tension of a tightening condition, T 0 is axial tension equivalent to 100% of tightening condition. 100% tightening condition is defined as the condition of that bolted joint is tightened to standard tightening axial tension. In our study, for not easy calibration of partially loosening bolt, and pattern overlapping obviously existence in our large structure since numerous structure joint bolts distribute densely, only the tight S = 100%, partially

11 106

loosening state S = 35%, 70% and completely loose state S = 0% of bolts are considered. Hence, there are thirty-seven classes for the bolt loosening. The excitation signal of PZT SA is the sine wave. The number of sampled data is 600, and the measured time is 0.06 ms. The frequency of the exciting sine wave is 100 kHz since with a higher excitation frequency the wave has a smaller wave length and is more sensitive, and lots of experiments [12] have also shows that the vibration response of the structure under this excitation frequency is sensitive to the bolt loosening. For sine wave excitation response, the experiment [12] shows that the peak change is obvious before and after the bolt loosening. In the experiment, PZT SA controls twelve PZT sensor circularly and periodically to excite and sense the structural dynamic strain signal. In Fig. 15, the signal of PZT3 as actuator exciting the sine wave with 9V and PZT 2 signals of four health and damage states of Bolt 8 are given. Twenty-four acquired signal peak changes of the twelve sensors on the plate border for PZT SA are combined to be a feature vector. For the chosen bolts and its different levels, twelve samples are measured to train parameters of the classifiers. When the 70 N load occurs in Position 11, Strain Gauge SA7 and FBG SA finds that the absolute sum of their sensors’ strain change exceeds to its threshold w, and the damage in the boundary is focused on. The collected quasi static signal waveforms of sensor Strain 1 and S33 are shown in Fig. 16. According to the finite element modeling analysis, the simulated strain responses w of four sensors under the 70 N load, which is respectively loaded on 21 positions in the sub-area or on the border, and the upper bound of the absolute sum of w in the sub-area is set to be the threshold. Then, the load experiment is adopted to verify the threshold. Finally, the threshold value is set to 68 le for the Strain Gauge SA, and 0.089 nm for the FBG SA, as shown in Tables 1 and 2. Next, five classifier agents are utilized to classify the calculated features of the stain distribution and the joint failure. The parameter of the classifier is saved in SIMA according to the subarea number. When the classifier works, it uploads the model parameter from SIMA in light to the damage type and the subarea number. The relevant parameters setup for these classifiers can be found in Table 3, in which SVM’s parameters are adopted by mean of Leave-One-Out Cross Validation. Table 4 gives the test accuracy of the five classifier agents. In the experiment, the classification accuracy is evaluated using a ratio of number of the samples classified correctly to the total sample. It can be seen that the classification accuracy of static load is better than the ones obtained from joint failure. The best classification accuracy for static load is 0.8571 and 0.8095 of SVM and k-NN agent. For joint failure, the accuracy of SVM agent is 0.7027. As far as performance of the five classifier agents in concerned, SVM and k-NN agent produces superior results and followed by GMM

Table 2 Strain distribution of Subarea 3 of plate structure applied on by 70 N load. Load location no. Central wavelength change (nm)

1 0.038

2 0.069

3 0.088

4 0.129

5 0.331

6 0.124

7 0.310

8 0.038

9 0.070

10 0.090

Load location no. Central wavelength change (nm)

12 0.077

13 0.091

14 0.040

15 0.033

16 0.058

17 0.077

18 0.116

19 0.282

20 0.141

21 0.207

11 0.101

11

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12 Table 3 Parameters of individual classifier. Classifier

SVM

Parameters setup

Kernel function: 2

kðx; yÞ ¼ ð0:7xT y þ 1Þ Euclidean distance type, penalty coefficient = 10

C4.5

k-NN

IIS

Percentage of incorrectly assigned samples at a node = 5

k=1

Number of iterations = 50

Table 4 Classification results and computation time. Data

Classifier agent SVM

C4.5

Fusion agent k-NN

IIS

GMM

Voting

Strain distribution Accuracy 0.8571 Time (ms) 4.310

0.2380 8.182

0.8095 0.002

0.1904 3.401

0.7619 19.147

0.9048 0.057

Joint failure Accuracy 0.7027 Time (ms) 9.628

0.2162 11.701

0.6216 0.002

0.4865 4.856

0.6216 11.331

0.7838 0.113

agent. IIS and C4.5 agents are not suitable in this work for static load and joint failure. It implies that the two types of classifiers do not fit for the scatter of training samples. However, in practice, it is almost impossible that all the predetermined classifiers will achieve the best performance at the same time. Otherwise, the fusion of a pure good or bad classifiers group may not necessarily improve the accuracy [24]. Therefore, IIS and C4.5 agents are still reserved for the fusion method. Table 4 shows the multi-agent

fusion with voting method is effective for static load and joint failure. In the experiment, the contract net interaction time is 4.99964 ms between SPA and five classifier agents. The computation time of the classifier agent is given in Table 4. It can be seen that the computation time of static load is less than the ones obtained from joint failure. The most computation time for static load is 19.147 ms of GMM agent. Except SVM, IIS and k-NN agent, the GMM and C4.5 agents’ computation time is more than 4.99964 ms. For joint failure, only IIS and k-NN agent’s one is less than the contract net interaction time. Hence, there may be the idle agent among five classifier agents. Considering the monitoring speed and two kinds of damage occurring uninterruptedly in different subareas, two neighborhood subsystems’ classifier DEAs are chosen, so MNB and TC are set to be 10 and 8 ms. In order to utilize every agent and improve the work efficiency, the introduced multi-agent contract net coordination method is very meaningful for the task distribution between two different subsystems and monitoring two kinds of damage occurring uninterruptedly in different subareas. At last, the user interface of the SHM MAS is shown in Fig. 17. In the figure, the damage monitoring history is demonstrated in the table, and the load location is labeled with the red box on the picture of the structure. The 64 lights represent the 64 bolts. When a light is turned on, its bolt is deemed to be loosening, and the meter control is adopted to denote the bolt loosening level. In our experiment, the multi-agent fusion with majority voting method is introduced. Hence, in the fusion principle, it is the same as the sensor fusion method. However, since different evaluation agents and fusion agent in our multi-agent system are of parallel, autonomous and not tightly-coupled, their computation speed

Fig. 17. The user interface of the SHM MAS.

Table 5 Comparison between sensor fusion and multi-agent method. Data

Classifier accuracy/time (ms)

Joint failure

SVM

C4.5

k-NN

IIS

GMM

Fusion accuracy/time (ms) Voting

Sensor fusion Multi-agent fusion

0.7027/9.950 0.7027/10.345 0.7027/9.825 0.7027/9.628

0.2162/11.802 0.2162/12.132 – 0.2162/11.701

0.6216/0.002 – 0.6216/0.002 0.6216/0.002

0.4865/4.949 – 0.4865/3.359 0.4865/4.856

0.6216/11.581 0.6216/10.947 – 0.6216/11.331

0.7838/0.124 0.6757/0.072 0.7568/0.096 0.7838/0.113

12

D. Liang, S. Yuan / Advances in Engineering Software 86 (2015) 1–12

and preciseness, the run state, and the coordination relation will directly affect the multi-agent fusion system’s real-time and accuracy, not affecting the single classifier’s time and preciseness. For instance, at different moments, each agent’s state is different, for which some are idle, some other are busy that is indicated by ‘–’, so, which agents take part in the fusion task varies as time goes on, and the final got result and the run time are fully different from the traditional centralized sensor fusion method. Comparison result between sensor fusion and multi-agent method is given in Table 5. It is shown that the preciseness of multi-agent method is the similar to the sensor fusion, and the dynamic distribution method saves the classifier recourse and time to different extent.

4. Discussion and conclusion The study has shown that a distributed SHM combined with multi-agent system offers the result of fast and precisely locating the position of static load and joint loosening for large structures. The accuracy at which the method performs illustrates that it is available in the distributed SHM systems. Its important advantage is that the technique self-organizes the sensor networks, coordinates different function modules and fuses the strengths of different evaluation methods with the distributed coordination for improving the preciseness and real-time of two damage location. Not like a typical centralized SHM system, it is comprised of a large number of sensors and the centralized data acquisition system. If the system addresses the stringent real-time operations, the system can become overburdened with computational tasks because its each component is not of parallel. Moreover, since its function modules are tightly-coupled, it is not useful for sufficiently and efficiently utilizing the distributed information from sensors to obtain the real-time and precise damage evaluation result. Experiments are carried out on a large aerospace aluminum plate structure with a loading equipment and a bolt loosening. It is shown that the point of static load and bolt loosening can be correctly predicted by our presented technique when a multi-point and synchronous sensor network and different evaluation methods are considered. Although the method is not guaranteed to give the true optimum, it is shown to produce encouraging results if compared with our previous studies. This paper shows that the multi-agent system can improve the strain distribution change and joint failure monitoring performance for the large structure. In the future work, the validation work of multi-agent system on the large scale composite material structure will be considered.

Acknowledgements This work is supported by National Natural Science Foundation of China (Grant Nos. 51405409), and the Fundamental Research Funds for the Central Universities.

References [1] Boller C, Buderath M. Fatigue in aerostructures-where structural health monitoring can contribute to a complex subject. Philos T R Soc A 2007;365:561–87. [2] Charles RF, Keith W. An introduction to structural health monitoring. T R Soc 2007;365:303–15. [3] Ren W, Cao YC. Distributed coordination of multi-agent networks. Berlin: Springer-Verlag Press; 2011. [4] McArthur Stephen DJ, Strachan Scott M, Jahn Gordon. The design of a multiagent transformer condition monitoring system. IEEE T Power Syst 2004;19:1845–52. [5] Lyell MJ, Krueger W, Zhang G, Xu R, Haynes L. An agent-based approach to health monitoring systems applied to the ISS power system. The 45th AIAA Aerospace Sciences Meeting and Exhibit. Reno, Nevada, USA; 2008. p. 1–17. [6] Ng Yew Seng, Srinivasan Rajagopalan. Multi-agent based collaborative fault detection and identification in chemical processes. Eng Appl Artif Intel 2010;23:934–49. [7] Abbott David, Doyle Briony, Dunlop John et al. Development and evaluation of sensor concepts for ageless aerospace vehicles NASA technical report NASA/ CR-2002-211773. Hampton, Virginia: Langley Research Center; 2002. [8] Esterline Albert, Gandluri Bhanu, Sundaresan Mannur, Sankar Jagannathan. Verified models of multi-agent systems for vehicles health management Smart Structures and Materials 2005: modeling signal processing and control, in proceedings of SPIE 5757; 2005. [9] Esterline Albert, Gandluri Bhanu, Sundaresan Mannur. Characterizing environmental information for monitoring agents. In: Hinchey MG et al. (Eds.), Innovative concepts for autonomic and agent-based systems. Springer; 2006. p. 74–85. [10] Yuan Shenfang, Lai Xiaosong, Zhao Xia, Xu Xin, Zhang Liang. Distributed structural health monitoring system based on smart wireless sensor and multi-agent technology. Smart Mater Struct 2006;15:1–8. [11] Zhao Xia, Yuan Shenfang, Yu Zhenhua. Designing strategy for multi-agent system based large structural health monitoring. Expert Syst Appl 2008;34:1154–68. [12] Zhao Xia, Yuan Shenfang, Zhou Hengbao, Sun Hongbing, Lei Qiu. An evaluation on the multi-agent system based structural health monitoring for large scale structures. Expert Syst Appl 2009;36:4900–14. [13] Hongbing Sun, Shenfang Yuan, Dong Liang, Weibo Yang. Structural health monitoring system based on multi-agent collaboration and case-based reasoning. Int J Adv Comput Technol 2012;4(20):470–8. [14] Lechleiter A, Bosse S. Structural health and load monitoring with materialembedded sensor networks and self-organizing multi-agent systems. Proc Technol 2014;15:669–91. [15] Qiu Zixue, Wu Jian, Yuan Shenfang. A wireless sensor network design and evaluation for large structural strain field monitoring. Meas Sci Technol 2011;22:1–7. [16] Qiu Lie, Yuan Shenfang. On development of a multi-channel PZT array scanning system and its evaluating application on UAV wing box. Sensor Actuat A – Phys 2009;151:220–30. [17] Saberi M, Azadeh A, Nourmohammadzadeh A et al. Comparing performance and robustness of SVM and ANN for fault diagnosis in a centrifugal pump. In: The 19th international congress on modeling and simulation; 2011. p. 433–39. [18] Kotsiantis SB. Supervised machine learning: a review of classification techniques. Informatica 2007;31:249–68. [19] Viswanath P, Sarma TH. An improvement to k-nearest neighbor classifier. Recent Adv Intell Comput Syst 2011:227–31. [20] Jin R, Yan R, Zhang J, Hauptmann AG. A faster iterative scaling algorithm for conditional exponential model. In: Proceedings of the 21th international conference on machine learning. Washington, DC; 2003. p. 282–89. [21] Hong QY, Kwong S. A genetic classification method for speaker recognition. Eng Appl Artif Intel 2005;18:13–9. [22] Daniel D. Corkill. Collaborating software: blackboard and multi-agent systems & the future. In: Proceedings of the international lisp conference. New York; 2003 [October]. [23] Zhang YP, Zhou LH, Mao E. Adaptive contract net protocol based on ant colony optimization algorithm. Appl Mech Mater 2012;109:666–70. [24] Petrakos M, Benediktsson JA. The effect of classifier agreement on the accuracy of the combined classifier in decision level fusion. IEEE T Geosci Remote 2001;39:2539–46.