Improved impact damage characterisation in CFRP samples using the fusion of optical lock-in thermography and optical square-pulse shearography images

Journal Pre-proof Improved impact damage characterisation in CFRP samples using the fusion of optical lock-in thermography and optical square-pulse shearography images Bernardo C.F. de Oliveira, Philipp Nienheysen, Crhistian R. Baldo, G. Armando Albertazzi Jr., Robert H. Schmitt PII: DOI: Reference:

S0963-8695(19)30503-1 https://doi.org/10.1016/j.ndteint.2020.102215 JNDT 102215

To appear in:

NDT and E International

Received date : 22 August 2019 Revised date : 11 November 2019 Accepted date : 2 January 2020 Please cite this article as: B.C.F. de Oliveira, P. Nienheysen, C.R. Baldo et al., Improved impact damage characterisation in CFRP samples using the fusion of optical lock-in thermography and optical square-pulse shearography images. NDT and E International (2020), doi: https://doi.org/10.1016/j.ndteint.2020.102215. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2020 Published by Elsevier Ltd.

Journal Pre-proof

Improved impact damage characterisation in CFRP samples using the fusion of optical lock-in thermography and optical square-pulse shearography images Bernardo C. F. de Oliveiraa,∗, Philipp Nienheysenb , Crhistian R. Baldoc , Armando Albertazzi G. Jr.a , Robert H. Schmittb a Federal University of Santa Catarina, Campus Universit´ ario UFSC, P. O. box 5053, Florian´opolis, SC, Brazil Aachen University, Chair of Production Metrology and Quality Management, Campus-Boulevard 30, Aachen, NRW, Germany c Federal University of ABC, Avenida dos Estados 5001, Santo Andr´ e, SP, Brazil

of

b RWTH

pro

Abstract

Image fusion methods with optical lock-in thermography (OLT) and optical square-pulse shearography (OSS) images are proposed to characterise impact damages in carbon fibre reinforced plastic (CFRP) plates. The samples were damaged with low-energy impacts and inspected using OLT, OSS and the reference ultrasound (US) time-of-flight C-scans. A total of 1113 combinations of decomposition, preprocessing, segmentation and fusion tools were proposed and compared with OLT, US and OSS results using the equivalent diameter criterion and the Matthews’ correlation coefficient. The results indicated a reduction of 72.21% in the equivalent

re-

diameter measurement error and a metric enhancement of 8.05% when using the fusion, showing that one of the developed image fusion methods can successfully perform improved impact damage inspections.

Keywords: Data fusion, Infrared thermography, Shearography, Impact, FRP (fibre reinforced plastic). used in the aerospace field, different ultrasound (US) techniques

urn al P

1. Introduction

are considered very reliable, although they can present issues

The replacement of metallic alloys for composite materials

related to their cost, the surface geometry and the mechani-

is an interesting alternative adopted in fields like aerospace and automotive for reducing the weight of components while pro5

20

ing it with the transducer. Optical lock-in thermography (OLT)

viding a mechanical behaviour comparable to metals. However,

and optical square-pulse shearography (OSS) are suitable for

composites like the carbon fibre reinforced plastics (CFRPs)

CFRP inspection, contactless and can be comparatively low-

are typically anisotropic materials, which means a high me-

cost. Their different sensitivities to the discontinuities gener-

chanical complexity of damages such as the ones caused by

impacts. This may lead to problematic scenarios, since many 10

25

ated by impacts, however, can lead to individually poorer detectabilities. Combining these techniques to achieve more reli-

impact damages can be difficult to detect visually, being called

able and more cost-effective results can be thus interesting [4–

barely visible damages (BVDs) [1–3].

7].

Jo

Non-destructive testing (NDT) methods are procedures that can enhance the capability of detection of BVDs. Each NDT

15

cal properties of the inspected part, and the need of contact-

Data fusion provides tools to combine data from two or more

technique has its particularities, being adequate to investigate 30

sources in a synergistic way. Some examples of data fusion ini-

only a fraction of all the possible abnormalities that compos-

tiatives with NDT methods can be given: in [8], the fusion of

ite materials may present. For CFRPs with impact damages

thermography images with other techniques such as shearography, radiography and US is shown, emphasising advantages like

∗ Corresponding

author Email address: [email protected] (Bernardo

C. F. de Oliveira ) Preprint submitted to NDT&E International

the complementarity of the techniques and the possibility of the 35

use of different loading methods; a new way to combine lockJanuary 5, 2020

Journal Pre-proof 2.1. Impact damage inspection and inspected samples

in thermography thermal phase images produced with different excitation frequencies using scatter plots is given in [9]; sev-

Composites are the union of two materials resulting in an out-

eral image fusion approaches with images from different NDT

come with two distinct phases: a reinforcement, which mainly

methods are described in [10], pointing out the enhancement 75

thermography is proposed in [11]. No data fusion approaches

and the transmission of mechanical loading between them [24].

comprising shearography and thermography for impact damage

Due to their anisotropy, there is a considerable rise in the com-

inspection were found.

45

plexity of their mechanical behaviour, which is especially in-

This work proposes a new, enhanced impact damage inspec-

tricate with any dynamic loading such as low-velocity impacts,

tion procedure to evaluate CFRP plates with different impact 80

capable of triggering several mechanisms and demanding spe-

energies from BVDs to perforation stages using image fusion

cific models to assess them. This topic is critical in the case of

procedures from OLT and OSS data, as an alternative to US

CFRPs due to their low impact resistance [1, 24, 25].

inspections. Optimal combinations of previously-tested image

Low-velocity impacts in CFRP laminates are related to de-

processing tools for filtering and segmentation of the abnormal-

fects such as matrix cracking, delamination, debonding and fi-

ities, such as the ones described in [12–21], are applied to OLT 85

bre cracking. All of these abnormalities are capable of hin-

re-

50

protect the reinforcements and to guarantee their positioning

of

of the complementarity; and a combination of holography and

pro

40

is responsible for the mechanical properties; and a matrix, to

dering the mechanical properties of the impacted body, but the

which are proposed [14]: (1) a binary fusion (BF) method, (2)

real danger of impacts is typically more related to the detec-

an algebraic fusion (AF) method and (3) a PCA-based fusion

tion difficulty of the damages than to their severity[1]. Heavy

(PCAF) method. Their fusion rules and other characteristics

loading defects are usually easy to detect and thus to repair or

urn al P

55

and OSS images before or after the three fusion procedures

are described. The results are evaluated using the equivalent 90

replace the damaged component. BVDs caused by impacts, es-

diameter (ED) [22] and the binary classifier Matthews’ correla-

pecially the ones capable of growing on duty, are more crucial

tion coefficient (MATT) [23].

to be revealed. The detection capability of these abnormalities

It has been shown that BF enhances the capability of impact

is enhanced with the use of an NDT method like thermography,

damage inspection since it is closer to the US inspections either 60

shearography, US, i.a. [2, 3].

analysing with ED and MATT in relation to the individual use 95

Considering CFRP plates with impact damages, US tech-

of OLT and OSS. This is a promising outcome since BF can be

niques are considered the most reliable technologies for in-

considered then an interesting alternative to many conservative

spection and, for many applications, seen as reference methods

industries which rely exclusively on US techniques.

[4, 6, 26]. US is extensively studied in many fields due to its good performance in NDT tasks, but it depends on the mechan-

100

2. Non-destructive testing procedures

70

needs to be in contact with the inspected sample. Active ther-

Jo

65

ical properties and the inspected surface. Also, in most cases, it mography and shearography are contactless, image-based tech-

This section starts with an introduction to impact damage in-

niques that are very known in the composite inspection field

spection applied to composite materials and a description of the

and can have lower costs when compared to US [3, 4, 27].

CFRP samples used in this work in subsection 2.1. Then, fun-105

Since they have different sensitivities when inspecting impact

damental aspects and the data acquisition procedures related to

damages in composites and this may lead to misinterpretations,

OLT are detailed in subsection 2.2, related to OSS in subsection

the fusion of such techniques can be interesting to provide a

2.3 and related to US in subsection 2.4.

more cost-effective and more robust CFRP assessment method. 3

Journal Pre-proof

110

Image fusion tools can be used in this sense with promising

f () is the error function. To fit the experimental and theoretical

outcomes to combine the image of these two sources in a syn-

curves and obtain αz , least-squares were used to minimise the

ergistic way [3, 5, 7, 14, 27].

term Er for all the τ temperature measurements in the expression (2):

The standard DIN 6603-1 (2000) [28] describes a procedure to perform impact tests with composite materials. The impact

Er =

tests responsible for the impact damages in the CFRP samples

120

impact damage in a controlled way. The drop-weight impact

where T meas is the temperature measured at a time t and T theo

test was conducted by impacting with a semi-spherical striker

is theoretical temperature at a time t for a thermal diffusivity

normal to the centre of the test specimen, which is clamped be-135

αz . The uncertainty of the final value of thermal diffusivity was

tween two metallic disks with central holes. The striker passes

calculated using the Bayesian approach described in [30].

through this hole and then hits the tested sample. The impact

2.2. Optical lock-in thermography

mass was (5.081 ± 0.022) kg. The impact energies were the following, in joules: 1.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0

2.2.1. Fundamentals of thermography

and 12.0, all with uncertainty of ± 0.2 J (95.45% of confidence).

re-

Thermography is a technique capable of producing images

The highest speed was (2.170 ± 0.019) m/s in the highest height.140

the heat diffusion inside material is directly related to the ther-

tection with NDT techniques is more challenging than higher

mal properties, which are modified with the presence of an ab-

energy ones.

normality [31]. This non-homogeneity can be detected on an

urn al P

130

based on the infrared spectrum of a scene. It relies on the fact

Low-energy impacts were prioritised in this work since their de-

The used samples were plates with 60.0 mm x 60.0 mm x

infrared image when the body is subjected to a heating proce-

2.0 mm made of seven layers of Huntsman bisphenol A epoxy145

dure, since the discontinuity may lead to a hot or a cold spot,

reinforced with TOHO Tenax carbon fibres (standard modulus,

depending on its characteristics. For impact damage, defects

twill 2x2, fibre 3k, 0/90) in the proportion 50/50. For each im-

cause the appearance of hotter regions on the image and by

pact energy, the four samples are named S1, S2, S3 and S4.

processing this image it is possible to efficiently characterise

A

thermal

diffusivity

for

the

samples

equal

the damage [4, 32].

to

(0.05976 ± 0.00051) mm2 /s in a direction normal of the

150

body to reach a constant average temperature, several images

impacts by heating the opposite surface with a higher, uniform

are captured during the measurement periods. By using a pixel-

temperature and then by measuring the temperature on four

wise FFT tool in this image sequence, a thermal amplitude im-

symmetrically-distributed points on the impacted face. These four sets of temperature in time were fit to the theoretical

155

age and a thermal phase image can be generated, as shown in figure 1. The thermal amplitude image is related to how much

function given by expression (1) [29]:

x T (x, t) = T s + (T i − T s ) f ( √ ), 2 αz t

For OLT, a sine wave is used to modulate the heating source.

After some conditioning periods for allowing the inspected

impacted surface was determined experimentally before the

Jo

125

(2)

k=1

of

in this work were performed based on this standard, to impose

τ X [T meas (k) − T theo (k, αz )]2

pro

115

v t

the regions are heated and the thermal phase image to the regional thermal delay imposed by the abnormality. Usually, the

(1)

thermal phase image is used more often since the defective re-

where T is the temperature at the time t in the distance x from160

gion has more contrast on it than on the thermal amplitude im-

the heated surface, T i is the initial temperature of the plate, T s

age. Yet, the use of modulated excitation makes easier to obtain

is the constant temperature of the uniformly heated surface and

images that are useful for defect characterisation especially due 4

Journal Pre-proof to the reduction of noise. Also, it is possible to obtain depth185

per loading frequency to consider repeatability.

information by changing the frequency of the modulation [7].

The infrared camera was calibrated to correct the distortion caused by the lens imperfections using captured images of a chessboard pattern. A tool, detailed in [33], was employed to

An Edevis OTvis 5000 system was used for image acquiring

determine the position in the space of the corners between its

and was composed of a FLIR SC5650 infrared camera with a190

black and white squares and to measure the distortion caused

spectral sensitivity from 2.5 µm to 5.1 µm, a spatial resolution

by the lens imperfections with a numerical model, allowing the

of 640 pixels × 512 pixels, a temporal resolution of 0.02 s, a

minimisation of issues such as aberrations.

mm. A lock-in halogen lamp excitation system is equipped with

2.3. Optical square-pulse shearography

two 1500 W halogen light sources in the reflection arrangement

2.3.1. Fundamentals of shearography

of

170

2.2.2. Data acquisition with optical lock-in thermography

integration time of 2.6 ms and a lens with focal length of 27

pro

165

at 250 mm was used as a loading system, but only 525 W on195 each lamp was needed.

laterally-displaced images, which allows one to calculate the

tests shown that five conditioning cycles per measurement were

derivative of the deformation field on a surface in a direction

sufficient to guarantee that the samples had their average tem-

parallel to the lateral displacement one. This method is sensi-

perature constant in time. After that, the image acquisition for200

tive to defects inside a body because these abnormalities may

the measurement was carried out with five cycles. The 17 fol-

affect the way this body deforms when subjected to a loading

lowing loading frequencies were applied, in hertz: 0.010, 0.020,

procedure (figure 2), such as heating. The presence of a defect

0.030, 0.040, 0.050, 0.060, 0.070, 0.080, 0.090, 0.100, 0.150,

generates a fringe pattern in the interferometric phase difference

0.200, 0.250, 0.300, 0.350, 0.400, 0.450. All those frequencies

map, which is the subtraction of an interferometric phase map

present an expanded uncertainty of 0.001 Hz with 95% of con-205

obtained before and after a loading procedure [5].

re-

Regarding the number of modulation cycles, preliminary

urn al P

180

ence of the displacements between the overlapped points of the

fidence. This procedure was repeated three times per sample

A single-shot shearography principle was used in this work

for generating interferometric phase images, which is described in [34]. With a special optical arrangement composed basically of an aperture and a wedge prism, this device is capable of pro-

Jo

175

OSS is an image-based NDT method that detects the differ-

Figure 1: A diagram with the lock-in thermography method showing the generation of thermal amplitude and phase images.

Figure 2: Fringes caused by the presence of a defect in an interferometric phase difference map.

5

Journal Pre-proof

210

ducing interference speckles using carrier fringes, separating

inspected object and different parameters can be measured from

the interferometric phase information in the frequency domain.

the waves which return, such as the amplitude, the frequency

The interferometric phase maps are thus computed through a

and the time of arrival [8].

two-dimensional Fourier transform method by filtering in the250

transducer and a processing unit, as can be seen in figure 3.

rection. This configuration does not demand any component

The pulser is responsible for generating high voltage electri-

such as moving mirror, being very compact and robust.

cal pulses, which are converted by the transducer into sonic waves. These waves are introduced into the material and travel

2.3.2. Data acquisition with optical square-pulse shearogra255

phy

whose magnitude depends on several characteristics of the dis-

D729MU PixeLink camera, with a CMOS sensor of 3840 pix-

continuity and the material. The reflected waves are the input

els × 2500 pixels, a pixel size of 2.4 µm × 2.4 µm and temporal

information of another transducer (or the same, depending on

resolution of 0.05 s, and (2) an optical arrangement with a lens with focal length of 3.5 mm, a wedge prism with a diameter

260

re-

a way that longitudinal waves are emitted by thickness oscilla-

plate with diameter of 25 mm with an aperture of 1 mm to pro-

tions. The received signal characteristics, such as amplitude,

duce carrier fringes, needed for the used shearography principle. A working distance was equal to (100.0 ± 5.0) mm between

shape and arrival time provide information about the geometry

265

which were reflected by discontinuities, provide different in-

a shear at the working distance of (8.15 ± 0.07) mm.

formation depending on how it is processed, i.e., the scanning

To allow fast real-time processing, an effective resolution of

mode. The most usual scanning modes are: (a) the A-scan,

2048 pixels x 2048 pixels was used, resulting in an effective

which is the output of a single location in terms of amplitude

temporal resolution of 0.2 s. During the measurements, 17 frames per measurement were acquired and three repetitions

270

duced by scanning a line (c) the C-scan, the result of scanning

of the process. Yet, the loading was performed with a halogen

an area, being the two-dimensional representation of a cross-

lamp with 300 W in transmission mode at 100 mm from the plates for 8 s.

As applied to OLT, the OSS camera was calibrated with a

section parallel to the part’s surface or visualising the time-of-

275

chessboard pattern and an image processing tool based on corfections.

2.4. Ultrasound time-of-flight C-scan

Ultrasound is one of the most used NDT methods, regarded

defects to be inspected are sub-surface imperfections or discontinuities including porosity, inclusions, and delaminations. It is 280

2.4.1. Fundamentals of ultrasound

245

flight (TOF) of the reflected waves (TOF C-scan) [6, 8, 35].

as a reference in many fields, especially in situations whose

ner detection to correct the distortion caused by the lens imper-

Jo

240

versus time; (b) the B-scan, being the two-dimensional representation of a cross-section vertical to the part’s surface pro-

of the procedure were performed to evaluate the repeatability 235

of the abnormality. The received sonic waves by the transducer,

urn al P

the shearography system and the inspected sample, resulting in

230

the configuration of the system), which converts and sends them to a processing or display unit. The orientation is chosen in such

of 25 mm and wedge angle of 3◦ to produce sheared images, a 225

through it. When abnormalities in the measurand structure are in the path of these waves, a part of the wave energy is reflected,

The image acquisition system was composed of a PL220

of

Fourier spectrum the desired component related to the shear di-

pro

215

A typical ultrasound device is composed of a pulser, a

also a reference because it has been used for many years and there is already a great number of studies regarding its perfor-

Ultrasonic techniques are based on the use of sonic waves

mance, which justifies its application in conservative fields in

above hearing level with frequencies higher than 20 kHz ap-

which the damage tolerance is low. However, US methods usu-

plied to the inspected material. These waves travel through the

ally have to balance these advantages with potential drawbacks 6

Journal Pre-proof

285

such as issues with the roughness of the inspected surface, with

and robustness. It can be classified into three types: pixel level,

the need of contacting this inspected object and with the speed.

feature level, and decision level [14, 36].

Specific techniques can avoid some of those issues, but they can

A general framework for image fusion in pixel-level, the vari-

be considerably more expensive [4]. In this context, an alterna-

ant used in this work, can be divided into registration and fu-

tive using the fusion of other NDT procedures is interesting.

315

sion. The first can be summarised in spatial alignment, temporal alignment, and radiometric calibration. The second is the

295

2.4.2. Data acquisition with US reference technique

of

application of the fusion rule itself that defines how the fusion

The US measurements were performed with an Olympus

procedure is performed. The spatial alignment comprises ac-

OmniScan MX2 UT device with the phased-array probe Olym-

tions to guarantee all the input images to be in the same spa-

pus 10L64-I1 with a mean frequency of 10 MHz consisting of320

tial coordinate system. Temporal alignment is similar to spatial

64 single transducers arranged in a line array. It was config-

alignment but in the temporal scale. The radiometric calibration

ured in a way that the sound beam was pulsed by a group of 16

is the normalisation of the pixel intensity levels of the input im-

active elements electronically moving from the 1st to the 64th

ages [14, 36, 37].

pro

290

element. The phased-array probe was connected to a uniaxial Olympus Mini-Wheel encoder providing lateral position infor-325

ticularities, advantages and disadvantages.

mation with a resolution of 12 steps per mm. The depth resolu-

can be given: algebraic operators [14], Boolean operators,

tion in z was equal to 0.015 mm and the measurement area was

Brovey transform [38], colour normalisation [39], multiscale

25 mm x 40 mm. The minimum focus depth was 0.5 mm and

algorithms such as wavelet decomposition (WD) and pyramid

the applied wave speed was calibrated to 3000 m/s.

transform [14, 40, 41], high-pass filtering [38], intensity-hue-

re330

3. Image fusion

Ehlers algorithm [39], i.a. Other methods can be seen in nice reviews in [43] and [44]. The tools which were chosen to be

Image fusion is the combination of images or information 305

Some examples

saturation method [41], principal component analysis [14, 42],

urn al P

300

There are several image fusion tools, with their own par-

used in this work will be described in section 4.

coming from images of the same or different sensors to gen-

erate a new image or data set producing a more complete or

4. Description of the employed methods

versatile view of the phenomenon which is investigated, preserving the most of the useful information on the input images without introducing artefacts. It leads also to the reduction of

In this section, the tools applied for image processing and

fusing images are described. An overview of the procedures

measurement uncertainty and the enhancement of confidence

is presented in subsection 4.1 and then they are detailed in the next subsections.

Jo

310

335

4.1. Overview of the employed methods 340

Several image processing and image fusion methods were tested and some of them were considered to have the highest potential for the proposed inspection task. Table 1 presents their names and acronyms. Figure 4 shows the flowchart representing the interaction be-

Figure 3: The operation of a generic US device.

345

7

tween the methods named in table 1.

Journal Pre-proof Procedure objective Dimension reduction Component extraction

Segmentation

Fusion Evaluation

pro

of

Preprocessing

Procedure name Linear interpolation (LI) Laplacian Pyramid (PYR) Two-dimensional WD for spatial registration (WDSR) No component extraction (NCE) Principal component analysis (PCA) No preprocessing (NPP) Low-pass filtering using two-dimensional Fast Fourier Transform (2DFFT) Absolute thermal contrast (ATC) Contrast enhancement (CE) Two-dimensional WD for low-pass filtering (WDLP) Otsu’s global segmentation (OTSU) Bradley’s adaptive segmentation (BRAD) Otsu’s global segmentation and morphological operators (GSMO) Multithreshold (MT) Extended minima and maxima transforms (EXMMT) Fast march method (FMM) Boolean image fusion of binary images (BF) Algebraic image fusion (AF) Principal component analysis fusion (PCAF) Equivalent diameter (ED) Matthews correlation coefficient (MATT)

Jo

urn al P

re-

Table 1: The image processing and image fusion methods employed in this work.

Figure 4: Flowchart of the proposed fusion procedures.

8

Journal Pre-proof After the descriptions of all methods, the number of combi-

loaded state and an unloaded state [5]:

nations that could be achieved with them are detailed in sec∆Φ(x, y) = Φal (x, y) − Φbl (x, y),

tion 4.9. 365

4.2. Calculation of thermal phase images

where Φal (x, y) and and Φbl (x, y) are interferometric phase maps after and before the loading application, respectively.

The thermal phase image generation was performed using a

The filtering procedure is a multiple pass sine-cosine tool de-

of

pixel-wise one-dimensional Fast Fourier Transform (FFT) tool

tailed in [4], which calculates in each iteration the sine and

with the OLT images, as described in [7, 32, 45], considering

cosine images of ∆Φ(x, y), applies to both of those images a

each pixel position (x, y) an one-dimensional temperature signal

two-dimensional Gaussian filter [21] with sigma equals to 2 and

pro

in time t. Assuming that the thermograms can be interpreted

with a kernel with dimensions 3 × 3 and them computes the arc-

as a three-dimensional matrix I(x, y, t), this tools calculates an

tangent image I f iltered (x, y) to merge those images according to

amplitude image A(x, y, f ) and a phase image Φ(x, y, f ) with the

expression (6):

real and imaginary parts of Y(x, y, f ). The result of the pixelwise FFT is given by:

I f iltered (x, y) = arctan

q Re[Y(x, y, f )]2 + Im[Y(x, y, f )]2

and

350

where I sin, f ilt (x, y) and Icos, f ilt (x, y) are the filtered sine and cosine images, respectively. A number of iterations equal to five was used in this work.

(4)

370

The phase unwrapping tool used was the one described in

[20] and is based on the reliability of the pixel, i.e. it considers

ing frequency, are the amplitude and phase images related to the

more important those points with the lowest module 2π gra-

studied phenomenon. As detailed in [46], Φ(x, y, fload ) is more

dients in relation to their neighbours, thus using them first in

favourable than A(x, y, fload ), being thus the images used in this

a non-continuous queue of priority to perform the unwrapping

375

4.3. Computation of interferometric phase difference images

operation. Then, the secondary fringe attenuation removes the gradient

present on the image background due to the CFRP sample heat

The filtered interferometric phase difference map genera-

dilation. If the unwrapped optical phase difference map is seen

tion starts from the interferometric phase maps obtained as ex-

as three-dimensional data, considering thus that the grey levels

plained in section 2.3 and then includes the following steps:380

of the pixels represent heights, the background gradient is usu-

(1) the generation of interferometric phase difference map; (2)

ally something like a ramp. Thus, a plane is fitted using three

the iterative sine-cosine filtering; (3) the interferometric phase

points of the background and then subtracted to remove this

unwrapping; and (4) the secondary fringe effect attenuation us-

background effect. A similar procedure can be found in [47].

Jo

360

(6)

A(x, y, fload ) and Φ(x, y, fload ), with fload being the lock-in load-

work.

355

Im[Y(x, y, f )] . Re[Y(x, y, f )]

(3)

urn al P

Φ(x, y, f ) = tan−1

I sin, f ilt (x, y) , Icos, f ilt (x, y)

re-

A(x, y, f ) =

(5)

ing a plane fitting procedure. In all these steps, it is important

It is important yet to state that the lateral displacement of the

to manage these images in radians, not in 8-bit depth intensity385

OSS images was not compensated, since the fusion, preprocess-

levels.

ing and segmentation tools in sections 4.8, 4.7 and 4.7, used to

The expression 5 shows the calculation of interferometric

characterise the damaged area of the CFRP samples, were opti-

phase difference map ∆Φ(x, y) by subtracting the map from a

mised to not require any action in this sense. 9

Journal Pre-proof 4.4. Spatial image registration and dimension reduction 390

415

of the grey level of the pixels in the nearest 4 × 4 neighbour-

hood [21].

An automatic spatial registration tool was developed to put all OLT and OSS images in the same coordinate system, using

PYR computed a Gaussian pyramid to reduce an input image

the CFRP sample itself as reference. This tool was based on

by one level [19], resulting in an image with half the resolution

intensity gradient and was enough to find the sample position

of the original image. WDSR reduced images with a pyramidal algorithm based

on the image, despite the amount of noise present in OLT and420

on convolutions with quadrature mirror filters [13]. One iter-

of

OSS images.

ation of this tool is capable of decompose the input image in

grouping the grey level values of N lines in vectors l(i) for each

the following images, all of them with half the resolution of

edge. The tool considered that a border was found when the

the original one: (1) one approximation image, containing low-

condition l(i + 1) − l(i) ≥ ∆g was met, where ∆g is the inten-425

frequency information of the input image; (2) one vertical detail

sity difference which defines a border, defined experimentally

image, containing the high-frequency information of the input

in previous tests. The group of points (xk , yk ) of each edge of

image in the vertical direction; (3) one horizontal detail im-

the sample was used as input in a least-squares algorithm to find

age, containing the high-frequency information in the horizon-

the best slope a1 and constant a0 that represented the edge on

tal direction; and (4) one diagonal detail image, containing the

the image by minimising ρ in expression (7):

pro

The corners of the sample on the images were found by

re-

395

430

high-frequency information in the direction at 45◦ . The approx-

imation image was further processed following the procedures

ρ=

N X k=1

[yk − (a1 xk + a0 )]2 .

described in table 1. A number of decomposition levels equal to

(7)

one was necessary to match the OSS approximation image dimensions from the OLT image ones and the Haar wavelet was

intersections may have been computed and then the corners of435

employed.

urn al P

With the four lines which represent the edges calculated, their the sample on the image were found.

4.5. Component extraction

Iterative affine geometric transforms with Mattes mutual in400

formation as similarity metric and one-plus-one evolutionary

Component extraction can be interesting to isolate the most

method as optimisation technique[48] were used to put the im-

important information in one image. In this sense, two alterna-

ages in a coordinate system with origin in the sample superior

tives were used: NCE and PCA.

left corner, x positive to the right and y positive downwards.

410

PCA is a multivariate statistical analysis which reduced the

Matching the resolution of OSS and OLT images was the last

dimensionality of a dataset with many interrelated variables but

stage of the spatial registration needed to perform fusion proce-

keeping as much as possible information by generating the prin-

dures in pixel level, since the resolution of the OSS images is

cipal components of this dataset [49]. For OSS, a time series of

higher than the one of OLT images [14]. As table 1 shown,

images was used to compute component images and, for OLT,

three methods of dimension reduction for images were tested:445

the PCA was applied to a frequency set of images, resulting

(1) LI, (2) PYR and (3) WDSR.

in component images with specific variabilities depending on

Jo

405

440

these frequencies.

LI was a tool which resized an input image according to a desired output image size, in this case the OLT spatial resolution

To generate the component images, it was necessary to per-

in x equal to 326 pixels and in y also equal to 326 pixels. The in-

form a pixel-wise grey level analysis, in frequency for OLT and

terpolation method was the bicubic interpolation, in which the450

in time for OSS. The series of images were rearranged to a two-

output grey level of a pixel (x, y) value was a weighted average

dimensional matrix A in which one dimension was the product 10

Journal Pre-proof

455

of the spatial resolutions in the directions x and y and the other

suming a Gaussian distribution. The resulting binary image was

was their behaviour in frequency or in time. Yet, to consider475

inverted and then multiplied by the original Fourier plane, filter-

only the variation of the grey levels of the pixels, the average

ing it. An inverse two-dimensional FFT transform was applied

grey level of each pixel in time or frequencies was calculated

and the result was then a filtered image.

and a matrix B was generated by subtracting each column of

ATC is widely used for processing pulse thermography im-

matrix A from its average value.

ages. The objective of ATC is the comparison of the grey level of each pixel, which is related with the temperature of each

variance matrix C relative to matrix B that satisfy the expres-

point, with the average grey level of a k × k defect-free zone to

of

The principal components are the eigenvectors vi of the co-

make it possible to attenuate the effect of non-uniform heating

sion (8): Cvi = (B B)vi = Vi vi

and to remove some high frequency components [7, 32]. The

(8)

pro

T

expression (10) summarises the use of this tool:

where Vi is the eigenvalue relative to the vi eigenvector. The resulting dimension of the matrix C was very large and difficult

T ac (x, y) = T pixel (x, y) − T sound

to be processed. The authors of [42] proposed a solution to this problem, which was to work with (BBT ) instead of (BT B),

with

re-

obtained with mathematical manipulations and resulting in a

smaller matrix. After reducing the number of eigenvectors to n,

T sound

a matrix D with dimensions n × (R x × Ry ), in which R x is the

expression (9):

trast image, T is the grey level of a pixel and (xd f , yd f ) is the

urn al P

where T ac is the grey level in (x, y) of the absolute thermal con-

D=V A

(9)480

OSS images, as shown in [32] as an absolute phase contrast.

m is the dimension of the n eigenvectors. D was rearranged,

generating the n component images which represent the most

changing the grey levels of the image to new ones in such a way that 1% of the data was saturated at low and 1% at high intensities of the input data. The first step was to find the grey

As seen in table 1, the alternatives explored as preprocessing

level value which corresponds to 1% gl0.01 and to 99% gl0.99

tools in this work were five: (1) NPP, (2) 2DFFT, (3) ATC, (4) CE and (5) WDLP.

Regarding CE, there were different ways to enhance the con-

trast of images. The one chosen for this work was done by

useful information present on the input image series. 4.6. Preprocessing tools

initial coordinates of the defect-free zone. An analogous approach from the provided by equation (10) could be thought to

where V 0 is a n × m matrix of stacked eigenvectors in which

465

k−1 X k−1 X T [xd f + i − 0.5(k − 1), yd f + j − 0.5(k − 1)] = k2 i=0 j=0

resolution in direction x and Ry in y, could be calculated with 0

460

(10)

in the cumulative distribution function of the image histogram. These grey levels were then adjusted to the 8-bit range with a

The 2DFFT started with the calculation of the Fourier plane

Jo

look-up table ruled by the expression (11):

of frequency components. It was described in [12] a way to generate a Fourier plane based on the two-dimensional infor-

Iout (x, y) =

mation present on the image showing the frequency informa470

Iin (x, y) − gl0.01 gl0.99 − gl0.01

(11)

where Iout is the output image and Iin is the input image.

tion. Bradley’s adaptive threshold [16] identified in the Fourier plane the frequency component which are more relevant, i.e.

Finally, WDLP is a tool similar to the one described in sec-

the brighter ones. The parameters used in this tool were the485

tion 4.4. In this case, the decomposition was performed on three

sensitivity factor of 0.7 and the dark foreground polarity, as-

levels. This number was chosen based on the relation between 11

Journal Pre-proof

age without changing significantly the form of the defect, which

defective regions.

may have led to errors when measuring the defective areas. Dif-

MT was a n-level multi-threshold segmentation tool based on

ferently from the other preprocessing methods, in this case, the

the Otsu’s global segmentation method. It used a search-based

WD was done in one way, the segmentation method was per-

optimisation method to find locally the thresholds, as described

formed and then the inverse WD was done. The detail images520

in [21]. The number n was adjusted according to the input im-

for the reconstruction were ignored since they did not bring

age, being equal to 3 for OLT images and 4 for OSS images.

of

495

the black lobe and the white lobe must have been segmented as

too much useful information for the output image and then null

EXMMT was the binary union between the images resulting

matrices were used instead. Since high-frequency components

from the extended-minima and the extended-maxima transform

were removed, this application worked as a low-pass filter in

images, which are the regional minima and maxima of the H-

the frequency domain.

minima and the H-maxima transforms, respectively, as seen in

pro

490

keeping the most of the information in the approximation im-515

the expression (13):

4.7. Segmentation tools Following what table 1 presented, six methods of segmenta500

tion were employed: (1) OTSU, (2) BRAD, (3) GSMO, (4) MT,

re-

ima image and Iextmin is the input extended minima image. H-

method, described in [15]. The goal of this tools is to maximise

extrema transformations can filter the image extrema using a

the inter-class variance looking for a value t that minimises the525

contrast criterion, which means that the h-maxima suppresses

intra-class variance, i.e. to find a t which considers that in the

all maxima whose depths are lower than a threshold t and the

image intensity histogram there are two peaks and there is an

h-minima suppresses all the minima below t. The threshold t

optimal t that splits these peaks.

used in this work was 80, obtained experimentally. The use of

urn al P

OTSU was the direct use of Otsu’s global segmentation

Then, BRAD was the direct use of Bradley’s adaptive seg-

both extrema for such application was also due to the presence

mentation tool [16] with a sensibility coefficient of 0.7, a dark530

of black and white lobes, which means that this tool was meant

foreground polarity and assuming Gaussian distribution for the

to have better results in shearography images [50].

evaluation.

Yet, the FMM joint method proposed in this work was also a

binary union, but of two images resulting from the fast march

GSMO was the joint use of global segmentation of Otsu [15]

method, as shown in the expression (14):

and the morphological opening [21] with a structuring element. The morphological operation of opening, indicated with the

symbol ◦, is the procedure to smooth the borders of objects and to eliminate fine saliencies [21]. The image and its nega-

Iout = (Iin ◦ S disk ) ∪ [ (255 − Iin )2 ◦ S disk ]

(14)

resulted from the fast march method applied to the darkest point

procedures, as described by the expression (12): p

Iout = I f mmblack ∪ I f mmwhite

where Iout is the output image, I f mmdark is the input image which

tive were combined after the global segmentation and opening

Jo

510

(13)

where Iout is the output image, Iextmax is the input extended max-

(5) EXMMT and (6) FMM.

505

Iout = Iextmax ∪ Iextmin

on the image and I f mmwhite is the input image which resulted (12)535

from the fast march method applied to the brightest point on the image. The seed locations were logical arrays, one with one

where Iout is the output image, Iin is the input image and S disk

true value approximately in the centre of the darkest point and

is the structuring element ’disk’ of size 7. The union gives the

another with the centre in the brightest point. The threshold

hint that this tool was designed for OSS images, in which both

level was 0.01. The weights were the absolute values of the 12

Journal Pre-proof

550

such decomposition, two stages were used with both OLT and

of the points in the central positions of the seed locations [18].

OSS images after matching their dimensions, independently

OLT with NCE and OSS with NCE techniques reached this

from the used dimension reduction. This number of decompo-

segmentation stage with all the filtered lock-in frequency im-

sitions was used as a commitment between the filtering effect

ages or filtered images taken after the heat pulse, respectively.

and the information loss caused by the dimension reduction.

For this reason, after the application of the segmentation tools

For OSS images in all AF methods, it was necessary to turn

described in this section, the 17 frequency images of OLT

the black lobes of the defects into white lobes during the appli-

were merged using the pixel-wise Boolean operator OR and

cation of the fusion rules, since these regions have low-intensity

the 17 interferometric phase difference images of OSS were

values that were difficult to segment together with the natural

also merged using the same operator, generating one OLT seg-

white lobes, and this was done by applying expression (16):

of

545

differences between the intensities of the pixel and grey value570

pro

540

mented image and one OSS segmented image for each combi-

Iout = Iin Mmax,in + Icomp Mmax,comp

nation of image processing tools. 4.8. Fusion methods

575

where Iout is the output image, is the element-wise multipli-

cation operator, Iin is the input image, Icomp is the 8-bit com-

The three fusion methods employed in this work were: (1) BF, (2) AF and (3) PCAF. They are described in the following

plement image of Iin , Mmax,in is the binary image in which

sections.

only the pixels (x, y) where Iin (x, y) > Icomp (x, y) are ones, and

re-

555

(16)

Mmax,comp is the binary image in which only the pixels (x, y)

4.8.1. BF

where Icomp (x, y) > Iin (x, y) are ones.

The first fusion method is the BF and, as seen in figure 4, this

The basic fusion rules applied to AF were: the maximum rule

was the only procedure which entirely processed the images

(MAX), the minimum rule (MIN), the mean rule (MEAN) and

before the fusion itself. This was interesting because each opti-

the sum rule (SUM).

urn al P

560

580

mised segmentation can be directed to the particularities of each

MAX was defined by expression (17):

OLT and OSS images, but it was also more time-consuming.

The first step was to match the OLT and OSS images by reducing the segmented OSS images using LI, PYR or WD. The fusion rule is given by the expression (15):

n X [(IOLT,i Mmax,OLT,i ) + (IOS S ,i Mmax,OS S ,i )], (17) i=1

where Iout,max is the output fused image, n is the number of im-

585

Iout = IOLT seg ∪ IOS S seg

Iout,max =

(15)

ages, IOLT,i is the i-OLT frequency image, IOS S ,i is the i-OSS

temporal image, Mmax,OLT,i is the binary image related to the i-OLT frequency image in which only the pixels (x, y) where

where Iout is the output image, IOLT seg is the segmented OLT

IOLT,i (x, y) > IOS S ,i (x, y) are ones, and Mmax,OS S ,i is the binary

Jo

image and IOS S seg is the segmented OSS image.

In figure 5 an example of BF fusion result is given. 565

4.8.2. AF

In AF procedures, the image processing tools were applied after the fusion. Also, since WD can provide more possibilities of fusion combinations with the use of detail images, the OLT

Figure 5: An example of BF. In (a), the OLT image. In (b), the OSS image. In (c), the fused image.

images can also be decomposed, named WD fusion (WDF). For 13

Journal Pre-proof of OSS and OLT detail images using the maximum fusion rule;

pixels (x, y) where IOS S ,i (x, y) > IOLT,i (x, y) are ones.

and (6) minimum details, using the fusion of OSS and OLT detail images with the minimum fusion rule as detail images.

Then, MIN was described by expression (18):

i=1

In figure 6 an example of AF fusion result is given. [(IOLT,i Mmin,OLT,i ) + (IOS S ,i Mmin,OS S ,i )], (18)

where Iout,min is the output fused image, n is the number of im-

4.8.3. PCAF

used to fuse images in this work, the only difference was that the

ages, IOLT,i is the i-OLT frequency image, IOS S ,i is the i-OSS

matrix of dimension-reduced images was organised with both

temporal image, Mmin,OLT,i is the binary image related to the

OLT and OSS images of each sample and repetition. The eigen-

i-OLT frequency image in which only the pixels (x, y) where 595

vectors were then calculated in the same way. The outcome was

IOLT,i (x, y) < IOS S ,i (x, y) are ones, and Mmin,OS S ,i is the binary image related to the i-OSS temporal image in which only the

625

other images were more related to OLT. Thus, the first com-

MEAN was defined by expression (19):

i=1

(

IOLT,i + IOS S ,i ), 2

ponent image was the only one used in further image processing procedures. Ninety images resulted from this method with

re-

Iout,mean =

(19)

630

figure 7 an example of PCAF fusion result is given.

urn al P

images, IOLT,i is the i-OLT frequency image and IOS S ,i is the i-OSS temporal image.

4.9. Combination of methods

Finally, SUM was described by expression (20): Iout,sum =

all the combinations of decomposition, component extraction, preprocessing and segmentation tools, and the fusion rules. In

where Iout,mean is the output fused image, n is the number of

600

that the first component was the most significant one, carrying most of the information of both OLT and OSS techniques. The

pixels (x, y) where IOS S ,i (x, y) < IOLT,i (x, y) are ones.

n X

The procedure similar to the one described in section 4.5 was

620

of

Iout,min =

n X

pro

590

image related to the i-OSS temporal image in which only the615

n X (IOLT,i + IOS S ,i ),

635

(20)

As pointed out earlier, several combinations could have been

made considering the presented image processing and image fusion tools. Table 2 presents the numbers of combinations pro-

i=1

vided by each stage.

where Iout,sum is the output fused image, n is the number of images, IOLT,i is the i-OLT frequency image and IOS S ,i is the i-OSS temporal image.

With AF, however, WDF could be applied and thus both OLT 605

and OSS images were decomposed in consequence of the possibility of using detail images as fusion alternatives. Six new

Figure 6: An example of AF fusion, with SUM as the fusion rule. In (a), one of the OLT images. In (b), one of the OSS images. In (c), the fused image.

Jo

image fusion methods based on changes on detail images were applied: (1) zero-detail, in which the detail images were replaced with null matrices; (2) original shearography details, us610

ing only the detail images of OSS decomposition; (3) original thermography details, using only the detail images of OLT decomposition; (4) mean details, in which the detail images were the fusion of OSS and OLT detail images using the mean fusion

Figure 7: An example of PCAF fusion. In (a), one of the OLT images. In (b), one of the OSS images. In (c), the first component image.

rule; (5) maximum details, so the detail images were the fusion 14

Journal Pre-proof

tions of tools. For generating the BF images, only the best com-

ied between 80 and 100% full screen height (FSH). Therefore,

bination of tools for OLT and OSS were used, resulting in only

the fixed threshold for detecting the front side echo was defined

3 alternatives depending on the decomposition method used for

to 80% FSH. The value of the defect echo threshold between

matching the dimension of OSS and OLT images. AF resulted665

the front and the rear side has been applied as exponentially

in 900 combinations considering the 4 fusion rules applied to

decreasing between the front and the rear side thresholds.

LI images, 4 to PYR images and 22 applied to WDSR due to

Regarding peak detection itself, the front side echo was de-

the alternatives provided by the detail images. Finally, PCAF

fined as the first local maximum in each A-scan exceeding the

also gives 90 possible outcomes. Summing all the combina-

front side reference threshold. The rear side echo, however, was

tions given by each NDT method, a total of 1113 combinations670

a local maximum exceeding the rear side reference threshold in

could be obtained.

an expected region known from the thickness of the plate and the position of the front side echo. A decreased rear side echo

4.10. Ultrasound data processing

660

erence damaged areas. The TOF C-scans were generated by675

ceeding the exponential threshold curve. If no defect has been

applying a peak detection method on the US raw data, which is

detected, the position of the rear side echo was then mapped on

described as follows, and mapping the defect depth information

a colour scale to form the TOF C-scan by a two-dimensional

in each A-scan on a colour scale, shown in figure 8(a).

representation of size and depth of the detected defects.

re-

was localised by the position of the first local maximum ex-

The peak detection comprised three steps: (1) the detection

For comparing the depth information of the TOF C-scans

and localisation of the front side echo, (2) of the rear side echo680

with the thermography data the TOF C-scans have been con-

and (3) of the defect echo. For the amplitudes of both front side

verted to 8-bit to generate the grey level scans by normalising

and rear side echoes, fixed thresholds have been calibrated us-

the depth range into the grey level 8-bit intensity range, as pre-

ing US data of undamaged reference plates, e.g. due to incon-

sented in figure 8(b). The resulting images were then binarised

stant coupling conditions in manual ultrasound measurement

with the Otsu’s global segmentation tool [15], as shown in fig-

urn al P

655

was seen as an indicator for a defect, while the defect position

Ultrasound TOF C-scans of each sample were used as ref-

685

Table 2: Combinations of image processing and image fusion tools.

Combination stage Decomposition Component extraction Preprocessing Segmentation Fusion rule Total

OLT 2 5 6 60

NDT methods OSS BF AF 3 2 5 5 6 6 1 30 60 3 900

ure 8(c). With this binary representations, those images have been used as references with MATT and their ED have been calculated. More information regarding the evaluation tools is

PCAF 3 5 6 1 90

Jo

650

of

645

the amplitude of the front side echo in undamaged plates var-

pro

640

OLT and OSS provided each one with 60 possible combina-

given in section 4.11. 4.11. Evaluation tools

690

The first criterion was MATT, good for unbalanced binary

classes. Its outcome is a value between -1 and 1 [51]. The metric was used in two opportunities: (1) to evaluate the performance of the image processing tools, determining which was the optimal segmentation for each method using then manually 695

Figure 8: Sequence of the processing procedures with the ultrasound data with examples of images at each stage. In (a), the TOF C-scan. In (b), the result of the conversion to 8-bit. In (c), the result of the segmentation.

segmented images as references; and (2) to evaluate the best methods determined in (1) in relation to the reference method, the TOF C-scan US measurements. This approach was adopted

15

Journal Pre-proof

700

because the best methods must be defined according to the best735

be associated with the good performance of the preprocessing

that they can retrieve from the images of each NDT technique.

tools and with the division of information among the principal

To compare the segmented image without determining the best

components. Yet, it is shown by the difference of OSS-NCE

methods could lead to mistakes due to false positives and false

and OLT-NCE that, for the proposed image processing tools, it

negatives of methods with poor performances. Yet, t-tests were

is more difficult to segment defects on OSS images than OLT

used to attest that the best methods are significantly better than740

images, mainly due to the higher amount of noise.

the other.

of

ages of OLT and OSS resulting from the combinations pointed

mm limit for diameters of damages in composites commonly

out in the last paragraph were used. Now the dimension reduc-

adopted in the aerospace field as damage criterion for several

tion tools are also being compared. In figure 12, the MATT

aircraft parts. ED is the diameter of a circle with the same area745

outcomes regarding the average outcomes of all impact energy

of the detected damage on the image. It is a similar concept than

sets applied are shown. The results from this fusion method are

the one used in fluid dynamics to calculate flows in tubes [22].

indeed good, with a high MATT outcome. BF with WDSR as

When ED resulted in a diameter lower than 6 mm, the defect

dimension reduction tool was the best method, showing that al-

would be considered admissible [52–55].

though LI, PYR and WDSR dimension reduction methods gave

pro

710

As explained earlier, for generating the BF images, the im-

The second evaluation method, ED, is related to the 6

re-

705

Figure 9 shows examples of metric results applied to images of a sample with an impact of 9 J: in (a), an OSS image; in (b), 715

an OLT image; in (c), an BF image; in (d), an AF image; in (e), an PCAF image; and in (f), an US image. The metric results

5. Results and discussion

urn al P

are shown in the upper right corner of the images.

In this section, the results concerning all the samples of this 720

work are presented.

Figure 10 shows the results of MATT evaluations regarding the average outcomes of all impact energy sets for the combinations OSS-NCE (black columns) and OSS-PCA (white columns). 725

MATT results regarding the average outcomes of all impact energy sets related to OLT-NCE (black columns) and OLT-PCA

Jo

images (white columns) are shown in figure 11.

As shown by those figures, the combination of NCE, WDLP with MT was the best method for OSS images and, for OLT, 730

the preprocessing with 2DFFT combined with the segmentation with BRAD also without component extraction provided the best result for impact damage characterisation for the proFigure 9: Examples of metric results applied to images of a sample with an impact of 9 J: in (a), an OSS image; in (b), an OLT image; in (c), an BF image; in (d), an AF image; in (e), an PCAF image; and in (f), an US image.

posed impact energies. PCA did not give an outcome comparable to NCE when processing the OLT images, which can 16

Journal Pre-proof

NCE

0.400

PCA

0.200

E VI

EV

E IV

E III

E II

EI

DV

D VI

D IV

D III

DI

D II

C VI

CV

C IV

C II

C III

CI

B VI

BV

B III

B IV

B II

BI

AV

A VI

A IV

A III

A II

0.000 −0.200

AI

MATT

0.600

E VI

EV

E IV

E III

E II

EI

DV

D VI

of D IV

D III

DI

D II

C VI

CV

pro

PCA

C IV

C II

C III

CI

B VI

BV

B III

B IV

B II

BI

AV

A VI

A IV

A III

NCE

A II

0.800 0.600 0.400 0.200 0.000 −0.200

AI

MATT

Figure 10: MATT results for OSS using as reference the manually-segmented images. Preprocessing methods: (A) NPP, (B) 2DFFT, (C) ATC, (D) CE and (E) WDLP. Segmentation methods: (I) OTSU, (II) BRAD, (III) GSMO, (IV) MT, (V) EXMMT, (VI) FMM.

MATT

0.760

re-

Figure 11: MATT results for OLT using as reference the manually-segmented images. Preprocessing methods: (A) NPP, (B) 2DFFT, (C) ATC, (D) CE and (E) WDLP. Segmentation methods: (I) OTSU, (II) BRAD, (III) GSMO, (IV) MT, (V) EXMMT, (VI) FMM.

765

LI in black, PYR in grey and WDSR in white, also in relation to

0.740

the average value of all impact energy data sets. The best pro-

0.720

cedure was the combination of WDLP and MT, with the com-

0.700 PYR

WDSR

urn al P

LI

ponent extraction using LI. This was also an interesting result

Figure 12: MATT results for BF using as reference the manually-segmented770 images.

750

tion coming from the OSS images was present in the outcomes that the best image processing combination was similar to the best for OSS images.

volving which fusion rule was the most appropriate in the task775

Figure 15 shows MATT results of PCAF regarding the av-

of defect characterisation, considering yet the different rules

erage outcomes of all impact energy sets. WDSR with WDLP

for WDF given by the interactions of approximation and de-

and MT was the best method for characterising impact damages

tail images. These results are realted to the average outcomes

in images from this method. High variability was present in the

of all impact energy sets. The combinations of different approx-

results, even for repetitions with the same sample. This hap-

imation images and different detail images are denoted with an780

pened probably because of the characteristic of the PCA usage,

acronym in the format ’aXdY’, where X and Y are the fusion

in which the division of useful information between the compo-

rules applied to the approximation images and the detail im-

nents can be different depending on the input images and then

ages, respectively. One notices that MIN was the best fusion

even small variations may lead to differences in what is shown

rule for LI, PYR and WDSR, and the combination LI-MIN was

in each component. Only the first component image was used,

Jo

760

because it can be observed from the results that more informathan OLT information. In such a sense, it was an expectation

similar results, it is shown that they are different.

Figure 13 shows the results of the first analysis of AF, in-

755

whose results are shown in figure 14, being the charts related to

the best of all.

785

so this division was very important.

The second analysis in this fusion procedure was the best

The comparison of the best method for OSS, OLT, BF, alge-

combination of tools to segment and characterise the defects,

braic fusion and PCAF with their manual references and with 17

Journal Pre-proof

MATT

0.600 0.400 0.200 C XXI

C XXII

C XX

C XIX

C XVII

C XVIII

C XVI

C XV

C XIV

C XII

C XIII

C XI

CX

C IX

C VII

C VIII

C VI

CV

C III

C IV

C II

CI

B IV

B II

B III

BI

A IV

A II

A III

AI

0.000

E VI

EV

E IV

E III

E II

EI

D VI

DV

D III

D IV

D II

DI

C VI

C III

re-

C II

CI

B VI

BV

B IV

B III

BI

B II

A VI

AV

A IV

A III

−0.200

A II

0.000

WDSR

CV

PYR

C IV

LI

0.200

AI

MATT

0.400

pro

of

Figure 13: MATT results for AF in relation to the fusion rules using as reference the manually-segmented images. Dimension reduction methods: (A) LI, (B) PYR and (C) WDSR. Segmentation methods: (I) MAX, (II) MEAN, (III) MIN and (IV) SUM, (V) aMAXdMAX, (VI) aMAXdMEAN, (VII) aMAXdMIN, (VIII) aMAXdSHEARO, (IX) aMAXdTHERMO, (X) aMAXdZERO, (XI) aMEANdMAX, (XII) aMEANdMEAN, (XIII) aMEANdMIN, (XIV) aMEANdSHEARO, (XV) aMEANdTHERMO, (XVI) aMEANdZERO, (XVII) aMINdMAX, (XVIII) aMINdMEAN, (XIX) aMINdMIN, (XX) aMINdSHEARO, (XXI) aMINdTHERMO and (XXII) aMINdZERO.

Figure 14: MATT results for AF combined with LI (black column), AF combined with PYR (grey column) and AF combined with WDSR (white column) in relation to the image processing tools using as reference the manually-segmented images. Preprocessing methods: (A) NPP, (B) 2DFFT, (C) ATC, (D) CE and (E) WDLP. Segmentation methods: (I) OTSU, (II) BRAD, (III) GSMO, (IV) MT, (V) EXMMT, (VI) FMM.

E VI

EV

E IV

E III

E II

EI

D VI

DV

D III

D IV

D II

DI

C VI

WDSR

CV

C IV

C III

C II

CI

B VI

BV

B IV

B III

BI

B II

A VI

A IV

A III

−0.200

A II

0.000

AV

0.200

PYR

urn al P

LI

AI

MATT

0.400

Figure 15: MATT results for PCAF with LI (black column), with PYR (grey column) and with WDSR (white column) using as reference the manually-segmented images. Preprocessing methods: (A) NPP, (B) 2DFFT, (C) ATC, (D) CE and (E) WDLP. Segmentation methods: (I) OTSU, (II) BRAD, (III) GSMO, (IV) MT, (V) EXMMT, (VI) FMM.

790

the ultrasound, the reference NDT technique, can be seen in800

these techniques result in images with different particularities.

figure 16 in black and in white, respectively, in relation to the

Comparing OSS and OLT, it seems to be easier to characterise

average outcomes of all impact energy sets.

defects in OLT images than in OSS images, since the OLT result was (37.6 ± 0.19)% better than the OSS one. This is also ex-

BF provided the closest result concerning its manual refer-

pected because OSS images were more complex to process due

Jo

ence and the US reference technology, followed by OLT, and

the difference between them in both cases was assured by hy-

805

defects on the image, especially for BVDs caused by impacts.

pothesis tests. BF is (14.80 ± 0.12)% higher than OLT for man-

795

to secondary fringes, noise and the characteristic low-frequency

ual reference and (8.05 ± 0.13)% for US reference. Both AF

Figure 17 presents the results regarding ED measurements:

and PCAF results were worse than OLT and OSS. About the

in (a), at 1 J; in (b), at 3 J; in (c), at 4 J; in (d), at 5 J; in (e), at

image fusion methods, one may say that BF was expected to be

6 J; in (f), at 7 J; in (g), at 8 J; in (h), at 9 J; in (i), at 10 J; in

better since it is simpler to segment OLT and OSS images sep-810

(j), at 12 J. From these figures, it can be seen that BF was better

arately before the fusion than to fuse the images first because

than other methods in almost all the measurement situations, 18

Journal Pre-proof

Manual reference

MATT

0.800

US reference

0.600 0.400 0.200 OSS-NCE-WDLP-MT

OLT-NCE-2DFFT-BRAD

BF-WDSR-NCE

AF-LI-MIN-WDLP-MT PCAF-WDSR-WDLP-MT

providing less deviation and measurement uncertainty. As ex-

Besides the adopted criteria, one must consider yet the ro-

pected, the uncertainties were higher to lower impact energies,

bustness increase provided by OLT and OSS, since they are

since was more difficult to accurately detect defects, but BF re-845

sensitive to different phenomena. Also, an economic analysis

sulted yet in better outcomes is relation to the other techniques.

must be taken into consideration in future works.

pro

815

of

Figure 16: Best methods MATT results in relation to their manual references (black column) and to ultrasound (white column).

There was also a higher measurement deviation at 10 J and 12 J, which was related to the own capacity of OSS and OLT as

6. Conclusions

NDT methods, since at these energies the defective areas were not similar to the damage at the US images anymore.

825

re-

Considering all the energies, BF presented in average (72.27

and shearography non-destructive testing methods for inspect-

± 0.56)% less measurement error in ED when compared to850

ing impact damages in carbon fibre reinforced plastics. The

OLT, the closest alternative, supporting then the use of image

impacts were induced in the samples in a controlled way from

fusion for impact damage characterisation. Moreover, AF and

1 J to 12 J. Data acquisition parameters were tested and op-

urn al P

820

This work proposed the use of image fusion of thermography

PCAF results were very poor, which was also expected consid-

timised for the shearography and thermography systems, gen-

ering the MATT results presented earlier.

erating proper input images. Several image processing tools

From ED measurements, one could also verify if the best855

835

techniques correctly indicated when the damages had a diam-

ages before or after the fusion task. Three fusion methods

eter greater than 6 mm, a limit of operation in many fields as

were studied to test image fusion contribution to inspect im-

described earlier. According to the US measurements, used as

pact damages: (1) binary image fusion; (2) algebraic fusion;

reference method, the 3 J impact energy samples already have

(3) principal component analysis fusion. These methods were

an ED greater than 6 mm. Figure 18 shows that only US, BF860

evaluated by calculating the equivalent diameter of each defect

and OSS provided correct indications in this sense, and this is

and also with the Matthews’ correlation coefficient. Binary fu-

another contribution of the use of image fusion for such an in-

sion provided the best results when compared to its manually-

spection task, since OLT alone was not capable to reliably pro-

segmented references and to the reference technology, the ul-

vide this outcome.

trasound D-scan, since its measurement error in relation to the

Jo

830

Both ED and MATT led to the conclusion that there is at least865

840

were combined and used to segment the defect on the im-

equivalent diameter was the smallest one and their Matthews’

one method of fusing images from OLT and OSS, BF, which

coefficients were the highest. Together with shearography, bi-

leads to a better characterisation of impact damages and thus

nary fusion was the only of the most appropriate methods which

the image fusion is beneficial. Further improvements can yet

correctly indicated defects with less and more than 6 mm. Be-

be made to reduce even more the difference between BF and

sides the good outcomes, the use of image fusion is yet inter-

US using other segmentation techniques such as convolutional870

esting since using techniques with different principles and sen-

neural networks for defect characterisation.

sitiveness enhances the complementarity of the non-destructive 19

OSS

OLT

BF

AF

PCAF

5.00 0.00 −5.00

−10.00

S1

S2

S3

S4

10.00

OLT

BF

AF

PCAF

S2

S3

S4

−20.00

S1

10.00 0.00

S1

OSS

AF

PCAF

S2

S3

S4

0.00 −10.00 −20.00 −30.00

OLT

BF

AF

PCAF

Meas. error (mm)

Meas. error (mm)

OSS

S1

−30.00

S1

10.00

OSS

S2

S3

S4

BF

S1

S2

AF

PCAF

Meas. error (mm)

OLT

Jo

Meas. error (mm)

OSS

S4

OLT

BF

AF

PCAF

S2

S3

S4

S3

S4

BF

AF

PCAF

−10.00 −20.00 −30.00

S1

S2

S3

S4

(f) ED at 7 J

5.00 0.00 −5.00 −10.00 −15.00 −20.00 −25.00

OSS

S1

OLT

BF

S2

AF

S3

PCAF

S4

(h) ED at 9 J

0.00

OSS

OLT

BF

AF

PCAF

−10.00 −20.00 −30.00 S1

(i) ED at 10 J

OLT

0.00

(g) ED at 8 J

5.00 0.00 −5.00 −10.00 −15.00 −20.00 −25.00

S3

−20.00

(e) ED at 6 J

10.00

S2

−10.00

re-

BF

Meas. error (mm)

OLT

urn al P

Meas. error (mm)

OSS

PCAF

(d) ED at 5 J

(c) ED at 4 J

5.00 0.00 −5.00 −10.00 −15.00 −20.00

AF

−10.00

pro

S1

BF

(b) ED at 3 J

Meas. error (mm)

Meas. error (mm)

OSS

OLT

0.00

(a) ED at 1 J

5.00 0.00 −5.00 −10.00 −15.00 −20.00

OSS

of

10.00

Meas. error (mm)

Meas. error (mm)

Journal Pre-proof

S2

S3

S4

(j) ED at 12 J

Figure 17: Measurement error related to the ED: in (a), at 1 J; in (b), at 3 J; in (c), at 4 J; in (d), at 5 J; in (e), at 6 J; in (f), at 7 J; in (g), at 8 J; in (h), at 9 J; in (i), at 10 J; in (j), at 12 J.

20

Journal Pre-proof OLT

BF

AF PCAF US

[7] C. M. Z¨ocke, Quantitative analysis of defects in composite material by means of optical lock-in thermography, Ph.D. thesis, Universit¨at des Saarlandes (2009). 905

[8] D. Balageas, X. Maldague, D. Burleigh, V. P. Vavilov, B. Oswald-Tranta,

4 -S

3

techniques for improved material inspection, Journal of Nondestructive Evaluation 35 (1). doi:10.1007/s10921-015-0331-7.

3J

2

-S 3J

1

-S 3J

3

4

-S 3J

-S 1J

2 -S

-S 1J

1J

-S

1

J.-M. Roche, C. Pradere, G. M. Carlomagno, Thermal (IR) and other NDT

1J

ED (mm)

OSS 12.00 9.00 6.00 3.00 0.00

[9] G. Busse, C. Spiessberger, A. Gleiter, Application of data fusion to thermal wave imaging with lockin thermography, Journal of Physics: Con-

910

of

Figure 18: Analysis of the 6 mm limit for each best method.

ference Series 214 (2010) 012070. doi:10.1088/1742-6596/214/1/

testing joint method, contributing to a more reliable inspection.

012070.

[10] R. Boudjemaa, A. B. Forbes, Parameter estimation methods for data fu-

eration, it was shown that there is at least one method of fusing915 875

sion, Tech. Rep. CMSC 38–04, National Physical Laboratory (2004).

pro

Even though a financial criterion must yet be taken into consid-

[11] M. P. Georges, J.-F. Vandenrijt, C. Thizy, I. Alexeenko, G. Pedrini,

images from thermography and shearography which leads to a

B. Vollheim, I. Lopez, I. Jorge, J. Rochet, W. Osten, Combined holography and thermography in a single sensor through image-plane holography

more complete characterisation of impact damages, with better

at thermal infrared wavelengths, Optics Express 22 (21) (2014) 25517.

metrological results, improving the capability of impact damage inspection.

doi:10.1364/oe.22.025517.

[12] M. Takeda, H. Ina, S. Kobayashi, Fourier-transform method of fringe-

re-

920

pattern analysis for computer-based topography and interferometry, Journal of the Optical Society of America 72 (1) (1982) 156. doi:10.1364/

Acknowledgements 880

josa.72.000156.

[13] S. Mallat, A theory for multiresolution signal decomposition: the wavelet

This study was financed in part by the Coordenac¸a˜ o (CAPES) - Finance Code 001. The authors also would like

in the context of the project IDD-Metro BRAGECRIM 030/14 and Petrobras.

telligence 11 (7) (1989) 674–693. doi:10.1109/34.192463.

[14] H. B. Mitchell, Image fusion : theories, techniques and applications,

to thank the support of the German agency for research DFG

885

representation, IEEE Transactions on Pattern Analysis and Machine In-

urn al P

de Aperfeic¸oamento de Pessoal de N´ıvel Superior - Brasil

925

Springer, Berlin, 2010.

[15] N. Otsu, A threshold selection method from gray-level histograms, IEEE

930

Transactions on Systems, Man, and Cybernetics 9 (1) (1979) 62–66. doi: 10.1109/tsmc.1979.4310076.

[16] D. Bradley, G. Roth, Adaptive thresholding using the integral image, Jour-

References

[1] V. Silberschmidt, Dynamic deformation, damage and fracture in compos-935 ite materials and structures, Woodhead Publishing, Duxford, UK, 2016.

[2] G. Minak, M. Fotouhi, M. Ahmadi, Low-velocity impact on laminates, 890

in: Dynamic Deformation, Damage and Fracture in Composite Materials and Structures, Elsevier, 2016, pp. 147–165.

Jo

940

ods, Journal of the Optical Society of America A 11 (1) (1994) 107. doi:10.1364/josaa.11.000107. versity Press, Cambridge, 1999.

10.1109/tcom.1983.1095851.

[4] V. M. Karbhari, Non-Destructive Evaluation (NDE) Of Polymer Matrix

[20] M. A. Herr´aez, D. R. Burton, M. J. Lalor, M. A. Gdeisat, Fast two-

Composites, Elsevier Science, Burlington, 2013.

900

weighted phase unwrapping that uses fast transforms and iterative meth-

IEEE Transactions on Communications 31 (4) (1983) 532–540. doi:

Victoria, B.C, 2004.

[5] L. Yang, Digital shearography : new developments and applications,

[17] D. C. Ghiglia, L. A. Romero, Robust two-dimensional weighted and un-

[19] P. Burt, E. Adelson, The laplacian pyramid as a compact image code,

[3] M. Kessler, Advanced topics in characterization of composites, Trafford,

895

2007.10129236.

[18] J. Sethian, Level set methods and fast marching methods, Cambridge Uni-

doi:10.1016/

b978-0-08-100080-9.00006-3.

nal of Graphics Tools 12 (2) (2007) 13—-21. doi:10.1080/2151237x.

945

dimensional phase-unwrapping algorithm based on sorting by reliability

SPIE, Bellingham, Washington (1000 20th St. Bellingham WA 98225-

following a noncontinuous path, Applied Optics 41 (35) (2002) 7437.

6705 USA, 2016.

doi:10.1364/ao.41.007437. [21] R. Gonzalez, R. E. Woods, Digital image processing, Pearson, New York,

[6] G. Workman, Nondestructive testing handbook, American Society for

NY, 2018.

Nondestructive Testing, Columbus, Ohio, 2012.

21

Journal Pre-proof

950

[22] F. White, Fluid mechanics, McGraw-Hill Education, New York, NY,

[40] D. P. Bavirisetti, R. Dhuli, Multi-focus image fusion using multi-scale im-

2016.

age decomposition and saliency detection, Ain Shams Engineering Jour-

[23] B. Matthews, Comparison of the predicted and observed secondary struc-1000

955

nal 9 (4) (2018) 1103–1117. doi:10.1016/j.asej.2016.06.011.

ture of T4 phage lysozyme, Biochimica et Biophysica Acta (BBA) - Pro-

[41] S. Masood, M. Sharif, M. Yasmin, M. A. Shahid, A. Rehman, Image fu-

tein Structure 405 (2) (1975) 442–451. doi:10.1016/0005-2795(75)

sion methods: A survey, Journal of Engineering Science and Technology

90109-9.

Review 10 (6) (2017) 186–195. doi:10.25103/jestr.106.24.

[24] P. K. Mallick, Composites engineering handbook, M. Dekker, New York, 1997.

[42] J.-F. Vandenrijt, N. Li`evre, M. P. Georges, Improvement of defect detection in shearography by using principal component analysis, in: In-

1005

[25] A. Kelly, Comprehensive composite materials, Elsevier, Amsterdam New

of

terferometry XVII: Techniques and Analysis, SPIE, 2014, pp. 1–10.

York, 2000. 960

doi:10.1117/12.2062831.

[26] P. Moore, Nondestructive testing handbook, American Society for Non-

[43] J. Du, W. Li, K. Lu, B. Xiao, An overview of multi-modal medical image

destructive Testing, Columbus, Ohio, 1998.

fusion, Neurocomputing 215 (2016) 3–20. doi:10.1016/j.neucom.

[27] P. K. Varshney, Multisensor data fusion, in: Intelligent Problem Solving.1010

pro

Methodologies and Approaches, Springer Berlin Heidelberg, 2000, pp.

2015.07.160.

[44] F. E.-Z. A. El-Gamal, M. Elmogy, A. Atwan, Current trends in medical

1–3. doi:10.1007/3-540-45049-1_1. 965

image registration and fusion, Egyptian Informatics Journal 17 (1) (2016)

[28] International Standard Organization, ISO DIN 6603-1: Determination of

99–124. doi:10.1016/j.eij.2015.09.002.

puncture impact behaviour of rigid plastics - Part 1: Non-instrumented impact testing (2000).

[45] P. Menner, Zerst¨orungsfreie pr¨ufung von modernen werkstoffen mit dynamischen shearografie-verfahren, Ph.D. thesis, Universit¨at Stuttgart

1015

[29] F. Incropera, Fundamentals of heat and mass transfer, John Wiley, Hobo-

970

(2013).

re-

ken, NJ, 2007.

[46] G. Busse, Optoacoustic phase angle measurement for probing a metal,

[30] S. Phillips, W. Estler, M. Levenson, K. Eberhardt, Calculation of mea-

Applied Physics Letters 35 (10) (1979) 759–760. doi:10.1063/1.

surement uncertainty using prior information, Journal of Research of

90960.

the National Institute of Standards and Technology 103 (6) (1998) 625.1020

shearography for the detection of transport-induced damages on artwork,

[31] X. Maldague, Infrared and thermal testing, American Society for Nonde975

structive Testing, Columbus, OH, 2001.

[32] C. Ibarra-Castanedo, Quantitative subsurface defect evaluation by pulsed

rigid multimodality image registration, in: Medical Imaging 2001: Image

sit´e Laval (2005).

Processing, SPIE, 2001, pp. 1–10. doi:10.1117/12.431046.

[49] I. T. Jolliffe, Principal Component Analysis, Springer, New York, NY,

tributor, Weinheim Chichester, 2006.

surements in three shearing directions, Optics and Lasers in Engineering

Springer Science & Business Media, Berlin, Heidelberg, 2013.

[51] D. Powers, Evaluation: From precision, recall and f-measure to roc, in-

111 (2018) 86–92. doi:10.1016/j.optlaseng.2018.07.018.

[35] A. Fahr, Aeronautical applications of non-destructive testing, DEStech Publications Inc, Lancaster, Pennsylvania, 2014.

formedness, markedness correlation, Journal of Machine Learning Technologies 2 (1) (2011) 37–63. doi:10.9735/2229-3981.

[52] P. R. Spendley, Design Allowables for Composite Aerospace Structures,

[36] D. Amarsaikhan, T. Douglas, Data fusion and multisource image classi-1035 fication, International Journal of Remote Sensing 25 (17) (2004) 3529–

Doctor thesis, University of Surrey (2012).

Jo

[53] E. P´eronnet, M.-L. Pastor, R. Huillery, O. Dalverny, S. Mistou, H. Wele-

3539. doi:10.1080/0143116031000115111.

mane, Non destructive investigation of defects in composite structures by

[37] T. Stathaki, Image fusion : algorithms and applications, Academic

three infrared thermographic techniques, in: Proceedings of the Interna-

Press/Elsevier, Amsterdam Boston, 2008.

tional conference on experimental mechanics ICEM 15, Porto, 2012, pp.

[38] Z. Wang, D. Ziou, C. Armenakis, D. Li, Q. Li, A comparative analysis1040

995

2002.

[50] P. Soille, Morphological Image Analysis - Principles and Applications,

bertazzi, Multiple-aperture one-shot shearography for simultaneous mea-1030

990

1–10. doi:10.1117/12.2270278.

[48] D. Mattes, D. R. Haynor, H. Vesselle, T. K. Lewellyn, W. Eubank, Non-

[34] E. S. Barrera, A. V. Fantin, D. P. Willemann, M. E. Benedet, A. Al-

985

in: Optics for Arts, Architecture, and Archaeology VI, SPIE, 2017, pp.

phase thermography: depth retrieval with the phase, Ph.D. thesis, Univer-1025 [33] A. Hornberg, Handbook of machine vision, Wiley-VCH John Wiley, dis980

[47] D. Buchta, C. Heinemann, G. Pedrini, C. Krekel, W. Osten, Lock-in-

urn al P

doi:10.6028/jres.103.042.

1–9. doi:10.1080/09593330.2013.828094.

of image fusion methods, IEEE Transactions on Geoscience and Remote

[54] E. P´eronnet, M.-L. Pastor, H. Welemane, R. Huillery, Non destructive

Sensing 43 (6) (2005) 1391–1402. doi:10.1109/tgrs.2005.846874.

testing of artificial defects in composite structures by thermal full field

[39] S. Klonus, M. Ehlers, Fusion 2009 : the 12th International Conference on

measurement methods, in: Proceedings of the PhotoMechanics 2013,

Information Fusion : Grand Hyatt Seattle, Seattle, Washington, USA, 6-9 July, 2009, IEEE, Piscataway, N.J, 2009.

Montpellier, 2013, pp. 1–4. doi:10.1016/j.suscom.2014.08.005.

1045

22

[55] M. Kaufmann, D. Zenkert, C. Mattei, Cost optimization of composite

Journal Pre-proof

aircraft structures including variable laminate qualities, Composites Science and Technology 68 (13) (2008) 2748–2754.

doi:10.1016/j.

Jo

urn al P

re-

pro

of

compscitech.2008.05.024.

23

Journal Pre-proof

urn al P

re-

pro

of

1.113 combinations of fusion and image processing tools 72.21% less error of the equivalent diameter when using the fusion approaches Correct identification of situations above and below the diameter limit of 6 mm An enhancement of 8.05% in defect characterization metric with fusion approaches

Jo

• • • •

Journal Pre-proof

Declaration of interests ☒ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Jo

urn al P

re-

pro

of

☐The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: