Automated quality assurance as an intelligent cloud service using machine learning

Available online at www.sciencedirect.com Available online at www.sciencedirect.com

ScienceDirect ScienceDirect

Available online atonline www.sciencedirect.com Available at www.sciencedirect.com Procedia CIRP 00 (2019) 000–000 Procedia CIRP 00 (2019) 000–000

ScienceDirect ScienceDirect

www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia

Procedia CIRP 00 (2017) 000–000 Procedia CIRP 86 (2019) 185–191 www.elsevier.com/locate/procedia

7th CIRP CIRP Global Global Web Web Conference, Towards shifted production value stream patterns inference of data, 7th Web Conference, Conference, Towards Towardsshifted shiftedproduction productionvalue valuestream streampatterns patternsinference inferenceof ofdata, data, models and technology models and technology models and technology 28th CIRP Design Conference, May 2018, Nantes, France

Automated Automated quality quality assurance assurance as as an an intelligent intelligent cloud cloud service service using learning A new methodology to analyze the functional and physical architecture of using machine machine learning existing products for an assembly oriented product family identification a,* a a a,b a,c M. Schreiber a,*, J. Klöber-Kocha, J. Bömelburg-Zachariasa, S. Braunreuthera,b, G. Reinharta,c M. Schreiber , J. Klöber-Koch , J. Bömelburg-Zacharias , S. Braunreuther , G. Reinhart Fraunhofer Research Institution for Casting, Composite and Processing Technology (IGCV), Provinostr. 52, 86153 Augsburg, Germany Stief *, Jean-Yves Alain Etienne, Ali Siadat Fraunhofer ResearchPaul Institution for Casting, Composite andDantan, Processing Technology (IGCV), Provinostr. 52, 86153 Augsburg, Germany Hochschule Augsburg University of Applied Sciences, An der Hochschule 1, 86161 Augsburg, Germany a a

b

b Hochschule Augsburg University of Applied Sciences, An der Hochschule 1, 86161 Augsburg, Germany for Machine Tools and Industrial Management (iwb), Technical University Munich, Boltzmannstr. 15, 85748 Germany École Nationale Supérieure d’Arts et Métiers, Arts et Métiers ParisTech, LCFC EAof 4 Rue Augustin Fresnel, MetzGarching, 57078, France Institute for Machine Tools and Industrial Management (iwb), Technical University of4495, Munich, Boltzmannstr. 15, 85748 Garching, Germany * Corresponding author. Tel.: +49 (0)821-90678-180; fax: +49 (0)821-90678-199. E-mail address: [email protected] * Corresponding author. Tel.: +49 (0)821-90678-180; fax: +49 (0)821-90678-199. E-mail address: [email protected] * Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: [email protected] c cInstitute

Abstract Abstract Abstract The amount of data generated in production systems increases continuously due to the integration of cyber-physical systems and additional The amount of data generated in production systems increases continuously due to the integration of cyber-physical systems and additional data contains potentially useful knowledge can be used to improve production processes and quality. Machine Insensors. today’sThis business environment, the trend towards morethat product variety and customization is unbroken. Dueproduct to this development, the learning need of sensors. This data contains potentially useful knowledge that can be used to improve production processes and product quality. Machine learning algorithms offer the potential of improvements for quality assurance. However, companies often dofamilies. not haveTo thedesign necessary know-howproduction to extract agile and reconfigurable production systems emerged to cope with various products and product and optimize algorithms offer the potential of improvements for quality assurance. However, companies often do not have the necessary know-how to extract this knowledge. This thereforeproduct presents a service-based optical quality assurance using machine algorithms. systems as well as to publication choose the optimal product system analysisfor are needed. Indeed, most of the learning known methods aimThe to this knowledge. This publication therefore presentsmatches, a service-based system formethods optical quality assurance using machine learning algorithms. The intelligent cloud service is testedfamily and validated by an industrial use caseproduct for transparent injection molded parts. analyze a product or one product on the physical level. Different families, however, may differ largely in terms of the number and intelligent cloud service is tested and validated by an industrial use case for transparent injection molded parts. © 2019 Authors. Published Elsevier nature ofThe components. This fact by impedes anB.V. efficient comparison and choice of appropriate product family combinations for the production © The Authors. Published Published by Elsevier B.V. © 2019 2019 The Authors. by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) system. A new methodology isunder proposed to analyze existing products in view of their functional and physical architecture. The aim is to cluster This is an open access article the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the scientific CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the committee of the 7th CIRP Global Webassembly Conference these products in new assembly oriented product families for the optimization of existing lines and the creation of future reconfigurable Peer-review committee of CIRP Global Global Web Conference Conference Peer-review under under responsibility responsibility of of the the scientific scientific committee of the the 7th 7th CIRP Web assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and Keywords: methodology; quality assurance; machine learning a Keywords: functionalmethodology; analysis is performed. Moreover, hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the quality assurance; machinea learning similarity between product families by providing design support to both, production system planners and product designers. An illustrative example of a nail-clipper is used to explain the proposed methodology. An industrial case study on two product families of steering columns of thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach. Introduction quality requirements, requirements, aa random random sample sample inspection inspection of of the the parts parts ©1. The Authors. Published by Elsevier B.V. 1.2017 Introduction quality produced is no longer sufficient and 100% inspection is Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.sufficient and 100% inspection is produced is no longer

Manufacturing companies companies all all over over the the world world are are exposed exposed to to Manufacturing strategy to strategy to ensure competitiveness is to manufacture products of the ensure competitiveness is to manufacture products of the highest [1]. highest quality quality in in highly highly automated automated production production systems systems [1]. Furthermore, due to increasingly shorter product life cycles, due to increasingly shorter product life cycles, 1.Furthermore, Introduction production lines have have to to be be adapted adapted and and production production processes processes production lines have to be redesigned at ever-shorter intervals. This can lead to to have ever-shorter intervals. can lead Dueto be toinredesigned the fastatprocesses development in theThis domain of problems production and therefore product quality problems in production therefore product quality communication and anprocesses ongoing and trend of digitization and needs to be continuously monitored by quality management. needs to be continuously monitored by quality management. digitalization, manufacturing enterprises are facing important Quality management management plans plans and and controls controls all all quality-related quality-related Quality challenges in objectives today’s market environments: a continuing activities and of the company [1]. One the activitiestowards and objectives of of theproduct company [1]. One of oftimes the most most tendency reduction development and frequently used quality control methods is visual inspection, frequently used quality control methods is visual inspection, shortened product lifecycles. In addition, there is an increasing which is is generally generally carried carried out out by by aa worker worker based based on on samples samples of of which demand of customization, being atThe thetask same time ininspection a global the manufactured products [1, 2]. of visual the manufactured products [1, all 2]. over The task of visualThis inspection competition with competitors the and world. trend, is often often monotonous, monotonous, difficult to to reproduce reproduce the assessments assessments is difficult and the which is inducing the development from macro to micro are influenced influenced by by the the natural natural performance performance fluctuations fluctuations of of the the are markets, results the in diminished lotday sizes dueDue to augmenting workers during course of the [3, 4]. to increased workersvarieties during the course of the day [3, 4]. Due to increased product (high-volume to low-volume production) [1]. high competitive competitive pressure in saturated saturated markets. One One Keywords: Assembly;pressure Design method; Family identification high in markets.

required [2]. 100% inspection inspection is is aa quality quality inspection inspection of of all all required [2]. 100% components of the inspection lot [1]. Solving this task with components of the inspection lot [1]. Solving this task with manual visual visual inspection inspection by by workers workers is is expensive expensive and and in in many many manual cases cannot be integrated into the production lines within the cases cannot be integrated into the production lines within the required production cycle [5]. required production [5]. of the product rangecycle and characteristics manufactured and/or Automated quality assurance can remedy remedy this situation situation and Automated quality assurance can this and assembled in this system. In this context, the main challenge in enable fatigue-free, reproducible and economical 100% enable fatigue-free, reproducible and toeconomical 100% modelling and analysis is now not only cope with single inspection of of products products [4]. [4]. Recent Recent approaches approaches in in image image inspection products, a limited product range or existing product families, processing have shown that machine learning algorithms can processing shown that and machine learning algorithms can but also tothe behave able to analyze toascompare products totodefine improve accuracy as well the adaptability new improve thefamilies. accuracyIt can as well as the that adaptability to new new product be observed classical existing products [5, 6]. 6]. products [5, product familiesautomated are regrouped in function of clients or features. In highly highly production systems using cyberIn automated production systems using However, assembly oriented product families are hardly tocyberfind. physical systems (CPS) data availability is increasing physical systems family (CPS)level, dataproducts availability is increasing On the product differ mainly in two continuously [7]. [7]. These These data data can can be be used used to to optimize optimize production production continuously main characteristics: (i) the number of components and (ii) the processes and and the the quality quality of of products products [8]. [8]. Nevertheless, Nevertheless, many many processes type of components (e.g. mechanical, electrical, electronical). companies do do not not have have the the know-how know-how to to use use the the available available data data companies Classical methodologies considering mainly single products and to gain knowledge from it [9]. One solution to this problem andsolitary, to gain knowledge from it [9]. One solution this problem or already existing product familiesto analyze the

To cope with this augmenting variety as well as to be able to product structure on a physical level (components level) which 2212-8271 © 2019 The Authors. Published by Elsevier B.V. 2212-8271 possible © 2019 The optimization Authors. Publishedpotentials by Elsevier B.V. identify in the existing causes difficulties regarding an efficient definition and This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CCtoBY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) production system, it is important have a precise knowledge of different product families. Addressing this Peer-review under responsibility of the scientific committee of the 7th CIRP Globalcomparison Web Conference Peer-review under responsibility of the scientific committee of the 7th CIRP Global Web Conference 2212-8271 © 2019 The Authors. Published by Elsevier B.V. This is an©open article Published under theby CC BY-NC-ND 2212-8271 2017access The Authors. Elsevier B.V. license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of scientific the scientific committee theCIRP 7th CIRP Web2018. Conference Peer-review under responsibility of the committee of the of 28th DesignGlobal Conference 10.1016/j.procir.2020.01.034

186 2

M. Schreiber et al. / Procedia CIRP 86 (2019) 185–191 Schreiber et al. / Procedia CIRP 00 (2019) 000–000

is the application of smart services in a service platform [8]. The service platform enables a collaboration and knowledge sharing between operators and service providers [8]. The operator uses a smart service to analyze the available data and the smart service helps to increase the productivity. The objective of this publication is to give a review of existing automated quality assurance systems for the inspection of products as a service for highly automated production lines. The identified research gap and an approach for an automated quality assurance as an intelligent cloud service using machine learning is presented. In section four, the industrial case study is presented. A summary is given in the concluding section. 2. Image processing for quality assurance 2.1. Image processing chain The automation of visual inspection can be implemented using image processing systems, in which images are automatically evaluated by software. The process of image processing can be divided into five steps [10]: image capturing, image pre-processing, segmentation, feature extraction and classification. In the image capturing step, the images of the products to be evaluated are captured. The hardware for image acquisition consists of three components: illumination, camera lens and camera sensor [10]. The quality and information content of the images can be increased by using polarization filters and color illumination [11]. By means of targeted illumination, an increase in contrast between the objects and the background can be achieved for colored products. Polarization of reflected light is typical for metallic or dielectric surfaces and polarization filters are used for suppressing them [10]. Images often contain inhomogeneities from uneven illumination or image noise and are therefore preprocessed in the second step [12]. Operations such as image smoothing, brightness and contrast compensation as well as position and rotation corrections can be used to filter out unwanted image noise or reflections and to correct geometric distortions [12]. In the third step, images are divided into elementary sections. The assignment of pixels to objects is done by extracting meaningful regions. Threshold operations and region detection by means of edge detections are widely used segmentation approaches [10, 11]. In the fourth step, features are extracted. The determination of characteristic properties for the classification object is decisive for image classification. These characteristic properties are called features and can be measured quantitatively. Two of the most commonly described features are region features and gray value features. Region features characterize the geometry of the region independently of the image content. Gray value features, on the other hand, take the underlying image information into account. [10] Classification is carried out as the fifth and last step based on the available parameters [5]. Classification according to [13] is defined as (1) 𝐶𝐶𝑀𝑀(𝜃𝜃) : 𝐷𝐷 → 𝑌𝑌

with classifier 𝐶𝐶, model 𝑀𝑀 with parameters θ, data space 𝐷𝐷 and classification result Y. Training in this context means that optimal parameters θ are found for a given amount of data D. Classification means that the classifier 𝐶𝐶𝑀𝑀(𝜃𝜃) is applied to type D objects. [13, 14] 2.2. Approaches for quality assurance using image processing [15] introduce an approach using deep convolutional neural networks to classify resin molded articles. The defects to be classified are crack, burr, protrusion and chipping. Data augmentation techniques like rotating, translating and scaling are employed to generate additional images for training. However, the transferability of the developed image classifier to other use cases is not considered. [16] present an approach for automated surface inspection that relies exclusively on fault-free samples. A fully connected neural network is trained to detect defects in surfaces without manually labelled data. Synthesized defects are injected into fault-free surface images to reach a higher classification accuracy. The approach does not include real images of the defects that are supposed to be detected later, which might compromise robustness when applied on a use case. An approach for defect detection in additive manufacturing using supervised machine learning is presented by [17]. Multiple images of the build layers are collected, and multidimensional visual features are extracted. A linear support vector machine is trained for classification and an accuracy of more than 80% is achieved. The approach requires a great effort in the manual feature extraction. [18] present an approach for quality control in manufacturing systems using defect detection through binary classification (e.g. good, suspect). Two case studies with highly unbalanced data are used for validation. However, due to the binary concept, distinguishing between different classes of defects is not possible. [19] introduce a machine learning-based multistage quality control. Several binary classification models are evaluated on image data from a multistage automotive assembly line. In case of a severe change in the appearance of the products (e.g. change of material) the machine learning model would have to be adapted or redesigned. The authors state that by incorporating a service-oriented approach an adaption or redesign of the entire model could be avoided. 2.3. Image processing with machine learning Machine learning is a branch of artificial intelligence and is the generation of knowledge from experience by developing algorithms from examples in a complex model [20]. The model is an acquired knowledge representation that can be applied to new data of the same kind. Machine learning can be divided into two categories based on the learning strategy: supervised learning and unsupervised learning [20]. In image processing and optical defect recognition, machine learning algorithms, in particular artificial neural networks (ANN) and deep learning are increasingly being applied for



M. Schreiber et al. / Procedia CIRP 86 (2019) 185–191 Schreiber et al. / Procedia CIRP 00 (2019) 000–000

classification tasks [14]. In contrast to the image processing chain using SVMs, feature extraction for ANNs is done implicitly from the original image without the intervention of a developer [4]. In order to train an ANN effectively, a large amount of labelled training data is necessary. In general, the more useful data a machine learning algorithm has been trained on, the more accurate it can become [21]. However, the amount of useful data available for specialized applications, especially in production environments, is often limited and therefore limits performance. The limitation in case of quality assessment is a result of the restricted amount of products not fulfilling the requirements. The amount of training data can be increased artificially by using data augmentation methods. Each ANN has an input and an output layer. In between these two layers, there is at least one hidden intermediate layer. The neurons are linked by directional connections. The application-specific structure of an ANN is also referred to as the architecture of the ANN. Each neuron of the ANN receives the input signals 𝑥𝑥1 to 𝑥𝑥𝑘𝑘 , which it weights with the weighting factors 𝑤𝑤1 to 𝑤𝑤𝑘𝑘 and then adds them up. The output signal is calculated from this sum via the activation function [11]. A model may describe the training data well but classify unknown data with insufficient accuracy or not at all and is called overfitting. According to [22] overfitting occurs when models are too complex or have too many parameters during training and testing. The opposite of overfitting is underfitting and it occurs when the model does not adequately represent the relationships between input data and output data. 2.4. Conclusions In the presented approaches machine learning has been widely used in numerous domains and each approach comes up with an innovative aspect. Nevertheless, none of the approaches presents a defect classification algorithm as a service. Most approaches focus on optimizing accuracy of the classification in their particular use case. The effort for the development of automated quality assurance for an individual use case can be reduced significantly by development of an intelligent cloud service using machine learning. The classical image processing chain includes feature extraction by a developer, which can result in loss of important information due to subjectivity. This may prevent the classifier from unfolding its full performance potential and needs to be prevented [14]. Systems using machine learning often need a large amount of data sets to reach an acceptable classification rate, which often cannot be provided in real production scenarios. Therefore the incorporation of data augmentation methods is needed. Nevertheless, data augmentation and the selection of suitable parameters of an ANN is not trivial. Under- and overfitting of the network must be avoided. Consequently, the development of an intelligent service, using machine learning, must be combined with a comprehensive experimental design to fit to the use case. In conclusion, to the best of the authors’ knowledge no coherent automated quality assurance as an intelligent cloud service using machine learning exists.

187 3

3. Approach for automated quality assurance using machine learning In the following section the approach for automated quality assurance is derived and the parameters to be considered for data augmentation and training and testing are deduced. 3.1. Smart services in production Service-oriented platforms are based on the concept of service-oriented architecture (SOA) [23]. SOA is a design paradigm of computer science in which an application landscape is created from individual functional application components. Each application component executes a clearly defined task and is coupled with other application components [24]. In contrast to a conventional monolithic program structure, service-oriented architecture implements the paradigm for structuring and use of distributed functionalities for which different owners are responsible [24]. A single service is self-contained, can be used independently and is available via an interface within a network. This allows different users to use the services and integrate them into their added value [8]. In particular, the possible reusability for similar applications, user-friendliness and system controllability through encapsulation of the technical application and economies of scale through access to a large network are to be mentioned as advantages [24]. The economic potential of additional services for manufacturing companies, especially in the area of plant operation, is estimated to be very high [25]. Possible examples of innovative services are predictive maintenance services for a condition-based maintenance strategy and risk-oriented production planning [26, 27]. 3.2. Selection of machine learning algorithms and parameters Deep Neural Networks (DNN) are a set of machine learning algorithms that use Artificial Neural Networks (ANN) with many intermediate layers. This type of algorithm has proven particularly successful in applications [28, 29]. [30] define a DNN as an ANN with four or more layers. Convolutional Neural Networks (CNN) are a special type of DNN with a grid-like architecture, and use convolution instead of matrix multiplication in at least one layer [28]. The pooling function replaces the CNN output with a statistic summarizing the nearby outputs at a certain point [28]. In this way, the pooling function contributes to make the model approximately invariant to small changes in the input data. The output therefore does not change with small shifts in the input data [28]. Due to the successful usage of CNNs for image classification, a CNN is also used in the context of this research project. The selection of a suitable architecture for a CNN is an important step in the increase of its classification accuracy. Depending on the application, various architectures may be suitable. Larger CNNs can store more data. Thus, the selection of network size should depend on how detailed the features that

188 4

M. Schreiber et al. / Procedia CIRP 86 (2019) 185–191 Schreiber et al. / Procedia CIRP 00 (2019) 000–000

are to be extracted from an image are [31]. [2] show that the highest classification accuracy is achieved by three and four layer CNNs for defect detection. Consequently, an architecture based on three CNN layers is used here for the CNN. The output of the CNN is a classification of the products’ images into good parts and rejected parts. The rejected parts are further categorized into different defect categories. As a result, the CNN is a multi-class classifier, which is a necessary requirement for many areas of optical quality assurance. CNN models that were trained based on a small amount of data often show a tendency to overfit to the training data set and therefore, often fail validation with the test data set due to a lack of generalization. The amount of training samples available has empirically been shown to be critical to achieve high classification accuracy [32]. In order to provide the classifier with a larger amount of data for training, data augmentation techniques are applied to the training data set. In order to augment the data available to the algorithm, small mutations are applied through scaling, cropping, rotating or flipping of the original training images (see Table 1) [14, 15, 30]. These techniques have been chosen through the indication in extensive experiments that they outperform the other enhancement approaches and are more effective on smaller data sets [33]. In addition to data augmentation, further techniques are available in order to avoid overfitting of a CNN: a combination of several CNNs and dropouts [34]. The combination of several different CNN models is an approach to increase classification accuracy. However, since training only one CNN often takes several days for large amounts of data, this is rarely feasible in practice. A more time-efficient approach is the dropout method, in which the weights of individual neurons are set to zero by a random mechanism, which then contribute neither to the forward pass nor to the backpropagation of the CNN. Thus, the architecture is slightly changed by chance during each pass and the tendency to overfit the data is reduced. The neurons are thereby forced to extract independent and robust features, because they cannot rely on the presence of other neurons [34]. The effectiveness of the dropout technique for the industrial use case in this paper is evaluated during experiments. Table 1. Data augmentation methods [14, 15, 30] Data Augmentation Method

Description

Scaling

Scale the image by a factor

Cropping

Crop a part of the original image and resize the cropped image

Rotating

Rotate the image by any angle

Flipping

Flip the image horizontally

4. Industrial case study In the following section, the industrial use case is presented. Subsequently, the semi-factorial experimental plan for validation of the developed concept is carried out.

4.1. Production line and test bench of injection molded parts The system is prototypically applied in a production line for injection molded parts of a household appliances manufacturer. The production line consists of an injection molding machine, a removal unit, a transport robot and two delivery belts with manual quality inspection stations. The products manufactured are transparent plastic parts as visual components for household refrigerators. Due to the large variety of refrigerators, a big variety of parts is produced at the production line. The parts are built into the refrigerators and represent a direct interface to the customer. Therefore, the product quality requirements are extraordinarily high. Currently, the quality of the components is ensured by visual inspection by a worker at the end of the production line. The products are examined with regard to various criteria such as homogeneity, surface quality and foreign particles. During the inspection, the worker classifies the parts as fulfilling the requirements or not. Due to the increasing requirements on quality and increasing workload, a flexible test bench and an intelligent service for automated quality assurance needed to be developed. Within the test bench, image capturing is realized by a camera and in conjunction with the coordinates of the robot enables the localization of defects. After the component has been captured, it is picked up again by the robot and placed in a position that allows easy integration into the material flow for further processing. The service enables the classification of different types of defects like scratches, inclusions, streaks and black dots. Both systems are flexible designed to be easily integrated into other production lines and be adaptable to various types of products. The test bench and the parts are to be seen in figure 1. a

b

Fig. 1. (a) test bench; (b) transparent injection molded part.

4.2. Training and testing of the service for automated quality assurance A semi-factorial experimental plan is used for the determination of the parameters of the CNN that achieve the highest possible classification accuracy for the test data set to detect the defective products. For the semi-factorial experimental plan, it is assumed that classification accuracy can be influenced in two basic ways. First, the accuracy is influenced by the amount and quality of data available for training the classifier. Second, the specifications of the classifier's training influence accuracy. The training data quantity can be influenced by the size of the data set and the percentage ratio between the training data set and the test data set. Data augmentation is used to artificially increase the amount of data.

Schreiber et al. / Procedia CIRP 00 CIRP (2019)86000–000 M. Schreiber et al. / Procedia (2019) 185–191



5 189

Table 2. Excerpt of the semi-factorial design plan Exp. Number

Size of the Data Set

Classification Result during Training

Classification Result during Test

[px x px]

[parts]

[%/100]

[%/100]

Yes

150x150

33

0.7650

0.7850

300

Yes

150x150

33

0.8331

0.7871

300

Yes

150x150

33

0.7500

0.7797

60/40

300

Yes

150x150

33

0.9605

0.7391

60/40

300

Yes

150x150

33

0.9491

0.7480

0.2

60/40

300

Yes

150x150

33

0.7188

0.7119

…

…

…

…

…

…

…

…

…

0

70/30

50

No

512x512

50

…

…

…

…

80/20

…

…

…

…

…

…

Parameter 1:

Parameter 2:

Parameter 3:

Parameter 4:

Parameter 5:

Batch Size

Dropout Rate

Epochs

Data Image Augmentation Resolution

[images]

[%/100]

Split Ratio Training / Test Data Set [%/%]

[-]

[-]

1

32

0.5

60/40

300

2

16

0.5

60/40

3

8

0.5

60/40

4

32

0.2

5

16

0.2

6

8

…

…

… …

The quality of the data can be influenced through image resolution. With regards to the training specifications, the number of training iterations and the size of the training data set can be varied per run. For the experimental design, the semi-factorial experimental plan thus consists of a total of seven parameters. Parameter 1 is the batch size. Since one cannot pass an entire dataset through a neural network at once, the dataset is divided into batches. One batch is propagated through the neural network during one training iteration. The batch size describes the number of image samples that are included in one batch. Parameter 2 is the dropout rate. Parameter 3 is the split ratio between the training and the test data sets. The data set is randomly divided into training and test data sets specified by a percentage of the data set. The chosen ratios between training and test data sets for the application are 60/40, 70/30, and 80/20. Parameter 4 is the number of epochs. During one epoch, the entire dataset is passed forward and backward through the neural network. It is comprised of numerous batches. Parameter 5 is the application of data augmentation to the data set. This parameter can be either yes (data augmentation is applied) or no (data augmentation is not applied). Parameter 6 is the quality of the image resolution specified in pixels. Parameter 7 is the size of the data set specified in the amount of different parts that are included. Several images have been taken of each part from different perspectives. Classification quality is evaluated on the basis of the accuracy achieved in the defect classification in the test data record. The classification accuracy is calculated as an average value of the accuracies achieved in the individual epochs. This increases generalizability and robustness against statistical outliers. For implementation of automated quality assurance using machine learning, the TensorFlow framework and the highlevel API Keras is used. A total of 268 images without errors as well as 61 images with scratches, 50 images with streaks and 10 images with black dots were available for the training and test phase of the CNN. This corresponds to a small data set for CNN. An excerpt of the semi-factorial design plan is shown in table 2. The experiments were carried out in such a way that all

Parameter 6:

Parameter 7:

seven parameters’ influences on the classification results could be investigated independently. The first parameter to be examined is the batch size. For this, the batch size is varied across its three levels 32, 16 and 8, while all other parameters are kept constant. In order to investigate interactions between the parameters, the batch size is reviewed again at all levels, while another parameter is changed. In this example, the dropout rate is varied. The parameters that achieved the highest classification accuracy of 86.25 % after having been varied according to the semi-factorial design plan for this use case are seen in table 3. It could be seen that use of the dropout method within the CNN does not have any significant influence on the classification result for this application case. Both results for a high dropout rate and without dropout are similar. Dropout is an established and successful procedure in image recognition to avoid overfitting of a CNN. In most applications of image recognition very large amounts of data (order of magnitude 105 -106 ) are used. However, production applications often work with a very limited amount of data. Approaches that are successful for facial recognition or tasks in the field of autonomous driving cannot be adopted unseen for production applications. It can be assumed that for systems for automated quality assessment by defect detection and small amounts of data, use of the dropout method is not necessary. The complete semi-factorial design plan, that was executed in order to achieve and validate the parameters for the highest classification accuracy, can be found in the appendix to this publication. Table 3. Resulting training parameters for highest classification accuracy Parameter

Value

Batch Size

16

Training/Test Data Set

80/20

Dropout Rate

0

Epochs

50

Data Augmentation

Yes

Image Resolution

150px x 150px

190 6

M. Schreiber et al. / Procedia CIRP 86 (2019) 185–191 Schreiber et al. / Procedia CIRP 00 (2019) 000–000

5. Summary In image processing machine learning algorithms, in particular convolutional neural networks, are increasingly being used for classification tasks in quality assurance. The development of a system with high classification results by the selection of suitable parameters of a convolutional neural network is not trivial. For the experimental design, the semifactorial experimental plan consisting of a total of seven parameters was carried out. The system is prototypically applied in a production line for injection molded parts in a household appliances manufacturer. An accuracy of 86.25 % was determined as highest classification. In future work, these defects can be combined in an analysis with the process parameters, in order to enable a reliable prediction of component quality. Acknowledgements The OpenServ4P research and development project (www.openserv4p.de) is funded by the German Federal Ministry for Economic Affairs and Energy (BMWi) within the “Smart Service World” framework concept and is managed by the German Aerospace Center (DLR). References [1] Geiger, W., Kotte, W., 2008. Handbuch Qualität: Grundlagen und Elemente des Qualitätsmanagements: Systeme, Perspektiven, 5th ed. Friedr. Vieweg & Sohn Verlag, Wiesbaden. [2] Xu, B. (Ed.), 2017. Proceedings of 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC 2017): December 15-17, 2017, Chengdu, China. IEEE Press, Piscataway, NJ, 1 p. [3] Ahmadi, B., Javidi, B., Shahbazmohamadi, S., 2018. Automated detection of counterfeit ICs using machine learning. Microelectronics Reliability 8890, 371–377. [4] Aran, M.O., Nath, A.G., Shyna, A., 2016. Automated cashew kernel grading using machine vision, in: 2016 International Conference on Next Generation Intelligent Systems (ICNGIS), Kottayam, India. 9/1/2016 9/3/2016. IEEE, Piscataway, NJ, pp. 1–5. [5] Beyerer, J., Puente León, F., Frese, C., 2012. Automatische Sichtprüfung. Springer Berlin Heidelberg, Berlin, Heidelberg. [6] Ye, R., Pan, C.-S., Chang, M., Yu, Q., 2018. Intelligent defect classification system based on deep learning. Advances in Mechanical Engineering 10 (3). [7] Monostori, L., Kádár, B., Bauernhansl, T., Kondoh, S., Kumara, S., Reinhart, G., Sauer, O., Schuh, G., Sihn, W., Ueda, K., 2016. Cyberphysical systems in manufacturing. CIRP Annals 65 (2), 621–641. [8] Kagermann, H., Riemensperger, F., Hoke, D., Schuh, G., Scheer, A.-W., Spath, D., Leukert, B., Wahlster, W., Rohleder, B., Schweed, D., 2015. Smart Service Welt: Recommendations for the Strategic Initiative Webbased Services for Businesses. National Academy of Science and Engineering. [9] Lee, J., Kao, H.-A., Yang, S., 2014. Service Innovation and Smart Analytics for Industry 4.0 and Big Data Environment. Procedia CIRP 16, 3–8. [10] Steger, C., Ulrich, M., Wiedemann, C. (Eds.), 2018. Machine vision algorithms and applications, 2nd, completely revised and enlarged edition ed. Wiley-VCH, Weinheim, 494 pp. [11] Demant, C., Streicher-Abel, B., Springhoff, A., 2011. Industrielle Bildverarbeitung. Springer Berlin Heidelberg, Berlin, Heidelberg.

[12] Malamas, E.N., Petrakis, E.G.M., Zervakis, M., Petit, L., Legat, J.-D., 2003. A survey on industrial vision systems, applications and tools. Image and Vision Computing 21 (2), 171–188. [13] Han, J., Kamber, M., 2010. Data mining: Concepts and techniques, 2. ed., [Nachdr.] ed. Elsevier/Morgan Kaufmann, Amsterdam, 770 pp. [14] Richter, J., Streitferdt, D., Rozova, E., 2017 - 2017. On the development of intelligent optical inspections, in: 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA. 09.01.2017 - 11.01.2017. IEEE, pp. 1–6. [15] Nagata, F., Tokuno, K., Otsuka, A., Ikeda, T., Ochi, H., Tamano, H., Nakamura, H., Watanabe, K., Habib, M.K., 2018. Design Tool of Deep Convolutional Neural Network for Visual Inspection, in: Tan, Y., Shi, Y., Tang, Q. (Eds.), Data Mining and Big Data, vol. 10943. Springer International Publishing, Cham, pp. 604–613. [16] Haselmann, M., Gruber, D.P., 2019. Pixel-Wise Defect Detection by CNNs without Manually Labeled Training Data. Applied Artificial Intelligence 33 (6), 548–566. [17] Gobert, C., Reutzel, E.W., Petrich, J., Nassar, A.R., Phoha, S., 2018. Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Additive Manufacturing 21, 517–528. [18] Escobar, C.A., Abell, J.A., Hernández-de-Menéndez, M., MoralesMenendez, R., 2018. Process-Monitoring-for-Quality — Big Models. Procedia Manufacturing 26, 1167–1179. [19] Peres, R.S., Barata, J., Leitao, P., Garcia, G., 2019. Multistage Quality Control Using Machine Learning in the Automotive Industry. IEEE Access 7, 79908–79916. [20] Döbel, I., Leis, M., Vogelsang, M.M., Neustroev, D., Petzka, H., Riemer, A., Rüping, S., 2018. Maschinelles Lernen: Eine Analyse zu Kompetenzen, Forschung und Anwendung. Fraunhofer-Gesellschaft. [21] Wang, J., Perez, L., 2017. The Effectiveness of Data Augmentation in Image Classification using Deep Learning. Computing Research Repository (CoRR) 2017 (1712.04621). [22] Wendler, T., Gröttrup, S., 2016. Data Mining with SPSS Modeler. Springer International Publishing, Cham. [23] Ertugrul, M., 2013. Serviceorientierte Plattformen in mittelstandischen Unternehmen. Grin Verlag. [24] Richter, J.-P., Haller, H., Schrey, P., 2005. Serviceorientierte Architektur. Informatik Spektrum 28 (5), 413–416. [25] Beierle, C., Kern-Isberner, G., 2014. Methoden wissensbasierter Systeme: Grundlagen, Algorithmen, Anwendungen, 5., überarb. und erw. Aufl. ed. Springer Vieweg, Wiesbaden, 545 pp. [26] Klöber-Koch, J., Schreiber, M., Klimm, B., Reinhart, G., 2018. Vorausschauende Instandhaltung für Fertigungsressourcen: wt Werkstattstechnik online 108 (3), 155–159. [27] Schreiber, M., Klöber-Koch, J., Richter, C., Reinhart, G., 2018. Integrated Production and Maintenance Planning for Cyber-physical Production Systems. Procedia CIRP 72, 934–939. [28] Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep Learning: Adaptive Computation and Machine Learning. MIT Press, Cambridge, Massachusetts, USA, 775 pp. [29] Luckow, A., Cook, M., Ashcraft, N., Weill, E., Djerekarov, E., Vorster, B., 2016. Deep Learning in the Automotive Industry: Applications and Tools. IEEE International Conference on Big Data (Big Data) 2016, 3759–3768. [30] Nagata, F., Tokuno, K., Tamano, H., Nakamura, H, T., M., K., K., O., A., I., 2018. Basic application of deep convolutional neural network to visual inspection. Proceedings of International Conference on Industrial Application ICIAE2018, 5. [31] Ahlin, B.T.I., Gärdin, E.M., 2017. Automated Classification of Steel Samples: An investigation using Convolutional Neural Netwroks. Degree Project in Technology, Stockholm, Schweden. [32] Inoue, H., 2018. Data Augmentation by Pairing Samples for Images Classification. [33] Jia, S., Wang, P., Jia, P., Hu, S., 2018. Research on data augmentation for image classification based on convolution neural networks. Chinese Automation Congress, 4165–4170. [34] Krizhevsky, A., Ilya Sutskever, and Geoffrey E. Hinton, 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems (25), 1097–1105.

Schreiber et al. / Procedia CIRP 00 (2019) 000–000 M. Schreiber et al. / Procedia CIRP 86 (2019) 185–191



7 191

Appendix Table 4. Complete semi-factorial design plan that was executed in order to achieve and validate the parameters for the highest classification accuracy. Exp. Number

Size of the Data Set

Classification Result during Training

Classification Result during Test

[px x px]

[parts]

[%/100]

[%/100]

Yes

150x150

33

0.7650

0.7850

Yes

150x150

33

0.8331

0.7871

300

Yes

150x150

33

0.7500

0.7797

300

Yes

150x150

33

0.9605

0.7391

60/40

300

Yes

150x150

33

0.9491

0.7480

60/40

300

Yes

150x150

33

0.7188

0.7119

0

60/40

300

Yes

150x150

33

0.7015

0.6886

0

60/40

300

Yes

150x150

33

0.9626

0.7642

8

0

60/40

300

Yes

150x150

33

0.6164

0.7627

16

0.5

70/30

300

Yes

150x150

33

0.8164

0.7087

11

8

0.5

70/30

300

Yes

150x150

33

0.8009

0.7471

12

16

0.2

70/30

300

Yes

150x150

33

0.9219

0.7282

13

8

0.2

70/30

300

Yes

150x150

33

0.8295

0.7658

14

16

0

70/30

300

Yes

150x150

33

0.9023

0.7476

15

8

0

70/30

300

Yes

150x150

33

0.9091

0.7748

16

16

0.5

80/20

300

Yes

150x150

33

0.8355

0.7667

17

8

0.5

80/20

300

Yes

150x150

33

0.8594

0.7833

18

16

0.2

80/20

300

Yes

150x150

33

0.8725

0.7542

19

8

0.2

80/20

300

Yes

150x150

33

0.7969

0.8000

20

16

0

80/20

300

Yes

150x150

33

0.9452

0.7705

21

8

0

80/20

300

Yes

150x150

33

0.8594

0.8333

22

8

0.5

80/20

50

Yes

150x150

33

0.6719

0.6833

23

8

0.2

80/20

50

Yes

150x150

33

0.7031

0.6667

24

8

0

80/20

50

Yes

150x150

33

0.6250

0.7500

25

16

0.5

80/20

50

Yes

150x150

33

0.7690

0.8125

26

16

0.2

80/20

50

Yes

150x150

33

0.7937

0.8167

27

16

0

80/20

50

Yes

150x150

33

0.8203

0.8625

28

16

0

80/20

50

No

150x150

33

0.9804

0.8458

29

16

0

80/20

50

No

150x150

50

0.9490

0.7897

30

16

0

80/20

50

Yes

150x150

50

0.7922

0.8016

31

16

0

80/20

50

Yes

512x512

50

0.6992

0.6746

32

16

0

80/20

300

Yes

512x512

50

0.6602

0.6429

33

8

0

80/20

50

Yes

150x150

50

0.7188

0.6875

34

8

0

80/20

300

Yes

150x150

50

0.7188

0.6935

35

16

0

80/20

300

Yes

512x512

50

0.6746

0.6667

Parameter 1:

Parameter 2:

Parameter 3:

Parameter 4:

Parameter 5:

Parameter 6:

Batch Size

Dropout Rate

Epochs

Data Image Augmentation Resolution

[images]

[%/100]

Split Ratio Training / Test Data Set [%/%]

[-]

[-]

1

32

0.5

60/40

300

2

16

0.5

60/40

300

3

8

0.5

60/40

4

32

0.2

60/40

5

16

0.2

6

8

0.2

7

32

8

16

9 10

Parameter 7:

2212-8271 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 7th CIRP Global Web Conference