- Email: [email protected]
Big Data Analytics for Processing Time Analysis in an IoT-enabled manufacturing Shop Floor
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Procedia Manufacturing 00 (2018) 000–000 Available online atatwww.sciencedirect.com Available online www.sciencedirect.com Procedia Manufacturing 00 (2018) 000–000
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
ScienceDirect ScienceDirect Procedia Manufacturing 26 (2018) 1411–1420 Procedia Manufacturing 00 (2017) 000–000
www.elsevier.com/locate/procedia
46th th
SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA 46 SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA
Big Big Data Data Analytics Analytics for for Processing Processing Time Time Analysis Analysis in in an an IoT-enabled IoT-enabled Shop Floor Manufacturing Engineeringmanufacturing Society International Conference 2017, MESIC 2017, 28-30 June manufacturing Shop Floor 2017, Vigo (Pontevedra), Spain Daniel D. Kho, Seungmin Lee, Ray Y. Zhong** Daniel D. Kho, Seungmin Lee, Ray Y. Zhong
Costing models for capacity optimization in Industry 4.0: Trade-off Department of Mechanical Engineering, The University of Auckland, Auckland, New Zealand Department of Mechanical Engineering, The University of Auckland, Auckland, New Zealand between used capacity and operational efficiency * Corresponding author. Tel.: +64-9-923-1584. a a,* b * Corresponding Tel.: +64-9-923-1584. E-mail address:author. [email protected] E-mail address: [email protected] a University of Minho, 4800-058 Guimarães, Portugal b Unochapecó, 89809-000 Chapecó, SC, Brazil
A. Santana , P. Afonso , A. Zanin , R. Wernkeb
Abstract Abstract Internet of things (IoT) technology has been widely used in manufacturing where great myriad of data have been generated. This Internet of things a(IoT) technology hasforbeen widely in manufacturing where great myriad datawhich have uses been radio generated. This paper introduces big data analytics Internet of used Things (IoT)-enabled manufacturing shop of floor frequency Abstract paper introduces a bigtechnology data analytics for Internet Things (IoT)-enabled manufacturing shop which uses radio frequency identification (RFID) for capturing theofreal-time production data. This involves twofloor machine learning techniques: kidentification (RFID) technology for capturing the for real-time production data. This involves machine this learning techniques: kmeans clustering and gradient descent optimization the data from the manufacturing sites. two In particular, research deals with Under the concept of "Industry 4.0", production be pushedsites. to In beparticular, increasingly interconnected, means and gradient descent processes optimization the processes databatches from thewill manufacturing this research deals with the realclustering RFID data related to various for for individual and processing time. This could produce valid predictions information onmanufacturing a real time basis and, necessarily, much more efficient. In this context, capacity optimization the real RFIDbased data related to various processes individual and processing This could produce valid predictions about expected overall time for a for given number batches of manufacturing batch time. inputs. goes traditional aim of capacity also forinputs. organization’s profitability and value. aboutbeyond expectedthe overall manufacturing time for amaximization, given number ofcontributing manufacturing batch Indeed, lean management continuous © 2018 The Authors. Publishedand by Elsevier B.V. improvement approaches suggest capacity optimization instead of © 2018 2018 The The Authors. Published by Elsevier B.V. B.V. © Authors. Published by Elsevier Peer-review under responsibility of the scientific committee NAMRI/SME. maximization. The study of capacity optimization and costing models is American an important research Research topic that deserves Peer-review under responsibility of the scientific committee ofofthe 46th SME North Manufacturing Conference. Peer-review under responsibility of the scientific committeeperspectives. of NAMRI/SME. contributions from both the practical and theoretical This paper presents and discusses a mathematical
Keywords: Data Analytics; RFID; IoT; Manufacturing; Shopcosting Floor. models (ABC and TDABC). A generic model has been model forBig capacity management based on different Keywords: Big Data Analytics; RFID; IoT; Manufacturing; Shop Floor. developed and it was used to analyze idle capacity and to design strategies towards the maximization of organization’s value. The trade-off capacity maximization vs operational efficiency highlighted andremarkable it is showntechnologies that capacity practices,is and one of the is 1. Introduction optimization might hide operational inefficiency. practices, and one of the remarkable technologies 1. Introduction radio frequency identification (RFID) [6-8]. is It © 2017 The Authors. Published by Elsevier B.V. radio frequency identification [6-8]. It enhances the configurability and (RFID) reusability during Internet of Things (IoT) technology has been Peer-review under responsibility of technology the scientific committee of the Manufacturing Engineering Society International Conference enhances the configurability and reusability during Internet of Things (IoT) has been manufacturing processes [9]. Smart objects with the widely used for enabling intelligent manufacturing so 2017. manufacturing processes Smart objects with widely used for enablingobjects intelligent manufacturing so RFID tags installed can [9]. capture fluctuations in the the that statuses of physical could be tracked and RFID tags installed can capture fluctuations in the that statuses of physical objects could be tracked and manufacturing surroundings and changes in quantity traced [1-3]. IoT is an ecosystem of technologies Keywords: Cost Models; ABC; TDABC; Capacity Management; Idle Capacity; Operational Efficiency manufacturing and changesofinthe quantity traced [1-3]. is anofecosystem of technologies [10]. The mostsurroundings beneficial usefulness IoTmonitoring theIoT statuses physical objects, capturing [10]. The most beneficial usefulness of the IoTmonitoring the statuses of physical objects, capturing enabled manufacturing is the reporting of data in realmeaningful data, and transferring the information enabled manufacturing is the reporting of data in realmeaningful data, and transferring the information time [11-14]. This empowers the users to have agilities through IP networks to software applications [4, 5]. 1. Introduction time [11-14]. This empowers users have agilities through IP networks to software applicationsindustry [4, 5]. in making accurate decisions the based onto such data. IoT technology is used in manufacturing in making accurate decisions based on such data. IoT technology is used in manufacturing industry The cost of idle capacity is a fundamental information for companies and their management of extreme importance in modern systems. In general, it isB.V. defined as unused capacity or production potential and can be measured 2351-9789 ©production 2018 The Authors. Published by Elsevier 2351-9789 2018responsibility The Authors. Published by Elsevier B.V.hours in several©under ways: tons of production, available of manufacturing, etc. The management of the idle capacity Peer-review of the scientific committee of NAMRI/SME. Peer-review underTel.: responsibility the761; scientific committee NAMRI/SME. * Paulo Afonso. +351 253 of 510 fax: +351 253 604of741 E-mail address: [email protected]
2351-9789 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the Manufacturing Engineering Society International Conference 2017. 2351-9789 © 2018 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 46th SME North American Manufacturing Research Conference. 10.1016/j.promfg.2018.07.107
1412 2
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
Big Data Analytics is the process of evaluating a large volume of various datasets to discover patterns, correlations, market trends and other useful information that can help organizations to make more informed decisions [15-17]. This involves manipulating any types of information that can be computed to represent characteristics of an object or a person, such as user ID or a person's age. The Big Data Analytics will be of paramount use in the performances of the manufacturing plants. For instance, a mix of business intelligence (BI) to analyse market, detect fraud and carry out financial risk assessment is popularly used by businesses [18]. A realization of analysis is achieved by numerous Big Data Analytics related tools, such as Oracle's Big Data appliance and Cloudera's Hadoop distribution [19]. Other typical tools include GridGain for high performance computing cluster (HPCC), Storm for dealing with huge data sets with distributed real-time computation capabilities and Talend for providing a number of BI service [20]. Processing large data and producing meaningful patterns instantly are the preeminent advantages of these tools. Amazon Elasticsearch Service provides log analytics service which interprets logs generated by many sources, such as websites, mobile devices, servers and sensors for application monitoring, fraud detection and IoT [21]. Apache Kafka is used for building real-time data pipelines streaming applications by having publish and subscribe mechanism to have streams of data like a messaging system, and stores data in a distributed replicated cluster [22]. These tools are accessible as platforms and tools and programming languages to stimulate businesses to improve their processes and efficiencies through exploring Big Data Analysis. There was a satisfactory amount of cases in manufacturing industries using Big Data Analytics for different purposes [23]. However, it was scarcely reported that the streams of real-time data are analysed through the interoperable machines in the IoT-enabled manufacturing and the application of decentralized decisions through the adoption of machine-learning techniques to implement maximum automation. Despite the data being post-processed in this research, it is highly feasible that the algorithm that is used for extracting notable interpretations could be accommodated for the real-time implementation. It was essential to capture the real-time production data due to the need for testing if the proposed algorithm would work well with the actual data. Furthermore, studies on the use of RFID data from smart manufacturing objects (SMOs) on an intelligent shop floor is limitedly reported [24]. With the primary aim
of capacitating these inadequacies, this study proposes to devise a Big Data Analytics for the IoT-enabled manufacturing. This paper is organized as follows. Section 2 gives the dataset definition. Section 3 presents the proposed Big Data Analytics approach. Section 4 shows the experimental results and some discussions. Section 5 concludes this paper by highlighting our contributions and future research directions. 2. Definition of Datasets The RFID data are from an IoT-enabled manufacturing shop floor. A piece of RFID data is recorded when there is a detection between a reader and a tag or an operation of the RFID readers, such as a clicking on the keyboard. Table 1. Definitions of RFID Datasets. Data Code ID
Definition
Product Code
BatchMainID Required group of materials to manufacture a product UserID ProcCode
Individual worker/operator at manufacturing plant Type of mechanical manufacturing process
ProcSeqnum Order in which each process is taken place Quantity GoodNum Time Location TimeFloat
Number of product produced Quantity of product that passes the acceptance threshold Period of record at each event occurrence Warehouse that archives the batches of materials Time expressed as float 32 bits in days from 01/01/2000
There are four main relationships that can be deduced from manipulating these datasets. One is tracking the specific BatchMainID over the period of time and comparing the total time taken to finish the batch. UserID and ProcCode can be used to observe the performance of the operator on the shop floor. The relationship between the locations in each BatchMainID stays over buffering time can be also established. Also, tracking each BatchMainID to see the number of operators required to complete the batch would be significant.
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
3. Big Data Analytics Approach for Production Performance Analysis The proposed Big Data Analytic approach involves data analysis, machine learning and data visualizations. TensorFlow open-source software library with the capacity of accomplishing these task operations is selected in this paper. 3.1. Containerization Containerization is a virtualization method for deploying and running distributed applications. It is enabled by providing an additional layer of abstraction, called container engine, which are run on a single control host and access a single kernel [25]. This approach not only achieves virtualization, cost reduction, scalability, and unnecessity of kernel resource duplication, but also its advantages come from compatible configurations. This means, when moved from one computing environment to another, the containers will be easily configured for new applications. This could be from a laptop to a test environment, from a staging environment into production, and from a physical machine in a data center to a virtual machine in a private or public cloud. Therefore, an application from the software will be consisted of all its dependencies, libraries and other binaries, and configuration files. By containerizing the application platform and its dependencies, differences in OS distributions and underlying infrastructure are abstract. That means end-users are empowered to adopt the software compatibly regardless of hardware discrepancies.
1413 3
The command used is docker exec -it ContainerName /bin/bash. Inside the container, a python file is created to implement the algorithm and the run command is PythonFileName.py. The container can be exited with the command exit and it can stop the running container behind by the command docker stop ContainerName. If the container needs to be started again, simple command docker restart ContainerName is used. 3.2. Visualization TensorFlow includes TensorBoard shown in Fig. 1, which is a data visualization toolkit. This will be the graphical platform for displaying nodes and different types of graphs to suit different kinds of data analysis. Fig. 1 also shows the number of graphs that were obtained by implementing some algorithms in this paper. The web-based localhost server display of the data analytic visualization promotes user-friendliness and clear UI Workflow.
Fig. 2. Architecture of the proposed approach
3.3. Architecture
Fig. 1. TensorBoard
3.3.1. Workflow The general workflow of the proposed solution is shown in Fig. 2. This architecture involves four main processes: data cleansing or pre-processing, classification, pattern recognition and visualization. It begins with a large set of RFID data from the manufacturing shop floor and is completed by producing a visualization of useful information regarding the performance of the manufacturing plant. The whole design works in a way in which the K-
1414 4
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
means clustering forms different clusters to represent closely-related data. These individual clusters provide the linear or non-linear regressions for predictions. The predictions are verified by the proposed probability algorithm that is tested by streaming a series of inputs with varying batch sizes. 3.3.2. Data Pre-processing The RFID data is provided by the commencement without the TimeFloat. It was modified to be a correct format during importing the data into the TensorFlow program. As the Time follows an arrangement in days, months and years format consecutively (DD/MM/YYYY), the program was unable to accept the values as it only allowed the numeric types, such as integer or float. Based on the date starts on 1900/01/01, the conversion into float type results in 39512.62 days, representing the first data entry which is recorded in 2008/05/03. The initial date was changed to 2000/01/01 which made the time difference to be 3012.618 days between 2000/01/01 and 2008/05/03. The floating-point numbers represent the smaller units of time, such as hours, minutes and seconds which can be calculated by multiplying by 24, (24x60) and (24x60x60) respectively. However, it must be noted that this kind of conversion ignores the effect of the Leap year. With the Leap year effect, a transition of the initial counting time from 1900/01/01 to 2000/01/01 has the time difference of about 36524 days. This is because the Leap year states that there are actually 365.2422 days in a year. For the convenience of calculations, it was assumed that the Leap year effect is negligible. The RFID dataset contains a total of 413,472 pieces. Time is an important feature for the analysis when the completion time of the BatchMainIDs are considered. Therefore, the data with no time entries were all removed for more relevance and faster processing. The total number of remaining data was amounted to 376,746 samples. Consequently, any analysis or programming using the data cannot exceed more than the data size. Although the data size is limited to 376,746 entries, these have been collected from the advanced sensors from the manufacturing shop floors which have been insufficiently reported. Different samples of data will be used to evaluate the predicted model by comparing the regression line and the centroids. It indicates that the model based on the first 10,000 data needs to be examined against the last 10,000 data, which are input for the examination. To compensate for the further randomness and validate
the model, the random sampled 10,000 data were compared in the same way. 3.4. BatchMainID vs Time Data analysis can be carried out by using the relationship between the BatchMainID and Time to produce statistically important remarks. Fig. 3 shows the plotting of all the 376,746 samples of matching BatchMainID processes and Time. A region covered by all the samples so that the hidden patterns or trends are difficult to identify. Additionally, the computation cost of the whole data is immense. From this figure, the datasets are challengeable to be clustered.
Fig. 3. All of BatchMainID VS Time in days
3.5. Batch Identification Algorithm The algorithm tracks the individual BatchMainIDs to obtain the start and finish Time and counts a number of processes to complete a batch. BatchMainID and TimeFloat are used for this purpose. Fig. 4 illustrates a result for a specific BatchMainID. The 5 scatter points means this BatchMainID had been conducted in 5 manufacturing processes. A time difference (diffTime) and a number of processes to complete a specific BatchMainID (noofProc) have been generated for further algorithms. The diffTime has been calculated by subtracting the last TimeFloat from start TimeFloat for a BatchMainID. The noofProc has been calculated by counting the number of manufacturing processes. Therefore, how much time was used and how many processes were used to complete a batch could be identified.
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018) 𝑘𝑘
𝑗𝑗
𝐽𝐽 = ∑ ∑‖𝑥𝑥𝑖𝑖 − 𝑐𝑐𝑗𝑗 ‖ 𝑗𝑗=1 𝑖𝑖=1
where 𝑗𝑗
𝑛𝑛
1415 5 2
2
‖𝑥𝑥𝑖𝑖 − 𝑐𝑐𝑗𝑗 ‖ is the squared distance between the n data 𝑗𝑗 points 𝑥𝑥𝑖𝑖 and the k centroids 𝑐𝑐𝑗𝑗 .
Fig. 4. Identification of a Single Batch
3.6. Machine Learning Algorithms A machine learning method used in this paper is an integration of supervised and unsupervised learning processes. The supervised learning has the labelled data which means the independent variable forms a relationship with the dependent variable which is already defined by humans, whilst the unsupervised approach has no labelled data. In this research, the kmeans clustering which is an unsupervised learning method is used first to group the data into a number of clusters. The gradient descent optimization algorithm which is a supervised learning method is then used to calculate the predictions and reduce the loss of the cost or the object function.
In this paper, 5 clusters were used for an adequate classification. The colors used were red, blue, green, black and purple to represent cluster 0, 1, 2, 3 and 4 respectively. The individual clusters are separately graphed, and each of them undergoes gradient descent optimizations. The overall graph is shown in Fig. 5 where the centroids are depicted as cross signs.
Fig. 5. Five clusters in 10000 samples
3.6.1. K-means Clustering
3.6.2. Gradient Descent Optimization
The K-means algorithm is one of the well-known unsupervised learning algorithm that solves the clustering problem [26]. To use this algorithm, the number of k centroids needs to be defined manually which corresponds to the number of clusters the user aims to have. The centroids need to be placed relatively separated from each other to achieve a clear distinction between each cluster. The next step is to associate each data point to its nearest centroid. When all the points are assigned, k new centroids need to be re-positioned to update from the previous step. The allocated data points among the clusters go to the loop again. It can be noticed that k centroids move for every step until no more changes are required. Mathematically, the K-means algorithm minimizes the squared error function J.
The Gradient Descent Optimization algorithm seeks to search for the global minimum of a function [27]. It takes the derivative of a function at a point that gives the direction to move towards. It makes steps down the function J in the direction in the negative gradient. The size of each step is determined by the learning rate α. 𝜕𝜕 𝐽𝐽(𝜃𝜃0 , 𝜃𝜃1 ) 𝜃𝜃𝑗𝑗 ∶= 𝜃𝜃𝑗𝑗 − α 𝜕𝜕𝜃𝜃𝑗𝑗 where 𝑗𝑗 = 0,1 represents the index number. θ is the parameter in the function. By changing 𝜃𝜃𝑗𝑗 for each iteration, the average squared distance between the object function and the proposed regression is minimized. The results could differ slightly due to the random weights and constant values generated during the
1416 6
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
experiments. After testing, it was resolved that the weights can change between 0 and 50, constants can change between -10 and 10, the learning rate is 0.015 and the iterations are 100 times. The weights located in the positive plane should be selected for the regression. The range between the negative and positive values used for the constants is to account for the position of the regression. As shown in Fig. 6, inverse functions were fitted to each cluster except the cluster 0 and 1 as they used logarithmic functions and undefined function respectively.
Fig. 6. Cluster 0-4
3.7. Prediction Algorithm The prediction algorithm is implemented by feeding the numbers range from 1 to the maximum number of BatchMainID processes. These need to be committed into different clusters. This was attempted by collecting the number of colored cluster points at which the numbered process is fed. The algorithm calculates the probability of having the most number of colored points over the total number of colored points. In Fig. 5, there are 7 purple points and 17 green points at 𝑥𝑥 = 12. If the new BatchMainID takes 12 processes to complete, the probability of the time that will take to finish the processes is 17/24 = 70.83% and the process belongs to a green cluster since it is the dominant color at 𝑥𝑥 = 12.
4. Results and Discussion 4.1. Predictions
The results from the prediction algorithm are shown in Table 2. The input is the array of one dimensional tensor of numbers which depict the
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
number of processes. The output shows the assignment of those processes to the cluster numbers respectively. The prediction percentage shows how accurate each assignment of batches to a certain cluster is decided. It is observed that this is a more reliable approach than calculating the general probability for the predictions. The general probability only considers the likelihood of retrieving one scatter point out of all. Table 2. Prediction Results.
Prediction Input, (Number of Processes) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Prediction Cluster 3 3 3 3 3 4 2 2 2 2 2 2 2 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0
Probability or Prediction Percentage (%), 100.00 100.00 82.47 92.31 44.62 84.23 80.95 87.50 73.47 91.30 70.37 70.83 100.00 53.33 77.78 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00
Prediction Output, (Time Difference in Days) 2.00 1.17 0.90 0.76 0.68 2.49 8.93 8.17 7.57 7.10 6.71 6.39 6.11 5.88 5.67 8.42 9.22 9.98 10.69 11.37 12.02 12.64 13.23 13.79 14.33 14.85 15.35 15.83
This approach depends on the total number of batch processes. However, the use of the proposed prediction algorithm enables the probability to reflect on the dominant number of scatter points that belong to specific clusters whilst also considering the total number of scatter points. It increases likelihood of the future BatchMainID placed into the most relevant cluster. In the Fig. 7, the computation cost is examined. It is observed that the computation cost is increasing as the number of samples increases. This is very reasonable due to the time taken to operate our algorithm for the data pre-processing, visualizing and optimizing. Due to the uncertainty of the error whilst
1417 7
processing the big data, the computation cost has been calculated up to 10,000 samples. The experiment took place as the 5 operations were run and they were averaged out per number of samples. This was because the computation cost is different for each run even if the number of samples is the same due to the random number generations within the algorithm, such as weights, constants and Prediction Input values. It is observed from Fig. 7, the computation cost has a linear relationship with the number of samples.
Fig. 7. Computation Cost
The proposed approach is tested with the Intel(R) Core(TM) i7-6500U CPU at 2.50GHz. The computation cost could be generally improved if the program was experimented with the NVIDIA GPU support, because TensorFlow programs are significantly faster on a GPU than on a CPU.
Fig. 8. Gradient Descent Optimizations (a) Cluster 0; (b) Cluster 2; (c) Cluster 3
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
1418 8
4.2. Parameters The parameters of our interest are weights (w) and constants (b). Fig. 8 is the illustration of the gradient descent optimizations that were conducted by 100 times for each cluster. This resulted in producing a non-linear regression lines in each cluster. The results are shown in multiples of 10 for a clear visualization. w and b represent weights and constant values respectively. They are the factors that determine the position and the direction of the regression lines. The weights and constant values change as the iterations are completed. These two variables change in a way that the loss is effectively reduced to zero. Fig. 8 clearly displays the loss is being reduced significantly over the iterations. In Fig. 8 (a), the Cluster 0 experienced the loss of 1311.06 initially and it has been reduced to 12.6655, which is the 99% improvement after 100 iterations. Even the least improvement in the Cluster 3 from Fig. 8 (c) experienced 86% improvement in minimizing the loss. These results reinforce the reliability and the strong diminishing loss effect of our model. Therefore, equations can be formulated for each cluster by computing the 100th weights and constants. For instances, for Cluster 0, the model equation will be: 𝑓𝑓(𝑥𝑥) = 13.25 × ln(𝑥𝑥) − 28.32 and the equation for the Cluster 2 will be 42.71 + 2.83. 𝑓𝑓(𝑥𝑥) = 𝑥𝑥
The model parameters, such as ln(𝑥𝑥) and inverse function have been selected after various trials of other mathematical models, such as linear and quadratics. This approach had to be intuitive as the focus is minimizing the loss of the chosen objective functions to represent the predictions of the data. This is validated by the number of trials when the loss appears to be improving no more. Therefore, the training performance results could be identified how much the loss has been reduced. Further adjustment could be made to decrease the loss by changing the range of weights and constants, so that the gradient descent algorithm can explore more opportunities to reach its most accurate global minima. The learning rate can also be adjusted to influence on the loss. The current learning rate is 0.015. However, the adjustment should be considered by the computation time and the direction of the gradient. Lower learning rate will induce immense computation time due to the limited computing resources. In contrary, higher learning rate
will drive the direction of the gradient opposite to the global minima. Furthermore, the number of optimizations completed also have a noticeable impact on the level of loss. In this experiment, if less iterations were conducted instead of 100 iterations, the loss would have been inaccurate as it can be seen in the Fig. 8. This means raising the number of optimizations would improve the quality of the optimization results. However, this can be limited by the nature of our mathematical model as the loss was not lessened much in the Fig. 8 (a), Cluster 0. Normally, this type of steady loss should be left as it is, because the change in any variables may be able to reduce it further, but this may have other effects on the other clusters.
Fig. 9. Evaluation with last 10,000 samples (a) Cluster 2; (b) Cluster 3; (c) Cluster 4
4.3. Evaluation Evaluation is required to validate the models. In this research, different samples are applied to the model for evaluation purposes. Not all data has been used for the analysis algorithms due to the large computational cost. Therefore, the model generated by first 10,000 samples are compared with lastly selected 10,000 samples. This is due to by experience that analyzing the data size over 10,000 creates a high computation cost when the learning takes place using all the proposed algorithms. Also, the more the data is plotted, the less the quality of the visibility becomes. It is compromised to depict the whole data set within the data size of 10,000 limit based on the tests of number of program runs with varying data sizes. The clustering result depends on the input size. This is validated by the shapes of the graphs of the last 10,000 samples. There is a significant similarity in the form and the contour between the first 10,000 samples and
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
the last 10,000 samples. Figure 9 (a) shows a classified scatter plot with last 10,000 samples. It describes that no dataset is assigned to red and blue clusters due to a shorter diffTime and noofProc range of the last 10,000 samples. As the raw dataset has the chronological trends, the last part of the data usually contains latter manufacturing RFID data. It is assumed that the manufacturing company retrieves the experiences of its products and increases the efficiency of manufacturing processes and time. However, it used the same K-means clustering centroids as the first 10,000 samples. Therefore, it is acceptable to assume that the similar batch products are classified as the same clusters. In addition, a visual analysis of Fig. 9 (b), (c), and (d) shows that the non-linear regression is also valid for these samples. 5. Conclusion This paper introduces a Big Data Analytics approach for examining the massive RFID data from an IoT-enabled manufacturing shop floor. The approach includes data processing and visualization. Evaluation studies are conducted for validating the approach. Some contributions are significant from this paper. The prediction method returned the time output with above 50% of validation. The big data acquired from the manufacturing site can be analyzed to make predictions and to improve efficiency. A combination of gradient descent and clustering methods is proposed. The losses were minimized at a reasonable level when the algorithm is tested with a wide range of samples. The computation cost will likely to increase in a non-linear manner as the data size increases. Future research directions will be carried out as follows. First of all, the visualization of the analysis could be implemented in numerous platforms, such as mobiles and tablets for the ease of access. Different parameters could be compared to produce further meaningful findings to the manufacturers, for example comparing the UserID against the ProcCode, specific BatchMainIDs in different locations against time and different BatchMainIDs against the number of UserIDs. Secondly, some questions such as many processes are handled by a single employee, how many employees are required to complete the batch are required further investigation. Finally, the software
1419 9
design principles could be taken into account to improve the readability, scalability, effectiveness and efficiency of the proposed approach. Acknowledgements Authors would like to give thanks to Huaiji Dengyun Auto-Parts for providing the data necessary for this analysis. References [1] R. Y. Zhong, X. Xu, and L. H. Wang, "IoT-enabled Smart Factory Visibility and Traceability using Laser-scanners," Procedia Manufacturing, 2017. [2] R. Y. Zhong, L. H. Wang, and X. Xu, "An IoT-enabled Realtime Machine Status Monitoring Approach for Cloud Manufacturing," Procedia CIRP, vol. 63, pp. 709-714, 2017. [3] X. Qiu, H. Luo, G. Y. Xu, R. Y. Zhong, and G. Q. Huang, "Physical assets and service sharing for IoT-enabled Supply Hub in Industrial Park (SHIP)," International Journal of Production Economics, vol. 159, pp. 4-15, 2015. [4] R. Y. Zhong, G. Q. Huang, S. L. Lan, and M. L. Wang, "IoTenabled Building Information Modelling Cloud for Prefabrication Construction," The International Conference on Flexible Automation and Intelligent Manufacturing (FAIM), 23-26 June, Wolverhampton, UK, 2015. [5] F. Xia, L. T. Yang, L. Wang, and A. Vinel, "Internet of things," International Journal of Communication Systems, vol. 25, p. 1101, 2012. [6] R. Y. Zhong, Y. Peng, F. Xue, J. Fang, W. Zou, H. Luo, et al., "Prefabricated construction enabled by the Internet-of-Things," Automation in Construction, vol. 76, pp. 59-70, 2017. [7] S. Mejjaouli and R. F. Babiceanu, "RFID-wireless sensor networks integration: Decision models and optimization of logistics systems operations," Journal of Manufacturing Systems, vol. 35, pp. 234-245, 2015. [8] C. Lee, G. Ho, K. Choy, and G. Pang, "A RFID-based recursive process mining system for quality assurance in the garment industry," International Journal of Production Research, vol. 52, pp. 4216-4238, 2014. [9] J. H. Lee, J. H. Song, K. S. Oh, and N. Gu, "Information lifecycle management with RFID for material control on construction sites," Advanced Engineering Informatics, vol. 27, pp. 108-119, 2013. [10] R. Y. Zhong, Q. Dai, T. Qu, G. Hu, and G. Q. Huang, "RFIDenabled real-time manufacturing execution system for masscustomization production," Robotics and Computer-Integrated Manufacturing, vol. 29, pp. 283-292, 2013. [11] M. L. Wang, T. Qu, R. Y. Zhong, Q. Y. Dai, X. W. Zhang, and J. B. He, "A radio frequency identification-enabled real-time manufacturing execution system for one-of-a-kind production manufacturing: a case study in mould industry," International Journal of Computer Integrated Manufacturing vol. 25, pp. 2034, 2012. [12] Q. Y. Dai, R. Y. Zhong, G. Q. Huang, T. Qu, T. Zhang, and T. Y. Luo, "Radio frequency identification-enabled real-time manufacturing execution system: a case study in an automotive part manufacturer," International Journal of Computer Integrated Manufacturing vol. 25, pp. 51-65, 2012. [13] Y. F. Zhang, P. Jiang, and G. Huang, "RFID-based smart kanbans for just-in-time manufacturing," International Journal of Materials and Product Technology, vol. 33, pp. 170-184, 2008.
1420 10
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
[14] Y. F. Zhang, T. Qu, O. K. Ho, and G. Q. Huang, "Agent-based smart gateway for RFID-enabled real-time wireless manufacturing," International Journal of Production Research, vol. 49, pp. 1337-1352, 2011. [15] R. Y. Zhong, C. Xu, C. Chen, and G. Q. Huang, "Big Data Analytics for Physical Internet-based intelligent manufacturing shop floors," International Journal of Production Research, vol. 55, pp. 2610-2621, 2017. [16] A. Zaslavsky, C. Perera, and D. Georgakopoulos, "Sensing as a service and big data," Proceedings of the International Conference on Advances in Cloud Computing (ACC), vol. 2, pp. 1-8, 2013. [17] R. M. Ward, R. Schmieder, G. Highnam, and D. Mittelman, "Big data challenges and opportunities in high-throughput sequencing," Systems Biomedicine, vol. 1, pp. 0--1, 2013. [18] G. Schuh, C. Reuter, J.-P. Prote, F. Brambring, and J. Ays, "Increasing data integrity for improving decision making in production planning and control," CIRP Annals-Manufacturing Technology, vol. 66, pp. 425-428, 2017. [19] R. Y. Zhong, S. T. Newman, G. Q. Huang, and S. Lan, "Big Data for supply chain management in the service and manufacturing sectors: Challenges, opportunities, and future perspectives," Computers & Industrial Engineering,vol. 101, pp. 572-591, 2016. [20] D. Loshin, Big data analytics: from strategic planning to enterprise integration with tools, techniques, NoSQL, and graph: Elsevier, 2013. [21] K. H. Tan, Y. Zhan, G. Ji, F. Ye, and C. Chang, "Harvesting big data to enhance supply chain innovation capabilities: An analytic infrastructure based on deduction graph," International Journal of Production Economics, vol. 165, pp. 223-233, 2015. [22] R. Ranjan, "Streaming big data processing in datacenter clouds," IEEE Cloud Computing, vol. 1, pp. 78-83, 2014. [23] H. Chen, R. H. Chiang, and V. C. Storey, "Business intelligence and analytics: From big data to big impact," MIS quarterly, vol. 36, pp. 1165-1188, 2012. [24] R. Y. Zhong, G. Q. Huang, S. L. Lan, Q. Y. Dai, C. Xu, and T. Zhang, "A Big Data Approach for Logistics Trajectory Discovery from RFID-enabled Production Data," International Journal of Production Economics, vol. 165, pp. 260-272, 2015. [25] M. J. Scheepers, "Virtualization and containerization of application infrastructure: A comparison," Proceeding in 21st Twente Student Conference on IT, pp. 1-7, 2014. [26] D. Feldman, M. Schmidt, and C. Sohler, "Turning big data into tiny data: Constant-size coresets for k-means, PCA and projective clustering,1434-1453," in SODA, 2013. [27] S. Ruder, "An overview of gradient descent optimization algorithms," arXiv preprint arXiv:1609.04747, 2016.
ScienceDirect ScienceDirect
Procedia Manufacturing 00 (2018) 000–000 Available online atatwww.sciencedirect.com Available online www.sciencedirect.com Procedia Manufacturing 00 (2018) 000–000
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
ScienceDirect ScienceDirect Procedia Manufacturing 26 (2018) 1411–1420 Procedia Manufacturing 00 (2017) 000–000
www.elsevier.com/locate/procedia
46th th
SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA 46 SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA
Big Big Data Data Analytics Analytics for for Processing Processing Time Time Analysis Analysis in in an an IoT-enabled IoT-enabled Shop Floor Manufacturing Engineeringmanufacturing Society International Conference 2017, MESIC 2017, 28-30 June manufacturing Shop Floor 2017, Vigo (Pontevedra), Spain Daniel D. Kho, Seungmin Lee, Ray Y. Zhong** Daniel D. Kho, Seungmin Lee, Ray Y. Zhong
Costing models for capacity optimization in Industry 4.0: Trade-off Department of Mechanical Engineering, The University of Auckland, Auckland, New Zealand Department of Mechanical Engineering, The University of Auckland, Auckland, New Zealand between used capacity and operational efficiency * Corresponding author. Tel.: +64-9-923-1584. a a,* b * Corresponding Tel.: +64-9-923-1584. E-mail address:author. [email protected] E-mail address: [email protected] a University of Minho, 4800-058 Guimarães, Portugal b Unochapecó, 89809-000 Chapecó, SC, Brazil
A. Santana , P. Afonso , A. Zanin , R. Wernkeb
Abstract Abstract Internet of things (IoT) technology has been widely used in manufacturing where great myriad of data have been generated. This Internet of things a(IoT) technology hasforbeen widely in manufacturing where great myriad datawhich have uses been radio generated. This paper introduces big data analytics Internet of used Things (IoT)-enabled manufacturing shop of floor frequency Abstract paper introduces a bigtechnology data analytics for Internet Things (IoT)-enabled manufacturing shop which uses radio frequency identification (RFID) for capturing theofreal-time production data. This involves twofloor machine learning techniques: kidentification (RFID) technology for capturing the for real-time production data. This involves machine this learning techniques: kmeans clustering and gradient descent optimization the data from the manufacturing sites. two In particular, research deals with Under the concept of "Industry 4.0", production be pushedsites. to In beparticular, increasingly interconnected, means and gradient descent processes optimization the processes databatches from thewill manufacturing this research deals with the realclustering RFID data related to various for for individual and processing time. This could produce valid predictions information onmanufacturing a real time basis and, necessarily, much more efficient. In this context, capacity optimization the real RFIDbased data related to various processes individual and processing This could produce valid predictions about expected overall time for a for given number batches of manufacturing batch time. inputs. goes traditional aim of capacity also forinputs. organization’s profitability and value. aboutbeyond expectedthe overall manufacturing time for amaximization, given number ofcontributing manufacturing batch Indeed, lean management continuous © 2018 The Authors. Publishedand by Elsevier B.V. improvement approaches suggest capacity optimization instead of © 2018 2018 The The Authors. Published by Elsevier B.V. B.V. © Authors. Published by Elsevier Peer-review under responsibility of the scientific committee NAMRI/SME. maximization. The study of capacity optimization and costing models is American an important research Research topic that deserves Peer-review under responsibility of the scientific committee ofofthe 46th SME North Manufacturing Conference. Peer-review under responsibility of the scientific committeeperspectives. of NAMRI/SME. contributions from both the practical and theoretical This paper presents and discusses a mathematical
Keywords: Data Analytics; RFID; IoT; Manufacturing; Shopcosting Floor. models (ABC and TDABC). A generic model has been model forBig capacity management based on different Keywords: Big Data Analytics; RFID; IoT; Manufacturing; Shop Floor. developed and it was used to analyze idle capacity and to design strategies towards the maximization of organization’s value. The trade-off capacity maximization vs operational efficiency highlighted andremarkable it is showntechnologies that capacity practices,is and one of the is 1. Introduction optimization might hide operational inefficiency. practices, and one of the remarkable technologies 1. Introduction radio frequency identification (RFID) [6-8]. is It © 2017 The Authors. Published by Elsevier B.V. radio frequency identification [6-8]. It enhances the configurability and (RFID) reusability during Internet of Things (IoT) technology has been Peer-review under responsibility of technology the scientific committee of the Manufacturing Engineering Society International Conference enhances the configurability and reusability during Internet of Things (IoT) has been manufacturing processes [9]. Smart objects with the widely used for enabling intelligent manufacturing so 2017. manufacturing processes Smart objects with widely used for enablingobjects intelligent manufacturing so RFID tags installed can [9]. capture fluctuations in the the that statuses of physical could be tracked and RFID tags installed can capture fluctuations in the that statuses of physical objects could be tracked and manufacturing surroundings and changes in quantity traced [1-3]. IoT is an ecosystem of technologies Keywords: Cost Models; ABC; TDABC; Capacity Management; Idle Capacity; Operational Efficiency manufacturing and changesofinthe quantity traced [1-3]. is anofecosystem of technologies [10]. The mostsurroundings beneficial usefulness IoTmonitoring theIoT statuses physical objects, capturing [10]. The most beneficial usefulness of the IoTmonitoring the statuses of physical objects, capturing enabled manufacturing is the reporting of data in realmeaningful data, and transferring the information enabled manufacturing is the reporting of data in realmeaningful data, and transferring the information time [11-14]. This empowers the users to have agilities through IP networks to software applications [4, 5]. 1. Introduction time [11-14]. This empowers users have agilities through IP networks to software applicationsindustry [4, 5]. in making accurate decisions the based onto such data. IoT technology is used in manufacturing in making accurate decisions based on such data. IoT technology is used in manufacturing industry The cost of idle capacity is a fundamental information for companies and their management of extreme importance in modern systems. In general, it isB.V. defined as unused capacity or production potential and can be measured 2351-9789 ©production 2018 The Authors. Published by Elsevier 2351-9789 2018responsibility The Authors. Published by Elsevier B.V.hours in several©under ways: tons of production, available of manufacturing, etc. The management of the idle capacity Peer-review of the scientific committee of NAMRI/SME. Peer-review underTel.: responsibility the761; scientific committee NAMRI/SME. * Paulo Afonso. +351 253 of 510 fax: +351 253 604of741 E-mail address: [email protected]
2351-9789 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the Manufacturing Engineering Society International Conference 2017. 2351-9789 © 2018 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 46th SME North American Manufacturing Research Conference. 10.1016/j.promfg.2018.07.107
1412 2
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
Big Data Analytics is the process of evaluating a large volume of various datasets to discover patterns, correlations, market trends and other useful information that can help organizations to make more informed decisions [15-17]. This involves manipulating any types of information that can be computed to represent characteristics of an object or a person, such as user ID or a person's age. The Big Data Analytics will be of paramount use in the performances of the manufacturing plants. For instance, a mix of business intelligence (BI) to analyse market, detect fraud and carry out financial risk assessment is popularly used by businesses [18]. A realization of analysis is achieved by numerous Big Data Analytics related tools, such as Oracle's Big Data appliance and Cloudera's Hadoop distribution [19]. Other typical tools include GridGain for high performance computing cluster (HPCC), Storm for dealing with huge data sets with distributed real-time computation capabilities and Talend for providing a number of BI service [20]. Processing large data and producing meaningful patterns instantly are the preeminent advantages of these tools. Amazon Elasticsearch Service provides log analytics service which interprets logs generated by many sources, such as websites, mobile devices, servers and sensors for application monitoring, fraud detection and IoT [21]. Apache Kafka is used for building real-time data pipelines streaming applications by having publish and subscribe mechanism to have streams of data like a messaging system, and stores data in a distributed replicated cluster [22]. These tools are accessible as platforms and tools and programming languages to stimulate businesses to improve their processes and efficiencies through exploring Big Data Analysis. There was a satisfactory amount of cases in manufacturing industries using Big Data Analytics for different purposes [23]. However, it was scarcely reported that the streams of real-time data are analysed through the interoperable machines in the IoT-enabled manufacturing and the application of decentralized decisions through the adoption of machine-learning techniques to implement maximum automation. Despite the data being post-processed in this research, it is highly feasible that the algorithm that is used for extracting notable interpretations could be accommodated for the real-time implementation. It was essential to capture the real-time production data due to the need for testing if the proposed algorithm would work well with the actual data. Furthermore, studies on the use of RFID data from smart manufacturing objects (SMOs) on an intelligent shop floor is limitedly reported [24]. With the primary aim
of capacitating these inadequacies, this study proposes to devise a Big Data Analytics for the IoT-enabled manufacturing. This paper is organized as follows. Section 2 gives the dataset definition. Section 3 presents the proposed Big Data Analytics approach. Section 4 shows the experimental results and some discussions. Section 5 concludes this paper by highlighting our contributions and future research directions. 2. Definition of Datasets The RFID data are from an IoT-enabled manufacturing shop floor. A piece of RFID data is recorded when there is a detection between a reader and a tag or an operation of the RFID readers, such as a clicking on the keyboard. Table 1. Definitions of RFID Datasets. Data Code ID
Definition
Product Code
BatchMainID Required group of materials to manufacture a product UserID ProcCode
Individual worker/operator at manufacturing plant Type of mechanical manufacturing process
ProcSeqnum Order in which each process is taken place Quantity GoodNum Time Location TimeFloat
Number of product produced Quantity of product that passes the acceptance threshold Period of record at each event occurrence Warehouse that archives the batches of materials Time expressed as float 32 bits in days from 01/01/2000
There are four main relationships that can be deduced from manipulating these datasets. One is tracking the specific BatchMainID over the period of time and comparing the total time taken to finish the batch. UserID and ProcCode can be used to observe the performance of the operator on the shop floor. The relationship between the locations in each BatchMainID stays over buffering time can be also established. Also, tracking each BatchMainID to see the number of operators required to complete the batch would be significant.
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
3. Big Data Analytics Approach for Production Performance Analysis The proposed Big Data Analytic approach involves data analysis, machine learning and data visualizations. TensorFlow open-source software library with the capacity of accomplishing these task operations is selected in this paper. 3.1. Containerization Containerization is a virtualization method for deploying and running distributed applications. It is enabled by providing an additional layer of abstraction, called container engine, which are run on a single control host and access a single kernel [25]. This approach not only achieves virtualization, cost reduction, scalability, and unnecessity of kernel resource duplication, but also its advantages come from compatible configurations. This means, when moved from one computing environment to another, the containers will be easily configured for new applications. This could be from a laptop to a test environment, from a staging environment into production, and from a physical machine in a data center to a virtual machine in a private or public cloud. Therefore, an application from the software will be consisted of all its dependencies, libraries and other binaries, and configuration files. By containerizing the application platform and its dependencies, differences in OS distributions and underlying infrastructure are abstract. That means end-users are empowered to adopt the software compatibly regardless of hardware discrepancies.
1413 3
The command used is docker exec -it ContainerName /bin/bash. Inside the container, a python file is created to implement the algorithm and the run command is PythonFileName.py. The container can be exited with the command exit and it can stop the running container behind by the command docker stop ContainerName. If the container needs to be started again, simple command docker restart ContainerName is used. 3.2. Visualization TensorFlow includes TensorBoard shown in Fig. 1, which is a data visualization toolkit. This will be the graphical platform for displaying nodes and different types of graphs to suit different kinds of data analysis. Fig. 1 also shows the number of graphs that were obtained by implementing some algorithms in this paper. The web-based localhost server display of the data analytic visualization promotes user-friendliness and clear UI Workflow.
Fig. 2. Architecture of the proposed approach
3.3. Architecture
Fig. 1. TensorBoard
3.3.1. Workflow The general workflow of the proposed solution is shown in Fig. 2. This architecture involves four main processes: data cleansing or pre-processing, classification, pattern recognition and visualization. It begins with a large set of RFID data from the manufacturing shop floor and is completed by producing a visualization of useful information regarding the performance of the manufacturing plant. The whole design works in a way in which the K-
1414 4
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
means clustering forms different clusters to represent closely-related data. These individual clusters provide the linear or non-linear regressions for predictions. The predictions are verified by the proposed probability algorithm that is tested by streaming a series of inputs with varying batch sizes. 3.3.2. Data Pre-processing The RFID data is provided by the commencement without the TimeFloat. It was modified to be a correct format during importing the data into the TensorFlow program. As the Time follows an arrangement in days, months and years format consecutively (DD/MM/YYYY), the program was unable to accept the values as it only allowed the numeric types, such as integer or float. Based on the date starts on 1900/01/01, the conversion into float type results in 39512.62 days, representing the first data entry which is recorded in 2008/05/03. The initial date was changed to 2000/01/01 which made the time difference to be 3012.618 days between 2000/01/01 and 2008/05/03. The floating-point numbers represent the smaller units of time, such as hours, minutes and seconds which can be calculated by multiplying by 24, (24x60) and (24x60x60) respectively. However, it must be noted that this kind of conversion ignores the effect of the Leap year. With the Leap year effect, a transition of the initial counting time from 1900/01/01 to 2000/01/01 has the time difference of about 36524 days. This is because the Leap year states that there are actually 365.2422 days in a year. For the convenience of calculations, it was assumed that the Leap year effect is negligible. The RFID dataset contains a total of 413,472 pieces. Time is an important feature for the analysis when the completion time of the BatchMainIDs are considered. Therefore, the data with no time entries were all removed for more relevance and faster processing. The total number of remaining data was amounted to 376,746 samples. Consequently, any analysis or programming using the data cannot exceed more than the data size. Although the data size is limited to 376,746 entries, these have been collected from the advanced sensors from the manufacturing shop floors which have been insufficiently reported. Different samples of data will be used to evaluate the predicted model by comparing the regression line and the centroids. It indicates that the model based on the first 10,000 data needs to be examined against the last 10,000 data, which are input for the examination. To compensate for the further randomness and validate
the model, the random sampled 10,000 data were compared in the same way. 3.4. BatchMainID vs Time Data analysis can be carried out by using the relationship between the BatchMainID and Time to produce statistically important remarks. Fig. 3 shows the plotting of all the 376,746 samples of matching BatchMainID processes and Time. A region covered by all the samples so that the hidden patterns or trends are difficult to identify. Additionally, the computation cost of the whole data is immense. From this figure, the datasets are challengeable to be clustered.
Fig. 3. All of BatchMainID VS Time in days
3.5. Batch Identification Algorithm The algorithm tracks the individual BatchMainIDs to obtain the start and finish Time and counts a number of processes to complete a batch. BatchMainID and TimeFloat are used for this purpose. Fig. 4 illustrates a result for a specific BatchMainID. The 5 scatter points means this BatchMainID had been conducted in 5 manufacturing processes. A time difference (diffTime) and a number of processes to complete a specific BatchMainID (noofProc) have been generated for further algorithms. The diffTime has been calculated by subtracting the last TimeFloat from start TimeFloat for a BatchMainID. The noofProc has been calculated by counting the number of manufacturing processes. Therefore, how much time was used and how many processes were used to complete a batch could be identified.
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018) 𝑘𝑘
𝑗𝑗
𝐽𝐽 = ∑ ∑‖𝑥𝑥𝑖𝑖 − 𝑐𝑐𝑗𝑗 ‖ 𝑗𝑗=1 𝑖𝑖=1
where 𝑗𝑗
𝑛𝑛
1415 5 2
2
‖𝑥𝑥𝑖𝑖 − 𝑐𝑐𝑗𝑗 ‖ is the squared distance between the n data 𝑗𝑗 points 𝑥𝑥𝑖𝑖 and the k centroids 𝑐𝑐𝑗𝑗 .
Fig. 4. Identification of a Single Batch
3.6. Machine Learning Algorithms A machine learning method used in this paper is an integration of supervised and unsupervised learning processes. The supervised learning has the labelled data which means the independent variable forms a relationship with the dependent variable which is already defined by humans, whilst the unsupervised approach has no labelled data. In this research, the kmeans clustering which is an unsupervised learning method is used first to group the data into a number of clusters. The gradient descent optimization algorithm which is a supervised learning method is then used to calculate the predictions and reduce the loss of the cost or the object function.
In this paper, 5 clusters were used for an adequate classification. The colors used were red, blue, green, black and purple to represent cluster 0, 1, 2, 3 and 4 respectively. The individual clusters are separately graphed, and each of them undergoes gradient descent optimizations. The overall graph is shown in Fig. 5 where the centroids are depicted as cross signs.
Fig. 5. Five clusters in 10000 samples
3.6.1. K-means Clustering
3.6.2. Gradient Descent Optimization
The K-means algorithm is one of the well-known unsupervised learning algorithm that solves the clustering problem [26]. To use this algorithm, the number of k centroids needs to be defined manually which corresponds to the number of clusters the user aims to have. The centroids need to be placed relatively separated from each other to achieve a clear distinction between each cluster. The next step is to associate each data point to its nearest centroid. When all the points are assigned, k new centroids need to be re-positioned to update from the previous step. The allocated data points among the clusters go to the loop again. It can be noticed that k centroids move for every step until no more changes are required. Mathematically, the K-means algorithm minimizes the squared error function J.
The Gradient Descent Optimization algorithm seeks to search for the global minimum of a function [27]. It takes the derivative of a function at a point that gives the direction to move towards. It makes steps down the function J in the direction in the negative gradient. The size of each step is determined by the learning rate α. 𝜕𝜕 𝐽𝐽(𝜃𝜃0 , 𝜃𝜃1 ) 𝜃𝜃𝑗𝑗 ∶= 𝜃𝜃𝑗𝑗 − α 𝜕𝜕𝜃𝜃𝑗𝑗 where 𝑗𝑗 = 0,1 represents the index number. θ is the parameter in the function. By changing 𝜃𝜃𝑗𝑗 for each iteration, the average squared distance between the object function and the proposed regression is minimized. The results could differ slightly due to the random weights and constant values generated during the
1416 6
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
experiments. After testing, it was resolved that the weights can change between 0 and 50, constants can change between -10 and 10, the learning rate is 0.015 and the iterations are 100 times. The weights located in the positive plane should be selected for the regression. The range between the negative and positive values used for the constants is to account for the position of the regression. As shown in Fig. 6, inverse functions were fitted to each cluster except the cluster 0 and 1 as they used logarithmic functions and undefined function respectively.
Fig. 6. Cluster 0-4
3.7. Prediction Algorithm The prediction algorithm is implemented by feeding the numbers range from 1 to the maximum number of BatchMainID processes. These need to be committed into different clusters. This was attempted by collecting the number of colored cluster points at which the numbered process is fed. The algorithm calculates the probability of having the most number of colored points over the total number of colored points. In Fig. 5, there are 7 purple points and 17 green points at 𝑥𝑥 = 12. If the new BatchMainID takes 12 processes to complete, the probability of the time that will take to finish the processes is 17/24 = 70.83% and the process belongs to a green cluster since it is the dominant color at 𝑥𝑥 = 12.
4. Results and Discussion 4.1. Predictions
The results from the prediction algorithm are shown in Table 2. The input is the array of one dimensional tensor of numbers which depict the
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
number of processes. The output shows the assignment of those processes to the cluster numbers respectively. The prediction percentage shows how accurate each assignment of batches to a certain cluster is decided. It is observed that this is a more reliable approach than calculating the general probability for the predictions. The general probability only considers the likelihood of retrieving one scatter point out of all. Table 2. Prediction Results.
Prediction Input, (Number of Processes) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Prediction Cluster 3 3 3 3 3 4 2 2 2 2 2 2 2 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0
Probability or Prediction Percentage (%), 100.00 100.00 82.47 92.31 44.62 84.23 80.95 87.50 73.47 91.30 70.37 70.83 100.00 53.33 77.78 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00
Prediction Output, (Time Difference in Days) 2.00 1.17 0.90 0.76 0.68 2.49 8.93 8.17 7.57 7.10 6.71 6.39 6.11 5.88 5.67 8.42 9.22 9.98 10.69 11.37 12.02 12.64 13.23 13.79 14.33 14.85 15.35 15.83
This approach depends on the total number of batch processes. However, the use of the proposed prediction algorithm enables the probability to reflect on the dominant number of scatter points that belong to specific clusters whilst also considering the total number of scatter points. It increases likelihood of the future BatchMainID placed into the most relevant cluster. In the Fig. 7, the computation cost is examined. It is observed that the computation cost is increasing as the number of samples increases. This is very reasonable due to the time taken to operate our algorithm for the data pre-processing, visualizing and optimizing. Due to the uncertainty of the error whilst
1417 7
processing the big data, the computation cost has been calculated up to 10,000 samples. The experiment took place as the 5 operations were run and they were averaged out per number of samples. This was because the computation cost is different for each run even if the number of samples is the same due to the random number generations within the algorithm, such as weights, constants and Prediction Input values. It is observed from Fig. 7, the computation cost has a linear relationship with the number of samples.
Fig. 7. Computation Cost
The proposed approach is tested with the Intel(R) Core(TM) i7-6500U CPU at 2.50GHz. The computation cost could be generally improved if the program was experimented with the NVIDIA GPU support, because TensorFlow programs are significantly faster on a GPU than on a CPU.
Fig. 8. Gradient Descent Optimizations (a) Cluster 0; (b) Cluster 2; (c) Cluster 3
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
1418 8
4.2. Parameters The parameters of our interest are weights (w) and constants (b). Fig. 8 is the illustration of the gradient descent optimizations that were conducted by 100 times for each cluster. This resulted in producing a non-linear regression lines in each cluster. The results are shown in multiples of 10 for a clear visualization. w and b represent weights and constant values respectively. They are the factors that determine the position and the direction of the regression lines. The weights and constant values change as the iterations are completed. These two variables change in a way that the loss is effectively reduced to zero. Fig. 8 clearly displays the loss is being reduced significantly over the iterations. In Fig. 8 (a), the Cluster 0 experienced the loss of 1311.06 initially and it has been reduced to 12.6655, which is the 99% improvement after 100 iterations. Even the least improvement in the Cluster 3 from Fig. 8 (c) experienced 86% improvement in minimizing the loss. These results reinforce the reliability and the strong diminishing loss effect of our model. Therefore, equations can be formulated for each cluster by computing the 100th weights and constants. For instances, for Cluster 0, the model equation will be: 𝑓𝑓(𝑥𝑥) = 13.25 × ln(𝑥𝑥) − 28.32 and the equation for the Cluster 2 will be 42.71 + 2.83. 𝑓𝑓(𝑥𝑥) = 𝑥𝑥
The model parameters, such as ln(𝑥𝑥) and inverse function have been selected after various trials of other mathematical models, such as linear and quadratics. This approach had to be intuitive as the focus is minimizing the loss of the chosen objective functions to represent the predictions of the data. This is validated by the number of trials when the loss appears to be improving no more. Therefore, the training performance results could be identified how much the loss has been reduced. Further adjustment could be made to decrease the loss by changing the range of weights and constants, so that the gradient descent algorithm can explore more opportunities to reach its most accurate global minima. The learning rate can also be adjusted to influence on the loss. The current learning rate is 0.015. However, the adjustment should be considered by the computation time and the direction of the gradient. Lower learning rate will induce immense computation time due to the limited computing resources. In contrary, higher learning rate
will drive the direction of the gradient opposite to the global minima. Furthermore, the number of optimizations completed also have a noticeable impact on the level of loss. In this experiment, if less iterations were conducted instead of 100 iterations, the loss would have been inaccurate as it can be seen in the Fig. 8. This means raising the number of optimizations would improve the quality of the optimization results. However, this can be limited by the nature of our mathematical model as the loss was not lessened much in the Fig. 8 (a), Cluster 0. Normally, this type of steady loss should be left as it is, because the change in any variables may be able to reduce it further, but this may have other effects on the other clusters.
Fig. 9. Evaluation with last 10,000 samples (a) Cluster 2; (b) Cluster 3; (c) Cluster 4
4.3. Evaluation Evaluation is required to validate the models. In this research, different samples are applied to the model for evaluation purposes. Not all data has been used for the analysis algorithms due to the large computational cost. Therefore, the model generated by first 10,000 samples are compared with lastly selected 10,000 samples. This is due to by experience that analyzing the data size over 10,000 creates a high computation cost when the learning takes place using all the proposed algorithms. Also, the more the data is plotted, the less the quality of the visibility becomes. It is compromised to depict the whole data set within the data size of 10,000 limit based on the tests of number of program runs with varying data sizes. The clustering result depends on the input size. This is validated by the shapes of the graphs of the last 10,000 samples. There is a significant similarity in the form and the contour between the first 10,000 samples and
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
the last 10,000 samples. Figure 9 (a) shows a classified scatter plot with last 10,000 samples. It describes that no dataset is assigned to red and blue clusters due to a shorter diffTime and noofProc range of the last 10,000 samples. As the raw dataset has the chronological trends, the last part of the data usually contains latter manufacturing RFID data. It is assumed that the manufacturing company retrieves the experiences of its products and increases the efficiency of manufacturing processes and time. However, it used the same K-means clustering centroids as the first 10,000 samples. Therefore, it is acceptable to assume that the similar batch products are classified as the same clusters. In addition, a visual analysis of Fig. 9 (b), (c), and (d) shows that the non-linear regression is also valid for these samples. 5. Conclusion This paper introduces a Big Data Analytics approach for examining the massive RFID data from an IoT-enabled manufacturing shop floor. The approach includes data processing and visualization. Evaluation studies are conducted for validating the approach. Some contributions are significant from this paper. The prediction method returned the time output with above 50% of validation. The big data acquired from the manufacturing site can be analyzed to make predictions and to improve efficiency. A combination of gradient descent and clustering methods is proposed. The losses were minimized at a reasonable level when the algorithm is tested with a wide range of samples. The computation cost will likely to increase in a non-linear manner as the data size increases. Future research directions will be carried out as follows. First of all, the visualization of the analysis could be implemented in numerous platforms, such as mobiles and tablets for the ease of access. Different parameters could be compared to produce further meaningful findings to the manufacturers, for example comparing the UserID against the ProcCode, specific BatchMainIDs in different locations against time and different BatchMainIDs against the number of UserIDs. Secondly, some questions such as many processes are handled by a single employee, how many employees are required to complete the batch are required further investigation. Finally, the software
1419 9
design principles could be taken into account to improve the readability, scalability, effectiveness and efficiency of the proposed approach. Acknowledgements Authors would like to give thanks to Huaiji Dengyun Auto-Parts for providing the data necessary for this analysis. References [1] R. Y. Zhong, X. Xu, and L. H. Wang, "IoT-enabled Smart Factory Visibility and Traceability using Laser-scanners," Procedia Manufacturing, 2017. [2] R. Y. Zhong, L. H. Wang, and X. Xu, "An IoT-enabled Realtime Machine Status Monitoring Approach for Cloud Manufacturing," Procedia CIRP, vol. 63, pp. 709-714, 2017. [3] X. Qiu, H. Luo, G. Y. Xu, R. Y. Zhong, and G. Q. Huang, "Physical assets and service sharing for IoT-enabled Supply Hub in Industrial Park (SHIP)," International Journal of Production Economics, vol. 159, pp. 4-15, 2015. [4] R. Y. Zhong, G. Q. Huang, S. L. Lan, and M. L. Wang, "IoTenabled Building Information Modelling Cloud for Prefabrication Construction," The International Conference on Flexible Automation and Intelligent Manufacturing (FAIM), 23-26 June, Wolverhampton, UK, 2015. [5] F. Xia, L. T. Yang, L. Wang, and A. Vinel, "Internet of things," International Journal of Communication Systems, vol. 25, p. 1101, 2012. [6] R. Y. Zhong, Y. Peng, F. Xue, J. Fang, W. Zou, H. Luo, et al., "Prefabricated construction enabled by the Internet-of-Things," Automation in Construction, vol. 76, pp. 59-70, 2017. [7] S. Mejjaouli and R. F. Babiceanu, "RFID-wireless sensor networks integration: Decision models and optimization of logistics systems operations," Journal of Manufacturing Systems, vol. 35, pp. 234-245, 2015. [8] C. Lee, G. Ho, K. Choy, and G. Pang, "A RFID-based recursive process mining system for quality assurance in the garment industry," International Journal of Production Research, vol. 52, pp. 4216-4238, 2014. [9] J. H. Lee, J. H. Song, K. S. Oh, and N. Gu, "Information lifecycle management with RFID for material control on construction sites," Advanced Engineering Informatics, vol. 27, pp. 108-119, 2013. [10] R. Y. Zhong, Q. Dai, T. Qu, G. Hu, and G. Q. Huang, "RFIDenabled real-time manufacturing execution system for masscustomization production," Robotics and Computer-Integrated Manufacturing, vol. 29, pp. 283-292, 2013. [11] M. L. Wang, T. Qu, R. Y. Zhong, Q. Y. Dai, X. W. Zhang, and J. B. He, "A radio frequency identification-enabled real-time manufacturing execution system for one-of-a-kind production manufacturing: a case study in mould industry," International Journal of Computer Integrated Manufacturing vol. 25, pp. 2034, 2012. [12] Q. Y. Dai, R. Y. Zhong, G. Q. Huang, T. Qu, T. Zhang, and T. Y. Luo, "Radio frequency identification-enabled real-time manufacturing execution system: a case study in an automotive part manufacturer," International Journal of Computer Integrated Manufacturing vol. 25, pp. 51-65, 2012. [13] Y. F. Zhang, P. Jiang, and G. Huang, "RFID-based smart kanbans for just-in-time manufacturing," International Journal of Materials and Product Technology, vol. 33, pp. 170-184, 2008.
1420 10
Daniel D. Kho et al. / Procedia Manufacturing 26 (2018) 1411–1420 D. Kho et al./ Procedia Manufacturing (2018)
[14] Y. F. Zhang, T. Qu, O. K. Ho, and G. Q. Huang, "Agent-based smart gateway for RFID-enabled real-time wireless manufacturing," International Journal of Production Research, vol. 49, pp. 1337-1352, 2011. [15] R. Y. Zhong, C. Xu, C. Chen, and G. Q. Huang, "Big Data Analytics for Physical Internet-based intelligent manufacturing shop floors," International Journal of Production Research, vol. 55, pp. 2610-2621, 2017. [16] A. Zaslavsky, C. Perera, and D. Georgakopoulos, "Sensing as a service and big data," Proceedings of the International Conference on Advances in Cloud Computing (ACC), vol. 2, pp. 1-8, 2013. [17] R. M. Ward, R. Schmieder, G. Highnam, and D. Mittelman, "Big data challenges and opportunities in high-throughput sequencing," Systems Biomedicine, vol. 1, pp. 0--1, 2013. [18] G. Schuh, C. Reuter, J.-P. Prote, F. Brambring, and J. Ays, "Increasing data integrity for improving decision making in production planning and control," CIRP Annals-Manufacturing Technology, vol. 66, pp. 425-428, 2017. [19] R. Y. Zhong, S. T. Newman, G. Q. Huang, and S. Lan, "Big Data for supply chain management in the service and manufacturing sectors: Challenges, opportunities, and future perspectives," Computers & Industrial Engineering,vol. 101, pp. 572-591, 2016. [20] D. Loshin, Big data analytics: from strategic planning to enterprise integration with tools, techniques, NoSQL, and graph: Elsevier, 2013. [21] K. H. Tan, Y. Zhan, G. Ji, F. Ye, and C. Chang, "Harvesting big data to enhance supply chain innovation capabilities: An analytic infrastructure based on deduction graph," International Journal of Production Economics, vol. 165, pp. 223-233, 2015. [22] R. Ranjan, "Streaming big data processing in datacenter clouds," IEEE Cloud Computing, vol. 1, pp. 78-83, 2014. [23] H. Chen, R. H. Chiang, and V. C. Storey, "Business intelligence and analytics: From big data to big impact," MIS quarterly, vol. 36, pp. 1165-1188, 2012. [24] R. Y. Zhong, G. Q. Huang, S. L. Lan, Q. Y. Dai, C. Xu, and T. Zhang, "A Big Data Approach for Logistics Trajectory Discovery from RFID-enabled Production Data," International Journal of Production Economics, vol. 165, pp. 260-272, 2015. [25] M. J. Scheepers, "Virtualization and containerization of application infrastructure: A comparison," Proceeding in 21st Twente Student Conference on IT, pp. 1-7, 2014. [26] D. Feldman, M. Schmidt, and C. Sohler, "Turning big data into tiny data: Constant-size coresets for k-means, PCA and projective clustering,1434-1453," in SODA, 2013. [27] S. Ruder, "An overview of gradient descent optimization algorithms," arXiv preprint arXiv:1609.04747, 2016.