- Email: [email protected]
A Digital Twin for Integrated Inspection System in Digital Manufacturing
13th IFAC Workshop on Intelligent Manufacturing Systems 13th IFAC Workshop on Intelligent Manufacturing Systems August 12-14, 2019. Oshawa, Canada 13th IFAC Workshop on Intelligent Manufacturing Systems Available online at www.sciencedirect.com August 12-14, 2019. Oshawa, Canada 13th IFAC Workshop on Intelligent Manufacturing Systems August 12-14, 2019. Oshawa, Canada August 12-14, 2019. Oshawa, Canada
ScienceDirect
IFAC PapersOnLine 52-10 (2019) 182–187
A Digital Twin for Integrated Inspection System in Digital Manufacturing A Digital Twin for Integrated Inspection System in Digital Manufacturing A Digital Twin for Integrated Inspection System in Digital Manufacturing Hossein Gohari* Cody Berry*in Digital Manufacturing A Digital Twin for Integrated Inspection System Hossein Gohari* Cody Berry*
HosseinAhmad Gohari* Cody Berry* Barari* HosseinAhmad Gohari* Cody Berry* Barari* HosseinAhmad Gohari* Cody Berry* Barari* * Faculty of Engineering and Applied Science, Ahmad Barari* of Engineering and Applied Science, * Faculty University of Ontario Institute of Technology (UOIT), * Faculty of Engineering and Applied Science, University of Ontario Institute of Technology (UOIT), Oshawa, Ontario, Canada * Faculty of Engineering and Applied Science, University of Ontario Institute ofCanada Technology (UOIT), Oshawa, Ontario, (e-mails: [email protected] , [email protected] , [email protected]) University of Ontario ofCanada Technology (UOIT), Oshawa,Institute Ontario, (e-mails: [email protected] , [email protected] , [email protected]) Oshawa, Ontario, Canada (e-mails: [email protected] , [email protected] , [email protected]) (e-mails: [email protected] , [email protected] , [email protected]) Abstract: Coordinate metrology is a crucial part in advanced manufacturing industries to achieve and and Abstract: Coordinate metrology is a crucial part in advanced manufacturing industries to achieve maintain conformance of high-quality products within design specifications. Meanwhile, softwareAbstract: Coordinate metrology is a crucial part in advanced manufacturing industries to achieve and maintain conformance of high-quality products within design specifications. Meanwhile, softwarecomponents are increasingly becoming anproducts essential part ofdesign the manufacturing inspection process becauseto ofachieve increasing Abstract: Coordinate metrology is a crucial part in advanced industries and maintain conformance of high-quality within specifications. Meanwhile, softwarecomponents are increasingly becoming an essential part of the inspection process because of increasing part complexities in design and the high-volume of dataofdesign captured from different sensors of in increasing hardwaremaintain conformance of high-quality products within specifications. Meanwhile, softwarecomponents are increasingly becoming an essential part the inspection process because part complexities in design and athe high-volume dataparallel captured from differentinspection sensors insystem hardwarecomponents. Thisincreasingly paper presents virtual toof work to an integrated (IIS) components are becoming anreplica essential part the inspection process because increasing part complexities in design and the high-volume ofwork dataof captured from different sensors of insystem hardwarecomponents. This paper presents a virtual replica to parallel to an integrated inspection (IIS) for inspection of freeform and complex surfaces based on a metric of their geometric complexity. In this part complexities in design and the high-volume of data captured from different sensors in hardwarecomponents. This paper presents a virtualsurfaces replicabased to work parallel antheir integrated inspection systemIn(IIS) for inspection of freeform and complex onconducted a metricto of geometric complexity. this approach, an intelligently guided sampling is virtually from a large dataset, instead of the components. This paper presents a virtual replica to work parallel to an integrated inspection system (IIS) for inspection of freeform and complex surfaces based onconducted a metric offrom theira geometric complexity. In this approach, an intelligently guided sampling is virtually large dataset, instead of the physical sampling process when the sample points are traditionally selected randomly from the measured for inspection of freeform and complex surfaces based on a metric of their geometric complexity. In this approach, an intelligently guidedthesampling is virtually conducted selected from a randomly large dataset, instead of the physical sampling process when sample points are traditionally from the measured surface. Implementation a closed-loop between the traditionally main tasks inselected IIS isa considered in developing approach, an intelligently guided is virtually conducted from large dataset, instead of this the physical sampling processof when thesampling sample points are randomly from the measured surface. Implementation ofuncertainties a closed-loop between the traditionally main tasks inselected IIS is considered in is developing this digital twin to reduce the associated with the inspection process. A method introduced to physical sampling process when the sample points are randomly from the measured surface. Implementation a closed-loop betweenwith the main tasks in IIS is considered in is developing this digital twin tolocal reduce theofuncertainties associated the inspection process. A method introduced to estimate the densities of the measured points required for virtual sampling from each patch on the surface. Implementation of a closed-loop between the main tasks in IIS is considered in developing this digital twin tolocal reduce the uncertainties associated with the inspection process. A method is introduced to estimate the densities of the measured points required for virtual sampling from each patch on the work-pieces’s surface based ontheitsmeasured geometric complexity. Two case studies are conducted to verify digital twin reduce the uncertainties associated with the inspection process. A method is introduced to estimate the tolocal densities ofon points required for virtual sampling from each patch on the the work-pieces’s surface based its geometric complexity. Two case studies are conducted to verify the effectiveness ofsurface thedensities methodology. observed efficiency selection of the are important measured data in estimate the local ofontheitsThe measured points requiredin for virtual sampling from each patch on the the work-pieces’s based geometric complexity. Two case studies conducted to verify effectiveness ofsurface the methodology. efficiencystrategy in selection ofimplemented the are important datathe in the proposed sampling strategy itobserved a better sampling to be in measured a digital twin work-pieces’s based onmakes itsThe geometric complexity. Two case studies conducted to verify effectiveness of the methodology. The observed efficiencystrategy in selection ofimplemented the important measured datafor in the proposed sampling strategy makes it a better sampling to be in a digital twin for IISs. effectiveness of the methodology. The itobserved efficiencystrategy in selection ofimplemented the important measured datafor in the proposed sampling strategy makes a better sampling to be in a digital twin IISs. the proposed sampling strategy makes it a better sampling strategy to be implemented in a digital twin for IISs. © 2019, IFAC (International of Automatic Control) Hosting by Elsevier All rights reserved. Keywords: Digital Twin, Federation Virtual Sampling, Coordinate Metrology, Point Ltd. Measurement Planning, IISs. Keywords: Digital Twin, VirtualDeviation Sampling, Coordinate Metrology, Point Measurement Planning, Substitute Geometry Evaluation, Zone Estimation, Integrated inspection system , geometric Keywords: Digital Twin, VirtualDeviation Sampling, Coordinate Metrology, Point Measurement Planning, Substitute Geometry Evaluation, Zone Estimation, Integrated inspection system , geometric complexity, Keywords: Digital Manufacturing. Twin, VirtualDeviation Sampling, Coordinate Metrology, Measurement Planning, Substitute Evaluation, Zone Estimation, Integrated Point inspection system , geometric complexity,Geometry Digital Manufacturing. Substitute Geometry Evaluation, Deviation Zone Estimation, Integrated inspection system , geometric complexity, Digital Manufacturing. complexity, Digital Manufacturing. Barari, 2019). Research studies in coordinate metrology show 1. INTRODUCTION Barari, Research in coordinate metrology show that the2019). uncertainties in studies IIS operations are highly affected by 1. INTRODUCTION Barari, 2019). Research studies in coordinate metrology show that the uncertainties in IIS operations are highly affected by 1. INTRODUCTION the number of measured points (Feng, Saal, Salsbury, Ness, & Today’s manufacturing systems have the flexibility to Barari, 2019). Research studies in coordinate metrology show that the uncertainties in IIS operations are highly affected by 1. INTRODUCTION Today’s manufacturing systems have the flexibility to the number of measured points (Feng, Saal, Salsbury, Ness, & Lin, 2007; Hammad Mian & Al-Ahmari, 2014; Vrba, fabricate a large variety of parts with high complexity, low that the uncertainties in IIS operations are highly affected by number ofHammad measured Mian points (Feng, Saal, Salsbury, Ness, & Today’s amanufacturing systems have thecomplexity, flexibility low to the fabricate large variety of parts with high Lin, 2007; & Al-Ahmari, 2014; Vrba, Palencar, …, & (Feng, n.d.).Salsbury, Recently, laser volume, and restricted tolerances. In order low number ofHammad measured Mian points Saal, & Today’s have flexibility to the Lin, 2007;Hadzistevic, &2015, Al-Ahmari, 2014;Ness, Vrba, fabricate amanufacturing largewith variety ofsystems parts with highthecomplexity, Palencar, Hadzistevic, …, & 2015, n.d.). Recently, laser volume, and with restricted tolerances. In order to scanners are widely being used in IIS because of their accommodate these requirements, intelligent process control is Lin, 2007; Hammad Mian & Al-Ahmari, 2014; Vrba, fabricate a large variety of parts with high complexity, low Hadzistevic, …, &used 2015, n.d.). Recently, volume, and these withrequirements, restricted intelligent tolerances.process In order to scanners are widely being in inspection IIS because of laser their accommodate control is Palencar, and widely time efficiency in2015, thein tasks. needed notandonly torequirements, assist in providing theprocess flexibility and Palencar, are Hadzistevic, …, &used n.d.). Recently, laser volume, with restricted tolerances. In order to scanners being IIS because of These their accommodate these intelligent control is versatility needed not only to assist in providing the flexibility and versatility and time efficiency in the inspection tasks. These provide a large numberused of pointstasks. thatofshould accuracy, alsotorequirements, toassist be implemented inthe real-time and and inscanners are being IIS because their accommodate these intelligent process control is sensors versatility and widely time efficiency in measured thein inspection These needed notbut only in providing flexibility sensors provide a large number of measured points that should accuracy, but also to be implemented in real-time and inbe analysed to make a proper decision about the product. process controlling procedures. In common practice, versatility and time efficiency in the inspection tasks. These needed notbutonly totoassist in providinginthe flexibility and sensors provide a large number of measured points that should accuracy, also be implemented real-time and inanalysed to make a proper decision aboutpoints the product. process controlling procedures. practice, manufactured at theIn ofreal-time a manufacturing sensors provide a collection large number of measured thatprocess should accuracy, but parts also are to inspected be implemented incommon and in- be be analysed to make a proper decision about the product. process controlling procedures. Inend common practice, After the data process, an automated manufactured parts are inspected at the end of a manufacturing process. The most practical and adaptable metrology approach be analysed to make a proper decision about the product. process controlling procedures. In common practice, After the data collection process, an automated process manufactured partspractical are inspected at the endmetrology of a manufacturing employs datacollection to find theprocess, best substitute geometryprocess which process. The most and adaptable approach thethe data an automated to the digital manufacturing environment inspect the After manufactured parts are inspected at the endmetrology of atomanufacturing employs the data to find the best substitute geometry which process. The most practical and adaptable approach results in considerable reduction in the inspection time as the thethedata an automated process to the digital manufacturing environment to inspect the After employs datacollection to find theprocess, bestinsubstitute geometry which geometric and dimensional attributes is coordinate metrology. process. The most practical and adaptable metrology approach results in considerable reduction the inspection time as the to the digital manufacturing environment to inspect the part no longer needs to be manually oriented in a specific employs the data to find the best substitute geometry which geometric and Metrology dimensional attributes is coordinate metrology. results in considerable reduction in the inspection time asway the A Coordinate Machine (CMM) typically utilizes a to the digital manufacturing environment to inspect the part no longer needs to be manually oriented in a specific geometric and Metrology dimensional attributes is coordinate metrology. to obtain optimal scanning results (Zhao, Xu, industry, & in considerable reduction in the inspection time asway the A Coordinate Machine (CMM) typically utilizestoa results part no longer needs to be manually oriented in a specific way combination of tactile or contact probes and optical sensors geometric and Metrology dimensional attributes is coordinate metrology. obtain optimal scanning results (Zhao, Xu, industry, & A Coordinate Machine (CMM) typically utilizestoa to 2009, n.d.). The final information that needs to be determined part no longer needs to be manually oriented in a specific way combination of tactile or contact probes and optical sensors obtain optimal scanning results (Zhao, Xu,beindustry, & capture 3D discrete points from the work-piece surfaces. Using A Coordinate Metrology Machine (CMM) typically utilizes a to 2009, n.d.). The final information that needs to determined combination of tactile or contact probes and optical sensors to from the measured points areresults the that geometric and dimensional to obtain optimal scanning (Zhao, Xu, industry, & capture 3D discrete points from the work-piece surfaces. Using 2009, n.d.). The final information needs to be determined the tactile sensors is typically slow and more expensive combination of tactile or contact probes optical sensors to from the measured points are the geometric and dimensional capture 3D discrete points from the work-piece surfaces. Using deviations from an ideal model. The algorithms that are needed 2009, n.d.). The final information that needs to be determined the tactile sensors is typically slow and more expensive the measured points are the and comparing to usingpoints the optical sensors. Themore optical sensors capture 3D discrete from the work-piece surfaces. Using from deviations from an ideal model. Thegeometric algorithms thatdimensional are needed the tactile sensors is typically slow and expensive findthe themeasured number and the distribution of the measured points, from points are the and comparing to using the optical sensors. The optical sensors to deviations from an ideal model. Thegeometric algorithms thatdimensional are needed can capture hundreds of thousands of data points in a fraction the tactile sensors is typically slow and more expensive to find the number and the distribution of the measured points, comparing to using the optical sensors. The optical sensors evaluation of the best substitute geometry, and also the deviations from an ideal model. The algorithms that are needed can capture hundreds ofthey thousands of dataThe points in a fraction to find the number and the distribution of the measured points, of a second. However, are typically more vulnerable to comparing to using the optical sensors. optical sensors evaluation of the best substitute geometry, and also the can capture hundreds of thousands of data points in a fraction calculation of deviation zones are the major cyber components to find the number and the distribution of the measured points, of a capture second. However, they are typically more vulnerable to evaluation of the best substitute geometry, and also the noise sources and their data collection accuracy and precision can hundreds of thousands of data points in a fraction calculation of deviation zones are the major cyber components of a second. However, they collection are typically more and vulnerable to evaluation in an Integrated Inspection System and of deviation the best substitute geometry, alsotheir the noise sources and their data accuracy precision of zones are the major (IIS), cyberand components are than tactile probes (Lalehpour, Berry, & Barari, 2017). of aless second. However, they are typically more vulnerable to calculation in an Integrated Inspection System (IIS), and bytheir noise sources and their data collection accuracy and precision corresponding computational cost is highly dominated the calculation of deviation zones are the major cyber components are less than tactile probes (Lalehpour, Berry, & Barari, 2017). an Integrated Inspection System (IIS), and bytheir Because of the high tolerances demanded by & industry and by in noise sources and their data collection accuracy and precision computational costthe is highly dominated the are less than tactile probes (Lalehpour, Berry, Barari, 2017). number the sample points and quality(IIS), of sampling. The in an of Integrated Inspection System and bytheir Because of the high tolerances demanded by industry and by corresponding corresponding computational cost is highly dominated the consumers, it is vital that parts are inspected accurately and are less than tactile probes (Lalehpour, Berry, & Barari, and 2017). number of the sample points and the quality of sampling. The Because of the high tolerances demanded by industry by IIS cyberofcomponents have been traditionally conducted inthea corresponding computational costthe is highly dominated byThe consumers, it is vital that parts are inspected accurately and number the sample points and quality of sampling. efficiently. However, the increasing geometric complexity in Because of the by industry andand by IIS cyberofcomponents have been traditionally in2)a consumers, it ishigh vitaltolerances thatincreasing partsdemanded are inspected accurately sequence of: 1) Point Measurement Planning (PMP),The the sample points and the quality of conducted sampling. efficiently. the geometric complexity in number IIS cyber components have been traditionally conducted in2)a the productHowever, uncertainties in and the consumers, it results is vital in thatcomputational parts are inspected accurately sequence of: 1) Point Measurement Planning (PMP), efficiently. However, the increasing geometric complexity in Substitute Geometry Estimation (SGE), and 3) Deviation Zone IIS cyber components have been traditionally conducted in2)a the productprocess results(Ahmad in computational uncertainties in the sequence of: 1) Point Measurement Planning (PMP), inspection Barari, 2013; Mahboubkhah & efficiently. However, the increasing geometric complexity in Substitute Geometry Estimation (SGE), and 3) Deviation Zone the productprocess results(Ahmad in computational uncertainties in the sequence Geometry of: 1) Point Measurement Planning (PMP), 2) inspection Barari, 2013; Mahboubkhah & Substitute Estimation (SGE), and 3) Deviation Zone the productprocess results(Ahmad in computational uncertainties in the inspection Barari, 2013; Mahboubkhah & Substitute Geometry Estimation (SGE), and 3) Deviation Zone 2405-8963 IFAC (International Federation of Automatic Control) Copyright@ 2019 IFAC inspection© 2019, process (Ahmad Barari, 2013; Mahboubkhah & 182Hosting by Elsevier Ltd. All rights reserved. Copyright@ 2019 IFAC 182Control. Peer review under responsibility of International Federation of Automatic Copyright@ 2019 IFAC 182 10.1016/j.ifacol.2019.10.020 Copyright@ 2019 IFAC 182
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Evaluation (DZE) (Lalehpour et al., 2017). PMP presents a sampling strategy and a detailed plan to capture data points from a surface. The location and number of the measured points decided by PMP have direct effect on the time and accuracy of an inspection procedure. Evaluation of the best geometry that minimizes the deviations between the measured points and a nominal geometry, typically presented by a Computer Aided Design (CAD) model is SGE’s task. SGE is a delegate computational process which its stability and convergence are highly affected by the density of the sample points. Using the evaluated substitute geometry, the deviation zone of the actual measured surface is estimated using a DZE algorithm. DZE may construct a detailed deviation zone measured surface by simulating a skin model for the measured surface (Lalehpour & Barari, 2017; Jamiolahmadi & Barari, 2014;).
183
dataset not necessary helps the SGE and DZE tasks to obtain the best results. In many cases, particularly for the surfaces with high levels of geometric complexities, increasing the size of datasets makes the SGE’s optimization problem highly nonlinear and the simulation in the DZE process very expensive to solve. Therefore, it is very desired to select only an optimum number of the most representative sample points, referred as high quality sample points, by PMP for SGE and DZE process. This paper proposes developing a digital twin of IIS, when the high quality sample points are virtually selected from a large set of point cloud according to the feedbacks that are received by the closed-loops of PMP-SGE and PMP-DZE tasks. A methodology is developed to estimate the density of the required measurement points based on the geometric complexities on the measured surface. The reduction in number of sample points reduces the inspection time and uncertainties in evaluation of the substitute geometry and estimation of the deviation zones.
The isolated and sequential completion of PMP, SGE, and DZE results in significant uncertainties. Each of these three steps is a software-component in a cyber-physical system that is used to inspect manufactured parts. However, they can be integrated to form an IIS. The IIS is an attempt to reduce the inspection uncertainties by implementing closed-loops between these tasks. The closed-loop approach results in sharing information between these tasks. Examples of algorithms that can be used in for the tasks integrations in inspection and manufacturing can be found in (A Barari, ElMaraghy, & Orban, 2009; Berry & Barari, 2018). It results in detecting the errors and compensating them in the intermediate results to reduce the uncertainties in inspection system. As reported in (Ahmad Barari & Pop-Iliev, 2009), the quality of final products can be increased by implementing individual closed-loops for different steps of engineering activities.
2. STRUCTURE OF INTEGRATED INSPECTION SYSTEM Having the above mentioned characteristics, IIS becomes a perfect candidate to be the legitimate inspection system in a digital manufacturing environment, and the ideal choice for Industry 4.0 implementation. However, its implementation requires exploring innovative approaches and building novel tools to model, simulate, design, and analyze the performance and efficiencies of forward-thinking inspection by integrating the PMP, SGE, and DZE tasks capable of Manufacturing Error Compensation and Geometric Defect Correction. New generations of computational algorithms need to be developed for compensation of errors and correction of the defects by analysing the inspection data. The needed methods should use the upstream manufacturing data and consider the requirements in the downstream operations, including repair, intermittent manufacturing operation, post-processing and finishing operations. Figure 1 presents the flow of the operations in an IIS. Developing inspection strategies initially starts using the up-stream manufacturing data. Then, the method of inspection is determined based on the type of design features and specifications. This determination is then used to start PMP. The sample size and location can be determined in PMP, as well as the type and number of sensors required for data acquisition. Based on the paths required and the geometric features being examined, the sensor’s optimal parameters must be determined for the specific measurement environment. Following PMP, SGE and DZE can both be carried out. The information gathered in these two tasks can be fed back into the PMP for dynamic control of the process. This ensures that the collected data is optimal for changing manufacturing environments. Once all these steps are completed, the acceptability of the part is determined, and it can be rejected and sent for repair, or passed to the downstream manufacturing operations. At all stages in the IIS, the data gathered or created is stored in a database that can be accessed and utilized by all other activities in IIS, and the entire digital manufacturing environment. The information can be used to study the manufacturing errors (Ahmad Barari, 2013). With all the information gathered, it is important that it can be operated as quick and efficient as possible so that the production lines do
IIS is a system of measurement and analysis built directly into the manufacturing process. As a part is manufactured, it should be examined periodically to ensure that it is maintaining an adherence to standards. The system is integrated because the data gleaned from the part is directly fed into a digital manufacturing database that holds all data relevant to the system. This data comes from a variety of sources on the machine, especially in today’s culture of Industry 4.0 from sensors on individual machines responsible for material manipulation, to data collected from the work-piece itself, in the form of point clouds, substitute geometries, and detailed deviation zones. All of this data has to be used to better inform the process in real-time, and be used to dynamically control the process for optimal results. If the part is affected negatively by a machine, for example a milling machine damaging the surface of the part due to a worn cutting tool, it will be known from the simulated skin models. The cost and reliability of IIS is highly dependent on the geometric complexity of the measured surface. Customized algorithms are needed for various geometric cases to reduce the computational uncertainties (A. Barari, ElMaraghy, & Knopf, 2007; A Barari, 2008; Ahmad Barari, ElMaraghy, & ElMaraghy, 2009; Ahmad Barari, ElMaraghy, & Knopf, 2007; Gohari & Barari, 2016; Jamiolahmadi & Barari, 2014; Lalehpour et al., 2017). Although the optical sensors including the laser scanners can capture large size of data points, provide the entire collected 183
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada 184
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Up-stream manufacturing digital data
Inspection based on a discrete data and point clouds
Inspection based on a geometric model (constructed geometries, parametric surfaces, tessellated surfaces)
Feature descriptors evaluation
Feature recognition
Primitive shapes (plane, sphere, cone)
Feature inspection
Freeform shapes
Whole body inspection Integrated Digital Design and Manufacturing DATA
Point Measurement Planning (PMP)
Sample size
Multiple sensors
Sample location
Single sensors
Determination of sensor optimum parameters Substitute Geometry Evaluation (SGE) Deviation Zone Estimation (DZE) Down-stream operation (work-piece finishing, assembly, part correction, process maintenance, …) Fig. 1. Flow of tasks in integrated inspection system (IIS) not get delayed by the computation. Considering the importance of minimizing the number and quality of sample points the role of PMP is highly critical in this system. The objective in developing a new sampling strategy in this paper is to increase the accuracy of the part information produced in a digital twin for IIS.
points should be analysed to find the deviation zones. The number of the measured points directly effects the computation time and convergence of analyses in SGE and DZE processes. The classic sampling methods seen in coordinate metrology are typically uniform sampling, random sampling and stratified sampling (A. Barari & Mordo, 2013; Obeidat & Raman, 2009). Adaptive sampling is a more recently used sampling approach in which the distribution of measured points is defined based on measuring surface characteristics.
3. MEASURED POINTS DENSITY EVALUATION BASED ON GEOMETRIC COMPLEXITY In an inspection procedure, a part is measured from discrete points using measuring devices and the resulted measured 184
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
The characteristics can be defined based on area, surface signature, or hybrid attitudes. Adaptive approaches in sampling are more efficient and reliable in terms of determination of density of the measured points for the freeform and complex surfaces especially when laser scanners are being used for the process of inspection. However, defining a criteria for the determination of the complexity and finding the measured points distribution from the complexity, is challenging. Curvature can be considered as the instant value that represents the complexity for the curves. There are different definitions for curvature in 3D space such as mean curvature, principal curvatures and Gaussian curvature. In this research, mean curvature is considered as the proper estimation for the instant complexity. A complexity index based on mean curvature for each patch can be defined as the following equation: 𝐶𝐶𝐶𝐶𝑝𝑝𝑝𝑝 = ∬|𝐻𝐻 − 𝐻𝐻𝑐𝑐 |√𝐸𝐸𝐸𝐸 −
𝐹𝐹 2 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
Fig. 2. Surface patches with different complexities. Planar patches are considered as the patches with minimum density. 4. RESULTS AND DISCUSSION The proposed density estimation algorithm is implemented to find the density of the distribution of the measured points for two different freeform surfaces. Two case studies are presented in Fig. 3 and Fig. 4. The density of the points extracted from the surfaces is highly dependent on the surface’s complexity. Each surface is divided to a grid of 20 × 20 and the measured points, shown in red, are extracted from the surface. Based on the methodology, the patch that has the minimum complexity is selected and based on its complexity the coefficient 𝛼𝛼 is determined to normalize the complexity indexes. The minimum number of measured points for the patch with minimum complexity for both surfaces is considered as 𝒩𝒩𝑏𝑏 = 4, as the patch is assumed to be a rectangular plane. The plots show a proper agreement between the number of the extracted points and the complexity of the patch which could result in considerable reduction in the inspection time.
(1)
where, 𝐸𝐸 = 𝑆𝑆𝑢𝑢 𝑆𝑆𝑢𝑢 , 𝐹𝐹 = 𝑆𝑆𝑢𝑢 𝑆𝑆𝑣𝑣 and 𝐺𝐺 = 𝑆𝑆𝑣𝑣 𝑆𝑆𝑣𝑣 are the first fundamental coefficients and 𝑆𝑆𝑢𝑢 , 𝑆𝑆𝑣𝑣 are the surface derivative with respect to each parameter 𝑢𝑢 and 𝑣𝑣. H is the instant mean curvature and 𝐻𝐻𝑐𝑐 is the average value of mean curvatures on the corresponding domain. The mean curvature can be calculated form the following equations. 𝐻𝐻 =
𝐿𝐿𝐿𝐿 − 2𝑀𝑀𝑀𝑀 + 𝑁𝑁𝑁𝑁 2(𝐸𝐸𝐸𝐸 − 𝐹𝐹 2 )
(2)
where 𝐿𝐿 = 𝑛𝑛𝑆𝑆𝑢𝑢𝑢𝑢 , 𝑀𝑀 = 𝑛𝑛𝑆𝑆𝑢𝑢𝑢𝑢 and 𝑁𝑁 = 𝑛𝑛𝑆𝑆𝑣𝑣𝑣𝑣 are the coefficients of the second fundamental of a surface, 𝑛𝑛 is the surface normal and 𝑆𝑆𝑢𝑢𝑢𝑢 , 𝑆𝑆𝑢𝑢𝑢𝑢 and 𝑆𝑆𝑣𝑣𝑣𝑣 are the second derivatives of a parametric surface. To normalize the complexity indexes, all the values can be divided by the maximum value of the indexes. 𝑁𝑁𝑁𝑁𝑁𝑁𝑝𝑝𝑝𝑝 =
𝐶𝐶𝐶𝐶𝑝𝑝𝑝𝑝 max(𝐶𝐶𝐶𝐶)
(3)
The final normalized complexity index (NCI) values will be a decimal number between 0 and 1. The minimum number of required points for the inspection of each one of the patches can be defined as a base factor 𝒩𝒩𝑏𝑏 . The base factor, 𝒩𝒩𝑏𝑏 , is the minimum number of measured points required to measure the simplest geometric patch on the workpiece surface. For example, in Fig. 2, the simplest patches are the planar patches at each corner of the surface. To find the number of points for each patch based on the normalized complexity index (NCI) and minimum number of points for the simplest patch 𝒩𝒩𝑏𝑏 can be calculated based on the following coefficient: 𝛼𝛼 =
𝒩𝒩𝑏𝑏 𝑁𝑁𝑁𝑁𝑁𝑁𝑝𝑝𝑝𝑝
(4)
Therefore the number of measured points for each patch is as follows: 𝑁𝑁𝑝𝑝𝑝𝑝 = 𝛼𝛼 × 𝑁𝑁𝑁𝑁𝑁𝑁𝑝𝑝𝑝𝑝
185
(5)
The number of sample points determined for each patch can be determined based on the defined complexity value.
185
Fig. 3. Sampling instruction calculated based on the geometric complexity for Case Study #1.
2019 IFAC IMS 186 August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Fig. 4. Sampling instruction calculated based on the geometric complexity for Case Study #2.
sampling method is providing more accurate information about the deviation zone on the actual work-piece than the stratified random sampling, while the same number of sample points are used for SGE process.
In order to validate the efficiency of this method it is important to study if the developed method is capable to virtually sample a set of high quality sample points in a digital twin of IIS. These points may be missed if a poor sampling strategy for virtual sampling is used. In Fig. 5, a stratified random sampling strategy is used to generate 49 sample points on a free-form surface in Case Study #2. After fitting this data to the ideal model, a deviation zone of 2.955 mm is found. This is the total distance between the two most deviant points in negative and positive sides of the surface.
Fig. 7. Result of SGE for 294 points selected using the developed density- based method from a grid of 7 × 7.
5. CONCLUSIONS The concept of employing a digital twin for the IIS is presented in this paper. The proposed digital twin conducts a virtual sampling from a large 3D data point cloud captured from the physical worckpiece on the production line. A methodology is presented to select a limited group of well representative sample points during the virtual sampling process considering the geometric complexities on the measured surface. The developed method defines the location and number of this group of high quality sample points. The method was examined in two separate case studies of parts with differing complexity. The developed sampling strategy has been more successful in detecting the underlying geometric errors on the measured surface compared with the same number of sample points gained from a stratified sampling method. Providing the highest amount of important data is needed for the implementation of IISs, where the detailed deviation zone simulated model should be updated in real-time. The results were promising and show that more work should be done in this area, and that this is a viable path towards a new virtual sampling strategy for coordinate metrology in IISs.
Fig. 5. Result of SGE for 49 stratified sample points from a grid of 7 × 7. In Fig. 6, 294 points are sampled from the same work-piece. These points are also chosen using a stratified random sampling strategy. With this sample set, deviation zone of 3.466 is found. This is a 17.2% improvement in detecting the deviation zone with a 500% increase in the number of sampled points.
ACKNOWLEDGMENT The research support provided by the Natural Science and Engineering Research Council of Canada (NSERC) is greatly appreciated.
Fig. 6. Result of SGE for 294 stratified sample points from a grid of 7 × 7. In Fig. 7, 294 points are selected again from the same surface, this time using the density based sampling method developed in this paper. The data retrieved using this method shows a deviation zone of 3.818 mm. This is a 29.2% increase over the deviation zone found with 49 points, and a 10.2% increase over the deviation zone found through stratified random sampling of 294 points. This shows that the density based
REFERENCES Barari, A., ElMaraghy, H. A., & Knopf, G. K. (2007). Evaluation of Geometric Deviations in Sculptured Surfaces Using Probability Density Estimation. In Models for Computer Aided Tolerancing in Design and Manufacturing (pp. 135–146). Springer. https://doi.org/10.1007/1-4020-5438-6_15 186
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Barari, A., & Mordo, S. (2013). Effect of sampling strategy on uncertainty and precision of flatness inspection studied by dynamic minimum deviation zone evaluation. International Journal of Metrology and Quality Engineering, 4(1), 3–8. https://doi.org/10.1051/ijmqe/2012039 Barari, A. (2008). Sources of uncertainty in coordinate metrology of automotive body. In Proceedings of 2nd CIRP International Conference on Assembly Technologies and Systems (CATS 2008), Toronto, ON, Canada, Sept (pp. 21–23). Barari, A, ElMaraghy, H. A., & Orban, P. (2009). NURBS representation of actual machined surfaces. International Journal of Computer Integrated Manufacturing, 22(5), 395–410. Barari, Ahmad. (2013). Inspection of the machined surfaces using manufacturing data. Journal of Manufacturing Systems, 32(1), 107. https://doi.org/10.1016/j.jmsy.2012.07.011 Barari, Ahmad, ElMaraghy, H. A., & ElMaraghy, W. H. (2009). Design for Manufacturing of Sculptured Surfaces: A Computational Platform. Journal of Computing and Information Science in Engineering, 9(2), 021006. https://doi.org/10.1115/1.3130143 Barari, Ahmad, ElMaraghy, H. A., & Knopf, G. K. (2007). Search-Guided Sampling to Reduce Uncertainty of Minimum Deviation Zone Estimation. Journal of Computing and Information Science in Engineering, 7(4), 360. https://doi.org/10.1115/1.2798114 Barari, Ahmad, & Pop-Iliev, R. (2009). Reducing rigidity by implementing closed-loop engineering in adaptable design and manufacturing systems. Journal of Manufacturing Systems, 28, 47–54. https://doi.org/10.1016/j.jmsy.2009.04.003 Berry, C., & Barari, A. (2018). Closed-Loop Coordinate Metrology for Hybrid Manufacturing System. IFACPapersOnLine, 51(11), 752–757. https://doi.org/10.1016/J.IFACOL.2018.08.409 Feng, C.-X. J., Saal, A. L., Salsbury, J. G., Ness, A. R., & Lin, G. C. S. (2007). Design and analysis of experiments in CMM measurement uncertainty study. Precision Engineering, 31(2), 94–101. https://doi.org/10.1016/J.PRECISIONENG.2006.03.00 3 Gohari, H., & Barari, A. (2016). A quick deviation zone fitting in coordinate metrology of NURBS surfaces using principle component analysis. Measurement: Journal of the International Measurement Confederation, 92, 352–364. https://doi.org/10.1016/j.measurement.2016.05.050 Hammad Mian, S., & Al-Ahmari, A. (2014). New developments in coordinate measuring machines for manufacturing industries. International Journal of Metrology and Quality Engineering, 5(1), 101. https://doi.org/10.1051/ijmqe/2014001 Jamiolahmadi, S., & Barari, A. (2014). Surface Topography of Additive Manufacturing Parts Using a Finite Difference Approach. Journal of Manufacturing Science and Engineering, 136(6), 061009. https://doi.org/10.1115/1.4028585
187
Lalehpour, A., Berry, C., & Barari, A. (2017). Adaptive data reduction with neighbourhood search approach in coordinate measurement of planar surfaces. Journal of Manufacturing Systems, 45, 28–47. https://doi.org/10.1016/J.JMSY.2017.07.001 Mahboubkhah, M., & Barari, A. (2019). International Journal of Computer Integrated Manufacturing Design and development of a novel 4-DOF parallel kinematic coordinate measuring machine (CMM) Design and development of a novel 4-DOF parallel kinematic coordinate measuring machine (CMM). https://doi.org/10.1080/0951192X.2019.1610576 Obeidat, S. M., & Raman, S. (2009). An intelligent sampling method for inspecting free-form surfaces. The International Journal of Advanced Manufacturing Technology, 40(11–12), 1125–1136. https://doi.org/10.1007/s00170-008-1427-3 Vrba, I., Palencar, R., Hadzistevic, M., … B. S.-M. science, & 2015, undefined. (n.d.). Different approaches in uncertainty evaluation for measurement of complex surfaces using coordinate measuring machine. Degruyter.Com. Retrieved from https://www.degruyter.com/downloadpdf/j/msr.2015.15 .issue-3/msr-2015-0017/msr-2015-0017.xml Zhao, F., Xu, X., industry, S. X.-C. in, & 2009, undefined. (n.d.). Computer-aided inspection planning—the state of the art. Elsevier. Retrieved from https://www.sciencedirect.com/science/article/pii/S016 6361509000451
187
ScienceDirect
IFAC PapersOnLine 52-10 (2019) 182–187
A Digital Twin for Integrated Inspection System in Digital Manufacturing A Digital Twin for Integrated Inspection System in Digital Manufacturing A Digital Twin for Integrated Inspection System in Digital Manufacturing Hossein Gohari* Cody Berry*in Digital Manufacturing A Digital Twin for Integrated Inspection System Hossein Gohari* Cody Berry*
HosseinAhmad Gohari* Cody Berry* Barari* HosseinAhmad Gohari* Cody Berry* Barari* HosseinAhmad Gohari* Cody Berry* Barari* * Faculty of Engineering and Applied Science, Ahmad Barari* of Engineering and Applied Science, * Faculty University of Ontario Institute of Technology (UOIT), * Faculty of Engineering and Applied Science, University of Ontario Institute of Technology (UOIT), Oshawa, Ontario, Canada * Faculty of Engineering and Applied Science, University of Ontario Institute ofCanada Technology (UOIT), Oshawa, Ontario, (e-mails: [email protected] , [email protected] , [email protected]) University of Ontario ofCanada Technology (UOIT), Oshawa,Institute Ontario, (e-mails: [email protected] , [email protected] , [email protected]) Oshawa, Ontario, Canada (e-mails: [email protected] , [email protected] , [email protected]) (e-mails: [email protected] , [email protected] , [email protected]) Abstract: Coordinate metrology is a crucial part in advanced manufacturing industries to achieve and and Abstract: Coordinate metrology is a crucial part in advanced manufacturing industries to achieve maintain conformance of high-quality products within design specifications. Meanwhile, softwareAbstract: Coordinate metrology is a crucial part in advanced manufacturing industries to achieve and maintain conformance of high-quality products within design specifications. Meanwhile, softwarecomponents are increasingly becoming anproducts essential part ofdesign the manufacturing inspection process becauseto ofachieve increasing Abstract: Coordinate metrology is a crucial part in advanced industries and maintain conformance of high-quality within specifications. Meanwhile, softwarecomponents are increasingly becoming an essential part of the inspection process because of increasing part complexities in design and the high-volume of dataofdesign captured from different sensors of in increasing hardwaremaintain conformance of high-quality products within specifications. Meanwhile, softwarecomponents are increasingly becoming an essential part the inspection process because part complexities in design and athe high-volume dataparallel captured from differentinspection sensors insystem hardwarecomponents. Thisincreasingly paper presents virtual toof work to an integrated (IIS) components are becoming anreplica essential part the inspection process because increasing part complexities in design and the high-volume ofwork dataof captured from different sensors of insystem hardwarecomponents. This paper presents a virtual replica to parallel to an integrated inspection (IIS) for inspection of freeform and complex surfaces based on a metric of their geometric complexity. In this part complexities in design and the high-volume of data captured from different sensors in hardwarecomponents. This paper presents a virtualsurfaces replicabased to work parallel antheir integrated inspection systemIn(IIS) for inspection of freeform and complex onconducted a metricto of geometric complexity. this approach, an intelligently guided sampling is virtually from a large dataset, instead of the components. This paper presents a virtual replica to work parallel to an integrated inspection system (IIS) for inspection of freeform and complex surfaces based onconducted a metric offrom theira geometric complexity. In this approach, an intelligently guided sampling is virtually large dataset, instead of the physical sampling process when the sample points are traditionally selected randomly from the measured for inspection of freeform and complex surfaces based on a metric of their geometric complexity. In this approach, an intelligently guidedthesampling is virtually conducted selected from a randomly large dataset, instead of the physical sampling process when sample points are traditionally from the measured surface. Implementation a closed-loop between the traditionally main tasks inselected IIS isa considered in developing approach, an intelligently guided is virtually conducted from large dataset, instead of this the physical sampling processof when thesampling sample points are randomly from the measured surface. Implementation ofuncertainties a closed-loop between the traditionally main tasks inselected IIS is considered in is developing this digital twin to reduce the associated with the inspection process. A method introduced to physical sampling process when the sample points are randomly from the measured surface. Implementation a closed-loop betweenwith the main tasks in IIS is considered in is developing this digital twin tolocal reduce theofuncertainties associated the inspection process. A method introduced to estimate the densities of the measured points required for virtual sampling from each patch on the surface. Implementation of a closed-loop between the main tasks in IIS is considered in developing this digital twin tolocal reduce the uncertainties associated with the inspection process. A method is introduced to estimate the densities of the measured points required for virtual sampling from each patch on the work-pieces’s surface based ontheitsmeasured geometric complexity. Two case studies are conducted to verify digital twin reduce the uncertainties associated with the inspection process. A method is introduced to estimate the tolocal densities ofon points required for virtual sampling from each patch on the the work-pieces’s surface based its geometric complexity. Two case studies are conducted to verify the effectiveness ofsurface thedensities methodology. observed efficiency selection of the are important measured data in estimate the local ofontheitsThe measured points requiredin for virtual sampling from each patch on the the work-pieces’s based geometric complexity. Two case studies conducted to verify effectiveness ofsurface the methodology. efficiencystrategy in selection ofimplemented the are important datathe in the proposed sampling strategy itobserved a better sampling to be in measured a digital twin work-pieces’s based onmakes itsThe geometric complexity. Two case studies conducted to verify effectiveness of the methodology. The observed efficiencystrategy in selection ofimplemented the important measured datafor in the proposed sampling strategy makes it a better sampling to be in a digital twin for IISs. effectiveness of the methodology. The itobserved efficiencystrategy in selection ofimplemented the important measured datafor in the proposed sampling strategy makes a better sampling to be in a digital twin IISs. the proposed sampling strategy makes it a better sampling strategy to be implemented in a digital twin for IISs. © 2019, IFAC (International of Automatic Control) Hosting by Elsevier All rights reserved. Keywords: Digital Twin, Federation Virtual Sampling, Coordinate Metrology, Point Ltd. Measurement Planning, IISs. Keywords: Digital Twin, VirtualDeviation Sampling, Coordinate Metrology, Point Measurement Planning, Substitute Geometry Evaluation, Zone Estimation, Integrated inspection system , geometric Keywords: Digital Twin, VirtualDeviation Sampling, Coordinate Metrology, Point Measurement Planning, Substitute Geometry Evaluation, Zone Estimation, Integrated inspection system , geometric complexity, Keywords: Digital Manufacturing. Twin, VirtualDeviation Sampling, Coordinate Metrology, Measurement Planning, Substitute Evaluation, Zone Estimation, Integrated Point inspection system , geometric complexity,Geometry Digital Manufacturing. Substitute Geometry Evaluation, Deviation Zone Estimation, Integrated inspection system , geometric complexity, Digital Manufacturing. complexity, Digital Manufacturing. Barari, 2019). Research studies in coordinate metrology show 1. INTRODUCTION Barari, Research in coordinate metrology show that the2019). uncertainties in studies IIS operations are highly affected by 1. INTRODUCTION Barari, 2019). Research studies in coordinate metrology show that the uncertainties in IIS operations are highly affected by 1. INTRODUCTION the number of measured points (Feng, Saal, Salsbury, Ness, & Today’s manufacturing systems have the flexibility to Barari, 2019). Research studies in coordinate metrology show that the uncertainties in IIS operations are highly affected by 1. INTRODUCTION Today’s manufacturing systems have the flexibility to the number of measured points (Feng, Saal, Salsbury, Ness, & Lin, 2007; Hammad Mian & Al-Ahmari, 2014; Vrba, fabricate a large variety of parts with high complexity, low that the uncertainties in IIS operations are highly affected by number ofHammad measured Mian points (Feng, Saal, Salsbury, Ness, & Today’s amanufacturing systems have thecomplexity, flexibility low to the fabricate large variety of parts with high Lin, 2007; & Al-Ahmari, 2014; Vrba, Palencar, …, & (Feng, n.d.).Salsbury, Recently, laser volume, and restricted tolerances. In order low number ofHammad measured Mian points Saal, & Today’s have flexibility to the Lin, 2007;Hadzistevic, &2015, Al-Ahmari, 2014;Ness, Vrba, fabricate amanufacturing largewith variety ofsystems parts with highthecomplexity, Palencar, Hadzistevic, …, & 2015, n.d.). Recently, laser volume, and with restricted tolerances. In order to scanners are widely being used in IIS because of their accommodate these requirements, intelligent process control is Lin, 2007; Hammad Mian & Al-Ahmari, 2014; Vrba, fabricate a large variety of parts with high complexity, low Hadzistevic, …, &used 2015, n.d.). Recently, volume, and these withrequirements, restricted intelligent tolerances.process In order to scanners are widely being in inspection IIS because of laser their accommodate control is Palencar, and widely time efficiency in2015, thein tasks. needed notandonly torequirements, assist in providing theprocess flexibility and Palencar, are Hadzistevic, …, &used n.d.). Recently, laser volume, with restricted tolerances. In order to scanners being IIS because of These their accommodate these intelligent control is versatility needed not only to assist in providing the flexibility and versatility and time efficiency in the inspection tasks. These provide a large numberused of pointstasks. thatofshould accuracy, alsotorequirements, toassist be implemented inthe real-time and and inscanners are being IIS because their accommodate these intelligent process control is sensors versatility and widely time efficiency in measured thein inspection These needed notbut only in providing flexibility sensors provide a large number of measured points that should accuracy, but also to be implemented in real-time and inbe analysed to make a proper decision about the product. process controlling procedures. In common practice, versatility and time efficiency in the inspection tasks. These needed notbutonly totoassist in providinginthe flexibility and sensors provide a large number of measured points that should accuracy, also be implemented real-time and inanalysed to make a proper decision aboutpoints the product. process controlling procedures. practice, manufactured at theIn ofreal-time a manufacturing sensors provide a collection large number of measured thatprocess should accuracy, but parts also are to inspected be implemented incommon and in- be be analysed to make a proper decision about the product. process controlling procedures. Inend common practice, After the data process, an automated manufactured parts are inspected at the end of a manufacturing process. The most practical and adaptable metrology approach be analysed to make a proper decision about the product. process controlling procedures. In common practice, After the data collection process, an automated process manufactured partspractical are inspected at the endmetrology of a manufacturing employs datacollection to find theprocess, best substitute geometryprocess which process. The most and adaptable approach thethe data an automated to the digital manufacturing environment inspect the After manufactured parts are inspected at the endmetrology of atomanufacturing employs the data to find the best substitute geometry which process. The most practical and adaptable approach results in considerable reduction in the inspection time as the thethedata an automated process to the digital manufacturing environment to inspect the After employs datacollection to find theprocess, bestinsubstitute geometry which geometric and dimensional attributes is coordinate metrology. process. The most practical and adaptable metrology approach results in considerable reduction the inspection time as the to the digital manufacturing environment to inspect the part no longer needs to be manually oriented in a specific employs the data to find the best substitute geometry which geometric and Metrology dimensional attributes is coordinate metrology. results in considerable reduction in the inspection time asway the A Coordinate Machine (CMM) typically utilizes a to the digital manufacturing environment to inspect the part no longer needs to be manually oriented in a specific geometric and Metrology dimensional attributes is coordinate metrology. to obtain optimal scanning results (Zhao, Xu, industry, & in considerable reduction in the inspection time asway the A Coordinate Machine (CMM) typically utilizestoa results part no longer needs to be manually oriented in a specific way combination of tactile or contact probes and optical sensors geometric and Metrology dimensional attributes is coordinate metrology. obtain optimal scanning results (Zhao, Xu, industry, & A Coordinate Machine (CMM) typically utilizestoa to 2009, n.d.). The final information that needs to be determined part no longer needs to be manually oriented in a specific way combination of tactile or contact probes and optical sensors obtain optimal scanning results (Zhao, Xu,beindustry, & capture 3D discrete points from the work-piece surfaces. Using A Coordinate Metrology Machine (CMM) typically utilizes a to 2009, n.d.). The final information that needs to determined combination of tactile or contact probes and optical sensors to from the measured points areresults the that geometric and dimensional to obtain optimal scanning (Zhao, Xu, industry, & capture 3D discrete points from the work-piece surfaces. Using 2009, n.d.). The final information needs to be determined the tactile sensors is typically slow and more expensive combination of tactile or contact probes optical sensors to from the measured points are the geometric and dimensional capture 3D discrete points from the work-piece surfaces. Using deviations from an ideal model. The algorithms that are needed 2009, n.d.). The final information that needs to be determined the tactile sensors is typically slow and more expensive the measured points are the and comparing to usingpoints the optical sensors. Themore optical sensors capture 3D discrete from the work-piece surfaces. Using from deviations from an ideal model. Thegeometric algorithms thatdimensional are needed the tactile sensors is typically slow and expensive findthe themeasured number and the distribution of the measured points, from points are the and comparing to using the optical sensors. The optical sensors to deviations from an ideal model. Thegeometric algorithms thatdimensional are needed can capture hundreds of thousands of data points in a fraction the tactile sensors is typically slow and more expensive to find the number and the distribution of the measured points, comparing to using the optical sensors. The optical sensors evaluation of the best substitute geometry, and also the deviations from an ideal model. The algorithms that are needed can capture hundreds ofthey thousands of dataThe points in a fraction to find the number and the distribution of the measured points, of a second. However, are typically more vulnerable to comparing to using the optical sensors. optical sensors evaluation of the best substitute geometry, and also the can capture hundreds of thousands of data points in a fraction calculation of deviation zones are the major cyber components to find the number and the distribution of the measured points, of a capture second. However, they are typically more vulnerable to evaluation of the best substitute geometry, and also the noise sources and their data collection accuracy and precision can hundreds of thousands of data points in a fraction calculation of deviation zones are the major cyber components of a second. However, they collection are typically more and vulnerable to evaluation in an Integrated Inspection System and of deviation the best substitute geometry, alsotheir the noise sources and their data accuracy precision of zones are the major (IIS), cyberand components are than tactile probes (Lalehpour, Berry, & Barari, 2017). of aless second. However, they are typically more vulnerable to calculation in an Integrated Inspection System (IIS), and bytheir noise sources and their data collection accuracy and precision corresponding computational cost is highly dominated the calculation of deviation zones are the major cyber components are less than tactile probes (Lalehpour, Berry, & Barari, 2017). an Integrated Inspection System (IIS), and bytheir Because of the high tolerances demanded by & industry and by in noise sources and their data collection accuracy and precision computational costthe is highly dominated the are less than tactile probes (Lalehpour, Berry, Barari, 2017). number the sample points and quality(IIS), of sampling. The in an of Integrated Inspection System and bytheir Because of the high tolerances demanded by industry and by corresponding corresponding computational cost is highly dominated the consumers, it is vital that parts are inspected accurately and are less than tactile probes (Lalehpour, Berry, & Barari, and 2017). number of the sample points and the quality of sampling. The Because of the high tolerances demanded by industry by IIS cyberofcomponents have been traditionally conducted inthea corresponding computational costthe is highly dominated byThe consumers, it is vital that parts are inspected accurately and number the sample points and quality of sampling. efficiently. However, the increasing geometric complexity in Because of the by industry andand by IIS cyberofcomponents have been traditionally in2)a consumers, it ishigh vitaltolerances thatincreasing partsdemanded are inspected accurately sequence of: 1) Point Measurement Planning (PMP),The the sample points and the quality of conducted sampling. efficiently. the geometric complexity in number IIS cyber components have been traditionally conducted in2)a the productHowever, uncertainties in and the consumers, it results is vital in thatcomputational parts are inspected accurately sequence of: 1) Point Measurement Planning (PMP), efficiently. However, the increasing geometric complexity in Substitute Geometry Estimation (SGE), and 3) Deviation Zone IIS cyber components have been traditionally conducted in2)a the productprocess results(Ahmad in computational uncertainties in the sequence of: 1) Point Measurement Planning (PMP), inspection Barari, 2013; Mahboubkhah & efficiently. However, the increasing geometric complexity in Substitute Geometry Estimation (SGE), and 3) Deviation Zone the productprocess results(Ahmad in computational uncertainties in the sequence Geometry of: 1) Point Measurement Planning (PMP), 2) inspection Barari, 2013; Mahboubkhah & Substitute Estimation (SGE), and 3) Deviation Zone the productprocess results(Ahmad in computational uncertainties in the inspection Barari, 2013; Mahboubkhah & Substitute Geometry Estimation (SGE), and 3) Deviation Zone 2405-8963 IFAC (International Federation of Automatic Control) Copyright@ 2019 IFAC inspection© 2019, process (Ahmad Barari, 2013; Mahboubkhah & 182Hosting by Elsevier Ltd. All rights reserved. Copyright@ 2019 IFAC 182Control. Peer review under responsibility of International Federation of Automatic Copyright@ 2019 IFAC 182 10.1016/j.ifacol.2019.10.020 Copyright@ 2019 IFAC 182
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Evaluation (DZE) (Lalehpour et al., 2017). PMP presents a sampling strategy and a detailed plan to capture data points from a surface. The location and number of the measured points decided by PMP have direct effect on the time and accuracy of an inspection procedure. Evaluation of the best geometry that minimizes the deviations between the measured points and a nominal geometry, typically presented by a Computer Aided Design (CAD) model is SGE’s task. SGE is a delegate computational process which its stability and convergence are highly affected by the density of the sample points. Using the evaluated substitute geometry, the deviation zone of the actual measured surface is estimated using a DZE algorithm. DZE may construct a detailed deviation zone measured surface by simulating a skin model for the measured surface (Lalehpour & Barari, 2017; Jamiolahmadi & Barari, 2014;).
183
dataset not necessary helps the SGE and DZE tasks to obtain the best results. In many cases, particularly for the surfaces with high levels of geometric complexities, increasing the size of datasets makes the SGE’s optimization problem highly nonlinear and the simulation in the DZE process very expensive to solve. Therefore, it is very desired to select only an optimum number of the most representative sample points, referred as high quality sample points, by PMP for SGE and DZE process. This paper proposes developing a digital twin of IIS, when the high quality sample points are virtually selected from a large set of point cloud according to the feedbacks that are received by the closed-loops of PMP-SGE and PMP-DZE tasks. A methodology is developed to estimate the density of the required measurement points based on the geometric complexities on the measured surface. The reduction in number of sample points reduces the inspection time and uncertainties in evaluation of the substitute geometry and estimation of the deviation zones.
The isolated and sequential completion of PMP, SGE, and DZE results in significant uncertainties. Each of these three steps is a software-component in a cyber-physical system that is used to inspect manufactured parts. However, they can be integrated to form an IIS. The IIS is an attempt to reduce the inspection uncertainties by implementing closed-loops between these tasks. The closed-loop approach results in sharing information between these tasks. Examples of algorithms that can be used in for the tasks integrations in inspection and manufacturing can be found in (A Barari, ElMaraghy, & Orban, 2009; Berry & Barari, 2018). It results in detecting the errors and compensating them in the intermediate results to reduce the uncertainties in inspection system. As reported in (Ahmad Barari & Pop-Iliev, 2009), the quality of final products can be increased by implementing individual closed-loops for different steps of engineering activities.
2. STRUCTURE OF INTEGRATED INSPECTION SYSTEM Having the above mentioned characteristics, IIS becomes a perfect candidate to be the legitimate inspection system in a digital manufacturing environment, and the ideal choice for Industry 4.0 implementation. However, its implementation requires exploring innovative approaches and building novel tools to model, simulate, design, and analyze the performance and efficiencies of forward-thinking inspection by integrating the PMP, SGE, and DZE tasks capable of Manufacturing Error Compensation and Geometric Defect Correction. New generations of computational algorithms need to be developed for compensation of errors and correction of the defects by analysing the inspection data. The needed methods should use the upstream manufacturing data and consider the requirements in the downstream operations, including repair, intermittent manufacturing operation, post-processing and finishing operations. Figure 1 presents the flow of the operations in an IIS. Developing inspection strategies initially starts using the up-stream manufacturing data. Then, the method of inspection is determined based on the type of design features and specifications. This determination is then used to start PMP. The sample size and location can be determined in PMP, as well as the type and number of sensors required for data acquisition. Based on the paths required and the geometric features being examined, the sensor’s optimal parameters must be determined for the specific measurement environment. Following PMP, SGE and DZE can both be carried out. The information gathered in these two tasks can be fed back into the PMP for dynamic control of the process. This ensures that the collected data is optimal for changing manufacturing environments. Once all these steps are completed, the acceptability of the part is determined, and it can be rejected and sent for repair, or passed to the downstream manufacturing operations. At all stages in the IIS, the data gathered or created is stored in a database that can be accessed and utilized by all other activities in IIS, and the entire digital manufacturing environment. The information can be used to study the manufacturing errors (Ahmad Barari, 2013). With all the information gathered, it is important that it can be operated as quick and efficient as possible so that the production lines do
IIS is a system of measurement and analysis built directly into the manufacturing process. As a part is manufactured, it should be examined periodically to ensure that it is maintaining an adherence to standards. The system is integrated because the data gleaned from the part is directly fed into a digital manufacturing database that holds all data relevant to the system. This data comes from a variety of sources on the machine, especially in today’s culture of Industry 4.0 from sensors on individual machines responsible for material manipulation, to data collected from the work-piece itself, in the form of point clouds, substitute geometries, and detailed deviation zones. All of this data has to be used to better inform the process in real-time, and be used to dynamically control the process for optimal results. If the part is affected negatively by a machine, for example a milling machine damaging the surface of the part due to a worn cutting tool, it will be known from the simulated skin models. The cost and reliability of IIS is highly dependent on the geometric complexity of the measured surface. Customized algorithms are needed for various geometric cases to reduce the computational uncertainties (A. Barari, ElMaraghy, & Knopf, 2007; A Barari, 2008; Ahmad Barari, ElMaraghy, & ElMaraghy, 2009; Ahmad Barari, ElMaraghy, & Knopf, 2007; Gohari & Barari, 2016; Jamiolahmadi & Barari, 2014; Lalehpour et al., 2017). Although the optical sensors including the laser scanners can capture large size of data points, provide the entire collected 183
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada 184
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Up-stream manufacturing digital data
Inspection based on a discrete data and point clouds
Inspection based on a geometric model (constructed geometries, parametric surfaces, tessellated surfaces)
Feature descriptors evaluation
Feature recognition
Primitive shapes (plane, sphere, cone)
Feature inspection
Freeform shapes
Whole body inspection Integrated Digital Design and Manufacturing DATA
Point Measurement Planning (PMP)
Sample size
Multiple sensors
Sample location
Single sensors
Determination of sensor optimum parameters Substitute Geometry Evaluation (SGE) Deviation Zone Estimation (DZE) Down-stream operation (work-piece finishing, assembly, part correction, process maintenance, …) Fig. 1. Flow of tasks in integrated inspection system (IIS) not get delayed by the computation. Considering the importance of minimizing the number and quality of sample points the role of PMP is highly critical in this system. The objective in developing a new sampling strategy in this paper is to increase the accuracy of the part information produced in a digital twin for IIS.
points should be analysed to find the deviation zones. The number of the measured points directly effects the computation time and convergence of analyses in SGE and DZE processes. The classic sampling methods seen in coordinate metrology are typically uniform sampling, random sampling and stratified sampling (A. Barari & Mordo, 2013; Obeidat & Raman, 2009). Adaptive sampling is a more recently used sampling approach in which the distribution of measured points is defined based on measuring surface characteristics.
3. MEASURED POINTS DENSITY EVALUATION BASED ON GEOMETRIC COMPLEXITY In an inspection procedure, a part is measured from discrete points using measuring devices and the resulted measured 184
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
The characteristics can be defined based on area, surface signature, or hybrid attitudes. Adaptive approaches in sampling are more efficient and reliable in terms of determination of density of the measured points for the freeform and complex surfaces especially when laser scanners are being used for the process of inspection. However, defining a criteria for the determination of the complexity and finding the measured points distribution from the complexity, is challenging. Curvature can be considered as the instant value that represents the complexity for the curves. There are different definitions for curvature in 3D space such as mean curvature, principal curvatures and Gaussian curvature. In this research, mean curvature is considered as the proper estimation for the instant complexity. A complexity index based on mean curvature for each patch can be defined as the following equation: 𝐶𝐶𝐶𝐶𝑝𝑝𝑝𝑝 = ∬|𝐻𝐻 − 𝐻𝐻𝑐𝑐 |√𝐸𝐸𝐸𝐸 −
𝐹𝐹 2 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
Fig. 2. Surface patches with different complexities. Planar patches are considered as the patches with minimum density. 4. RESULTS AND DISCUSSION The proposed density estimation algorithm is implemented to find the density of the distribution of the measured points for two different freeform surfaces. Two case studies are presented in Fig. 3 and Fig. 4. The density of the points extracted from the surfaces is highly dependent on the surface’s complexity. Each surface is divided to a grid of 20 × 20 and the measured points, shown in red, are extracted from the surface. Based on the methodology, the patch that has the minimum complexity is selected and based on its complexity the coefficient 𝛼𝛼 is determined to normalize the complexity indexes. The minimum number of measured points for the patch with minimum complexity for both surfaces is considered as 𝒩𝒩𝑏𝑏 = 4, as the patch is assumed to be a rectangular plane. The plots show a proper agreement between the number of the extracted points and the complexity of the patch which could result in considerable reduction in the inspection time.
(1)
where, 𝐸𝐸 = 𝑆𝑆𝑢𝑢 𝑆𝑆𝑢𝑢 , 𝐹𝐹 = 𝑆𝑆𝑢𝑢 𝑆𝑆𝑣𝑣 and 𝐺𝐺 = 𝑆𝑆𝑣𝑣 𝑆𝑆𝑣𝑣 are the first fundamental coefficients and 𝑆𝑆𝑢𝑢 , 𝑆𝑆𝑣𝑣 are the surface derivative with respect to each parameter 𝑢𝑢 and 𝑣𝑣. H is the instant mean curvature and 𝐻𝐻𝑐𝑐 is the average value of mean curvatures on the corresponding domain. The mean curvature can be calculated form the following equations. 𝐻𝐻 =
𝐿𝐿𝐿𝐿 − 2𝑀𝑀𝑀𝑀 + 𝑁𝑁𝑁𝑁 2(𝐸𝐸𝐸𝐸 − 𝐹𝐹 2 )
(2)
where 𝐿𝐿 = 𝑛𝑛𝑆𝑆𝑢𝑢𝑢𝑢 , 𝑀𝑀 = 𝑛𝑛𝑆𝑆𝑢𝑢𝑢𝑢 and 𝑁𝑁 = 𝑛𝑛𝑆𝑆𝑣𝑣𝑣𝑣 are the coefficients of the second fundamental of a surface, 𝑛𝑛 is the surface normal and 𝑆𝑆𝑢𝑢𝑢𝑢 , 𝑆𝑆𝑢𝑢𝑢𝑢 and 𝑆𝑆𝑣𝑣𝑣𝑣 are the second derivatives of a parametric surface. To normalize the complexity indexes, all the values can be divided by the maximum value of the indexes. 𝑁𝑁𝑁𝑁𝑁𝑁𝑝𝑝𝑝𝑝 =
𝐶𝐶𝐶𝐶𝑝𝑝𝑝𝑝 max(𝐶𝐶𝐶𝐶)
(3)
The final normalized complexity index (NCI) values will be a decimal number between 0 and 1. The minimum number of required points for the inspection of each one of the patches can be defined as a base factor 𝒩𝒩𝑏𝑏 . The base factor, 𝒩𝒩𝑏𝑏 , is the minimum number of measured points required to measure the simplest geometric patch on the workpiece surface. For example, in Fig. 2, the simplest patches are the planar patches at each corner of the surface. To find the number of points for each patch based on the normalized complexity index (NCI) and minimum number of points for the simplest patch 𝒩𝒩𝑏𝑏 can be calculated based on the following coefficient: 𝛼𝛼 =
𝒩𝒩𝑏𝑏 𝑁𝑁𝑁𝑁𝑁𝑁𝑝𝑝𝑝𝑝
(4)
Therefore the number of measured points for each patch is as follows: 𝑁𝑁𝑝𝑝𝑝𝑝 = 𝛼𝛼 × 𝑁𝑁𝑁𝑁𝑁𝑁𝑝𝑝𝑝𝑝
185
(5)
The number of sample points determined for each patch can be determined based on the defined complexity value.
185
Fig. 3. Sampling instruction calculated based on the geometric complexity for Case Study #1.
2019 IFAC IMS 186 August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Fig. 4. Sampling instruction calculated based on the geometric complexity for Case Study #2.
sampling method is providing more accurate information about the deviation zone on the actual work-piece than the stratified random sampling, while the same number of sample points are used for SGE process.
In order to validate the efficiency of this method it is important to study if the developed method is capable to virtually sample a set of high quality sample points in a digital twin of IIS. These points may be missed if a poor sampling strategy for virtual sampling is used. In Fig. 5, a stratified random sampling strategy is used to generate 49 sample points on a free-form surface in Case Study #2. After fitting this data to the ideal model, a deviation zone of 2.955 mm is found. This is the total distance between the two most deviant points in negative and positive sides of the surface.
Fig. 7. Result of SGE for 294 points selected using the developed density- based method from a grid of 7 × 7.
5. CONCLUSIONS The concept of employing a digital twin for the IIS is presented in this paper. The proposed digital twin conducts a virtual sampling from a large 3D data point cloud captured from the physical worckpiece on the production line. A methodology is presented to select a limited group of well representative sample points during the virtual sampling process considering the geometric complexities on the measured surface. The developed method defines the location and number of this group of high quality sample points. The method was examined in two separate case studies of parts with differing complexity. The developed sampling strategy has been more successful in detecting the underlying geometric errors on the measured surface compared with the same number of sample points gained from a stratified sampling method. Providing the highest amount of important data is needed for the implementation of IISs, where the detailed deviation zone simulated model should be updated in real-time. The results were promising and show that more work should be done in this area, and that this is a viable path towards a new virtual sampling strategy for coordinate metrology in IISs.
Fig. 5. Result of SGE for 49 stratified sample points from a grid of 7 × 7. In Fig. 6, 294 points are sampled from the same work-piece. These points are also chosen using a stratified random sampling strategy. With this sample set, deviation zone of 3.466 is found. This is a 17.2% improvement in detecting the deviation zone with a 500% increase in the number of sampled points.
ACKNOWLEDGMENT The research support provided by the Natural Science and Engineering Research Council of Canada (NSERC) is greatly appreciated.
Fig. 6. Result of SGE for 294 stratified sample points from a grid of 7 × 7. In Fig. 7, 294 points are selected again from the same surface, this time using the density based sampling method developed in this paper. The data retrieved using this method shows a deviation zone of 3.818 mm. This is a 29.2% increase over the deviation zone found with 49 points, and a 10.2% increase over the deviation zone found through stratified random sampling of 294 points. This shows that the density based
REFERENCES Barari, A., ElMaraghy, H. A., & Knopf, G. K. (2007). Evaluation of Geometric Deviations in Sculptured Surfaces Using Probability Density Estimation. In Models for Computer Aided Tolerancing in Design and Manufacturing (pp. 135–146). Springer. https://doi.org/10.1007/1-4020-5438-6_15 186
2019 IFAC IMS August 12-14, 2019. Oshawa, Canada
Hossein Gohari et al. / IFAC PapersOnLine 52-10 (2019) 182–187
Barari, A., & Mordo, S. (2013). Effect of sampling strategy on uncertainty and precision of flatness inspection studied by dynamic minimum deviation zone evaluation. International Journal of Metrology and Quality Engineering, 4(1), 3–8. https://doi.org/10.1051/ijmqe/2012039 Barari, A. (2008). Sources of uncertainty in coordinate metrology of automotive body. In Proceedings of 2nd CIRP International Conference on Assembly Technologies and Systems (CATS 2008), Toronto, ON, Canada, Sept (pp. 21–23). Barari, A, ElMaraghy, H. A., & Orban, P. (2009). NURBS representation of actual machined surfaces. International Journal of Computer Integrated Manufacturing, 22(5), 395–410. Barari, Ahmad. (2013). Inspection of the machined surfaces using manufacturing data. Journal of Manufacturing Systems, 32(1), 107. https://doi.org/10.1016/j.jmsy.2012.07.011 Barari, Ahmad, ElMaraghy, H. A., & ElMaraghy, W. H. (2009). Design for Manufacturing of Sculptured Surfaces: A Computational Platform. Journal of Computing and Information Science in Engineering, 9(2), 021006. https://doi.org/10.1115/1.3130143 Barari, Ahmad, ElMaraghy, H. A., & Knopf, G. K. (2007). Search-Guided Sampling to Reduce Uncertainty of Minimum Deviation Zone Estimation. Journal of Computing and Information Science in Engineering, 7(4), 360. https://doi.org/10.1115/1.2798114 Barari, Ahmad, & Pop-Iliev, R. (2009). Reducing rigidity by implementing closed-loop engineering in adaptable design and manufacturing systems. Journal of Manufacturing Systems, 28, 47–54. https://doi.org/10.1016/j.jmsy.2009.04.003 Berry, C., & Barari, A. (2018). Closed-Loop Coordinate Metrology for Hybrid Manufacturing System. IFACPapersOnLine, 51(11), 752–757. https://doi.org/10.1016/J.IFACOL.2018.08.409 Feng, C.-X. J., Saal, A. L., Salsbury, J. G., Ness, A. R., & Lin, G. C. S. (2007). Design and analysis of experiments in CMM measurement uncertainty study. Precision Engineering, 31(2), 94–101. https://doi.org/10.1016/J.PRECISIONENG.2006.03.00 3 Gohari, H., & Barari, A. (2016). A quick deviation zone fitting in coordinate metrology of NURBS surfaces using principle component analysis. Measurement: Journal of the International Measurement Confederation, 92, 352–364. https://doi.org/10.1016/j.measurement.2016.05.050 Hammad Mian, S., & Al-Ahmari, A. (2014). New developments in coordinate measuring machines for manufacturing industries. International Journal of Metrology and Quality Engineering, 5(1), 101. https://doi.org/10.1051/ijmqe/2014001 Jamiolahmadi, S., & Barari, A. (2014). Surface Topography of Additive Manufacturing Parts Using a Finite Difference Approach. Journal of Manufacturing Science and Engineering, 136(6), 061009. https://doi.org/10.1115/1.4028585
187
Lalehpour, A., Berry, C., & Barari, A. (2017). Adaptive data reduction with neighbourhood search approach in coordinate measurement of planar surfaces. Journal of Manufacturing Systems, 45, 28–47. https://doi.org/10.1016/J.JMSY.2017.07.001 Mahboubkhah, M., & Barari, A. (2019). International Journal of Computer Integrated Manufacturing Design and development of a novel 4-DOF parallel kinematic coordinate measuring machine (CMM) Design and development of a novel 4-DOF parallel kinematic coordinate measuring machine (CMM). https://doi.org/10.1080/0951192X.2019.1610576 Obeidat, S. M., & Raman, S. (2009). An intelligent sampling method for inspecting free-form surfaces. The International Journal of Advanced Manufacturing Technology, 40(11–12), 1125–1136. https://doi.org/10.1007/s00170-008-1427-3 Vrba, I., Palencar, R., Hadzistevic, M., … B. S.-M. science, & 2015, undefined. (n.d.). Different approaches in uncertainty evaluation for measurement of complex surfaces using coordinate measuring machine. Degruyter.Com. Retrieved from https://www.degruyter.com/downloadpdf/j/msr.2015.15 .issue-3/msr-2015-0017/msr-2015-0017.xml Zhao, F., Xu, X., industry, S. X.-C. in, & 2009, undefined. (n.d.). Computer-aided inspection planning—the state of the art. Elsevier. Retrieved from https://www.sciencedirect.com/science/article/pii/S016 6361509000451
187