- Email: [email protected]
AI for design: Virtual design assistant
CIRP Annals - Manufacturing Technology 68 (2019) 141–144
Contents lists available at ScienceDirect
CIRP Annals - Manufacturing Technology jou rnal homep age : ht t p: // ees .e lse vi er . com /ci r p/ def a ult . asp
AI for design: Virtual design assistant Sang-Gook Kim (1)a,*, Sang Min Yoon b, Maria Yang a, Jungwoo Choi b, Haluk Akay a, Edward Burnell a a b
Massachusetts Institute of Technology, Cambridge, MA, USA Kookmin University, Seoul, Republic of Korea
A R T I C L E I N F O
Keywords: Design method Machine learning Hybrid intelligence
A B S T R A C T
Engineering faces many wicked problems: irreducibly interdisciplinary with multiple competing objectives, and of such large scale and complexity that will require processes to deeply rely on human insights and power of computation. The resurgence of machine learning offers the possibility for new forms of human/computer collaboration where each fuels hybrid intelligence in complementary ways. A concept of virtual design assistant (VDA) is developed as a platform to bring the hybrid intelligence in solving complex design challenges. A deep learning-based abstraction process is developed to provide VDA a function to extract structured functional requirements from fragmental design specifications and customer needs. © 2019 Published by Elsevier Ltd on behalf of CIRP.
1. Introduction Recently developed techniques in machine learning and other artificial intelligence (AI) tools have brought breakthroughs from processing of images, speech and audio, and language translation to driving of autonomous cars. Since the 1990s, CIRP members have intensively investigated the use of conventional machining learning for tool monitoring, process inspection and sensor fusion [1–3], conceptual design [4], CAD [5], process planning [6], process monitoring and parameter optimization [7], and production control [8] among others. Conventional machine learning techniques used for the studies of manufacturing and design, however, were limited in their ability to process huge amounts of raw data due to the shallow neural network architecture and much slower computing hardware available decades ago and thereby research interest on them has fizzled out. From 1990 to 2000, 27 papers were published in CIRP Annals Vol. 1 while only 5 were published after 2001. The resurgence of machine learning (i.e. deep learning) and AI is the result of the high-powered Graphic Processing Units readily available at very low cost, deep architecture with massively increased layers and neurons, and massive amounts of data available on line to train the network. As this new technology spreads to reshape the world and society, bringing the power of computing and AI to all fields of study and disciplines, it is urgently necessary to revisit the use of AI (deep learning, narrowly speaking) for the advancement of design and manufacturing technologies. The design of real-world systems (engineering, architecture, software, industrial, financial and social systems) is often a tumultuous endeavor fraught with great triumphs and, at times,
* Corresponding author. E-mail address: [email protected] (S.-G. Kim). https://doi.org/10.1016/j.cirp.2019.03.024 0007-8506/© 2019 Published by Elsevier Ltd on behalf of CIRP.
significant failures. Many believe that only human experts can conceptualize and orchestrate large and complex projects at the upstream of designing systems. There are two challenging issues in the current practice of heuristic systems design. First, it takes tens of years for humans to become area experts through the accumulation of experience in both successes and failures. Secondly, human experts also fail sometimes, especially at the critical time. The questions one might ask at this stage are “How could we teach junior engineers, architects and scientists to design complex systems successfully without spending years of effort on the job training? Could we assist human experts to minimize the probability of failure by leveraging recent developments in AI and big data?” We have looked into the use of AI in the up-stream of design where all the key concepts are determined, and experience-based human insight is necessary. Could machine intelligence help this early stage of designing beyond routine design toward the generation of good and novel designs? Our proposed solution is the use of the hybrid intelligence which is an approach in which human intelligence co-evolves with machine intelligence. In this paper, we describe a deep learning-based design assistant as a first step to hybrid intelligence that combines human intelligence which grows through experience and machine intelligence which can learn from past successes and failures. 2. Hybrid intelligence for systems design Many designers, engineers, and researchers emphasize the importance of early stage system design, calling it “system architecting.” System architecting is different from “system engineering,” which focuses more on modeling-based system analysis such as single variable optimization and multi-variable trade-offs [9]. Great system architects, however, sometimes fail also, especially at critical times. As the scale of a system goes up or
142
S.-G. Kim et al. / CIRP Annals - Manufacturing Technology 68 (2019) 141–144
system heterogeneity increases, complexity can become unmanageable by heuristics alone. Incorrect understanding of a problem in the early stages of design significantly increases the likelihood of system failure which has been mostly served only by the experts’ intuition, heuristics and experiential perceptions. Engineering relies on precise tools, computation, and scientific methods while heuristic approach is a pragmatic guideline from lessons learned from past successes and failures, taken to a level of abstraction. The former alone (without extensive experience) has little success for solving big complex systems (engineering, architecture, software, industrial, financial and social systems); the latter has been the guideline for most practitioners in real world cases. In Hybrid Intelligence, our concern is not with the intelligence of artificial systems, or with constraining human designers, but with the effectiveness of their collaboration. Instead of finding humans replaceable by computational systems such as machine intelligence, we see humans and computers as working together within an ecosystem where each must bring their strengths to bear. Preliminary work in enhancing the tacit knowledge of human has included geometric programming to support co-evolutionary design among team members through a software package GPKit created by one of the authors [10]. A simple machine learningbased design example developed by another author is “SketchHelper” [11] which provides stroke guidance by training a stroke guidance neural network that learns the mapping between the step-wise stroke relations to predict the user’s next stroke. We propose a framework which will become a tool for broad investigations of this likely convergence and co-evolution of man machine intelligence in the field of design and manufacturing.
in the retrospectively generated FR-DP tree structures. For each discipline, we will be able to collect huge amount of data for training of VDA. In 1978, Nam Suh at MIT introduced a principlebased design approach to manufacturing systems [12]. AD has provided a way of design thinking that the ad hoc nature of engineering design could be better structured with the concept of domains and the principles to guide decision making in mapping between the domains [13]. While the merits of using AD theory have been evidenced in academic research papers, we have seen limited use of AD by industrial practitioners. Many experienced designers agree that AD helps them to cultivate insightful thinking. But many still find it difficult to apply AD principles to design practice since using AD effectively also requires designer’s insights and experience in AD. Benefits of machine learning will help designers effectively use AD for making early design decisions. In the AD theory, this principle has been formally described based on the concept of “design domains” and “mapping.” It is the key for designers to learn the what and how relationship between design domains. Functional requirements (FRs) are derived from customers’ needs and multiple solution concepts (by the choice of design parameters, DPs) are conceived in the “functional space,” as shown in Fig. 1. AD defines “design” as a mapping between functional domain and physical domain, which is in the “Functional Space” in Fig.1, while most practitioners perform stepwise iterative design activities along the time-domain, which is in the “Process Space.”
3. Virtual design assistant (VDA): a platform toward hybrid intelligence A concept of “virtual design assistant” (VDA) is developed, like Siri or Google Assistant which uses voice queries and a natural language interface to understand the intent of a user, answer questions and perform proper actions. VDA should offer more than simple answers and actions of the virtual assistant apps, in order to have human designers reach proper design solutions through the collaboration. Expected advices and recommendations include the following. What would a designer do first when facing a new problem to solve? Could they ask machines as they ask the experts when they don’t know where to start? Identifying the functional requirements of a completely new system (or a problem) which has never existed is even more complex and difficult since there are no data or knowledge available from prior cases and trials. Answering these questions addresses the following major challenges and functions that could be provided by VDA: - Finding abstraction process for system architecting in terms of finding and structuring correct functional requirements. - Integrating the abstraction process with a generalization design principle such as Axiomatic Design (AD) to map and decompose to physical parameters of design. This will also train the junior designers to have structured thinking. - Developing a unified framework for systems architecture across disciplines and scenarios. VDA needs to understand designer’s intent written in natural language on the design needs and questions. VDA also needs to have design principles to assess the design decisions at the earliest possible stage and to provide assistive suggestions. And the VDA should be able to learn from past successes and failures though deep learning, providing heuristic expertise collected in each discipline to the designer. The key for a successful machine learning will be how to train VDA with existing design cases and data, and on which principle the DVA recommends/assists the designer. VDA uses a principlebased design framework namely, “an Axiomatic Design (AD),” which provides a set of principles that determine good design practices. VDA can use AD to learn from past successes and failures
Fig. 1. Illustration of the use of Axiomatic Design AD to reverse-map past design successes and failures in terms of functional requirements and design parameters, which will then form hierarchical trees between the functional and physical domain via design.
VDA should be able to oversee both Functional Space and Process Space while most designers tend to think at the Process Space. VDA helps designers check the design matrix at each level of mapping from what to how, following the first design axiom (Independence Axiom), in order not to have their FRs coupled by the chosen DPs. The second design axiom, Information Axiom, concerns the complexity of a design solution, which can be applied when there are two or more non-coupled designs competing against each other (Fig. 1). AD can be also used to reverse-map the past design successes and failures into trees of functional requirements and design parameters as shown in the Functional Space in Figs. 1 and 2. The reverse-mapped, decomposed and hierarchical past design data can be generated in volume and be used to train the reinforced neural network of VDA.
Fig. 2. Machine intelligence assisted FR-DP mapping and decomposition process, which provides rewards to make the design decisions conform the AD principles.
S.-G. Kim et al. / CIRP Annals - Manufacturing Technology 68 (2019) 141–144
143
The VDA will provide interactive advice to a human designer to reach good design decisions at each stages of design:
combined loss function consisting of the reconstruction loss and a clustering assignment hardening loss function.
- Suggest where to begin when a human designer struggles with unstructured, fragmental customer requirements. - Help establish solution neutral functional requirements (FR) from customers’ needs. - Help map FRs to design parameters (DP). Decomposition continues through a top-down, zig-zag process (Fig. 2). - Analyze a design matrix at the early stage of design to verify that the design satisfies the independence axiom and the information axiom. - Suggest revising DPs to avoid coupled designs which violate the axioms.
4.3. Recursive keyword-based cluster decomposition
There are the three core functions of VDA we need to develop to achieve the above: 1) understanding of the user needs and extract functional requirements, 2) providing suggestions on the goodness of design decisions based on AD, 3) accumulating big data from the past design successes and failures. This paper focuses on the first step of VDA which is to understands the user needs and intension and extract functional requirements in the correct AD syntax machine can vectorize them.
4. Extracting functional requirements Abstraction and generalization have played an important role in the minds of scientists to solve complex problems [14]. Abstraction in AI is a technique that reduces the complexity of a problem by filtering the irrelevant properties while preserving all important and necessary details to solve a given problem. Instead of the rulebased technique, a data-driven hierarchical system representation and abstraction framework through deep learning have resulted much improved and successful AI tools for natural language processing, text mining and even novel writing in recent years. Extracting functional requirements from a string of words, oftenfragmented and imperfect user specifications, can be accomplished by recognizing the designer’s intent contextually and abstracting of them by hiding low level details as well as decomposing the subset data using recursive clustering algorithm. For effective key feature abstraction and decomposition, we first applied existing deep neural network-based systematic taxonomy of clustering methods to abstract the text data [15]. Hierarchical clustering algorithm is composed of three steps: (1) preprocessing to remove the meaningless words and convert the words to high-dimensional vectorized features; (2) feature abstraction using deep learningbased clustering; (3) recursive seed keyword-based subgroup decomposition (with the input from the human designer). 4.1. Preprocessing Natural languages are composed of various characters, symbols, and numbers. First, VDA reads the documents and tokenizes them as the collection of words. It filters predefined “stop” words from the tokenized collection of words. To represent the words as unique entities, each word is vectorized using word2vec, which is a two-layered neural network trained to reconstruct the linguistic contexts of words [16]. 4.2. Clustering From a large set of vectorized words (we used 300 dimensions for vectorization), we abstract the feature vectors by utilizing the deep neural network-based systematic taxonomy to cluster the data. The autoencoder-based deep neural network is used to reduce the dimension of the feature vector, and then reconstruct the data from the reduced representation of the original input data. Autoencoder is trained to use standard mean square-based loss function, and the features are voted into a few subgroups by using
To understand the meaning of the subgroups which are divided using deep neural network-based clustering, we use a seed keyword-based subset decomposition algorithm. The seed keywords are provided by users (interactively) and each group has analyzed the relationship between seed words and its words in the subgroup. When k seed keywords are provided by the user, the seed word feature vectors are intentionally represented as much as orthogonal to the rest of the vectors to ensure functional independence. 4.4. Experiment1 We validated the proposed hierarchical clustering algorithm of VDA by showing how to effectively abstract the highly scattered data in text into subgroups (Fig. 3a) and to extract the meaningful features (key words or center words) of each subgroup. We used Boston Airbnb review comments dataset (https://www.kaggle. com/airbnb/boston) which is simply chosen among public natural language datasets available. The proposed algorithm is implemented using Pytorch, an open source deep learning platform, and MATLAB. The dataset was first tokenized as a collection of words and then filtered with the predefined stop. Secondly, we extracted 29,380 words in 300-dimension vector using word2vec. We grouped the words into 15 clusters to separate out reviews with foreign languages. By using the loss function, which is combined of reconstruction loss and clustering loss, the deep neural networkbased clustering provides clear separation with different languages. After removing language reviews, the second run deep learning based clustering shows very distinct subgroups (Fig. 3b).
Fig. 3. Deep neural network-based clustering from scattered feature vectors, a) filtered English words using t-SNE domain, b) deep learning-based clustering result (the color represents the number of subgroup), c) seed word-based clustering using t-SNE domain.
We intentionally added 5 seed words into filtered feature vectors to accelerate the clustering; location, value, communication, host, and cleanness. Then the DL clusters 15,414 words in the filterer data into 5 subgroups as shown in Fig.3c). 81.15% of the words are projected into the “location” keyword, indicating that most Airbnb reviews are very closely related to “location.” 4.5. Experiment2 The same method was applied to a set of unstructured design requirements in manufacturing. From a fully decomposed FR tree of a Chemical Mechanical Polishing machine designed and built at MIT [17], 146 design requirements were collected and put into a bag of words. Doc2vec, an extension of the word2vec for vectorizing phrases rather than words, was used to vectorize each requirement into a 784-dimensional feature vector [18]. Autoencoder-based deep clustering was applied to recursively generate a binary tree structure but resulted in unsuccessful clustering and could not extract higher level FRs either. A key learning from the two experiments is that AI abstraction tools for natural language such as word2vec or doc2vec to vectorize FRs in design is not effective in
144
S.-G. Kim et al. / CIRP Annals - Manufacturing Technology 68 (2019) 141–144
extracting functional requirements from natural language expression of design requirements. This finding suggested a new way of vectorizing FRs in the design domain as described below. 4.6. Function embedding vs. word embedding The existing word embedding tools are for clustering words in documents based on the proximity of word vectors and frequency of them. But to extract a functional requirement from a short string of words cannot be achieved by simple frequency counting of word vectors in proximity. For this challenge, we created a concept of “function embedding” instead of “word embedding.” “Function embedding” is to vectorize a sting of words which describes a function wanted. Once vectorized, functions with similar context occupy close spatial positions in n-dimensional vector space, then the cosine of the angle between such vectors could be used to determine the orthogonality, redundancy and functional coupling of them. Function embedding can be explained with a case below to show how FRs can be extracted, vectorized and used thereafter. Consider a case of a building façade design with an ambiguous designer’s need such as, “I want my building to have easy access.” Firstly, a string of user specified requirements needs to be encoded following a syntax of FRs: “action + object + attributes + hereditary information (from higher level FRs).” Then the designer’s statement can be rephrased such as, “FR1: Provide access to my building.” This encoding can be done by training a shallow network with the FR syntax and many FR examples. Using the Axiomatic Design framework, a corresponding design parameter DP1 can be chosen by the designer or suggested by VDA as “DP1: a kind of door.” From the solution neutral status of design, we are not supposed to rush to choose a type of door (slide, hung etc.) as DP1. FR1 needs to be decomposed further as shown in Fig. 4 (a). Per VDA’s suggestion, a designer can make lower level FRs in free form text, which can also be encoded into FRs.
design principles for good design decisions, like Axiomatic Design (AD), and to provide assistive design suggestions for designers to navigate between functional and physical domains. A key major challenge in developing a platform for hybrid intelligence is identified as the extraction of functional requirements from unstructured and, sometimes, ill-defined user specifications. We found the existing word embedding tools of deep learning do not extract functional requirements directly from the user specifications. A syntax for FRs has been applied to translate (encode) user needs and specifications to a collection of functional requirements (FRs) that can be vectorized and understood by the machine learning tools to create structure and hierarchy. Once confirmed with the design with interactive communication, the vectorized FRs will be able to utilize the rich set of AD principles, theorems and corollaries to assess the designer’s decision and provide adequate advices. Leveraging the Deep Learning’s recent progress, we expect to find a way to transform the art of design to a scientifically solvable problem, by integrating human and machine intelligence effectively. The VDA will have a significant influence on the designing across the wide spectrum of engineering systems including the future advanced manufacturing systems, such as cyber physical production systems (CPPS) where design and system architecture still are based on traditional engineering methods facing increased system complexity [19]. VDA as a deep learning platform and the wealth of manufacturing technology accumulated in the CIRP over the years will enable exemplary cases of human/computer collaboration toward smart and intelligent product design and manufacturing. Acknowledgement This work was supported by SUTD IDC Flagship Project Funding and MIT/SenseTime Alliance on Artificial Intelligence.
References
Fig. 4. FRs of a building entrance design case (a) and t-SNE visualization of building entrance FR vectors (b).
Using doc2vec, each FRs were vectorized, and then t-SNE visualization tool was used to compare the vectors in two dimensions (Fig.4(b)). The parent FR is located at a central position compared to the sub FRs in this example, which shows the positive feasibility that functions can be vectorized with the existing word embedding tools if the FRs are encoded following the specific syntax. Once the functions are encoded into vectors, a metric for the degree to which child FRs are related to their parent can be established. Similarly, child FRs that may not belong in this branch can be identified easily. With further development, the virtual design assistant will apply the hereditary nature of FRs along the same branch in order to structure unstructured and fragmental requirements into a structured one. Also, vectorized FRs can be mathematically analyzed to find whether the FRs are coupled or not and, if coupled, how serious the coupling is. 5. Conclusion A concept of “virtual design assistant” (VDA) is developed to have three core functions to understand designer’s natural language in the syntax of functional requirements, to implement
[1] Dornfeld D, De Vries M (1990) Neural Network Sensor Fusion for Tool Condition Monitoring. CIRP Annals 39(1):101. [2] Noori-Khajavi A, Komanduri R (1993) On Multi-Sensor Approach to Drill Wear Monitoring. CIRP Annals 42(1):71. [3] Kumara S, Ham I (1990) Use of Associative Memory and Self-Organization in Conceptual Design. CIRP Annals 39(1):117. [4] Krause F, Fisher A, Gross N, Barhak J (2003) Reconstruction of Freeform Objects with Arbitrary Topology Using Neural Networks and Subdivision. CIRP Annals 52(1):125. [5] Teti R, Langella A, D’Addona D (1999) An Intelligent Computation Approach to Process Planning in Multiple-Step Cold Forging. CIRP Annals 48(1):175. [6] Monosotori L, Prohaska J (1993) A Step Towards Intelligent Manufacturing: Modelling and Monitoring of Manufacturing Processes Through Artificial Neural Networks. CIRP Annals 42(1):485. [7] Choi GH, Lee KD, Chang N, Kim SG (1994) Optimization of the Process Parameters of Injection Molding with Neural Network. CIRP Annals 43(1):449. [8] Scholz-Reiter B, Muller S, Wiendhal HP (2000) Throughput Time Control in Production Systems Supported by Neural Networks. CIRP Annals 49(1):331. [9] Rechtin E (1991) Systems Architecting: Creating & Building Complex Systems, Prentice Hall, New Jersey. [10] https://gpkit.readthedocs.io/en/latest/. [11] Choi J, Cho H, Song J, Yoon SM (2019) SketchHelper: Real-Time Stroke Guidance for Freehand Sketch Retrieval. IEEE Transactions on Multimedia. http://dx.doi. org/10.1109/TMM.2019.2892301. in press. [12] Suh NP, Kim S, Bell AC, et al (1978) Optimization of Manufacturing Systems through Axiomatics. CIRP Annals 31:383–388. [13] Suh NP (1990) The Principles of Design, Oxford University Press. [14] Dreyfus T (1991) in Tall D, (Ed.) Advanced Mathematical Thinking Processes, Advanced Mathematical Thinking, Springer. [15] Aljalbout E, Golkov V, Siddiqui Y, Cremers D (2018) Clustering with Deep Learning: Taxonomy and New MethodsCoRR, Vol. abs/1801.07648 . [16] Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient Estimation of Word Representations in Vector SpacearVix:1301,3781 . [17] Melvin JW, Suh NP (2002) Axiomatic Design of a Chemical Mechanical Polishing (CMP), MIT Ph.D. Thesis, MIT. [18] Le Q, Mikolov T (2014) Proceedings of the 31st international conference on machine learning. PMLR 32(2):1188–1196. [19] Stark R, Kinda S, Neumeyer S (2017) Innovations in Digital Modelling for Next Generation Manufacturing System Design. CIRP Annals 66(1):169.
Contents lists available at ScienceDirect
CIRP Annals - Manufacturing Technology jou rnal homep age : ht t p: // ees .e lse vi er . com /ci r p/ def a ult . asp
AI for design: Virtual design assistant Sang-Gook Kim (1)a,*, Sang Min Yoon b, Maria Yang a, Jungwoo Choi b, Haluk Akay a, Edward Burnell a a b
Massachusetts Institute of Technology, Cambridge, MA, USA Kookmin University, Seoul, Republic of Korea
A R T I C L E I N F O
Keywords: Design method Machine learning Hybrid intelligence
A B S T R A C T
Engineering faces many wicked problems: irreducibly interdisciplinary with multiple competing objectives, and of such large scale and complexity that will require processes to deeply rely on human insights and power of computation. The resurgence of machine learning offers the possibility for new forms of human/computer collaboration where each fuels hybrid intelligence in complementary ways. A concept of virtual design assistant (VDA) is developed as a platform to bring the hybrid intelligence in solving complex design challenges. A deep learning-based abstraction process is developed to provide VDA a function to extract structured functional requirements from fragmental design specifications and customer needs. © 2019 Published by Elsevier Ltd on behalf of CIRP.
1. Introduction Recently developed techniques in machine learning and other artificial intelligence (AI) tools have brought breakthroughs from processing of images, speech and audio, and language translation to driving of autonomous cars. Since the 1990s, CIRP members have intensively investigated the use of conventional machining learning for tool monitoring, process inspection and sensor fusion [1–3], conceptual design [4], CAD [5], process planning [6], process monitoring and parameter optimization [7], and production control [8] among others. Conventional machine learning techniques used for the studies of manufacturing and design, however, were limited in their ability to process huge amounts of raw data due to the shallow neural network architecture and much slower computing hardware available decades ago and thereby research interest on them has fizzled out. From 1990 to 2000, 27 papers were published in CIRP Annals Vol. 1 while only 5 were published after 2001. The resurgence of machine learning (i.e. deep learning) and AI is the result of the high-powered Graphic Processing Units readily available at very low cost, deep architecture with massively increased layers and neurons, and massive amounts of data available on line to train the network. As this new technology spreads to reshape the world and society, bringing the power of computing and AI to all fields of study and disciplines, it is urgently necessary to revisit the use of AI (deep learning, narrowly speaking) for the advancement of design and manufacturing technologies. The design of real-world systems (engineering, architecture, software, industrial, financial and social systems) is often a tumultuous endeavor fraught with great triumphs and, at times,
* Corresponding author. E-mail address: [email protected] (S.-G. Kim). https://doi.org/10.1016/j.cirp.2019.03.024 0007-8506/© 2019 Published by Elsevier Ltd on behalf of CIRP.
significant failures. Many believe that only human experts can conceptualize and orchestrate large and complex projects at the upstream of designing systems. There are two challenging issues in the current practice of heuristic systems design. First, it takes tens of years for humans to become area experts through the accumulation of experience in both successes and failures. Secondly, human experts also fail sometimes, especially at the critical time. The questions one might ask at this stage are “How could we teach junior engineers, architects and scientists to design complex systems successfully without spending years of effort on the job training? Could we assist human experts to minimize the probability of failure by leveraging recent developments in AI and big data?” We have looked into the use of AI in the up-stream of design where all the key concepts are determined, and experience-based human insight is necessary. Could machine intelligence help this early stage of designing beyond routine design toward the generation of good and novel designs? Our proposed solution is the use of the hybrid intelligence which is an approach in which human intelligence co-evolves with machine intelligence. In this paper, we describe a deep learning-based design assistant as a first step to hybrid intelligence that combines human intelligence which grows through experience and machine intelligence which can learn from past successes and failures. 2. Hybrid intelligence for systems design Many designers, engineers, and researchers emphasize the importance of early stage system design, calling it “system architecting.” System architecting is different from “system engineering,” which focuses more on modeling-based system analysis such as single variable optimization and multi-variable trade-offs [9]. Great system architects, however, sometimes fail also, especially at critical times. As the scale of a system goes up or
142
S.-G. Kim et al. / CIRP Annals - Manufacturing Technology 68 (2019) 141–144
system heterogeneity increases, complexity can become unmanageable by heuristics alone. Incorrect understanding of a problem in the early stages of design significantly increases the likelihood of system failure which has been mostly served only by the experts’ intuition, heuristics and experiential perceptions. Engineering relies on precise tools, computation, and scientific methods while heuristic approach is a pragmatic guideline from lessons learned from past successes and failures, taken to a level of abstraction. The former alone (without extensive experience) has little success for solving big complex systems (engineering, architecture, software, industrial, financial and social systems); the latter has been the guideline for most practitioners in real world cases. In Hybrid Intelligence, our concern is not with the intelligence of artificial systems, or with constraining human designers, but with the effectiveness of their collaboration. Instead of finding humans replaceable by computational systems such as machine intelligence, we see humans and computers as working together within an ecosystem where each must bring their strengths to bear. Preliminary work in enhancing the tacit knowledge of human has included geometric programming to support co-evolutionary design among team members through a software package GPKit created by one of the authors [10]. A simple machine learningbased design example developed by another author is “SketchHelper” [11] which provides stroke guidance by training a stroke guidance neural network that learns the mapping between the step-wise stroke relations to predict the user’s next stroke. We propose a framework which will become a tool for broad investigations of this likely convergence and co-evolution of man machine intelligence in the field of design and manufacturing.
in the retrospectively generated FR-DP tree structures. For each discipline, we will be able to collect huge amount of data for training of VDA. In 1978, Nam Suh at MIT introduced a principlebased design approach to manufacturing systems [12]. AD has provided a way of design thinking that the ad hoc nature of engineering design could be better structured with the concept of domains and the principles to guide decision making in mapping between the domains [13]. While the merits of using AD theory have been evidenced in academic research papers, we have seen limited use of AD by industrial practitioners. Many experienced designers agree that AD helps them to cultivate insightful thinking. But many still find it difficult to apply AD principles to design practice since using AD effectively also requires designer’s insights and experience in AD. Benefits of machine learning will help designers effectively use AD for making early design decisions. In the AD theory, this principle has been formally described based on the concept of “design domains” and “mapping.” It is the key for designers to learn the what and how relationship between design domains. Functional requirements (FRs) are derived from customers’ needs and multiple solution concepts (by the choice of design parameters, DPs) are conceived in the “functional space,” as shown in Fig. 1. AD defines “design” as a mapping between functional domain and physical domain, which is in the “Functional Space” in Fig.1, while most practitioners perform stepwise iterative design activities along the time-domain, which is in the “Process Space.”
3. Virtual design assistant (VDA): a platform toward hybrid intelligence A concept of “virtual design assistant” (VDA) is developed, like Siri or Google Assistant which uses voice queries and a natural language interface to understand the intent of a user, answer questions and perform proper actions. VDA should offer more than simple answers and actions of the virtual assistant apps, in order to have human designers reach proper design solutions through the collaboration. Expected advices and recommendations include the following. What would a designer do first when facing a new problem to solve? Could they ask machines as they ask the experts when they don’t know where to start? Identifying the functional requirements of a completely new system (or a problem) which has never existed is even more complex and difficult since there are no data or knowledge available from prior cases and trials. Answering these questions addresses the following major challenges and functions that could be provided by VDA: - Finding abstraction process for system architecting in terms of finding and structuring correct functional requirements. - Integrating the abstraction process with a generalization design principle such as Axiomatic Design (AD) to map and decompose to physical parameters of design. This will also train the junior designers to have structured thinking. - Developing a unified framework for systems architecture across disciplines and scenarios. VDA needs to understand designer’s intent written in natural language on the design needs and questions. VDA also needs to have design principles to assess the design decisions at the earliest possible stage and to provide assistive suggestions. And the VDA should be able to learn from past successes and failures though deep learning, providing heuristic expertise collected in each discipline to the designer. The key for a successful machine learning will be how to train VDA with existing design cases and data, and on which principle the DVA recommends/assists the designer. VDA uses a principlebased design framework namely, “an Axiomatic Design (AD),” which provides a set of principles that determine good design practices. VDA can use AD to learn from past successes and failures
Fig. 1. Illustration of the use of Axiomatic Design AD to reverse-map past design successes and failures in terms of functional requirements and design parameters, which will then form hierarchical trees between the functional and physical domain via design.
VDA should be able to oversee both Functional Space and Process Space while most designers tend to think at the Process Space. VDA helps designers check the design matrix at each level of mapping from what to how, following the first design axiom (Independence Axiom), in order not to have their FRs coupled by the chosen DPs. The second design axiom, Information Axiom, concerns the complexity of a design solution, which can be applied when there are two or more non-coupled designs competing against each other (Fig. 1). AD can be also used to reverse-map the past design successes and failures into trees of functional requirements and design parameters as shown in the Functional Space in Figs. 1 and 2. The reverse-mapped, decomposed and hierarchical past design data can be generated in volume and be used to train the reinforced neural network of VDA.
Fig. 2. Machine intelligence assisted FR-DP mapping and decomposition process, which provides rewards to make the design decisions conform the AD principles.
S.-G. Kim et al. / CIRP Annals - Manufacturing Technology 68 (2019) 141–144
143
The VDA will provide interactive advice to a human designer to reach good design decisions at each stages of design:
combined loss function consisting of the reconstruction loss and a clustering assignment hardening loss function.
- Suggest where to begin when a human designer struggles with unstructured, fragmental customer requirements. - Help establish solution neutral functional requirements (FR) from customers’ needs. - Help map FRs to design parameters (DP). Decomposition continues through a top-down, zig-zag process (Fig. 2). - Analyze a design matrix at the early stage of design to verify that the design satisfies the independence axiom and the information axiom. - Suggest revising DPs to avoid coupled designs which violate the axioms.
4.3. Recursive keyword-based cluster decomposition
There are the three core functions of VDA we need to develop to achieve the above: 1) understanding of the user needs and extract functional requirements, 2) providing suggestions on the goodness of design decisions based on AD, 3) accumulating big data from the past design successes and failures. This paper focuses on the first step of VDA which is to understands the user needs and intension and extract functional requirements in the correct AD syntax machine can vectorize them.
4. Extracting functional requirements Abstraction and generalization have played an important role in the minds of scientists to solve complex problems [14]. Abstraction in AI is a technique that reduces the complexity of a problem by filtering the irrelevant properties while preserving all important and necessary details to solve a given problem. Instead of the rulebased technique, a data-driven hierarchical system representation and abstraction framework through deep learning have resulted much improved and successful AI tools for natural language processing, text mining and even novel writing in recent years. Extracting functional requirements from a string of words, oftenfragmented and imperfect user specifications, can be accomplished by recognizing the designer’s intent contextually and abstracting of them by hiding low level details as well as decomposing the subset data using recursive clustering algorithm. For effective key feature abstraction and decomposition, we first applied existing deep neural network-based systematic taxonomy of clustering methods to abstract the text data [15]. Hierarchical clustering algorithm is composed of three steps: (1) preprocessing to remove the meaningless words and convert the words to high-dimensional vectorized features; (2) feature abstraction using deep learningbased clustering; (3) recursive seed keyword-based subgroup decomposition (with the input from the human designer). 4.1. Preprocessing Natural languages are composed of various characters, symbols, and numbers. First, VDA reads the documents and tokenizes them as the collection of words. It filters predefined “stop” words from the tokenized collection of words. To represent the words as unique entities, each word is vectorized using word2vec, which is a two-layered neural network trained to reconstruct the linguistic contexts of words [16]. 4.2. Clustering From a large set of vectorized words (we used 300 dimensions for vectorization), we abstract the feature vectors by utilizing the deep neural network-based systematic taxonomy to cluster the data. The autoencoder-based deep neural network is used to reduce the dimension of the feature vector, and then reconstruct the data from the reduced representation of the original input data. Autoencoder is trained to use standard mean square-based loss function, and the features are voted into a few subgroups by using
To understand the meaning of the subgroups which are divided using deep neural network-based clustering, we use a seed keyword-based subset decomposition algorithm. The seed keywords are provided by users (interactively) and each group has analyzed the relationship between seed words and its words in the subgroup. When k seed keywords are provided by the user, the seed word feature vectors are intentionally represented as much as orthogonal to the rest of the vectors to ensure functional independence. 4.4. Experiment1 We validated the proposed hierarchical clustering algorithm of VDA by showing how to effectively abstract the highly scattered data in text into subgroups (Fig. 3a) and to extract the meaningful features (key words or center words) of each subgroup. We used Boston Airbnb review comments dataset (https://www.kaggle. com/airbnb/boston) which is simply chosen among public natural language datasets available. The proposed algorithm is implemented using Pytorch, an open source deep learning platform, and MATLAB. The dataset was first tokenized as a collection of words and then filtered with the predefined stop. Secondly, we extracted 29,380 words in 300-dimension vector using word2vec. We grouped the words into 15 clusters to separate out reviews with foreign languages. By using the loss function, which is combined of reconstruction loss and clustering loss, the deep neural networkbased clustering provides clear separation with different languages. After removing language reviews, the second run deep learning based clustering shows very distinct subgroups (Fig. 3b).
Fig. 3. Deep neural network-based clustering from scattered feature vectors, a) filtered English words using t-SNE domain, b) deep learning-based clustering result (the color represents the number of subgroup), c) seed word-based clustering using t-SNE domain.
We intentionally added 5 seed words into filtered feature vectors to accelerate the clustering; location, value, communication, host, and cleanness. Then the DL clusters 15,414 words in the filterer data into 5 subgroups as shown in Fig.3c). 81.15% of the words are projected into the “location” keyword, indicating that most Airbnb reviews are very closely related to “location.” 4.5. Experiment2 The same method was applied to a set of unstructured design requirements in manufacturing. From a fully decomposed FR tree of a Chemical Mechanical Polishing machine designed and built at MIT [17], 146 design requirements were collected and put into a bag of words. Doc2vec, an extension of the word2vec for vectorizing phrases rather than words, was used to vectorize each requirement into a 784-dimensional feature vector [18]. Autoencoder-based deep clustering was applied to recursively generate a binary tree structure but resulted in unsuccessful clustering and could not extract higher level FRs either. A key learning from the two experiments is that AI abstraction tools for natural language such as word2vec or doc2vec to vectorize FRs in design is not effective in
144
S.-G. Kim et al. / CIRP Annals - Manufacturing Technology 68 (2019) 141–144
extracting functional requirements from natural language expression of design requirements. This finding suggested a new way of vectorizing FRs in the design domain as described below. 4.6. Function embedding vs. word embedding The existing word embedding tools are for clustering words in documents based on the proximity of word vectors and frequency of them. But to extract a functional requirement from a short string of words cannot be achieved by simple frequency counting of word vectors in proximity. For this challenge, we created a concept of “function embedding” instead of “word embedding.” “Function embedding” is to vectorize a sting of words which describes a function wanted. Once vectorized, functions with similar context occupy close spatial positions in n-dimensional vector space, then the cosine of the angle between such vectors could be used to determine the orthogonality, redundancy and functional coupling of them. Function embedding can be explained with a case below to show how FRs can be extracted, vectorized and used thereafter. Consider a case of a building façade design with an ambiguous designer’s need such as, “I want my building to have easy access.” Firstly, a string of user specified requirements needs to be encoded following a syntax of FRs: “action + object + attributes + hereditary information (from higher level FRs).” Then the designer’s statement can be rephrased such as, “FR1: Provide access to my building.” This encoding can be done by training a shallow network with the FR syntax and many FR examples. Using the Axiomatic Design framework, a corresponding design parameter DP1 can be chosen by the designer or suggested by VDA as “DP1: a kind of door.” From the solution neutral status of design, we are not supposed to rush to choose a type of door (slide, hung etc.) as DP1. FR1 needs to be decomposed further as shown in Fig. 4 (a). Per VDA’s suggestion, a designer can make lower level FRs in free form text, which can also be encoded into FRs.
design principles for good design decisions, like Axiomatic Design (AD), and to provide assistive design suggestions for designers to navigate between functional and physical domains. A key major challenge in developing a platform for hybrid intelligence is identified as the extraction of functional requirements from unstructured and, sometimes, ill-defined user specifications. We found the existing word embedding tools of deep learning do not extract functional requirements directly from the user specifications. A syntax for FRs has been applied to translate (encode) user needs and specifications to a collection of functional requirements (FRs) that can be vectorized and understood by the machine learning tools to create structure and hierarchy. Once confirmed with the design with interactive communication, the vectorized FRs will be able to utilize the rich set of AD principles, theorems and corollaries to assess the designer’s decision and provide adequate advices. Leveraging the Deep Learning’s recent progress, we expect to find a way to transform the art of design to a scientifically solvable problem, by integrating human and machine intelligence effectively. The VDA will have a significant influence on the designing across the wide spectrum of engineering systems including the future advanced manufacturing systems, such as cyber physical production systems (CPPS) where design and system architecture still are based on traditional engineering methods facing increased system complexity [19]. VDA as a deep learning platform and the wealth of manufacturing technology accumulated in the CIRP over the years will enable exemplary cases of human/computer collaboration toward smart and intelligent product design and manufacturing. Acknowledgement This work was supported by SUTD IDC Flagship Project Funding and MIT/SenseTime Alliance on Artificial Intelligence.
References
Fig. 4. FRs of a building entrance design case (a) and t-SNE visualization of building entrance FR vectors (b).
Using doc2vec, each FRs were vectorized, and then t-SNE visualization tool was used to compare the vectors in two dimensions (Fig.4(b)). The parent FR is located at a central position compared to the sub FRs in this example, which shows the positive feasibility that functions can be vectorized with the existing word embedding tools if the FRs are encoded following the specific syntax. Once the functions are encoded into vectors, a metric for the degree to which child FRs are related to their parent can be established. Similarly, child FRs that may not belong in this branch can be identified easily. With further development, the virtual design assistant will apply the hereditary nature of FRs along the same branch in order to structure unstructured and fragmental requirements into a structured one. Also, vectorized FRs can be mathematically analyzed to find whether the FRs are coupled or not and, if coupled, how serious the coupling is. 5. Conclusion A concept of “virtual design assistant” (VDA) is developed to have three core functions to understand designer’s natural language in the syntax of functional requirements, to implement
[1] Dornfeld D, De Vries M (1990) Neural Network Sensor Fusion for Tool Condition Monitoring. CIRP Annals 39(1):101. [2] Noori-Khajavi A, Komanduri R (1993) On Multi-Sensor Approach to Drill Wear Monitoring. CIRP Annals 42(1):71. [3] Kumara S, Ham I (1990) Use of Associative Memory and Self-Organization in Conceptual Design. CIRP Annals 39(1):117. [4] Krause F, Fisher A, Gross N, Barhak J (2003) Reconstruction of Freeform Objects with Arbitrary Topology Using Neural Networks and Subdivision. CIRP Annals 52(1):125. [5] Teti R, Langella A, D’Addona D (1999) An Intelligent Computation Approach to Process Planning in Multiple-Step Cold Forging. CIRP Annals 48(1):175. [6] Monosotori L, Prohaska J (1993) A Step Towards Intelligent Manufacturing: Modelling and Monitoring of Manufacturing Processes Through Artificial Neural Networks. CIRP Annals 42(1):485. [7] Choi GH, Lee KD, Chang N, Kim SG (1994) Optimization of the Process Parameters of Injection Molding with Neural Network. CIRP Annals 43(1):449. [8] Scholz-Reiter B, Muller S, Wiendhal HP (2000) Throughput Time Control in Production Systems Supported by Neural Networks. CIRP Annals 49(1):331. [9] Rechtin E (1991) Systems Architecting: Creating & Building Complex Systems, Prentice Hall, New Jersey. [10] https://gpkit.readthedocs.io/en/latest/. [11] Choi J, Cho H, Song J, Yoon SM (2019) SketchHelper: Real-Time Stroke Guidance for Freehand Sketch Retrieval. IEEE Transactions on Multimedia. http://dx.doi. org/10.1109/TMM.2019.2892301. in press. [12] Suh NP, Kim S, Bell AC, et al (1978) Optimization of Manufacturing Systems through Axiomatics. CIRP Annals 31:383–388. [13] Suh NP (1990) The Principles of Design, Oxford University Press. [14] Dreyfus T (1991) in Tall D, (Ed.) Advanced Mathematical Thinking Processes, Advanced Mathematical Thinking, Springer. [15] Aljalbout E, Golkov V, Siddiqui Y, Cremers D (2018) Clustering with Deep Learning: Taxonomy and New MethodsCoRR, Vol. abs/1801.07648 . [16] Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient Estimation of Word Representations in Vector SpacearVix:1301,3781 . [17] Melvin JW, Suh NP (2002) Axiomatic Design of a Chemical Mechanical Polishing (CMP), MIT Ph.D. Thesis, MIT. [18] Le Q, Mikolov T (2014) Proceedings of the 31st international conference on machine learning. PMLR 32(2):1188–1196. [19] Stark R, Kinda S, Neumeyer S (2017) Innovations in Digital Modelling for Next Generation Manufacturing System Design. CIRP Annals 66(1):169.