- Email: [email protected]
End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine
ARTICLE IN PRESS
JID: COMM
[m5G;October 27, 2018;11:10]
Computer Methods and Programs in Biomedicine xxx (xxxx) xxx
Contents lists available at ScienceDirect
Computer Methods and Programs in Biomedicine journal homepage: www.elsevier.com/locate/cmpb
End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine Qinan Hu a,b, Tong Yu c,∗, Jinghua Li c, Qi Yu c, Ling Zhu c, Yueguo Gu a,b a
Institute of Linguistics, Chinese Academy of Social Sciences, Beijing 100732, China China Multilingual and Multimodal Corpora and Big Data Research Centre, Beijing 100089, China c Institute of Information on Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China b
a r t i c l e
i n f o
Article history: Received 1 July 2017 Revised 22 September 2018 Accepted 11 October 2018 Available online xxx Keywords: Traditional Chinese medicine Syndrome differentiation Yin deficiency Yang deficiency Text classification End-to-end Convolutional neural networks FastText
a b s t r a c t Background and Objective. Yin and Yang, two concepts adapted from classical Chinese philosophy, play a diagnostic role in Traditional Chinese Medicine (TCM). The Yin and Yang in harmonious balance indicate health, whereas imbalances to either side indicate unhealthiness, which may result in diseases. Yin-yang disharmony is considered to be the cause of pathological changes. Syndrome differentiation of yin-yang is crucial to clinical diagnosis. It lays a foundation for subsequent medical judgments, including therapeutic methods, and formula, among many others. However, because of the complexities of the mechanisms and manifestations of disease, it is difficult to exactly point out which one, yin or yang, is disharmonious. There has been inadequate research conducted on syndrome differentiation of yin and yang from a computational perspective. In this study, we present a computational method, viz. an end-to-end syndrome differentiation of yin deficiency and yang deficiency. Methods. Unlike most previous studies on syndrome differentiation, which use structured datasets, this study takes unstructured texts in medical records as its inputs. It models syndrome differentiation as a task of text classification. This study experiments on two state-of-the-art end-to-end algorithms for text classification, i.e. a classic convolutional neural network (CNN) and fastText. These two systems take the n-grams of several types of tokens as their inputs, including characters, terms, and words. Results. When evaluated on a data set with 7326 modern medical records in TCM, it is observed that CNN and fastText generally give rise to comparable performances. The best accuracy rate of 92.55% comes from the system taking inputs as raw as n-grams of characters. It implies that one can build at least a moderate system for the differentiation of yin deficiency and yang deficiency even if he has no glossary or tokenizer at hand. Conclusions. This study has demonstrated the feasibility of using end-to-end text classification algorithms to differentiate yin deficiency and yang deficiency on unstructured medical records. © 2018 Elsevier B.V. All rights reserved.
1. Introduction Yin and yang are the general terms describing two opposite, yet complementary and inter-related forces found in all matter in nature [1]. In the eight principle syndrome differentiation, yin and yang are taken as the fundamental ones, wherein external, excess
∗
Corresponding author. E-mail addresses: [email protected] (Q. Hu), [email protected] (T. Yu), [email protected] (J. Li), [email protected] (Q. Yu), [email protected] (L. Zhu), [email protected] (Y. Gu).
and heat syndromes are indicators of yang deficiencies, while internal, deficiency and cold syndromes reveal yin deficiencies. Yin-yang disharmony is considered to be the cause of pathological changes. Yin/yang-based syndrome differentiation lays a foundation for subsequent medical judgements, such as therapeutic methods and formulas. It is claimed in Yellow Emperor’s Inner Canon that ’when observing complexion and palpating the pulse, those who are skilled at diagnosis first ascertain Yin and Yang.’ () Jingyue Zhang stated in his Classified Canon - Yin-Yang Classes that no matter how complicated the manifestation of a disease
https://doi.org/10.1016/j.cmpb.2018.10.011 0169-2607/© 2018 Elsevier B.V. All rights reserved.
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM 2
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
seems to be, it originates from either yin or yang disharmony in essence. ,……,,,, ( .) However, because of the complexities of the mechanisms and manifestations of diseases, it’s not easy to exactly point out which one, yin or yang, is disharmonious. As indicated by Qin’ an Zheng, a famous physician in Qing dynasty, while it’s more difficult to differentiate syndromes than learning pharmacy, it’s even more difficult to differentiate yin and yang than differentiating syndromes. , , , () Although syndrome differentiation of yin-yang has been investigated intensively and extensively in Traditional Chinese Medicine (TCM), only a few studies have been conducted from a computational perspective.
2. Background Machine learning and data mining technologies have been widely used in syndrome differentiation in modern research of TCM, giving rise to a large volume of literatures and reviews [2– 5]. Jiang et al. presented an overall profile for syndrome-related studies in both Chinese and English literatures, and generated semantic networks based on the frequencies of co-occurrences in the literatures [6]. Song et al. investigated the differentiation of 7 syndromes in depression from 364 patients [7]. It’s reported Support Vector Machines (SVM) achieves the highest accuracies of more than 90% except for yin deficiency. Zhao et al. proposed to use manifold ranking (MR) to explore the relation between syndromes for viral hepatitis [8]. Yao et al. used a supervised topic model and TCM domain ontology to discover treatment patterns using 3090 medical records [9]. Zhao conducted an extensive survey of more than one hundred papers on patient classification from a machine learning perspective [5]. To classify patients, these studies employ Naive Bayes, decision trees, SVM, graphical models and many other machine learning algorithms. Nowadays, deep learning and related neural networks have become the mainstream technologies in processing speeches, images, and texts. There have been several studies on syndrome differentiation in TCM using deep learning techniques as well. Liu et al. employed multi-label learning based on Deep Belief Network to construct the syndrome diagnostic model for chronic gastritis in TCM [10]. They used structured data to represent whether one of eight syndromes are present in a patient, such as damp heat accumulating in the spleen-stomach, and dampness obstructing the spleen-stomach, to predict chronic gastritis. Zhu et al. adopted a deep learning approach to damp-heat syndrome differentiation using medical records represented in terms of word vectors and TFIDF [11]. Yao et al. [12] studied medical record classification using document embedding technologies enriched with domain knowledge [13,14]. It has become a typical practice to use structured data in patient and case classifications. However, great potentials lie in the unstructured texts in the medical records in literature and from physicians as well [15,16]. Textual medical records in literature play an important role in the inheritance of medical knowledge from generation to generation. In addition, physicians in TCM used to record symptoms, the degrees of symptoms, syndromes, and many other valuable information using texts, based on their four diagnostic examinations of inspection, auscultation and olfaction, interrogation, and palpation.
Fig. 1. Comparison of end-to-end learning and traditional systems.
3. Syndrome differentiation as an end-to-end text classification task We model syndrome differentiation of yin-yang as a text classification task, which takes the texts in medical records as its inputs. Text classification is a widely used task in Natural Language Processing (NLP), and has been applied to the classification of news [17], spam filtering [18], and sentiment analysis [19], among many others. 3.1. End-to-End framework We adopt an end-to-end approach to text classification using two state-of-the-art neural network models, Convolutional Neural Network (CNN), and fastText. The idea of end-to-end is that the system takes in raw inputs, learn the features by itself, and then map the features to outputs. A comparison of end-to-end learning and traditional systems is shown in Fig. 1. The operations in gray boxes are performed automatically, while those in white boxes require human experts. As no feature engineering is involved, human efforts are reduced in end-to-end learning. As it turns out, end-toend systems sometimes outperform traditional systems [20], since their parameters are learned jointly throughout the entire learning pipeline. 3.2. Convolutional Neural Network Convolutional neural networks have been adopted to classify wrist pulse signals [21,22] and tongue colors [23] in TCM. In our study, the CNN model proposed in [24] is used to classify syndromes in textual TCM medical records. Its architecture is shown in Fig. 2. It includes three layers. In the input layer, the input words in a medical record are embedded into low-dimensional vectors. In the convolutional layer, the embedded word vectors are convoluted using multiple filter sizes, sliding over n-grams at a time. The results from the convolutional layer are then fed to a max-pooling layer to capture the most salient features. Finally, the classification is made using softmax in the fully-connected layer. 3.3. FastText We also use fastText (https://github.com/facebookresearch/ fastText) [25] to differentiate yin-yang deficiencies. Its architecture is shown in Fig. 3. In the input layer, the words in a medical record are represented as low-dimensional vectors. The word vectors are averaged in the hidden layer, and then go through a softmax classifier in the output layer. It is reported that, while dramatically cutting the time of training from several days to a few seconds, fast-
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
3
Fig. 2. Architecture of CNN for syndrome differentiation of yin-yang.
Fig. 3. Architecture of fastText for syndrome differentiation of yin-yang.
Text is often on par with deep neural networks in terms of classification accuracy. The experiments are conducted using a computer equipped with Nvidia Titan X Pascal GPU.
4. Data set We have collected more than 10 0,0 0 0 modern medical records from about 40,0 0 0 journal articles in 353 journals on TCM. Each medical record includes sex, age, and medical conditions of a patient, and diagnostic impressions from doctors. The impressions include syndromes, diseases, therapeutic principles and methods, treatments, formulas, and so on. In our systems, sex, age, and medical conditions of patients are used as the inputs, since these are the information used by doctors before making diagnostic decisions in clinical practice. Since the impressions are generally inferred on the basis of syndromes, they are excluded from the inputs. The labels of yin deficiency and yang deficiency are explicitly designated by the authors of the journal articles. For instance, the medical record in Example 1 is used to demonstrate the ”pattern of yin deficiency with effulgent fire” in a journal article [26], indicating that this medical record is a case of ”yin deficiency”. We extract this explicit indicative information from the journal articles.
In our data set, 7326 medical records are labeled as yin deficiency or yang deficiency, accounting for 6.88% of all medical records. 3720 (51%) records are yin deficiency, and 3606 (49%) records are yang deficiency. The numbers of yin and yang deficiencies are in balance in general. Example 1: Journal article snippet: “1.2.3 , , , , , , , ... 6 ,,52,2005-04-04..., ,,,,,, , ,, ...,... :12 g,...” (1.2.3 Pattern of yin deficiency with effulgent fire Palpitation, dryness-heat and sweating, reddened complexion, yellow urine, dry stool, dry mouth, drinking a lot, dry lips, red tongue, rapid pulses. The appropriate treatments are enriching yin and purging fire... Example 6: Ms Li, female, 52-year old, worker. 1st visit on 2005-04-04. ... paroxysmal palpitation, feeling hot, easy to sweat, dry mouth, drinking a lot, difficult to open eyes, preferring cold food, short breath when talking, poor sleep, slightly dry stool, yellow urine, irregular menstruation, ... red tongue with a yellow
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM 4
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
Fig. 4. Accuracies of fastText w/ characters, terms, and words as inputs.
cover, rapid pulses, ..., Dangguiliuhuang decoction addition and subtraction, prescriptions: prunella 12g, ...) Input: 52 ,,,,, ,,,, ,, (female 52 paroxysmal palpitation, feeling hot, easy to sweat, dry mouth, drinking a lot, difficult to open eyes, preferring cold food, short breath when talking, poor sleep, slightly dry stool, yellow urine, irregular menstruation, red tongue with a yellow cover, rapid pulses) Output: (yin deficiency) It’s worthy to note that the differentiation of syndromes involving both yin and yang deficiencies, such as (dual deficiency in yin and yang), and (detriment to yang affecting yin), are excluded from the current study, due to a lack of data of such kinds of syndromes in our corpus. 5. Results We have conducted two sets of experiments on syndrome differentiation of yin deficiency and yang deficiency. To avoid overfitting, all experimental results are obtained using 10-fold cross validation. The data set is randomly split into 10 subsets of equal size. In each of the 10 folds, a model is trained on 9 subsets, and tested on the remaining one subset. The performance is averaged over the 10 folds. Specifically, in our study, the entire data set of 7326 medical records is randomly split into 10 subsets, each of which includes 733 medical records. In each fold, a model is trained on 6593 medical records and tested on the remaining 733 records. 5.1. End-to-End learning Applying the end-to-end model, vanilla settings of fastText are used. We first investigate three types of raw inputs to the system, i.e. characters, terms and words. When using medical terms as the inputs, the medical conditions are filtered using a glossary having more than 170 millions of terms in Traditional Chinese Medicine Language System (TCMLS) [26–28]. The idea
Table 1 Precisions, recalls and F1 scores of fastText.
Yin deficiency Yang deficiency
Precision
Recall
F1
0.9243 0.9174
0.9203 0.9223
0.9222 0.9197
is to remove the stop words and the words unrelated to the medical domain from the medical records. The words are obtained using the tokenizer from Language Technology Platform (https://github.com/HIT-SCIR/ltp) [29] equipped with the glossary from TCMLS. Fig. 4 shows the performances of faxtText using characters, terms and words as its inputs, with varying sizes of corpora. It is obvious that faxtText using characters and words as inputs significantly outperform that using terms. The poor performances of term-based systems may arise from the low recall of medical terms. The terms remained for example 1 are shown below; other terms in the medical conditions are out of the glossary and have been discarded from the inputs. Example 1 (term) Input: 52 (female 52 palpitation, paroxysm, shortness of breath, poor sleep, stool, urine, menstruation, red tongue) Surprisingly, as far as the task of syndrome differentiation of yin deficiency and yang deficiency is concerned, character-based systems generally have an accuracy very close to the word-based systems, which depend heavily on domain and linguistic knowledge. The differences between them are no larger than 1%. Furthermore, since fastText algorithm doesn’t take the positions of tokens into consideration, we try to use n-grams to capture local dependencies among tokens to improve its performance. We observe a significant improvement when extending from 1-grams to 3-grams in all of the three types of tokens, as shown in Figs. 5, 6, and 7. While accuracies depict overall performances of these models, we further look into the class-specific evaluation metrics for details, as shown in Table 1. fastText models using 5-grams present a balanced performance on the differentiation of yin deficiency and
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
ARTICLE IN PRESS
JID: COMM
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
5
Fig. 5. Accuracies of fastText w/ n-gram characters as inputs.
Fig. 6. Accuracies of fastText w/ n-gram terms as inputs. Table 2 Hyper-parameters used in CNN and fastText. Algorithm
Embedding dimension
Drop-out
Batch size
Epochs
Other
CNN fastText
128 100
0.5 0.5
64 64
100 100
filter sizes: 3, 4, 5 5-grams
yang deficiency, as the precisions, recalls, and F1 scores of these two classes are very close. 5.2. CNN vs. fastText In our experiments on a comparison between CNN and fastText, we generally use the vanilla settings of the hyper-parameters in these models, as shown in Table 2. The embeddings of tokens are
randomly initialized. The numbers of epochs in both models are set to be 100. To capture local dependencies among tokens, the sizes of convolutional filters in CNN are set to 3, 4, and 5; and 5grams of tokens are fed to fastText as its inputs. As shown in Fig. 8, the accuracies of syndrome differentiation of yin deficiency and yang deficiency using CNN and fastText are 92.55% and 92.11%, respectively. The best accuracy comes from the CNN system using 5-grams of characters as its inputs. It implies
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM 6
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
Fig. 7. Accuracies of fastText w/ n-gram words as inputs.
Fig. 8. Accuracies of CNN and fastText.
that one can at least get a moderate system even if he has no glossary or tokenizer at hand. The differences between the performances of CNN and fastText systems are insignificant. An analysis on the bad cases discovers that the classifiers are likely to make errors for the medical records in return visits, because the medical conditions in subsequent visits are not as complete as the first one, and the descriptions of the changes may lead to an opposite prediction, as illustrated in the 2nd visit in Example 2. Example 2: 1st visit: Input: 42 , , , ,,,
(male 42 lassitude of spirit, lack of strength, bright pale complexion, edema all over the body, pit remaining after pressing the edema, pale and enlarged tongue with thin white fur, sunken fine soggy pulse) Output: (yang deficiency) 2nd visit: Input: 42 ,,, (male 42 edema gone, tongue becoming pale red with thin fur, sunken fine pulse) Output: (yang deficiency) 6. Future plans The current research is our first attempt at syndrome differentiation using deep learning technologies. It would be an interesting
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
topic to explore whether the simple character-based text classification models are applicable to more complicated syndrome differentiation tasks, such as the differentiation of qi deficiency and blood deficiency, the deficiencies in viscera and bowels, and the determination of therapeutic methods. 7. Conclusions This study has demonstrated the feasibility of using text classification technologies on syndrome differentiation of yin deficiency and yang deficiency using unstructured medical records. Both CNN and fastText models used in this study are open source, and the application of end-to-end paradigm requires minimal efforts on preprocessing. Nor are efforts required on feature engineering, making it easy to be conducted by physicians and other medical practitioners with limited computer skills for their own studies. Acknowledgment This work is supported by “the Fundamental Research Funds for the Central public welfare research institutes (NO. ZZ070804)” References [1] World Health Organization and others, WHO International Standard Terminologies on Traditional Medicine in the Western Pacific Region, Manila: WHO Regional Office for the Western Pacific, 2007. [2] Y. Huang, Y. Gao, B. Ma, Review of data mining methods frequently used in tcm syndrome study [j], Acta Chi. Medicine Pharmacol. 3 (2010) 004. [3] M. Jiang, C. Lu, C. Zhang, J. Yang, Y. Tan, A. Lu, K. Chan, Syndrome differentiation in modern research of traditional chinese medicine, J. Ethnopharmacol. 140 (3) (2012) 634–642. [4] J. Poon, S.K. Poon, Data analytics for traditional Chinese medicine research, Springer, 2014. [5] C. Zhao, G.-Z. Li, C. Wang, J. Niu, Advances in patient classification for traditional Chinese medicine: a machine learning perspective, Evidence-Based Complemen. Altern. Med. 2015 (2015). [6] M. Jiang, C. Zhang, G. Zheng, H. Guo, L. Li, J. Yang, C. Lu, W. Jia, A. Lu, Traditional chinese medicine zheng in the era of evidence-based medicine: a literature analysis, Evidence-Based Complemen. Altern. Med. 2012 (2012). [7] J. Song, X. Liu, Q. Deng, W. Dai, Y. Gao, L. Chen, Y. Zhang, J. Wang, M. Yu, P. Lu, et al., A network-based approach to investigate the pattern of syndrome in depression, Evidence-Based Complemen. Altern. Med. 2015 (2015). [8] Y.-f. Zhao, L.-y. He, B.-y. Liu, J. Li, F.-y. Li, R.-l. Huo, X.-h. Jing, Syndrome classification based on manifold ranking for viral hepatitis, Chinese J. Integrative Med. 20 (5) (2014) 394–399. [9] L. Yao, Y. Zhang, B. Wei, W. Wang, Y. Zhang, X. Ren, Y. Bian, Discovering treatment pattern in traditional chinese medicine clinical cases by exploiting supervised topic model and domain knowledge, J. Biomed. Inform. 58 (2015) 260–267.
7
[10] G.-P. Liu, J.-J. Yan, Y.-Q. Wang, W. Zheng, T. Zhong, X. Lu, P. Qian, Deep learning based syndrome diagnosis of chronic gastritis, Comput. Math. Methods Med. 2014 (2014). [11] W. Zhu, W. Zhang, G.-Z. Li, C. He, L. Zhang, et al., A study of damp-heat syndrome classification using word2vec and tf-idf, in: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE, 2016, pp. 1415–1420. [12] L. Yao, Y. Zhang, B. Wei, Z. Li, X. Huang, Traditional Chinese medicine clinical records classification using knowledge-powered document embedding, in: Bioinformatics and Biomedicine (BIBM), 2016 IEEE International Conference on, IEEE, 2016, pp. 1926–1928. [13] X. Zhou, Y. Peng, B. Liu, Text mining for traditional chinese medical knowledge discovery: a survey, J. Biomed. Inform. 43 (4) (2010) 650–660. [14] X. Zhou, B. Liu, X. Zhang, Q. Xie, R. Zhang, Y. Wang, Y. Peng, Data mining in real-world traditional Chinese medicine clinical data warehouse, in: Data Analytics for Traditional Chinese Medicine Research, Springer, 2014, pp. 189–213. [15] Y. Wang, Z. Yu, Y. Jiang, Y. Liu, L. Chen, Y. Liu, A framework and its empirical study of automatic diagnosis of traditional chinese medicine utilizing raw free-text clinical records, J. Biomed. Inform. 45 (2) (2012) 210–223. [16] Y. Wang, Z. Yu, L. Chen, Y. Chen, Y. Liu, X. Hu, Y. Jiang, Supervised methods for symptom name recognition in free-text clinical records of traditional chinese medicine: an empirical study, J. Biomed. Inform. 47 (2014) 91–104. [17] X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, in: Advances in neural information processing systems, 2015, pp. 649–657. [18] M. Sahami, S. Dumais, D. Heckerman, E. Horvitz, A bayesian approach to filtering junk e-mail, in: Learning for Text Categorization: Papers from the 1998 workshop, 62, Madison, Wisconsin, 1998, pp. 98–105. [19] B. Pang, L. Lee, S. Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, in: Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, Association for Computational Linguistics, 2002, pp. 79–86. [20] S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, D. Quillen, Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection, Int. J. Rob. Res. 37 (4–5) (2018) 421–436. [21] X. Hu, H. Zhu, J. Xu, D. Xu, J. Dong, Wrist pulse signals analysis based on deep convolutional neural networks, in: Computational Intelligence in Bioinformatics and Computational Biology, 2014 IEEE Conference on, IEEE, 2014, pp. 1–7. [22] S.-R. Zhang, Q.-F. Sun, Human pulse recognition based on convolutional neural networks, in: Computer, Consumer and Control (IS3C), 2016 International Symposium on, IEEE, 2016, pp. 366–369. [23] J. Hou, H.-Y. Su, B. Yan, H. Zheng, Z.-L. Sun, X.-C. Cai, Classification of tongue color based on cnn, in: Big Data Analysis (ICBDA), 2017 IEEE 2nd International Conference on, IEEE, 2017, pp. 725–729. [24] Y. Kim, Convolutional neural networks for sentence classification, arXiv preprint arXiv:1408.5882 (2014). [25] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, Bag of tricks for efficient text classification, arXiv preprint arXiv:1607.01759 (2016). [26] O. Bodenreider, The unified medical language system (umls): integrating biomedical terminology, Nucleic Acids Res. 32 (suppl_1) (2004) D267–D270. [27] Semantic Types in Unified Medical Language System (UMLS), https://www. nlm.nih.gov/research/umls/new_users/online_learning/SEM_003.html. [28] Semantic Relationships in Unified Medical Language System (UMLS), https:// www.nlm.nih.gov/research/umls/new_users/online_learning/SEM_004.html. [29] W. Che, Z. Li, T. Liu, LTP: A chinese language technology platform, in: Proceedings of the 23rd International Conference on Computational Linguistics: Demonstrations, Association for Computational Linguistics, 2010, pp. 13–16.
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM
[m5G;October 27, 2018;11:10]
Computer Methods and Programs in Biomedicine xxx (xxxx) xxx
Contents lists available at ScienceDirect
Computer Methods and Programs in Biomedicine journal homepage: www.elsevier.com/locate/cmpb
End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine Qinan Hu a,b, Tong Yu c,∗, Jinghua Li c, Qi Yu c, Ling Zhu c, Yueguo Gu a,b a
Institute of Linguistics, Chinese Academy of Social Sciences, Beijing 100732, China China Multilingual and Multimodal Corpora and Big Data Research Centre, Beijing 100089, China c Institute of Information on Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China b
a r t i c l e
i n f o
Article history: Received 1 July 2017 Revised 22 September 2018 Accepted 11 October 2018 Available online xxx Keywords: Traditional Chinese medicine Syndrome differentiation Yin deficiency Yang deficiency Text classification End-to-end Convolutional neural networks FastText
a b s t r a c t Background and Objective. Yin and Yang, two concepts adapted from classical Chinese philosophy, play a diagnostic role in Traditional Chinese Medicine (TCM). The Yin and Yang in harmonious balance indicate health, whereas imbalances to either side indicate unhealthiness, which may result in diseases. Yin-yang disharmony is considered to be the cause of pathological changes. Syndrome differentiation of yin-yang is crucial to clinical diagnosis. It lays a foundation for subsequent medical judgments, including therapeutic methods, and formula, among many others. However, because of the complexities of the mechanisms and manifestations of disease, it is difficult to exactly point out which one, yin or yang, is disharmonious. There has been inadequate research conducted on syndrome differentiation of yin and yang from a computational perspective. In this study, we present a computational method, viz. an end-to-end syndrome differentiation of yin deficiency and yang deficiency. Methods. Unlike most previous studies on syndrome differentiation, which use structured datasets, this study takes unstructured texts in medical records as its inputs. It models syndrome differentiation as a task of text classification. This study experiments on two state-of-the-art end-to-end algorithms for text classification, i.e. a classic convolutional neural network (CNN) and fastText. These two systems take the n-grams of several types of tokens as their inputs, including characters, terms, and words. Results. When evaluated on a data set with 7326 modern medical records in TCM, it is observed that CNN and fastText generally give rise to comparable performances. The best accuracy rate of 92.55% comes from the system taking inputs as raw as n-grams of characters. It implies that one can build at least a moderate system for the differentiation of yin deficiency and yang deficiency even if he has no glossary or tokenizer at hand. Conclusions. This study has demonstrated the feasibility of using end-to-end text classification algorithms to differentiate yin deficiency and yang deficiency on unstructured medical records. © 2018 Elsevier B.V. All rights reserved.
1. Introduction Yin and yang are the general terms describing two opposite, yet complementary and inter-related forces found in all matter in nature [1]. In the eight principle syndrome differentiation, yin and yang are taken as the fundamental ones, wherein external, excess
∗
Corresponding author. E-mail addresses: [email protected] (Q. Hu), [email protected] (T. Yu), [email protected] (J. Li), [email protected] (Q. Yu), [email protected] (L. Zhu), [email protected] (Y. Gu).
and heat syndromes are indicators of yang deficiencies, while internal, deficiency and cold syndromes reveal yin deficiencies. Yin-yang disharmony is considered to be the cause of pathological changes. Yin/yang-based syndrome differentiation lays a foundation for subsequent medical judgements, such as therapeutic methods and formulas. It is claimed in Yellow Emperor’s Inner Canon that ’when observing complexion and palpating the pulse, those who are skilled at diagnosis first ascertain Yin and Yang.’ () Jingyue Zhang stated in his Classified Canon - Yin-Yang Classes that no matter how complicated the manifestation of a disease
https://doi.org/10.1016/j.cmpb.2018.10.011 0169-2607/© 2018 Elsevier B.V. All rights reserved.
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM 2
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
seems to be, it originates from either yin or yang disharmony in essence. ,……,,,, ( .) However, because of the complexities of the mechanisms and manifestations of diseases, it’s not easy to exactly point out which one, yin or yang, is disharmonious. As indicated by Qin’ an Zheng, a famous physician in Qing dynasty, while it’s more difficult to differentiate syndromes than learning pharmacy, it’s even more difficult to differentiate yin and yang than differentiating syndromes. , , , () Although syndrome differentiation of yin-yang has been investigated intensively and extensively in Traditional Chinese Medicine (TCM), only a few studies have been conducted from a computational perspective.
2. Background Machine learning and data mining technologies have been widely used in syndrome differentiation in modern research of TCM, giving rise to a large volume of literatures and reviews [2– 5]. Jiang et al. presented an overall profile for syndrome-related studies in both Chinese and English literatures, and generated semantic networks based on the frequencies of co-occurrences in the literatures [6]. Song et al. investigated the differentiation of 7 syndromes in depression from 364 patients [7]. It’s reported Support Vector Machines (SVM) achieves the highest accuracies of more than 90% except for yin deficiency. Zhao et al. proposed to use manifold ranking (MR) to explore the relation between syndromes for viral hepatitis [8]. Yao et al. used a supervised topic model and TCM domain ontology to discover treatment patterns using 3090 medical records [9]. Zhao conducted an extensive survey of more than one hundred papers on patient classification from a machine learning perspective [5]. To classify patients, these studies employ Naive Bayes, decision trees, SVM, graphical models and many other machine learning algorithms. Nowadays, deep learning and related neural networks have become the mainstream technologies in processing speeches, images, and texts. There have been several studies on syndrome differentiation in TCM using deep learning techniques as well. Liu et al. employed multi-label learning based on Deep Belief Network to construct the syndrome diagnostic model for chronic gastritis in TCM [10]. They used structured data to represent whether one of eight syndromes are present in a patient, such as damp heat accumulating in the spleen-stomach, and dampness obstructing the spleen-stomach, to predict chronic gastritis. Zhu et al. adopted a deep learning approach to damp-heat syndrome differentiation using medical records represented in terms of word vectors and TFIDF [11]. Yao et al. [12] studied medical record classification using document embedding technologies enriched with domain knowledge [13,14]. It has become a typical practice to use structured data in patient and case classifications. However, great potentials lie in the unstructured texts in the medical records in literature and from physicians as well [15,16]. Textual medical records in literature play an important role in the inheritance of medical knowledge from generation to generation. In addition, physicians in TCM used to record symptoms, the degrees of symptoms, syndromes, and many other valuable information using texts, based on their four diagnostic examinations of inspection, auscultation and olfaction, interrogation, and palpation.
Fig. 1. Comparison of end-to-end learning and traditional systems.
3. Syndrome differentiation as an end-to-end text classification task We model syndrome differentiation of yin-yang as a text classification task, which takes the texts in medical records as its inputs. Text classification is a widely used task in Natural Language Processing (NLP), and has been applied to the classification of news [17], spam filtering [18], and sentiment analysis [19], among many others. 3.1. End-to-End framework We adopt an end-to-end approach to text classification using two state-of-the-art neural network models, Convolutional Neural Network (CNN), and fastText. The idea of end-to-end is that the system takes in raw inputs, learn the features by itself, and then map the features to outputs. A comparison of end-to-end learning and traditional systems is shown in Fig. 1. The operations in gray boxes are performed automatically, while those in white boxes require human experts. As no feature engineering is involved, human efforts are reduced in end-to-end learning. As it turns out, end-toend systems sometimes outperform traditional systems [20], since their parameters are learned jointly throughout the entire learning pipeline. 3.2. Convolutional Neural Network Convolutional neural networks have been adopted to classify wrist pulse signals [21,22] and tongue colors [23] in TCM. In our study, the CNN model proposed in [24] is used to classify syndromes in textual TCM medical records. Its architecture is shown in Fig. 2. It includes three layers. In the input layer, the input words in a medical record are embedded into low-dimensional vectors. In the convolutional layer, the embedded word vectors are convoluted using multiple filter sizes, sliding over n-grams at a time. The results from the convolutional layer are then fed to a max-pooling layer to capture the most salient features. Finally, the classification is made using softmax in the fully-connected layer. 3.3. FastText We also use fastText (https://github.com/facebookresearch/ fastText) [25] to differentiate yin-yang deficiencies. Its architecture is shown in Fig. 3. In the input layer, the words in a medical record are represented as low-dimensional vectors. The word vectors are averaged in the hidden layer, and then go through a softmax classifier in the output layer. It is reported that, while dramatically cutting the time of training from several days to a few seconds, fast-
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
3
Fig. 2. Architecture of CNN for syndrome differentiation of yin-yang.
Fig. 3. Architecture of fastText for syndrome differentiation of yin-yang.
Text is often on par with deep neural networks in terms of classification accuracy. The experiments are conducted using a computer equipped with Nvidia Titan X Pascal GPU.
4. Data set We have collected more than 10 0,0 0 0 modern medical records from about 40,0 0 0 journal articles in 353 journals on TCM. Each medical record includes sex, age, and medical conditions of a patient, and diagnostic impressions from doctors. The impressions include syndromes, diseases, therapeutic principles and methods, treatments, formulas, and so on. In our systems, sex, age, and medical conditions of patients are used as the inputs, since these are the information used by doctors before making diagnostic decisions in clinical practice. Since the impressions are generally inferred on the basis of syndromes, they are excluded from the inputs. The labels of yin deficiency and yang deficiency are explicitly designated by the authors of the journal articles. For instance, the medical record in Example 1 is used to demonstrate the ”pattern of yin deficiency with effulgent fire” in a journal article [26], indicating that this medical record is a case of ”yin deficiency”. We extract this explicit indicative information from the journal articles.
In our data set, 7326 medical records are labeled as yin deficiency or yang deficiency, accounting for 6.88% of all medical records. 3720 (51%) records are yin deficiency, and 3606 (49%) records are yang deficiency. The numbers of yin and yang deficiencies are in balance in general. Example 1: Journal article snippet: “1.2.3 , , , , , , , ... 6 ,,52,2005-04-04..., ,,,,,, , ,, ...,... :12 g,...” (1.2.3 Pattern of yin deficiency with effulgent fire Palpitation, dryness-heat and sweating, reddened complexion, yellow urine, dry stool, dry mouth, drinking a lot, dry lips, red tongue, rapid pulses. The appropriate treatments are enriching yin and purging fire... Example 6: Ms Li, female, 52-year old, worker. 1st visit on 2005-04-04. ... paroxysmal palpitation, feeling hot, easy to sweat, dry mouth, drinking a lot, difficult to open eyes, preferring cold food, short breath when talking, poor sleep, slightly dry stool, yellow urine, irregular menstruation, ... red tongue with a yellow
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM 4
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
Fig. 4. Accuracies of fastText w/ characters, terms, and words as inputs.
cover, rapid pulses, ..., Dangguiliuhuang decoction addition and subtraction, prescriptions: prunella 12g, ...) Input: 52 ,,,,, ,,,, ,, (female 52 paroxysmal palpitation, feeling hot, easy to sweat, dry mouth, drinking a lot, difficult to open eyes, preferring cold food, short breath when talking, poor sleep, slightly dry stool, yellow urine, irregular menstruation, red tongue with a yellow cover, rapid pulses) Output: (yin deficiency) It’s worthy to note that the differentiation of syndromes involving both yin and yang deficiencies, such as (dual deficiency in yin and yang), and (detriment to yang affecting yin), are excluded from the current study, due to a lack of data of such kinds of syndromes in our corpus. 5. Results We have conducted two sets of experiments on syndrome differentiation of yin deficiency and yang deficiency. To avoid overfitting, all experimental results are obtained using 10-fold cross validation. The data set is randomly split into 10 subsets of equal size. In each of the 10 folds, a model is trained on 9 subsets, and tested on the remaining one subset. The performance is averaged over the 10 folds. Specifically, in our study, the entire data set of 7326 medical records is randomly split into 10 subsets, each of which includes 733 medical records. In each fold, a model is trained on 6593 medical records and tested on the remaining 733 records. 5.1. End-to-End learning Applying the end-to-end model, vanilla settings of fastText are used. We first investigate three types of raw inputs to the system, i.e. characters, terms and words. When using medical terms as the inputs, the medical conditions are filtered using a glossary having more than 170 millions of terms in Traditional Chinese Medicine Language System (TCMLS) [26–28]. The idea
Table 1 Precisions, recalls and F1 scores of fastText.
Yin deficiency Yang deficiency
Precision
Recall
F1
0.9243 0.9174
0.9203 0.9223
0.9222 0.9197
is to remove the stop words and the words unrelated to the medical domain from the medical records. The words are obtained using the tokenizer from Language Technology Platform (https://github.com/HIT-SCIR/ltp) [29] equipped with the glossary from TCMLS. Fig. 4 shows the performances of faxtText using characters, terms and words as its inputs, with varying sizes of corpora. It is obvious that faxtText using characters and words as inputs significantly outperform that using terms. The poor performances of term-based systems may arise from the low recall of medical terms. The terms remained for example 1 are shown below; other terms in the medical conditions are out of the glossary and have been discarded from the inputs. Example 1 (term) Input: 52 (female 52 palpitation, paroxysm, shortness of breath, poor sleep, stool, urine, menstruation, red tongue) Surprisingly, as far as the task of syndrome differentiation of yin deficiency and yang deficiency is concerned, character-based systems generally have an accuracy very close to the word-based systems, which depend heavily on domain and linguistic knowledge. The differences between them are no larger than 1%. Furthermore, since fastText algorithm doesn’t take the positions of tokens into consideration, we try to use n-grams to capture local dependencies among tokens to improve its performance. We observe a significant improvement when extending from 1-grams to 3-grams in all of the three types of tokens, as shown in Figs. 5, 6, and 7. While accuracies depict overall performances of these models, we further look into the class-specific evaluation metrics for details, as shown in Table 1. fastText models using 5-grams present a balanced performance on the differentiation of yin deficiency and
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
ARTICLE IN PRESS
JID: COMM
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
5
Fig. 5. Accuracies of fastText w/ n-gram characters as inputs.
Fig. 6. Accuracies of fastText w/ n-gram terms as inputs. Table 2 Hyper-parameters used in CNN and fastText. Algorithm
Embedding dimension
Drop-out
Batch size
Epochs
Other
CNN fastText
128 100
0.5 0.5
64 64
100 100
filter sizes: 3, 4, 5 5-grams
yang deficiency, as the precisions, recalls, and F1 scores of these two classes are very close. 5.2. CNN vs. fastText In our experiments on a comparison between CNN and fastText, we generally use the vanilla settings of the hyper-parameters in these models, as shown in Table 2. The embeddings of tokens are
randomly initialized. The numbers of epochs in both models are set to be 100. To capture local dependencies among tokens, the sizes of convolutional filters in CNN are set to 3, 4, and 5; and 5grams of tokens are fed to fastText as its inputs. As shown in Fig. 8, the accuracies of syndrome differentiation of yin deficiency and yang deficiency using CNN and fastText are 92.55% and 92.11%, respectively. The best accuracy comes from the CNN system using 5-grams of characters as its inputs. It implies
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM 6
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
Fig. 7. Accuracies of fastText w/ n-gram words as inputs.
Fig. 8. Accuracies of CNN and fastText.
that one can at least get a moderate system even if he has no glossary or tokenizer at hand. The differences between the performances of CNN and fastText systems are insignificant. An analysis on the bad cases discovers that the classifiers are likely to make errors for the medical records in return visits, because the medical conditions in subsequent visits are not as complete as the first one, and the descriptions of the changes may lead to an opposite prediction, as illustrated in the 2nd visit in Example 2. Example 2: 1st visit: Input: 42 , , , ,,,
(male 42 lassitude of spirit, lack of strength, bright pale complexion, edema all over the body, pit remaining after pressing the edema, pale and enlarged tongue with thin white fur, sunken fine soggy pulse) Output: (yang deficiency) 2nd visit: Input: 42 ,,, (male 42 edema gone, tongue becoming pale red with thin fur, sunken fine pulse) Output: (yang deficiency) 6. Future plans The current research is our first attempt at syndrome differentiation using deep learning technologies. It would be an interesting
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011
JID: COMM
ARTICLE IN PRESS
[m5G;October 27, 2018;11:10]
Q. Hu et al. / Computer Methods and Programs in Biomedicine 000 (2018) 1–7
topic to explore whether the simple character-based text classification models are applicable to more complicated syndrome differentiation tasks, such as the differentiation of qi deficiency and blood deficiency, the deficiencies in viscera and bowels, and the determination of therapeutic methods. 7. Conclusions This study has demonstrated the feasibility of using text classification technologies on syndrome differentiation of yin deficiency and yang deficiency using unstructured medical records. Both CNN and fastText models used in this study are open source, and the application of end-to-end paradigm requires minimal efforts on preprocessing. Nor are efforts required on feature engineering, making it easy to be conducted by physicians and other medical practitioners with limited computer skills for their own studies. Acknowledgment This work is supported by “the Fundamental Research Funds for the Central public welfare research institutes (NO. ZZ070804)” References [1] World Health Organization and others, WHO International Standard Terminologies on Traditional Medicine in the Western Pacific Region, Manila: WHO Regional Office for the Western Pacific, 2007. [2] Y. Huang, Y. Gao, B. Ma, Review of data mining methods frequently used in tcm syndrome study [j], Acta Chi. Medicine Pharmacol. 3 (2010) 004. [3] M. Jiang, C. Lu, C. Zhang, J. Yang, Y. Tan, A. Lu, K. Chan, Syndrome differentiation in modern research of traditional chinese medicine, J. Ethnopharmacol. 140 (3) (2012) 634–642. [4] J. Poon, S.K. Poon, Data analytics for traditional Chinese medicine research, Springer, 2014. [5] C. Zhao, G.-Z. Li, C. Wang, J. Niu, Advances in patient classification for traditional Chinese medicine: a machine learning perspective, Evidence-Based Complemen. Altern. Med. 2015 (2015). [6] M. Jiang, C. Zhang, G. Zheng, H. Guo, L. Li, J. Yang, C. Lu, W. Jia, A. Lu, Traditional chinese medicine zheng in the era of evidence-based medicine: a literature analysis, Evidence-Based Complemen. Altern. Med. 2012 (2012). [7] J. Song, X. Liu, Q. Deng, W. Dai, Y. Gao, L. Chen, Y. Zhang, J. Wang, M. Yu, P. Lu, et al., A network-based approach to investigate the pattern of syndrome in depression, Evidence-Based Complemen. Altern. Med. 2015 (2015). [8] Y.-f. Zhao, L.-y. He, B.-y. Liu, J. Li, F.-y. Li, R.-l. Huo, X.-h. Jing, Syndrome classification based on manifold ranking for viral hepatitis, Chinese J. Integrative Med. 20 (5) (2014) 394–399. [9] L. Yao, Y. Zhang, B. Wei, W. Wang, Y. Zhang, X. Ren, Y. Bian, Discovering treatment pattern in traditional chinese medicine clinical cases by exploiting supervised topic model and domain knowledge, J. Biomed. Inform. 58 (2015) 260–267.
7
[10] G.-P. Liu, J.-J. Yan, Y.-Q. Wang, W. Zheng, T. Zhong, X. Lu, P. Qian, Deep learning based syndrome diagnosis of chronic gastritis, Comput. Math. Methods Med. 2014 (2014). [11] W. Zhu, W. Zhang, G.-Z. Li, C. He, L. Zhang, et al., A study of damp-heat syndrome classification using word2vec and tf-idf, in: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE, 2016, pp. 1415–1420. [12] L. Yao, Y. Zhang, B. Wei, Z. Li, X. Huang, Traditional Chinese medicine clinical records classification using knowledge-powered document embedding, in: Bioinformatics and Biomedicine (BIBM), 2016 IEEE International Conference on, IEEE, 2016, pp. 1926–1928. [13] X. Zhou, Y. Peng, B. Liu, Text mining for traditional chinese medical knowledge discovery: a survey, J. Biomed. Inform. 43 (4) (2010) 650–660. [14] X. Zhou, B. Liu, X. Zhang, Q. Xie, R. Zhang, Y. Wang, Y. Peng, Data mining in real-world traditional Chinese medicine clinical data warehouse, in: Data Analytics for Traditional Chinese Medicine Research, Springer, 2014, pp. 189–213. [15] Y. Wang, Z. Yu, Y. Jiang, Y. Liu, L. Chen, Y. Liu, A framework and its empirical study of automatic diagnosis of traditional chinese medicine utilizing raw free-text clinical records, J. Biomed. Inform. 45 (2) (2012) 210–223. [16] Y. Wang, Z. Yu, L. Chen, Y. Chen, Y. Liu, X. Hu, Y. Jiang, Supervised methods for symptom name recognition in free-text clinical records of traditional chinese medicine: an empirical study, J. Biomed. Inform. 47 (2014) 91–104. [17] X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, in: Advances in neural information processing systems, 2015, pp. 649–657. [18] M. Sahami, S. Dumais, D. Heckerman, E. Horvitz, A bayesian approach to filtering junk e-mail, in: Learning for Text Categorization: Papers from the 1998 workshop, 62, Madison, Wisconsin, 1998, pp. 98–105. [19] B. Pang, L. Lee, S. Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, in: Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, Association for Computational Linguistics, 2002, pp. 79–86. [20] S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, D. Quillen, Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection, Int. J. Rob. Res. 37 (4–5) (2018) 421–436. [21] X. Hu, H. Zhu, J. Xu, D. Xu, J. Dong, Wrist pulse signals analysis based on deep convolutional neural networks, in: Computational Intelligence in Bioinformatics and Computational Biology, 2014 IEEE Conference on, IEEE, 2014, pp. 1–7. [22] S.-R. Zhang, Q.-F. Sun, Human pulse recognition based on convolutional neural networks, in: Computer, Consumer and Control (IS3C), 2016 International Symposium on, IEEE, 2016, pp. 366–369. [23] J. Hou, H.-Y. Su, B. Yan, H. Zheng, Z.-L. Sun, X.-C. Cai, Classification of tongue color based on cnn, in: Big Data Analysis (ICBDA), 2017 IEEE 2nd International Conference on, IEEE, 2017, pp. 725–729. [24] Y. Kim, Convolutional neural networks for sentence classification, arXiv preprint arXiv:1408.5882 (2014). [25] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, Bag of tricks for efficient text classification, arXiv preprint arXiv:1607.01759 (2016). [26] O. Bodenreider, The unified medical language system (umls): integrating biomedical terminology, Nucleic Acids Res. 32 (suppl_1) (2004) D267–D270. [27] Semantic Types in Unified Medical Language System (UMLS), https://www. nlm.nih.gov/research/umls/new_users/online_learning/SEM_003.html. [28] Semantic Relationships in Unified Medical Language System (UMLS), https:// www.nlm.nih.gov/research/umls/new_users/online_learning/SEM_004.html. [29] W. Che, Z. Li, T. Liu, LTP: A chinese language technology platform, in: Proceedings of the 23rd International Conference on Computational Linguistics: Demonstrations, Association for Computational Linguistics, 2010, pp. 13–16.
Please cite this article as: Q. Hu et al., End-to-End syndrome differentiation of Yin deficiency and Yang deficiency in traditional Chinese medicine, Computer Methods and Programs in Biomedicine (2018), https://doi.org/10.1016/j.cmpb.2018.10.011