A Shapelet Selection Algorithm for Time Series Classification: New Directions

Available online at www.sciencedirect.com Available online at www.sciencedirect.com

Available online at www.sciencedirect.com

ScienceDirect

Procedia Computer Science 00 (2018) 000–000 Procedia Computer Science (2018) 000–000 Procedia Computer Science 12900 (2018) 461–467

www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia

2017 International Conference on Identification, Information and Knowledge in the Internet of 2017 International Conference on Identification, Information and Knowledge in the Internet of Things Things

A A Shapelet Shapelet Selection Selection Algorithm Algorithm for for Time Time Series Series Classification: Classification: New New Directions Directions Cun Jiaa , Shijun Liua,b,∗ , Chenglei Yanga,b , Li Pana,b , Lei Wua,b , Xiangxu Menga,b Cun Ji , Shijun aLiua,b,∗, Chenglei Yanga,b , Li Pana,b , Lei Wua,b , Xiangxu Menga,b School of Computer Science & Technology, Shandong University, Jinan, China

a School of Computer Science & Technology, Shandong University, Jinan, China b Engineering Research Center of Digital Media Technology, Ministry of Education, Jinan, b Engineering Research Center of Digital Media Technology, Ministry of Education, Jinan,

China China

Abstract Abstract Time series classification (TSC) has attracted significant interest over the past decade. One of the promising recent approaches Time series classification (TSC)which has attracted significantmore interest over the decade. One of the promising recent are shapelet based algorithms, are interpretable, accurate, andpast faster than most classifiers. However, theapproaches high time are shapelet based algorithms, which are interpretable, more accurate, and faster than most classifiers. However, highpaper time complexity of shapelet selection process hinders its application in real time data procession. To overcome this, the in this complexity shapelet selection process hinders its application in real timethe data procession. To overcome in this In paper we propose of a fast shapelet selection algorithm (FSS), which sharply reduces time consumption of shapeletthis, selection. our we proposewe a fast selection which consumption shapelet selection. In our algorithm, first shapelet sample some time algorithm series from(FSS), training data sharply set withreduces the helpthe of atime subclass splitting of method. Then FSS identifies algorithm, we firstdeviation sample some series for from trainingtime dataseries set with helpthe of asubsequences subclass splitting method. Then FSS identifies the local farthest pointstime (LFDPs) sampled andthe selects between two nonadjacent LFDPs theshapelet local farthest deviation pointsthese (LFDPs) for sampled time series and selects the subsequences between two nonadjacent LFDPs as candidates. Through two steps, the number of shapelet candidates is greatly reduced, which leads to an obvious as shapeletincandidates. Through these twomethods steps, thewhich number of shapelet candidates is process greatly at reduced, whichofleads to anaccuracy, obvious reduction time complexity. Unlike other accelerate shapelet selection the expense reducing reduction in time complexity. Unlike other which accelerate shapelet selection process the expense of reducing accuracy, the experimental results demonstrate that methods FSS is thousands of times faster than ST with no at accuracy reduced. Our results also the experimental results demonstrate that FSS is thousands times faster than ST with nohave accuracy reduced. Our also demonstrate that our methods is the fastest method among theofshapelet-based methods which the leading level of results accuracy. demonstrate that our methods is the fastest method among the shapelet-based methods which have the leading level of accuracy. c 2018 Copyright Copyright  © 2018 Elsevier Elsevier Ltd. Ltd. All All rights rights reserved. reserved. c 2018 Copyright  Elsevierunder Ltd. All rights reserved. Selection peer-review responsibility of Selection and and peer-review under responsibility of the the scientific scientific committee committee of of the the 2017 2017 International International Conference Conference on on Identification, Identification, Selection andand peer-review under responsibility of the (IIKI2017). scientific committee of the 2017 International Conference on Identification, Information Knowledge in Internet Information and Knowledge in the the Internet of of Things Things (IIKI2017). Information and Knowledge in the Internet of Things (IIKI2017). Keywords: time series classification; shapelet transform; shapelet selection; subclass split; local farthest deviation points Keywords: time series classification; shapelet transform; shapelet selection; subclass split; local farthest deviation points

1. Introduction 1. Introduction A time series T = (t1 , t2 , · · · , ti , · · · , tm ) is a sequence of m real-valued data points measured successively at uniform A time seriesTime T = (tseries · , ti ,are · · · always , tm ) is aconsidered sequence ofasm areal-valued datathan points successively uniform 1 , t2 , · ·data time intervals. whole rather asmeasured individual numerical at fields [6]. time intervals. Time series data are always considered as a whole rather than as individual numerical fields [6]. Effective time series classification (TSC) has been an important research problem for both academic researchers and Effective time series classification (TSC) has been an important research problem for both academic researchers and industry practitioners in the last decade. In TSC, an unlabeled time series is assigned to one of two or more predefined industry practitioners the last decade.high In TSC, an unlabeled is assigned to one of twofound or more predefined classes [16]. The highindimensionality, feature correlationtime and series typically high levels of noise in time series classes [16]. The high dimensionality, high feature correlation and typically high levels of noise found in time series bring great challenges to TSC [16]. bring great challenges to TSC [16]. ∗ ∗

Corresponding author. Tel.: +86-13793165737 ; fax: +86-0531-88390059. Corresponding Tel.: +86-13793165737 ; fax: +86-0531-88390059. E-mail address:author. [email protected] E-mail address: [email protected] c 2018 Elsevier Ltd. All rights reserved. 1877-0509 Copyright  c 2018 Elsevier 1877-0509and Copyright  Ltd. Allof rights scientific reserved. committee of the 2017 International Conference on Identification, Information and Selection peer-review responsibility 1877-0509 Copyright © under 2018 Elsevier Ltd. Allthe rights reserved. Selection and peer-review responsibility of the scientific committee of the 2017 International Conference on Identification, Information and Knowledge in the Internet under of Things (IIKI2017). Selection and peer-review under responsibility of the scientific committee of the 2017 International Conference on Identification, Information Knowledge in theinInternet of Things (IIKI2017). and Knowledge the Internet of Things (IIKI2017). 10.1016/j.procs.2018.03.025

462 2

Cun Ji et al. / Procedia Computer Science 129 (2018) 461–467 Cun Ji, Shijun Liu, Chenglei Yang, Li Pan, Lei Wu, Xiangxu Meng / Procedia Computer Science 00 (2018) 000–000

A lot of TSC algorithms have been proposed. Among them, shapelet based algorithms can meet these requirements better. First, they are more compact than many alternatives, resulting in faster classification. Second, shapelets are directly interpretable. Third, shapelets allow for the detection of shape-based similarities in subsequences. This type of similarity is often hard to detect when examining the whole series. However, the training time in shapelet based algorithms is high, even though it is computed offline [5], because there are too many shapelet candidates and the time complexity to evaluate each candidate is high. As shown in [18, 12], nearly all subsequences of time series are selected as shapelet candidates. To address this problem, in this work a fast shapelet selection algorithm (FSS) is proposed. Unlike previous speed up technologies for shapelet discovery [12, 25, 20], which mainly aim to reduce the time complexity of evaluating every candidate, our algorithm aims to reduce the number of shapelet candidates. In our algorithm, we first sample some time series from training data set with the help of a subclass splitting method. Then FSS identifies LFDPs for sampled time series and selects the subsequences between two nonadjacent LFDPs as shapelet candidates. Through these two steps, the number of shapelet candidates is greatly reduced, which leads to a significant reduction in time complexity. The remainder of this paper is structured as follows. Section 2 gives some related works on TSC, especially shapelet based algorithms, the preliminary of ST and some related speed up technologies for shapelet selection process are also given in this section. The proposed FSS algorithm is introduced in Section 3. Experimental results are presented in Section 4 and our conclusions are given in Section 5. 2. Related Work Since being introduced in 2009 [26], shapelet based classifiers have aroused the interest of many researchers. shapelet based algorithms are promising for they are interpretable, more accurate, and faster than other algorithms. After that, a lot of shapelet based algorithms are proposed. There are three main types of shapelet based algorithms:1) Shapelet discovery algorithms [20, 26, 27, 21]. 2) Shapelet transformation algorithms [18, 12]. 3) Learning shapelets algorithms [9, 24, 17]. We use three classical algorithm FS [21], ST [18, 12], LS [9] on behalf of these three types of algorithms respectively. They are compared as follow: 1) Classification Accuracy: “The published results for FS are significantly worse than those for LS and ST. There is no significant difference between the LS and ST published results.[1]” The classification accuracies of ST and LS is higher than FS. LS and ST are in the same level. 2) Running Time: As far as time complexity concerned, FS is best, then LS is better and ST is worst. However, running time is not exactly the same as time complexity. “We found that LS was not noticeably faster, and required more memory, than ST with the optimizations included. [3]”. All in all, the running time of FS is far less than ST and LS, and the running time of ST and LS is in the same level. 3) Extensive Applications: Ensemble algorithms are more and more popular in recent TSC research. ST is adopted by some ensemble algorithms such as the Hierarchical Vote Collective of Transformation-based Ensembles (Hive-COTE, [19]) and COTE [3]. Until now, none of shapelet discovery algorithms or learning shapelets algorithms are adopted by public ensemble algorithms. In a word, ST has a wider range of applications. All in all, ST and LS are better than FS on classification accuracies. Whilst, FS is much faster than ST and LS. On classification accuracies and running time, ST and LS are in the same level. In addition, ST can be adopted in a wider range of ensemble algorithms. Our work is based on ST [18, 12]. The main advantages of ST are that it optimizes the process of shapelet selection and it allows various classification strategies to be adopted. However, the training time of ST is high. Some acceleration strategies are aimed to reduce the time complexity of evaluating each candidate shapelet. Ye and Keogh developed two speedup methods: Subsequence Distance Early Abandon (SDEA) and Admissible Entropy Pruning (AEP) [27]. In SDEA, once the distance is larger than the current smallest distance, the computation is abandoned. SDEA can help reduce the runtime by a factor of two. AEP calculates a cheap-to-compute upper bound of the information gain, and uses this to admissibly prune certain candidates. AEP reduces the runtime by more than two orders of magnitude. Mueen et al. accelerated the time required to find the same shapelet by reusing computations and pruning the search space [20]. Their algorithm uses a matrix to cache the distance computations for future use, and then applies the triangle inequality to prune some candidates. Xing et al. accelerated the shapelet selection process by sharing the computation among different local shapelets [25]. The computation is shared by storing additional information regarding the matching distances. Also, there are some technologies speeding up the shapelets selection process by



Cun Ji et al. / Procedia Computer Science 129 (2018) 461–467 Cun Ji, Shijun Liu, Chenglei Yang, Li Pan, Lei Wu, Xiangxu Meng / Procedia Computer Science 00 (2018) 000–000

463 3

reducing the number of candidates. In [25], Xing et al. set the minimum length of shapelets to 5. If the length of time series is bigger than 100, then the maximum length of the shapelets is set to half of the time series length. What’s more, the step size is set to 3. In [12], Hills et al. defined a simple algorithm for estimating the minimum and maximum length of the shapelets. They randomly selected ten series to find the best ten shapelets in this subset of the data and repeated it ten times. The shapelets are sorted by length, with the length of the 25th shapelet returned as minimum length and the length of the 75th shapelet returned as maximum length. Zhang et al. [28] filtered shapelet candidates with key points. Meanwhile, Ji. et al [14] reduced the number of candidates by selecting subsequences which contained one or more important data points. Gordon et al. [7, 8] introduce a shapelet sampling algorithm (which named SALSA-R) for fast computation of shapelet-based classification trees which does not examine all possible shapelets. Renard et al. [22] proposed a random-shapelet (RS) to reduce dramatically the time required. Their method is based on the randomization of the discovery process. Grabocka et al. [10, 11] process a Scalable shapelet Discovery (SD) which reduces the numbers of shaplet candidates through an online clustering pruning technique. Karlsson et al. [15] reduced shaplet candidates by generalizing random shapelet forests (gRSF). However, these methods are aimed to speed up shapelet shapelet discovery algorithms. 3. FSS: Accelerating ST In this paper, we proposed a fast shapelet selection algorithm (FSS) for ST through reducing the number of shapelet candidates. In FSS, there are two acceleration strategies : 1) Sample time series: Sample some time series from training data set with the help of a subclass splitting method; 2) Filter shapelet candidates: Filter some shapelet candidates with the help of a time series piecewise linear representation method. 3.1. Strategy 1: Sample Time Series It has been demonstrated that sampling one or some time series per class directly is insufficient and greatly reduces the accuracies [23]. One important reason is that subclasses are typically present in most real world data sets. For this, we sample time series with the help of a subclass splitting method. In this paper, we will use the split subclass method as [23]. And then we will sample time series from every subclass. Suppose there are nc time series T i = {t1 , t2 , · · · , t j , · · · , tm } in one class Dc = {T 1 , T 2 , · · · , T i , · · · , T nc } and the time series length is m. There are three steps to sample time series: Step 1: Select criteria time series. The time series which is closest to the mean sum value is selected as criteria time series. The sum value of one time series  is the sum of the individual values. The sum value of every time series T i can be calculated as S umi = mj=1 t j . nc

S um

The mean sum value can be calculate as Mean = i=1nc i . The criteria time series can be described as TC = argmini(1,2,··· ,nc ) (|sumi − Mean|). Step 2: Split subclass. After get the criteria time series, we can calculate Euclidean distance between time series in the class and criteria. The Euclidean distance between T i = {t1 , t2 , · · · , t j , · · · , tm } and  TC = {c1 , c2 , · · · , c j , · · · , cm } can be calculated as dist = Σmj=1 (t j − c j )2 . Then we sort time series by these distance values, and we can get the adjacent discrepancies of the distance values. Next, we calculate the standard deviation value of these adjacent discrepancies. At last, we can separate the data into subclasses by splitting at the sequence that has difference larger than half of the computed standard deviation. Step 3: Sample time series. Finally, we sample one time series from every subclass. In every subclass, the time series which has minimum sum distance to other time series in this subclass is selected as sample. 3.2. Strategy 2: Filter Shapelet Candidates As [28] described, shapelet must have a certain significance. In other words, subsequences with no obvious features can be filtered out from the shapelets candidate. If a subsequence is approximately like a line segment, we can filter out it. Based on this principle, we can use the time series piecewise linear representation method to filter out shapelet candidates. Note that, we do not make a linear representation of the time series. We only use the endpoints in piecewise linear representation method to filter out shapelet candidates. An example is given in Fig. 1, where there is one time series of “GunPoint” [4]. The time series can be linear representation as dotted line. The endpoints of dotted line are marked out with red dots. The subsequence S 1, which

464 4

Cun Ji et al. / Procedia Computer Science 129 (2018) 461–467 Cun Ji, Shijun Liu, Chenglei Yang, Li Pan, Lei Wu, Xiangxu Meng / Procedia Computer Science 00 (2018) 000–000

Fig. 1: An example of shapelet candidates filter.

is marked out with green color, is approximately like a line segment. It will be filtered out from shapelet candidates. The subsequence S 2, which is marked out with dark red, is approximately like two line segments. It will be remained in the candidates. There are two steps in this speedup strategy: Step 1: Identity endpoints. In [13], Ji et al. proposed a piecewise linear representation method based on important data points for time series data. In this paper, we will used their method in [13] to get endpoints of time series. The points we identified are called LFDPs. As [13], LFDPs are defined as those points which satisfy the following conditions: the point is in the subsequence S which has the maximum weight and the point has a maximum distance from the fitting line of S . The weight of one subsequence is expressed as weight = max(dist sum , 2 ∗ distmax ), dist sum is the sum of the distances of all points in the subsequence and distmax is the maximum among these distances. The distances are the fitting errors of the points to the fitting line. In this paper, we will identify LFDPs by the given number. The LFDP selection algorithm is shown in Algorithm 1. At first, the start and end points are selected as LFDPs. Next, we choose LFDPs through loop process. In the loop process, we sort the time series segments set by weight and select the time series segment which has maximum weight. Then the point which has the maximum distance to the fitting line of the selected time series segment will be identify as LFDP. The selected time series segment is cut into two segments. The information of new generated segments are put into time series segments set and the segment which has been cut are removed out. We pick one LFDP in each iteration. This loop stops until the number of LFDPs reaches the given number. Step 2: Generate shapelet candidates. After getting LFDPs in Step 1, we will use them to generate shapelet candidates. Subsequences which use non-adjacent LFDPs as endpoints are selected as shapelet candidates. 4. Experiments and Evaluation 4.1. Experimental Setup We implemented our algorithm based on code that is freely accessible from an online repository [2]. Our code and detailed results is open1 so that the results can be independently replicated. The experiments were carried out in Java on a 3.10 GHz Intel Core i5 CPU with 16 GB, 1333 MHz DDR3 internal storage, using MyEclipse with JDK 1.8. In all of our experiment, the number of shapelets (k) selected is set to half the size of the training set (The reason why set this value please read [18, 12] ). And the number of LFDP (q) is set to 0.05 ∗ m + 2 (m is time series length, 2 represents that the first and last points in time series must be selected as LFDPs. We use information gain as the assess method for shapelet quality measures. In this paper, we selected 12 data sets from the UEA & UCR Time Series Classification Repository [4]. The results of comparative experiments on the 12 data sets can be found in the relevant literature. 1 Our

code: https://github.com/sdujicun/FSS.



Cun Ji et al. / Procedia Computer Science 129 (2018) 461–467 Cun Ji, Shijun Liu, Chenglei Yang, Li Pan, Lei Wu, Xiangxu Meng / Procedia Computer Science 00 (2018) 000–000

465 5

Algorithm 1 LFDPs Selection Algorithm Input: time series data: T = {t1 , t2 , · · · , tm }, LFDP number: q Output: the index of LFDP; 1: int[] LFDP=new int[q] 2: List list=new ArrayList(); 3: list.add(T ); 4: LFDP[0]=1; LFDP[1]=m; 5: int tempNumber=2; 6: while {tempNumber < q} do 7: sortSegmentByWeight(list); 8: Segment s= list.get(0); 9: int LFDPIndex= getLFDPIndex(s); 10: LFDP[tempNumber]=LFDPIndex 11: tempNumber++; 12: Segment sL =leftSegment(s,LFDPIndex); 13: Segment sR =rightSegment(s,LFDPIndex); 14: list.add(sL ); list.add(sR ); 15: list.remove(0); 16: end while 17: return sortLFDPFromSmallToBig(LFDP) Table 1: Comparatione of Classification Accuracy

method Adiac Beef ChlorineConcentration Coffee DiatomSizeReduction ItalyPowerDemand Lightning7 MedicalImages MoteStrain Symbols Trace TwoLeadECG average rate average rank

ST 0.783 0.900 0.700 0.964 0.925 0.948 0.726 0.670 0.897 0.882 1.000 0.997 0.866 2.708

LS 0.522 0.867 0.592 1.000 0.980 0.960 0.795 0.664 0.883 0.932 1.000 0.996 0.849 3.042

FSS 0.780 0.833 0.607 0.929 0.912 0.926 0.740 0.721 0.899 0.895 1.000 0.972 0.851 3.333

gRSF 0.732 0.633 0.658 0.964 0.779 0.944 0.726 0.697 0.952 0.755 1.000 0.991 0.819 3.708

Acceleration Methods SALSA-R SD 0.726 0.583 0.609 0.975 0.671 0.553 0.960 0.961 0.769 0.896 0.951 0.920 0.695 0.652 0.686 0.676 0.854 0.783 0.864 0.865 1.000 0.965 0.958 0.867 0.812 0.808 4.452 5.583

FS 0.593 0.567 0.546 0.929 0.866 0.917 0.644 0.624 0.777 0.934 1.000 0.924 0.777 5.917

RS 0.516 0.324 0.572 0.769 0.774 0.924 0.635 0.529 0.815 0.795 0.934 0.914 0.708 7.250

4.2. Contrast Experiments for Classification Accuracy First of all, we will show that classification accuracy of FSS is in a leading level among shapelet based algorithms. We contrast FSS with the three famous shapelet based methods (FS [21], ST [18, 12], LS [9]) and four other methods (SD [10, 11], RS [22], gRSF [15], SALSA-R [7, 8]) which accelerate shapelet selection process by sampling. Table 1 lists the classification accuracy of them. Note that: 1) the accuracy listed for ST, FS and LS is taken from [9](In [9], the accuracy for ST is higher than the result in the paper [18, 12] which proposed ST). 2) The accuracy listed for SD is taken from [10, 11]. 3). The accuracy listed for RS is taken from [22]. We use the results of randomly selecting 10% subsequences as candidates. 4) The accuracy listed for SALSA-R is taken from [7, 8]. We use the results of sampling

Cun Ji et al. / Procedia Computer Science 129 (2018) 461–467

466 6

Cun Ji, Shijun Liu, Chenglei Yang, Li Pan, Lei Wu, Xiangxu Meng / Procedia Computer Science 00 (2018) 000–000 Table 2: Running Time Comparison

ChlorineConcentration Coffee DiatomSizeReduction Lightning7 MoteStrain Symbols Trace

FSS 3.1 0.1 0.3 10.4 0.01 4.7 17.9

gRSF 624.4 6.4 13.3 124 1.9 40.2 80.1

FS 175.5 6.8 8.3 114.9 0.3 37.5 55.5

ST 21921.6 548.5 332.6 13646.8 2.1 4477.3 6902

LS 6389.5 241.1 780 14535.9 23.5 4781.7 5332.5

30% subsequences as candidates. 5) The accuracy listed for gRSF is gettings by using the tool which is given in [15] 2 . As Table 1 shows, FSS has the highest average classification rate and smallest average rank in all acceleration methods. It demonstrates that FSS is better than other acceleration methods on accuracy. The average rank in Table 1 also demonstrates that we can find that FSS is still keeping a high level as ST and LS on accuracy. 4.3. Contrast Experiments for Running Time In Table 1, the average ranks of ST, LS, FSS and gRSF are smaller than FS, SD, RS and SALSA-R. And the average classification rate of ST, LS, FSS and gRSF are bigger than FS, SD, RS and SALSA-R. The results in Table 1 demonstrate that ST, LS, FSS and gRSF are better than FS, SD, RS and SALSA-R on accuracy. Next, we will show that our methods FSS is fastest among the methods (ST, LS, FSS and gRSF) which accuracies are high. FS is also be used as baseline. Table 2 lists the shapelet selection time of FSS, ST, LS, FS and gRSF 3 . Because the running time for ST and LS is too long (For example, the trainning time for ST on ”CinCECGtorso” is about 20 days in our experiment environments), we select 7 data sets in this experiments. As Table 2 shows, FSS is the fastest method among the methods which accuracies are high. FSS is tens of times faster than gRSF or FS. And it is thousand of times faster than ST or LS. In conclusion, our method is fastest among the shapelet based methods which accuracies are high. 5. Conclusion Shapelet transformation algorithms have attracted a lot of attention in the last decade. However, the time complexity of the shapelet selection process in shapelet transformation algorithms is too high. In order to accelerate shapelet selection process with no accuracy reduced, we have presented a fast shapelet selection algorithm, FSS, for ST. In our algorithm, we first sample some time series from training data set with the help of a subclass splitting method. Then FSS identifies LFDPs for sampled time series and selects the subsequences between two nonadjacent LFDPs as shapelet candidates. Through these two steps, the number of shapelet candidates can be greatly reduced, which leads to a significant reduction in time complexity. The experimental results demonstrate that our proposed FSS is thousands of times faster than the original shapelet transformation method with no accuracy reduced. Our results also demonstrate that our methods is the faster method among the shapelet methods which have the leading level of accuracy. Acknowledgements The authors would like to acknowledge the support provided by the National Natural Science Foundation of China (61402263, 91546203), the National Key Research and Development Program of China (2016YFB0201405), the 2 We

use default parameters for the tool. it handles in parallel, we use CPU Time as running time for gRSF.

3 Because



Cun Ji et al. / Procedia Computer Science 129 (2018) 461–467 Cun Ji, Shijun Liu, Chenglei Yang, Li Pan, Lei Wu, Xiangxu Meng / Procedia Computer Science 00 (2018) 000–000

467 7

Natural Science Foundation of Shandong Province (ZR2014FQ031), the Key Research and Development Program of Shandong Province of China(2017CXGC0605), the Shandong Provincial Science and Technology Development Program (2016GGX106001, 2016GGX101008, 2016ZDJS01A09), the Fundamental Research Funds of Shandong University (2016JC011), and the special funds of Taishan scholar construction project. References [1] Bagnall, A., Bostrom, A., Large, J., Lines, J., 2016a. The great time series classification bake off: an experimental evaluation of recently proposed algorithms. Extended Version. CoRR, abs/1602.01711 . [2] Bagnall, A., Bostrom, A., Lines, J., 2016b. The uea tsc codebase. https://bitbucket.org/aaron_bostrom/ time-series-classification. [3] Bagnall, A., Lines, J., Hills, J., Bostrom, A., 2015. Time-series classification with cote: the collective of transformation-based ensembles. IEEE Transactions on Knowledge and Data Engineering 27, 2522–2535. [4] Bagnall, A., Lines, J., Vickers, W., Keogh, E., 2016c. The uea & ucr time series classification repository. URL www.timeseriesclassification.com . [5] Chang, K.W., Deka, B., Hwu, W.W., Roth, D., 2012. Efficient pattern-based time series classification on gpu, in: 2012 IEEE 12th International Conference on Data Mining (ICDM), IEEE. pp. 131–140. [6] Fu, T.c., 2011. A review on time series data mining. Engineering Applications of Artificial Intelligence 24, 164–181. [7] Gordon, D., Hendler, D., Rokach, L., 2012. Fast randomized model generation for shapelet-based time series classification. arXiv preprint arXiv:1209.5038 . [8] Gordon, D., Hendler, D., Rokach, L., 2015. Fast and space-efficient shapelets-based time-series classification. Intelligent Data Analysis 19, 953–981. [9] Grabocka, J., Schilling, N., Wistuba, M., Schmidt-Thieme, L., 2014. Learning time-series shapelets, in: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM. pp. 392–401. [10] Grabocka, J., Wistuba, M., Schmidt-Thieme, L., 2015. Scalable discovery of time-series shapelets. arXiv preprint arXiv:1503.03238 . [11] Grabocka, J., Wistuba, M., Schmidt-Thieme, L., 2016. Fast classification of univariate and multivariate time series through shapelet discovery. Knowledge and Information Systems 49, 429–454. [12] Hills, J., Lines, J., Baranauskas, E., Mapp, J., Bagnall, A., 2014. Classification of time series by shapelet transformation. Data Mining and Knowledge Discovery 28, 851–881. [13] Ji, C., Liu, S., Yang, C., Wu, L., Pan, L., Meng, X., 2016. A piecewise linear representation method based on importance data points for time series data, in: 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD), IEEE. pp. 111–116. [14] Ji, C., Zhao, C., Pan, L., Liu, S., Yang, C., Wu, L., 2017. A fast shapelet discovery algorithm based on important data points. International Journal of Web Services Research (IJWSR) 14, 67–80. [15] Karlsson, I., Papapetrou, P., Bostr¨om, H., 2016. Generalized random shapelet forests. Data Mining and Knowledge Discovery 30, 1053–1085. [16] Keogh, E., Kasetty, S., 2003. On the need for time series data mining benchmarks: a survey and empirical demonstration. Data Mining and knowledge discovery 7, 349–371. [17] Kwok, L.H.J.T., Zurada, J.M., 2016. Efficient learning of timeseries shapelets . [18] Lines, J., Davis, L.M., Hills, J., Bagnall, A., 2012. A shapelet transform for time series classification, in: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM. pp. 289–297. [19] Lines, J., Taylor, S., Bagnall, A., 2016. Hive-cote: The hierarchical vote collective of transformation-based ensembles for time series classification, in: 2016 IEEE 16th International Conference on Data Mining (ICDM). [20] Mueen, A., Keogh, E., Young, N., 2011. Logical-shapelets: an expressive primitive for time series classification, in: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM. pp. 1154–1162. [21] Rakthanmanon, T., Keogh, E., 2013. Fast shapelets: A scalable algorithm for discovering time series shapelets, in: Proceedings of the 2013 SIAM International Conference on Data Mining, SIAM. pp. 668–676. [22] Renard, X., Rifqi, M., Erray, W., Detyniecki, M., 2015. Random-shapelet: an algorithm for fast shapelet discovery, in: 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), IEEE. pp. 1–10. [23] Sathianwiriyakhun, P., Janyalikit, T., Ratanamahatana, C.A., 2016. Fast and accurate template averaging for time series classification, in: 2016 8th International Conference on Knowledge and Smart Technology (KST), IEEE. pp. 49–54. [24] Shah, M., Grabocka, J., Schilling, N., Wistuba, M., Schmidt-Thieme, L., 2016. Learning dtw-shapelets for time-series classification, in: Proceedings of the 3rd IKDD Conference on Data Science, 2016, ACM. p. 3. [25] Xing, Z., Pei, J., Yu, P.S., Wang, K., 2011. Extracting interpretable features for early classification on time series, in: Proceedings of the 2011 SIAM International Conference on Data Mining, SIAM. pp. 247–258. [26] Ye, L., Keogh, E., 2009. Time series shapelets: a new primitive for data mining, in: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM. pp. 947–956. [27] Ye, L., Keogh, E., 2011. Time series shapelets: a novel technique that allows accurate, interpretable and fast classification. Data mining and knowledge discovery 22, 149–182. [28] Zhang, Z., Zhang, H., Wen, Y., Yuan, X., 2016. Accelerating time series shapelets discovery with key points, in: Asia-Pacific Web Conference, Springer. pp. 330–342.