Indoor corner recognition from crowdsourced trajectories using smartphone sensors

Accepted Manuscript

Indoor Corner Recognition from Crowdsourced Trajectories using Smartphone Sensors Yuchen Sun, Bang Wang PII: DOI: Reference:

S0957-4174(17)30268-3 10.1016/j.eswa.2017.04.024 ESWA 11257

To appear in:

Expert Systems With Applications

Received date: Revised date: Accepted date:

9 September 2016 10 April 2017 11 April 2017

Please cite this article as: Yuchen Sun, Bang Wang, Indoor Corner Recognition from Crowdsourced Trajectories using Smartphone Sensors, Expert Systems With Applications (2017), doi: 10.1016/j.eswa.2017.04.024

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT 1

AC

CE

PT

ED

M

AN US

CR IP T

Highlights • Address the fake corner and pose diversity problem in indoor corner recognition • Propose a hierarchical architecture consisting of three classifiers • Propose to classify diverse poses into only two pose groups • Conduct field experiments to confirm the superiority of the proposed scheme

ACCEPTED MANUSCRIPT 2

Indoor Corner Recognition from Crowdsourced Trajectories using Smartphone Sensors Yuchen Sun and Bang Wang

CR IP T

localization (Calderoni et al., 2015; Torres-Sospedra et al., 2015; Rubio et al., 2016; Wang et al., 2015a,b). The basic assumption of fingerprinting localization is that each indoor spatial location can be identified by a unique measurable feature. The mostly used fingerprint is based on the received signal strength (RSS) from the access points (APs) of wireless local access networks (WLANs). The key challenge to the success of fingerprinting localization is to obtain a fingerprint map for a given indoor environment. Site survey can be conducted to establish a fingerprint map via manually field measurements at given locations, however, it is too time-consuming, labor-intensive and cost-prohibitive to be implemented in large environments (Hossain and Soh, 2015). Recently, the crowdsourcing approach has been applied to help relieving or even eliminating the burden of site survey by exploiting casually collected fingerprints without explicitly labeled exact location information (Hossain and Soh, 2015; Wang et al., 2016a). Fingerprint annotation is to label where a crowdsourced fingerprint is collected, which can be implemented from crowdsourced movement trajectories. A movement trajectory contains not only the time series of RSS measurements but also the time series of smartphone internal sensors’ measurements. On the other hand, a typical indoor layout often includes many landmarks, such as corners, staircases, elevators and etc., which could induce detectable sensor measurement changes in a trajectory. If we can detect such landmarks from the signal space of a movement trajectory, then it is possible to match some fingerprints in a trajectory to the detected landmarks to implement fingerprint annotation. In this paper, we study the problem of indoor corner recognition from crowdsourced trajectories. Some recent work have applied corner detection in an indoor positioning system, which, however, adopt a very simple signal change detection technique (Park et al., 2013; Zhou et al., 2015a,b; Shang et al., 2015). Basically, these approaches are to detect a turning in a movement trajectory. Although a turning is likely to be caused due to passing by a corner, it may also be caused due to some turnaround in an open space without actually passing by a true corner. We call it the fake corner problem. Furthermore, these approaches seem to adopt a fixed pose for experimenting corner detection. However, in practice different users may take different poses to hold or place his smartphone when collecting trajectory measurements. Even worse, a user may switch his pose during trajectory collection. From our field experiments, we find that these existing corner detection approaches perform poorly when taking different poses. We call it the pose diversity problem. In this paper, we propose a corner recognition scheme

ED

M

AN US

Abstract—Recently, fingerprint crowdsourcing from pedestrian movement trajectories has been promoted to alleviate the site survey burden for radio map construction in fingerprintingbased indoor localization. Indoor corners, as one of the most common indoor landmarks, play an important role in movement trajectory analysis. This paper studies the problem of indoor corner recognition in crowdsourced movement trajectories. In a movement trajectory, smartphone internal sensor measurements experience some signal changes when passing by a corner. However, the state-of-the-art solutions based on signal change detection cannot well deal with the fake corner problem and pose diversity problem in most practical movement trajectories. In this paper, we study the corner recognition problem from an expert system viewpoint by applying machine learning techniques. In particular, we extract recognition features from both the time and frequency domain and propose a hierarchical corner recognition scheme consisting of three classifiers. The first pose classifier is to classify various poses into only two groups according to whether or not a smartphone is kept in a fixed position relative to a user upper body when collecting sensor measurements. Feature selection is then applied to train two corner classifiers each for one pose group. Field experiments are conducted to compare our proposed scheme with three state-of-the-art algorithms. In all cases, our scheme outperforms the best of these algorithms in terms of much higher F1-measure and precision for corner recognition. The results also provide insights on the potentials of using more advanced techniques from expert systems in indoor localization.

PT

Index Terms—indoor corner recognition, fake corner problem, pose diversity problem, indoor positioning system, machine learning.

I. I NTRODUCTION

AC

CE

In the last decade, indoor localization has been drawing great interests in both academia and industry due to its wide applications in expert and intelligent systems like targeted advertising, health care, smart parking, location-based recommender systems and so on (Hossain and Soh, 2015; Chiu et al., 2012; Lan and Shih, 2014; Calderoni et al., 2015; Kefalas and Manolopoulos, 2017; Tuan et al., 2017; Tsai et al., 2017). Although the global positioning system (GPS) works well in outdoor environments, its indoor performance is much unsatisfactory because of its weak reception of satellite signals. Nowadays, fingerprint-based indoor positioning systems which leverage the wide adoption of wireless networks and mobile devices have been promoted as a promising solution for indoor Yuchen Sun and Bang Wang are with the School of Electronic Information and Communications, Huazhong University of Science and Technology(HUST), Luoyu Lu #1037, Wuhan, Hubei, 430074, China. E-mail: [email protected] (Yuchen Sun), [email protected] (Bang Wang). The corresponding author is Bang Wang.

ACCEPTED MANUSCRIPT 3

CR IP T

match a movement trajectory from the signal space of sensor measurement to the physical space of indoor layout. Generally, these approaches can be divided into two groups based on whether or not taking landmark detection for a trajectory. A movement trajectory can be decomposed into several components, including walking steps, heading directions and stride length, via processing smartphone inertial sensor measurements by the pedestrian dead reckoning (PDR) technique (Harle, 2013; Xiao et al., 2015; Waqar et al., 2016; He et al., 2015). With such decomposition, the steps and directions of a trajectory can be used to convert a trajectory from the signal space to the Euclidean space. After obtaining its spatial representation, a trajectory can be matched to a feasible physical indoor route with the aid of a floor plan (hoon Jung et al., 2016; Zhou et al., 2015c; Rai et al., 2012). For example, Zee (Rai et al., 2012) leverages underlying physical constraints, such as walls, pathways and partitions, to obtain feasible walking routes based on an indoor layout. If a trajectory indicates a zigzag path and only one such feasible route exists, then this trajectory can be matched to this route. Following this basic matching idea, some improvements have been proposed (Wu et al., 2015; Shahidi and Valaee, 2015). LiFS (Wu et al., 2015) proposes to convert an indoor layout into a new stress-free floor plan for trajectory matching. (Shahidi and Valaee, 2015) propose to first generate a trajectory semantic graph and then apply the hidden Markov Model (HMM) approach for trajectory matching. (Xiao et al., 2015) propose to conduct lifelong learning with a feedback loop to iteratively improve trajectory estimation with only using PDR. However, the PDR-based trajectory decomposition might contain estimation errors, and even worse such errors can be accumulated, which could lead to poor performance of trajectory spatial representation. Indoor landmark detection can be used to reduce the PDR accumulative errors for trajectory matching (Park et al., 2013; Zhou et al., 2015a,b; Shang et al., 2015; Wang et al., 2012; Abdelnasser et al., 2015). Wang et al. (Wang et al., 2012) first propose the concept of indoor landmark and use landmarks to calibrate the accumulative error of the PDR algorithm. It is also worth of noting that indoor landmarks can also be directly exploited for fingerprint annotation and indoor localization. (Abdelnasser et al., 2015) include landmarks within the simultaneous localization and mapping (SLAM) framework and use a particle filter algorithm to reduce both the localization error and convergence time. (Zhou et al., 2015b) build a link-node model using landmarks for trajectory matching, where nodes represent the landmarks and links represent the walkable path between landmarks. (Zhou et al., 2015a) further propose a trajectory matching scheme using HMMs based on landmark sequences and PDR trajectories. Indoor corners have been regarded as one of common and important indoor landmarks in movement trajectory analysis. Although previous work has not systematically studied the corner recognition problem, some did have proposed algorithms for corner detection (Zhou et al., 2015b,a; Shang et al., 2015; Gu et al., 2016a; Chen et al., 2015; Wang et al., 2016b; Lan and Shih, 2014; Niu et al., 2015). We note that these algorithms are mostly related to our work in this paper and

AC

CE

PT

ED

M

AN US

to solve the fake corner problem and pose diversity problem from a machine learning approach. To the best of our knowledge, this paper is the first to systematically study the corner recognition problem. From the time series of sensor measurements, we first extract features from both the time and frequency domain to represent the most significant signal characteristics. From our field measurements, we observe that smartphone displacement and rotation relative to a walking user are the most impact factors interfering sensor signals for corder recognition. Thereby, we propose to classify all possible poses into two groups, namely, fixed and unfixed pose group, based on whether the smartphone is kept in a fixed position relative to a user upper body. Instead of using a single unified classifier for corner recognition, we propose a hierarchical classification architecture with three classifiers. At first, we use a pose classifier to classify a feature window into either fixed or unfixed pose group. We next use different corner detectors for the two pose groups. Furthermore, feature selection is applied to train different classifiers. Field measurements are collected to evaluate our proposed scheme. Based on whether containing the pose diversity instances and/or fake corner instances, field measurements can be divided into four groups: (i) only with ideal instances; (ii) with pose diversity instances; (iii) with fake corner instances; (iv) with all instances. The F1-measures of our proposed scheme for the four groups are (i) 100%, (ii) 90.5%, (iii) 91.9% and (iv) 73.2%, respectively. The F1-measures of the best of the state-of-the-art algorithms (Zhou et al., 2015a,b; Shang et al., 2015) are (i) 95.0%, (ii) 50.6%, (iii) 64.6% and (iv) 31.1%, respectively. Furthermore, the precisions of our proposed scheme are also higher than these algorithms in all groups. The experiment results indicate that our proposed scheme can be readily applied in practical corner recognition scenarios, which also suggest that the expert system for indoor localization like (Lan and Shih, 2014) is possible to implement in actual environments. Our contributions can be summarized as follows: • Address the fake corner and pose diversity problem in indoor corner recognition. • Propose a hierarchical architecture consisting of three classifiers. • Propose to classify diverse poses into only two pose groups. • Propose to train one classifier for each pose group with feature selection. • Conduct field experiments to confirm the superiority of the proposed scheme. The remainder of this paper is organized as follows: Section II reviews the related work. Section III describes the research problems, and our proposed scheme is presented and evaluated in Section IV and Section V, respectively. Section VI concludes the paper with some discussions. II. R ELATED W ORK Some previous schemes have been proposed to exploit crowdsourced movement trajectories to build a fingerprint map for indoor positioning systems. The main challenge is how to

ACCEPTED MANUSCRIPT

Fig. 1. Example of the trajectories which contain a true corner or not in the figure of the indoor layout. In this Figure, the part I(b) is the detailed drawing ^ of the part I(a), the same as II(b) for II(a). Each of the trajectories CDE, ^ ^ F GH, N P Q contains a true corner, as illustrated by the dotted boxes; While g IJK, g contains no corner or a fake corner. ] LM each of the trajectories AB,

CE

PT

ED

M

AN US

we compare them with our work in Table I. Most of these algorithms are based on the gyroscope signal peak detection and compass signal variation comparison, except one method in (Niu et al., 2015) that is based on the Wi-Fi signal change. Moreover, almost all of these methods have not considered the pose diversity problem, and except the one (Lan and Shih, 2014) that has considered to filter out fake corners with the aid of indoor layout map. These existing corner detectors only exploit the single signal peak or variation feature with some simple data preprocessing technique like using a sliding window, yet other useful signal features have not been considered. Our experiments in Section V show that these existing detectors cannot well deal with the fake corner and pose diversity challenges, which are very common in practical crowdsourcing scenarios. Different from the existing corner detectors, this paper studies the corner recognition problem from the expert system viewpoint by applying machine learning techniques. The pose diversity problem has not been received enough attention in the indoor localization domain, however, it has been considered in the human activity recognition (HAR) domain. HAR refers to recognizing a user current activity, including but not limited to walking, running, lying, going upstairs and downstairs, via wearable sensors or smartphone sensors (D.Lara and Labrador, 2013; Shoaib et al., 2015). To the best of our knowledge, we note that corner detection has not been researched in the HAR domain. It is noted in a recent HAR survey (Shoaib et al., 2015) that most HAR studies have assumed a fixed smartphone position, as the activity recognition performance is sensitive to smartphone positions. Among a few studies considering the pose diversity problem, two approaches are adopted: One is to train a single classifier for all poses with or without feature discriminant analysis (Anjum and Ilyas, 2013; Siirtola and Roning, 2013; Khan et al., 2014). Another is to first train a pose classifier, and then for each pose, train an individual activity classifier (Martłn et al., 2013). In this paper, we follow the second approach. But instead of considering enumerable poses, we propose to classify all poses into only two groups.

CR IP T

4

III. P ROBLEM D ESCRIPTION

AC

Indoor corner recognition is to detect whether a corner exists within a pedestrian movement trajectory. In building a crowdsourcing-based indoor localization system, corner recognition is the first step to match a movement trajectory from the signal space to the physical space. As illustrated in Fig.1, there are several crowdsourced movement trajectories. For ease of presentation, we plot a trajectory as a dot-marked poly-line segment. In this paper, a true corner is defined as a physical corner formed by corridors or obstacles. As an abstraction counterexample, we also define a fake corner as a false detected corner while a user is actually not passing a ^ F ^ true corner. In Fig. 1, each of the trajectories CDE, GH, ^ N P Q contains a true corner, while each of the trajectories g IJK, g contains no corner or a fake corner. ] LM AB, We use the most common smartphone sensors, including accelerometer, gyroscope and magnetometer, to obtain the raw

measurements along with each trajectory for corner recognition. An accelerometer and gyroscope measure the acceleration and the angular velocity in the device coordinate system, respectively. While a magnetometer measures the device direction in the magnetic field. Notice that we can separate two components from the accelerometer measurements, i.e, the linear acceleration and the gravity. Furthermore, we can obtain the digital compass readings to measure the device direction in the earth coordinate system from the accelerometer and gyroscope measurements. Therefore, the sensor measurements used for corner recognition include the linear acceleration, gravity, gyroscope, digital compass, and magnetometer. For a trajectory, we use a sliding window to divide it into a series of consecutive windows. A window may contain a true corner or not. Our objective is to use the sensor measurements to detect whether a trajectory window contains a true corner. However, due to the measurement noise and the diversity of walking pattern, the corner recognition from crowdsourced trajectories faces the following two main challenges: One is the fake corner problem, and another is the the pose diversity problem. The fake corner problem may come from the fact that a person makes a turning or even turnaround without actually passing a true corner. As shown in Fig.2, the gyroscope and ^ ] in compass measurements of two trajectories F GH, and IJK ] contains no true Fig.1 look very similar. The trajectory IJK corner, however, a window could be falsely detected due to the signal similarity. Many previous work using peak detection by gyroscope or the variation of two neighboring windows in compass would lead to false detection in such cases. The pose diversity problem is especially serious in crowdsourced trajectories, as we cannot mandate a fixed pose for a person when collecting trajectory measurements. According to (Park et al., 2013), four typical poses for holding a smartphone are generally considered, including texting, phoning, swinging, and pocket. Fig. 3 plots the gyroscope and compass

ACCEPTED MANUSCRIPT 5

TABLE I C OMPARISON OF I NDOOR C ORNER R ECOGNITION A LGORITHMS Method of Indoor Corner Recognition (Zhou et al., 2015a) ALIMC (Zhou et al., 2015b) APFiLoc (Shang et al., 2015), (Gu et al., 2016a) (Chen et al., 2015) (Wang et al., 2016b) (Lan and Shih, 2014) WicLoc (Niu et al., 2015)

Consider Pose Diversity No No No No No No No Yes, grouping poses

Consider Fake Corner No No No No No Yes, filtered by indoor map No Yes, training different classifiers with feature selection

AN US

CR IP T

Our Proposed Method

Sensors in use gyroscope, compass gyroscope gyroscope, compass gyroscope gyroscope, compass gyroscope Wi-Fi RSS gyroscope, accelerometer, magnetometer

Fig. 2. Illustration of the signal similarity between a turning event passing a true corner (left), and a turning event without passing a true corner (right).

CE

PT

ED

M

measurements of the trajectories passing by a same true corner but with different poses. It can be seen that they are quite different when using different poses. Many previous corner recognition approaches (Park et al., 2013; Zhou et al., 2015a,b; Shang et al., 2015) have not considered the pose diversity problem, resulting in poor recognition performance in such cases. In this paper, our objective is to design a robust indoor corner recognition system, trying to solve the fake corner and pose diversity problem.

IV. T HE P ROPOSED C ORNER R ECOGNITION S YSTEM

AC

Fig. 4 presents the proposed system architecture, which consists of four parts: data sensing and preprocessing, feature extraction, pose classification and corner recognition. From the raw sensor measurements, we first obtain the time series data for linear acceleration, gravity, gyroscope, digital compass and magnetometer. Data preprocessing is to filter out noises and perform window segmentation. From the window series, we next perform feature extraction and obtain in total 440 features from both the time and frequency domain. Instead of using all features to train one corner classifier, we propose to first construct a pose classifier to group different poses into two groups, namely, fixed pose group and unfixed pose group. For each pose group, we then train one corner classifier accordingly. Note that we build in total three classifiers each trained by different features.

Fig. 4. The proposed system architecture for corner recognition.

A. Data preprocessing The raw data from smartphone sensors usually contains much noises, such as the electromagnetic measurement noises and random environment noises. We first use a median filter and a first order low-pass Butterworth filter. The median filter is used to filter out the isolated point which is greatly different from the next and the previous point in a series. The Butterworth filter is to reduce the random noises in high frequency. According to (Khusainov et al., 2013), the energy spectrum of human body motion lies mainly within the range of 0Hz to 10Hz, so we set the cutoff frequency as 10Hz. After data filtering, let the tuple < L, G, R, C, M > represent the data series of linear acceleration, gravity, gyroscope, digital compass, and magnetometer measurements for one trajectory. Notice that each of them contains three axial components. For example, the linear acceleration L can be → − → − → − → − further written as L = ( L x , L y , L z ), where L x is its x-axis data series. Therefore, there are in total 15 data series from sensor measurements.

ACCEPTED MANUSCRIPT

AN US

CR IP T

6

Fig. 3. The sensor signals when passing a same true corner yet with different poses. (a) holding, (b) phoning, (c) swinging, and (c) putting in trouser pocket. The red/blue/green curve represents the x/y/z-axis data series. It can be observed that the signals are with quite distinguishable characteristics in the holding and phoning pose, such as the peak in one gyroscope axis and a steep drop in one digital compass axis; While in the later two poses, such characteristics are not prominent, which may lead to undetected true corners.

TABLE II E LEMENTS OF AN EXTRACTED WINDOW FEATURE

Dimension (a × b × c) 3 × 15 × 1 = 45 3 × 15 × 1 = 45 2 × 15 × 1 = 30 4 × 5 × 1 = 20 1 × 15 × 10 = 150 1 × 15 × 10 = 150

M

Series L, G, R, C, M L, G, R, C, M L, G, R, C, M L, G, R, C, M L, G, R, C, M L, G, R, C, M

ED

Features MAX, MIN, MEAN VAR, RMS, MAD DIFF, RANGE SMA, ρxy , ρxz , ρyz AR coefficients α Energy

AC

CE

PT

We then use a sliding window with overlap to generate data segments for each of the fifteen data series. Considering that a user normally needs about two seconds to pass through a true corner, we set each window size of 2s. And two consecutive windows have an 50% overlap. We note that such data segmentation has been successfully applied in human activity recognition (D.Lara and Labrador, 2013; Shoaib et al., 2015). For ease of presentation, we use W =< W1 , W2 , ..., WK > to denote the window series. Notice that each window W ∈ W could denote one source of the filtered sensor data series. Each window W contains n data samples, W =< w1 , w2 , ..., wn >. Our feature extraction is conducted on these n data samples for each window and for each source of sensor data series. B. Feature extraction Feature extraction tries to extract relevant information and obtain quantitative measures for window comparison. For each window W , we consider to extract its features from both time domain and frequency domain. Table. II summarizes the extracted elements to compose a window feature. Notice that in the table, the Dimension (a × b × c) indicates the total

number of features in this row, where a represents the number of features in this row, b the source types of each feature in this row and c the dimensionality of each feature in this row. A full feature vector of one window contains 440 features. In the time domain, the feature elements include the minimum value (MIN), maximum value (MAX), mean value (MEAN), variance (VAR), root mean square (RMS), difference of window end sample values (DIFF), range of window sample values (RANGE), mean absolute deviation (MAD), signal magnitude area (SMA), correlation coefficient in between axis (ρ), autoregressive model coefficients (α). In the frequency domain, the features mainly include fast Fourier transform (FFT) energy. We next provide some discussions about the features. The features of MIN, MAX, MEAN and RMS are commonly used central tendency measures, and RMS is computed by v u n u1 X RM S = t w2 . (1) n i=1 i The feature DIFF is used to describe the change of the last and first sample value in a window: DIF F = |wn − w1 |,

(2)

where wn and w1 is the last and the first sample value of a window, respectively. The feature RANGE describes the value span of a window, which is defined as RAN GE = M AX − M IN.

(3)

The features of VAR and MAD are dispersion metrics, and

ACCEPTED MANUSCRIPT 7

v u u M AD = t

n

1 X |wi − w|, ¯ n − 1 i=1

(4)

where w ¯ is the mean value of W . Notice that for one source of sensor data, it contains triaxial components. For one data source, we use xi , yi and zi to denote the corresponding element in the three axis windows, respectively. The feature SMA is a statistical measure of the magnitude of a varying quantity, which is computed by SM A =

n X (|xi | + |yi | + |zi |).

(5)

i=1

AN US

The correlation coefficient ρxy is used to measure the correlation between two data series, which is computed by Pn (xi − x ¯)(yi − y¯) , (6) ρxy = i=1 σx σy

1) Feature selection for pose Classification: In this paper, pose is referred to as the means of a person carrying his smartphone while walking. The inertial sensors are sensitive to the smartphone displacement and rotation. A slight pose change may result in significant changes in the inertial sensors’ signals. Most previous corner detection algorithms assume a fixed pose in experiments without considering the pose diversity problem. On the other hand, there are lots of different poses and even more pose switches in practice, which makes it difficult to enumerate all of them for experiments. Therefore, we need to classify different poses into a few groups. In our experiments, we have collected many crowdsourced trajectories with different poses. From these trajectories, we observed that if a smartphone is held in a fixed position relative to a human upper body, then the sensor rotation could be mainly attributed to passing by a true corner. In such cases, we can use a subset yet important features to improve corner detection performance. On the other hand, when not passing by a true corner, some poses may cause sensor rotation and generate similar signals as if passing a true corner. In such cases, care must be taken to avoid false corner detections. Therefore, we classify a window into either a fixed or an unfixed pose group in this paper. Typical fixed poses include holding, phoning and fastening. The holding pose refers to that a person holds a smartphone in front of his body for reading and texting; The phoning pose refers to that a user holds a smartphone close to his head; The fastening pose refers to that a smartphone is fastened to somewhere in the upper body, such as being tightly placed in a waist pack or a shirt pocket. There may exist other poses that could be classified into the fixed pose group, as long as they can be regarded as being placed in a fixed position relative to a human upper body. Some general unfixed poses include swinging, rotating and loosening. The swinging pose refers to that a user carries a smartphone in hand and wing naturally in a side of body while walking; The rotating pose refers to that a person rotates his smartphone; The loosening pose is the contrary to the fastening pose, where a smartphone is placed somewhere in the lower body, such as in a trouser pocket, with some degree of freedom for displacement and rotation relative to the upper body. Furthermore, those pose switches, i.e., switching from one pose to another, are also considered as unfixed poses. We use the supervised learning approach to design a pose classifier. Instead of using all the 440 features, we propose to select a subset of them that can describe most of differences in between two pose groups to reduce computation complexity. Recall that the core difference between the two pose groups is the smartphone displacement and rotation relative to upper body while walking. Such displacements and rotations would influence variation degree of acceleration, angular velocity and azimuth angle, which may introduce undesirable violent signal fluctuation even when walking straight. So we first select the VAR, MAD, SMA and RMS features to as measures for these physical quantity changes. Furthermore, smartphone rotations relative to upper body would generate some degree of extra centripetal acceleration in accelerometer signals and introduce undesirable influences on the linear acceleration and

CR IP T

MAD is computed by

PT

ED

M

where x ¯ and σx are the mean and standard deviation of the x-axis data window. The AR model utilizes the history of a signal to extract important information hidden in the signal. According to the AR model, each sample of the series can be predicted as a weighted sum of the previous sample values of the same series plus an error term. We select Levinson-Durbin Algorithm (Kay and Marple, 1981) to calculate the AR model coefficient, and set the order as 10. In the frequency domain, we mainly consider the fast Fourier transform (FFT) energy feature. Since we use a Butterworth filter with 10Hz cutoff frequency, we compute a 10 × 1 energy feature. Besides the direct-current component, we calculate the other nine FFT energy components in the range between 0.5Hz to 9.5Hz with a step of 1Hz. Let fi denote the ith value in the FFT spectrum, and l and m denote the start and end index of a frequency bin. The jth energy component is computed by m

X 1 fi2 , l−m+1

CE Ej =

(7)

i=l

AC

Therefore, the energy feature E = (E1 , ..., E10 ) contains ten elements. Although a window contains 440 features with almost all relevant information ready for corner recognition, some features may be more significant than the others and should be discriminatively used. Furthermore, to deal with the pose diversity problem, we may need to first classify the pose types and design different corner detectors accordingly. C. Pose classification and corner recognition Our corner recognition subsystem contains two parts: pose classification and corner detection. We first classify different poses into two groups: fixed and unfixed; and then for each pose group we train a corner detector.

ACCEPTED MANUSCRIPT 8

TABLE III S ELECTED P OSE F EATURES Series L, G, R, C, M L, G, R L, G L, G, R, C, M L, G, R, C, M

Dimension (a × b × c) 1×5×1=5 1×9×1=9 1×6×1=6 2 × 15 × 1 = 30 1 × 15 × 1 = 15

Series − → R ts − → − → M, R ts , C x − → M, R ts − → M, C x

Dimension (a × b × c) 5×1×1=5 2 × 5 × 1 = 10 1×4×1=4 1×4×1=4

data axismax,i is defined as follows (Park et al., 2013): axismax,i = arg max(accx,i , accy,i , accz,i ).

(8)

When a user is making a turn, the angular velocity around the turning axis can be used to reflect the direction of gravity. → − So we compose the gyroscope turning data series R ts by choosing the turning axis gyroscope data from its original triaxial data series. Furthermore, when a user is making a turn, his horizontal direction normally experiences significant changes. Since the x-axis data in compass measures the horizontal direction in the earth coordinate system, so we also use → − the compass x-axis series C x for feature extraction. Finally, when a user is making a turn, each axis of magnetometer measurements on earth magnetic field will experience significant changes in the smartphone coordinate system. So we also use − → − → − → the triaxial magnetometer data series M = (M x , M y , M x ) for feature extraction. The turning detection proposed in (Zhou et al., 2015a,b; Shang et al., 2015) are based on gyroscope peak detection → − and compass variation comparison from the data series R ts → − and C x , respectively. Unlike these algorithms, we propose to extract time domain features from the selected data series → − → − R ts , C x and M to train a F-detector. Table IV summarizes the selected features, and in total 23 features are selected.

AC

CE

PT

ED

M

AN US

gravity signals. So we next select the MEAN feature for linear acceleration and gravity. In some unfixed poses like swing, all signals of accelerometer, gyroscope and magnetometer will fluctuate periodically while a person walking step by step. In general, the period of arm swing is around two steps, and a person may go one to three steps each second in average. According to the periodicity of human walking, we reduce the ten dimension FFT energy feature to only one dimension and calculate the energy contained in between 0.5Hz and 3.5Hz. Table III summarizes the selected features for pose classification, where the L, G, C, R, M denote the linear acceleration, gravity, compass, gyroscope angular velocity and magnetometer, respectively. We select in total 65 features and use them to train a pose classifier. 2) Feature selection for corner detection: After pose group classification, we need to further detect whether a window contains a corner. In this paper, we propose to train two corner detectors each for one pose group. If a window is classified as a fixed pose, we use a F-detector for corner detection; While for an unfixed pose window, we use a U-detector. For U-detector, the training data set consists of only unfixed pose windows. In an unfixed pose window, since the useful signal changes caused by passing a corner might be obscured due to sensor displacement and rotation relative to upper body, it is difficult to discriminate which features are more significant for corner detection. So we decide to use all the 440 features to train a U-detector. For F-detector, the training data set consists of only fixed pose windows. Unlike U-detector, we need to select appropriate features for training a F-detector. This is because with very few or even without sensor displacements and rotations relative to upper body, the characteristics of signal change due to passing a corner can be easily captured. This is also the design principle for many existing corner detectors based on gyroscope peak detection and compass variation comparison(Zhou et al., 2015a,b; Shang et al., 2015). However, observed from our experiment results, we found that these detectors are not robust enough in practice. So we propose to select the following features for training a F-detector. Notice that passing a corner also corresponds to a turning activity in a trajectory. So we first apply the turning detection approach proposed in (Zhou et al., 2015a,b; Shang et al., 2015) to compose a new gyroscope turning data series for each data window based on the turning axis detection. The turning axis is defined as the axis mostly affected by gravity changes, which can be obtained from the acceleration data series. Let (accx,i , accy,i , accz,i ) denote the ith triaxial data of the acceleration data series. The turning axis for the ith

Features SMA, RMS, Mean, Max, Min Var, MAD RANGE DIFF

CR IP T

Features SMA RMS MEAN VAR, MAD Energy 0.5Hz-3.5Hz

TABLE IV S ELECTED FEATURES TO TRAIN A F- DETECTOR

3) Classifier Selection: For supervised learning, the commonly used classifiers include the decision tree, Bayesian approach, nearest neighbor, and support vector machine (SVM), and etc (Tan et al., 2005). In this paper, we choose the decision tree approach. We build a hierarchical tree-like decision model with nodes and edges, respectively, representing features and feature significance. Each branch from the root to a leaf node is a classification rule. The mature implementation of C4.5 decision tree in WEKA has been widely used in the area of human activity recognition (D.Lara and Labrador, 2013; Shoaib et al., 2015). In this paper, we use J48 decision tree, a version of C4.5 for our classifier design with the following considerations: (i) The decision tree model has excellent descriptiveness and explanation, which can help for manual analysis; (ii) The computation complexity is relatively low, which can be easily applied to smartphone platform. The training computation complexity is O(mn log n), and the classification complexity is O(log n), where m is the number of training windows and n the number of features; (iii) The decision tree has good capability of addressing unbalanced data problem.

ACCEPTED MANUSCRIPT 9

TABLE V T HE DETAILS ABOUT DATA COLLECTION .

TABLE VI T HE PARTITION OF DATA S ET Pose Type (Window) Fixed Uixed 816 821 1894 1910

Corner Type(Window) Positive Negative 308 1329 347 3457

V. E XPERIMENT R ESULTS A. Experiment Setup

CE

PT

ED

M

The motivation of our research is to address the fake corner and pose diversity problem in corner recognition. So we collect in total 9 kinds of walking trajectories each along with a selected pose. We consider four main poses: the texting and phoning poses are labeled as fixed pose, and swinging and trouser pocket are labeled as unfixed pose. We also consider trajectories along with pose switches to be labeled as unfixed pose. For data collection, we used our Android APP implemented in a Meizu MX5 smartphone. The sensor data include accelerometer, gyroscope, compass and magnetometer, and the sampling rate is set as 50Hz for all sensors. During data collection, we also use a second chronograph to record the time span passing a true corner. Fig. 1 illustrates some kinds of trajectories in our data. g is the one going straight, F ^ The trajectory AB GH contains ^ ^ a corridor corner, CDE and N P Q are trajectories going through a corner consisted of some obstacles and doors, reg ] contains a turnaround and LM spectively. The trajectory IJK is an arc-shaped turning in open area. In total, we collect 284 trajectories, and use T 1 ∼ T 9 to label these trajectory kinds. After data preprocessing, each trajectory will be segmented into consecutive data windows. For a data window, we first label it as either a fixed pose window or an unfixed one based on our collection record. For a data window, we also label it as a positive window, i.e., containing a true corner, if more than half of the window are within the time span passing a true corner. Otherwise, a window is labeled as a negative one. More details about the trajectories are presented in Table V. From all trajectories, we compose two data sets: P and PN. The data set P contains in total 144 trajectories from the trajectory kinds T 1 ∼ T 4, where each trajectory passes a true corner. There are in total 1637 windows in P. For pose labeling, 816 windows are labeled as fixed pose windows, and the rest 821 are unfixed pose windows; For corner labeling,

AC

Total Contain Corners? Yes Yes Yes Yes No No No No Portion -

Total (Window) 1631 3804

Trajectory Number 144 140

AN US

P PN

Trajectory Class T1-T4 T1-T9

Trajectories 48 48 32 16 32 32 24 20 32 284

CR IP T

(T1) Pass Left Corner (T2) Pass Right Corner (T3) Pass Indoor Obstacle Corner (T4) Pass Indoor-outdoor Corner (T5) Walk Straight (T6) Walk Straight Turnaround (T7) Walk Straight + Device Rotation (T8) Walk Straight + Pose Switch (T9) Walk in Arc-shaped in Open Area Total

Trajectory Number of Different Pose Texting Phoning Swing Pocket Other 12 12 12 12 0 12 12 12 12 0 8 8 8 8 0 4 4 4 4 0 8 8 8 8 0 8 8 8 8 0 24 0 0 0 0 0 0 0 0 20 8 8 8 8 0 84 60 60 60 20

308 windows are labeled positive windows containing a true corner, and the rest 1329 are negative windows. The data set PN includes all 284 trajectories from the trajectory kinds T 1 ∼ T 9, which is close to real trajectory crowdsourcing conditions. There are in total 3804 windows in PN. For pose labeling, 1894 windows are labeled as fixed pose windows, and the rest 1910 are unfixed pose windows; For corner labeling, 347 windows are labeled as positive windows, and the rest 3457 are negative windows. Details of the two data sets are summarized in Table VI. Considering that our scheme consists of both pose classification and corner detection, our experiments are designed as follows: We first evaluate and compare our pose classification with a state-of-the-art algorithm; We then investigate the corner recognition performance for our scheme with different settings; Finally, we compare our scheme with other state-ofthe-art corner detection algorithms. For all experiments, we use 10-fold cross validations. B. Pose classification comparison We first compare the performance of our pose classification with a state-of-the-art algorithm for motion mode classification, which classify smartphone motion into either a symmetric or an asymmetric mode by using an energy detector to compare acceleration and angular periodicity (Xiao et al., 2015). For our data collection, the fixed pose group consists of only the texting and phoning pose, which can correspond to the symmetric motion mode. The rest unfixed poses correspond to the asymmetric motion mode. We implement an energy detector and compare the FFT energy Ea of acceleration magnitude and Eω of angular velocity in turning axis between 1.0Hz and 2.5Hz. If Ea ≥ αEω , the motion mode is classified as a symmetric mode; Otherwise, an asymmetric mode. We set α = 20 through optimal tuning in our experiments. Since the two labeled pose groups have similar number of windows, we use the accuracy for comparing pose classifica-

ACCEPTED MANUSCRIPT

CR IP T

10

Fig. 6. Illustration of corner recognition results. A rectangle refers to a window.

TABLE VII C ONFUSION MATRIX OF THE POSE GROUP CLASSIFICATION Fixed pose group 795 22 Fixed pose group 1837 62

Unfixed pose group 21 799 Unfixed pose group 57 1848

tion performance, which is defined as

M

Data set: P Fixed pose group Unfixed pose group Data set: PN Fixed pose group Unfixed pose group

AN US

Fig. 5. Result of Pose Group Classification.

T , (9) N where T is the number of the windows being correctly classified and N the number of all windows. Table VII provides the confusion matrices for the two data sets by using our proposed pose classification. Out approach can achieve 97.5% accuracy for dataset P and 96.9% accuracy for data set PN. Fig. 5 compares the accuracy of our proposed pose classification with the FFT-based energy detector (Xiao et al., 2015). For the energy detector with 2s window size, the accuracy is 77.7% and 76.3% for the data set P and PN, respectively. Ours can achieve 20.2% improvement in average. We also notice that the performance of the energy detector is also dependent on the window size. From our experiments, it achieves the best accuracy performance by using a window size of 4s. However, the performance is poorer than ours. Furthermore, using a larger window will introduce the delay problem, which may not be appropriate for capturing pose changes.

AC

CE

PT

ED

Accuracy =

being trained with different features. One decision tree first classifies a window to either the fixed or the unfixed pose group; After the pose classification, then two decision trees are used for corner detection, each for one pose group. One of the variant is named as P C + 3DT , which also construct three decision trees. But different from F S + P C + 3DT , all three decision trees use the the full 440 dimension features, i.e., without feature selection. The other variant is named as 1DT only, which uses the full 440 dimension features to train only one decision tree for corner detection, i.e., without feature selection and without pose classification. For a trajectory containing a true corner, we note that passing a corner normally requires around two seconds. Since our data segmentation uses a window of 2s and with 50% overlap, one window or at most three consecutive windows would be labeled as positive when passing a true corner. If one of such positive window is detected, we say that a true corner is recognized correctly, i.e., a true positive (TP) occurs. On the contrary, if none of such positive windows labeled for one corner is recognized, we say that a true corner is not recognized, i.e., a false negative (FN) occurs. Furthermore, if one or some consecutive unlabeled negative windows are recognized as a true corner, then a false positive (FP) occurs. Fig. 6 illustrates such cases of TP, FN and FP. Since the negative windows are much more than the positive ones, we adopt the precision, recall and F1-measure as performance metrics for corner recognition as follows:

C. Corner recognition performance The proposed scheme consists of feature selection, decision tree-based pose classification and corner detection. To verify the effect of the feature selection and pose classification, we compare the proposed scheme with two variants of our the proposed scheme. We name the original proposed scheme as F S + P C + 3DT , where we use three decision trees each

precision = recall = F 1 − measure =

TP TP + FP

TP TP + FN

2 ∗ recall ∗ precision recall + precision

(10) (11) (12)

Table VIII presents the experiment results for the two data sets. We have the following observations: • For all schemes and in both data sets, the corner recognition performance for the fixed pose group is better than their respective peers for the unfixed pose group. This is not unexpected. As discussed before, the fixed pose group contains those poses considered as fixed relative to upper body. Therefore, some prominent signal characteristics

ACCEPTED MANUSCRIPT 11

TABLE VIII C ORNER RECOGNITION PERFORMANCE COMPARISON FOR DIFFERENT SCHEME SETTINGS .

P

PN

+ PC + 3DT recall F1-measure 1.000 1.000 1.000 1.000 1.000 1.000 0.861 0.785 0.972 0.854 0.917 0.820 0.958 0.905 0.975 0.918 1.000 0.920 0.988 0.919 0.750 0.606 0.750 0.645 0.750 0.577 0.869 0.732

precision 0.875 0.818 0.845 0.689 0.761 0.725 0.783 0.771 0.816 0.794 0.508 0.566 0.472 0.612

ED

AC

CE

PT

•

PC + 3DT recall F1-measure 0.972 0.921 1.000 0.900 0.986 0.910 0.861 0.765 0.972 0.854 0.917 0.810 0.951 0.859 0.925 0.841 1.000 0.899 0.963 0.870 0.750 0.606 0.750 0.645 0.750 0.580 0.856 0.714

D. Comparison with other state-of-the-art algorithms In this subsection, we compare our scheme with other stateof-the-art algorithms, namely, the ActSeq (Zhou et al., 2015a), ALIMC (Zhou et al., 2015b), and APFiLoc algorithm (Shang et al., 2015). The three algorithms use gyroscope peak detection and compass variation comparison for corner detection. → − Recall that we use R ts to denote the gyroscope turning axis → − data series and C x the digital compass x-axis data series. The ALIMC algorithm first determines the peak data samples → − → − in R ts . The ith data in R ts is defined as a peak sample p ri , if |ri | > |ri−1 | and |ri | > |ri+1 |. Then the average value of peak samples in the jth window Wj is computed

precision 0.923 0.857 0.889 0.679 0.660 0.670 0.766 0.673 0.796 0.733 0.508 0.549 0.492 0.601

1DT only recall F1-measure 1.000 0.960 1.000 0.923 1.000 0.941 1.000 0.809 0.917 0.767 0.958 0.789 0.979 0.860 0.875 0.761 0.975 0.876 0.925 0.818 0.800 0.621 0.700 0.615 0.750 0.594 0.838 0.700

by r¯j = mean(rip ), rip ∈ Wj . A peak sample rip ∈ Wj is called a corner peak, if |rip | > rthres and |rip | ≥ α¯ rj . → − In R ts , if any one corner peak is within the labeled corner span, then a corner is detected. In our experiments, we set rthres = 60 degree/s and α = 0.6 to obtain its best corner detection performance. Besides the above gyroscope peak detection, the APFiLoc (Shang et al., 2015) further uses compass variation comparison to refine the turn detection → − result. Let cxi denote the ith data sample in C x . It first computes the mean azimuth angle of the jth window by cxj = mean(cxi ), cxi ∈ Wj . If |cxj −cxj−1 | ≥ 30o , a turn is then claimed, which is regarded as a corner in our experiments. The ActSeq (Zhou et al., 2015a) also uses the above gyroscope peak detection and compass variation comparison for turn detection. However, the ActSeq (Zhou et al., 2015a) further uses compass variation comparison to distinguish a U-turn. If |cxj − cxj−1 | ≥ 135o , a U-turn is detected, which is not regarded as a corner in our experiments. Fig. 7 compares the proposed scheme with the these stateof-the-art algorithms. • We first examine their capability of dealing with the pose diversity problem by the data set P, which contains simple trajectories without fake corners. From Fig. 7(a), we can observe that all algorithms can achieve high recognition precision for the fixed pose group. But for the unfixed pose group, only ours can achieve high precision, and the others perform very poorly. This is not unexpected as these algorithms did not take pose diversity into design consideration, which would limit their applications in practice. Note that although the ALMIC has higher recall performance, its overall performance by F1-measure is still very unsatisfiable due to its poor recognition precision. • We next examine their capability of dealing with the fake corner problem by considering only the fixed pose group in both data sets. Recall that the data set PN contains more complicated trajectories with many potential fake corners. As shown in Fig. 7(b), all algorithms suffer some performance degradation in data set PN. This indicates that even when taking the fixed pose, the fake corner problem could much impact on the recognition

AN US

can be observed, when passing a true corner by taking a fixed pose. But such signal characteristics would be much obscured by taking unfixed poses. For all schemes, the average corner recognition performance of the data set P is better than that of the data set PN. Recall that the data set PN contains more complicated trajectories with many potential fake corners due to turns or even turnarounds in open spaces. Such turns in open spaces would impact the corner recognition performance for both fixed pose and unfixed pose groups. We note that our scheme can help to relieve the fake corner problem, as also to be compared with other state-of-the-art corner recognition algorithms in the next subsection. Yet how to minimize the fake corner impacts needs further study in our future work. Among the three schemes, our proposed one achieves the best corner recognition performance in both data sets for either pose-wise comparison or in-average comparison. This validates the effectiveness of using pose classification before corner recognition and performing feature selection to train different classifiers. As discussed before, the corner recognition should well deal with the pose diversity and fake corner problem. Using a single classifier with all features can well address the pose diversity problem, and using three classifiers without feature selection cannot well differentiate the feature significance when passing a true corner.

M

•

Fixed Holding Pose Phoning Group Subtotal Unfixed Swing Pose Pocket Group Subtotal Total Fixed Holding Pose Phoning Group Subtotal Unfixed Swing Pose Pocket Group Subtotal Total

FS precision 1.000 1.000 1.000 0.721 0.761 0.742 0.857 0.867 0.851 0.859 0.508 0.566 0.469 0.632

CR IP T

Data set

ACCEPTED MANUSCRIPT 12

AN US

CR IP T

(a) Performance comparison for the data set P.

M

(b) Performance comparison for the fixed pose group in both data set P and PN.

ED

(c) Performance comparison for the data set PN. Fig. 7. Corner recognition performance comparison between the proposed algorithm and the state-of-the-art algorithms.

PT

AC

CE

•

performance. However, our proposed algorithm performs much better than the others, which validates its capability of relieve the fake corner problem. We finally examine the overall performance for dealing with both the pose diversity and fake corner problem by the data set PN. Again, it can be observed from Fig. 7(c) that our proposed algorithm achieves the highest precision and F1-measure in all cases. We notice that the overall performance of F1-measure of our algorithm is 73.2%, which still needs further improvements in our future work.

In summary, based on whether containing the pose diversity instances and/or fake corner instances, field measurements can be divided into four groups: (i) only with ideal instances; (ii) with pose diversity instances; (iii) with fake corner instances; (iv) with all instances. The F1-measures of our proposed scheme for the four groups are (i) 100%, (ii) 90.5%, (iii) 91.9% and (iv) 73.2%, respectively. The F1-measures of the best of the algorithms (Zhou et al., 2015a,b; Shang et al., 2015) are (i) 95.0%, (ii) 50.6%, (iii) 64.6% and (iv) 31.1%, respectively. Furthermore, the precisions of our proposed scheme are (i) 100%, (ii) 85.7%, (iii) 85.9%, (iv) 63.2%, respectively. And

the precisions of the algorithms (Zhou et al., 2015a,b; Shang et al., 2015) are (i) 97.1%, (ii) 39.6%, (iii) 50.3%, (iv) 20.1%, respectively.

Indoor landmark detection has shown to play an important role in movement trajectory analysis for indoor localization systems, such as the PDR correction for online pedestrian tracking and the fingerprint annotation for offline radio map construction (Gu et al., 2016b; Luo et al., 2017; Abdelnasser et al., 2015). Although the previous corner detection methods (Zhou et al., 2015a) have reported high precision performance, we argue that they had not fully take into account of the pose diversity and fake corner challenges, which are very common in practical movement scenarios. Our work has treated the corner detection problem from an expert system viewpoint by applying machine learning techniques: Not only the recognition performance can be improved, but also the methodological potential for other indoor landmark detection problems could be expected.

ACCEPTED MANUSCRIPT 13

•

•

AN US

In this paper, we have provided a systematical study on the indoor corner recognition problem from crowdsourced movement trajectories. Two challenges in practical crowdsourcing scenarios have been identified, namely, the fake corner problem and the pose diversity problem. We have proposed a hierarchical corner recognition scheme consisting of three classifiers to solve the two problems. The first pose classifier is to classify various poses into only two pose groups; then for each pose group, we propose to train a corner classifier respectively. Compared with the existing methods that are mainly based on sensor signal peak detection and/or variation comparison, we put the corner detection problem within the framework of expert systems and our method takes a machine learning approach by extracting and selecting multifarious sensor signal features to train different classifiers. Field experiments have shown that our proposed scheme can better deal with the fake corner and pose diversity challenges in terms of much higher F1-measures and precisions than the state-of-the-art algorithms in practical corner recognition scenarios. We note that our solution also inherits the common weakness of supervised learning in that we need to manually label windows and select features. In our future work, we shall try semi-supervised and/or unsupervised learning approaches and investigate more advanced landmark recognition algorithms.

many noises and outliers (Zhang et al., 2015). How to construct and update an indoor fingerprint map from error prone crowdsourced fingerprints deserves further studies (Majeed et al., 2016; Chang et al., 2015). Indoor behavior recognition and tracking: Although human activity recognition is another hot topic being widely researched (Ronao and Cho, 2016; Shoaib et al., 2015), we note that their methodologies and results may also benefit the indoor localization community (Wang et al., 2015c; Bobkov et al., 2015). People may make certain activities at special locations, such as going upstairs or downstairs, resting on a sofa, walking in a corridor and etc. How to construct an indoor expert and intelligent system for human daily behavior tracking and recognition would be an interesting direction for both research communities. Indoor location-based intelligent applications: Indoor localization services shall find many exciting applications for improving user experiences, such as location-based advertising, point-of-interest recommendation, and indoor crowd analysis and so on (Yu, 2016; Tuan et al., 2017; Tsai et al., 2017; Ilarri et al., 2015). How to provide and integrate real-time location service to other intelligent applications deserves more research and engineering efforts.

CR IP T

VI. C ONCLUSIONS AND D ISCUSSIONS A. Conclusions

M

B. Discussions

AC

CE

PT

ED

We first discuss the link between our work and expert systems. We note that the fingerprint-based indoor localization system resembles a lot to a knowledge-based expert system. We need to first establish a knowledge base, i.e., indoor fingerprint map, for online positioning. For how to construct the knowledge base, site survey might be regarded as a supervised learning method; while crowdsourcing could be regarded as an unsupervised or semi-supervised learning. The online positioning algorithms are actually some similarity function to compute the similarity between an online fingerprint with those offline ones in the fingerprint map. The landmark detection is a typical pattern classification problem, which is to detect and label whether a data window from a time series contains some true indoor landmark. Therefore, we adopt and improve the widely used decision tree as our classifiers. We next would like to point out some potential future research directions: • Indoor landmark series detection: A movement trajectory may contain one or more landmarks of different types. More advanced algorithms for classifying various landmarks with high precision and robustness need to be further investigated. Besides single landmark detection, how to detect consecutive landmarks in a movement trajectory is also worth of further study. • Indoor fingerprint map construction and management: Although movement trajectory analysis helps to construct an indoor fingerprint map, the crowdsourced fingerprints may be not accurate enough, even containing

ACKNOWLEDGEMENT

This work is partly supported by the National Natural Science Foundation of China (Grant No. 61371141) and the Fundamental Research Funds for the Central Universities (No. HUST2015QN081). R EFERENCES

Abdelnasser, H., Mohamed, R., Elgohary, A., Alzantot, M. F., Wang, H., Sen, S., Choudhury, R. R., and Youssef, M. (2015). Semanticslam: Using environment landmarks for unsupervised indoor localization. IEEE Transactions on Mobile Computing, 15(7):1770–1782. Anjum, A. and Ilyas, M. (2013). Activity recognition using smartphone sensors. the 2013 IEEE Consumer Communications and Networking Conference (CCNC), pages 914–919. Bobkov, D., Grimm, F., Steinbach, E., Hilsenbeck, S., and Schroth, G. (2015). Activity recognition on handheld devices for pedestrian indoor navigation. 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN). Calderoni, L., Ferrara, M., Franco, A., and Maio, D. (2015). Indoor localization in a hospital environment using random forest classifiers. Expert Systems with Applications (Elsevier), 42(1):125–134. Chang, L., Chen, X., Wang, J., Fang, D., Liu, C., Tang, Z., and Nie, W. (2015). Talc: Time adaptive indoor localization with little cost. ACM/IEEE International Conference on Mobile Computing and Networking, pages 49–54. Chen, Z., Zou, H., Jiang, H., Zhu, Q., Soh, Y., and Xie, L. (2015). Fusion of wifi, smartphone sensors and landmarks using the kalman filter for indoor localization. Sensors (MDPI), 15(1):715– 732. Chiu, W.-Y., Chen, B.-S., and Yang, C.-Y. (2012). Robust relative location estimation in wireless sensor networks with inexact position problems. IEEE Transactions on Mobile Computing, 11(6):935– 946. D.Lara, O. and Labrador, M. A. (2013). A survey on human activity recognition using wearable sensors. IEEE Communications Surveys and Tutorials, 15(3):1192–1209.

ACCEPTED MANUSCRIPT 14

CR IP T

64(1):434–443. Shahidi, S. and Valaee, S. (2015). Hidden markov model based graph matching for calibration of localization maps. IEEE International Conference on Communications (ICC), pages 4606–4611. Shang, J., Gu, F., Hu, X., and Kealy, A. (2015). Apfiloc: An infrastructure-free indoor localization method fusing smartphone inertial sensors, landmarks and map information. Sensors (MDPI), 15(10):27251–27272. Shoaib, M., Bosch, S., Incel, O. D., Scholten, H., and Havinga, P. J. (2015). A survey of online activity recognition using mobile phones. Sensors (MDPI), 15:2059–2085. Siirtola, P. and Roning, J. (2013). Ready-to-use activity recognition for smartphones. the 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), pages 59–64. Tan, P.-N., Steinbach, M., and Kumar, V. (2005). Introduction to Data Mining. Addison Wesley. Torres-Sospedra, J., Montoliu, R., Trilles, S., scar Belmonte, and Huerta, J. (2015). Comprehensive analysis of distance and similarity measures for wi-fi fingerprinting indoor positioning systems. Expert Systems with Applications (Elsevier), 42(23):9263–9278. Tsai, C.-Y., Li, M.-H., and Kuo, R. (2017). A shopping behavior prediction system: Considering moving patterns and product characteristics. Computers & Industrial Engineering (Elsevier), 106:192–204. Tuan, C.-C., Hung, C.-F., and Wu, Z.-H. (2017). Collaborative location recommendations with dynamic time periods. Pervasive and Mobile Computing (Elsevier), 35(1):1–14. Wang, B., Chen, Q., Yang, L. T., and Chao, H.-C. (2016a). Indoor smartphone localization via fingerprint crowdsourcing: Challenges and approaches. IEEE Wireless Communications, 23(3):82–89. Wang, B., Zhou, S., Liu, W., and Mo, Y. (2015a). Indoor localization based on curve fitting and location search using received signal strength. IEEE Transactions on Industrial Electronics, 62(1):572– 582. Wang, B., Zhou, S., Yang, L., and Mo, Y. (2015b). Indoor positioning via subarea fingerprinting and surface fitting with received signal strength. Pervasive and Mobile Computing (Elsevier), 23(2):23– 48. Wang, H., Sen, S., Elgohary, A., Farid, M., Youssef, M., and Choudhury, R. R. (2012). No need to war-drive: Unsupervised indoor localization. ACM International Conference on Mobile Systems, Applications, and Services (MobiSys), pages 197–210. Wang, Q., Luo, H., Zhao, F., and Shao, W. (2016b). An indoor selflocalization algorithm using the calibration of the online magnetic fingerprints and indoor landmarks. 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pages 1–8. Wang, W., liu, A. X., Shahzad, M., Ling, K., and Lu, S. (2015c). Understanding and modeling of wifi signal based human activity recognition. Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 65–76. Waqar, W., Chen, Y., and Vardy, A. (2016). Smartphone positioning in sparse wifi environments. Computer Communications (Elsevier), 73:108–117. Wu, C., Yang, Z., and Liu, Y. (2015). Smartphones based crowdsourcing for indoor localization. IEEE Transactions on Mobile Computing, 14(2):444–457. Xiao, Z., Wen, H., Markham, A., and Trigoni, N. (2015). Robust indoor positioning with lifelong learning. IEEE Journal on Selected Areas in Communications, 33(11):2287–2301. Yu, W. (2016). Spatial co-location pattern mining for location-based services. Expert Systems with Applications (Elsevier), 46(15):324– 335. Zhang, L., Valaee, S., Zhang, L., Xu, Y., and Ma, L. (2015). Signal propagation-based outlier reduction technique (sport) for crowdsourcing in indoor localization using fingerprints. 2015 IEEE 26th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), pages 2008–2013. Zhou, B., Li, Q., Mao, Q., Tu, W., and Zhang, X. (2015a). Activity sequence-based indoor pedestrian localization using smartphones.

AC

CE

PT

ED

M

AN US

Gu, F., Kealy, A., Khoshelham, K., and Shang, J. (2016a). Efficient and accurate indoor localization using landmark graphs. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pages 509–514. Gu, F., Khoshelham, K., Shang, J., and Yu, F. (2016b). Sensory landmarks for indoor localization. 2016 4th Internation Conference on Ubiquitous Positioning, Indoor Navigation and Location Based Services (UPINLBS), pages 201–206. Harle, R. (2013). A survey of indoor inertial position systems for pedestrains. IEEE Coummunications Surveys and Tutorials, 15(3):1281–1293. He, S., Chan, S. H. G., Yu, L., and Liu, N. (2015). Calibrationfree fusion of step counter and wireless fingerprints for indoor localization. ACM International Joint Conference on Pervasive and Ubiquitous Computing (UBICOMP), pages 897–908. hoon Jung, S., chul Moon, B., and Han, D. (2016). Unsupervised learning for crowdsourced indoor localization in wireless networks. IEEE Transactions on Mobile Computing, 15(11):2892–2906. Hossain, A. and Soh, W.-S. (2015). A survey of calibration-free indoor positioning systems. Computer Communications (Elsevier), 66(1):1–13. Ilarri, S., Stojanovic, D., and Ray, C. (2015). Semantic management of moving objects: A vision towards smart mobility. Expert Systems with Applications (Elsevier), 42(15):1418–1435. Kay, S. M. and Marple, S. L. (1981). Spectrum analysis-a modern perspective. Proceeding of the IEEE, 69(11):1380–1419. Kefalas, P. and Manolopoulos, Y. (2017). A time-aware spatio-textual recommender system. Expert Systems with Applications (Elsevier), 78(15):396–406. Khan, A. M., Tufail, A., Khattak, A. M., and Laine, T. H. (2014). Activity recognition on smartphones via sensor-fusion and kdabased svms. International Journal of Distributed Sensor Networks (Hindawi), pages 1–14. Khusainov, R., Azzi, D., Achumba, I. E., and Bersch, S. D. (2013). Real-time human ambulation, activity, and physiological monitoring: Taxonomy of issues, techniques, applications, challenges and limitations. Sensors (MDPI), 13(10):12852–12902. Lan, K.-C. and Shih, W.-Y. (2014). An intelligent driver location system for smart parking. Expert Systems with Applications (Elsevier), 41(5):2443–2456. Luo, C., Cheng, L., Chan, M. C., Gu, Y., and Li, J. (2017). Pallas: Self-bootstrapping fine-grained passive indoor localization using wifi monitors. IEEE Transactions on Mobile Computing, 16(2):466–481. Majeed, K., Sorour, S., Alnaffouri, T. Y., and Valaee, S. (2016). Indoor localization and radio map estimation using unsupervised manifold alignment with geometry perturbation. IEEE Transactions on Mobile Computing, 15(11):2794–2808. Martłn, H., Bernardos, A. M., Iglesias, J., and Casar, J. R. (2013). Activity logging using lightweight classification techniques in mobile device. Personal and Ubiquitous Computing (Springer), 17(4):675–695. Niu, J., Wang, B., Cheng, L., and Rodrigues, J. J. P. C. (2015). Wicloc: An indoor localization system based on wifi fingerprints and crowdsourcing. IEEE International Conference on Communications (ICC), pages 3008–3013. Park, K., H.Shin, and H.Cha (2013). Smartphone-based pedestrian tracking in indoor corridor environments. Personal and Ubiquitous Computing (Springer), 17(2):359–370. Rai, A., Chintalapudi, K., Padmanabhan, N., and Sen, R. (2012). Zee: Zero-effort crowdsourcing for indoor localization. ACM International Conference on Mobile Computing and Networking (MobiCom), pages 293–304. Ronao, C. A. and Cho, S.-B. (2016). Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications (Elsevier), 59(15):235–244. Rubio, F., Martłnez-Gmez, J., Flores, M. J., and Puerta, J. M. (2016). Comparison between bayesian network classifiers and svms for semantic localization. Expert Systems with Applications (Elsevier),

ACCEPTED MANUSCRIPT 15

AC

CE

PT

ED

M

AN US

CR IP T

IEEE Transactions on Human-Machine Systems, 45(5):562–574. Zhou, B., Mao, Q., Li, Q., Tu, W., Zhang, X., and Chen, L. (2015b). Alimc: Activity landmark-based indoor mapping via crowdsourcing. IEEE Transactions on Intelligent Transportation Systems, 16(5):2774–2785. Zhou, M., Zhang, Q., Tian, Z., Xu, K., Qiu, F., and Wu, H. (2015c). Imlours: Indoor mapping and localization using time-stamped wlan received signal strength. IEEE Wireless Communications and Networking Conference (WCNC), pages 1817–1822.