Using accelerometer for counting and identifying swimming strokes

Accepted Manuscript Using accelerometer for counting and identifying swimming strokes Meng-Shiuan Pan, Ku-Chen Huang, Tzu-Hsiu Lu, Ze-Yan Lin PII: DOI: Reference:

S1574-1192(16)00023-7 http://dx.doi.org/10.1016/j.pmcj.2016.01.011 PMCJ 680

To appear in:

Pervasive and Mobile Computing

Received date: 8 September 2015 Revised date: 9 December 2015 Accepted date: 29 January 2016 Please cite this article as: M.-S. Pan, K.-C. Huang, T.-H. Lu, Z.-Y. Lin, Using accelerometer for counting and identifying swimming strokes, Pervasive and Mobile Computing (2016), http://dx.doi.org/10.1016/j.pmcj.2016.01.011 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

swimmer.tex Click here to view linked References 1

Using Accelerometer For Counting and Identifying Swimming Strokes Meng-Shiuan Pan, Ku-Chen Huang, Tzu-Hsiu Lu, and Ze-Yan Lin

Abstract Stroke styles and stroke counts are the two important parameters for calculating swimming efficiency indices for swimmers. Assume that swimmers wear accelerometers when swimming. In this paper, we design a swimming analysis scheme, which can distinguish swimming styles and count strokes in realtime. The proposed scheme is composed by a data processing phase and a stroke analysis phase. First, the data processing phase gathers linear acceleration values, eliminates floating data, and finds feature points in the perceived sensory data. According to the data from the data processing phase, the stroke analysis phase identifies the stroke style that the swimmer is performing, and then adopts the concept of correlation coefficient to count strokes. We implement the proposed scheme on a water-proof Android platform. The experiment results indicate that the proposed scheme can identify swim styles and count strokes with high accuracy. Index Terms accelerometer, stroke count, stroke style, swimming stroke analysis.

I. I NTRODUCTION Swimming is a popular exercise. Recently, with the develop of sensing technologies, some commercial products, e.g., Withings activite [1] and Garmin Swim [3], propose to use wearable sensors to calculate distances, consumed calories, speed, stroke frequency, and stroke range for swimmers when they are training. Swimmers can utilize these swimming efficiency indices to review their training procedures and to improve their stroke skills. In the literatures [5][8][12], the above swimming efficiency indices can be derived by swimmers’ stroke styles, stroke counts, and drill times. Since the drill times can be obtained easily, in this work, we design a swimming analysis scheme, which utilizes accelerometers worn on swimmers to distinguish swimming styles and to count strokes in realtime. We first conduct an experiment to motivate our scheme. In this experiment, we observe the changes on linear acceleration sensory data perceived from the accelerometer attached on a swimmer’s right palm. Fig. 1(a) and Fig. 1(b) shows the visual results on the gathered sensory values when the swimmer strokes by front crawl and breaststroke styles, respectively. We can see that the perceived linear acceleration values on X, Y, and Z axes exhibit periodic and regular changes over time. So, we can use this characteristic to derive stroke counts. Moreover, from Fig. 1(a) and Fig. 1(b), the tendencies of different stroke styles are not the same, and thus we can use this characteristic to identify stroke styles. Meng-Shiuan Pan, Tzu-Hsiu Lu, and Ze-Yan Lin are with the Department of Computer Science and Information Engineering, Tamkang University, New Taipei City, Taiwan. e-mail: [email protected], [email protected], and [email protected] Ku-Chen Huang is with the Office of Physical Education, Tamkang University, New Taipei City, Taiwan. e-mail: [email protected] M.-S. Pan’s research is sponsored by MOST Grant 104-2221-E-032-004

2

3

4

5

6

7

8

X

Acc (m/s2)

Y

Acc (m/s2)

Z

Acc (m/s2)

Acc (m/s2)

0 -5 -10 -15 -20 -25 30 20 10 0 -10 -20

Acc (m/s2)

20 10 0 -10 -20

Acc (m/s2)

2

9

10 5 0 -5 -10 -15 10 5 0 -5 -10 20 15 10 5 0 -5

10

X

Y

Z

2

3

4

time (s)

(a) Fig. 1.

5

6 7 time (s)

8

9

10

11

(b)

Example on the perceived linear acceleration values on X, Y, Z axes when stroking by (a) front crawl and (b) breaststroke.

In this work, the proposed swimming analysis scheme contains two phases, named data processing phase and stroke analysis phase. First, the data processing phase involves processing perceived linear acceleration values. This phase adopts a low pass filter to reduce noise signals, and then removes fluctuations caused by floating during a swimming stroke. From the perceived sensory data, this phase further finds feature points, which will be taken as possible start time points of strokes. According to the found feature points, the stroke analysis phase first finds intervals formed by any two feature points. If the correlation coefficient of the sensory values in an interval is high, this interval will be taken as a segment. (In this work, a segment is considered to have two strokes.) After determining the first segment, the stroke analysis phase utilizes the designed procedure to identify the swimmer’s stroke style to be front crawl, breaststroke, butterfly stroke, or backstroke. Then, this phase can continue on finding segments (or say strokes). In this paper, we implement the designed scheme on a water-proof Android smartphone. Our experiment results reveal that the proposed algorithm can effectively identify swimmers’ stroke styles and calculate stroke counts. We also compare the proposed scheme with a commercial swimming watch. The results indicate that the proposed scheme will have higher accuracy. Moreover, the contributions of this work are threefold. First, compare to existing classification schemes, the proposed scheme can identify stroke styles with less computation complexity, less training overheads, and higher accuracy. Second, the proposed stroke analysis phase can adapt to irregularity on swimmers’ postures, and can effectively count strokes for recreational and elite swimmers. Third, the proposed scheme can be implemented on wearable devices and can be extended to other sports, e.g., cycling or running. The rest of this paper is organized as follows. Section II describes some related works. Section III gives an overview on the proposed swimming analysis scheme. Then, Section IV and Section V present the designed data processing phase and stroke analysis phase, respectively. Section VI shows our prototyping and experiment results. Finally, we conclude this paper in Section VII.

3

II. R ELATED WORKS Recently, many studies (e.g., [6][14][16][18]) propose to infer users’ activities or postures by placing sensors on users’ hands or trunks. These schemes are not suitable for applying to analyze stroke styles and stroke counts since they need a lot of computations or a complicated training procedure. In references [10][19], the authors propose to adopt inertial sensors for swimming analysis. The proposed schemes aim to analyze stroke stages in each of strokes. They do not discuss how to count number of strokes, and can only work offline. In reference [15], the authors use video analysis and inertial sensors to calculate stroke counts. In the proposed scheme, inertial sensors report acceleration sensory values, and an underwater camera capture the corresponding timing of each stroke. Then, by the gathered sensory data and the video, the proposed scheme can identify peaks in the collected sensory values, and take the number of peaks as stroke counts. Although the proposed scheme can achieve high accuracy, there are two drawbacks. First, a high resolution underwater camera is expensive. Second, the proposed scheme cannot report stroke counts in realtime. Moreover, in [9], the authors utilize smartphones’ barometers to calculate stroke counts. In this work, smartphones are placed on swimmers’ upper arms. The proposed idea is based on the characteristic that the sensed values of a barometer are different when the sensor is in air or in water. So, a smartphone can count strokes easily by detecting the changes on sensory values from barometers. The proposed scheme can indeed counts the number of strokes with high accuracy, but barometers are not commonly used in mobile devices or wearable devices. In references [7][21], the authors place sensors on swimmers’ wrists or backs. After collecting acceleration sensory values, the proposed schemes detect peak values among the collected sensory data. If the value of an identified peak exceeds a predefined threshold, one stroke will be counted. In reference [20], the authors observe that when swimming, swimmers’ bodies will rotate regularly. The authors gather sensory values from the accelerometer and gyroscope disposed at the side of goggles, and then calculate the changes of angle. A stroke count will be recorded when the derived angle is more than a threshold. Reference [11] counts strokes by the accelerometer placed on swimmers’ backs. The authors observe tendencies on the collected acceleration sensory values, and then propose a stroke count method, which contains the following two stages. First, the proposed scheme finds the peaks of acceleration values on Z axis and records the corresponding time instances. Second, by a recorded time instance, the proposed scheme further checks if the acceleration value on X axis crosses zero at that time. If so, a stroke is identified. We can see above works [7][11][20][21] set threshold values and use peak detection to count strokes. These solutions are easy to implement but how to find a universal threshold for swimmers is a critical issue. For example, according to our observations, a recreational swimmer may have redundant

4 D ata p ro cessin g p h a se D ata collection m odu le G atherlinear acceleration valuesperiodically

F loatin g elim in ation m odu le F eatu re p oin t d etection m odu le

Processed sensory data S tro k e a n aly sis p h a se enable data

feedback

M ain thread stroke v ng xthrea _ di idi d stroke style= _ undefined stroke v ngy threa _ di idi d

yes

stroke scor ng _ i R eportstoke counts

stroke v ng zthrea _ di idi d

Fig. 2.

The flowchart of the designed swimming analysis scheme.

motions when swimming, and thus the gathered sensory values will have many small vibrations. So, if the threshold is set to a low value, it is possible that some vibrations on sensory values can pass the threshold easily. As a result, the stroke counts will be incorrect. In reference [13], the authors place accelerometers on swimmers’ wrists, and propose two algorithms to calculate stroke counts. The first one is designed based on the observation that the maximum and minimum values of the acceleration values on Y axis can be detected at the time instants when the palms enter and leave the water, respectively. So, stroke counts can be obtained by the observed numbers of maximum and minimum sensory values. The second one is to utilize the zero crossing concept. The idea is to mark a start time point as the time instant when a perceived sensory value is crossing zero point. Then, when there is another sensory value crossing the zero point again, a stop time is recorded. The stroke counts are calculated based on the recorded numbers of start and stop time pairs. Reference [17] shows a stroke detection method, which is similar to the second algorithm in [13]. Next, reference [22] proposes to place accelerometers on users’ waist sacrum, and set the Y axis of the accelerometer is directing to the moving direction of swimmers. According to the placement on accelerometers, the authors observe that they can increment stroke counts if the perceived sensory values on X axis are oscillated between 1g and −1g.

The above works [13][17][22] utilize zero crossing concept to design their algorithm. But, the sensory values may frequently cross zero points (due to noises), and thus the stroke count will be incremented unexpected.

5

III. OVERVIEW

ON THE

P ROPOSED S CHEME

Fig. 2 gives an overview on the proposed swimming analysis scheme, which contains two phases. First, the main objective of the data processing phase is to preprocessing the perceived linear acceleration values. This phase contains three modules, named data collection module, floating elimination module, and feature point detection module. In this phase, the data collection module periodically gathers sensory values from acclerometer, and then forwards the filtered data to floating elimination module and feature point detection module. Then, the stroke analysis phase is triggered by the sensory data from the data processing phase. The main objective of this phase is to calculate stroke counts and identify stroke style of the swimmer. This phase contains four threads, one main thread and three threads for dividing strokes based on sensory values of X, Y, and Z axes. Initially, the main thread only demands the stroke dividing thread for X axis to operate, and uses a designed stroke score procedure to identify stroke style based on the sensory values of X axis. After knowing the stroke style, the main thread will further enable the stroke dividing threads for Y and Z axes. Finally, the stroke counts will be reported based on the results of stroke dividing threads on X, Y, Z axes. For more details on the above two phases, please refer to Section IV and Section V, respectively. IV. DATA P ROCESSING P HASE In the following, we present the details of the designed three modules in this phase. A. Data Collection Module This module gathers linear acceleration values from the accelerometer every n millisecond. At a time ti , the perceived linear acceleration values on X, Y, and Z axes are modeled as ax,ti , ay,ti , az,ti , respectively. To restrain the effects caused by buoyant forces, water flow, and sensor’s noises, this module adopts the concept of exponential moving average to implement a low pass filter. When filtering, this module sets the weight of a new sensory value to be w. Let ti−1 be the previous sampling time. At the time ti , this module first replaces the perceived sensory value on X axis ax,ti by ax,ti × w + ax,ti−1 × (1 − w), where

ax,ti−1 is the filtered result on the X axis at time ti−1 , and then puts the filtered results to a set Ax . This module further applies the same procedure to deal with the sensory values on Y and Z axes. Finally, this module keeps those filtered sensory values by a set A = {Ax , Ay , Az }. Note that the size of set A will be

continuously increased (if there are new incoming data) and then decreased (after performing the stroke

analysis phase). Also note that in most cases, deciding w is usually an empirical task. When setting a small w, the output will adjust slowly and include less noise. On the other hand, when setting a large w, the output will adjust quickly and be sensitive to noise. In this work, in order to eliminate noise of the accelerometer, we set w = 0.1 in our implementation.

6

20

Z

Acc (m/s2)

15 10 5

floating

0 -5

p1

p2

-10 0

5

10

15

20

time (s)

Fig. 3.

The changes of Z-axis linear acceleration values.

B. Floating Elimination Module When swimming, most people float for a while after pushing off the wall and have a short floating between strokes. For example, Fig. 3 shows the changes of acceleration values of Z axis when stroking by front crawl. The segment marked by blue color is a floating duration. We can see that the collected sensory values during floating are meaningless to us. Moreover, from Fig. 3, we can see that the sensory values of a floating duration will be vibrated within a certain range. So, in this module, we set two F threshold values, TlbF and Tub , to be the lower bound and upper bound of floating. A sensory value, which F locates between (TlbF , Tub ), will be taken as a floating sensory value. However, when swimming, a floating

duration often lasts for a period of time. So, we further check if the length of a floating duration exceeds F a predefined time threshold Ttime . In summary, this module eliminates those floating sensory data within F a floating duration if the length of the floating duration is larger than a threshold Ttime .

Assume that this module is processing a sensor value ati in any axis (in A). The floating elimination module works as follows. F F 1) This module checks if ati satisfies one of the following two conditions: (i) ati−1 > Tub and ati < Tub

or (ii) ati−1 < TlbF and ati > TlbF . If so, this module records the start time of a floating duration tf s to be ti , and then executes the step 2. Otherwise, this module performs this step again and checks the next sensory value ati+1 . 2) This module keeps checking the sensory value ati+n , where n = 1, 2, .., until finding a sensory value F F ati+k , which satisfies one of the following two conditions: (i) ati+k < Tub and ati+k+1 > Tub or (ii)

ati+k > TlbF and ati+k+1 < TlbF . Then, this module records the stop time of the floating duration tf e to be ti+k . F 3) This module checks if tf e − tf s > Ttime . If so, this module takes the time interval between tf s and

tf e as a floating duration. Then, this module eliminates those data between tf s and tf e , and updates the corresponding time information of the sensory values in A.

7

4) This module goes back step 1 to check sensory values from the next of ati+k+1 . Note that in step 1 and step 2, this module finds those sensory values that are located between TlbF and F Tub , and identifies a floating duration bounded by tf s and tf e . Also note that all sensory values in all axes

in A will be processed by this module. Moreover, when floating, the perceived sensory values on three F axes should be zero, and the determined TlbF and Tub values should be near the 0 m/s2 . To determine F suitable TlbF and Tub values, a preliminary experiment (e.g., Fig. 3) to observe vibrations of accelerometer

(near 0 m/s2 ) is needed. Recall that the floating elimination module is to trim some sensory values. If F the gap between the TlbF and Tub is large (resp., small), more (resp. few) sensory data will be considered

as floating points, and thus the set A will be smaller (resp., larger). However, based on our experiments, the size of A only affects the computation times of the feature point detection model, but does not bring impacts on the results of identifying stroke styles and counting strokes. C. Feature Point Detection Module In this module, we find feature points in A. In our scheme, feature points are taken as potential start or end time points of strokes. According to the characteristics of accelerometers, the linear acceleration values change greatly when a user starts to perform some actions. For example, in Fig. 3, the swimmer starts to stroke around the time p1 . So, we can take p1 as a feature point because that it may be a start time point of a regular fluctuation. From the visual result in Fig. 3, we can mutually find that the stroke approximately ends around p2 (which has the same linear acceleration value as p1 ). By the above example, a fluctuation starts and ends with same sensory value, and thus we can take the sensory value as a boundary. The basic idea of this module is to find the corresponding time instance of a sensory value as a feature point if its next sensory value crosses a predefined boundary. To determine boundaries, we have the following observations from preliminary experiments. 1) The regular fluctuations on X axis of front crawl, breaststroke, butterfly stroke, and backstroke start near 0 m/s2 . For example, the visual examples in Fig. 1 and Fig. 8. 2) The sensory values on Y (or Z) axis of front crawl, breaststroke, butterfly stroke, and backstroke fluctuate does not have same characteristic. For example, in Fig. 8, most peak sensory values on Y axis of front crawl and backstrokes are near 0 m/s2 , while the ones of butterfly stroke are below 0 m/s2 . B So, in this work, for the X axis of all stroke styles, we assign the same lower boundary Tx,lb and upper B B B boundary Tx,ub values. But, for the Y (resp., Z) axis, we give different lower boundary Ty,lb (resp., Tz,lb ) B B and upper boundary Ty,ub (resp., Tz,ub ) values for different stroke styles. Moreover, the basic principle of

selecting a lower (resp., upper) boundary is to set a value that can be higher (resp., lower) than the observed

8

valley (resp., peak) sensory values in preliminary experiments. We only need to assign “reasonable” but not “optimized” boundary values. This is because that the design of setting boundary is used to find out those feature points that may be the start and end time points of a regular fluctuation. So, different boundary values only affect the start timings of regular fluctuation and number of feature points. For example, in Fig. 3, if we set a lower boundary, which value is smaller than p1 ’s value, the regular fluctuation will be deferred. On the other hand, if we set a boundary value that is larger than p1 ’s value, there will have more feature points. However, after deciding boundary values, it is possible that the perceived sensory values on X, Y, or Z axis are not crossing the upper or lower boundaries. In this case, our scheme can still count strokes (unless all sensory values on all axes are not crossing boundaries). Furthermore, recall that the boundaries for different strokes styles are different. But, when collecting linear acceleration values, we have no idea about the stroke style that the swimmer is performing, and thus we cannot know how to apply boundary values. Recall that the boundaries of X axis for all stroke styles are the same, and thus the feature points on X axis for all stroke styles are the same. So, in our scheme, this feature point detection module first finds feature points on X axis, and then gives the results to the stroke analysis phase in Section V. After the stroke analysis phase feedbacks the identified stroke style, this module will further find feature points on Y and Z axes. B B Given the assigned the boundaries Tx,ub and Tx,lb , this feature point detection module works as follows:

For a time ti , this module puts ti to a feature point set Fx if one of the following two conditions is satisfied. • •

B B ax,ti−1 < Tx,ub and ax,ti ≥ Tx,ub

B B ax,ti−1 > Tx,lb and ax,ti ≤ Tx,lb

After knowing the identified stroke style, this module can also generate Fy and Fz by boundaries of Y axis B B B B )) using the same procedure on deciding Fx . As a result, , Ty,lb )) and Z axis (i.e., (Tz,ub , Tz,lb (i.e., (Ty,ub

this module puts feature points to a set F = {Fx , Fy , Fz }. Note that the idea of feature point detection

module is similar to the floating elimination module. But, the feature point detection module aims to find

sensory values that are going to cross predefined boundaries while the floating elimination module aims to find sensory values that are located within a specific range. V. S TROKE A NALYSIS P HASE In the following, we first show the basic ideas on counting strokes and identifying stroke styles. Then, we show the detailed procedures of this phase.

9

A. Ideas of Counting Strokes As we mentioned above, when swimming, the perceived linear acceleration values of strokes fluctuate regularly. In this work, our scheme analyzes strokes based on segments, and a segment contains two (nearby) regular fluctuations. The basic ideas of counting strokes are as follows: Given a feature point set Fx , Fy , or Fz , we first find two feature points, say tfi and tfj , and take those sensory values between tfi and tfj as a temporary segment SG1 . Then, we divide sensory values in SG1 into two equal parts, say S0 and S1 , and calculate the correlation coefficient of S0 and S1 by a function ρia (tfi , tfj ). If the result of ρia (tfi , tfj ) can be higher than a threshold Tia , we say that the segment SG1 contains two regular fluctuations, i.e., contains two strokes. Moreover, recall that strokes are continuous when swimming. So, we need to further calculate the correlation coefficient of two nearby segments. Given a segment SG2 , which boundaries are tfj and tfk . (The right boundary of SG1 is the same as the left boundary of SG2 .) Assume that the result of ρia (tfj , tfk ) also satisfies the predefined threshold Tia . At this time, we further calculate the correlation coefficient of SG1 and SG2 by a function ρsg (tfi , tfj , tfk ). If ρsg (tfi , tfj , tfk ) is also higher than another threshold Tsg , we can say that SG1 and SG2 are two nearby similar segments. Then, we can further find a segment SG3 , which is nearby by the SG2 , and execute the above checks. By the above procedures, we can find segments such that each of them contain two strokes, and sensory values within every two nearby segments have similar tendencies. However, when analyzing strokes, we have to consider the following two factors: First, we observe that the stroke frequencies of recreational swimmers are not stable, which leads to the sizes of segments are not the same. As a result, the correlation coefficient of two segments cannot be derived. So, given two nearby segments (tfi , tfj ) and (tfj , tfk ), in this work, the function ρsg (tfi , tfj , tfk ) will contain the following three steps: 1) In this step, we enlarge segments (tfi , tfj ) and (tfj , tfk ) to a predefined size Td . Take the segment (tfi , tfj ) as an example. We first calculate the magnification l = Td /(tfj − tfi ). When enlarging a

segment, we need to modify the sensory values in the segment to preserve its characteristics. Let

at be a sensory value in the segment (tfi , tfj ). After enlarging the segment, the new location of the value at will be t × l. By the displacement formula S = 12 × a × t2 , when the time scale is enlarged,

it implies that a movement will spend more time to complete. Under the condition that the total movement is unchanged, when the time scale is enlarged by l times, the acceleration values will

be degraded by l2 times. As a result, the sensory value at has to change to at /l2 . In this step, we adjust sensory values in segments (tfi , tfj ) and (tfj , tfk ) by the above rules. 2) When calculating correlation coefficient, the numbers of quantities within (tfi , tfj ) and (tfj , tfk ) should be the same. So, this step uses the concept of linear interpolation to adjust sensory values

10

9

X

15

Acc (m/s2)

Acc (m/s2)

10 5 0 -5

6

3

3

0 -3

-10

-6

-15

-9

-20

9

X

6 Acc (m/s2)

20

0.5

1

1.5 time (s)

2

2.5

3

0 -3 -6 -9

-12 0

X

-12 0

0.5

1

(a)

1.5 time (s)

2

2.5

3

(b)

0

0.5

1

1.5 time (s)

2

2.5

3

(c)

Fig. 4. An example on processing training data for front crawl. (a) The original sensory data of 15 strokes. (b) The normalized result. (c) The alignment result.

in (tfi , tfj ) and (tfj , tfk ) to fixed intervals. Given a time instance ti , the corresponding previous and next perceived sensory values on the timeline are at′ and at , where t′ < ti < t, respectively. ′

i −t Then, the value of ati will be set to ati = at′ + (at − at′ ) tt−t ′ . After calculating sensory values on

all fixed time intervals, this step will clear the original perceived sensory values.

3) This step uses the modified (tfi , tfj ) and (tfj , tfk ) to calculate correlation coefficient, and returns the result to the caller of ρsg (tfi , tfj , tfk ). The second factor that we should consider is that according to our experiments, when a swimmer is going to stop swimming, the swimmer may produce redundant sensory values. For example, in Fig. 3, the line segment marked by red color near 19s. These redundant values may have similar tendency as previous segments, and will induce additional stroke counts. As we can see in Fig. 3, when a segment contains redundant values, the sensory values within the segment will have a large vibration. In this work, we use a function, named cal amp(tfi , tfj ), to calculate amplitude value within the segment (tfi , tfj ). The function cal amp(tfi , tfj ) operates as follows. 1) By tfi and tfj , this function first finds the middle time point, say p′ , between tfi and tfj . 2) According to p′ , this function searches the maximum and minimum values between tfi and p′ (resp., p′ and tfj ), and record the difference between the maximum and minimum values to be x1 (resp., x2 ). 3) This function returns |x1 − x2 |. In our scheme, if the return value of cal amp(tfi , tfj ) exceeds a threshold Tamp , the sensory values in the segment (tfi , tfj ) are considered to have redundant values. (We will show how to handle redundant values later.) Moreover, when a swimmer strokes continuously, the peak and valley values of the perceived sensory values on strokes will be similar. So, we can set the Tamp to be Dp + Dv , where Dp (resp., Dv ) represents the maximum difference between the perceived peak (resp., valley) sensory values observed in preliminary experiments. Again, it is hard to find an optimal Tamp value. When setting a large Tamp value,

11 15

An given stroke Model of front crawl Model of backstroke

12 Acc (m/s2)

9 6 3 0 -3 -6 0

Fig. 5.

0.5

1

1.5 time (s)

2

2.5

3

An example of identifying stroke style.

some extra stroke counts will be recorded. On the other hand, when setting a small Tamp value, some stroke counts may be missed. However, swimmers’ stokes may not be stable all the time and accelerometers may suffer from noises. Based on our experiments, a larger Tamp value is preferred. B. Ideas of Identifying Stroke Styles This phase identifies stroke styles by checking if the perceived sensory values have similar patterns with any of four stroke style models, which are constructed by the following three steps. 1) We first collect elite swimmers’ linear acceleration values. Then, we mutually identify strokes based on the collected sensory values on X axis. For example, Fig. 4(a) shows the gathered 15 front crawl strokes of five swimmers. 2) For each stroke, we use the same procedure in Section V-A to extend its time scale to Td /2 and to interpolate sensory values in fixed intervals. Fig. 4(b) indicates the results that each stroke is extended to 3 seconds. (Note that in Section V-A, a segment contains two strokes so we only need to extend the time scale of a stroke to Td /2.) 3) We align the time instants of peak sensory values in the collected strokes, and round those sensory values that exceed 0 or Td /2 to the rear or front (as shown in Fig. 4(c)). Then, we calculate the averaged values on each sampling time point. The averaged result will be taken as the model for the corresponding stroke style. (The final result of the example in Fig. 4 is shown in Section VI). Given a stroke, again, we can adopt the correlation coefficient concept to classify the stroke to be one of the four stoke styles. Fig. 5 shows an example that the given stroke is front crawl. In this example, the correlation coefficient of the given stroke and front crawl (resp., backstroke) model is 0.57 (resp., 0.62). We can see that if we only use the correlation coefficient to identify the stroke’s style, the results may not be correct, e.g., the stroke in Fig. 5 will be taken as backstroke. This is because that the correlation coefficient value of two data sets will be high if these two data sets have similar tendency. But, it does not guarantee that these two data sets are closed. (For example, in Fig. 5, during 1.25 to 2.25 s, the gaps between the sensory data of the given stroke and the backstroke model are large.) So, when identifying

12

stroke styles, we propose to further check if the sensory values of the given strokes and the derived models are closed. The details on identifying stroke styles are as follows: Let models of front crawl, breaststroke, butterfly stroke, and backstroke be M0 , M1 , M2 , and M3 , respectively. We design a stroke scoring procedure, named (stroke scoring()), to identify the stroke style of a given segment SG by the following steps. 1) For the segment SG, we first extend its time scale to Td , and then interpolate sensory values. Then, we divide the modified SG into two strokes S0 and S1 such that S0 and S1 are the first half and latter half of SG, respectively. 2) For M0 , M1 , M2 , and M3 (resp., S0 and S1 ), we record the time instances of their peak sensory values by tm0 , tm1 , tm2 , and tm3 (resp., ts0 and ts1 ). 3) For an Si , where i = 0, 1, and an Mj , where j = 0..3, we calculate scorej,i as follows. a) Let tdif f = |tmj − tsi | be the time difference between peaks of Si and Mj . This step shifts the

time scale of Si by tdif f to let the peaks of Mj and Si are aligned. For those sensory values

that exceed before 0 (resp., behind Td /2) will be rounded to the rear (resp., front) of Si . b) Then, we calculate a score scorej,i =

CC , Dist

where

•

CC is the correlation coefficient of Mj and the shifted Si .

•

Dist is the sum of differences on the sensory values at each sampling point in Mj and the shifted Si .

4) Let sty = argmaxj {scorej |j = 0..3}, where scorej = scorej,0 + scorej,1. This procedure reports the corresponding stroke style of sty.

Note that in the stroke scoring procedure, a stroke Si is considered to be similar with the a model Mj if (i) the shifted Si and Mj have similar trend, i.e., the correlation coefficient of the shifted Si and Mj is high and (ii) the sensory values of the shifted Si and Mj are closed, i.e., the dist value is small. (We will show how to integrate the stroke scoring procedure to our algorithm later.) C. The Detailed Procedures In the following, we present the detailed procedures of the designed stroke analysis phase. In this phase, there are four threads, i.e., one main thread and three stroke dividing(.) threads for sensory values on X, Y, and Z axes, respectively. Initially, the main thread sets stroke style to be undefined and activates the stroke dividing(.) of X axis only. The stroke dividing(.) thread of X axis first finds segments, and then performs the stroke scoring(.) procedure to decide the stroke style. After deciding the stroke style, the main thread notifies the feature point detection module in Section IV-C to generate Fy and Fz . Then, the main thread can activate the stroke dividing(.) threads of Y and Z axes.

13

In a stroke dividing(.) thread, its initial state is instable. When the thread can find consecutive segments, its state will change to stable. Moreover, in our design, a stroke dividing(.) thread, which is in the stable state, may fail to find consecutive segments because that the swimmer starts a new lap or the perceived sensory values do not contain regular fluctuations. At this time, the state of stroke dividing(.) thread changes to instable state, and then the thread will try to find consecutive segments again. Moreover, if the states of all stroke dividing(.) threads become instable at the same time, the stroke style will change to undefined. Then, the main thread will inactivate stroke dividing(.) of Y and Z axes, and the stroke dividing(.) thread of X axis will try to identify the new stroke style. Note that when the stroke style changes to undefined, the main thread also notifies the feature point detection module to stop generating Fy and Fz . Before showing the details of the stroke dividing(.) thread, we first present the used parameters. There are four global variables: (i) The flag stable, which is used to indicate that whether consecutive segments are found. (The default value is false.) (ii) The variable W , which is used to record the estimated period of a stroke. (iii) The variable tfcur , which represents the current processing feature point. (iv) The variable tfprev , which records the start point of the previous segment. Moreover, we define the least size of a segment to be dmin , i.e., the time needed for two strokes is at least dmin . We further demand the maximum size of a segment to be dmax . Furthermore, if the correlation coefficient of ρia (tfi , tfj ) (resp., ρsg (tfi , tfj , tfk )) exceeds a threshold Tia (resp., Tsg ), we say a segment is found (resp., consecutive segments are found). Given a set (Ax , Fx ), (Ay , Fy ), or (Az , Fz ), the stroke dividing(tfcur ) thread derives stroke counts Skx , Sky , or Skz by the following procedures. (Note that in the following, we only show the procedure of using Ax and Fx . The same procedures can be applied to process sensory values of Y and Z axes.) stroke dividing(tfcur ): 1) If stable is false, we perform procedure SP 1(tfcur ). 2) Otherwise, we perform procedure SP 2(tfcur ). procedure SP1(tfcur ): 1) From Fx , we find those feature points, which locate between tfcur + dmin and tfcur + dmax , and then put them into a queue Qp1 . 2) Let tfj = dequeue(Qp1 ). There will have three cases. a) If tfj = ∅, which means that we cannot find a stroke, we update tfcur = next of(tfcur ), where next of(tfcur ) represents the next feature point in Fx . Then, we use the updated tfcur as the

input parameter to execute the stroke dividing again. b) If ρia (tfcur , tfj ) ≥ Tia and cal amp(tfcur , tfj ) ≤ Tamp , which means that we successfully find

14

a segment and the segment does not have redundant sensory values, we set W = tfj − tfcur

and change stable to be true. Then, we update the stroke count Skx = Skx + 2, tfprev = tfcur , and tfcur = tfj . If the stroke style is undefined yet, we execute stroke scoring(tfcur , tfj ), and then send the stroke style result to the main thread. Then, we execute the stroke dividing again with the updated tfcur . c) For other cases, we go back to step 2 to check the next feature point in Qp1 . procedure SP2(tfcur ): 1) From Fx , we find those feature points, which locate between tfcur + W − δ and tfcur + W + δ, where δ is a deviation for W , and them put into a queue Qp2 .

2) Let tfj = dequeue(Qp2 ). There will have four cases. a) If tfj = ∅, which represents that we cannot find consecutive strokes, we set stable=false, W = 0, and tfprev = null. Then, we set tfcur = next of(tfcur ) and execute the stroke dividing again. b) If ρia (tfcur , tfj ) ≥ Tia , ρsg (tfprev , tfcur , tfj ) ≥ Tsg , and cal amp(tfcur , tfj ) ≤ Tamp , which means that consecutive strokes are found, we first set W = tfj − tfcur and update Skx =

Skx + 2, tfprev = tfcur , and tfcur = tfj . Then, we execute the stroke dividing again. c) If ρia ( tfprev2+tfcur ,

tfcur +tfj ) 2

≥ Tia and cal amp( tfprev2+tfcur ,

tfcur +tfj ) 2

≤ Tamp , which means that

the latter half of the previous segment is similar to the first half of the current segment, we set stable = f alse, W = 0, and tfprev = null and update Skx = Skx + 1. Then, we set tfcur = tfj and execute the stroke dividing again. d) For other cases, we go back to step 2 to check the next feature point in Qp2 . Note that in step 2(c) of the procedure SP2, this case is so designed to handle the scenario that the actual stroke count of the swimmer is odd value. Also note that in some cases, stroke dividing threads for X, Y, or Z axes may not be able to find strokes, and thus the derived stroke counts based on (Ax , Fx ), (Ay , Fy ), and (Az , Fz ) will be different. In this work, we assign the final stroke count using the result of stroke dividing threads that can identify the most number of strokes. In other words, the main thread will set the final stroke counts to be max{Skx , Sky , Skz }. VI. P ROTOTYPING

AND

E XPERIMENT R ESULTS

A. Prototyping Results We implement the proposed swimming analysis scheme on an Android-based waterproof smartphone. According to the specification of the smartphone, the X, Y, and Z axes of the accelerometer are as shown in Fig. 6(a). By the Android sensor API, we set the sampling frequency of the smartphone to be “SENSOR DELAY FASTEST”. As shown in Fig. 6(b), we place the smartphone on the swimmer’s

15 Z

X

Y (a)

Fig. 6.

(b )

(a) The X, Y, and Z axes of the smartphone’s accelerometer. (b) The placement of the smartphone on swimmer’s palm. TABLE I I MPLEMENTATION PARAMETERS . Parameter Filter weight w Floating elimination module related F Tub , TlbF F Ttime Feature point detection module related B B Tx,ub , Tx,lb of all styles B B B B Ty,ub , Ty,lb , Tz,ub , Tz,lb of front crawl (resp., breaststroke) (resp., butterfly) (resp., backstroke) Segment predefined size Td Segment difference threshold Tamp Tia , Tsg dmin , dmax δ

Values 0.1 1.5, -1.5 m/s2 0.5 s 2.5, -2.5 m/s2 -2.5, -6.5, 2.5, -2.5 m/s2 2.5, -6.5, 2.5, -2 m/s2 0, -6.5, 5, -5 m/s2 5, -5, 5, -5 m/s2 6s 8 m/s2 0.45, 0.3 1, 6 s 1.5 s

right palm, and we further pack the smartphone to enhance waterproof. Moreover, we invite five elite swimmers, who can stroke by four stroke styles, to conduct preliminary experiments. By the experiment results, we obtain the stroke style models, as shown in Fig. 7, and threshold values, as listed in Table I. When constructing stork style models, for each elite swimmer, we randomly pick three strokes per stroke style. In other words, each stroke style model is constructed by 15 strokes (as the example on obtaining front crawl model in Fig. 4). We invite 24 swimmers to test the designed scheme. Twelve of them are elite swimmers and are able to swim using four stroke styles. For the other 12 swimmers, 6 of them can also swim by four stroke styles, and the other 6 swimmers stroke by front crawl only. In a test, a swimmer uses front crawl, breaststroke, butterfly stroke, or backstroke to swim for 50 meters. We mutually record swimmers’ stroke styles and stroke counts, and these records are taken as ground truths.

16 6

X Acc (m/s2)

Acc (m/s2)

6 3 0 -3

X

3 0 -3 -6

-6 0

0.5

1

1.5 2 time (s)

2.5

3

0

0.5

(a)

2.5

3

2.5

3

(b)

3 0 -3

X

6 3 0 -3

-6 0

0.5

1

1.5 2 time (s)

2.5

3

(c) Fig. 7.

1.5 2 time (s)

9

X Acc (m/s2)

Acc (m/s2)

6

1

0

0.5

1

1.5 2 time (s)

(d)

The models of (a) front crawl, (b) breaststroke, (c) butterfly stroke, and (d) backstroke.

B. Experiment Results on Identifying Stroke Styles Table II(a) shows the experiment results of identifying stroke styles of those 12 elite swimmers. From the results, we can see that the proposed scheme can exactly identify front crawl, breaststroke, and backstroke styles. We can see that two butterfly strokes are identified as breaststroke and backstroke, respectively. In our design, we only use the first two strokes for identifying stroke styles. Since the butterfly strokes are harder to control, it is possible that the swimmer’s postures are not stable at that time, and thus cause the mismatches. Table II(b) shows the stroke style identification results of those 12 elite swimmers when using a commercial product, Garmin swim [3]. We can see that the Garmin swim can exactly identify the breaststroke only. Five of the 12 butterfly strokes are identified as front crawl, and one backstroke is identified to be breaststroke. The Garmin swim has a state N/A, and one front crawl stroke is divided into the N/A state. As a result, we can see that the proposed scheme can have better performance on identifying stroke styles than the Garmin swim. We further compare our scheme with the support vector machine (SVM) classifier (which is implemented by the Python scikit-learn library [4]). In SVM, for each stroke style, we use the gathered 15 strokes from 5 elite swimmers as training data. Then, we use the recorded first stroke of those 12 elite swimmers (in every stroke style) as the testing data. The time scale of all data are extended to Td /2. As shown in Table II(c), the prediction accuracies of SVM on four stroke styles are lower than 50%. The accuracy is low because of the following two reasons. (i) For each stroke style, we only use 15 stroke data as training data. According to the characteristic of SVM, the classification results can have a higher accuracy when there are sufficient training data. (ii) As discussed in [9], the accuracy of SVM can be further improved if the system can feedback testing results to the training database. So, we further conduct an addition experiment to feedback parts of those 12 elite swimmers’ strokes as training data to SVM. The results

17

TABLE II T HE COMPARISONS ON

Front crawl l t ua Ac

rea st stroke B utterfly stroke B ack stroke B

Front crawl 12

t ua Ac

Predict by our scheme rea stx utterfly Bstroke Bstroke 0 0

Front crawl

ackx stroke B 0

Front crawl

0

12

0

0

l

0

1

10

1

Ac

0

0

12

0

t ua

Predict by SVM classifier rea stx utterfly Bstroke Bstroke

B

Front crawl

5

1

4

ackx stroke B 2

rea st stroke B utterfly stroke B ack stroke B

1

5

1

5

l

3

1

6

2

Ac

0

2

6

4 (c)

B B

(a) Front crawl

l

IDENTIFYING STROKE STYLES OF ELITE SWIMMERS .

B B B

/A N 1

rea st stroke

0

12

0

0

0

utterfly stroke

5

0

7

0

0

ack stroke

0

1

0

11

0

(b)

Front crawl t ua

11

Predict by Garmin swim rea stx ackx utterfly stroke Bstroke Bstroke B 0 0 0

Front crawl 2

Predict by DTW rea stx utterfly Bstroke Bstroke 0 9

ackx stroke B 1

rea st stroke

0

4

5

3

utterfly stroke

0

0

10

2

ack stroke

0

0

5

7

(d)

indicate that the accuracy will be improved to about 90%. To summarize, compare to the SVM classifier, the proposed scheme can achieve high accuracy by some training data. Although the SVM can have better accuracy than our scheme by a feedback mechanism, it is inconvenient for users to give feedbacks when they are swimming. Next, we compare our scheme with dynamic time wrapping (DTW) technique (which is implemented by the Python DTW library [2]). The DTW is commonly used to measure the similarity of two temporal sequences that may have different lengths and frequencies. In DTW, we use the models in Fig. 7 as the training data, and use the recorded first stroke of those 12 elite swimmers (in every stroke style) as the testing data. Note that the time scales of training data is Td /2, but the ones of testing data are not extended to Td /2. For a testing data, the DTW computes the distances to four stroke style models. Then, we simply select the stroke style, which induces the least distance to the testing data, as the final result. Table II(d) indicates the results when using DTW. From the results, the DTW can only have better accuracies on identifying butterfly stroke and backstroke. But, the overall accuracy of using DTW is low. However, we found that it is still possible to increase the accuracy of DTW by some preprocessing on testing data. (But, how to increase the accuracy of DTW is out of scope of this work.) Although it seems that the DTW scheme can be applied to identify stroke styles, the problem of the DTW is that it needs a lot of computing time to obtain a result. In our experiment, the DTW takes almost 3 seconds to make a decision (by a desktop PC). So, the DTW scheme is not suitable for the needs of realtime decision making. We further conduct experiments to identify stroke styles of other 12 recreational swimmers. The results are shown in Table III. We can see that the accuracy rate is lower than the results in Table II(a), but the results can still outperform the ones of SVM and DTW. In this experiment, we found that the postures of

↺ㇳ㫉㔠 ↺ㇳ㫉㔠

18

TABLE III T HE PERFORMANCE ON IDENTIFYING STROKE STYLES OF RECREATIONAL SWIMMERS . ⧃㦪 Classific a⧃㦪 tio n so n re c re atio n al sw imme r s F ro n t c raw l l t ua Ac

B

0

re aststro k e tte r fly stro k e

Bu ac k stro k e B

Predi ctby oursch em e tte r fly re ast¿ Bstro k e B ustro k e 0 0

F ro n t c raw l 1 0

3

3 0

3 0

0

2 0

ac k ¿ Bstro k e 2 0 1 6

TABLE IV T HE OVERALL ACCURACY OF OUR SCHEME AND G ARMIN SWIM . ↮栆㭼庫 ↮栆㭼庫

O u r s c h e m e

G a r m in s w im

F ro n t c ra w l

9 1 .6 %

9 1 .6 %

B re a st stro k e

8 3 .3 %

1 0 0 %

B u tte rfly stro k e

7 7 .8 %

5 8 .3 %

B a c k stro k e

1 0 0 %

9 1 .6 %

Ove r a l l a c c u r a c y

8 7 . 2%

8 5 . 4%

those recreational swimmers are not stable, and thus affect the results. For example, in a test, a female recreational swimmer uses front crawl stroke style, but our scheme identifies she strokes by backstroke. This is because that when a front crawl swimmer pushes the water slighter, the recorded linear acceleration values will be flat, which will be similar to the ones of the latter half of the backstroke model (as in Fig. 7(d)). Finally, Table IV summaries the overall accuracy of our scheme and Garmin swim. In this table, the overall accuracies of our scheme (resp., Garmin swim) are averaged by all 12 elite and 12 recreational swimmers (resp., only 12 elite swimmers only). We can see that compare to Garmin swim, our scheme has less accuracy on classifying breaststroke because that our scheme reports wrong answers on three recreational swimmers. But, overall speaking, our scheme can perform better than the Garmin swim even that our results include some recreational swimmers. C. Experiment Results on Identifying Stroke Counts Fig. 8 gives four visual stroke dividing results. The segment, which is colored other than gray, represents the identified two strokes. Fig. 8(a) shows the dividing result of front crawl. In this experiment, the swimmer strokes 16 times. From the result, the proposed scheme finds 12, 16, and 16 strokes based on sensory values on X, Y, and Z axes, respectively. The stroke counts on X axis are incorrect because that the sensory values of X axis may be affected easily if the swimmer swings her palm slightly when swimming. In this example, we can see that the found strokes are not exactly aligned to the swimmer’s strokes. This is because that our scheme dividing strokes based on discrete feature points, and thus the found strokes may be shifted. Recall that in our scheme, we use the maximum number of strokes in X, Y, or Z axis as the final decision. So, the number of found strokes are the same as the ground truth in this

0

Y

Acc (m/s2)

X

Acc (m/s2)

10 0 -10 -20

-10 -20 10 0 -10 -20

Acc (m/s2)

2

Acc (m/s )

2

Acc (m/s )

2

Acc (m/s )

19

Z

5

10

15

20

25

30

35

10

X

0 -10 Y

0 -10 20

Z

10 0 5

40

10

15

20

25

time (s)

Acc (m/s2) Acc (m/s2)

X

Y

-10 -20 20 10 0 -10 -20

Z

0

5

10

15

45

50

20

25

30

35

20

55

60

X

10 0 -10 10

Y

0 -10 -20 10

Z

0 -10 10

40

15

20

25

time (s)

(c) Fig. 8.

40

(b)

10 0 -10 -20 0

Acc (m/s2)

2

Acc (m/s )

2

Acc (m/s )

2

Acc (m/s )

(a)

30 35 time (s)

30 35 time (s)

40

45

50

55

(d)

The stroke dividing results of (a) front crawl, (b) breaststroke, (c) butterfly stroke, and (d) backstroke. TABLE V T HE ACCURACY OF THE PROPOSED SCHEME ON COUNTING STROKES . ↺ㇳ㫉㔠

Front crawl Breaststroke Butterfly stroke Back stroke Averaged

Our sc he e

m 96.7% 92.7% 87.3% 92.6% 92.3%

r

s

c

Ga min wim Peak dete tion 85.7% 89 .2% 88.5% 86.1% 3% 4. 6 83. % 9 87.8% 94.1% 0. 6% 86. 7 % 9

⧃㦪 example. Fig. 8(b) indicates the result of breaststroke. In this example, we can see that the last segment

of Z axis contains three strokes. This result shows the effectiveness of step 2(c) of procedure SP2. We can see that in the X (resp. Y) axis, there are five strokes (resp. one stroke) cannot be identified. However, according to the result of Z axis, the found stroke counts are the same with the ground truth. Moreover, the result of butterfly stroke is shown in Fig. 8(c). In this experiment, we can see that a divided segment of X axis contains only four strokes. This is because that we set the maximum length of a segment dmax ↮栆㭼庫 to be up to 6 seconds. In this case, the dividing result of Y axis can fit to the ground truth. Fig. 8(d)

shows the result of backstroke. Based on the Y and Z axes, our scheme can find 16 strokes, which is the same with the ground truth. Next, Table V indicates the averaged accuracy of the proposed scheme on counting strokes (for all

20

12 elite and 12 recreational swimmers). In a test, i.e., a swimmer swims for 50 meters, the accuracy is calculated by 1 −

|Sk−Gt| , Gt

where Sk and Gt represent our stroke counting result and the ground truth,

respectively. From the results, we can see that the averaged accuracy of the proposed scheme is above 92%. The butterfly stroke will have a lower accuracy value. This is because that when stroking by butterfly stroke, swimmers’ postures cannot be stable all the time, and thus slightly affects stroke counting results. Moreover, from Table V, we can see that the average accuracy of Garmin swim (for 12 elite swimmers only) is about 90.7%. The accuracy value of butterfly and backstroke are slightly higher than our scheme. But, the Garmin swim has lower accuracy on counting front crawl and breaststroke. Table V also indicates that the average accuracy when using the peak detection method in [7][21]. The results of peak detection are obtained off-line, and are based on the collected sensory values when experimenting on our scheme. Before experimenting, we need to decide peak threshold values. We observe that for front crawl, breaststroke, and backstroke styles (resp. for the butterfly stroke style), it is not easy to find a suitable peak threshold value for sensory values on Y axis (resp., X and Y axes). So, we only rely on the linear acceleration values on X or Z axes to count strokes. The threshold values on X axis for front crawl, breaststroke, and backstrokes are set to 10, 5, and 10 m/s2 , respectively. The threshold values on Z axis for front crawl, breaststroke, butterfly stroke, and backstrokes are set to 10, 10, 10, and 8 m/s2 , respectively. We record the result (based on X or Z axis), which have larger stroke counts, as the final result. We observe that the peak detection method can perform well only when the performed strokes of swimmers are stable. The butterfly stroke will have the worst accuracy because that the style is more difficult than the other three styles. From the experiment, we also see that it is hard to find suitable thresholds for all swimmers. Some experiments may less stroke counts because that the swimmers push water with less strength in some strokes. To conclude, when counting stroke numbers, our scheme can outperform the Garmin swim and peak detection scheme. VII. C ONCLUSIONS In this paper, we have proposed a swimming analysis scheme to count and identify swim strokes by

accelerometer. The proposed algorithm contains a data processing phase and a stroke analysis phase. The data processing phase collects sensory data, eliminates floating sensory data, and finds feature points. The stroke analysis phase identifies stroke styles and counts strokes by finding correlated segments, which can be taken as swimmers’ strokes. In this work, we implement the designed scheme on a waterproof Android platform. The experiment results indicate that the designed scheme can effectively identify stroke styles and count strokes with more than 87% and 94% accuracies in average, respectively. In the future, there are two directions. First, we can try to improve the accuracy on identifying stroke styles by continuously classifying strokes (when swimming), and then our scheme can report a style with most confidence.

21

Second, we are going to implement our scheme on Android smartwatches and design an algorithm to detect turns in the swimming pool. R EFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]

Withings activite. http://www2.withings.com/us/en/products/activite/. Python dtw 1.0. https://pypi.python.org/pypi/dtw/1.0. Garmin swim. http://sites.garmin.com/en-US/swim/. Scikit-learn: machine learning in Python. http://scikit-learn.org/stable/. Rhythm, timing and stroke rate in swimming. http://www.swimsmooth.com/strokerate.html. A. Anjum and M. U. Ilyas. Activity recognition using smartphone sensors. In Proc. of IEEE Int’l Consumer Communications and Networking Conference (CCNC), 2013. M. B¨achlin and G. Tr¨oster. Swimming performance and technique evaluation with wearable acceleration sensors. Elsevier Pervasive and Mobile Computing, 8(1):68–81, 2012. J. M. Barden and R. T. Kell. Relationships between stroke parameters and critical swimming speed in a sprint interval training set. Journal of Sports Sciences, 27(3):227–235, 2009. W. Choi, J. Oh, T. Park, S. Kang, M. Moon, U. Lee, I. Hwang, and J. Song. MobyDick: an interactive multi-swimmer exergame. In Proc. of ACM Int’l Conference on Embedded Network Sensor Systems (SenSys), 2014. F. Dadashi, F. Crettenand, G. P. Millet, and K. Aminian. Front-crawl instantaneous velocity estimation using a wearable inertial measurement unit. MDPI Sensors, 12(10):12927–12939, 2012. S. Daukantas, V. Marozas, and A. Lukosevicius. Inertial sensor for objective evaluation of swimmer performance. In Proc. of Int’l Biennial Baltic Electronics Conference (BEC), 2008. M. Franken, F. Diefenthaeler, F. C. Mor´e, R. P. Silveira, and F. A. d. S. Castro. Critical stroke rate as a parameter for evaluation in swimming. Motriz: Revista de Educac¸a˜ o F´ısica, 19(4):724–729, 2013. R. M. Hagem, S. G. O’Keefe, T. Fickenscher, and D. V. Thiel. Self contained adaptable optical wireless communications system for stroke rate during swimming. IEEE Sensors Journal, 13(8):3144–3151, 2013. H. Junker, O. Amft, P. Lukowicz, and G. Tr¨oster. Gesture spotting with body-worn inertial sensors to detect user activities. Elsevier Pattern Recognition, 41(6):2010–2024, 2008. B. Khoo, B. Lee, S. Senanayake, M. Arosha, and B. Wilson. System for determining within-stroke variations of speed in swimming (SWiSS). In Proc. of IEEE/ASME Int’l Conference on Advanced Intelligent Mechatronics (AIM), 2009. N. Krishnan, C. Juillard, D. Colbry, and S. Panchanathan. Recognition of hand movements using wearable accelerometers. Journal of Ambient Intelligence and Smart Environments, 1(2):143–155, 2009. T. Le Sage, A. Bindel, P. Conway, L. Justham, S. Slawson, and A. West. Embedded programming and real-time signal processing of swimming strokes. Springer Sports Engineering, 14(1):1–14, 2011. Q. Li, J. Stankovic, M. Hanson, A. T. Barth, J. Lach, and G. Zhou. Accurate, fast fall detection using gyroscopes and accelerometerderived posture information. In Proc. of IEEE Int’l Conference on Body Sensor Networks (BSN), 2009. Y. Ohgi, H. Ichikawa, M. Homma, and C. Miyaji. Stroke phase discrimination in breaststroke swimming using a tri-axial acceleration sensor device. Springer Sports Engineering, 6(2):113–123, 2003. J. Pansiot, B. Lo, and G.-Z. Yang. Swimming stroke kinematic analysis with BSN. In Proc. of IEEE Int’l Conference on Body Sensor Networks (BSN), 2010. P. Siirtola, P. Laurinen, J. R¨oning, and H. Kinnunen. Efficient accelerometer-based swimming exercise tracking. In Proc. of IEEE Int’l Symposium on Computational Intelligence and Data Mining (CIDM), 2011. A. Stamm, D. A. James, R. M. Hagem, and D. V. Thiel. Investigating arm symmetry in swimming using inertial sensors. In Proc. of IEEE Sensors Conference, 2012.