# Features for time series classification

I consider the problem of (multiclass) classification based on time series of variable length $T$, that is, to find a function $$f(X_T) = y \in [1..K]\\ \text{for } X_T = (x_1, \dots, x_T)\\ \text{with } x_t \in \mathbb{R}^d ~,$$ via a global representation of the time serie by a set of selected features $v_i$ of fixed size $D$ independent of $T$, $$\phi(X_T) = v_1, \dots, v_D \in \mathbb{R}~,$$ and then use standard classification methods on this feature set. I'm not interested in forecasting, i.e. predicting $x_{T+1}$. For example, we may analyse the way a person walks to predict the gender of the person.

What are the standard features that I may take into account ? In example, we can obviously use the mean and variance of the serie (or higher order moments) and also look into the frequency domain, like the energy contained in some interval of the Discrete Fourier Transform of the serie (or Discrete Wavelet Transform).

## Simple statistical features

• Means in each of the $d$ dimensions
• Standard deviations of the $d$ dimensions
• Skewness, Kurtosis and Higher order moments of the $d$ dimensions
• Maximum and Minimum values

## Time serie analysis related features

• The $d \times d-1$ Cross-Correlations between each dimension and the $d$ Auto-Correlations
• Orders of the autoregressive (AR), integrated (I) and moving average (MA) part of an estimated ARIMA model
• Parameters of the AR part
• Parameters of the MA part

## Frequency domain related features

See Morchen03 for a study of energy preserving features on DFT and DWT

• frequencies of the $k$ peaks in amplitude in the DFTs for the detrended $d$ dimensions
• $k$-quantiles of these DFTs
• Emile, This question is similar to one I just posed (stats.stackexchange.com/questions/51475/…). Would you be able to post any R code for the DFT features? – B_Miner Mar 6 '13 at 15:43
• Is there any method based on shapelets for variable length time series? – Simone Jun 9 '15 at 4:18

Emile, I think the features listed in your answer are pretty good starting points, though as always, I think some domain expertise (or at least a good long think) about your problem is equally important.

You may want to consider including features calculated from the derivatives (or integrals) of your signal. For example, I would wager that rapid acceleration/deceleration is a reasonably good predictors of accident-prone driving. That information is obviously still present in the position signal, but it's not nearly as explicit.

You may also want to consider replacing the Fourier coefficients with a wavelet or wavelet packet representation. The major advantage of wavelets is that they allow you to localize a feature in both frequency and time, while the traditional Fourier coefficients are restricted to only time. This might be particularly useful if your data contains components that switch on/off irregularly or has square wave-like pulses that can be problematic for Fourier methods.

As the other answers suggested, there is a huge number of time series characteristics that can be used as potential features. There are simple features such as the mean, time series related features such as the coefficients of an AR model or highly sophisticated features such as the test statistic of the augmented dickey fuller hypothesis test.

# Comprehensive Overview over possible time series features

The python package tsfresh automates the extraction of those features. Its documentation describes the different calculated features. You can find the page with the calculated features here.

Disclaimer: I am one of the authors of tsfresh.

I suggest you, instead of using classic approaches for extracting hand-engineered features, utilise autoencoders. Autoencoders plays an important role in the feature extraction of deep learning architecture.

The autoencoder tries to learn a function $f(X_T)≈X_T$. In other words, it is trying to learn an approximation to the identity function, so as to output $\hat X_T$ that is similar to $X_T$.

The identity function seems a particularly trivial function to be trying to learn; but by placing constraints on the network, such as by limiting the number of hidden units, we can discover interesting structure about the data. In this way, your desired $ϕ(X_T)=v_1,…,v_D \in R$ will be equivalent to the output values of the middlemost layer in a deep autoencoder, If you limit the number of hidden units in the middlemost to $D$.

Additionally, you can use many flavors of autoencoder for finding the best solution to your problem.

The linked paper will be somewhat enlightening, since it is interested in the more or less the same issue in another context.

Paper abstract (in the Internet Archive)

Paper PDF

Depending on the length of your time series, the usual approach is to epoch the data into segments, e.g. 10 secs.

However, often prior to breaking the time-series into segments it is necessary to perform some preprocessing such as filtering and artifact rejection. You can then compute a variety of features such as those based on frequency (i.e. take an FFT for each epoch), time (e.g. mean, variance etc of the time-series in that epoch) or morphology, (i.e. the shape of the signal/time-series in each epoch).

Usually the features used to classify segments (epochs) of a time-series/signal are domain-specific but Wavelet/Fourier analysis are simply tools to allow you examine your signal in the frequency/time-frequency domains rather than being features in themselves.

In a classification problem each epoch will have a class label e.g. 'happy' or 'sad', you would then train a classifier to distinguish between 'happy' and 'sad' epochs using the 6 features calculated for each epoch.

In the event that each time series represents a single case for classification, you need to calculate each feature across all samples of the time series. The FFT is only relevant here if the signal is linear time invariant (LTI), i.e. if the signal can be considered to be stationary over the whole time series, if the signal is not stationary over the period of interest, a wavelet analysis may be more appropriate. This approach will mean that each time series will produce one feature vector and will constitute one case for classification.

• I've always felt that breaking time series, a naturally continuous process, into discrete periods (or epochs), results in a loss of information. Unless there is natural epochs in the series, how does one choose epochs? Likely epochs can be found that fits any desired outcome. – Cam.Davidson.Pilon Feb 25 '13 at 16:02
• Not sure that an epoch can be found to suit every outcome but for any practical problem With a non-stationary signal you need to find someway to take into account temporal variations (if a signal/time-series adequately described analytically or is linear time invariant this not necessary). The epoch length is again domain specific but usually chosen to be short enough that the signal is stationary under the time period (epoch) of interest. – BGreene Feb 25 '13 at 16:10
• BGreene, could you explain what is the benefit of partitioning the serie into epochs for selecting features used in classification ? I see this partitioning as a helper for later computations (i.e. FFT), but not as something related to the selection of features themselves. Maybe it is related to the "morphology" you mentioned. – Emile Feb 25 '13 at 16:20
• The partitioning into epochs has nothing to do with feature selection. If you have a long signal (e.g. 10 hours of recording sampled @100Hz) then in order to examine how the signal changes over time you need to break it into epochs. In a classification problem each epoch will have a class label e.g. 'happy' or 'sad', you would then train a classifier to distinguish between 'happy' and 'sad' epochs using the 6 features calculated for each epoch. – BGreene Feb 25 '13 at 16:39
• Ok:) It is definitely unrelated to my original question. I consider the case where the class labels the whole serie. I'm going to edit my question to add a clarifying example. – Emile Feb 25 '13 at 16:52