6
$\begingroup$

I am looking for methods for feature selection (or feature extraction) for time series data. Of course I did some research before, but it was not satisfying.

I am aware of methods like PCA, importance matrix from random forest, linear regression, etc. for feature selection or extraction, but are those methods also applicable to time series data?

The task would be to find a set of variables which is a good predictor of a certain time series variable.

Thanks for any suggestions!

$\endgroup$
  • $\begingroup$ I would start with domain knowledge and expert opinion as opposed to data mining. $\endgroup$ – forecaster Mar 30 '15 at 20:42
  • 1
    $\begingroup$ Perhaps you could start with some large general model (AR with exogenous regressors and their lags) and use regularization (LASSO, ridge regression, elastic net). Meanwhile, PCA assumes independent observations so its use in a time series context is a bit "illegal". A dynamic factor model (Pena & Poncela "Nonstationary dynamic factor analysis" (2006)) could be a PCA counterpart for time series but it may be difficult to estimate (Kalman filter would be slow for a large system). $\endgroup$ – Richard Hardy Mar 30 '15 at 20:47
  • $\begingroup$ In my case there are more than 100 time series data and I want to extract a set of time series to predict a specific time series variable. What would be the best approach to start to find the best predictors? $\endgroup$ – MikeHuber Mar 30 '15 at 20:49
  • $\begingroup$ Thanks Hardy for the advice about PCA with time series data. I expected that it's a common problem and there are some common and well known methods for feature extraction on time series data as there are for sectional data. $\endgroup$ – MikeHuber Mar 30 '15 at 21:09
2
$\begingroup$

The Cross Correlation function will help you identify relationships in your X variables. Box-Jenkins discussed this in their text book. Time Series Analysis: Forecasting and Control

Of course, you will also need to identify outliers as the relationship can be impacted by these events along with changes in trend and level.

Plotting the Y and X in standardized form in a scatterplot and line plot will also support your hypothesis.

$\endgroup$
  • $\begingroup$ Thanks Tom, I am already using cross correlations, but it only works pairwise as far as I know. I was looking for a multivariate approach to explore the relationship of a set of time series variables on the target variable. $\endgroup$ – MikeHuber Mar 30 '15 at 21:06
  • 1
    $\begingroup$ Are you Pre-whitening first? And then doing the cross correlation process? Post your data to dropbox.com Do you have any knowledge about the relationships? Can there be lead relationships? Lag relationships? $\endgroup$ – Tom Reilly Mar 30 '15 at 21:13
  • $\begingroup$ No, honestly not. Is it the goal to filter the autocorrelated part out of it? If so, can I apply PCA after prewhitening the signals? $\endgroup$ – MikeHuber Mar 30 '15 at 21:24
  • 1
    $\begingroup$ Yes, you remove the within to then find the among. Forget PCA in time series. I strongly recommend you consult the book. $\endgroup$ – Tom Reilly Mar 30 '15 at 22:58
2
$\begingroup$

I was also on the search for a list of time series features quite a while ago. There are publications inspecting individual features but I was not able to find a comprehensive list of features.

tsfresh automates extraction of features

While working on industrial machine learning projects I made my own list of features that proved helpful in different applications. This list is contained in the python package tsfresh, which allows to automatically extract a huge of number of features and filter them for their importance.

Comprehensive list of time series features

So regarding your question: You can find inspiration about other features in the comprehensive documentation about the calculated features of tsfresh here. There are simple features such as the mean, time series related features such as the coefficients of an AR model or highly sophisticated features such as the test statistic of the augmented dickey fuller hypothesis test.

I am sure you will find some interesting features for your application there.

Disclaimer: I am one of the authors of tsfresh.

$\endgroup$
  • $\begingroup$ Where can I find example of tsfresh + regressor? $\endgroup$ – SpanishBoy Nov 4 '16 at 19:18
  • $\begingroup$ You can find a collection of examples contained in ipython notebooks at github.com/blue-yonder/tsfresh/tree/master/notebooks $\endgroup$ – MaxBenChrist Nov 5 '16 at 14:39
  • $\begingroup$ I think the question is about finding relevant covariates (features, regressors, indicators...) which help to forecast future values of the time series of interest not 'features' of the time series itself such as a local maximum. Does tsfresh can identify relevant covariates? $\endgroup$ – Arne Jonas Warnke Jun 26 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.