Pre-processing time series data for data mining / predictive modeling input

Question

What are some ways to prepare/pre-process time series data to use the series data as a predictor(s) in a predictive model (classification or regression)? Specifically, what are the methods to be considered in order to

• Get the most predicatively useful signal from the data?
• Reduce the dimensionality of the series

As a concrete example, I have 90 days of ending balance data (amount on deposit in a checking account). I want to use that data to predict if the owner of the account will close it in the next 2 weeks (I have an indicator for this occurring or not 2 weeks after the end of the series).

ADDITION:

After reviewing the responses, I think I was looking for a list of commonly used techniques.

• Certainly there are the feature creation that Matt Krause wrote about (each customers balance series are treated separately in all these methods): Things like differences and % changes in the series values each day or week. There are sliding window aggrgations like weekly averages, min, max and standard variations. Also number of increases and decreases, indicators for changes in the balances (absolute or relative of a certain size).
• I have considered fitting a linear or polynomial regression to each serie and using the coefficients in the model as predictors.
• Other things I've wondered about is for each balance series compute the auto correlations for a maximum number of lags and use these values as predictors.
• Cluster the time series into a relatively small number of values and use the indicators for cluster membership as predictors (use dynamic time warping distance and hierarchical clustering for example).

Are there others?

• For example do Fourier Transforms work here? I may post a separate question about them.

Answer 1

I agree with what user765195 said: there's no magic bullet here that will work for all your problems. You've got to come up with potentially-useful features based on your domain knowledge. I've never worked in a bank, so take these suggestions with a grain of salt, but how about

• "Volatility" When I've changed banks, I tend to use the new account for a while before I close the old one, since it takes a while for payroll/recurring charges to get moved over. Maybe the variance of the balance (or changes in the variance over shorter time windows) would capture this?

• Transaction Size Taking the derivative of the daily balances would give you an idea of the (net) daily transactions. Maybe people make anamolously large withdrawals before closing out their accounts (e.g., to set up a new account elsewhere).

If I were you, I would start by making a long list of possible features. Carve your data up into a test set, a development set, and a training set. Test out new features on the training+development sets and see what works. Personally, I would throw them all in and see what happens first. There are lots of feature selection algorithms, ranging from the brain-dead but exhaustive (try all possible combinations!) to something like projection pursuit or hill-climbing, which might be more tractable for big data sets.

Then, once you've settled on a model, use the previously-untouched test data to evaluate its performance.

Answer 2

It all depends on how to plan to make predictions after you've finished pre-processing there data. For example, if you expect consecutive observations of a time series $y_t$ to be linearly related to previous measurements:

$$ y_{t} = \beta_0 + \beta_1 y_{t-1} + \cdots $$

then you'd better choose a linear transformation. Or, if you expect consecutive measurements to be multiplicatively related to each other:

$$ y_{t} = \eta_{0} \cdot y_{t-1}^{\eta_{1}} \cdot \cdot \cdot $$

then a log transformation (and subsequent linear model) would accomplish this since

$$ \log(y_{t}) = \log(\eta_0) + \eta_{1} \log(y_{t-1}) + \cdot \cdot \cdot $$

You may investigate various transformations to see which one provides the best predictive power. But, the "best predictor" will depend on your measure of prediction accuracy.

Regarding dimension reduction, your example only seems to involve 2 variables - dimension reduction often refers to the situation where you have a large number of predictors and want to lower the number in a principled way. If you have that situation, you may use standard techniques (e.g. PCA) for dimension reduction, which are not related to the fact that this is a time series.

If you mean dimension reduction in the sense of reducing the number of timepoints, the only types of transformations that will accomplish this will be things like averaging over timepoints (e.g. to calculate weekly averages from daily data). It would be hard to justify doing something like this unless it is drastically easier to fit weekly data than daily data (for example, for rainfall data, it is easier to predict the average rainfall over the course of a week than it is to predict the daily rainfall values).