# Time Series Features for Multi-Dimensional Problems

I am new to applying ML to time series data but I do have experience doing general supervised learning. I have time series that is multidimensional (so several variables over time) with one output variable. I tried following some online tutorials but I am confused on a few points that keep coming up.

Some tutorials stress checking whether the variables are stationary (Usually they do this using a Dickey-Fuller test).

1. Do I have to check every feature in my time series for this or just the output variable?
2. Can I keep in individual features that are not stationary?
3. Do I have to check if my dataset (features or output variable) are autocorrelated?
4. Do I have to exclude any that are overly autocorrelated?
5. Any other do's and don'ts for time series predictive analysis?

It seems like if I just lag my features with the output then I can readily apply supervised learning models like multiple linear regression or random forests if I cross-validate in a special fashion. Is that really all I have to do to make my time series a supervised learning task?

1. Generally speaking, multivariate time-series should be stationary because it reduces variance in the model. But this is usually looked at a case-by-case basis. The normalisation of such time-series could also prove to be important.

2. You can transform them into stationary time-series. Two main methods are using differencing or Box-Cox transformations. See this section and this answer for more details.

3. Absolutely! Highly correlated time-series make the estimation of the regression coefficients computationally difficult. To understand the correlation in time-series you'll need to first understand ACF and PACF. These links help with that.

It is important to understand that correlations are useful for forecasting, even when there is no causal relationship between the two variables, or when the correlation runs in the opposite direction to the model. However, often a better model is possible if a causal mechanism can be determined.

5. This link should help you differentiate the RNN vs Supervised learning approach for time-series. Generally, if you have a time-series X = [1,2,3,4,5,6,7,8,9,10] then that could be transformed into a supervised learning problem like -

# assuming we only consider lag = 1
| X(t) | X(t-1) | y    |
| ---- | ------ | ---- |
| 1    | 2      | 3    |
| 2    | 3      | 4    |
| 3    | 4      | 5    |
| 4    | 5      | 6    |
| 5    | 6      | 7    |


You can extend this for multivariate time-series like so -

# assuming we only consider lag = 1
| X1(t) | X1(t-1) | X2(t) | X2(t-1) | y    |
| ----- | ------- | ----- | ------- | ---- |
| 1     | 2       | 50    | 60      | 3    |
| 2     | 3       | 60    | 70      | 4    |
| 3     | 4       | 70    | 80      | 5    |
| 4     | 5       | 80    | 90      | 6    |
| 5     | 6       | 90    | 100     | 7    |

$$$$

• Thanks. So what I did so far was check that ALL variables (input/output) were stationary and transformed ones that weren't. I then calculated PACF on ALL variables to see which ones had significant lags (the PACF plot was outside the conf interval). I then added these significant lags as variables similar to how you wrote them in your answer. Is that correct? Can I just train/test a multivariate regression on this like usual? – guy May 19 at 13:45
• Yes, let's take an example. Suppose you have daily data and PACF suggested significant lag = 7 then would you be able to forecast accurately by just taking X(t), X(t-7)? Or would it perform better by incorporating inter-week seasonal patterns X(t), X(t-1)..., X(t-7)? Start with small lags but you might even have to go back 14 days/28 days based on your data. Really depends on a case-to-case basis – Ic3fr0g May 19 at 14:34
• Ah right I see your point. It may be worth exploring different lags nonetheless then. But suppose I wanted to do a regression with this setup. I just regress y onto X1(t), X1(t-1), X2(t), X2(t-1)` with the data in the order it came and test on a holdout set? – guy May 19 at 14:49
• – Ic3fr0g May 19 at 15:15