This question has three sub-parts, answering each of which probably doesn't require huge text. I hope that is okay.

I'm trying to understand time series prediction using ML. I have the target variable $y_t$, and suppose two other variables $x_t,z_t$ (e.g. if $y_t$ were the demand of an item, $x_t$ could be type of item or price of item, etc.). Also, let's say I'm using a random forest model because I've read it generally does okay out of the box.

i) From my understanding, if I include $y_{t-1}$ as a predictor, the model may just learn to predict $y_t=y_{t-1}$, for example there is autocorrelation with lag $1$. Given that, is it a bad idea to include $y_{t-1}$ as a feature?

ii) Each of the predictors $x_t,z_t$ may have one or the other typical time series characteristics, like non-stationarity, autocorrelation or seasonality. Is there some special method I have to follow or transformation (to the predictor) that I have to do if any of the predictors has any special characteristic?

iii) Typically, what are some best practices to go about such forecasting? My current thought is: use $x_t,z_t$ as predictors without transformation. Use ARIMA with grid searched parameters to fit the training data and validate. Use that as baseline. Finally, use random forest to predict the differenced time series $y_t-y_{t-1}$ using $x_{t-1},z_{t-1}$ as predictors and compare to baseline. Am I missing anything here or should I consider something additional?

Thanks in advance!


Typically what I see for time-series involving trees is two types of setups:

  1. Recursive
  2. Non-Recursive

Where recursive uses past values of the target variable such as last period's sales, moving average of last 4 periods, etc. This does complicate the setup because now you have to predict the next period then recalculate everything and continue predicting. So that brings us to non-recursive which simply doesn't use these types of features. Therefore predicting is straightforward.

Generally, recursive features from our target variable can help the forecast accuracy but lagging features such as last period price is less common. So I would stick with lagging your target and using a moving average/ moving std.

In terms of differencing or something like that, I wouldn't do it initially. Just pass calendar features such as the period month or week or whatever frequency you are at. This is assuming you have no reason to suspect that the target isn't just crazily increasing or decreasing, for example if you selling something on e-commerce and because of covid things are just exploding way out of bounds of what they were before. Trees won't be able to pick up on that trend so then you may want to de-trend or something.

Other features which can be used are things like product id and category. I have seen some neat features using average word vectors of the title but that probably is overkill.

I think a great resource for this would be kaggle's m5 since tree models were pretty heavily used: https://www.kaggle.com/c/m5-forecasting-accuracy/discussion

  • $\begingroup$ Thanks for the answer! Just a bit of clarification regarding the second paragraph: are you saying that it's more common to take $y_{t-1}$, BUT for features, instead of $x_{t-1}$ or $z_{t-1}$, we take the moving average of $x$ and $z$? The e-commerce case sounds like an interesting case study! If I'm trying to predict total views, say, in a week, I can think of the following features: weekly aggregated views in past weeks (lagged target), month of year, sale event - yes/no, price, discount%, weekly aggregated sales from that category, weekly aggregated sales from that brand, etc. $\endgroup$ Jun 24 at 19:06
  • $\begingroup$ Yeah I used some muddy words there, but I am saying that typically people use just lagged and moving average of our target y. Lagged features of our other variables can of course be used but are used less in a lot of competitive solutions I have seen around predicting a product demand (which I assumed as the target y). It's not a rule or anything just something that I tend to notice and you can try lagging those other features. Yeah those features sound swell, I would also add in things like the brand itself and product id and week of year. $\endgroup$
    – Tylerr
    Jun 24 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.