XGboost for Time series - using lag of target variables

I'm trying to make a time series forecast using XGBoost. I have already added many time related variables - day_of_week, month, week_of_month, holiday.

I want to add lagged values of target variable but not sure what is the right approach to build a model with lags. Should the train set have lagged values based on actual data to build model, and test set should have iterative/recursive approach for developing lags?

Thanks!!

The method you are looking for are Auto-Correlation and ARIMA (Auto-Regressive Integrated Moving Averages).

Pandas has a nice and easy implementation of auto-correlation plots that will help you to identify and visualize any temporal correlation in your data.

Next, read up on ARIMA as there are various approaches depending on your data and domain such as: adjusting for temporal trends, adjusting for seasonality, adjusting for scale etc.

One last point, rather than "I want to add lagged values of target variable" it is more common to conceptualize this as shifting the independent variables ($$X$$) backwards in time ($$t$$) i.e. ($$X^{t0}, X^{t-1}, X^{t-2}, X^{t-3}$$ etc) rather than shifting ($$y$$) forward in time.

Update:

You can include lagged versions of $$y$$ as independent variables. Note that if you take a 14 day lag, you will effectively remove the bottom 14 rows of your data - bare this in mind if your sample size is small. To alleviate this issue you can try feature engineering such as moving or running averages of $$y$$.

Also, be sure to check for collinearity if you are adding multiple lagged version of $$y$$.

• I have already tried using ARIMA, and even SARIMAX but the accuracy is very low in both cases. Hence, I wanted to implement XGBoost as it is often seen as a very accurate and robust algorithm for versatile problems. I do want to take values of the dependent variable (y) backwards in time, i.e. shifting (y) – ANP Apr 4 at 6:08
• So you want to include $y^{t-1}$ ...n as a predictor of $y^{t0}$, ie using the previous value of y to predict the next? – BenP Apr 4 at 7:04
• Yes, not just the previous but also values 7 days ago, 2 weeks ago, etc. – ANP Apr 4 at 8:36

I would say that yes, using actual observations during training and and predicted observations during real use is valid.

This is a common approach in natural language generation.

Just to answer your questions about including the lagged variables - Yes, this can be done and we did this recently for a model where we had to capture time variance but also have a number of other predictors tested in the model. We were using weekly data and used last 4 weeks of observed weekly data as lag1 - lag4 variables in the data and these helped the model significantly in our case.