2
$\begingroup$

I'm trying to make a time series forecast using XGBoost. I have already added many time related variables - day_of_week, month, week_of_month, holiday.

I want to add lagged values of target variable but not sure what is the right approach to build a model with lags. Should the train set have lagged values based on actual data to build model, and test set should have iterative/recursive approach for developing lags?

Thanks!!

Improve this question
$\endgroup$

4 Answers 4

Reset to default
1
$\begingroup$

The method you are looking for are Auto-Correlation and ARIMA (Auto-Regressive Integrated Moving Averages).

Pandas has a nice and easy implementation of auto-correlation plots that will help you to identify and visualize any temporal correlation in your data.

Next, read up on ARIMA as there are various approaches depending on your data and domain such as: adjusting for temporal trends, adjusting for seasonality, adjusting for scale etc.

One last point, rather than "I want to add lagged values of target variable" it is more common to conceptualize this as shifting the independent variables ($X$) backwards in time ($t$) i.e. ($X^{t0}, X^{t-1}, X^{t-2}, X^{t-3}$ etc) rather than shifting ($y$) forward in time.

Update:

You can include lagged versions of $y$ as independent variables. Note that if you take a 14 day lag, you will effectively remove the bottom 14 rows of your data - bare this in mind if your sample size is small. To alleviate this issue you can try feature engineering such as moving or running averages of $y$.

Also, be sure to check for collinearity if you are adding multiple lagged version of $y$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I have already tried using ARIMA, and even SARIMAX but the accuracy is very low in both cases. Hence, I wanted to implement XGBoost as it is often seen as a very accurate and robust algorithm for versatile problems. I do want to take values of the dependent variable (y) backwards in time, i.e. shifting (y) $\endgroup$
    – ANP
    Commented Apr 4, 2019 at 6:08
  • $\begingroup$ So you want to include $y^{t-1}$ ...n as a predictor of $y^{t0}$, ie using the previous value of y to predict the next? $\endgroup$
    – BenP
    Commented Apr 4, 2019 at 7:04
  • $\begingroup$ Yes, not just the previous but also values 7 days ago, 2 weeks ago, etc. $\endgroup$
    – ANP
    Commented Apr 4, 2019 at 8:36
1
$\begingroup$

I would say that yes, using actual observations during training and and predicted observations during real use is valid.

This is a common approach in natural language generation.

Improve this answer
$\endgroup$
1
$\begingroup$

Just to answer your questions about including the lagged variables - Yes, this can be done and we did this recently for a model where we had to capture time variance but also have a number of other predictors tested in the model. We were using weekly data and used last 4 weeks of observed weekly data as lag1 - lag4 variables in the data and these helped the model significantly in our case.

Improve this answer
$\endgroup$
0
$\begingroup$

Directly using lag of target variable as a feature is a good approach. However, you need to be careful about if model is overfitting due to the lag feature. If you would like to estimate rare peaks on the data along with normal days, previous lag may only be overfitting, you may estimate peaks as a normal day.

In addition to that, if other features are informative, and if lagged feature is highly correlated with the target variable, XGBoost may fail on capturing contribution of other features.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.