I have time series data for two and half year. I am performing xgb.cv (from Jan 2014 to March 2016) and its giving very good result, but when I am using new data (April 2016 to June 2016) to predict, the accuracy is getting worse. I am using R xgboost package. Any suggestion on improving accuracy?
It is likely that your features have time biases. Assume you want to predict month and year from your data. You would see very good performance during CV as you have enough data points at each month. However you can not predict the future(April 2016 to June 2016) very well using your predictor. Another example is that assume you have a feature that its density changes during the time. Then the difference between cv error and your prediction in the future would be big. To solve it, you can use following suggestions:
- Drop time biased features: time related indices, features that has a big shift during time, etc
- Create features that are partially independent of time: from date, get "day of week", "weekend" and so on
First, if there is a trend in time series, then tree-based model maybe not the good choice (because of tree model can't extrapolate, can't predict value bigger or smaller than the value in the training set), or you can remove the trend first, then using the xgboost to predict the residuals of linear models.
Second, as Mortezaaa suggests, if this time series correlation with day of week, holiday, weekends, or season and so on. Then you can construct many features to improve you prediction result! Beside it, the moving average of time series can be the features too. There is a lot of good example on kaggle, such as rossmann-store-sales prediction and bike-sharing-demand prediction, there are time series too, and the winners do a lot of feature engineering! You will learn a lot.