I am trying to predict sales of certain product using regression method. I am using XGboost and using MAPE as final metric for comparison between models. I added a new feature to my existing model and this feature came out to be highly important ( within top 3). But there is no significant change in the final metric MAPE at all. What could be going wrong and what are the possible next steps?
I am performing this in
R using xgboost library. Also using
xgb.importance() function to get variable importance and using Gain column to understand variable importance.
I am modeling time series data and it is at weekly level. It has some time varying components & non time varying features (like static product characteristics) as predictor variables.