New feature is highly important but not improving the existing model I am trying to predict sales of certain product using regression method. I am using XGboost and using MAPE as final metric for comparison between models. I added a new feature to my existing model and this feature came out to be highly important ( within top 3). But there is no significant change in the final metric MAPE at all. What could be going wrong and what are the possible next steps?
I am performing this in R using xgboost library. Also using xgb.importance() function to get variable importance and using Gain column to understand variable importance. 
I am modeling time series data and it is at weekly level. It has some time varying components & non time varying features (like static product characteristics) as predictor variables.
 A: My first thought would be that your new variable is correlated to the variables you were already using. It does not bring new information but rather 'sums up' the information contained in other variables.
Have you analyzed the correlations of your variables ?
A: Maybe you have not enough training data, so your model overfits your data. This is called high-variance. To validate it, check your model on the training data and also check it on the test data. If the training error is low and test error is high, it suffers from the high-variance problem and you should use more training data to train a better model that can generalize to the test data. You can also use feature selection methods to remove unimportant features. You can also use regularization to control the complexity of the trained model.
A: Beside the reason of it maybe highly correlated with other variables.
There is another reason: it decreases the iteration times (the number of trees) to get the minimum MAPE ( you can do cross validation to confirm that), so it's important features!
