I tried to use a GBM model to make predictions for the same data point, but it gave me very different answers. Please see the example below. When using the entire dataset for predicting the first data point, the predicted value is 49.30266. However, when I pulled out the first data point alone for prediction (with the correct factor level for treatment), the results were not consistent. Then, I put into the first observation manually, i.e., manualFirstrow0 and manualFirstrow1 (with wrong treatment value), but they produced the same estimation.
Obviously, the package (GBM version 2.1.4) may have some issues when it comes to categorical features. The correct prediction should be around 50 like presented above. Anyone has any ideas how to deal with this issue?
> set.seed(1) > data(OrchardSprays) > model <- gbm(decrease ~ rowpos+colpos+treatment, data=OrchardSprays, n.trees=1000, distribution="gaussian", interaction.depth=3, bag.fraction=0.5, train.fraction=1.0, shrinkage=0.1, keep.data=TRUE) > > predict(model,newdata=OrchardSprays,n.trees=100)  49.30266 > > firstrow <- OrchardSprays[1,] > manualFirstrow0 <- data.frame(decrease=57,rowpos=1,colpos=1,treatment="D") > manualFirstrow1 <- data.frame(decrease=57,rowpos=1,colpos=1,treatment="A") > > predict(model,newdata=firstrow,n.trees=100)  16.12713 > predict(model,newdata=manualFirstrow0,n.trees=100)  16.12713 > predict(model,newdata=manualFirstrow1,n.trees=100)  16.12713
Here is the link to the original question: Why does GBM predict different values for the same data. But it seems that we cannot solve this issue by fixing the factor issue.