# why is there a huge difference existed in coefficient of determination obtained from 10-fold cross validation?

I'm using gradient boosting regression model (GBRT).

To evaluate this model, I use 10-fold cross validation, in each of which I set same param and compute the coefficient of determination as a measure of fitting.

However, I find that there exists a huge difference in coefficient of determination obtained from each fold, e.g., the coefficient of determination from fold_1 to fold_10 is:

[ 0.95310245  0.89725342  0.886711    **0.97063794**  0.84182142  0.80870443
0.70535911  0.8888032   **0.42510782**  0.70421155]


Although the mean is 0.81 and std is 0.31, there is a fold in which the coefficient of determination is 0.4, while another fold is 0.97.

The only difference btw each fold is just the training & test data set, why does there exist such huge difference? Is such difference indicating that the performance of my model is not good?

• It certainly indicates some sort of problem, but what the problem is is impossible to determine from the limited information you provide. You should look at each model, but particularly the one with the very low value, and see what is going on. – Peter Flom - Reinstate Monica Sep 12 '14 at 11:50
• @PeterFlom Thanks, but I set same parameter for each fold, thus the only difference among folds is the data set. Any suggestion to do next in order to find out the problem? – ice_lin Sep 12 '14 at 12:11
• Provide the sample size, distribution of $Y$, and total number of parameters being fitted or entertained. – Frank Harrell Sep 12 '14 at 13:12