# Including class probabilities might skew a model in caret?

I've been training SVMs over some particular data for some time. I was quite happy with the Kappa and Accuracy measures caret gives, but adding some other metrics was not a bad idea at all. The thing is whenever I add classProbs=T to the trainControl function the Cohen's Kappa is diminished in the models generated during the grid search.

I'm quite intrigued by this behavior, but I can't create a reproducible code!

With my data, the drop in the Kappa was from around 0.70 to ~0.10. When trying the same with iris I get something like:

> set.seed(101);TRAIN1<-train(Species~., data = iris, method = "svmLinear",
+ trControl = trainControl(method = "boot", number = 10))
> set.seed(101);TRAIN2<-train(Species~., data = iris, method = "svmLinear",
+ trControl = trainControl(method = "boot", number = 10, classProbs=T))
> set.seed(101);TRAIN3<-train(Species~., data = iris, method = "svmLinear",
+ trControl = trainControl(method = "boot", number = 10, classProbs=TRUE))
>
> TRAIN1$resample Accuracy Kappa Resample 1 1.0000000 1.0000000 Resample01 2 0.9433962 0.9148822 Resample02 3 0.9803922 0.9705373 Resample03 4 0.9824561 0.9731132 Resample04 5 0.9821429 0.9727361 Resample05 6 1.0000000 1.0000000 Resample06 7 0.9649123 0.9470752 Resample07 8 0.9473684 0.9211618 Resample08 9 0.9661017 0.9489619 Resample09 10 0.9491525 0.9233766 Resample10 > TRAIN2$resample
Accuracy     Kappa   Resample
1  1.0000000 1.0000000 Resample01
2  0.9433962 0.9148822 Resample02
3  0.9803922 0.9705373 Resample03
4  0.9824561 0.9731132 Resample04
5  0.9821429 0.9727361 Resample05
6  0.9807692 0.9707042 Resample06
7  0.9473684 0.9205021 Resample07
8  0.9649123 0.9473684 Resample08
9  0.9661017 0.9489619 Resample09
10 0.9661017 0.9489619 Resample10
> TRAIN3\$resample
Accuracy     Kappa   Resample
1  1.0000000 1.0000000 Resample01
2  0.9433962 0.9148822 Resample02
3  0.9803922 0.9705373 Resample03
4  0.9824561 0.9731132 Resample04
5  0.9821429 0.9727361 Resample05
6  0.9807692 0.9707042 Resample06
7  0.9473684 0.9205021 Resample07
8  0.9649123 0.9473684 Resample08
9  0.9661017 0.9489619 Resample09
10 0.9491525 0.9233766 Resample10
>


As you can see, in the 6th and 10th resamples the results differ, but not as drastically as in my own data.

Is there any reason for that?