After running MCC, will the result displayed be for the whole data or the testing data. I read MCC uses 2/3rd data for training and 1/3rd for testing but while checking my results i still see results for whole data and not only for the testing data. I need clarity on how i can get only testing result data. Thank you.
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You seem to have some misunderstanding of cross validation in general.
The data are split not once, but many times -- in normal (k-fold) cross validation there are $k$ such splits and each observation will be both in the test set (for one split) and in the training set (in the remaining ones).
In Monte Carlo CV the splits are done at random each time (rather than cycled through a fixed set of partitions) but the same principle applies (that an observation can serve to be test sometimes and training other times, across multiple splits -- and if you have enough random splits almost all points will have been both).
See the discussion at K-fold vs. Monte Carlo cross-validation for more details about how these work. Also see the relevant section of the Wikipedia page on cross-validation particularly in relation to random sub-smapling
Since we're interested in out-of sample (test set) performance and every point (in k-fold) and all or almost all (in Monte Carlo) will have been in in the test set, we can get some test-set information for (essentially) the whole sample.
