When I use 5-k cross validation, is the mean absolute error (MAE) equal to
$$\text{mean}(\text{MAEs of each 5 steps})=\frac{1}{5}(MAE_1 + \cdots + MAE_5)$$
or equal to
$$\text{mean}(\text{absolute errors of all predictions})$$?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ If there are an even number of training examples, wont this be exactly the same? With an odd number then they differ slightly. $\endgroup$– B_MinerSep 19, 2011 at 15:20
-
$\begingroup$ @B_M How does that work? Perhaps you are thinking of medians? $\endgroup$– whuber ♦Sep 19, 2011 at 19:02
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
CV is a simulation of running testing algorithm on unseen data, so the first one.
The second is more an OOB error approximation of a bagging ensemble of your classifiers.
-
$\begingroup$ Indeed, it makes more sense. wikidoc.org/index.php/Cross-validation : "The K results from the folds then can be averaged (or otherwise combined) to produce a single estimation" $\endgroup$– hijackSep 20, 2011 at 14:26