How do i compute the bootstrapped mean?

Question

How do i compute the bootstrapped mean? I computed 3 validation accuracies using 3 fold accuracies, and wanted to compute the overall mean, or a bootstrapped mean?

Lets say i have these accuracies for the 3 folds being 0.1, 0.5 and 0.3 .

I understand the concept of bootstrapping, meaning to reuse data you already have, but how do i "reuse" the same data to compute the mean?..

Background:

So I am currently training a DNN acoustic model for speech recognition, and to ensure performance or accuracy of the validation accuracy, I need some idea of the of the range of which the accuracies lies in.

Usually would one do 10 fold CV, but given the amount of data and the time it would take for train all folds, I decided to go to the other way, and compute 3 fold CV, and bootstrapping the resulting distribution (an so called "accuracy distribution") => thereby extracting the mean, variance, and CI..

But doing that doesn't seem to be that simple or?

I initially thought that doing 10 fold CV and 3 fold CV with bootstrapping would in some sense give the same result, but that does not seem to be case?

Answer 1

Typically, you would not use bootstrapping to calculate the mean. Rather, the mean would be the empirical average from your original dataset, and the bootstrapping replicas (of which should there should be many more than 3, incidentally), would be used only to calculate the confidence interval of the mean.

One exception to this would be using bootstrapped bias correction (see Introduction To The Bootstrap, Or Michael Chernik's books), but, here too, you would start off with the mean from the original dataset. In each iteration, you would calculate the difference between the original mean, and the mean of the dataset. Using the frequencies for the biases, you could decide if the original mean is biased. Using bootstrapping for bias correction is dangerous, and is done more rarely.

Edit based on the question clarification

Here is the application of this to the specifics of your case. Suppose you divide your dataset into 3 parts, $A$, $B$, and $C$.

Using $A$ and $B$ you build a model, and predict $C'$; using $A$ and $C$ you build a model, and predict $B'$; using $B$ and $C$ you build a model, and predict $A'$.

Using $A, A'$, $B, B'$, and $C, C'$, you presumably build now $A''$, $B''$, and $C''$, indicating the cross-validated accuracy.

Define now $X = A'' + B'' + C''$, that is, $X$ is the concatenation of these three results. The estimated mean should be the mean of $X$. The confidence intervals should be calculated by bootstrapping $X$.