I am trying to make sense of the bootstrap method.

I am studying on Rice, "mathematical statistics and data analysis"

Here it is its explanation of the bootstrap method:

Imagine for the moment that we knew the true values $\lambda_0$ and $\alpha_0$.We could generate many, many samples of size $n$ from the gamma distribution with these parameter values, and from each of these samples we could calculate estimates of$\lambda$ and $\alpha$. A histogram of the values of the estimates of $\lambda$, for example, should then give us a good idea of the sampling distribution of $\hat \lambda$.

This makes sense. But now:

The only problem with this idea is that it requires knowing the true parameter values. So we substitute our estimates of $\lambda$ and $\alpha$ for the true values; that is we draw many, many samples of size $n$ from a gamma distribution with parameters $\alpha = \hat \alpha(\omega)$ and $\lambda = \hat \lambda(\omega)$

(Since $\hat \alpha$ and $\hat \lambda$ are random variables and not numbers, with $\hat \alpha(\omega) $ I mean the actual value that we observe with our data set (ie $\omega$) )

Now this does not make much sense; We want to study the probability distributions of our variables $\hat \alpha$ and $\hat \lambda$, but if in doing do we basically start with the assumption that $\alpha = \hat \alpha(\omega)$ and $\lambda = \hat \lambda(\omega)$!

Is there something I am missing? It does not seem very rigorous to me


It seems to me that you very close to a good understanding of the bootstrap. You are just missing the last stepping stone: we're using the bootstrapping to estimate the uncertainty of our estimate by assuming that the variation around the true value is the same as the variation around our estimate. A full explanation:

For notational simplicity, assume that we have some function of the parameters, $f(\beta)$, we would like to estimate. We have an estimator, e.g. $f(\hat{\beta})$, where $\hat{\beta}$ is some estimate of the parameters. We are now interested in the distribution of this estimator. We might not be able to estimate this theoretically, but we try using bootstrapping methods. From your description this will be a parametric bootstrap as the bootstrap samples are drawn from the estimated distribution.

To estimate the variability in our data, we first draw from the distribution with $\hat{\beta}$ as the true value. This will get us a new estimate $f(\bar{\beta})$. Now we know the true parameter value, because we simulated some data ourselves. Thus, we can evaluate the variation around the true value. We repeat a number of times, giving us a distribution of the estimator when $\hat{\beta}$ is the true value. If we assume that this variation is similar to the variation around the true value $f(\beta)$, we will have an estimate of the variation of $f(\hat{\beta})$.

For instance, if you're interested in a confidence interval for $f(\beta)$ you in some way or another translate the empirical distribution of $f(\hat{\beta}) - f(\bar{\beta})$ to an estimate of the variability in $f(\beta) - f(\hat{\beta})$. There are different methods for doing that, see e.g.

Bootstrapping on Wikipedia

The above article also describes other types of bootstraps, such as non-parametric bootstrapping.

In short, we want to estimate some true, unknown parameter (or function thereof) and evaluate the variability of this estimate. By bootstrapping (parametrically), we let the estimated parameter be the true parameter for a while, simply to estimate the variability of our estimator.

  • $\begingroup$ I am sorry for the long delay. Thank you very much, I think I understand it now.. we take for granted that the variability we find with our estimated parameter is very close to the true one. If I have other doubts I'll ask! Thanks again :) $\endgroup$
    – Ant
    Mar 14 '15 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.