1
$\begingroup$

When using a parametric bootstrap, we use MLEs to parameterize the model. When working on a normal distribution, I assume that we use MLEs for both the mean and variance. But the MLE of variance parameter is biased, and this makes me confused.

I think that the idea of bootstrap is to generate samples from a distribution that approximates the (unknown) true distribution. For a normal distribution, the unbiased estimate of variance better approximates the true variance, than the MLE. So if we used the unbiased variance, data generated from the distribution would better approximate samples from the true distribution.

My question is that do we really use the MLE for the variance of normal distribution? I searched some examples, but surprisingly they all used other non-normal distributions in their examples.

$\endgroup$
  • $\begingroup$ 1. "For a normal distribution, the unbiased estimate of variance better approximates the true variance, than the MLE." ... what is the basis of this statement? (Or ... in what sense to you intend "better" / how are you measuring better there?); why is bias more important to you than, say mean square error? 2. What leads you to imply that it is necessary to use ML (rather than some other consistent estimator) for a parametric bootstrap? There's good reasons to like MLEs and they're often used in a parametric bootstrap, but you speak as if it's required. Is that really the case? $\endgroup$ – Glen_b Jun 7 '17 at 1:17
  • $\begingroup$ It might be good to explain where those premises arise (that an unbiased variance estimate is better than ML and that parametric bootstrap must use ML) ... do you have a text that says these things, for example? $\endgroup$ – Glen_b Jun 7 '17 at 1:25
  • $\begingroup$ I can see that parametric bootstrap is a general method and ML is not a requirement, but educational documents generally use ML (I have never seen other examples) but they do not say ML must be used. $\endgroup$ – quibble Jun 7 '17 at 1:34
  • $\begingroup$ Example ocw.mit.edu/courses/mathematics/… $\endgroup$ – quibble Jun 7 '17 at 1:34
  • $\begingroup$ For example, if we know the true mean is 10 and true variance is 1. Ideally we want to resample from the true distribution. Because we cannot do that, we use bootstrap. But if we use MLE for the variance, we will likely resample from a distribution with smaller variance than the true variance (in this case it is 1). $\endgroup$ – quibble Jun 7 '17 at 1:38
0
$\begingroup$

You can use an unbiased estimate for variance if you want. Why not?

However a little bias is often less important than people seem to think.

If you really do need something to be unbiased, make sure it's the thing that matters for your problem, not some other arbitrary choice that could make the thing that you care about even more biased than the MLE is.

Ideally we want to resample from the true distribution. Because we cannot do that, we use bootstrap.

We bootstrap using something close to the true distribution. But which kind of closeness is more important in a particular simulation? (i.e. make sure you solve the right problem)

Bias in particular is tricky. If it's so essential to have unbiased estimators, why is variance the thing you want to be unbiased? Why not standard deviation or even precision? Perhaps they're more important to have unbiased for your problem -- or maybe you need some complicated function of the mean and standard deviation to be unbiased, for example. You won't get that by unbiasing the variance.

[Note that using the MLE yields ML for variance, standard deviation and precision at the same time, while you can only be unbiased for one -- the others must then be biased.]

if we use MLE for the variance, we will likely resample from a distribution with smaller variance than the true variance

In small samples, yes, you'll sample a little more often from below the true variance than above it.

On the other hand if you use the unbiased version, on average you'll simulate from a version that's further away from the true variance (I might be happy to take a little bias if I miss the target by less)*. This is why it's important to be clear why bias was more important than say mean square error (or indeed some other criterion).

* The ML estimate won't minimize MSE either, but it's closer to it than the unbiased one is.

Note that we can use simulation to investigate the suitability of various choices in some particular circumstance. For example, we can investigate the effect on coverage of a confidence interval on replacing an MLE with some other estimator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.