I would like to estimate the mean of a population and select a best estimator with minimum variance of the estimated mean. Suppose that I have two estimators est1 and est2, and they could refer to any type of sampling design, such as simple random sampling, stratified sampling, unequal probability sampling, etc. Here is one way that I see very often to compare the estimators:

1) Compute the sample mean $\hat{\mu}$ from the original sample

2) Conduct a bootstrapping simulation (Nboots=1000) of re-sampling the original samples with replacement. For every simulated sample, compute the sample mean $\hat{\mu}_{i}$ as well as the 95% confidence interval using each of the estimators $\hat{\mu}_{i}+/-1.96\times \sigma_{i}^{k}$, where $\sigma_{i}^{k}$ refers to the estimated variance of mean using the ith sample and estimator k (k=1,2).

3) Compute the lower-bound (LB), higher-bound (HB) and actual confidence interval (CI) for each estimator.

The LB for estimator 1 is defined as: $\frac{\text{number of runs that}\ \hat{\mu}>\hat{\mu}_{i}-1.96\times \sigma_{i}^{1}}{Nboots}$

The UB for estimator 1 is defined as: $\frac{\text{number of runs that}\ \hat{\mu}<\hat{\mu}_{i}+1.96\times \sigma_{i}^{1}}{Nboots}$

The actual CI for estimator 1 is defined as: $\frac{\text{number of runs that}\ \hat{\mu}_{i}-1.96\times \sigma_{i}^{1}<\hat{\mu}<\hat{\mu}_{i}+1.96\times \sigma_{i}^{1}}{Nboots}$

4) The estimator with a larger LB, HB and larger CI would then be selected as the best estimator.

My questions are:

1) Can someone provide me some theory or reference about using this method to compare estimators?

2) In case of one of my estimator is a bootstrapping estimator, does it mean that for each bootstrapping simulated sample $i$, I need to conduct again a bootstrapping simulation in order to estimate $\sigma_{i}^{\text{bootstrapping}}$?


  • $\begingroup$ Is there a reason you are not just calculating the MSE of both estimators using bootstrap simulation? If you know the true mean, then repeated re sampling will allow you to better estimate the bias (there should be none) and the variance of the estimator. As you take more boostrap samples, the MSE's will converge to the exact MSE of the empirical distribution (note, not the true MSE..you need an infinite number of data points for that). $\endgroup$ – user31668 Aug 1 '14 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.