3
$\begingroup$

It was my understanding that MLE estimates were asymptotically unbiased. The following simulation therefore confuses me. Can anyone help me with my understanding here?

I estimate the parameters of 50 random Weibulls that I've simulated with shape = scale = 1 (using the Wikipedia/R parameterisation)

After lots of simulations, we should* get a distribution of parameter estimates, the median of which equals the simulated values.

The result is that the median of the estimates are significantly different from the simulated parameters:

         shape     scale
est   1.018636 0.9954484
2.5%  1.015124 0.9919716
97.5% 1.021469 0.9984734

I expect I may be wrong at the "should*" above. If so, what is correct?

The R code is below:

I am simulating using rweibull() and estimating parameters using mle() in the core stats4 package and also fitdist() in the fitdistrplus. I am using the correct estimates as the starting parameters, so there shouldn't be any optimisation issues here. (?)

# fitdistrplus::fitdist() fitting
res <- replicate(1e4,
                 fitdist(
                   rweibull(50,shape=1,scale=1),
                   distr="weibull",
                   start=list(shape=1,scale=1)
                 )$est
)

# stats4::mle() fitting
res <- replicate(1e4,{
  y <- rweibull(50,shape=1,scale=1)
  nll <- function(shape,scale) -sum(dweibull(y,shape,scale,log=TRUE))
  res <- stats4::mle(
    nll,
    start=list(shape=1,scale=1)
  )@coef
})

# calculate median of the simulated estimates and its bootstrapped confidence interval
apply(res,1,function(x){
  c(est=median(x),
    quantile(
      replicate(1e3,
                median(sample(x,replace=TRUE))
                ),
      c(0.025,0.975)
      )
    )
})
Improve this question
$\endgroup$

2 Answers 2

Reset to default
6
$\begingroup$

Increase the sample size from fifty & the averages of your parameter estimates over many simulations should get closer still to the true ones—that's what asymptotically unbiased means.

[As @Joe's pointed out, bias (at least "bias" without further qualification—there is such a thing as median-unbiasedness) is the expected difference between the true value & its estimate, so you should be looking at the means rather than the medians in your simulations, if it's this you want to assess.]

Improve this answer
$\endgroup$
8
  • $\begingroup$ So you're saying that even if I had infinite simulations, but still a finite sample size, it would always be biased? I was originally interpreting asymptotically as meaning "If I simulate enough, it will eventually match the original parameters" $\endgroup$
    – sqrt
    Commented May 2, 2014 at 13:30
  • $\begingroup$ That's right. In statistics "asymptotically" means "as the sample size goes to infinity" unless otherwise qualified. You might also be interested in looking at the variance of your estimates, & hence the overall mean-square error. $\endgroup$
    – Scortchi ♦
    Commented May 2, 2014 at 13:33
  • $\begingroup$ OK, so are there ways of correcting this consistent bias for the finite sample size case, or is there a general statistical topic name for this? Thanks $\endgroup$
    – sqrt
    Commented May 2, 2014 at 14:06
  • $\begingroup$ What makes you think the median of the shapes will be equal to the shape? That doesn't mean that the method is biased for finite samples... What's the sampling distribution of the shape parameter? $\endgroup$
    – Joe
    Commented May 2, 2014 at 14:25
  • $\begingroup$ For the general ideas read the chapter on point estimation in a theoretical statistics book. Unbiased estimates sound good but ...(1) They may have higher variance; enough to make the overall MSE higher. (2) Unlike MLEs, they're not in general invariant to reparameterization. So you can get a unbiased estimator of the scale parameter for your Weibull distribution, but its reciprocal won't be an unbiased estimator of the rate. (3) If you choose an estimate that isn't the MLE, your observations were less probable (roughly speaking) if that estimate were right than if the MLE were. $\endgroup$
    – Scortchi ♦
    Commented May 2, 2014 at 14:28
3
$\begingroup$

Wikipedia has a section on finite-sample bias correction for MLEs.

http://en.wikipedia.org/wiki/Maximum_likelihood#Higher-order_properties

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.