1
$\begingroup$

I am studying the maximal likelihood method to get the estimators. I read the following sentence and I am confused about that

The standard deviation of the sampling distribution of estimator $\hat{\theta}$ is called the standard error denoted by $se(\hat{\theta})$. Since this measures the extent of variability in the estimator, and provided the bias of the estimator is negligible, $se(\hat{\theta})$ is implicitly a measure of how precise the estimator is: the smaller the standard error, the greater the precision.

How to understand this sentence?

In my study, I think if we want to compare two estimators which one is better, we need to compare the mean square error but not the standard error: $$ MSE(\hat{\theta})=Var(\hat{\theta})+bias^2(\hat{\theta}). $$

If I understand correct, we have $Var(\hat{\theta})=[se(\hat{\theta})]^2$. So even if standard error is smaller, it doesn't mean that MSE is smaller, right, because there is still a term bias...

Is this one true for MLE $\hat{\theta}$?

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ "and provided the bias of the estimator is negligible," $\endgroup$
    – Sextus Empiricus
    Commented Nov 7, 2022 at 20:12
  • $\begingroup$ @SextusEmpiricus How to verify this point? In one paper the author only has the practical data, and he only compared the standard error...but he did not state that the bias is $0$. $\endgroup$
    – Hermi
    Commented Nov 7, 2022 at 22:10
  • $\begingroup$ @SextusEmpiricus How about the $\hat{\theta}$ is the MLE of $\theta$? $\endgroup$
    – Hermi
    Commented Nov 7, 2022 at 22:34

1 Answer 1

Reset to default
2
$\begingroup$

  1. Who says that we have to measure square loss? If you look at some other loss function, the bias-variance decomposition does not apply. However, no matter the loss function, we can talk about estimator variance and standard error.

  2. In science, precision refers to tight clustering. For instance, the archer who always misses 2-2.1cm to the right would be considered a precise archer, despite his right bias away from the bullseye. In that sense, you are right that a precise estimator (or archer) could miss the mark (high MSE).

  3. More introductory statistics tends to focus on estimators that are unbiased or at least consistent (bias converges to zero as the sample size increases to infinity). Consequently, your estimator is either unbiased (leaving the entire MSE as a function of the variance) or almost unbiased with a large sample size. Either way, at least most of the MSE will come from the variance.

Note that you are studying maximum likelihood estimators, which all have the property of being consistent.

Improve this answer
$\endgroup$
4
  • $\begingroup$ "Almost unbiased with a large sample size" might be overlooking an important point: if the bias remains appreciably larger than the standard error of the estimate, even as the sample size grows, then bias likely is an issue to consider. The key to this question, IMHO, is that the assumption "provided the bias is negligible" puts us into the framework of minimum-variance unbiased estimators. $\endgroup$
    – whuber ♦
    Commented Nov 7, 2022 at 22:48
  • $\begingroup$ Thanks. So if I have an estimator $\hat{\theta}_1$ obtained from MLE which is consistency, and another estimator $\hat{\theta}_2$ obtained from the method of the moment which is still consistency. How do we compare these two estimators? Can we say the mean square error is smaller, the estimator is better for this case? $\endgroup$
    – Hermi
    Commented Nov 8, 2022 at 4:11
  • 1
    $\begingroup$ @Hermi That actually gets into deeper territory than the original question you posted, and I believe that it warrants a separate post. Briefly, such a decision about which estimator is “better” would come from an analysis of the risk profiles, which relates to notions like admissibility and minimax estimators. $\endgroup$
    – Dave
    Commented Nov 8, 2022 at 4:35
  • $\begingroup$ @Dave Thanks. I have posted this as a new question: stats.stackexchange.com/questions/595004/… $\endgroup$
    – Hermi
    Commented Nov 8, 2022 at 6:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.