2
$\begingroup$

If I can prove that for an estimator $\hat{k}( \theta)$ I can write: $$\frac{\partial l(X_1, \dots , X_n)}{\partial \theta} = a(n, \theta)(\hat{\theta} - \theta)$$

Am i sure that the estimator is unbiased? and consistent?

NB:

  • $l$: is the log likelihood
  • $X_1$ is generated from a regular model
  • $\hat{\theta}$ is the estimator for $\theta$
  • $a(\cdot,\cdot)$ is a function of $n$ and $\theta$ (without any particular meaning i guess)
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I'm not sure how your expression relates to efficiency... Could you please explain? Usually efficiency is only defined for unbiased estimators, so it doesn't make much sense to talk about biased efficient estimators (other than asymptotically). $\endgroup$
    – MånsT
    Commented Jun 28, 2012 at 8:25
  • 5
    $\begingroup$ 1. The equation appears to have no relationship to $\hat{k}(\theta)$ at all. Is it missing something? 2. Because one can define $a(n,\theta)=\frac{\partial l(X_1, \dots , X_n)}{\partial \theta} / (\hat{\theta}-\theta)$, the equation appears to add no information whatsoever. $\endgroup$
    – whuber ♦
    Commented Jun 28, 2012 at 13:16

2 Answers 2

Reset to default
2
$\begingroup$

Estimators that are asymptotically efficient are not necessarily unbiased but they are asymptotically unbiased and consistent. An estimator that is efficient for a finite sample is unbiased. Since efficient estimators achieve the Cramer-Rao lower bound on the variance and that bound goes to 0 as the sample size goes to infinity efficient estimators are consistent.

Improve this answer
$\endgroup$
12
  • 1
    $\begingroup$ -1. The statement you've written isn't true. Efficiency just means that the estimator performs the best in terms of your particular choice of loss function. The statement you've made not only tacitly assumes a particular loss function but also brings in asymptotic properties that don't necessarily follow. $\endgroup$
    – Macro
    Commented Jun 28, 2012 at 11:59
  • 1
    $\begingroup$ I am referring to efficiency in the sense of Fisher which does not involve a loss function and only relates to the Fisher information. Isn't that the commonly used form of the temr? $\endgroup$
    – Michael R. Chernick
    Commented Jun 28, 2012 at 12:09
  • 1
    $\begingroup$ Efficiency is a much more broad term than that. Without more clarification from the OP, I don't think the question can be answered. If we're talking about finite sample efficiency in the Cramer-Rao sense as you're assuming, your statement still isn't true - finite sample efficiency is only used to compare unbiased estimators in terms of MSE, since only the variance appears in the bound. See en.wikipedia.org/wiki/Efficient_estimator $\endgroup$
    – Macro
    Commented Jun 28, 2012 at 12:12
  • 1
    $\begingroup$ Michael, everything you've said is assuming a particular loss function - squared error loss. As I said in my original comment/link, efficiency can be conceived of much more generally. Therefore, your answer, as it currently stands, contains false statements. If you think your answer is perfectly fine in its current state then OK but I'm just explaining my downvote. $\endgroup$
    – Macro
    Commented Jun 28, 2012 at 14:40
  • 1
    $\begingroup$ Michael, the third paragraph of the wiki page you linked to: "Efficiencies are often defined using the variance or mean square error as the measure of desirability", so they are clearly laying out their assumptions, showing an awareness that efficiency actually is a far more general concept. You are not laying out your assumption about the loss function so all of your statements are not true in general. $\endgroup$
    – Macro
    Commented Jun 28, 2012 at 16:47
1
$\begingroup$

In an attempt to... un-unanswer this question:

If one can write the derivative of the log-likelihood as the OP states, then this estimator equals the true value always, irrespective of the realized sample (what a dream, hey?). This is because we choose the estimator so as to make this derivative zero:

$$\hat \theta : \frac{\partial l(\hat \theta \mid X_1, \dots , X_n)}{\partial \theta} =0$$

So, if $$\frac{\partial l(\hat \theta \mid X_1, \dots , X_n)}{\partial \theta} =a(n, \theta) \cdot (\hat{\theta} - \theta) =0 \Rightarrow \hat \theta = \theta$$

(the case $a(n, \theta) =0$ is trivial). In such a case the estimator does not really need any of the usual properties, obviously.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.