I want to know the asymptotic distribution of $\sqrt{n}(\hat\theta_n-\theta)$ to determine the efficiency of $\hat\theta_n$.

I know there is a theorem with lots of assumptions that immediately concludes that it is efficient and the distribution if $N(0,I(\theta)^{-1})$ but I'm wondering if there is another approach?

I have that $\hat\theta_n = X_{(1)}$ and I have computed $F_{\hat\theta_n}(x)$ and $f_{\hat\theta_n}(x)$ already.

$$f_\theta (x) = e^{\theta-x}1_{x\geq \theta}$$ $$F_{\hat\theta_n} (x) = \left[1-e^{n(\theta-x)} \right] 1_{x\geq \theta}$$ $$f_{\hat\theta_n} (x) = \left[ne^{n(\theta-x)} \right] 1_{x\geq \theta}$$

Is there a way to continue from here that uses the delta method or otherwise?

This is my attempt: We want the distribution of $Y_n = \sqrt{n}(\hat\theta_n-\theta)$. We can rewrite this as $\frac{Y_n}{\sqrt{n}}+\theta = \hat\theta_n$

$$f_{Y_n} (x) = f_{\hat\theta_n} \left(\frac{Y_n}{\sqrt{n}}+\theta\right)$$

$$= ne^{-\sqrt{n}Y_n}1_{Y_n \geq 0}$$

I don't know how to continue from here. I don't see how to arrive from this to a normal approximation so that I can figure out efficiency?! I also thought about using the Delta Method but it seems like it is not applicable in this case?!

  • $\begingroup$ So, just to clarify- $\hat \theta_n = \min_i X_i$ is the MLE for a parameter $\theta$ in a model with density (proportional to) $e^{\theta-x}$ ($x>0$)? $\endgroup$ – P.Windridge Oct 20 '17 at 13:19
  • $\begingroup$ @P.Windridge Yes, precisely so $\endgroup$ – ChineseStatistician Oct 20 '17 at 13:21
  • $\begingroup$ Apologies, I misread $x > \theta$ in the density as as $x > 0$ on my phone. Anyway, check the conditions for the asymptotic normality of the mle- does it apply in your case? You have $\hat \theta_n > \theta$ so $ $Y_n > 0$ and thus cannot be asymptotically normal $\endgroup$ – P.Windridge Oct 20 '17 at 13:45
  • $\begingroup$ No problem. It should be $x\geq \theta$, hence why $\hat\theta_n = X_{(1)}$ $\endgroup$ – ChineseStatistician Oct 20 '17 at 13:48
  • $\begingroup$ Fisher information is not defined for distributions whose support depend on the unknown parameter. math.stackexchange.com/q/3404365/321264 $\endgroup$ – StubbornAtom Apr 18 at 19:09

Part of the sufficient conditions for asymptotical normality of the MLE is that all models in the family have the same support. This fails in your example because it is $(\theta,\infty)$.

In particular this means that $\hat \theta_n = X_{(1)} > \theta$ and so $Y_n > 0$. Thus it cannot be asymptotically normal (other than being degenerate, in case you consider that "normal").

We can compute the distribution more explicitly. As you correctly noted we have $$ \mathbb{P}(Y_n < y) = \mathbb{P}(X_{(1)} < \theta + y/\sqrt{n}). $$ (Note you can't just put $\theta + y/\sqrt{n}$ straight into the density for $X_{(1)}$ - you need to differentiate the expression above and you'd find an extra $1/\sqrt{n}$).

This comes out as $$ \mathbb{P}(Y_n < y) = 1 - \exp(-y \sqrt{n}), $$ i.e. Exponential($\sqrt{n}$). This is not asymptotically normal. The variance is $1/n$. Assuming efficiency is defined by the ratio to the Cramer Rao lower bound, we must compute the Fisher Information $I(\theta)$ and examine the ratio $$ n/I(\theta) $$

The Fisher information is $$ I(\theta) = \mathbb{E}[l'(X;\theta)^2] \\ = \mathbb{E}[ \left((d/d\theta)(n\theta - \sum_i X_i)\right)^2] = n^2. $$

| cite | improve this answer | |
  • $\begingroup$ So the conclusion is that it is not efficient. But according to my exercise, it still approaches some normal distribution and my question was how do you find that normal distribution. $\endgroup$ – ChineseStatistician Oct 20 '17 at 14:12
  • $\begingroup$ Please clarify what you mean by "approaches some normal distribution". $\endgroup$ – P.Windridge Oct 20 '17 at 14:34
  • 1
    $\begingroup$ The question literally says: "Determine the asymptotic distribution of $\sqrt{n}(\hat\theta_n-\theta)$". However I think I was confusing "normal" and "asymptotic". The asymptotic distribution is indeed just the exponential as you have show. Then it ask whether $\hat\theta_n$ is efficient. The only definition of "efficient" that I have says that it should go towards a normal distribution with the variance of that distribution being $1/I(\theta)$. So if the distribution is not normal, but the variance is still $1/I(\theta)$, would that still be an efficient estimator? $\endgroup$ – ChineseStatistician Oct 20 '17 at 14:53
  • $\begingroup$ Thanks, your question is clear now. Often I have seen "efficiency" defined in terms of the ratio to $1/I(\theta)$ (the best according to Cramer Rao), which is $1/n^2$ in your example. So the efficiency here is $1/n$, which is quite poor. $\endgroup$ – P.Windridge Oct 20 '17 at 15:01
  • $\begingroup$ Information function is not usually defined for distributions with parameter in their support. As such the Cramer-Rao bound does not apply. Large sample efficiency of $\hat\theta_n$ in general is defined as the ratio $\operatorname{Var}(T)/\operatorname{Var}(\hat\theta_n)$ where $T$ is UMVUE of $\theta$ (assuming it exists). Here UMVUE does exist and it shows that the MLE $\hat\theta_n$ is indeed (asymptotically) efficient, as it is in most cases. $\endgroup$ – StubbornAtom Oct 23 '19 at 14:49

If in fact $F_{\hat\theta_n} (x) = \left[1-e^{n(\theta-x)} \right] 1_{x\geq \theta}$ then defining $Y_n = n(\hat{\theta}_n - \theta)$ shows that $$ F_{Y_n}(y) = P(\hat{\theta}_n \le \frac{y}{n} + \theta) = 1 - e^{-y}. $$ So $Y_n \sim \text{Exp}(1)$ exactly, not asymptotically, $\hat{\theta}$ converges faster to $\theta$ than root $n$, and you don't need to appeal to any results (just use CDFs). However, do note that I am defining $Y_n$ differently than you are.

Also, the Delta method is typically used when, say a different estimator $\hat{\theta}_n$ is asymptotically Normal. So you assume that $\sqrt{n}(\hat{\theta} - \theta)$ converges already. Then you can use it if you want the asymptotic distribution of some transformed estimator $f(\hat{\theta}_n)$, where $f$ is a function with certain requirements. Typically $\sqrt{n}(f(\hat{\theta}_n) - f(\theta))$ will also be asymptotically Normal with mean $0$ again but with a different variance depending on $f$. Even though it's very popular, I'm not sure it can be of much use here.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.