1
$\begingroup$

I am trying to figure out what I am getting wrong about the following:

Consider an iid Gaussian random variable $X$ with mean $\mu$ and variance $\sigma^2$. Suppose we know $\sigma^2$ and we are trying to estimate $\mu$ by maximum likelihood. By setting the first derivative of the Gaussian log likelihood function equal to zero we get that the MLE for $\mu$ is $\hat{\mu} = \bar{X}$, i.e., the sample mean. The fisher information matrix (which in this case is just a scalar, since we are only interested in $\mu$) is $I := - \text{E}_{\theta} \left[ \frac{\partial L(x|\theta)^2}{\partial \theta^2} \right] = n/\sigma^2$.

According to my econometrics textbook, the asymptotic distribution of the MLE is: $$\sqrt{n} (\bar{X} - \mu) \xrightarrow{d} N(0, I^{-1}).$$ However, this would imply that the asymptotic variance of the estimator is $\sigma^2 /n$, which does not make any sense because (1) the asymptotic variance depends on $n$ and (2) this is known to be the variance of $\bar{X}$.

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ why it makes no sense that the asymptotic variance depends on $n$? the more data you have, the surer you are about your estimate $\endgroup$
    – Alberto
    Commented Oct 2 at 17:33
  • $\begingroup$ The asymptotic variance is the limiting variance of $\sqrt{n}(\bar{X}-\mu)$ when $n \to \infty$, thus it does not depend on $n$. However, the variance (not asymptotic) of $\bar{X}$ is $\sigma^2 / n$ and decrease as $n \to \infty$, according with the fact that the sample mean is a consistent estimator for the population mean. $\endgroup$
    – Residual Claimant
    Commented Oct 2 at 17:37
  • $\begingroup$ Moreover, by the CLT we know that the asymptotic variance of the sample mean is just $\sigma^2$ $\endgroup$
    – Residual Claimant
    Commented Oct 2 at 17:39
  • 1
    $\begingroup$ Are you sure? $\bar{X}\sim N (\mu_x, \frac{\sigma_x^2}{n})$... $\endgroup$
    – Alberto
    Commented Oct 2 at 17:41
  • $\begingroup$ $\sigma_x / \sqrt{n}$ is the standard deviation of the sample mean, while the variance is $var((1/N) \sum_{i} x_i) = (1/N^2)(N \sigma^2) = \sigma^2 /N$. $\endgroup$
    – Residual Claimant
    Commented Oct 2 at 17:45

1 Answer 1

Reset to default
3
$\begingroup$

I think you are confusing the Fisher information matrix for the whole sample, which I will write as $I_n=n/\sigma^2$ and the Fisher information per observation, $I=1/\sigma^2$. You may also be confusing three uses of "asymptotic variance": the variance of the rescaled limiting distribution, the finite-sample variance approximation based on it, and the variance estimator based on that.

The variance of the asymptotic distribution is $I^{-1}$ if $I$ is the per-observation information, which does not depend on $n$ (in the iid case) and so makes sense on the right-hand side of the arrow $$\sqrt{n}(\bar X-\mu)\stackrel{d}{\to} N(0,I^{-1})\equiv N(0,\sigma^2)$$

The asymptotics-based variance approximation for $\bar X$ is $I_n^{-1}=\sigma^2/n$, which is exact in this case but not in general.

The asymptotic variance estimator is $\hat I_n^{-1}=\hat\sigma^2/n$, where $\hat\sigma^2$ is some sensible estimator of $\sigma^2$ -- in many cases it will be the MLE, but for the Normal it's more often the unbiased estimator.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you, I didn’t know there was a distinction between whole sample and per observation. Most sources on the web consider the two interchangeably without rigorous notation $\endgroup$
    – Residual Claimant
    Commented Oct 2 at 21:26
  • $\begingroup$ Just one clarification. Suppose that observations are not iid. Then, by writing $I^{-1}$ we just refer to $I^{-1}_n / n$, am I correct? This would be an average of all the per-observation informations, which are different since data is not iid $\endgroup$
    – Residual Claimant
    Commented Oct 3 at 8:51
  • 1
    $\begingroup$ Strictly, you need $I=\lim_{n\to\infty} I_n/n$ in the non-iid case, because the limit can't depend on $n$. $\endgroup$
    – Thomas Lumley
    Commented Oct 3 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.