3
$\begingroup$

Given a dataset of Gaussian i.i.d. $X_i's$, we can show that the maximum likelihood estimates for this Gaussian distribution's mean and variance are given as: \begin{align} \hat{\mu} &= \frac{1}{n}\sum_{i = 1}^{n} X_i \\[10pt] \widehat{\sigma^2} &= \frac{1}{n}\sum_{i = 1}^{n} (X_i - \hat{\mu})^2 \end{align} We can easily tell that $\hat{\mu}$ will be Gaussian distributed as it is a linear function of Gaussian random variables. But the same cannot be said for $\widehat{\sigma^2}$ as it is a non-linear function of $X_i's$. My question is, what is the best way to determine the variance estimator's distribution for arbitrary $n$?

For the case where $n = 2$ we have: \begin{align} \hat{\sigma} &= \frac{1}{2}\left( \big(X_1 - \frac{1}{2} \left( X_1 + X_2 \right)^2 \big) + \big(X_2 - \frac{1}{2} \left( X_1 + X_2 \right)^2 \big)\right) \\[10pt] \widehat{\sigma^2} &= \frac{1}{4} \left( X_1^2 + X_2^2 - 2X_1X_2 \right) \end{align} Even for this case it is not exactly clear to me how to go about determining the estimator's distribution analytically.

$\endgroup$
  • 2
    $\begingroup$ When the $X$'s are Gaussian, the variance will be distributed as chi-squared. Cf: Why is chi square used when creating a confidence interval for the variance? $\endgroup$ – gung - Reinstate Monica Nov 22 '15 at 18:32
  • 2
    $\begingroup$ So working with moment generating functions or characteristic functions would be the best way to solve for the distribution? Is there a treatment of going about it without using MGFs? Although I imagine it's unwieldy, I'd be interested in seeing how one might attempt it. $\endgroup$ – Falimond Nov 22 '15 at 18:53
1
$\begingroup$

It is possible to prove that $$S^2=\sum_{i=1}^n (X_i-\bar{X})^2\sim \chi^2_{n-1}$$without the mgf. This comes from the property that, if $Z$ is a $n$-dimension Normal random vector, $Z\sim\text{N}(0,I_n)$ and if $A$ is an $n\times n$ symmetric and idempotent matrix, then $$\xi^2=Z^\text{T} A Z \sim \chi^2_{\text{tr}(A)}$$ where $\text{tr}(A)$ denotes the trace of $A$. (The proof of this result does not necessarily involves the mgf.) Since $$S^2=\sum_{i=1}^n (X_i-\bar{X})^2=Z^\text{T} A Z$$when $$A=I_n-\frac{1}{n} \mathbf{1}\mathbf{1}^\text{T}$$and $A$ is idempotent with $n-1$ eigenvalues equal to $1$ and one equal to $0$ (with eigenvector $\mathbf{1}$), the above lemma applies and$$S^2\sim\chi^2_{n-1}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.