2
$\begingroup$

My slides say that the exponent of a multivariate normal distribution, $(\mathbf{X} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\mathbf{X} - \boldsymbol{\mu})$, follows a chi squared distribution with degrees-of-freedom $p$ (i.e., the length of the $\mathbf{X}$ vector)

My intuitive understanding of chi squared is that it is the sum of (squared) standard normals that are independent.

Therefore back to the first sentence, doesn't this m.variate normal distribution have to be independent (i.e., $X_1, X_2.... X_p$ independent of each other) for this to make sense? The slides did not specify independence.

Thus, what I think is that for the correlation matrix, it will be zero for all off-diagonals. Am I right?

$\endgroup$
1

1 Answer 1

2
$\begingroup$

The quadratic form you are looking at is the squared-Mahalanobis distance. The independence in this computation comes implicitly from the fact that this quadratic form includes the inverse-variance matrix (also called the precision matrix) in the middle. To derive the distributional result at issue, consider a normal random vector $\mathbf{X} \sim \text{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ and suppose we let $\boldsymbol{\Delta} = \sqrt{\boldsymbol{\Sigma}}$ be the principal square root of the variance matrix, so that $\boldsymbol{\Sigma} = \boldsymbol{\Delta} \boldsymbol{\Delta}$. (Note also that $\boldsymbol{\Delta}$ is symmetric, so you can transpose it without changing the result.) Then we can get a standardised normal random vector by taking:

$$\mathbf{Z} \equiv \boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \sim \text{N}(\mathbf{0}, \mathbf{I}).$$

This allows us to rewrite the quadratic form at issue as:

$$\begin{align} (\mathbf{X} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\mathbf{X} - \boldsymbol{\mu}) &= (\mathbf{X} - \boldsymbol{\mu})^\text{T} (\boldsymbol{\Delta} \boldsymbol{\Delta}^\text{T})^{-1} (\mathbf{X} - \boldsymbol{\mu}) \\[12pt] &= (\mathbf{X} - \boldsymbol{\mu})^\text{T} (\boldsymbol{\Delta}^\text{T})^{-1} \boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \\[12pt] &= (\mathbf{X} - \boldsymbol{\mu})^\text{T} (\boldsymbol{\Delta}^{-1})^\text{T} \boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \\[12pt] &= [\boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu})]^\text{T} [\boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu})] \\[12pt] &= \mathbf{Z}^\text{T} \mathbf{Z} \\[6pt] &= \sum_{i=1}^p Z_i^2 \\[6pt] &\sim \text{ChiSq}(p). \\[6pt] \end{align}$$

$\endgroup$
2
  • $\begingroup$ thank you so much for this answer, i'll need some time to digest it... but in short, does it mean that it is necessary for the individual Xs to be independent? since from what i gather from the defined Z, the correlation matrix is an identity matrix, therefore implying independence? $\endgroup$
    – jojorabbit
    Sep 26, 2022 at 16:47
  • 1
    $\begingroup$ No, it doesn't. The $X_i$ values are not independent but the $Z_i$ values are. $\endgroup$
    – Ben
    Sep 26, 2022 at 22:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.