6
$\begingroup$

It's about the demostration of the quadratic forms and chi-squared distribution.

Let's split the problem:

  1. We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$. Obviously $z'z$ is a $\chi^2_n$
  2. If we have a symmetric, idempotent matrix (i.e. called $A$) then $z'Az \thicksim \chi^2_p $ where $p$ is the rank of $A$. I got this splitting the $A$ matrix with the spectral theorem into: $Q'\Lambda Q$ hence: $z'Az = z'Q'\Lambda Qz$ we call $w= Qz$ hence $z'Az =w'\Lambda w$. Because it's a idempotent matrix we have only 1s or 0s as eigenvalues and the number of 1's is exactly the rank of $A$. Then we have $w'w$ with with different length now as the number of the rank of $A$. We backward the process and we get: $z'Q'Qz$ but $Q'Q= I_{rank(A)}$ (they are orthogonal since we consider the eigenvectors) so we have the summation of $p$ $Z^2$.
  3. Now the question comes: if we have a symmetric, positive definite matrix $\Sigma_{nxn}$ my teacher told me that $z'\Sigma z$ is a $\chi^2_n $ distribution. I was trying to find a proof, but I can't find one.
  4. My efforts are: use the spectral theorem again, hence we have: $z'Q' \Lambda Qz$ but now $\Lambda$ is a diagonal matrix with real positive numbers (cause it's positive definite).

    $\Lambda$= \begin{bmatrix} \lambda_{1} & 0 & 0 & \dots & 0 \\ 0 & \lambda_{2} & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & \lambda_{n} \end{bmatrix}

We have called, as before, $w=Qz$ hence we have: $w'\Lambda w$. Let's suppose, that w is 3-dimensional vector with 3 elements called a,b and c. So $w=$ \begin{bmatrix} a \\ b \\ c \\ \end{bmatrix}

so $w'\Lambda w= a^2 \lambda_1+b^2 \lambda_2+c^2 \lambda_3$ And here I'm stuck. Maybe this last point is wrong. Therefore I'm asking for clarification and a clear proof of what my teacher said.

[EDIT]

I found this:

Let $X$ be a $ K \times 1$ standard multivariate normal random vector, i.e., $X \thicksim N(0,I)$. Let $A$ be an orthogonal $K \times K$ real matrix. Define $Y=AX$ Then also $Y$ has a standard multivariate normal distribution, i.e., $Y \thicksim N(0,I)$. I'm okay with this, the proof is here: https://www.statlect.com/probability-distributions/normal-distribution-quadratic-forms So my problem would come something like this: $w'\Lambda w$ where $w=QZ$ and by this theorem $w \thicksim N(0,I)$ since Q is an orthogonal matrix. So, if w is a 3 dimensional vector I'll have something like this:

$\lambda_1 \chi^2_1 + \lambda_2 \chi^2_1 + \lambda_3 \chi^2_1 $ A lineare combination of chi-squared distribution with weights the value of the eigenvalues. Am I right? If I am, does this sum has a particular distribution?

$\endgroup$
  • 1
    $\begingroup$ Are you certain that in (3.), $z \sim \mathcal{N}(0,I)$ and not $z \sim \mathcal{N}(0,\Sigma^{-1})$? In which case clearly the quadratic is a $\chi^2$. Otherwise (for std. normal $z$ and symmetric PD $\Sigma$), I don't think $z'\Sigma z$ is distributed $\chi^2$: see answer (here)[stats.stackexchange.com/questions/9220/… for useful leads. $\endgroup$ – Nate Pope Oct 9 '17 at 2:30
  • $\begingroup$ I added a new part, I don't know if it will be a notification. I'm new here and I'll be glad if you can read my edit to the post. Thank you. $\endgroup$ – Mario Migliaccio Oct 9 '17 at 22:13
  • $\begingroup$ Take a look at the wikipedia page for the generalized $\chi^2$ distribution $\endgroup$ – Nate Pope Oct 9 '17 at 22:29
2
$\begingroup$

It is not true in general that $\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi^2_p$ for any symmetric positive-definite (variance) matrix $\mathbf{\Sigma}$. Breaking this quadratic form down using the spectral theorem you get:

$$\begin{equation} \begin{aligned} \mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} = \mathbf{z}^\text{T} \mathbf{Q} \mathbf{\Lambda} \mathbf{Q}^\text{T} \mathbf{z} &= (\mathbf{Q}^\text{T} \mathbf{z})^\text{T} \mathbf{\Lambda} (\mathbf{Q}^\text{T} \mathbf{z}) \\[6pt] &= \sum_{i=1}^p \lambda_i ( \mathbf{q}_i \cdot \mathbf{z} )^2. \\[6pt] \end{aligned} \end{equation}$$

Define the random variables $y_i = \mathbf{q}_i \cdot \mathbf{z}$. Since the eigenvectors $\mathbf{q}_1,...,\mathbf{q}_p$ are orthonormal this means that that $y_1,...,y_p \sim \text{IID N}(0,1)$, so we have:

$$\begin{equation} \begin{aligned} \mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} &= \sum_{i=1}^p \lambda_i ( \mathbf{q}_i \cdot \mathbf{z} )^2 \\[6pt] &= \sum_{i=1}^p \lambda_i \cdot y_i^2 \\[6pt] &\sim \sum_{i=1}^p \lambda_i \cdot \chi_1^2. \\[6pt] \end{aligned} \end{equation}$$

We can see that the distribution of the quadratic form is a weighted sum of $\chi_1^2$ random variables, where the weights are the eigenvalues of the variance matrix. In the special case where these eigenvalues are all one we do indeed obtain $\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi_n^2$, but in general this result does not hold. Bodenham and Adams (2015) examine some approximations to this distribution, and provide comparisons with simulations.

$\endgroup$
0
$\begingroup$

Regarding 3. above, suppose $\Sigma_{n\times n}$ is diagonal with diagonal elements $\{3, 2, 1\}$, then $\Sigma$ is symmetric positive definite. ...but $z'\Sigma z$ cannot be $\chi^2_n$ (chi-squared with 3 df), because $E\left[z'\Sigma z\right]=3E(z_1^2)+2E(z_2^2)+E(z_3^2)=6$, but the mean of a chi-squared with 3 df is 3.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.