# Quadratic form and Chi-squared distribution

Let's split the problem:

1. We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$. Obviously $z'z$ is a $\chi^2_n$
2. If we have a symmetric, idempotent matrix (i.e. called $A$) then $z'Az \thicksim \chi^2_p$ where $p$ is the rank of $A$. I got this splitting the $A$ matrix with the spectral theorem into: $Q'\Lambda Q$ hence: $z'Az = z'Q'\Lambda Qz$ we call $w= Qz$ hence $z'Az =w'\Lambda w$. Because it's a idempotent matrix we have only 1s or 0s as eigenvalues and the number of 1's is exactly the rank of $A$. Then we have $w'w$ with with different length now as the number of the rank of $A$. We backward the process and we get: $z'Q'Qz$ but $Q'Q= I_{rank(A)}$ (they are orthogonal since we consider the eigenvectors) so we have the summation of $p$ $Z^2$.
3. Now the question comes: if we have a symmetric, positive definite matrix $\Sigma_{nxn}$ my teacher told me that $z'\Sigma z$ is a $\chi^2_n$ distribution. I was trying to find a proof, but I can't find one.
4. My efforts are: use the spectral theorem again, hence we have: $z'Q' \Lambda Qz$ but now $\Lambda$ is a diagonal matrix with real positive numbers (cause it's positive definite).

$\Lambda$= \begin{bmatrix} \lambda_{1} & 0 & 0 & \dots & 0 \\ 0 & \lambda_{2} & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & \lambda_{n} \end{bmatrix}

We have called, as before, $w=Qz$ hence we have: $w'\Lambda w$. Let's suppose, that w is 3-dimensional vector with 3 elements called a,b and c. So $w=$ \begin{bmatrix} a \\ b \\ c \\ \end{bmatrix}

so $w'\Lambda w= a^2 \lambda_1+b^2 \lambda_2+c^2 \lambda_3$ And here I'm stuck. Maybe this last point is wrong. Therefore I'm asking for clarification and a clear proof of what my teacher said.

[EDIT]

I found this:

Let $X$ be a $K \times 1$ standard multivariate normal random vector, i.e., $X \thicksim N(0,I)$. Let $A$ be an orthogonal $K \times K$ real matrix. Define $Y=AX$ Then also $Y$ has a standard multivariate normal distribution, i.e., $Y \thicksim N(0,I)$. I'm okay with this, the proof is here: https://www.statlect.com/probability-distributions/normal-distribution-quadratic-forms So my problem would come something like this: $w'\Lambda w$ where $w=QZ$ and by this theorem $w \thicksim N(0,I)$ since Q is an orthogonal matrix. So, if w is a 3 dimensional vector I'll have something like this:

$\lambda_1 \chi^2_1 + \lambda_2 \chi^2_1 + \lambda_3 \chi^2_1$ A lineare combination of chi-squared distribution with weights the value of the eigenvalues. Am I right? If I am, does this sum has a particular distribution?

• Are you certain that in (3.), $z \sim \mathcal{N}(0,I)$ and not $z \sim \mathcal{N}(0,\Sigma^{-1})$? In which case clearly the quadratic is a $\chi^2$. Otherwise (for std. normal $z$ and symmetric PD $\Sigma$), I don't think $z'\Sigma z$ is distributed $\chi^2$: see answer (here)[stats.stackexchange.com/questions/9220/… for useful leads. Oct 9, 2017 at 2:30
• I added a new part, I don't know if it will be a notification. I'm new here and I'll be glad if you can read my edit to the post. Thank you. Oct 9, 2017 at 22:13
• Take a look at the wikipedia page for the generalized $\chi^2$ distribution Oct 9, 2017 at 22:29
• It has a chi-square distribution only under special conditions: stats.stackexchange.com/q/188626/119261. Jun 25, 2020 at 18:30

### In general, the quadratic form is a weighted sum of $$\chi_1^2$$

It is not true in general that $$\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi^2_p$$ for any symmetric positive-definite (variance) matrix $$\mathbf{\Sigma}$$. Breaking this quadratic form down using the spectral theorem you get:

\begin{equation} \begin{aligned} \mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} = \mathbf{z}^\text{T} \mathbf{Q} \mathbf{\Lambda} \mathbf{Q}^\text{T} \mathbf{z} &= (\mathbf{Q}^\text{T} \mathbf{z})^\text{T} \mathbf{\Lambda} (\mathbf{Q}^\text{T} \mathbf{z}) \\[6pt] &= \sum_{i=1}^p \lambda_i ( \mathbf{q}_i \cdot \mathbf{z} )^2, \\[6pt] \end{aligned} \end{equation}

where $$\mathbf{q}_1,...,\mathbf{q}_p$$ are the eigenvectors of $$\mathbf{\Sigma}$$ (i.e., the columns of $$\mathbf{Q}$$). Define the random variables $$y_i = \mathbf{q}_i \cdot \mathbf{z}$$. Since $$\mathbf{\Sigma}$$ is a real symmetric matrix, the eigenvectors $$\mathbf{q}_1,...,\mathbf{q}_p$$ are orthonormal, which means that $$y_1,...,y_p \sim \text{IID N}(0,1)$$. Thus, we have:

\begin{equation} \begin{aligned} \mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} &= \sum_{i=1}^p \lambda_i ( \mathbf{q}_i \cdot \mathbf{z} )^2 \\[6pt] &= \sum_{i=1}^p \lambda_i \cdot y_i^2 \\[6pt] &\sim \sum_{i=1}^p \lambda_i \cdot \chi_1^2. \\[6pt] \end{aligned} \end{equation}

We can see that the distribution of the quadratic form is a weighted sum of $$\chi_1^2$$ random variables, where the weights are the eigenvalues of the variance matrix. In the special case where these eigenvalues are all one we do indeed obtain $$\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi_n^2$$, but in general this result does not hold. In fact, we can see that in general, the quadratic form is distributed as a weighted sum of chi-squared random variables each with one degree-of-freedom. The general distribution for this form is complicated and its density function does not have a closed form representation. Bodenham and Adams (2015) examine some approximations to this distribution, and provide comparisons with simulations.

• Awesome, thanks! But, do we really need $\Sigma$ to be positive semi-definite for your proof? It seems to me that it's enough if it is symmetric. Then some $\lambda_i$ might be negative, but it still works. I did some calculations and it seems like allowing negative eigenvalues for $\Sigma$ works well! Thats not a proof, but I think your proof works with negative $\lambda_i$ as well. What do you think? Jun 25, 2020 at 14:10
• That is correct; if it is not positive semi-definite then you get at least one negative eigenvalue and the quadratic form has a non-zero probability of being negative. However, in most statistical contexts you will encounter, the matrix $\mathbf{\Sigma}$ is a variance matrix.
– Ben
Jun 25, 2020 at 22:10

Regarding 3. above, suppose $$\Sigma_{n\times n}$$ is diagonal with diagonal elements $$\{3, 2, 1\}$$, then $$\Sigma$$ is symmetric positive definite. ...but $$z'\Sigma z$$ cannot be $$\chi^2_n$$ (chi-squared with 3 df), because $$E\left[z'\Sigma z\right]=3E(z_1^2)+2E(z_2^2)+E(z_3^2)=6$$, but the mean of a chi-squared with 3 df is 3.