It's about the demostration of the quadratic forms and chi-squared distribution.

Let's split the problem:

  1. We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$. Obviously $z'z$ is a $\chi^2_n$
  2. If we have a symmetric, idempotent matrix (i.e. called $A$) then $z'Az \thicksim \chi^2_p $ where $p$ is the rank of $A$. I got this splitting the $A$ matrix with the spectral theorem into: $Q'\Lambda Q$ hence: $z'Az = z'Q'\Lambda Qz$ we call $w= Qz$ hence $z'Az =w'\Lambda w$. Because it's a idempotent matrix we have only 1s or 0s as eigenvalues and the number of 1's is exactly the rank of $A$. Then we have $w'w$ with with different length now as the number of the rank of $A$. We backward the process and we get: $z'Q'Qz$ but $Q'Q= I_{rank(A)}$ (they are orthogonal since we consider the eigenvectors) so we have the summation of $p$ $Z^2$.
  3. Now the question comes: if we have a symmetric, positive definite matrix $\Sigma_{nxn}$ my teacher told me that $z'\Sigma z$ is a $\chi^2_n $ distribution. I was trying to find a proof, but I can't find one.
  4. My efforts are: use the spectral theorem again, hence we have: $z'Q' \Lambda Qz$ but now $\Lambda$ is a diagonal matrix with real positive numbers (cause it's positive definite).

    $\Lambda$= \begin{bmatrix} \lambda_{1} & 0 & 0 & \dots & 0 \\ 0 & \lambda_{2} & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & \lambda_{n} \end{bmatrix}

We have called, as before, $w=Qz$ hence we have: $w'\Lambda w$. Let's suppose, that w is 3-dimensional vector with 3 elements called a,b and c. So $w=$ \begin{bmatrix} a \\ b \\ c \\ \end{bmatrix}

so $w'\Lambda w= a^2 \lambda_1+b^2 \lambda_2+c^2 \lambda_3$ And here I'm stuck. Maybe this last point is wrong. Therefore I'm asking for clarification and a clear proof of what my teacher said.


I found this:

Let $X$ be a $ K \times 1$ standard multivariate normal random vector, i.e., $X \thicksim N(0,I)$. Let $A$ be an orthogonal $K \times K$ real matrix. Define $Y=AX$ Then also $Y$ has a standard multivariate normal distribution, i.e., $Y \thicksim N(0,I)$. I'm okay with this, the proof is here: https://www.statlect.com/probability-distributions/normal-distribution-quadratic-forms So my problem would come something like this: $w'\Lambda w$ where $w=QZ$ and by this theorem $w \thicksim N(0,I)$ since Q is an orthogonal matrix. So, if w is a 3 dimensional vector I'll have something like this:

$\lambda_1 \chi^2_1 + \lambda_2 \chi^2_1 + \lambda_3 \chi^2_1 $ A lineare combination of chi-squared distribution with weights the value of the eigenvalues. Am I right? If I am, does this sum has a particular distribution?

  • 1
    $\begingroup$ Are you certain that in (3.), $z \sim \mathcal{N}(0,I)$ and not $z \sim \mathcal{N}(0,\Sigma^{-1})$? In which case clearly the quadratic is a $\chi^2$. Otherwise (for std. normal $z$ and symmetric PD $\Sigma$), I don't think $z'\Sigma z$ is distributed $\chi^2$: see answer (here)[stats.stackexchange.com/questions/9220/… for useful leads. $\endgroup$
    – Nate Pope
    Commented Oct 9, 2017 at 2:30
  • $\begingroup$ I added a new part, I don't know if it will be a notification. I'm new here and I'll be glad if you can read my edit to the post. Thank you. $\endgroup$ Commented Oct 9, 2017 at 22:13
  • $\begingroup$ Take a look at the wikipedia page for the generalized $\chi^2$ distribution $\endgroup$
    – Nate Pope
    Commented Oct 9, 2017 at 22:29
  • $\begingroup$ It has a chi-square distribution only under special conditions: stats.stackexchange.com/q/188626/119261. $\endgroup$ Commented Jun 25, 2020 at 18:30
  • $\begingroup$ See also this: math.stackexchange.com/questions/442472/… $\endgroup$ Commented Feb 19, 2021 at 0:35

2 Answers 2


In general, the quadratic form is a weighted sum of $\chi_1^2$

It is not true in general that $\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi^2_p$ for any symmetric positive-definite (variance) matrix $\mathbf{\Sigma}$. Breaking this quadratic form down using the spectral theorem you get:

$$\begin{equation} \begin{aligned} \mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} = \mathbf{z}^\text{T} \mathbf{Q} \mathbf{\Lambda} \mathbf{Q}^\text{T} \mathbf{z} &= (\mathbf{Q}^\text{T} \mathbf{z})^\text{T} \mathbf{\Lambda} (\mathbf{Q}^\text{T} \mathbf{z}) \\[6pt] &= \sum_{i=1}^p \lambda_i ( \mathbf{q}_i \cdot \mathbf{z} )^2, \\[6pt] \end{aligned} \end{equation}$$

where $\mathbf{q}_1,...,\mathbf{q}_p$ are the eigenvectors of $\mathbf{\Sigma}$ (i.e., the columns of $\mathbf{Q}$). Define the random variables $y_i = \mathbf{q}_i \cdot \mathbf{z}$. Since $\mathbf{\Sigma}$ is a real symmetric matrix, the eigenvectors $\mathbf{q}_1,...,\mathbf{q}_p$ are orthonormal, which means that $y_1,...,y_p \sim \text{IID N}(0,1)$. Thus, we have:

$$\begin{equation} \begin{aligned} \mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} &= \sum_{i=1}^p \lambda_i ( \mathbf{q}_i \cdot \mathbf{z} )^2 \\[6pt] &= \sum_{i=1}^p \lambda_i \cdot y_i^2 \\[6pt] &\sim \sum_{i=1}^p \lambda_i \cdot \chi_1^2. \\[6pt] \end{aligned} \end{equation}$$

We can see that the distribution of the quadratic form is a weighted sum of $\chi_1^2$ random variables, where the weights are the eigenvalues of the variance matrix. In the special case where these eigenvalues are all one we do indeed obtain $\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi_n^2$, but in general this result does not hold. In fact, we can see that in general, the quadratic form is distributed as a weighted sum of chi-squared random variables each with one degree-of-freedom.

The general distribution for this form is complicated and its density function does not have a closed form representation. Davies (1980) provides an algorithm to compute the distributions function (the algorithm is actually for a broader generalisation). Bodenham and Adams (2015) examine some approximations to this distribution, and provide comparisons with simulations.

  • $\begingroup$ Awesome, thanks! But, do we really need $\Sigma$ to be positive semi-definite for your proof? It seems to me that it's enough if it is symmetric. Then some $\lambda_i$ might be negative, but it still works. I did some calculations and it seems like allowing negative eigenvalues for $\Sigma$ works well! Thats not a proof, but I think your proof works with negative $\lambda_i$ as well. What do you think? $\endgroup$
    – Tomas
    Commented Jun 25, 2020 at 14:10
  • $\begingroup$ That is correct; if it is not positive semi-definite then you get at least one negative eigenvalue and the quadratic form has a non-zero probability of being negative. However, in most statistical contexts you will encounter, the matrix $\mathbf{\Sigma}$ is a variance matrix. $\endgroup$
    – Ben
    Commented Jun 25, 2020 at 22:10

Regarding 3. above, suppose $\Sigma_{n\times n}$ is diagonal with diagonal elements $\{3, 2, 1\}$, then $\Sigma$ is symmetric positive definite. ...but $z'\Sigma z$ cannot be $\chi^2_n$ (chi-squared with 3 df), because $E\left[z'\Sigma z\right]=3E(z_1^2)+2E(z_2^2)+E(z_3^2)=6$, but the mean of a chi-squared with 3 df is 3.

  • $\begingroup$ Yes, that is correct. Making use of the other answer here, for a diagonal matrix the eigenvalues are the diagonal entries, so in this case we would have $\mathbf{z}^\text{T} \boldsymbol{\Sigma} \mathbf{z} \sim 3 \cdot \chi_1^2 + 2 \cdot \chi_1^2 + 1 \cdot \chi_1^2$ (where each of those parts involve independent chi-squared-one random variables). $\endgroup$
    – Ben
    Commented Nov 12, 2022 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.