Let $A$ be an $m \times m$ symmetric matrix with $r=\text{rank}(A)$ and suppose that $x$ ~$N_m(0,I_m)$. Show that the distribution of $x'Ax$ can be expressed as a linear combination of r independent random variables. Note: $A$ is only symmetric, that is, we are not given that $A$ is idempotent.


Hint: use the eigenvalue-eigenvector decomposition.

For a square matrix of rank $r$, we have

$$ \mathbf{A} = \sum_{i=1}^r \lambda_i \mathbf{c}_i \mathbf{c}_i^{T}$$

where $\lambda$ denotes the eigenvalue and $\mathbf{c}$ the (normalized) eigenvector. We then need to exploit the spherical symmetry of the random variables.

We have

$$\mathbf{x}^{T} \mathbf{A} \mathbf{x} = \mathbf{x}^{T} \left( \sum_{i=1}^r \lambda_i \mathbf{c}_i \mathbf{c}_i^{T} \right) \mathbf{x} = \sum_{i=1}^r \lambda_i \left( \mathbf{c}_i^{T} \mathbf{x} \right)^2$$

But since the eigenvectors are normalized to have unit length, it is easy to verify that $\mathbf{c}_i^{T} \mathbf{x} \sim N(0, 1)$. Hence the squares follow the $\chi^2$ distribution and they are also independent because eigenvectors corresponding to different eigenvalues are orthogonal for symmetric matrices. You can see that by writing

$$cov\left(\mathbf{c}_i ^T \mathbf{x} , \mathbf{c}_j ^T \mathbf{x} \right) = \mathbf{c}_i ^T I_m \mathbf{c}_j = 0, \ \ i \neq j $$

and independence follows from the normality.

Note that we are not making any assumptions on the eigenvalues other that $r$ of them are nonzero, i.e. the rank is $r$. The eigenvalues can be positive and negative at the same time, in which case the matrix is indefinite, and the above would still hold.

The problem is that unless the matrix is idempontent, which in turn means that these $r$ nonzero eigenvalues equal 1, we cannot obtain a closed form for the distribution of that expression.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.