3
$\begingroup$

I'm studying multivariate distributions in general and I keep coming across an expression like:

$X = \mu + A Y$ where $ \mu $ and $A $ are constant vectors of dimensions d x 1 and d x k respectively. Then $X$ is d x 1 and $Y$ k x 1.

Then theres an expression saying $ \Sigma = AA'$ and my lecturer mentions how we get $A$ via cholesky decomposition. Here's what I dont understand. if $A$ is d x k , i.e. not necessarily a square matrix, how can it be derived using cholesky if cholesky only produces square matrices, at least I think it does. It also seems strage to me that k can be larger or smaller then d, but I think it makes some sense. For example say k was 1, in which case $X$ would be generated from a single univarite distrbution $Y$. Or if d was 2 and k was 5 then $X$ would be a bivarite distribution generated from a linear combination of those distributions making up $Y$ of dimension 5... This second part, which is a bit fuzzy in my head, I think is okay but I really am struggling to understanding how $A$ can be non-square if it's derived using choleskly decomposition.

$\endgroup$
1
$\begingroup$

If $\Sigma$ is positive definite, then it is full rank, and your Cholesky decomposition must be square. This is because $$ d = rank(\Sigma) = rank(AA') = rank(A) \le \min(d,k). $$

More info:

  1. https://math.stackexchange.com/questions/349738/prove-rank-ata-rank-a-for-any-a-m-times-n
  2. https://math.stackexchange.com/questions/782790/relation-between-rank-of-a-symmetric-positive-semi-definite-matrix-and-its-numbe
$\endgroup$
1
$\begingroup$

Your doubts are right: A matrix $A$ that fulfills $\Sigma = AA'$ for a given symmetric positive semidefinite $\Sigma$ is not unique at all. For example, once you have found such an $A$, the matrix $B=\frac{1}{\sqrt{2}}(\begin{array}{cc}A & A\end{array})$ would be another solution.

Using a Cholesky decomposition is just one way to find such an $A$, if $A$ is allowed to have the same size as $\Sigma$ and $\Sigma$ is positive definite. If $A$ is further required to be a triangular matrix, then the Cholesky decomposition is the only unique solution (and in fact, the Cholesky algorithm is the most efficient way to find it). But in general, $A$ is not unique.

This is of no distributional concern if you consider multivariate normal distributions. But have in mind if you want to simulate such random variables, that the single realizations again depend on the choice of $A$, of course. For more general models of $Y$, the choice of $A$ influences even the distribution of $X$.

Now what if $A$ is $k\times d$ and $k<d$? Then, $\Sigma$ is a singular matrix. Such distributions exist, but they don't have a density w.r.t. the Borel measure on $\mathbb{R}$. In particular, the multivariate normal distribution defined via this density doesn't permit singular $\Sigma$. So if $X$ follows a multivariate normal distribution, you can rely on the Cholesky decomposition.

If you still want to find $A$ in such a non-square setting, you can use a singular value decomposition. But again: This is no unique solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.