2
$\begingroup$

Examples I found explain this using standard Gaussian data, i.e. $\mathcal{N}(0, I)$ (e.g. in Andrew Ng CS229 lecture page 3), saying that if so, mixing matrix with arbitrary rotations can not be determined from the data, so we can't recover the original sources.

However, if we are using Gaussian data with non-identity covariance matrix, isn't it the case that such problem won't exist? I'm confused about it.

$\endgroup$
1
$\begingroup$

Consider a multivariate Gaussian vector $x \sim N(0,\Sigma)$ with a general covariance matrix $\Sigma \in \mathbb R^{d \times d}$. The goal of ICA is to find a matrix $W$ such that $s = Wx$ has independent components. Both $x$ and $s$ are in $\mathbb R^d$. We can make a couple of observations:

  • We have $\mathbb E (s) = 0$.
  • We know that a linear transformation of a (multivariate) Gaussian vector is again Gaussian, hence $s$ will have a (multivariate) Gaussian distribution no matter what $W$ we choose.
  • We know that the scaling of the individual components of $s$ cannot be recovered, hence, without loss of generality, we might as well assume $\mathbb E( s_i^2) = 1$.
  • We want the components of $s$ to be independent. This implies that the covariance matrix $s$ has to be diagonal.

From these it follows that we have $s \sim N(0,I_d)$, that is, $s$ has to be a Gaussian with the identity covariance matrix. Assuming that $W$ is invertible, with inverse $A = W^{-1}$, we have $x = A s$. Hence, the general Gaussian ICA problem is of the form $x = A s$ where $s \sim N(0,I_d)$.

TL;DR. The only zero-mean Gaussian distribution with independent coordinates and scaling fixed to have unit-variance coordinates is $N(0,I_d)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.