5
$\begingroup$

Why do we usually pre-whiten the data before doing independent components analysis (ICA), like making all eigenvalues equal? Doesn't that take away some information regarding the data?

$\endgroup$
1
  • $\begingroup$ A longer discussion to the same question has evolved here $\endgroup$ – Greg Ver Steeg Sep 19 '15 at 19:21
2
$\begingroup$

ICA assumes that the observed data $\mathbf x$ are a linear combination of independent components: $\mathbf x = \mathbf A \mathbf s$, where $\mathbf A$ is usually called mixing matrix.

Covariance matrix of the data is then given by $\mathbf A \mathbf A^\top$, because the covariance matrix of independent components $\mathbf s$ is identity (indeed, independent components must have zero correlation between each other and are assumed to be scaled to have unit variance each). This means that if the data are pre-whitened, then $\mathbf A$ must be an orthogonal matrix. This is often convenient.

Here is e.g. a quote from Hyvärinen & Oja, 2000, Independent Component Analysis: Algorithms and Applications, section 5.2, called "Whitenening":

Here we see that whitening reduces the number of parameters to be estimated. Instead of having to estimate the $n^2$ parameters that are the elements of the original matrix $\mathbf A$ , we only need to estimate the new, orthogonal mixing matrix $ \tilde{\mathbf A}$ . An orthogonal matrix contains $n ( n - 1)/ 2$ degrees of freedom. For example, in two dimensions, an orthogonal transformation is determined by a single angle parameter. In larger dimensions, an orthogonal matrix contains only about half of the number of parameters of an arbitrary matrix. Thus one can say that whitening solves half of the problem of ICA. Because whitening is a very simple and standard procedure, much simpler than any ICA algorithms, it is a good idea to reduce the complexity of the problem this way.

$\endgroup$
1
$\begingroup$

It does take away some information, but that information is irrelevant for ICA. The whitening step make algorithms for ICA converge faster.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.