What is meant by non-gaussianity in ICA? Why do we use in ICA? How is Non-Gaussianity is an important and essential principle in ICA estimation?

Following is the statement I found in a research paper. But I am not able to understand it. Please explain with help of an example and mathematical equations.

Separation of independent signals from their mixtures can be accomplished by making the linear signal transformation as non-Gaussian as possible. Non-Gaussianity is an important and essential principle in ICA estimation.


1 Answer 1


First we look at the central limit theorem, which is basically concerned with the tendancy of estimations of the mean of independently drawn variables of any arbitrary distribution to follow a Gaussian distribution. This matters because in real world samples we are often observing data that is in fact a composite of many underlying factors, and based on the central limit theorem we understand that linear combinations of independent variables create an aggregate variable that tends towards Gaussian in nature.

Non independent variable aggregates can retain non Gaussian distributions as the distributions are linked, but if independent then their combination will tend towards Gaussian (just as the sum of multiple independent fair dice tends towards a normal distribution).

What we want to achieve with ICA is to separate out the independent variables that underlie the observed data, i.e. reverse the central limit theorem. Since the linear combination of independent variables is more Gaussian than the original variables, unless at least one is Gaussian, it follows that using non - Gaussianality is required to identify the underlying variables.

Thus ICA is built on using the assumption of non-Gaussianality in the latent factors to tease them apart. If more than one underlying factor is Gaussian then they will not be separated by ICA since the separation is based on deviation from normality. Basically two Gaussian variables give a circular joint probability for which rotation is arbitrary, so there is no single solution.

https://web.archive.org/web/20210303213322/fourier.eng.hmc.edu/e161/lectures/ica/node3.html http://wwwf.imperial.ac.uk/~nsjones/TalkSlides/HyvarinenSlides.pdf


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.