4
$\begingroup$

I am new to Independent Component Analysis (ICA) and have a rudimentary understanding of the method.

I understand that PCA finds vectors on which the projected data has maximum variance. ICA, on the other hand, finds vectors on which the projected data is statistically independent however, what those ICA is actually producing as a benefit when applied to a dataset ?

Let say that I want to apply FastICA to the iris dataset of scikit-learn. (maybe not the best example as it only have 4 features however I use it for illustration)

from sklearn import datasets
from sklearn.preprocessing import Imputer
imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
imp.fit(X)
X = datasets.load_iris().data

ica = FastICA(n_components=2)
X_trans = ica.fit_transform(X)
print ica.components_
plt.scatter(X_trans[:, 0], X_trans[:, 1])

enter image description here

In this example I selected 2 components for ICA for visualization purpose however there should be a way to select the optimal number of components ? What does the plot is supposed to show me in this example ? What objet or property the vectors correspond to ? How can this be used in practice?

I also find this post and went through it however my question is a bit more specific. What ICA has to offer if applied on a dataset. In the example I gave, I use ICA. Why one would use ICA? What are the property of the vectors produced by ICA and in the example ?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
4
$\begingroup$

Typical use case for ICA is separation of signals from several independent sources "mixed up" together. A textbook example is the "cocktail party problem" - google that! - when the data is a recording of two or more parallel conversations - hardly intelligible, you might suppose! - and the goal is to recover every one of original discussions separately.

As with most unsupervised methods, the results evaluation and interpretation is closely connected to domain knowledge.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thanks, I have read the example of cocktail party problem however appart from that or maybe some edge face detection into picture dataset i don't see the point of using ICA in traditional dataset like in my example. How can I interpret the vector produced by ICA ? $\endgroup$
    – Michael
    Nov 27, 2015 at 14:43
  • 1
    $\begingroup$ As a rule of thumb, not every statistical method provides meaningful results for every dataset, so there actually may be no point in applying some techniques to a particular dataset. Let me give a brief matematical description, however. In the famous Iris dataset you have 4 features, let's call them $Y=[y_1,y_2,y_3,y_4]$. When you use ICA with two components, you assume the existence of variables $x_1,x_2$ and a 4x2 matrix $A$ such that $Y^T=A[x_1,x_2]$ and try to recover the values of those variables that "produced" your data, transformed by A. The fun part is that matrix A is unknown... $\endgroup$
    – Jacek Podlewski
    Nov 27, 2015 at 14:55
  • 1
    $\begingroup$ The matrix $A$ is unknown, so it has to be estimated somehow - forgive me for skipping the computational details in here, they may actually vary across different implementations - and the recovered vectors of points $[x_1,x_2]$ are what is shown in the plot you obtained in the end. In general, the algorithm maximises some measure of 'source separation', which - as I mentioned - may differ across implementations and algorithm variants. $\endgroup$
    – Jacek Podlewski
    Nov 27, 2015 at 15:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.