I am confused with ICA. With PCA I understood that it always gives the components with maximum variance. What does ICA return? Does it return components with maximum independence? How to find best number of components that describe the data? Someone told me about Kurtosis. The more kurtotic, the better. I am not sure what it meant.
What could I interpret from the ICA plot that was generated by running ICA with 2 components?
What does ICA return? Does it return components with maximum independence?
Yes, the goal of ICA is to find maximally independent components. But, it's not feasible to directly estimate and minimize measures of dependence like the mutual information between components. Instead, ICA works by optimizing an objective function that acts as a surrogate. Different variants of ICA use different objective functions. For a review, see Hyvarinen and Oja (2000).
Someone told me about Kurtosis. The more kurtotic, the better. I am not sure what it meant.
Maximizing independence between components is closely related to maximizing their non-Gaussianity (again, see Hyvarinen and Oja). In principle, this can be achieved by maximizing the absolute or squared kurtosis, which is one way to measure non-Gaussianity. But, estimating kurtosis is highly sensitive to outliers, so this doesn't provide a good objective function for ICA in practice.
How to find best number of components that describe the data?
The standard ICA model assumes the same number of sources as input dimensions, so it doesn't really provide a choice. One common way to fit fewer components is to reduce the dimensionality using PCA before running ICA. But, reducing the dimensionality too far may result in ICA finding components that are mixtures of the true sources. It can work to fit 'too many' components, then select a subset post hoc (in a way that depends on the specifics the problem).
In various problems where I've applied ICA, it typically worked to fit the full number of components, or to reduce the dimensionality with PCA to slightly more dimensions than the a priori estimated number of sources. A subset of 'meaningful' components could then be identified using domain knowledge about the signals (by examining the component weights and timecourses of the extracted signals) and/or using a model of the physical system that generated the data.
What could I interpret from the ICA plot that was generated by running ICA with 2 components?
Impossible to say, as you haven't described the data, your procedure, or what the axes, points, and colors represent.
References
Hyvarinen and Oja (2000). Independent Component Analysis: Algorithms and Applications.
