# How do I select the number of components for independent components analysis?

In the absence of good a priori guesses about the number of components to request in Independent Components Analysis, I'm looking to automate a selection process. I think that a reasonable criterion might be the number that minimizes the global evidence for correlation amongst the computed components. Here's pseudocode of this approach:

for each candidate number of components, n:
run ICA specifying n as requested number of components
for each pair (c1,c2) of resulting components:
compute a model, m1: lm(c1 ~ 1)
compute a model, m2: lm(c1 ~ c2)
compute log likelihood ratio ( AIC(m2)-AIC(m1) ) representing the relative likelihood of a correlation between c1 & c2
compute mean log likelihood ratio across pairs
Choose the final number of components as that which minimizes the mean log likelihood of component relatedness


I figure this should automatically penalize candidates larger than the "true" number of components because ICAs resulting from such candidates should be forced to distribute information from single true components across multiple estimated components, increasing the average evidence of correlation across pairs of components.

Does this make sense? If so, is there a faster way of achieving an aggregate metric of relatedness across estimated components than the mean log likelihood approach suggested above (which can be rather slow computationally)? If this approach doesn't make sense, what might a good alternative procedure look like?

-

I know that this post is a bit old, but thought I would answer the question.

The Variational Ensemble Learning approach to ICA can do this "automatic relevance detection". It automatically turns off components that are not required to improve the bound on the Bayesian Evidence.

Have a look at the thesis of James Miskin available here, which introduces the technique.

This is implemented very eligently in Java by John Winn (another PhD that implements Bayesian Ensemble Learning via a message passing Algorithm).

To learn the technique, I decided to implement Winn's algorithm in c++ which can be obtained from here (active development).

-