2
$\begingroup$

Can somebody explain ICA(Independently Component Analysis) with a small practical example over here. I have seen lot of programs and libraries written and you can just apply that to your data to find ICA components. One such library is famous python FastICA?

There is a whole Book on ICA, some Tutorial on ICA , some nicely explained PPT on ICA and some Practical Use of ICA to remove ECG artifacts But none of those references gave some practical small example to explain those mathematical concepts behind the ICA stepwise.

It would be I am sure very useful for beginners like me to understand the step wise mathematics of the ICA as just applying library is not enough for deep understanding if ICA.

Note:

I really appreciate the effort if somebody could do that using famous fastICA algorithm and explaining step by the step the mathematics involved in that with evidence of equivalent result using tools like python fastICA library.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ You already mentioned some very good resources (e.g. the Hyvarinen et al. book and tutorial paper). These contain exact descriptions of both the principles and math involved, and also example cases. Perhaps you could say more about what the remaining confusion is, or why these were unsatisfactory? $\endgroup$
    – user20160
    Mar 26, 2018 at 13:24
  • $\begingroup$ what I am asking here is a small workable example that somebody expert in ICA could solve for all of us. I want this resource to be added online so that every beginner could utilize that. $\endgroup$
    – Naseer
    Mar 26, 2018 at 13:42
  • 1
    $\begingroup$ It could help to elaborate or specify a bit more on that since it might still be unclear what the difference is with the already mentioned resources. $\endgroup$
    – Sextus Empiricus
    Mar 26, 2018 at 14:44
  • $\begingroup$ Are you sure? I believe the cocktail party problem would nicely qualify such an example, no? If not, then a more mathematical approach could be to compare it with PCA. The simplified hand-wavy version is that you assume that the signals are uncorrelated and that you can "clean" all of the signals in by finding the linear combination of the signals that gives least correlation. $\endgroup$
    – Andreas Storvik Strauman
    Mar 28, 2018 at 11:27
  • $\begingroup$ Some small workable example using any ICA algorithm..but I think fastICA is so far the famous algorithm for ICA. So if you could do that using fastICA then thats enough. $\endgroup$
    – Naseer
    Mar 28, 2018 at 14:02

1 Answer 1

Reset to default
2
+50
$\begingroup$

I found working out an ICA problem (a simple one about unmixing audios) for the Stanford cs229 course very helpful in understanding its inner working. The basics of ICA isn't that complicated.

Check these out:

Improve this answer
$\endgroup$
7
  • $\begingroup$ Please explain sigmoid part of your code it is a bit cryptic, normalized function and where did you get expected W matrix? $\endgroup$
    – Naseer
    Mar 31, 2018 at 12:48
  • 1
    $\begingroup$ It is a numerically stable sigmoid implementation and calculates positive and negative exponents separately. I've updated its docstring to make it more explicit. And W was initialized as an identity matrix first, see the line with W = np.eye(N) $\endgroup$
    – zyxue
    Mar 31, 2018 at 16:44
  • $\begingroup$ @zyuxe please also explain two things 1) what type of normalization transformation are you using? and 2) How you got expected W matrix of weights? $\endgroup$
    – Naseer
    Apr 1, 2018 at 7:07
  • $\begingroup$ @zyuxe It would have been much intuitive and easy to understand if you could first show us unmixed signals.Then you would mix those signals with some random mixing matrix. After that you applied ICA to compute W matrix and finally you unmix the mixed signals using the W matrix you obtained using ICA? $\endgroup$
    – Naseer
    Apr 1, 2018 at 7:12
  • 1
    $\begingroup$ @NaseerAhmed, not sure what you mean by normalization transformation. The W_expected is just the result I got after running ICA, I put it there as a test for the future if the code is to be modified, to ensure I still get the same W. I don't the raw separate audio, that downloaded from the course is already a mixed one, but you could listen to the unmixed one to see the difference. $\endgroup$
    – zyxue
    Apr 1, 2018 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.