Questions tagged [independent-component-analysis]

Independent Component Analysis separates the additive combination of multiple signals into their estimated components.

Filter by
Sorted by
Tagged with
69
votes
2answers
36k views

What is the relationship between independent component analysis and factor analysis?

I am new to Independent Component Analysis (ICA) and have just a rudimentary understanding of the the method. It seems to me that ICA is similar to Factor Analysis (FA) with one exception: ICA assumes ...
21
votes
1answer
530 views

Blind source separation of convex mixture?

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + ...
19
votes
2answers
4k views

Making sense of independent component analysis

I have seen and enjoyed the question Making sense of principal component analysis, and now I have the same question for independent component analysis. I mean I want to make a comprehensive question ...
11
votes
2answers
5k views

How do I select the number of components for independent components analysis?

In the absence of good a priori guesses about the number of components to request in Independent Components Analysis, I'm looking to automate a selection process. I think that a reasonable criterion ...
11
votes
3answers
3k views

PCA, ICA and Laplacian eigenmaps

Question I am very interested in the Laplacian Eigenmaps method. Currently, I am using it to do dimension reduction on my medical data sets. However, I have run into a problem using the method. For ...
10
votes
3answers
7k views

Does ICA require to run PCA first?

I reviewed an application-based paper saying that applying PCA before applying ICA (using fastICA package). My question is, does ICA (fastICA) require PCA to be run first? This paper mentioned that ...
10
votes
1answer
2k views

Using kurtosis to assess significance of components from independent component analysis

In PCA eigenvalues determine the order of components. In ICA I am using kurtosis to obtain the ordering. What are some accepted methods to assess the number, (given I have the order) of components ...
9
votes
1answer
2k views

PCA is to CCA as ICA is to?

PCA looks for factors in data that maximize explained variance. Canonical correlation analysis (CCA), as far as I understand, is like an PCA but looks for a factors that maximize cross covariance ...
9
votes
0answers
582 views

Why are Gaussian distributions the only “forbidden” source distribution for ICA?

I know it's commonly asked why Gaussians are forbidden from use in independent components analysis. This is because a gaussian source distribution will result in the same observed distribution no ...
8
votes
1answer
3k views

Linear independence vs statistical independence (PCA and ICA)

I'm reading this interesting paper on application of ICA to gene expression data. The authors write: [T]here is no requirement for PCA components to be statistically independent. That is true, ...
7
votes
1answer
1k views

What is mean by the non-gaussianity in the independent component analysis(ICA)?

What is mean by non-gaussianity in ICA? Why do we use in ICA? How is Non-Gaussianity is an important and essential principle in ICA estimation? Following is the statement I found in a research paper....
5
votes
2answers
4k views

Why do we whiten the data before running ICA?

Why do we usually pre-whiten the data before doing independent components analysis (ICA), like making all eigenvalues equal? Doesn't that take away some information regarding the data?
4
votes
4answers
1k views

When will PCA be equivalent to ICA?

$X = AS$ where $A$ is my mixing matrix and each column of $S$ represents my sources. $X$ is the data I observe. If the columns of $S$ are independent and Gaussian, will the components of PCA be ...
4
votes
1answer
638 views

What does the number of independent components produced by ICA depend on?

I'm a student working on my bachelor thesis performing independent component analysis (ICA) on some fMRI data using MELODIC FSL. I would like to ask some questions regarding the results of ICA. ...
4
votes
1answer
6k views

Why are we using ICA? [duplicate]

I am new to Independent Component Analysis (ICA) and have a rudimentary understanding of the method. I understand that PCA finds vectors on which the projected data has maximum variance. ICA, on ...
4
votes
1answer
2k views

SVD & ICA — or why doesn't the other rotation matrix in SVD solve for independent components?

When data are a linear mixture of non-gaussian sources, it can be shown that with a rotation, an independent rescaling of each of the rotated axes, and a second rotation you can recover the original, ...
4
votes
1answer
3k views

PCA vs FA vs ICA for dimensionality reduction in questionaire data

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where ...
3
votes
1answer
5k views

ICA - How do I know the optimal number of components?

I'm trying to use the FastICA algorithm in MATLAB. My question is: How do I know, which is the optimal number of ICs? I have a matrix of 62 samples with 1009 signals and the FastICA algorithm returns ...
3
votes
3answers
2k views

Evaluate output of different dimensionality reduction methods

I used PCA, ICA, and FA to perform dimensionality reduction on my data. How can I measure which method performed best? If I reduce my data to 3 dimensions and plot it, what type of trends would ...
3
votes
0answers
36 views

ICA: a question about the non-gaussian requirement

I'm new in the ICA processing and I'm trying to understand the non-gaussian requirement. I read that the problem is that, if the composed data is $\mathbf{x}=\mathbf{As}$ with $\mathbf{A}$ (unknown) ...
2
votes
1answer
458 views

Doesn't the non-Gaussian source assumption of ICA render it practically useless?

Gaussian distributions appear everywhere in nature, indeed this was largely the justification for most classical methods' reliance on assumption of normality. ICA assumes non-Gaussian sources, indeed ...
2
votes
1answer
7k views

The Independence in Independent Component Analysis - Intuitive Explanation

I could use a little bit of help in understanding a concept with regards to ICA: ICA decomposes a multivariate signal into 'independent' components through 1. orthogonal rotation and 2. maximizing ...
2
votes
1answer
1k views

Numerical Example of Independent Component Analysis

Can somebody explain ICA(Independently Component Analysis) with a small practical example over here. I have seen lot of programs and libraries written and you can just apply that to your data to find ...
2
votes
2answers
630 views

ICA, how to check for Gaussian components?

Independent Component Analysis (ICA) requires that at most one of the additive subcomponents of a multivariate signal is Gaussian. If I do not know the distributions of the subcomponents, how do I ...
2
votes
1answer
40 views

Identifiability of a particular Independent Component Analysis model

I am considering the model : $$ \mathbf{x} = \mathbf{A}\mathbf{s} $$ where $\mathbf A \in \mathcal{M}_{n,p}(\mathbb{R})$ and $\mathbf s \in \mathbb{R}^{p}$ such that the entries of $\mathbf s$ are i....
2
votes
1answer
71 views

Why ICA doesn't work on Gaussian data

Examples I found explain this using standard Gaussian data, i.e. $\mathcal{N}(0, I)$ (e.g. in Andrew Ng CS229 lecture page 3), saying that if so, mixing matrix with arbitrary rotations can not be ...
2
votes
1answer
764 views

Is it a good idea to use log scale on scree plots for PCA/ICA/FA?

I always found the concept of determining the "ideal" number of components/factors for an ICA/PCA/FA via a scree plot useful and quick, but also a bit shaky. In an effort to try to make the scree ...
2
votes
1answer
130 views

Proof that the mutual information I(X;Y) between two random variables X and Y is 0 if and only if X and Y are independent

On the Wikipedia of mutual information it says that $I(X;Y)=0$ if and only if $X$ and $Y$ are independent. I can proof that if $X$ and $Y$ are independent, then $I(X;Y)=0$, because $p(x,y)=p(x)p(y)$. ...
2
votes
0answers
142 views

ICA for noise reduction of covariance matrix

Trying to understand ICA in the context of noise reduction of covariance matrices (of dimensionality M). I understand in PCA, you can reconstruct the covariance matrix by squaring the first N ...
2
votes
1answer
66 views

ICA - extract one non-Gaussian source among many Gaussian sources with a-priori information

I have $N$ mixtures consisting of one non-Gaussian source (I know the distribution) and many (more than $N$) Gaussian sources. I also know how my non-Gaussian source is mixed into the signal (I know ...
2
votes
0answers
254 views

What is the difference between independent subspace analysis and independent component analysis?

What is the difference between independent subspace analysis (ISA) and independent component analysis (ICA)?
1
vote
1answer
2k views

Using only one mic in Cocktail Party Algorithm

So I came across this piece of code that separates 2 audio sources from 2 mixed audio sources has shown here: ...
1
vote
1answer
70 views

Why is infinite data required to verify statistical independence

I've been reading about Independent Component Analysis and the FastICA algorithm. The wiki page for FastICA states: FastICA is an efficient and popular algorithm for independent component ...
1
vote
1answer
152 views

FastICA results not exactly consistent on repetition

I have asked this on stack overflow but couldn't get an answer. I am using the fastICA implementation in R. Example code: ...
1
vote
1answer
774 views

What does ICA return?

I am confused with ICA. With PCA I understood that it always gives the components with maximum variance. What does ICA return? Does it return components with maximum independence? How to find best ...
1
vote
2answers
69 views

The lack of correlation determines the second-degree cross-moments (covariances) of a multivariate distribution?

It is given in the following image that lack of correlation determines the second-degree cross-moments (covariances) of a multivariate distribution,while in general statistical independence ...
1
vote
2answers
2k views

Project new data using independent components analysis

I’m trying to project new data into a space I created with icafast. The R package ica does not come with it's own predict function. Here is my attempt to do so. This is in vein of what I can do with ...
1
vote
2answers
59 views

independence in independent component analysis

ICA is quite popular for analyzing brain images (e.g. group ICA). One common assumption/constraint is that the signals in the brain come from "independent spatial sources". I'm confused ...
1
vote
0answers
15 views

How to use ICA as a specific factor rotation in orthogonal factor model

I am trying to understand the way ICA is used as factor rotation in the traditional orthogonal factor model. The idea of using ICA as a specific factor rotation is often mentioned in the literature (...
1
vote
0answers
29 views

Blind source separation using FastICA

Reading the example by scikit-learn on how to use the FastICA function, I couldn't understand the following plot: In the third plot ("ICA recovered signal") - why is the magnitude different ...
1
vote
0answers
33 views

Relationship between minimizing mutual information and maximizing non-Gaussianity in Independent Component Analysis (ICA)

In independent component analysis (ICA), we assume that the observe data $\bf{x}$ from $n$ channels come from linear mixing of $n$ independent sources $\bf{s}$: $$ \bf{x}=\bf{As} $$ And we try to find ...
1
vote
0answers
35 views

Can I combine independent components from different models using PCA?

I have a set of independent components for each subject in my dataset (i.e. an ica model was generated for each subject). The samples used to generate each set of ICs are aligned across subjects, and ...
1
vote
0answers
11 views

Blind source seperation on space data [closed]

in a uni project we gathered spectrum data from the ISS (time x frequency). The challenge is now to analyse this data and especially try to seperate the signals. As far as I understood it the common ...
1
vote
1answer
54 views

Testing statistical independence before using ICA

First a little bit of background. I'm interested in exploring the performance of Independent Component Analysis (ICA) in the context of disentangling intracranial EEG signals. These signals are ...
1
vote
1answer
32 views

FastICA of orthonormal inputs

In Wikipedia page of FastICA it says FastICA seeks an orthogonal rotation of prewhitened data Which means if the input is orthonormal, so would be the output. This can be examined numerically but ...
1
vote
0answers
756 views

Why does the Independent Component Analysis require non-gaussian?

This I found on google while I was going through the Independent Component Analysis in unsupervised learning. Let x = As where A is the Mixing Matrix. So, Lets assume that s here is Gaussian ...
1
vote
0answers
55 views

Connecting PCA singular values to generating parameters

I have a non-linear function of the following form $y_i = f(x_i,\theta)$ where $x_i$ is known and $\theta$ is a vector of parameters. I have generated $n$ realizations of $y_i$ by sampling from a ...
1
vote
0answers
55 views

Understanding Natural gradient learning Independent component analysis

I am quite a novice to statistics and am currently fighting my way through the "Neural Networks and Machine Learning" Book by Haykin. (p.516-518) In the discussion about independent ...
1
vote
1answer
218 views

ICA: plotting independent components

How do I find the directions of the independent components if I have already found the mixing matrix? Let's say that I have this mixing matrix: $$\mathcal W = \begin{bmatrix} 2 & -2 \\ 2 & 4 ...
1
vote
0answers
159 views

Reducing number of variables for Independent Component Analysis

Say I have a dataset with $n$ observations of $p$ random variables. Since $p$ is "large" (mine is 72), I would like to perform a fastICA only on a subset of $k$ variables, maintaining the same number ...