# PCA vs FA vs ICA for dimensionality reduction in questionaire data

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where N is the number of questions) to a more manageable number (usually chosen based on how much variance these dimensions may contain). A key thing to note is that given the fuzzy nature of personality traits it is expected that these dimensions are not orthogonal.

Generally psychologists like to do what I described above via Factor Analysis. I have a basic understanding of the distinctions between PCA, FA, and ICA. I am also aware that ICA is not commonly used for dimensionality reduction.

I have constructed a set of 2D data points distributed normal-ish along two non-orthogonal dimensions to assess the suitability of these methods. The full script for generating the data and plotting the figure can be found here. Admittedly this is about re-mapping the dimensionality, but reducing it would require data of a higher dimensionality than I can nicely plot.

An example of the sort of figure the script would produce is displayed below:

• The second Factor for FA is [0,0]. This does not change even if I manually require the function to return two factors. Why does FA try to squeeze everything into one factor (when it is obvious that is not the latent variable generating my data)? I heard one of the strengths of FA was that it could return non-orthogonal dimensions. Why is that not happening here?
• ICA seems to be doing the right job here. So why is it not used to re-map questionnaire data to more meaningful dimensions? I have heard ICA components are unordered - is that part of the issue? If so, why can't one determine how much of the variance each component explains, and order them accordingly?

So, why would anyone rather use FA than ICA when analyzing questionnaire data?

• Your questions are thoughtful but there are several of almost unrelated ones here. It's usually not a good strategy on CV, it's better to ask clearly focused questions, one at a time. (Q1) Why does FA extract only one factor? That's because you only have 2 dimensions; FA models off-diagonal terms in the covariance matrix and there is only a single unique off-diagonal term in the 2x2 matrix. (Q2) Why are ICA components unordered? I think this is a misconception; I have no idea why it's so widespread. One can order them. (Q3) What's better for questionnaire data? No idea. Commented May 13, 2016 at 20:34
• Does that mean that the capacities of FA cannot be demonstrated on 2D data? Commented May 14, 2016 at 3:38
• One more remark: ICA produces uncorrelated (in fact, it aims to produce independent) components; if your factors "are expected" to be non-orthogonal then it's not clear how you aim to use ICA for that. Commented May 14, 2016 at 16:44
• People are voting to close this as too broad. The main question here seems to be "Why does nobody use ICA to analyze questionnaire data, but only PCA and FA?" and I believe it is not too broad. Commented May 14, 2016 at 16:45
• But in the example above, the two dimensions are correlated - and ICA seems to detect them just fine. Better than FA and PCA in fact. Which brings me to my original question. Commented May 14, 2016 at 16:49