7
$\begingroup$

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where N is the number of questions) to a more manageable number (usually chosen based on how much variance these dimensions may contain). A key thing to note is that given the fuzzy nature of personality traits it is expected that these dimensions are not orthogonal.

Generally psychologists like to do what I described above via Factor Analysis. I have a basic understanding of the distinctions between PCA, FA, and ICA. I am also aware that ICA is not commonly used for dimensionality reduction.

I have constructed a set of 2D data points distributed normal-ish along two non-orthogonal dimensions to assess the suitability of these methods. The full script for generating the data and plotting the figure can be found here. Admittedly this is about re-mapping the dimensionality, but reducing it would require data of a higher dimensionality than I can nicely plot.

An example of the sort of figure the script would produce is displayed below:

enter image description here

  • The second Factor for FA is [0,0]. This does not change even if I manually require the function to return two factors. Why does FA try to squeeze everything into one factor (when it is obvious that is not the latent variable generating my data)? I heard one of the strengths of FA was that it could return non-orthogonal dimensions. Why is that not happening here?
  • ICA seems to be doing the right job here. So why is it not used to re-map questionnaire data to more meaningful dimensions? I have heard ICA components are unordered - is that part of the issue? If so, why can't one determine how much of the variance each component explains, and order them accordingly?

So, why would anyone rather use FA than ICA when analyzing questionnaire data?

Improve this question
$\endgroup$
20
  • 4
    $\begingroup$ Your questions are thoughtful but there are several of almost unrelated ones here. It's usually not a good strategy on CV, it's better to ask clearly focused questions, one at a time. (Q1) Why does FA extract only one factor? That's because you only have 2 dimensions; FA models off-diagonal terms in the covariance matrix and there is only a single unique off-diagonal term in the 2x2 matrix. (Q2) Why are ICA components unordered? I think this is a misconception; I have no idea why it's so widespread. One can order them. (Q3) What's better for questionnaire data? No idea. $\endgroup$
    – amoeba
    Commented May 13, 2016 at 20:34
  • $\begingroup$ Does that mean that the capacities of FA cannot be demonstrated on 2D data? $\endgroup$
    – TheChymera
    Commented May 14, 2016 at 3:38
  • $\begingroup$ One more remark: ICA produces uncorrelated (in fact, it aims to produce independent) components; if your factors "are expected" to be non-orthogonal then it's not clear how you aim to use ICA for that. $\endgroup$
    – amoeba
    Commented May 14, 2016 at 16:44
  • 2
    $\begingroup$ People are voting to close this as too broad. The main question here seems to be "Why does nobody use ICA to analyze questionnaire data, but only PCA and FA?" and I believe it is not too broad. $\endgroup$
    – amoeba
    Commented May 14, 2016 at 16:45
  • $\begingroup$ But in the example above, the two dimensions are correlated - and ICA seems to detect them just fine. Better than FA and PCA in fact. Which brings me to my original question. $\endgroup$
    – TheChymera
    Commented May 14, 2016 at 16:49

1 Answer 1

Reset to default
3
$\begingroup$

I was curious about your question, because I had never even heard of Independent Component Analysis (ICA), but I use factor analysis all the time. So looking up ICA, I found that one of the key assumptions was that "the values in each source signal have non-Gaussian distributions" (Wikipedia). This doesn't seem like a very helpful assumption if we're trying to discern or confirm a latent construct -- like a personality trait, if we're assuming that our item-responses are being drawn from a normal distribution, or that our latent construct is normally distributed. As such, ICA seems to be used for things like studying radio signals, and not personality traits.

Improve this answer
$\endgroup$
7
  • $\begingroup$ For somebody who knows FA well, a useful (but rarely discussed) perspective on ICA is that ICA is a particular way to do factor rotation in PCA. The criterion that is maximized is indeed (roughly speaking) "non-gaussianity" of each factor. $\endgroup$
    – amoeba
    Commented May 17, 2016 at 16:08
  • $\begingroup$ @5ayat I have become aware of that as well, though for my example above ICA grossly outperforms FA. This may be because my data is not at all what one would expect from a 2-item questionnaire with data based on two correlated latent variables. Can you help me figure out how to construct my data? In the comments on the original post I detail how I have tried to do that but I keep getting a diagonal blob, and FA just detects one factor. $\endgroup$
    – TheChymera
    Commented May 17, 2016 at 17:26
  • $\begingroup$ @amoeba : can you detail on this? In my implementation of software Inside-r and Matmate I've experimented with many, even exotic, rotation criteria. Perhaps a more detailed description of ICA-by-rotation would help me to finally understand that ICA-concept. Under the label "correspondending correlation" and "corresponding regression" (coined by W. Chambers in Semnet and elsewhere) I had in 1996 a longer discussion to recover uniform-distribution latents from mixtures - perhaps this is somehow related. $\endgroup$
    – Gottfried Helms
    Commented May 17, 2016 at 17:59
  • $\begingroup$ 5ayat, I am not convinced by the argument you put forward here. Consider the example provided by @TheChymera in the question. The data form two Gaussian clouds superimposed non-orthogonally; ICA is perfectly able to find these two latent variables, despite the fact that they are Gaussian. $\endgroup$
    – amoeba
    Commented May 17, 2016 at 19:32
  • $\begingroup$ @Gottfried, if you have a question or confusion about ICA then perhaps it's better you ask a separate question about that. I just meant that ICA is usually preceded by PCA-whitening of the data, which essentially means that ICA takes standardized PCA factors and rotates them. Rotation is chosen in order to maximize some measure of "non-Gaussianity", e.g. kurtosis. $\endgroup$
    – amoeba
    Commented May 17, 2016 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.