Singular covariance matrix in exploratory Factor Analysis

I'm kind of a noob to EFA and am trying to use the FANode object in Python. This is from the MDP library. I am using it on survey data to see which variables are tied together. Whenever I run it on my data, I get the following error:

mdp.NodeException: The covariance matrix of the data is singular. Redundant dimensions need to be removed.


I was wondering if anyone else has experienced these errors? What do they mean to you? Again, I am quite the noob and would prefer to ask the community before diving into a text book.

• Although some implementations of FA can cope with complete multicollinearity (singularity), that would be a palliative trick. Theoretically FA requires nonsingularity. It assumes that each variable has its independent part, which is also independent of the common factor(s). (That part is called unique factor). This assumption cannot hold under singularity. Jul 15, 2014 at 20:31

Covariance matrix of the data being singular means that some variables in your data set are linear functions of one another. Most typically, this is a full set of dummy variables corresponding to a categorical factor. You put categorical data into your tags, but you did not describe how exactly it shows up in your EFA. Technically speaking, categorical data violates assumptions of EFA (multivariate normal data), so you will probably need to modify your analysis somehow.

The error message, however, speaks of a somewhat poor implementation of EFA. There are EFA methods that can get away with a degenerate matrix, although of course it makes life harder for the methods that rely on inverses and determinants of the covariance matrix of the observed variables. A better implementation should crank through it with a warning.

• Thanks Stask that was super informative. I'll try changing up some of the data or using a different library. MUCH APPRECIATED! Jul 16, 2014 at 21:17
• Also StasK I changed my user name just for you! Jul 16, 2014 at 21:20

That's a surprisingly clear error message. As it says, redundant dimensions need to be removed. Factor analysis doesn't work on extremely collinear variables. For example, consider this data frame:

1    1      1      1
2    2      2      2
3    3      3      3
4    4      4      4
5    5      5      5
6    6      6      6
7    7      7      7
8    8      8      8
9    9      9      9


All covariances for these data are equal (7.5). Therefore the determinant of the covariance matrix is 0, the matrix is not invertible (i.e., is singular), and factor analysis cannot proceed. This can be much less obvious in real datasets, but generally entails that one or more of your variables is/are so strongly related to others as to be completely redundant. Factor analysis should generally be performed on variables that are at least slightly distinct from one another, though smoothing methods exist to help handle extreme collinearity under certain circumstances.

If by survey data you mean Likert scale ratings, see also Factor analysis of questionnaires composed of Likert items. There will be a lot of other issues to consider after addressing the singularity problem.

• Nick you're a god send! I love this site. Thank you so much for the article on Likert items. I think it will solve the problem. Jul 16, 2014 at 21:18
• Yeah, this site is pretty amazing :) Jul 16, 2014 at 21:27