"matrix is not positive definite" - even when highly correlated variables are removed I am running a factor analysis in SPSS and get a "matrix is not positive definite" error from my correlation matrix. I've tried removing correlated variables, but I have to remove all variables down to those with correlations of r = 0.8 before the "positive definite" issue is resolved. This seems weird, as I have read that this problem normally arises where two or more vbls are nearly perfectly collinear (to me, r = 0.8 is a high correlation, but not near-perfect.) Can anyone suggest a way of dealing with this issue that doesn't just involve arbitrary removal of variables until the problem goes away? Or perhaps explain why I am getting the issue even when I seem to be removing all the really highly correlated variables?            
[edit by Gottfried Helms:] By comment to an answer the OP says, there are 14 variables on 10 cases per correlation.
 A: Something to consider are the subtle ways your data may be collinear.
Multicollinearity: greatest hits & common mistakes:
1. Complete dummy variables (I'd guess the most common error)
Let's say your regression includes a constant and a dummy variable for January, a dummy for February, etc... all the way to December. Let's call these dummies $x_1$, $x_2$, ...., $x_{12}$.
$$ x_1 + x_2 + \ldots + x_{11} + x_{12} = 1$$
Therefore your data is collinear. If you're including dummy variables, you need to leave the dummy for one category out.
2. Components that add together to some total
I'll give a corporate finance example.
Imagine $x_1$ is short-term debt. $x_2$ is long-term debt, and $x_3$ is total debt. Obviously we have:
$$ x_3 = x_1 + x_2 $$
If you put all three on the right hand side your, your regressors are collinear.
3. Context specific linear identities
Imagine you have some regression with assets, liabilities, and shareholder's equity on the right hand side. Sound legit? Nope! The most basic accounting identity is that:
$$ \mathrm{Assets}_i = \mathrm{Liabilities}_i + \mathrm{ShareholderEquity}_i$$
Mechanically, that equation always holds true. Including two of the three would be fine. Including all three makes your data collinear.
4. Some more subtle examples...
Let $x_1$ be the age of your individual. Let $x_2$ be years of schooling.
If everyone in your sample starts school at the same age and doesn't drop out, then $x_1$ and $x_2$ are collinear.
Conclusion
Observe that in (1), (2), and (3), your pairwise correlations would be useless in identifying the collinearity problem. These are all examples where THREE or more variables are collinear, not the obvious situation where $x_1 = \alpha x_2$.
This is also why the Cholesky decomposition @Gottfried Helms discusses might be useful.
A: A correlation matrix is positive semidefinite, by definition.
However, the vast majority of correlation matrices are actually positive definite.  There is a detailed explanation of this at the following link:  http://www.uic.edu/classes/bstt/bstt580/jw4e/nts02.htm in sect 2.6)).  This means that you have at least one redundant variable in your analysis.  How many variables are you working with? 
A: This can happen if you have missing values and you are computing pairwise correlation matrix instead of listwise.
