# How to find factor that is making matrix singular

I have a 300+ column data.frame, and no matter how I break it up I get this error every time:

Error in solve.default(cv) :
Lapack routine dgesv: system is exactly singular: U[107,107] = 0


I tried breaking the dataframe up and running vlf() on it then removing factors where the result was infinity, but Ive done this multiple times (each with a smaller datset) and no luck. Is there a better way to tell which factors are causing problems?

• When you say "which factors are causing problems" what are you after? An expect linear combination of the columns that vanishes? Commented May 22, 2015 at 21:41
• Exactly. I am trying to end up with either a complete, invertible matrix or a list of columns that I need to remove Commented May 22, 2015 at 21:46

You can use an eigen-decomposition to find linear combinations of your columns that vanish, then remove enough columns participating in these linear combinations.

Here's a matrix with a vanishing column linear combination:

> M <- matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow=4,  byrow=TRUE)
> M
[,1] [,2] [,3]
[1,]    0    0    0
[2,]    1    0    1
[3,]    0    1    1
[4,]    1   -1    0


If a linear combination of the columns vanish, then the same is true if I cut off the bottom of the matrix to make it square:

> sM <- M[1:3, ]
> sM
[,1] [,2] [,3]
[1,]    0    0    0
[2,]    1    0    1
[3,]    0    1    1


Now compute the eigenvalues and eigenvectors:

> eigen(sM)
$values [1] 1.618034 -0.618034 0.000000$vectors
[,1]       [,2]       [,3]
[1,] 0.0000000  0.0000000  0.5773503
[2,] 0.5257311  0.8506508  0.5773503
[3,] 0.8506508 -0.5257311 -0.5773503


So there's an zero eigenvalue, which we expected, and it corresponds to the column linear combination:

$$.57 C_1 + .57 C_2 - .57 C_3 = 0$$

So removing one of these columns will result in a full column rank matrix.

The plm package has a couple of handy functions for doing this, and they don't require a square matrix. Let's start by loading the package and initializing @Matthew Drury's helpful example:

> library(plm)
> M = matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow = 4,  byrow = T)


The detect.lindep function can be applied directly to a matrix, and it will identify linear dependent columns. We have to add the intercept explicitly in order for the function to work properly; to be honest, I'm not entirely sure why this is.

> matrix.with.intercept = cbind(M, 1)
> detect.lindep(matrix.with.intercept)
[1] "Suspicious column number(s): 1, 2, 3"
[1] "Suspicious column name(s):   "


The alias function goes even further; it specifies the relationship among the linear dependent columns. However, it can't be applied directly to a matrix; it needs a fitted model.

> library(lme4)
> model.data = data.frame(M)
> model.data\$y = 1:4
> model.fit = lm(y ~ X1 + X2 + X3, data = model.data)
> alias(model.fit)
Model :
y ~ X1 + X2 + X3

Complete :
(Intercept) X1 X2
X3 0           1  1