How to find factor that is making matrix singular

Question

I have a 300+ column data.frame, and no matter how I break it up I get this error every time:

Error in solve.default(cv) : 
Lapack routine dgesv: system is exactly singular: U[107,107] = 0

I tried breaking the dataframe up and running vlf() on it then removing factors where the result was infinity, but Ive done this multiple times (each with a smaller datset) and no luck. Is there a better way to tell which factors are causing problems?

Answer 1

You can use an eigen-decomposition to find linear combinations of your columns that vanish, then remove enough columns participating in these linear combinations.

Here's a matrix with a vanishing column linear combination:

> M <- matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow=4,  byrow=TRUE)
> M
     [,1] [,2] [,3]
[1,]    0    0    0
[2,]    1    0    1
[3,]    0    1    1
[4,]    1   -1    0

If a linear combination of the columns vanish, then the same is true if I cut off the bottom of the matrix to make it square:

> sM <- M[1:3, ]
> sM
     [,1] [,2] [,3]
[1,]    0    0    0
[2,]    1    0    1
[3,]    0    1    1

Now compute the eigenvalues and eigenvectors:

> eigen(sM)
$values
[1]  1.618034 -0.618034  0.000000

$vectors
          [,1]       [,2]       [,3]
[1,] 0.0000000  0.0000000  0.5773503
[2,] 0.5257311  0.8506508  0.5773503
[3,] 0.8506508 -0.5257311 -0.5773503

So there's an zero eigenvalue, which we expected, and it corresponds to the column linear combination:

$$ .57 C_1 + .57 C_2 - .57 C_3 = 0 $$

So removing one of these columns will result in a full column rank matrix.