7
$\begingroup$

I have a 300+ column data.frame, and no matter how I break it up I get this error every time:

Error in solve.default(cv) : 
Lapack routine dgesv: system is exactly singular: U[107,107] = 0

I tried breaking the dataframe up and running vlf() on it then removing factors where the result was infinity, but Ive done this multiple times (each with a smaller datset) and no luck. Is there a better way to tell which factors are causing problems?

$\endgroup$
2
  • $\begingroup$ When you say "which factors are causing problems" what are you after? An expect linear combination of the columns that vanishes? $\endgroup$ Commented May 22, 2015 at 21:41
  • $\begingroup$ Exactly. I am trying to end up with either a complete, invertible matrix or a list of columns that I need to remove $\endgroup$
    – Rilcon42
    Commented May 22, 2015 at 21:46

2 Answers 2

7
$\begingroup$

You can use an eigen-decomposition to find linear combinations of your columns that vanish, then remove enough columns participating in these linear combinations.

Here's a matrix with a vanishing column linear combination:

> M <- matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow=4,  byrow=TRUE)
> M
     [,1] [,2] [,3]
[1,]    0    0    0
[2,]    1    0    1
[3,]    0    1    1
[4,]    1   -1    0

If a linear combination of the columns vanish, then the same is true if I cut off the bottom of the matrix to make it square:

> sM <- M[1:3, ]
> sM
     [,1] [,2] [,3]
[1,]    0    0    0
[2,]    1    0    1
[3,]    0    1    1

Now compute the eigenvalues and eigenvectors:

> eigen(sM)
$values
[1]  1.618034 -0.618034  0.000000

$vectors
          [,1]       [,2]       [,3]
[1,] 0.0000000  0.0000000  0.5773503
[2,] 0.5257311  0.8506508  0.5773503
[3,] 0.8506508 -0.5257311 -0.5773503

So there's an zero eigenvalue, which we expected, and it corresponds to the column linear combination:

$$ .57 C_1 + .57 C_2 - .57 C_3 = 0 $$

So removing one of these columns will result in a full column rank matrix.

$\endgroup$
1
$\begingroup$

The plm package has a couple of handy functions for doing this, and they don't require a square matrix. Let's start by loading the package and initializing @Matthew Drury's helpful example:

> library(plm)
> M = matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow = 4,  byrow = T)

The detect.lindep function can be applied directly to a matrix, and it will identify linear dependent columns. We have to add the intercept explicitly in order for the function to work properly; to be honest, I'm not entirely sure why this is.

> matrix.with.intercept = cbind(M, 1)
> detect.lindep(matrix.with.intercept)
[1] "Suspicious column number(s): 1, 2, 3"
[1] "Suspicious column name(s):   "

The alias function goes even further; it specifies the relationship among the linear dependent columns. However, it can't be applied directly to a matrix; it needs a fitted model.

> library(lme4)
> model.data = data.frame(M)
> model.data$y = 1:4
> model.fit = lm(y ~ X1 + X2 + X3, data = model.data)
> alias(model.fit)
Model :
y ~ X1 + X2 + X3

Complete :
   (Intercept) X1 X2
X3 0           1  1 
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.