I have a 300+ column data.frame, and no matter how I break it up I get this error every time:
Error in solve.default(cv) :
Lapack routine dgesv: system is exactly singular: U[107,107] = 0
I tried breaking the dataframe up and running vlf() on it then removing factors where the result was infinity, but Ive done this multiple times (each with a smaller datset) and no luck. Is there a better way to tell which factors are causing problems?
You can use an eigen-decomposition to find linear combinations of your columns that vanish, then remove enough columns participating in these linear combinations.
Here's a matrix with a vanishing column linear combination:
> M <- matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow=4, byrow=TRUE)
> M
[,1] [,2] [,3]
[1,] 0 0 0
[2,] 1 0 1
[3,] 0 1 1
[4,] 1 -1 0
If a linear combination of the columns vanish, then the same is true if I cut off the bottom of the matrix to make it square:
> sM <- M[1:3, ]
> sM
[,1] [,2] [,3]
[1,] 0 0 0
[2,] 1 0 1
[3,] 0 1 1
Now compute the eigenvalues and eigenvectors:
> eigen(sM)
$values
[1] 1.618034 -0.618034 0.000000
$vectors
[,1] [,2] [,3]
[1,] 0.0000000 0.0000000 0.5773503
[2,] 0.5257311 0.8506508 0.5773503
[3,] 0.8506508 -0.5257311 -0.5773503
So there's an zero eigenvalue, which we expected, and it corresponds to the column linear combination:
$$ .57 C_1 + .57 C_2 - .57 C_3 = 0 $$
So removing one of these columns will result in a full column rank matrix.
The plm package has a couple of handy functions for doing this, and they don't require a square matrix. Let's start by loading the package and initializing @Matthew Drury's helpful example:
> library(plm)
> M = matrix(c(0, 0, 0, 1, 0, 1, 0, 1, 1, 1, -1, 0), nrow = 4, byrow = T)
The detect.lindep function can be applied directly to a matrix, and it will identify linear dependent columns. We have to add the intercept explicitly in order for the function to work properly; to be honest, I'm not entirely sure why this is.
> matrix.with.intercept = cbind(M, 1)
> detect.lindep(matrix.with.intercept)
[1] "Suspicious column number(s): 1, 2, 3"
[1] "Suspicious column name(s): "
The alias function goes even further; it specifies the relationship among the linear dependent columns. However, it can't be applied directly to a matrix; it needs a fitted model.
> library(lme4)
> model.data = data.frame(M)
> model.data$y = 1:4
> model.fit = lm(y ~ X1 + X2 + X3, data = model.data)
> alias(model.fit)
Model :
y ~ X1 + X2 + X3
Complete :
(Intercept) X1 X2
X3 0 1 1
