2
$\begingroup$

I'm up to perform certain kinds of sparse decomposition methods on my dataset. However, I'm not sure: what's the tractable data size for the Sparse Decomposition methods?

The dataset is a $10^3\times10^5$ binary matrix, and the expected methods are Sparse PCA, which aims to find a decomposition with minimized coordinate L1-norm and positive constraint.

I found these things useful:

penalized package for R language.

nsprcomp pacakge for R

$\endgroup$
3
  • $\begingroup$ The matrix isn't that big (less than 1 GB). A conventional non-sparse approach should already do the trick just fine. I suggest trying this before you get out the big guns. $\endgroup$ Aug 1, 2013 at 13:26
  • $\begingroup$ "Tractable" is a function of your hardware as much as it's a function of your data. Will you be running this on conventional hardware? A supercomputer? A toaster? Pen and paper? $\endgroup$
    – David Marx
    Aug 1, 2013 at 14:17
  • $\begingroup$ @DavidMarx I'm on my PC only. No parallel clusters, no workstation. $\endgroup$ Aug 2, 2013 at 1:18

2 Answers 2

3
$\begingroup$

It depends on how you structure the problem. If you store the data as a sparse matrix, your datasets can get pretty large before you have a problem. e.g:

#Adapted from:
#http://stat.ethz.ch/R-manual/R-devel/library/Matrix/html/sparseMatrix.html
library(Matrix)
nrow <- 1e3
ncol <- 1e5
nnz <- nrow*ncol*.25
M1 <- sparseMatrix(i = sample(nrow, nnz, replace = TRUE),
                 j = sample(ncol, nnz, replace = TRUE),
                 dims = c(nrow, ncol))

However, the tradoff is that there's no sparse methods for PCA and sparse PCA that I know of. You can do SVD using irlba, which is pretty closely related to PCA:

library(irlba)
SVD <- irlba(M1, nv=25, nu=0) #Takes a long time
PCs <- M1 %*% SVD$v
head(PCs) #Not true principle components, but close

You can also do lasoo regression using glmnet:

library(glmnet)
betas <- runif(ncol)
betas[sample(ncol, ncol-50)] <- 0
betas[betas!=0]
Y <- as.numeric(M1 %*% betas + runif(nrow))/10
model <- cv.glmnet(M1, Y, alpha=1)
CF <- coef(model, model$lambda.1se)
as.numeric(CF[CF!=0])

However, if you truly want to do sparse PCA, you'll have to use one of the packages you referenced (or something similar), which require dense matrices, and which will probably choke on a 1e3 * 1e5 matrix.

$\endgroup$
2
  • $\begingroup$ Exactly. I cannot even assign that large a matrix. Memory overflow for my data. $\endgroup$ Aug 2, 2013 at 1:28
  • $\begingroup$ BTW, I'm trying to use Sparse PCA and it cannot be achieved by classical PCA since constrians other than minimizing square error is applied. $\endgroup$ Aug 2, 2013 at 1:48
1
$\begingroup$

As mentioned the glmnet package can handle datasets with thousands of variables without too many problems.

I just tried doing a PCA on a 1e3 * 1e5 matrix in base R, and it actually worked! Took some time, though (about 5 minutes). Memory usage never went above 3.5GB. This would be a good time to install a multithreaded BLAS, if you'll be doing these analyses frequently.

$\endgroup$
3
  • $\begingroup$ What package are you using? I got a memory error for that. $\endgroup$ Aug 2, 2013 at 1:26
  • $\begingroup$ Just prcomp in the stats package. This is with R 3.0.1 64-bit, on a Windows 7 machine with 12GB memory. $\endgroup$
    – Hong Ooi
    Aug 2, 2013 at 1:52
  • 1
    $\begingroup$ Bloody briliant memory. I've 4GB. $\endgroup$ Aug 2, 2013 at 3:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.