I have a large, sparse
Matrix of features I would like to use in a machine learning algorithm:
library(Matrix) set.seed(42) rows <- 500000 cols <- 10000 i <- unlist(lapply(1:rows, function(i) rep(i, sample(1:5,1)))) j <- sample(1:cols, length(i), replace=TRUE) M <- sparseMatrix(i, j)
Because this matrix has many columns, I would like to reduce it's dimensonality to something more manageable. I can use the excellent irlba package to perform SVD and return the first n principle components (5 shown here; I'll probably use 100 or 500 on my actual dataset):
library(irlba) pc <- irlba(M, nu=5)$u
However, I've read that prior to performing PCA, one should center the matrix (subtract the column mean from each column). This is very difficult to do on my dataset, and furthermore would destroy the sparsity of the matrix.
How "bad" is it to perform SVD on the un-scaled data, and feed it straight into a machine learning algorithm? Are there any efficient ways I could scale this data, while preserving the sparsity of the matrix?