Scaling a sparse matrix

Question

I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?

Answer 1

Sparse matrix is a matrix that has a lot of zeros. Technically, we store such matrices in a way that is more computationally efficient, by storing only non-zero elements, what saves us some memory. There are also specialized algorithms that make computations on sparse matrices more efficient, that on non-sparse matrices by using the fact that a lot of values are zeros.

Now, going back to your question, you cannot really scale the zeroes in the sparse matrix, since $0/c = 0$, you would scale only the non-zero elements. You can center them by subtracting the mean, but first of all, the mean is probably already close to zero if most of values are zeros. Second, this would make your matrix non-sparse, so it would grow in the memory. So centering and scaling does not sound like something that you really want to do.

As described in this thread there are ways how you could make use of the sparsity when doing PCA. Additionally, there are algorithms designed for sparse data, e.g. scikit-learn has RandomizedPCA method that handles them. I would first try some of the approaches designed for sparse data, rather than making it non-sparse. Of course, if after transforming it into non-sparse format it still fits your memory, you could just proceed as usual and ignore the fact that it is sparse.