I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?


Sparse matrix is a matrix that has a lot of zeros. Technically, we store such matrices in a way that is more computationally efficient, by storing only non-zero elements, what saves us some memory. There are also specialized algorithms that make computations on sparse matrices more efficient, that on non-sparse matrices by using the fact that a lot of values are zeros.

Now, going back to your question, you cannot really scale the zeroes in the sparse matrix, since $0/c = 0$, you would scale only the non-zero elements. You can center them by subtracting the mean, but first of all, the mean is probably already close to zero if most of values are zeros. Second, this would make your matrix non-sparse, so it would grow in the memory. So centering and scaling does not sound like something that you really want to do.

As described in this thread there are ways how you could make use of the sparsity when doing PCA. Additionally, there are algorithms designed for sparse data, e.g. scikit-learn has RandomizedPCA method that handles them. I would first try some of the approaches designed for sparse data, rather than making it non-sparse. Of course, if after transforming it into non-sparse format it still fits your memory, you could just proceed as usual and ignore the fact that it is sparse.

  • $\begingroup$ Thanks for the reply. The memory is not an issue here. Do you suggest doing the regular scaling then? $\endgroup$ – BlueSky Dec 30 '20 at 18:30
  • $\begingroup$ @BlueSky why not? $\endgroup$ – Tim Dec 30 '20 at 19:29
  • $\begingroup$ My concern is that scaling a matrix with lots of zeros might be an issue for sprasePCA? but I haven't read anywhere that it might be an issue. But I still need to double-check that with others who have more experience $\endgroup$ – BlueSky Dec 30 '20 at 20:01
  • $\begingroup$ @BlueSky SparsePCA creates sparse representation of the data, it has nothing to do with data being sparse. $\endgroup$ – Tim Dec 30 '20 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.