Is large scale PCA even possible? Principal component analysis' (PCA) classical way is to do it on an input data matrix which columns have zero mean (then PCA can "maximize variance"). This can be achieved easily by centering the columns. Howenver, when the input matrix is sparse, the centered matrix will now longer be sparse, and - if the matrix is very big - will thus not fit into memory anymore. Is there an algorithmic solution for the storage problem?
 A: Yes, it is possible.
If the data matrix does not fit into RAM, it is not the end of the world yet: there are efficient algorithms that can work with data stored on a hard drive. See e.g. randomized PCA as described in Halko et al., 2010, An algorithm for the principal component analysis of large data sets.
In Section 6.2 the authors mention that they tried their algorithm on 400k times 100k data matrix and that

The algorithm of the present paper required 12.3 hours to process
all 150 GB of this data set stored on disk, using the laptop computer
with 1.5 GB of RAM [...].

Note that this was in the old days of magnetic hard drives; today there are much faster solid state drives available, so I guess the same algorithm would perform considerably faster.
See also this old thread for more discussion of randomized PCA: Best PCA algorithm for huge number of features (>10K)? and this large 2011 review by Halko et al.: Finding Structure with Randomness:
Probabilistic Algorithms for
Constructing Approximate
Matrix Decompositions.
