This is the difference between centered (or demeaned) and uncentered PCA. One recent (2016) paper that among others discuss this is Principal component analysis:a review and recent
developments by Ian T. Jolliffe and Jorge Cadima, which can be found here. I will just cite what they say:
As was seen in §2, PCA amounts to an SVD of a column-centred data
matrix. In some applications [see below], centring the columns of the
data matrix may be considered inappropriate. In such situations, it
may be preferred to avoid any pre-processing of the data and to
subject the uncentred data matrix to an SVD or, equivalently, to carry
out the eigendecomposition of the matrix of noncentred second moments,
T, whose eigenvectors define linear combinations of the uncentred
variables. This is often referred to as an uncentred PCA and there has
been an unfortunate tendency in some fields to equate the name SVD
only with this uncentred version of PCA. Uncentred PCs are linear
combinations of the uncentred variables which successively maximize
non-central second moments, subject to having their crossed
non-central second moments equal to zero. Except when the vector of
column means x¯ (i.e. the centre of gravity of the original n-point
scatterplot in p-dimensional space) is near zero (in which case
centred and uncentred moments are similar), it is not immediately
intuitive that there should be similarities between both variants of
PCA. Cadima & Jolliffe ON RELATIONSHIPS BETWEEN UNCENTRED AND
COLUMN-CENTRED PRINCIPAL COMPONENT
ANALYSIS
have explored the relations between the standard (column-centred) PCA
and uncentred PCA and found them to be closer than might be expected,
in particular when the size of vector $\bar{x}$ is large. It is often
the case that there are great similarities between many eigenvectors
and (absolute) eigenvalues of the covariance matrix S and the
corresponding matrix of non-centred second moments, T. In some
applications, row centrings, or both row- and column-centring (known
as doublecentring) of the data matrix, have been considered
appropriate. The SVDs of such matrices give rise to row-centred and
doubly centred PCA, respectively.
One case when centering often is unnatural and doesn't help is for images. One example on this site is What is the intuition behind SVD?.
