Is there any advantage of SVD over PCA? I know how to calculate PCA and SVD mathematically, and I know that both can be applied to Linear Least Squares regression.
The main advantage of SVD mathematically seems to be that it can be applied to non-square matrices.
Both focus on the decomposition of the $X^\top X$ matrix. Other than the advantage of SVD mentioned, are there any additional advantages or  insights provided by using SVD over PCA?
I'm really looking for the intuition rather than any mathematical differences.
 A: As @ttnphns and @nick-cox said, SVD is a numerical method and PCA is an analysis approach (like least squares). You can do PCA using SVD, or you can do PCA doing the eigen-decomposition of $X^T X$ (or $X X^T$), or you can do PCA using many other methods, just like you can solve least squares with a dozen different algorithms like Newton's method or gradient descent or SVD etc.
So there is no "advantage" to SVD over PCA because it's like asking whether Newton's method is better than least squares: the two aren't comparable.
A: The question is really asking if you should do Z-score normalization of the columns before applying the SVD. This is because PCA is the above transformation followed by the SVD. Sometimes doing the normalization is quite harmful. If your data is for example (transformed) word counts which are positive, subtracting the mean is definitely harmful. This is because zeros which represent the absence of a word in a document will be mapped to negative numbers with high magnitude. In linear problems the higher magnitude should be used to represent the range where your features are most sensitive.  Also dividing by the standard deviation is harmful for this type of data.
