# When to use PCA vs. OLS

Let's say I want to build a factor model to explain (excess) stock returns--think Fama-French, for instance. Obviously one could use OLS to fit a model, but I've seen PCA used as well. What are the practical differences between the two methodologies? I know OLS minimizes squared residuals, while PCA sort of minimizes the (not sure if squared or absolute) errors diagonally (or orthogonally to the variance-covariance matrix eigenvector with the highest eigenvalue, to be more specific). This is shown graphically here: http://www.r-bloggers.com/principal-component-analysis-pca-vs-ordinary-least-squares-ols-a-visual-explanation/? So what are the implications of doing this vs. running an OLS regression? I'm not quite sure when you'd use one methodology over the other.

Another concern I have is the model output. You have a very clear model output with OLS: $$\hat y= \hat \beta_0 + \hat\beta_1X_1 + ... +\hat\beta_nX_n+\epsilon$$

And there are straightforward methods to assess the fit of your model (e.g., $$R^2$$ or t-tests).

I guess you could represent a PCA model similarly to how you'd represent an OLS model (as above) using the main eigenvector I referred to earlier as sort of your "line of best fit". But from there, are there simple tests the likes of $$R^2$$ or t-tests that are available?

More importantly, when would you use PCA as opposed to OLS generally speaking? What makes you decide whether your PCA model provides a good fit?

I looked at this blog post and it intrigued me quite a bit, but I have no clue what's going on: http://www.calculatinginvestor.com/2013/03/18/pca-factors-vs-fama-french-factors/. I'm not sure what the writer means by "PCA factors" and whether or not his use of $$R^2$$ is appropriate for PCA. Could you perhaps briefly comment on how PCA could be used in the context of factor analysis specifically, whether you're testing three known factors or trying to find them from a pool of, say, 10?

I hope my questions aren't all over the place, but I'd really appreciate any insight you might have on the whole OLS vs. PCA issue.

• You can use OLS with PCA. For instance, instead of running a regression with 100 Macroeconomic drivers, you could extract a few PCA components from the independent variables, and use them as variables in OLS. Also, if you have to forecast 20 highly correlated variables separately or in vector regression, you could extract 3 PCA factors, regress them separately on variables, then invert PCA to get your 20 variables back. Jan 27, 2016 at 22:17

PCA is a data transform tool that transforms the data to a new coordinate system. It is used to reduce dimension. For example, you can apply PCA to Fama-French's 3 factor to transform three time series into one time series. Then use OLS tech to find the $\beta$. I do not know exactly why people may use PCA before OLS to reduce dimension of the independent variables. Maybe reduced time series gives a better prediction.