# Why does the intercept of Principal Component Regression differ from the orginal regression?

I am comparing two regressions:

$$y= \beta_0+ \beta_1 x_1 + \beta_2 x_2 + \epsilon$$

and

$$y = \gamma_0 + \gamma_1 PC_1 + \gamma_2 PC_2 +\eta$$.

The regressors in the second regression, $$PC_1, PC_2$$, are principal components generated by a PCA of $$x_1,x_2$$.

Presumably if the PCA just linearly transforms the original regressors, the intercepts should be identical. However, I find them very different. What is the reason behind this difference?

## 1 Answer

Presumably if the PCA just linearly transforms the original regressors, the intercepts should be identical.

This is not correct.

The estimate for the intercept, $$\hat{\beta_0}$$, can be computed as $$\hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x_1} - \hat{\beta_2}\bar{x_2} - \dots,$$ where $$\bar{y}$$ denotes the sample mean of $$y$$, $$\hat{\beta_j}$$ is the sample estimate of $$\beta_j$$, and $$\bar{x_j}$$ is the sample mean of $$x_j$$.

So, for a simple counterexample, consider a linear transformation of $$x_1$$ where we just add 1 to every value. This would increase $$\bar{x_1}$$ by 1 and thus change the intercept.

• Adding a constant is an affine transformation, but not a linear transformation. – David Xiaoyu Xu Aug 12 at 4:18
• @DavidXiaoyuXu I assumed you meant linear as in f(x) = a*x + b, which adding a constant certainly is. In either case, you can see from the expression I wrote why your assumption about the intercept is incorrect – Jake Westfall Aug 12 at 4:25