I am learning about Principal Component Regression, but I am confused about centering the matrix. My question is described using the following example:
In MATLAB, I made some sample data, where $Y$ is a linear combination of the columns of $X$:
clc;clear all; X = rand(1E3,20); beta = rand(20,1); Y = X*beta;
Of course, linear regression works perfectly:
%Compute the least squares using SVD [U1, S1, V1 ] = svd(X,0) beta0 = V1*inv(S1)*U1'*Y; mean((X*beta0-Y).^2) %returns ~1E-30
Now, I will do PCR. First, let's get the principal components of X, using SVD:
X_c = bsxfun(@minus, X, mean(X,1)); [U2,S2,V2] = svd(X_c); %Principal components of X W= X_c *V2;
And now, I get the regression coefficients of the principal components:
%Compute the regression coefficients of the PCs. [U3, S3,V3] = svd(W,0); beta1 = V3*inv(S3)*U3'*Y;
But this linear combination of principal components does not recover Y.
Y2 = W*beta1; mean((Y2-Y).^2) %returns ~20.1
However, if I use the uncentered matrix on the loadings of the principal components, it works nicely:
Y3 = X*V2*beta1; mean((Y3-Y).^2) %returns 1E-26
I am confused about how centering plays into this. We compute the PCs (from the centered data), regress Y to the PCs, but then we need to project the original (uncentered data) using the loadings of the PCs, in order to use these regression coefficients. Why does this work? I would have thought that since the PCs were computed from the centered data, the regression coefficients would not be applicable to the uncentered data.