A principal component is a weighted linear combination of all your factors (X's).
example: PC1 = 0.1X1 + 0.3X2
There will be one component for each factor (though in general a small number are selected).
The components are created such that they have zero correlation (are orthogonal), by design.
Therefore, component PC1 should not explain any variation in component PC2.
You may want to do regression on your Y variable and the PCA representation of your X's, as they will not have multi-collinearity. However, this could be hard to interpret.
If you have more X's than observations, which breaks OLS, you can regress on your components, and simply select a smaller number of the highest variation components.
Principal Component Analysis by Jollife a very in-depth and highly cited book on the subject
This is also good: http://www.statsoft.com/textbook/principal-components-factor-analysis/
r
tag and what do you mean by "why this is so"? PC's are not correlated, i.e. they're orthogonal, additive, you cannot predict one PC with the another. Are you looking for a formula? $\endgroup$ – aL3xa Aug 2 '10 at 0:38