# In PCA, is there an intuitive explanation for why the second principal component chosen must be orthogonal to the first component?

In principal components analysis, the principal components are chosen according to three criteria. The first component is chosen to be the direction in the data with greatest variance. The second is chosen to be the direction with greatest variance GIVEN that it is orthogonal to the first.

I am wondering if there is an intuitive way to understand this without having to resort to the proof, of which is hard for me to extract intuition from? Thanks.

• If it weren't orthogonal (i.e. independent), it would explain variance that is already captured by the first component! Also see the great answers here: stats.stackexchange.com/questions/2691/… Commented Oct 6, 2017 at 0:51
• Is that the same as saying that we are creating principal components that are correlated, and hence may introduce a collinearity issue when trying to regress on these summary/principal component variables? Commented Oct 6, 2017 at 0:57
• Do you have experience applying PCA to data sets? If so, in those cases, what lead you to use PCA? Commented Oct 6, 2017 at 1:01
• They would indeed be correlated if they were not orthogonal. The original variables may or may not be too strongly correlated leading to multicollinearity, which is one of the reasons one might use PCA prior to regression (PCR). If regression is your goal, there exist other alternatives you may want to consider, such as PLS or ridge regression. What is the purpose of your research? Commented Oct 6, 2017 at 1:14
• If you didn't include the orthogonality constraint, the solution to the second optimization problem obviously would be the first principal component, because nothing would have changed. What constraint, then, do you propose erecting in place of orthogonality? (I don't see how your question could be answered without knowing what you have in mind.)
– whuber
Commented Oct 6, 2017 at 14:36