I was doing the Andrew Ng's ML course, and one of the solutions mentioned The first principal component is aligned with the direction of maximal variance.
I didn't get what it is trying to say.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Welcome to CV!
PCA finds the linear combination of your original input variables that contains the largest possible variance among all input variables. This is the first principal component, and it will thus by definition "align with the direction of maximal variance". The second principal component is then a linear combination independent of the first PC, with the largest remaining variance, and so on.
Consider this mock example:
There are two input variables (bacterial colony size and relative expression of a fluorescent protein). However, it turns out that larger colonies express less fluorescent protein (i.e., the input variables are correlated). The first PC will then be in the direction of this combined variance of the two input variables, because this is the largest total variance that a linear combination can find. The second PC will do the same, but perpendicular to PC1.