Principal Components and Noise When doing principal components it is intuitively clear that noise accumulates more towards the last components. What would be the formal explanation of how and why this happens? 
 A: It is not "intuitively clear" to me. In fact, I'd say it totally depends on the signal to noise ratio. PCA decomposes the data into uncorrelated vectors, with the first vector (first component) catching the largest portion of the variability in the data, the second -- the second largest etc.
If the noise is big enough, it will push the "signal" towards the smaller components.
Quite often I see that one of the "major" components is dominated by one or two outliers (usually technical errors in measurement), while the actual study variables come in third or fourth.
A: Maybe another way to state the same thing is that the first Principal Component grabs the majority of the Signal.  The 2nd Principal Component is stuck attempting to explain only what is unexplained by the 1st one.  It explains the residual of the 1st one.  By the time you reach the 3d Principal Component it really does not have much Signal left to work with or explain because the vast majority of it has been explained by the first two.  This logic entails that as you move away from the 1st Principal Component and towards the last one, you get less and less Signal... which means you get more and more Noise.      
