I am reading a note on PCA and in the introduction the author states that large diagonal values in the covariance matrix of the predictors correspond to a strong signal and large off-diagonal values --i.e. covariances-- correspond to high noise or distortion in our data.
He furthermore presents an example from the well-known iris dataset which can be found in base R among other places.
> cov(iris[1:4]) Sepal.Length Sepal.Width Petal.Length Petal.Width Sepal.Length 0.6856935 -0.0424340 1.2743154 0.5162707 Sepal.Width -0.0424340 0.1899794 -0.3296564 -0.1216394 Petal.Length 1.2743154 -0.3296564 3.1162779 1.2956094 Petal.Width 0.5162707 -0.1216394 1.2956094 0.5810063
In this dataset the covariances between Petal.Length and Sepal.Length and Petal.Width and Petal.Length are rather high.
I can intuitively understand the statement that large variances in the predictors imply a strong signal. I can also readily comprehend that large covariances signify redundancy in the Signal, i.e. high multicollinearity.
What I find difficult to understand intuitively is that high covariances relate to high noise and distortion in the data. For a starter, in the particular example of iris data brought forward by the author, the fact that certain attributes of the flowers appear correlated is in no way a distortion or a noise, simply a physical reality. But perhaps I do not comprehend the notion of distortion and noise as understood by the author of the note.
Your advice will be appreciated.