Let's imagine an arbitrary data set in $ D=3 $, is it possible that the covariance matrix consists of diagonal elements only?
I'd say such a data set can't exist. Or would all data points be incident with three axes?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
For instance, if we try to estimate linear regression model, we then check an assumption of an absence of autocorrelation (particular, in time series). We use, at first, covariance matrix between residuals with zero non diagonal elements. Then we use Newey West matrix with consistent estimates and compare this matrix to the previous (also, you could use tests to detect).
So, according to the theory, yes, this matrix exists, but in practice, we always get nonzero values (maybe, small, e.g. 1e-5) on non diagonal elements.