When a covariance matrix is a diagonal matrix then there is no correlation between random variables as seen here Difference between identity and diagonal covariance matrices However, what if I wanted the opposite. What if I wanted to have a covariance matrix where the variables, where each variable reprsented say a different companies value at a certain time, were perfectly correlated? What would that covariance matrix look like?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If the multidimensional data are perfectly correlated, the points form a line through the feature space, but in some arbitrary direction. The covariance matrix would not have any particularly distinguishing form. This is why covariance matrices are not very rich descriptors of multivariate data. You need to generate a more meaningful description of the data by doing something that will not change the structure of the data but will enhance its description. Like forming eigenvectors and eigenvalues of the covariance matrix. Now you have a richer descriptor of the same data. The eigenvectors are just a rotation of the coordinate system that optimizes the variance in the smallest number of features. Each eigenvalue is the variance measured in the direction of the corresponding eigenvector. If the data are perfectly correlated, there will be only one eigenvalue that is nonzero.
