# Covariance and Correlation Matrices

I have a somewhat dumb question. When determining the correlation or covariance (doesn't matter I suppose) amongst random vectors, is the covariance computed among features or among observations?

For instance, say I have some data matrix $$X = \begin{bmatrix} x_{11} & x_{12} & x_{13} & \dots & x_{1p} \\ x_{21} & x_{22} & x_{23} & \dots & x_{2p} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ x_{n1} & x_{d2} & x_{d3} & \dots & x_{dp} \end{bmatrix}$$

In which there are $$n$$ observations and $$p$$ features/predictors. Is the covariance matrix going to be an $$n \times n$$ symmetric matrix or a $$p \times p$$ symmetric matrix?

The reason for asking is that I have seen both $$(X - \mu)^T(X - \mu)$$ and $$(X - \mu)(X - \mu)^T$$ being used to define covariance matrices.

• Are your observations independent? If so, that implies their covariance matrix is diagonal and little calculation is needed. The reason you might have seen both formulas is that some people represent their data with the transpose of your $X.$ – whuber Feb 21 at 18:47
• @whuber so you calculate the correlation between features then? But what if the data is a video or images where each frame or snapshot is a column in my data matrix? – David Sarpong Feb 21 at 18:55
• It sounds like your frames are your observations and the pixels are your variables ("features"). – whuber Feb 21 at 19:13