# Covariance and correlation in multivariate random variables

I have this experiment where there are two random vectors $P_1 = (x_1,y_1)$ and $P_2 = (x_2,y_2)$. These two vectors represents two measurements for the location of two nearby points ($10$ meters apart) using two independent sensors. I want to calculate the covariance matrix and correlation coefficient between $P1$ and $P2$ using $10,000$ measurements I have. I know how do this for two random variables but not in the case of two random vectors.

• Pick two components at a time. Apply what you know.
– whuber
Commented Jun 3, 2016 at 20:10
• +1 for whuber's answer. That's the absolute most basic thing you can do. Are you familiar and comfortable with matrix notation? What programming language/environment are you using? There are extremely concise ways to do this... Commented Jun 3, 2016 at 20:47
• @whuber: I am not sure about what you mean by "two components at a time". would you elaborate?
– user117835
Commented Jun 3, 2016 at 21:19
• @MatthewGunn: Yes I am totally fine with the matrix notation, and I am using Matlab.
– user117835
Commented Jun 3, 2016 at 21:19
• You have four vector components: $x_1, y_1, x_2, y_2$. Pick any two. You know how to compute their covariance matrix and their correlation coefficient. Do this for all $6$ pairs and you will be done.
– whuber
Commented Jun 3, 2016 at 22:03

Let $$\mathbf{x}$$ be a random column vector. In matrix notation, the covariance matrix for $$\mathbf{x}$$ can be expressed as:

$$\Sigma = E\left[\left( \mathbf{x} - E[\mathbf{x}]\right) \left(\mathbf{x} - E[\mathbf{x}]\right)' \right]$$

The sample analogue is: $$\hat{\Sigma} = \frac{1}{n-1} \sum_i \left( \mathbf{x}_i - \hat{\boldsymbol{\mu}}\right) \left(\mathbf{x}_i - \hat{\boldsymbol{\mu}}\right)' \quad \quad \hat{\boldsymbol{\mu}} = \frac{1}{n} \sum_i \mathbf{x}_i$$ where each $$\mathbf{x}_i$$ is a column vector containing the $$i$$th observation.

Something standard is to put your $$n$$ observations in an $$n$$ by $$k$$ data matrix $$X$$ where each row is an observation. That's standard convention in statistical texts and something similar is standard practice in many programming environments.

$$X = \left[ \begin{array}{c} \mathbf{x}_1' \\ \mathbf{x}_2' \\ \ldots \\ \mathbf{x}_n' \\ \end{array} \right]$$

Various operations can be expressed quite elegantly with matrix notation using the data matrix $$X$$. The sample covariance matrix can be written as

$$(X - \hat{\boldsymbol{\mu}}')'(X - \hat{\boldsymbol{\mu}}') / (n - 1)$$

where $$X - \hat{\mathbf{u}}'$$ means you subtract the row vector $$\hat{\mathbf{u}}'$$ from each row of $$X$$.

(Note: bold letters are vectors, upper case are matrices, lower case are scalars, and $$'$$ means taking the transpose.)

## Matlab comment:

In Matlab, you can easily follow the formulas exactly: make a data matrix $$X$$, compute $$\hat{\boldsymbol{\mu}}$$, and compute $$\hat{\Sigma}$$. There are also built-in functions, mean and cov respectively, which will do it for you.