How to express a correlation matrix in terms of a covariance matrix?

Question

I'd like to know, if I have the following Normal multivariate structure $$\left( {\begin{array}{*{20}{c}} {{Y_{r_1}}}\\ {{Y_{r_2}}} \end{array}} \right)\sim{\mathcal{N}_{r}}\left( {\left[ {\begin{array}{*{20}{c}} {{\mu_1}}\\ {{\mu_2}} \end{array}} \right],\left[ {\begin{array}{*{20}{c}} {{\Sigma _{11}}}&{{\Sigma _{12}}}\\ {{\Sigma _{21}}}&{{\Sigma _{22}}} \end{array}} \right]} \right),$$ then $Y_{r_1}\mid Y_{r_2} \sim \mathcal{N}_r(\mu, \Sigma)$, where $\mu=\mu_1 + \Sigma_{12}\Sigma^{-1}_{22}(Y_{S_n}-\mu_2)$ and $\Sigma=\Sigma_{11}-\Sigma_{12}\Sigma^{-1}_{22}\Sigma_{21}$.

Another result we have is the correlation one, where $Cor(Y_{r_1},Y_{r_2})=\cfrac{Cov(Y_{r_1},Y_{r_2})}{\sqrt{Var(Y_{r_1})}\sqrt{Var(Y_{r_2})}}$.

Therefore, using this 2 facts, can I use the correlation expression in this following way? (It will help me a lot to prove a result)

$Cor(Y_{r_1},Y_{r_2})= \Sigma_{12}\Sigma_{22}^{-1/2}\Sigma_{11}^{-1/2}$

Answer 1

When there are $n$ variables $X_1, \ldots, X_n,$ their mutual covariances can be assembled into a matrix $\Sigma = (\sigma_{ij})$ where

$$\sigma_{ij} = \operatorname{Cov}(X_i,X_j).$$

Note that the diagonal elements $\sigma_{ii} = \operatorname{Cov}(X_i,X_i) = \operatorname{Var}(X_i)$ are the variances. It will be convenient to extract these elements into a diagonal matrix by setting all off-diagonal entries to zero; sometimes this is written

$$\operatorname{Diag}(\Sigma)_{ij} = \left\{\matrix{\sigma_{ii} & i=j \\ 0 & \text{otherwise.}}\right.$$

Note, too, that a positive square root of this matrix is well-defined and unique because all its entries are positive:

$$\sqrt{\operatorname{Diag}(\Sigma)}_{ij} = \left\{\matrix{\sqrt{\sigma_{ii}} & i=j \\ 0 & \text{otherwise.}}\right.$$

One formula for the corresponding correlation coefficient, as given in the question, is

$$\rho_{ij} = \operatorname{Cor}(X_i,X_j) = \frac{\operatorname{Cov}(X_i,X_j)}{\sqrt{\operatorname{Var}(X_i)\operatorname{Var}(X_j)}} = \frac{\sigma_{ij}}{\sqrt{\sigma_{ii}\,\sigma_{jj}}}\ .$$

These can be assembled into a correlation matrix $P= (\rho_{ij}).$

It follows directly from the definitions of matrix multiplication and inversion that the preceding formula is the result of the following combination of operations:

$$P = \sqrt{\operatorname{Diag}(\Sigma)}^{-1}\ \Sigma\ \sqrt{\operatorname{Diag}(\Sigma)}^{-1}.$$

By focusing on part of the matrices, this formula extends directly to the block matrix setup of the question, where the covariances of the variables in the $k$-vector $Y_{r_1}$ and those in the $m$-vector $Y_{r_2}$ appear in the $k\times m$ matrix $\Sigma_{12},$ $\Sigma_{11}$ is the $k\times k$ covariance matrix of $Y_{r_1},$ and $\Sigma_{22}$ is the $m\times m$ covariance matrix of $Y_{r_2}.$ As before, the correlations between the variables in $Y_{r_1}$ and those in $Y_{r_2}$ can be arranged into a $k\times m$ mutual correlation matrix $P_{12}$ and

$$P_{12} = \sqrt{\operatorname{Diag}(\Sigma_{11})}^{-1}\ \Sigma_{12}\ \sqrt{\operatorname{Diag}(\Sigma_{22})}^{-1}.$$

---

For example, let $$\Sigma = \pmatrix{1 & -1 & 2 \\ -1 & 4 & -1 \\ 2 & -1 & 9}.$$ Then

$$\operatorname{Diag}(\Sigma) = \pmatrix{1 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & 9},$$

$$\sqrt{\operatorname{Diag}(\Sigma)} = \pmatrix{1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3},$$

$$\sqrt{\operatorname{Diag}(\Sigma)}^{-1} = \pmatrix{1 & 0 & 0 \\ 0 & \frac{1}{2} & 0 \\ 0 & 0 & \frac{1}{3}},$$

giving the correlation matrix

$$P = \pmatrix{1 & 0 & 0 \\ 0 & \frac{1}{2} & 0 \\ 0 & 0 & \frac{1}{3}}\ \pmatrix{1 & -1 & 2 \\ -1 & 4 & -1 \\ 2 & -1 & 9}\ \pmatrix{1 & 0 & 0 \\ 0 & \frac{1}{2} & 0 \\ 0 & 0 & \frac{1}{3}} = \pmatrix{1 & -\frac{1}{2} & \frac{2}{3} \\ -\frac{1}{2} & 1 & -\frac{1}{6} \\ \frac{2}{3} & -\frac{1}{6} & 1}.$$

Taking $k=1$ and $m=2$ makes $\Sigma_{11} = \pmatrix{1},$ $\Sigma_{22} = \pmatrix{4 & -1 \\ -1 & 9},$ and $\Sigma_{12} = \pmatrix{-1 & 2}.$ Thus the correlations between the first variable ($Y_{r_1}$) and the remaining two variables ($Y_{r_2}$) are given by the $1\times 2$ matrix

$$P_{12} = \pmatrix{\sqrt{1}}^{-1} \pmatrix{-1 & 2}\ \pmatrix{\sqrt{4} & 0 \\ 0 & \sqrt{9}}^{-1} = \pmatrix{-\frac{1}{2} & \frac{2}{3}},$$

which is precisely the upper right hand block of $P,$ as claimed.