# Gaussian covariance matrix basic concept

We have a $$\boldsymbol\mu= \begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}$$

$$\boldsymbol Y= \begin{bmatrix} Y_1 \\ Y_2 \end{bmatrix}$$

with

$$\begin{bmatrix} \Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22} \end{bmatrix}$$

Why they represent covariance with 4 separated matrix?

and what does $$\Sigma_{11} \Sigma_{12} \Sigma_{21} \Sigma_{22}$$ seperately mean in Gaussian?

• So that matrix consists of the covariance for every combination of your 2 parameters. So the diagonals are the variances of the two and the off diagonal are the covariances.
– VCG
Aug 24, 2016 at 0:46

Why they represent covariance with 4 separated matrices? I emphasize this each notion as matrix. what happen if each notion become a matrix

In this case the vectors ${\boldsymbol Y}$ and ${\boldsymbol \mu}$ are really block vectors. In the case of an $n$-dimensional ${\boldsymbol Y}$ vector we could expand it as follows:

$$\boldsymbol Y= \begin{bmatrix} \color{blue}{Y_1} \\ \color{red}{Y_2} \end{bmatrix}=\begin{bmatrix}\color{blue}{Y_{11}\\Y_{12}\\\vdots\\ Y_{1h}}\\\color{red}{Y_{21}\\Y_{22}\\\vdots\\ Y_{2k}}\end{bmatrix}\tag{n \times 1}$$

showing the partition of the $n$ coordinates into two groups of size $h$ and $k$, respectively, such that $n = h + k$. A parallel illustration would immediately follow for the $\boldsymbol \mu$ vector of population means.

The block matrix of covariances would hence follow as:

$$\begin{bmatrix} \Sigma_{\color{blue}{11}} & \Sigma_{\color{blue}{1}\color{red}{2}}\\ \Sigma_{\color{red}{2}\color{blue}{1}} & \Sigma_{\color{red}{22}} \end{bmatrix} \tag {n \times n}$$

where

$$\small\Sigma_{\color{blue}{11}}=\begin{bmatrix} \sigma^2({\color{blue}{Y_{11}}}) & \text{cov}(\color{blue}{Y_{11},Y_{12}}) & \dots & \text{cov}(\color{blue}{Y_{11},Y_{1h}}) \\ \text{cov}(\color{blue}{Y_{12},Y_{11}}) & \sigma^2({\color{blue}{Y_{12}}}) & \dots & \text{cov}(\color{blue}{Y_{12},Y_{1h}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}(\color{blue}{Y_{1h},Y_{11}}) & \text{cov}(\color{blue}{Y_{1h},Y_{12}}) &\dots& \sigma^2({\color{blue}{Y_{1h}}}) \end{bmatrix} \tag{h \times h}$$

with

$$\small \Sigma_{\color{blue}{1}\color{red}{2}}= \begin{bmatrix} \text{cov}({\color{blue}{Y_{11}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{11}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{11}},\color{red}{Y_{2k}}) \\ \text{cov}({\color{blue}{Y_{12}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{12}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{12}},\color{red}{Y_{2k}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}({\color{blue}{Y_{1h}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{1h}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{1h}},\color{red}{Y_{2k}}) \end{bmatrix}\tag{h \times k}$$

its transpose...

$$\small \Sigma_{\color{red}{2}\color{blue}{1}}= \begin{bmatrix} \text{cov}({\color{red}{Y_{21}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{21}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{21}},\color{blue}{Y_{1h}}) \\\text{cov}({\color{red}{Y_{22}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{22}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{22}},\color{blue}{Y_{1h}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}({\color{red}{Y_{2k}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{2k}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{2k}},\color{blue}{Y_{1h}}) \end{bmatrix}\tag{k \times h}$$

and

$$\small \Sigma_{\color{red}{22}}=\begin{bmatrix} \sigma^2({\color{red}{Y_{21}}}) & \text{cov}(\color{red}{Y_{21},Y_{22}}) & \dots & \text{cov}(\color{red}{Y_{21},Y_{2k}}) \\ \text{cov}(\color{red}{Y_{22},Y_{21}}) & \sigma^2({\color{red}{Y_{22}}}) & \dots & \text{cov}(\color{red}{Y_{22},Y_{2k}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}(\color{red}{Y_{2k},Y_{21}}) & \text{cov}(\color{red}{Y_{2k},Y_{22}}) &\dots& \sigma^2({\color{red}{Y_{2k}}}) \end{bmatrix} \tag{k \times k}$$

These partitions come into play in proving that the marginal distributions of a multivariate Gaussian are also Gaussian, as well as in the actual derivation of marginal and conditional pdf's.

• The notation in the question (which uses capital letters for the components of $Y$ and capital Greek letters for the components of the covariance matrix) suggests that this is a multivariate situation, not merely bivariate, and that the vector $Y$ has been partitioned into two separate vectors. This induces a block partitioning of the covariance matrix: each of the $\Sigma_{ij}$ are matrices. If the product of blocks as given in the question is to make any sense, they must all be square of the same dimensions.
– whuber
Aug 24, 2016 at 13:52
• @whuber I thought of this, but the title seemed to indicate that we wouldn't have to address block matrices. Please advise as to whether to leave it as is, or erase the answer. I'm ambivalent, and can't tell whether the person asking was looking for something else, since they are new to the forum. Aug 24, 2016 at 14:14
• To me, the title emphasizes that this is a question about matrices. But let's see how the OP reacts to your post.
– whuber
Aug 24, 2016 at 14:16
• @whuber . yes I emphasize this each notion as matrix. what happen if each notion become a matrix ? Aug 25, 2016 at 0:46
• @user6507246 It makes sense. There was a tentative approach on the OP that made me overlook the notation. I think now it should make sense. Aug 25, 2016 at 3:43