6
$\begingroup$

We have a $$ \boldsymbol\mu= \begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix} $$

$$ \boldsymbol Y= \begin{bmatrix} Y_1 \\ Y_2 \end{bmatrix} $$

with

$$ \begin{bmatrix} \Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} $$

Why they represent covariance with 4 separated matrix?

and what does $$ \Sigma_{11} \Sigma_{12} \Sigma_{21} \Sigma_{22} $$ seperately mean in Gaussian?

$\endgroup$
  • $\begingroup$ So that matrix consists of the covariance for every combination of your 2 parameters. So the diagonals are the variances of the two and the off diagonal are the covariances. $\endgroup$ – VCG Aug 24 '16 at 0:46
9
$\begingroup$

Why they represent covariance with 4 separated matrices? I emphasize this each notion as matrix. what happen if each notion become a matrix

In this case the vectors ${\boldsymbol Y}$ and ${\boldsymbol \mu}$ are really block vectors. In the case of an $n$-dimensional ${\boldsymbol Y}$ vector we could expand it as follows:

$$\boldsymbol Y= \begin{bmatrix} \color{blue}{Y_1} \\ \color{red}{Y_2} \end{bmatrix}=\begin{bmatrix}\color{blue}{Y_{11}\\Y_{12}\\\vdots\\ Y_{1h}}\\\color{red}{Y_{21}\\Y_{22}\\\vdots\\ Y_{2k}}\end{bmatrix}\tag{$n \times 1$}$$

showing the partition of the $n$ coordinates into two groups of size $h$ and $k$, respectively, such that $n = h + k$. A parallel illustration would immediately follow for the $\boldsymbol \mu$ vector of population means.

The block matrix of covariances would hence follow as:

$$\begin{bmatrix} \Sigma_{\color{blue}{11}} & \Sigma_{\color{blue}{1}\color{red}{2}}\\ \Sigma_{\color{red}{2}\color{blue}{1}} & \Sigma_{\color{red}{22}} \end{bmatrix} \tag {$n \times n$}$$

where

$$\small\Sigma_{\color{blue}{11}}=\begin{bmatrix} \sigma^2({\color{blue}{Y_{11}}}) & \text{cov}(\color{blue}{Y_{11},Y_{12}}) & \dots & \text{cov}(\color{blue}{Y_{11},Y_{1h}}) \\ \text{cov}(\color{blue}{Y_{12},Y_{11}}) & \sigma^2({\color{blue}{Y_{12}}}) & \dots & \text{cov}(\color{blue}{Y_{12},Y_{1h}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}(\color{blue}{Y_{1h},Y_{11}}) & \text{cov}(\color{blue}{Y_{1h},Y_{12}}) &\dots& \sigma^2({\color{blue}{Y_{1h}}}) \end{bmatrix} \tag{$h \times h$}$$

with

$$\small \Sigma_{\color{blue}{1}\color{red}{2}}= \begin{bmatrix} \text{cov}({\color{blue}{Y_{11}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{11}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{11}},\color{red}{Y_{2k}}) \\ \text{cov}({\color{blue}{Y_{12}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{12}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{12}},\color{red}{Y_{2k}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}({\color{blue}{Y_{1h}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{1h}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{1h}},\color{red}{Y_{2k}}) \end{bmatrix}\tag{$h \times k$} $$

its transpose...

$$\small \Sigma_{\color{red}{2}\color{blue}{1}}= \begin{bmatrix} \text{cov}({\color{red}{Y_{21}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{21}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{21}},\color{blue}{Y_{1h}}) \\\text{cov}({\color{red}{Y_{22}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{22}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{22}},\color{blue}{Y_{1h}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}({\color{red}{Y_{2k}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{2k}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{2k}},\color{blue}{Y_{1h}}) \end{bmatrix}\tag{$k \times h$} $$

and

$$\small \Sigma_{\color{red}{22}}=\begin{bmatrix} \sigma^2({\color{red}{Y_{21}}}) & \text{cov}(\color{red}{Y_{21},Y_{22}}) & \dots & \text{cov}(\color{red}{Y_{21},Y_{2k}}) \\ \text{cov}(\color{red}{Y_{22},Y_{21}}) & \sigma^2({\color{red}{Y_{22}}}) & \dots & \text{cov}(\color{red}{Y_{22},Y_{2k}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}(\color{red}{Y_{2k},Y_{21}}) & \text{cov}(\color{red}{Y_{2k},Y_{22}}) &\dots& \sigma^2({\color{red}{Y_{2k}}}) \end{bmatrix} \tag{$k \times k$}$$

These partitions come into play in proving that the marginal distributions of a multivariate Gaussian are also Gaussian, as well as in the actual derivation of marginal and conditional pdf's.

$\endgroup$
  • $\begingroup$ The notation in the question (which uses capital letters for the components of $Y$ and capital Greek letters for the components of the covariance matrix) suggests that this is a multivariate situation, not merely bivariate, and that the vector $Y$ has been partitioned into two separate vectors. This induces a block partitioning of the covariance matrix: each of the $\Sigma_{ij}$ are matrices. If the product of blocks as given in the question is to make any sense, they must all be square of the same dimensions. $\endgroup$ – whuber Aug 24 '16 at 13:52
  • $\begingroup$ @whuber I thought of this, but the title seemed to indicate that we wouldn't have to address block matrices. Please advise as to whether to leave it as is, or erase the answer. I'm ambivalent, and can't tell whether the person asking was looking for something else, since they are new to the forum. $\endgroup$ – Antoni Parellada Aug 24 '16 at 14:14
  • $\begingroup$ To me, the title emphasizes that this is a question about matrices. But let's see how the OP reacts to your post. $\endgroup$ – whuber Aug 24 '16 at 14:16
  • $\begingroup$ @whuber . yes I emphasize this each notion as matrix. what happen if each notion become a matrix ? $\endgroup$ – user6507246 Aug 25 '16 at 0:46
  • $\begingroup$ @user6507246 It makes sense. There was a tentative approach on the OP that made me overlook the notation. I think now it should make sense. $\endgroup$ – Antoni Parellada Aug 25 '16 at 3:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.