6
$\begingroup$

We have a $$ \boldsymbol\mu= \begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix} $$

$$ \boldsymbol Y= \begin{bmatrix} Y_1 \\ Y_2 \end{bmatrix} $$

with

$$ \begin{bmatrix} \Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} $$

Why they represent covariance with 4 separated matrix?

and what does $$ \Sigma_{11} \Sigma_{12} \Sigma_{21} \Sigma_{22} $$ seperately mean in Gaussian?

Improve this question
$\endgroup$
1
  • $\begingroup$ So that matrix consists of the covariance for every combination of your 2 parameters. So the diagonals are the variances of the two and the off diagonal are the covariances. $\endgroup$
    – VCG
    Commented Aug 24, 2016 at 0:46

1 Answer 1

Reset to default
10
$\begingroup$

Why they represent covariance with 4 separated matrices? I emphasize this each notion as matrix. what happen if each notion become a matrix

In this case the vectors ${\boldsymbol Y}$ and ${\boldsymbol \mu}$ are really block vectors. In the case of an $n$-dimensional ${\boldsymbol Y}$ vector we could expand it as follows:

$$\boldsymbol Y= \begin{bmatrix} \color{blue}{Y_1} \\ \color{red}{Y_2} \end{bmatrix}=\begin{bmatrix}\color{blue}{Y_{11}\\Y_{12}\\\vdots\\ Y_{1h}}\\\color{red}{Y_{21}\\Y_{22}\\\vdots\\ Y_{2k}}\end{bmatrix}\tag{$n \times 1$}$$

showing the partition of the $n$ coordinates into two groups of size $h$ and $k$, respectively, such that $n = h + k$. A parallel illustration would immediately follow for the $\boldsymbol \mu$ vector of population means.

The block matrix of covariances would hence follow as:

$$\begin{bmatrix} \Sigma_{\color{blue}{11}} & \Sigma_{\color{blue}{1}\color{red}{2}}\\ \Sigma_{\color{red}{2}\color{blue}{1}} & \Sigma_{\color{red}{22}} \end{bmatrix} \tag {$n \times n$}$$

where

$$\small\Sigma_{\color{blue}{11}}=\begin{bmatrix} \sigma^2({\color{blue}{Y_{11}}}) & \text{cov}(\color{blue}{Y_{11},Y_{12}}) & \dots & \text{cov}(\color{blue}{Y_{11},Y_{1h}}) \\ \text{cov}(\color{blue}{Y_{12},Y_{11}}) & \sigma^2({\color{blue}{Y_{12}}}) & \dots & \text{cov}(\color{blue}{Y_{12},Y_{1h}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}(\color{blue}{Y_{1h},Y_{11}}) & \text{cov}(\color{blue}{Y_{1h},Y_{12}}) &\dots& \sigma^2({\color{blue}{Y_{1h}}}) \end{bmatrix} \tag{$h \times h$}$$

with

$$\small \Sigma_{\color{blue}{1}\color{red}{2}}= \begin{bmatrix} \text{cov}({\color{blue}{Y_{11}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{11}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{11}},\color{red}{Y_{2k}}) \\ \text{cov}({\color{blue}{Y_{12}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{12}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{12}},\color{red}{Y_{2k}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}({\color{blue}{Y_{1h}}},\color{red}{Y_{21}}) & \text{cov}(\color{blue}{Y_{1h}},\color{red}{Y_{22}}) & \dots & \text{cov}(\color{blue}{Y_{1h}},\color{red}{Y_{2k}}) \end{bmatrix}\tag{$h \times k$} $$

its transpose...

$$\small \Sigma_{\color{red}{2}\color{blue}{1}}= \begin{bmatrix} \text{cov}({\color{red}{Y_{21}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{21}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{21}},\color{blue}{Y_{1h}}) \\\text{cov}({\color{red}{Y_{22}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{22}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{22}},\color{blue}{Y_{1h}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}({\color{red}{Y_{2k}}},\color{blue}{Y_{11}}) & \text{cov}(\color{red}{Y_{2k}},\color{blue}{Y_{12}}) & \dots & \text{cov}(\color{red}{Y_{2k}},\color{blue}{Y_{1h}}) \end{bmatrix}\tag{$k \times h$} $$

and

$$\small \Sigma_{\color{red}{22}}=\begin{bmatrix} \sigma^2({\color{red}{Y_{21}}}) & \text{cov}(\color{red}{Y_{21},Y_{22}}) & \dots & \text{cov}(\color{red}{Y_{21},Y_{2k}}) \\ \text{cov}(\color{red}{Y_{22},Y_{21}}) & \sigma^2({\color{red}{Y_{22}}}) & \dots & \text{cov}(\color{red}{Y_{22},Y_{2k}}) \\ \vdots & \vdots & & \vdots \\ \text{cov}(\color{red}{Y_{2k},Y_{21}}) & \text{cov}(\color{red}{Y_{2k},Y_{22}}) &\dots& \sigma^2({\color{red}{Y_{2k}}}) \end{bmatrix} \tag{$k \times k$}$$

These partitions come into play in proving that the marginal distributions of a multivariate Gaussian are also Gaussian, as well as in the actual derivation of marginal and conditional pdf's.

Improve this answer
$\endgroup$
5
  • $\begingroup$ The notation in the question (which uses capital letters for the components of $Y$ and capital Greek letters for the components of the covariance matrix) suggests that this is a multivariate situation, not merely bivariate, and that the vector $Y$ has been partitioned into two separate vectors. This induces a block partitioning of the covariance matrix: each of the $\Sigma_{ij}$ are matrices. If the product of blocks as given in the question is to make any sense, they must all be square of the same dimensions. $\endgroup$
    – whuber ♦
    Commented Aug 24, 2016 at 13:52
  • $\begingroup$ @whuber I thought of this, but the title seemed to indicate that we wouldn't have to address block matrices. Please advise as to whether to leave it as is, or erase the answer. I'm ambivalent, and can't tell whether the person asking was looking for something else, since they are new to the forum. $\endgroup$
    – Antoni Parellada
    Commented Aug 24, 2016 at 14:14
  • $\begingroup$ To me, the title emphasizes that this is a question about matrices. But let's see how the OP reacts to your post. $\endgroup$
    – whuber ♦
    Commented Aug 24, 2016 at 14:16
  • $\begingroup$ @whuber . yes I emphasize this each notion as matrix. what happen if each notion become a matrix ? $\endgroup$
    – user6507246
    Commented Aug 25, 2016 at 0:46
  • $\begingroup$ @user6507246 It makes sense. There was a tentative approach on the OP that made me overlook the notation. I think now it should make sense. $\endgroup$
    – Antoni Parellada
    Commented Aug 25, 2016 at 3:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.