0
$\begingroup$

I have a data matrix $X \in \mathbb{R}^{m \times 4}$, where $m$ is any number of rows, whose data follow a multivariate normal (MVN) distribution.

Suppose that for a given row $i$, the data for the 4th column is missing. We can use the conditional expectation formula for the MVN distribution to estimate the missing value. First, let's partition our data into two parts: $Y_1 = \{X_1, X_2, X_3\}, Y_2 = \{X_4\}$, where $X_i$ represents the column $i$ from matrix $X$. The conditional probability of $Y_2$ given $Y_1$ is expressed as:

$$\mathbb{E}[Y_2 | Y_1 = \mathbf{y_1}] = \mu_2 + \Sigma_{12} \ \Sigma_{11}^{-1} \ (\mathbf{y_1} - \mathbf{\mu_1})$$

The mean vector, $\mathbf{\mu}$, for all four columns is $\mathbf{\mu} = [40, 34, 42, 41]$. The covariance matrix for the data is

$\Sigma = \begin{bmatrix}20 & 7 & 15 & 11\\ 7 &19 &9 & 8\\ 15& 9 &28 & 18 \\ 11 &8 & 18 & 21\end{bmatrix}$

Question: How do I properly partition my matrix $\Sigma$ into the matrices the conditional expectation requires?

I thought $\Sigma_{11}$ was the submatrix that represents the variance and covariance among the non-missing data, $Y_1$, and $\Sigma_{22}$ represents the covariance of among missing ($Y_2$) and non-missing data. So,

$$\Sigma_{11} = \begin{bmatrix} 20 &7 & 15 \\ 7 & 19& 9 \\ 15 & 9 & 28\end{bmatrix}$$

$$\Sigma_{12} = \begin{bmatrix} 11 \\ 8 \\ 18\end{bmatrix}$$

However, with this interpretation, I could write:

$$\mathbb{E}[Y_2 | Y_1 = \mathbf{y_1}] = 41 + \begin{bmatrix} 11 \\ 8 \\ 18\end{bmatrix} \ \begin{bmatrix} 0.085 & -0.011 & -0.42 \\ -0.011 & 00063 & -0.014 \\ -0.042 & -0.014 & 0.060\end{bmatrix} \ \begin{bmatrix} y_1^{(1)} - 40 \\ y_2^{(1)} - 34 \\ y_3^{(1)} - 42 \end{bmatrix} $$

However the dimensions for the product $\Sigma_{12} \ \Sigma_{11}^{-1}$ are off. I could "simply" take the transpose of the $\Sigma_{12}$, but I don't think that'd be correct. I feel like my struggle lies in the poor understanding of this partition.

$\endgroup$
1
  • $\begingroup$ In this formula, $\Sigma_{12}=\pmatrix{11&8&18}.$ You have shown us $\Sigma_{21}=\Sigma_{12}^\prime.$ $\endgroup$
    – whuber
    Commented Mar 18, 2022 at 19:17

1 Answer 1

1
$\begingroup$

I think your equation for the expected value of $X_4$ is the same as the corresponding equation in here. By checking the dimension, I think the mistake that you have made is that the $\Sigma_{12}$ should be a row vector instead of a column vector; therefore, it should be a $1\times 3$ matrix.

With that fixed, your result will have a sensible dimension.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.