# How to apply the conditional expectation of the multivariate normal distribution to fill gaps in data?

I have a data matrix $$X \in \mathbb{R}^{m \times 4}$$, where $$m$$ is any number of rows, whose data follow a multivariate normal (MVN) distribution.

Suppose that for a given row $$i$$, the data for the 4th column is missing. We can use the conditional expectation formula for the MVN distribution to estimate the missing value. First, let's partition our data into two parts: $$Y_1 = \{X_1, X_2, X_3\}, Y_2 = \{X_4\}$$, where $$X_i$$ represents the column $$i$$ from matrix $$X$$. The conditional probability of $$Y_2$$ given $$Y_1$$ is expressed as:

$$\mathbb{E}[Y_2 | Y_1 = \mathbf{y_1}] = \mu_2 + \Sigma_{12} \ \Sigma_{11}^{-1} \ (\mathbf{y_1} - \mathbf{\mu_1})$$

The mean vector, $$\mathbf{\mu}$$, for all four columns is $$\mathbf{\mu} = [40, 34, 42, 41]$$. The covariance matrix for the data is

$$\Sigma = \begin{bmatrix}20 & 7 & 15 & 11\\ 7 &19 &9 & 8\\ 15& 9 &28 & 18 \\ 11 &8 & 18 & 21\end{bmatrix}$$

Question: How do I properly partition my matrix $$\Sigma$$ into the matrices the conditional expectation requires?

I thought $$\Sigma_{11}$$ was the submatrix that represents the variance and covariance among the non-missing data, $$Y_1$$, and $$\Sigma_{22}$$ represents the covariance of among missing ($$Y_2$$) and non-missing data. So,

$$\Sigma_{11} = \begin{bmatrix} 20 &7 & 15 \\ 7 & 19& 9 \\ 15 & 9 & 28\end{bmatrix}$$

$$\Sigma_{12} = \begin{bmatrix} 11 \\ 8 \\ 18\end{bmatrix}$$

However, with this interpretation, I could write:

$$\mathbb{E}[Y_2 | Y_1 = \mathbf{y_1}] = 41 + \begin{bmatrix} 11 \\ 8 \\ 18\end{bmatrix} \ \begin{bmatrix} 0.085 & -0.011 & -0.42 \\ -0.011 & 00063 & -0.014 \\ -0.042 & -0.014 & 0.060\end{bmatrix} \ \begin{bmatrix} y_1^{(1)} - 40 \\ y_2^{(1)} - 34 \\ y_3^{(1)} - 42 \end{bmatrix}$$

However the dimensions for the product $$\Sigma_{12} \ \Sigma_{11}^{-1}$$ are off. I could "simply" take the transpose of the $$\Sigma_{12}$$, but I don't think that'd be correct. I feel like my struggle lies in the poor understanding of this partition.

• In this formula, $\Sigma_{12}=\pmatrix{11&8&18}.$ You have shown us $\Sigma_{21}=\Sigma_{12}^\prime.$
– whuber
Commented Mar 18, 2022 at 19:17

I think your equation for the expected value of $$X_4$$ is the same as the corresponding equation in here. By checking the dimension, I think the mistake that you have made is that the $$\Sigma_{12}$$ should be a row vector instead of a column vector; therefore, it should be a $$1\times 3$$ matrix.