Peculiar Behaviour of Conditional Variance for Multivariate Normal Distributions Let $Z_i \sim \mathcal{N}(0,1)$ be independent normal distributions. Consider the following correlated variables, defined by
$$ X_1 = \frac{Z_1 + Z_2}{\sqrt{2}},\;\;\;X_2= \frac{Z_2 + Z_3}{\sqrt{2}},\;\;\;X_3= \frac{Z_3 + Z_4}{\sqrt{2}},\ldots$$
Thus each $X_i$ by itself is also a standard normal distribution but is correlated to the immediate neighbours $X_{i-1}$ and $X_{i+1}$. Consider the joint distribution of $(X_1,X_2)$ which is a joint normal with mean = $(0,0)$ and covariance matrix 
$$\begin{pmatrix} 1 & 1/2 \\ 1/2 & 1 \end{pmatrix} $$
Now the thing is according to the rules of conditional probability the conditional variance for $X_1$ is $\left(1-\rho^2\right)\sigma_1^2 = \frac{3}{4}$ in this case. So far so good.
Suppose we then consider the joint normal $\left(X_1,X_2,X_3\right)$, which has the covariance matrix 
$$\begin{pmatrix} 1 & 1/2 & 0 \\ 1/2 & 1 & 1/2 \\ 0 & 1/2 & 1 \end{pmatrix}$$
In this case, the conditional variance of $X_1$ is given by
$$ 1 - \begin{pmatrix} 1/2 & 0 \end{pmatrix}\begin{pmatrix} 1 & 1/2 \\ 1/2 & 1 \end{pmatrix}^{-1}\begin{pmatrix} 1/2 \\ 0 \end{pmatrix} = 2/3$$
The questions I have now are: 


*

*Since $X_1$ is not dependent on $X_3$ at all, why the conditional variance of $X_1$ drops from $3/4$ to $2/3$ when $X_3$ is taken into account? 

*If I further include $X_4,X_5,\ldots$ the conditional variance seems to drop further and reaches a limit of $1/2$ when I include a very large number of $X_i$. Is there any intuitive explanation for this limit?

 A: This is a super interesting question/observation.... First thing to note is that $$Var(X_1\mid Z_2) = 1/2$$
And the reason that 
$$Var(X_1\mid X_2) = 3/4 \neq 1/2$$
is because it's not possible to disentangle the actual value that $Z_2$ takes when you only observe $$X_2 = \frac{Z_2 + Z_3}{\sqrt{2}}$$
and as  the first commentor pointed out, observing $X_3, X_4, X_5, \dots$ tells you more and more information about what the actual value of $Z_2$ is, and thus brings you closer to that $1/2$ bound you mentioned.
To prove the we eventually hit that $1/2$ limit, denote the conditional variance of $X_1$ on $X_2, \dots, X_T$ by $\overline{\Sigma}_T$, and the unconditional covariance matrix of $X_1, X_2, \dots , X_T$ by $\Sigma_T$. We know that $$\overline{\Sigma}_T = \Sigma^{11}_T  - \Sigma^{12}_T[\Sigma^{22}_T]^{-1}\Sigma^{21}_T  $$
Where we've partitioned the unconditional covariance matrix $\Sigma_T$ as $$\Sigma_T=\begin{bmatrix}
\Sigma^{11}_T & \Sigma^{12}_T\\
\Sigma^{21}_T & \Sigma^{22}_T
\end{bmatrix} $$
We know that, in this case, $$\Sigma^{11}_T = 1$$
$$\Sigma^{12}_T = [1/2, \boldsymbol{0}]$$
$$\Sigma^{21}_T = [1/2, \boldsymbol{0}]^T$$ where $\boldsymbol{0}$ is an 1x(T-1) vectors of zeros (this is because $X_1$ is dependent only on $X_2$ and not on $X_3, X_4, \dots$).
Now, we can write $$[\Sigma^{22}_T]^{-1} = \begin{bmatrix}\Sigma^{11}_{T-1} & \Sigma^{12}_{T-1}\\
\Sigma^{21}_{T-1} & \Sigma^{22}_{T-1}
\end{bmatrix}^{-1} = 
\begin{bmatrix} (1 - \Sigma^{12}_{T-1} [\Sigma^{22}_{T-1}]^{-1}\Sigma^{21}_{T-1})^{-1} & (1 - \Sigma^{12}_{T-1} [\Sigma^{22}_{T-1}]^{-1}\Sigma^{21}_{T-1})^{-1}\Sigma^{12}_{T-1}[\Sigma^{22}_{T-1}]^{-1}\\ \vdots & \vdots
\end{bmatrix}$$
Where I left out the last row to save space (I'll show those entries don't matter anyways). This result holds by the Schur complement of a block matrix.
Next, since $$\Sigma^{12}_T = [1/2, \boldsymbol{0}]$$
$$\Sigma^{21}_T = [1/2, \boldsymbol{0}]^T$$ this all simplifies to $$\overline{\Sigma}_T = 1  - \frac{1}{4}(1 - \Sigma^{12}_{T-1} [\Sigma^{22}_{T-1}]^{-1}\Sigma^{21}_{T-1})^{-1}$$
However, since $\overline{\Sigma}_{T-1} = 1 - \Sigma^{12}_{T-1} [\Sigma^{22}_{T-1}]^{-1}\Sigma^{21}_{T-1}$, we see finally that 
$$\overline{\Sigma}_T = 1  - \frac{1}{4}[\overline{\Sigma}_{T-1}]^{-1}$$
Taking the limit as $T\rightarrow \infty$ yields $$\overline{\Sigma}_{\infty} = 1  - \frac{1}{4}[\overline{\Sigma}_{\infty}]^{-1}$$
Which can be solved to give the limiting conditional variance as $$\overline{\Sigma}_\infty = 0.5$$
