Interpretation of Total Law of Covariance

Question

let $X,Y,Z$ be random variables defined on the same probability space and let covariance of $X$ and $Y$ be finite, then the law of total covariance / covariance decomposition formula states: \begin{align} \text{Cov}(X,Y)=\underbrace{\mathbb{E}\big[\text{Cov}(X,Y\lvert Z)\big]}_{\text{(i)}}+\underbrace{\text{Cov}\big[\mathbb{E}(X\lvert Z),\mathbb{E}(Y\lvert Z)\big]}_{\text{(ii)}} \end{align} What is the interpretation of $\text{(i)}$ and $\text{(ii)}$?

My thoughts: in (ii) the two conditional expectations can be seen as random variables themselves, I also know that this is a generalization of the law of total variance / variance decomposition formula which can be shown by setting $X=Y$, where the interpretation is then that of a variation in $Y$ explained by $Z$ and unexplained by $Z$. But what is the correct interpretation in the above covariance formula for (i) and (ii)? Wikipedia offers a brief description which is not very satisfying.

Answer 1

The first term (i): $E[\operatorname{cov}(X,Y|Z)]$

Think of $\operatorname{cov}(X,Y)$ as a function of $Z$. As you examine different values of $Z$, you will correspondingly get a value for $\operatorname{cov}(X,Y)$. The expectation simply averages these different covariances with respect to $Z$.

The second term (ii): $\operatorname{cov}([E[X|Z],E[Y|Z])$

Think of $E[X|Z]$ and $E[Y|Z]$ as functions of $Z$. As you examine different values of $Z$, you will correspondingly get a value of $E[X|Z]$ and a value of $E[Y|Z]$ realized simultaneously. Therefore, for every value of $Z$, you will get an $(X, Y)$ coordinate. This term is simply the covariance of all these coordinate points.

Answer 2

Another possible interpretation in a hierarchal framework is simple a decomposition of the total covariance $\operatorname{cov}(X,Y)$ into two terms:

1. the within group ($E[\operatorname{cov}(X,Y|Z)]$) and
2. between group $\operatorname{cov}([E[X|Z],E[Y|Z])$

covariances. The first term represents in this example the average of the covariances of $X$ and $Y$ evaluated for each group while the second term is the covariance of the group-averages for $X$ and $Y$.