let $X,Y,Z$ be random variables defined on the same probability space and let covariance of $X$ and $Y$ be finite, then the law of total covariance / covariance decomposition formula states: \begin{align} \text{Cov}(X,Y)=\underbrace{\mathbb{E}\big[\text{Cov}(X,Y\lvert Z)\big]}_{\text{(i)}}+\underbrace{\text{Cov}\big[\mathbb{E}(X\lvert Z),\mathbb{E}(Y\lvert Z)\big]}_{\text{(ii)}} \end{align} What is the interpretation of $\text{(i)}$ and $\text{(ii)}$?

My thoughts: in (ii) the two conditional expectations can be seen as random variables themselves, I also know that this is a generalization of the law of total variance / variance decomposition formula which can be shown by setting $X=Y$, where the interpretation is then that of a variation in $Y$ explained by $Z$ and unexplained by $Z$. But what is the correct interpretation in the above covariance formula for (i) and (ii)? Wikipedia offers a brief description which is not very satisfying.


The first term (i): $E[\operatorname{cov}(X,Y|Z)]$

Think of $\operatorname{cov}(X,Y)$ as a function of $Z$. As you examine different values of $Z$, you will correspondingly get a value for $\operatorname{cov}(X,Y)$. The expectation simply averages these different covariances with respect to $Z$.

The second term (ii): $\operatorname{cov}([E[X|Z],E[Y|Z])$

Think of $E[X|Z]$ and $E[Y|Z]$ as functions of $Z$. As you examine different values of $Z$, you will correspondingly get a value of $E[X|Z]$ and a value of $E[Y|Z]$ realized simultaneously. Therefore, for every value of $Z$, you will get an $(X, Y)$ coordinate. This term is simply the covariance of all these coordinate points.


Another possible interpretation in a hierarchal framework is simple a decomposition of the total covariance $\operatorname{cov}(X,Y)$ into two terms:

  1. the within group ($E[\operatorname{cov}(X,Y|Z)]$) and
  2. between group $\operatorname{cov}([E[X|Z],E[Y|Z])$

covariances. The first term represents in this example the average of the covariances of $X$ and $Y$ evaluated for each group while the second term is the covariance of the group-averages for $X$ and $Y$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.