Variance of expectation vs. Expectation of variance Can we compare
$Var_X[E_Y(f(X,Y))]$
and
$E_Y[Var_X((f(X,Y))]$ where $f()$ is any function of $X$ and $Y$ iid? I suspect $Var_X[E_Y(f(X,Y))]$ is the smaller one. Though I couldn't find a single counter example, is there any formal proof to show (or counter example to count) this?
 A: The law of total variance is $$\mathrm{Var}_{(X,Y)}[f(X,Y)]=\mathrm{Var}_X[\mathbb E_{Y|X}\{f(X,Y)|X\}]+\mathbb E_X[\mathrm{Var}_{Y|X}\{f(X,Y)|X\}]\\=\mathrm{Var}_Y[\mathbb E_{X|Y}\{f(X,Y)|Y\}]+\mathbb E_Y[\mathrm{Var}_{X|Y}\{f(X,Y)|Y\}]\tag{0}$$
In the specific setting when
$$\mathbb E_{X|Y}[f(X,Y)|Y)]=0\qquad\text{a.s.}\tag{1}$$
we thus get
$$\mathrm{Var}_X[\mathbb E_{Y|X}\{f(X,Y)|X\}]+\mathbb E_X[\mathrm{Var}_{Y|X}\{f(X,Y)|X\}]=\mathbb E_Y[\mathrm{Var}_{X|Y}\{f(X,Y)|Y\}]$$
which implies
$$\mathrm{Var}_X[\mathbb E_{Y|X}\{f(X,Y)|X\}]\le\mathbb E_Y[\mathrm{Var}_{X|Y}\{f(X,Y)|Y\}]\tag{2}$$
Take $f(x,y)=a(x)b(y)$ and $X,Y$ independent. Then
$$\mathrm{Var}_X[\mathbb E_{Y|X}\{f(X,Y)|X\}]=\textrm{Var}_X(a(X))\mathbb E_{Y}[b(Y)]^2\tag{3}$$and
$$\mathbb E_Y[\mathrm{Var}_{X|Y}\{f(X,Y)|Y\}]=\mathbb E_Y[b(Y)^2]\mathrm{Var}_{X}(a(X))\tag{4}$$
implying that (3) is always smaller than (4).
The inequality
$$\mathrm{Var}_X[\mathbb E_{Y|X}\{f(X,Y)|X\}]\le\mathbb E_Y[\mathrm{Var}_{X|Y}\{f(X,Y)|Y\}]\tag{5}$$also holds when $f(x,y)=a(x)+b(y)+c$.
A last indication is that, when assuming wlog that $\mathbb E_{(X,Y)}[f(X,Y)]=0$, (5) is equivalent to
$$\mathbb E_{(X,Y)}[f(X,Y)^2]\ge \mathbb E_X[\mathbb E_{Y|X}\{f(X,Y)|X\}^2]+\mathbb E_Y[\mathbb E_{X|Y}\{f(X,Y)|Y\}^2]\tag{6}$$
Running a small Monte Carlo experiment with some highly varying and assymetric functions $f$ and a Student's $\mathfrak t_4$ on $X$ and $Y$ did not exhibit cases when (6) does not hold:
N=1e4;m=mean
fxy=outer(rt(N,4),rt(N,4),f)
fxy=fxy-m(fxy) #centred
m(fxy^2)-m(apply(fxy,1,m)^2)-m(apply(fxy,2,m)^2))

