The Law of Total Variance says:
if the variance of X is finite then $V(X) = E(V(X|Z)) + V(E(X|Z))$
Suppose $X\sim N(0,1)$, $Y\sim \text{Cauchy}(0,1)$, $X$ and $Y$ are independent.
Define $Z \equiv X + Y$.
In Mathematica & Julia I get:
$V(X)=1$
$E(V(X|Z))=0.9948052149591405$
$V(E(X|Z))=0.11285474154640088$
Clearly: $1=V(X) \neq E(V(X|Z))+V(E(X|Z))=1.1076599565055414$
What am I missing?
Btw, I realize both $Y$ and $Z$ have infinite variance, but I don't see why that should matter here since $X$ has finite variance as required by the Law of Total Variance...
Julia code:
using Distributions, Plots
N = 10^7
X_s = rand(Normal(), N)
Y_s = rand(Cauchy(), N)
Z_s = X_s + Y_s
ε = .5 #tolerance
z_grid = -20.0:.01:20.0
cef = [ mean(X_s[z-ε .≤ Z_s .≤ z+ε]) for z ∈ z_grid ] # E[X|Z]
cvf = [ var(X_s[z-ε .≤ Z_s .≤ z+ε]) for z ∈ z_grid ] # V[X|Z]
var(X_s) # V[X]
mean(cvf) + var(cef) # E[V[X|Z]] + V[E[X|Z]]
Update: following @Ben's advice: now I'm off by 2-orders of magnitude
using Distributions, Plots
N = 10^7
X_s = rand(Normal(), N)
Y_s = rand(Cauchy(), N)
Z_s = X_s + Y_s
ε = .5 #tolerance
z_grid = -20.0:.01:20.0 #cdf(Cauchy(), 35.0)
cef = [ mean(X_s[z-ε .≤ Z_s .≤ z+ε]) for z ∈ z_grid ] # E[X|Z]
cvf = [ var(X_s[z-ε .≤ Z_s .≤ z+ε]) for z ∈ z_grid ] # V[X|Z]
pdf_z = [ mean([z-ε .≤ Z_s .≤ z+ε][1][:]) for z ∈ z_grid ]
#
var(X_s) # V[X] = 0.9993787441998672
E_V = pdf_z'cvf # E[V[X|Z]] = 74.65203538665487
Ecef = pdf_z'cef # E[E[X|Z]] = -0.035781335395943976
V_E = sum([ pdf_z[ii] * (cef[ii]-Ecef)^2.0 for ii ∈ eachindex(z_grid) ])
E_V + V_E #96.8588174093247