Proving result about expectation under posterior predictive distribution

Question

I am trying to figure out how to prove a result about the expectation of a random variable under the posterior predictive distribution, that may or may not be true.

Let $X$ be a random variable, and $D$ be observed data of this random variable. Suppose $X$ belongs to a parametric family with parameter $\theta$. Then I wish to show, \begin{equation} \mathbb{E}_{X|D}[X]=\mathbb{E}_{X|D}[\mathbb{E}_{\theta|D}[X|\theta]] \end{equation}

where the outer expectation is taken under the posterior predictive, and the inner expectation is taken under the posterior distribution. It seems intuitively true to me because under the posterior predictive distribution we don't assume $X$ has any parameter, so when taking the expectation of $X$ under the predictive, we must first average out over the possible values of $\theta$.

My ideas about how to prove it used the tower rule (law of total expectation) and using that the posterior predictive pdf satisfies $f(x|D)=\mathbb{E}_{\theta|D}[f(x|\theta)]$ but neither of these worked. Does anyone have any suggestions about how to prove this, if it is even true?

Thanks.

EDIT: Here is an example of this in action.

Suppose $X \sim N(\theta,\sigma^2)$, with $\theta$ unknown. Then the posterior $\theta|D_n$ is Normal with parameters $\theta_n,\sigma^2_n$ say. The posterior-predictive distribution is also normal with parameters $\theta_n$ and $\sigma^2_n+\sigma^2$.

Then $\mathbb{E}_{X|D}[X]=\theta_n$ and \begin{align} \mathbb{E}_{X|D}[\mathbb{E}_{\theta|D}[X|\theta]]&=\mathbb{E}_{X|D}[\mathbb{E}_{\theta|D}[\theta+{\sigma}Z]]\\ &=\mathbb{E}_{X|D}[\theta_n+{\sigma}Z]=\theta_n+0=\theta_n \end{align}

This is just one example though. Still not convinced it is correct.

Answer 1

I am not sure about $\mathbb{E}_{\theta|D}[X|\theta]$, since the subscript $\theta|D$ implies that the expectation is averaging over all $\theta$, but the $|\theta$ part in the expectation itself implies that $\theta$ has already been observed. Therefore, I will assume that $\mathbb{E}_{\theta|D}[X|\theta] \equiv \mathbb{E}_{\theta|D}[X]$.

Using the definition of a conditional expectation:

$$ \mathbb{E}_{X|D}[\mathbb{E}_{\theta|D}[X]] = \mathbb{E}_{X|D}\left[\int_{\theta} x \ p(\theta|D) \ d\theta \right] $$

Using the definition of conditional expectation again:

$$ \begin{align} \mathbb{E}_{X|D}\left[\int_{\theta} x \ p(\theta|D) \ d\theta \right] &= \int_x p(x|D) \cdot \left[\int_{\theta} x \ p(\theta|D) \ d\theta \right] dx \\ &= \int_x x \ p(x|D) \cdot \left[\int_{\theta} \ p(\theta|D) \ d\theta \right] dx \\ &= \int_x x \ p(x|D) \ dx \\ &= \mathbb{E}_{X|D}[X] \end{align} $$