# Why Condition on a Random Variable in Iterated Expectation?

Say I wanted to take the following expectation for $$C$$ a constant and an RV $$\theta$$ with density $$p(\theta)$$.

$$E[\theta-C] = \int_{\theta}(\theta-C)p(\theta)d\theta$$

Now given $$p(y|\theta)$$ and a function $$\phi(y)$$, I would like to take the following expectation:

$$E[\theta-\phi(y)]$$

I am having trouble proving to myself the following:

$$E[\theta-\phi(y)] \ne \int_{\theta}(\theta-\phi(y))p(\theta)d\theta$$

But rather I believe the correct solution is:

$$E[\theta-\phi(y)] = \int_{y}\int_{\theta}(\theta-\phi(y))p(\theta|y)d\theta\ p(y)dy$$ $$=\int_{\theta}\int_{y}(\theta-\phi(y))\ p(\theta,y)dyd\theta$$ $$=E[E[(\theta-\phi(Y))|Y]]$$

Intuitively, I think the last line makes since, taking the average of $$\theta -\phi(y)$$ for a given $$y$$, and then taking the average of that over all $$y$$.

But I am still not convinced on why I shouldnt take the expectation treating $$\phi(y)$$ as a constant.

Is it necessary or sufficient to do the iterated expection if $$Y$$ is a random variable? Or is it only necessary when $$Y$$ depends on $$\theta$$ via $$p(y|\theta)$$?

Thanks for reading and any help on clairification!

• I think you should clarify whether you want $E_{\theta|y}(\theta - \phi(y))$ or $E_{\theta,y}(\theta - \phi(y))$. – Tim Mak Apr 27 '20 at 6:42

In fact both expressions for $$E[\theta -\phi(y)]$$ are correct, depending on which type of expectation you want to compute. The integral over $$\theta$$ only computes an expectation with respect to the marginal distribution of $$\theta$$, while the integral over both, $$\theta$$ and $$y$$, provides the expectation over the full joint distribution via the law of total expectation (see https://en.wikipedia.org/wiki/Law_of_total_expectation). Sometimes people use indices to indicate over which of the variables they intend to compute the expectation, so I think you simply got fooled by notation or the lack thereof.
Think of it this way: For most (but not all) probability distributions it is possible to compute an expectation. In the first case this is done for $$p(\theta)$$, the marginal distribution of $$\theta$$, while in the latter it is done for $$p(\theta,y)$$, i.e. the joint distribution of both variables or, equivalently, the random vector $$(\theta,y)$$.