How does one calculate $\text{Var}(E[Y|X])$? and other related quantities

Question

I know how to calculate $E[Y|X=x]$ from knowing how to calculate $P(Y|X=x)$.

What I don't understand is the meaning of $P(Y=y|X)$. Wouldn't be $P(X)$ = 1 since it is taken over the entire random variable? Or is it constraining X to some implicit set like $x_1 < X < x_2$, so you can then apply Bayes' theorem?

I have seen people suggest calculating $E[Y|X] = \int y p(y|X) dy$ which I don't understand.

My understanding on calculating $\text{Var}(E[Y|X)$ as it stands is calculating $E[Y|X=x]$ and then marginalizing over X for the variance part, but I think I am missing something when it comes to $P(y|X)$ or going from $E[Y|X]$ to $\text{Var}(E[Y|X)$.

Answer 1

When you specify the conditional expectation (or other moment) of a random variable $Y$ given a specific value of the conditioning variable $X=x$, the result is a function of the argument $x$. In generic function notation, we have:

$$\mathbb{E}(Y|X=x) = f(x).$$

When you replace the argument value $x$ with the actual random variable $X$ (by declining to specify a value for the conditioning variable) the result is a function of the random variable $X$. In generic function notation, we have:

$$\mathbb{E}(Y|X) = f(X).$$

Because this latter form is a function of a random variable, it is itself a random variable. That is, the conditional expectation of $Y$ conditional on $X$ is a random variable. The distribution of this random variable is obtained through the appropriate transformation rules using the distribution of $X$.

---

An example: Suppose we have $Y|X=x \sim \text{N}(2x, 1)$ and $X \sim \text{N}(\mu, 1)$. Then we have the conditional expectation function:

$$f(x) \equiv \mathbb{E}(Y|X=x) = 2x.$$

The random variable version of this is:

$$f(X) = \mathbb{E}(Y|X) = 2X \sim \text{N}(2 \mu, 4).$$

So in this case you would have:

$$\mathbb{V}(\mathbb{E}(Y|X)) = \mathbb{V}(2X) = 4.$$