If the distribution of $X$ is defined conditionally on variables $Z,W$, and we only use $W$ to estimate $X$, can $E(X)$ be defined?

Question

Suppose that $X,Z,W$ are random variables and that we are interested in estimating the expectation of $X$, $E(X)$. Suppose that the distribution of $X$ is defined as:

$$ X\mid Z=z, W=w \sim \mathcal{F}_{Z,W} $$

That is, $X$ is conditionally defined with respect to values of $Z$ and $W$. Suppose now that we would like to find $E(X)$ through a sampling procedure

$$ E(X)=E_{W}\bigg(E_{Z\mid W}\left(E\bigg(X\mid Z,W\bigg)\mid W\right)\bigg) $$

If a sampling procedure were to be taken, it would be

1. Draw a value of $w$ from the distribution of $W$.
2. Draw a value of $z$ from the conditional distribution of $Z\mid W$.
3. Draw a value of $x$ from the conditional distribution of $X\mid Z,W$.

I would then like to know what happens if we instead do the following procedure:

$$ E_{W}\bigg(E\bigg(X\mid W\bigg)\bigg) $$

That is, we

1. Draw a value of $w$ from the distribution of $W$.
2. Draw a value of $x$ from the conditional distribution of $X\mid W$.

By the iterated expectations, or the tower property, the above should be equal to $E(Y)$. However, I am unsure if such a thing can be defined, and what this sort of phenomena is normally referred to.

For example, if the issue is that $X$ needs $Z,W$ to be defined, we can rewrite as:

$$ E\bigg(X\mid W=w\bigg)= \sum_z E\bigg(X\mid W, Z=z\bigg)P\left(Z=z\mid W=w\right) $$

Would it then be a sum of distributions?

Answer 1

By the tower property for conditional expectations, it is always possible to write $$ \tag{1} \label{1} E[X] = E\big[E[X \mid W]\big] $$ for any random variables $X$ and $W$ (provided the expectation $E[X]$ exists).

This is easiest to see for discrete random variables: if $X$ and $W$ are discrete, then $$ \begin{aligned} E[X] &= \sum_x x P(X = x) \\ &= \sum_x x \sum_w P(X = x, W = w) \\ &= \sum_x x \sum_w P(X = x \mid W = w) P(W = w) \\ &= \sum_w \left(\sum_x x P(X = x \mid W = w\right) P(W = w) \\ &= \sum_w E[X \mid W = w] P(W = w) \\ &= E\big[E[X \mid W]\big]. \end{aligned} $$

In particular, \eqref{1} holds no matter what the relationship between $X$ and $W$ is; $X$ might be most easily defined in a conditional way depending on $W$, or it may be independent of $W$, or anything in between.

In your particular example, where the distribution of $X$ is most easily described by conditioning on random variables $W$ and $Z$, it is probably easiest to actually compute the expectation of $X$ as an expectation of iterated conditional expectations; e.g., $$ E[X] = E\big[E\big[E[X \mid W, Z] \,\big|\, W\big]\big] $$ or $$ E[X] = E\big[E\big[E[X \mid W, Z] \,\big|\, Z\big]\big]. $$ However, it is still correct to say that $$ E[X] = E\big[E[X \mid W]\big] $$ or $$ E[X] = E\big[E[X \mid Z]\big], $$ even if actually computing these expectations might be infeasible.