8
$\begingroup$

I hope someone can clear the doubt on conditional distribution, in a text, it mentions:

To interpret the notation $f_{X \mid Y}(x \mid y)$, we need to be clear that there is nothing random about the random variable $Y$, since it is already fixed. What remains is a sub-population distribution in terms of $X$ (i.e. the reduced sample space of $\Omega_X$ where $Y$ happened). Consequently, the conditional PMF/PDF $f_{X \mid Y}(x \mid y)$ is a distribution in terms of $X$.

I have no trouble understanding this, since conditional just means looking into the reduced sample space of $X$ where $Y$ has happened for a specific set. But soon after, when conditional expectation is mentioned, it says:

There are two points to note here. First, the expectation of $\mathbb{E}[X \mid Y=y]$ is taken with respect to $f_{X \mid Y}(x \mid y)$. We assume that the random variable $Y$ is already fixed at the state $Y=y$. Thus, the only source of randomness is $X$. Secondly, since the expectation $\mathbb{E}[X \mid Y=y]$ has eliminated the randomness of $X$, the resulting function is in $y$.

A quick read online reveals that the expectation of $X$ given $Y$ can indeed be a function of $Y$ if we treat this conditional expectation as random variable. I am just slightly confused why the conditional expectation is not a function of $X$, since the underlying conditional distribution is a function of $X$.

$\endgroup$

2 Answers 2

11
$\begingroup$

Without getting very much into measure theory,consider the random vector $(X,Y)$ with density $f_{X,Y}(\cdot,\cdot)$ (wrt a dominating measure $\text d\mu(x,y)$) decomposed as $$f_{X,Y}(x,y)=f_Y(y)\times f_{X|Y}(x|y)$$ where

  • $f_Y(\cdot)$ is a probability density (wrt the appropriate dominating measure $\text d\mu_2(y)$) attached with the random variable $Y$
  • $f_{X|Y}(\cdot|y)$ is a probability density (wrt the appropriate dominating measure $\text d\mu_1(x)$) for (almost) every $y\in\mathcal Y$, attached with a random variable $Z_y$. In this notation, $y$ is a parameter of the density.

For a fixed value of $y\in\mathcal Y$, $f_{X|Y}(\cdot|y)$ can thus be understood as a regular density over the set $\mathcal X$ and $$f_{X|Y}(\cdot|y):\ x\longmapsto f_{X|Y}(x|y)$$ is a non-negative integrable (measurable) function on $\mathcal X$ such that $$\int_\mathcal Xf_{X|Y}(x|y)\text d\mu_1(x)=1$$ This means that, for a fixed value of $y\in\mathcal Y$, the expectation of $Z_y\sim f_{X|Y}(\cdot|y)$ can considered and, provided it exists for this specific value of $y\in\mathcal Y$, be defined as $$\mathbb E_y[X] = \int_\mathcal X xf_{X|Y}(x|y)\text d\mu_1(x)\tag{1}$$ It is usually written as $\mathbb E[X|Y=y]$. In the event (1) exists for all values of $y\in\mathcal Y$, the function $$\varphi:\ y \longmapsto \mathbb E[X|Y=y]$$ is rigorously defined. It can therefore be called to transform the random variable $Y$ into the new random variable $\varphi(Y)$. It is usually written as $\mathbb E[X|Y]$ and is equal to (1) when the realisation of $Y$ is equal to $y$.

As seen above, this random variable $\varphi(Y)=\mathbb E[X|Y]$ is not a function of the random variable $X$, even though they may be correlated with one another. Hence, the citation

First, the expectation of $\mathbb E[X∣Y=y]$ is taken with respect to $f_{X∣Y}(x∣y)$. We assume that the random variable $Y$ is already fixed at the state $Y=y$. Thus, the only source of randomness is $X$.

should be restated as [with highlighted changes]:

First, the expectation $\mathbb E[X∣Y=y]$ is taken with respect to the distribution with density $f_{X∣Y}(x∣y)$. We assume that the random variable $Y$ is already observed at the realisation $Y=y$. Thus, the only remaining source of randomness in the expectation is $X$ with distribution $f_{X∣Y}(\cdot∣y)$, that is, the conditional distribution of $X$ given $Y=y$.

Similarly,

Secondly, since the expectation $\mathbb E[X∣Y=y]$ has eliminated the randomness of $X$, the resulting function is in $y$.

should state

Secondly, since the expectation $\mathbb E[X∣Y=y]$ has eliminated the (remaining) conditional randomness of $X$ given $Y=y$, the resulting function is a function of $y$.

$\endgroup$
3
  • 1
    $\begingroup$ Thanks for this rigorous explanation! $\endgroup$
    – jmarkov
    Jan 9, 2023 at 2:15
  • $\begingroup$ Thanks. This is rigourous and clarifies my doubt. From what you wrote, the PDF of $X$ given $Y$ is still a function of $X$ right? Just want to make sure that this is still true. $\endgroup$
    – nan
    Jan 9, 2023 at 7:12
  • 1
    $\begingroup$ The pdf of the rv $X$ given thr rv $Y$ is the function $f_{X|Y}(⋅|⋅)$ and, therefore, is a function/mapping from $\mathcal X$ to $\mathbb R^+$ indexed by the realisation of $Y$ or $Y$ itself, but definitely not a function of the random variable $X$. (Just as the marginal density $f_X$ is not a function of $X$. It determines the distribution of $X$ and thus in a very informal sense depends on $X$, but it is not a mathematical function of the rv $X$.) $\endgroup$
    – Xi'an
    Jan 9, 2023 at 10:04
12
$\begingroup$

If you take the expectation of $X$, it's not a function of $X$. You integrate $xf(x)$ over $x$, and so $x$ is gone, producing only a number.

If you take the expectation of $X|Y=y$, it's not a function of $X$. You integrate $xf(x|y)$ over $x$ (think of it as $xf(x)$ with a different $f(x)$ for different $y$), and so $x$ is gone, producing only a number, which may be different for different $y$s, hence a function of $y$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.