12
$\begingroup$

Generally, What is difference between $E(X|Y)$ and $E(X|Y=y)$?

Former is function of $y$ and latter is function of $x$? It's so confusing..

$\endgroup$
18
$\begingroup$

Roughly speaking, the difference between $E(X \mid Y)$ and $E(X \mid Y = y)$ is that the former is a random variable, whereas the latter is (in some sense) a realization of $E(X \mid Y)$. For example, if $$(X, Y) \sim \mathcal N\left(\mathbf 0, \begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$$ then $E(X \mid Y)$ is the random variable $$ E(X \mid Y) = \rho Y. $$ Conversely, once $Y = y$ is observed, we would more likely be interested in the quantity $E(X \mid Y = y) = \rho y$ which is a scalar.

Maybe this seems like needless complication, but regarding $E(X \mid Y)$ as a random variable in its own right is what makes things like the tower-law $E(X) = E[E(X \mid Y)]$ make sense - the thing on the inside of the braces is random, so we can ask what its expectation is, whereas there is nothing random about $E(X \mid Y = y)$. In most cases we might hope to calculate $$ E(X \mid Y = y) = \int x f_{X\mid Y}(x \mid y) \ dx $$

and then get $E(X \mid Y)$ by "plugging in" the random variable $Y$ in place of $y$ in the resulting expression. As hinted in an earlier comment, there is a bit of subtlety that can creep in with regards to how these things are rigorously defined and linking them up in the appropriate way. This tends to happen with conditional probability, due to some technical issues with the underlying theory.

$\endgroup$
6
$\begingroup$

Suppose that $X$ and $Y$ are random variables.

Let $y_0$ be a fixed real number, say $y_0 = 1$. Then, $E[X\mid Y=y_0]= E[X\mid Y = 1]$ is a number: it is the conditional expected value of $X$ given that $Y$ has value $1$. Now, note for some other fixed real number $y_1$, say $y_1=1.5$, $E[X\mid Y = y_1] = E[X\mid Y = 1.5]$ would be the conditional expected value of $X$ given $Y = 1.5$ (a real number). There is no reason to suppose that $E[X\mid Y = 1.5]$ and $E[X\mid Y = 1]$ have the same value. Thus, we can also regard $E[X\mid Y=y]$ as being a real-valued function $g(y)$ that maps real numbers $y$ to real numbers $E[X\mid Y = y]$. Note that the statement in the OP's question that $E[X\mid Y = y]$ is a function of $x$ is incorrect: $E[X\mid Y = y]$ is a real-valued function of $y$.

On the other hand, $E[X\mid Y]$ is a random variable $Z$ which happens to be a function of the random variable $Y$. Now, whenever we write $Z = h(Y)$, what we mean is that whenever the random variable $Y$ happens to have value $y$, the random variable $Z$ has value $h(y)$. Whenever $Y$ takes on value $y$, the random variable $Z = E[X\mid Y]$ takes on value $E[X\mid Y = y] = g(y)$. Thus, $E[X\mid Y]$ is just another name for the random variable $Z = g(Y)$. Note that $E[X\mid Y]$ is a function of $Y$ (not $y$ as in the statement of the OP's question).

As a a simple illustrative example, suppose that $X$ and $Y$ are discrete random variables with joint distribution $$\begin{align} P(X=0,Y=0) = 0.1, P(X=0, Y=1) = 0.2,\\ P(X=1,Y=0) = 0.3, P(X=1,Y=1) = 0.4. \end{align}$$ Note that $X$ and $Y$ are (dependent) Bernoulli random variables with parameters $0.7$ and $0.6$ respectively, and so $E[X] = 0.7$ and $E[Y] = 0.6$. Now, note that conditioned on $Y=0$, $X$ is a Bernoulli random variable with parameter $0.75$ while conditioned on $Y = 1$, $X$ is a Bernoulli random variable with parameter $\frac 23$. If you cannot see why this is so immediately, just work out the details: for example $$P(X=1\mid Y = 0) = \frac{P(X=1, Y=0)}{P(Y=0)} = \frac{0.3}{0.4} = \frac 34,\\ P(X=0\mid Y = 0) = \frac{P(X=0, Y=0)}{P(Y=0)} = \frac{0.1}{0.4} = \frac 14,$$ and similarly for $P(X=1\mid Y=1)$ and $P(X=0\mid Y = 1)$. Hence, we have that $$E[X\mid Y = 0] = \frac 34, \quad E[X \mid Y = 1] = \frac 23.$$ Thus, $E[X\mid Y = y] = g(y)$ where $g(y)$ is a real-valued function enjoying the properties: $$g(0) = \frac 34, \quad g(1) = \frac 23.$$

On the other hand, $E[X\mid Y] = g(Y)$ is a random variable that takes on values $\frac 34$ and $\frac 23$ with probabilities $0.4 = P(Y=0)$ and $0.6 = P(Y=1)$ respectively. Note that $E[X\mid Y]$ is a discrete random variable but is not a Bernoulli random variable.

As a final touch, note that $$E[Z] = E\left[E[X\mid Y]\right] = E[g(Y)] = 0.4\times \frac 34 + 0.6\times \frac 23 = 0.7 = E[X].$$ That is, the expected value of this function of $Y$, which we computed using only the marginal distribution of $Y$, happens to have the same numerical value as $E[X]$ !! This is an illustration of a more general result that many people believe is a LIE: $$E\left[E[X\mid Y]\right] = E[X].$$

Sorry, that's just a small joke. LIE is an acronym for Law of Iterated Expectation which is a perfectly valid result that everyone believes is the truth.

$\endgroup$
3
$\begingroup$

$E(X|Y)$ is the expectation of a random variable: the expectation of $X$ conditional on $Y$. $E(X|Y=y)$, on the other hand, is a particular value: the expected value of $X$ when $Y=y$.

Think of it this way: let $X$ represent the caloric intake and $Y$ represent height. $E(X|Y)$ is then the caloric intake, conditional on height - and in this case, $E(X|Y=y)$ represents our best guess at the caloric intake ($X$) when a person has a certain height $Y = y$, say, 180 centimeters.

$\endgroup$
  • 4
    $\begingroup$ I believe your first sentence should replace "distribution" with "expectation" (twice). $\endgroup$ – Glen_b Oct 10 '14 at 15:02
  • 4
    $\begingroup$ $E(X\mid Y)$ isn't the distribution of $X$ given $Y$; this would be more commonly denotes by the conditional density $f_{X \mid Y} (x \mid y)$ or conditional distribution function. $E(X \mid Y)$ is the conditional expectation of $X$ given $Y$, which is a $Y$-measurable random variable. $E(X \mid Y = y)$ might be thought of as the realization of the random variable $E(X \mid Y)$ when $Y = y$ is observed (but there is the possibility for measure-theoretic subtlety to creep in). $\endgroup$ – guy Oct 10 '14 at 15:24
  • 1
    $\begingroup$ @guy Your explanation is the first accurate answer yet provided (out of three offered so far). Would you consider posting it as an answer? $\endgroup$ – whuber Oct 10 '14 at 16:17
  • $\begingroup$ @whuber I would but I'm not sure how to strike the balance between accuracy and making the answer suitably useful to OP and I'm paranoid about getting tripped up on technicalities :) $\endgroup$ – guy Oct 10 '14 at 16:42
  • $\begingroup$ @Guy I think you have already done a good job with the technicalities. Since you are sensitive about communicating well with the OP (which is great!), consider offering a simple example to illustrate--maybe just a joint distribution with binary marginals. $\endgroup$ – whuber Oct 10 '14 at 17:00
1
$\begingroup$

$E(X|Y)$ is expected value of values of $X$ given values of $Y$ $E(X|Y=y)$ is expected value of $X$ given the value of $Y$ is $y$

Generally $P(X|Y)$ is probability of values $X$ given values $Y$, but you can get more precise and say $P(X=x|Y=y)$, i.e. probability of value $x$ from all $X$'s given the $y$'th value of $Y$'s. The difference is that in the first case it is about "values of" and in the second you consider a certain value.

You could find the diagram below helpful.

Bayes theorem diagram form Wikipedia

$\endgroup$
  • $\begingroup$ This answer discusses probability, while the question asks about expectation. What is the connection? $\endgroup$ – whuber Oct 13 '14 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.