If we want to predict one variable $Y$ based on another $X$, the best predictor is apparently $\mathbb{E}[Y \mid X = x]$. However, this apparently assumes two things:

  1. The distribution is symmetric.

    Which distribution needs to be symmetric? I think it is the conditional one, that is, $P(Y \mid X=x)$, because the mean would be the best predictor only if the distribution is symmetric. If it wasn't symmetric, then maybe the median would be a better predictor.

  2. We need to know the joint distribution of $Y$ and $X$, that is, we need to know $P(X=x, Y=y)$.

    Why do we strictly need to know $P(X=x, Y=y)$? I understand that, if we have the joint distribution, e.g. a table of all combinations of values of $X$ and $Y$, we can filter out all $Y$ values where $X \neq x$ and get only those where $X = x$, then we take the mean of those. Anyway, this should be the idea, which is applied in practice. But $\mathbb{E}[Y \mid X = x]$ should still be the best predictor (assuming symmetry), either we have the joint distribution or not.


In this context, the best predictor is the predictor that minimizes the variance $E[(Y-g(X))^2]$, where $g(X)$ is the prediction of $Y$, given the data $X$. Several analyses show that this expectation is minimized when $g(X)=E[Y|X]$. So, no symmetric assumption over any of the PDFs (PMFs). By the way, in general, you need to know $p_{Y|X}(y|x)$ to calculate the expected value. You can deduce it from the joint distribution, but having the complete joint is not necessary.

Median, or something else might sometimes result in a better estimate, but better in other sense. Not the variance defined above.

  • $\begingroup$ I have a source (which can be wrong) which states that the joint distribution needs to be known. So, why isn't the joint distribution necessary? Also, in which sense would the median be a "better" estimate, if not because symmetry needs to be taken into account? $\endgroup$ – nbro Jan 20 '19 at 13:06
  • $\begingroup$ Median minimizes $E[|Y-g(X)|]$. Having joint means having $p_X(X)$. If we have the specific value of $X$, i.e. $X=x$, we don't need $p_X(x)$ to calculate $E[Y|X=x]$. I can't think of a scenario for the necessity of $p(X)$, if I don't miss anything crucial in the problem. $\endgroup$ – gunes Jan 20 '19 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.