In model selection with $Y$ as the outcome and $ X = (X_1, \ldots, X_p)$ the explanatory variables, why do we care about $F_{Y|X}$? In model selection with $Y$ as the outcome variable and $ X = (X_1, \ldots, X_p)$ the explanatory variables, why do we care about $F_{Y|X}$, the conditional distribution of the outcome $Y$ given the covariates $X = (X_1, \ldots, X_p)$?
I read that this is often times the object of study on model selection. Can anyone tell me why we care about the conditional distribution? What would knowing the conditional distribution do for us in prediction?
 A: Suppose you want to build a predictive model.  That is, you have some data $(X, Y)$, and you want to use knowledge of $X$ to make a prediction for the value of $Y$. Practically, you want to construct a function of $X$, so that $f(X)$ can be in some way construed as a "prediction for $Y$".
The thing you would most hope for is that you can build an $f$ that satisfies $f(X) = Y$ for all possibilities of $X$ and $Y$, a perfect link between two phenomina.
In practice, this is just not possible.  Whether it because of boundaries in our knowledge, deficiencies in our measurement, or some actual randomness in a process, we do not often believe that we can build a perfect link between $X$ and $Y$.  Given this reality, the conditional distribution $Y \mid X$ is simply a mathematical tool that is used to describe our state of knowledge about $Y$ once we have complete knowledge of $X$.
So if we construe $f(X)$ as a prediction of $Y$, then $f(X)$ must tell us something about the conditional distribution $Y | X$.  Sometimes we try to predict the mean of the conditional distribution, sometimes the median, sometimes something else, but a predictive model is, at the end of the day, always trying to tell us something about the conditional distribution.
A: A regression model or a GLM (or indeed nay number of other models) are at heart a model of the conditional distribution, generally including an explicit description of the conditional mean in terms of the parameters. 
When you're estimating parameters in (say) a linear regression, you're actually working out how (in your model) the conditional mean of $Y$ changes as the things you're conditioning on (the $x$'s, the independent variables, the predictors) change.

In a regression model other aspects of the conditional distribution are also defined -- the conditional variance for example, may often be assumed to be constant.
Similarly, with a Poisson GLM, when I say that $Y|x$ is Poisson, with $$\mu(x) = E(Y|x) = \exp(\beta_0+\beta_1x)\,,$$ I am explicitly modelling the conditional distribution by writing a model for the conditional mean of $Y$ -- that's what the parameters $\beta_0$ and $\beta_1$ define; then the conditional distribution follows from the fact that once you have the mean of the Poisson, you have its distribution:
 
Here the model is that at any given value of $x$ (i.e. conditional on $x$) the $Y$ value has a Poisson distribution. Note that the marginal distribution of $Y$ is not Poisson. The blue points (some laying one on another) are - according to the model - generated from a Poisson distribution at each value of $x$ (the conditional means of which are related by a model). The "+" symbols mark the estimates of these conditional means at each $x$ value for which we had data.
