How to perform predictions using a probabilistic model? Suppose that we have fitted a probabilistic model (like GLM) to the data in order to perform predictions using test samples. The question is how to perform such predictions having the conditional distribution of response given data? Should we sample from the conditional distribution and count it as the predicted value? Or we should consider the mean or perhaps the median of the distribution as the predicted value? I have seen a paper where the median is used for prediction but I'm not sure whether it is correct.
 A: Say the input is $X$, the output is $Y$, and the fitted model gives the conditional distribution $P(Y \mid X)$. What you're looking to do is obtain a 'point estimate' from this distribution. Typically the mean $E[Y \mid X]$ is used. But, there's no reason you couldn't use the conditional mode, median, or even other quantiles. It just depends on your intended use.
One way to resolve the question is if you're specifically interested in one of these features of the distribution. Otherwise, say you're just interested in prediction performance. In that case, you have to specify a loss function, which measures how bad it is to make errors of a given size. You then choose the point estimate that minimizes the loss. For example, the ever-popular squared loss says that badness is proportional to the squared magnitude of the error. The mean is the point estimate that minimizes this loss function. The median minimizes the absolute loss, which says that badness is proportional to the magnitude of the error. These notes may be of interest.
