How best to summarize a predictive discrete distribution in a single number? I have generated a predictive distribution for a future discrete observable outcome, and would like to generate a single value $p$ which we would most likely encounter when we perform the experiment the next time.
I am confused: should I use the expectation or mean or mode? The mode seems the most reasonable to me as it represents the peak, thus implying that this value is most likely to be sampled.
 A: If you are indeed looking for the most likely outcome of the next run of your experiment and your outcomes are discrete, then you should indeed use the mode of your predictive PMF, i.e., the value where your predictive PMF has the highest value - because the PMF contains your predicted probabilities, so its mode is by definition the value which you predict to be most likely.
However, this value may not be terribly informative. For instance, you may have zero-inflation. Or simply a negative-binomial predictive distribution where the mode is zero - but expected values or outcomes in general can be very large.
As @Cagdas Ozgenc notes, your optimal point prediction will depend heavily on your loss function.


*

*If your loss function indeed depends simply on being wrong or right (if you predict the correct number, you get $x$ dollars; if not, you get nothing), then you should indeed predict the most likely value, or the mode (in the discrete case - in the continuous case, this loss function does not make sense).

*If your loss function depends on the absolute difference between your point prediction and the actual outcome, you should use the median of your predictive PMF, which will minimize this expected absolute difference (Hanley et al., 2001, The American Statistician).

*If your loss function depends on the squared difference between your point prediction and the actual outcome, you need the expected value of your predictive PMF. That the expectation minimizes the expected square loss is a very standard result in statistics.

*Finally, a quantile of your predictive PMF may be the optimal point forecast for a variety of other loss functions (Gneiting, 2011, International Journal of Forecasting). See also quantile-regression on this.


Note that except for the very first bullet point, all this is equally valid for both discrete and continuous predictive distributions. The only problem for the continuous case is that there is no "most likely" outcome here, since any specific outcome has probability zero.
