I am reading a bit about Bayesian analysis, but I cannot understand the difference between the classic quantiles and the Highest Posterior Density Intervals. What is the difference between the two? I have simulated and plotted some data, the Q95% and the 9%% HPDI seems to be similar but not identical. Are there situations where they differ to a larger amount? Thanks a lot
For unimodal, more-or-less symmetric distributions, HPD- and quantile-based credible intervals won't be too different. But consider a bimodal posterior distribution with well-separated modes: the HPD-based credible region will be two disjoint intervals whereas the central quantile-based credible region is a single interval by construction.
From a decision theory perspective, the two different kinds of intervals correspond to two different loss functions. The big difference is
that the HPD corresponds to a loss function that includes a penalty for the length of the credible region(s). (um, no, as guest implicitly points out, it's) that if the interval fails to cover the true value, the loss function for the quantile-based credible interval penalizes you for how wrong you are whereas in the loss function for the HPD interval, "a miss is as good as a mile".
A simple example would be if you bought a light-bulb with a lifetime which was exponentially distributed with a mean of 1000 days.
With a 95% credible region: would you tend to see it as likely to last for between 25 and 3689 days (based on the quantiles at 0.025 and 0.975), or would you see it as likely to last fewer than 2996 days (based on the Highest Density Interval)?
In other words, would it surprise you if it died almost as soon as you bought it, even though the mode of the distribution is at zero?