0
$\begingroup$

In prediction task, when using Bayesian fashion of predictors, I think in most the cases, people just use posterior mean for each individual estimate. I wonder if there's any utility of the higher order information, e.g. variance of the posterior predictive, in any task?

I can think of example where the whole distribution might be useful is that, making a false positive in the prediction is more costly than making a false negative. In this case, people might want to define some utility function, which use the whole distribution information.

It would be great if people can point me more examples and references. Thanks a lot in advance!


For a concrete example, suppose I perform a Bayesian linear regression, $\boldsymbol{\beta} \sim \pi(\cdot)$, $y_i = \mathbf{x}_i^\top \boldsymbol{\beta} + \epsilon_i$. With some particularly chosen prior, I get posterior of $\boldsymbol{\beta}$, $\pi(\boldsymbol{\beta} | \mathbf{x}_{1:N}, y_{1:N})$, with this posterior, I could get posterior predictive distribution for new data point $\mathbf{x}_{\text{new}}$: $$p(y_\text{new} | \mathbf{x}_{\text{new}}, \mathbf{x}_{1 :N}, y_{1:N}) = \int p(y_\text{new} | \mathbf{x}_{\text{new}}, \boldsymbol{\beta}) \pi(\boldsymbol{\beta} | \mathbf{x}_{1:N}, y_{1:N}) d\boldsymbol{\beta} $$ So for this new data point, I get a posterior distribution. For a real example, this could be the predicted susceptability for some disease for a potential patient. I may want to make decision whether to further check the patient. I can think of two examples: 1. use the posterior mean 2. use some tail density. It's not clear to me which one is more principled to use.

$\endgroup$
2
$\begingroup$

In a simple Bayesian application to a binomial experiment, one can begin with the prior $\mathsf{Beta}(5,7)$ that places about 95% of the probability of the success probability $\theta$ in the interval $(.15,.7).$ [Computation in R.]

diff(pbeta(c(.15,.7), 5,7))
[1] 0.9624924

Then suppose $n=200$ Bernoulli trials result in $x = 83$ Successes, so that the likelihood is proportional to $\theta^{83}(1-\theta)^{117}.$

Finally, the posterior distribution proportional to the product of the prior density and the binomial likelihood: $$\theta^{5+83-1}(1-\theta)^{7+117-1} = \theta^{88-1}(1-\theta)^{124-1},$$ which is the kernel of the posterior distribution $\mathsf{Beta}(88,124).$

Of course, the posterior mean is 0.415, but in many applications the posterior distribution is also used to get the 95% Bayesian posterior probability interval $(0.350, 0.482)$ for $\theta.$ Moreover, the posterior distribution can be used to argue that $\theta$ is very likely less than $1/2$ (probability above $0.99).$

qbeta(c(.025,.975), 88, 124)
[1] 0.3497507 0.4819556
pbeta(.5, 88, 124)
[1] 0.9934937

enter image description here

$\endgroup$
6
  • $\begingroup$ thanks for this example with R code! regarding "but in many applications the posterior distribution is also used to get the 95% Bayesian posterior probability interval". It would be great if you can provide more examples. I can imagine cases where one use this to obtain their belief about parameters, similar things like hypothesis testing. Also is there example for the posterior predictive? $\endgroup$ – david Sep 10 '20 at 15:42
  • $\begingroup$ Routine in Bayesian statistics, especially for beta-binomial, gamma-Poisson, nornal-normal conjugacy. See any elementary Bayesian stat book, and Wikipedia on 'binomial CI's', especially Jeffries binomial CI. Also in Gibbs sampling, example here. See book by Suess, Springer 2010. $\endgroup$ – BruceET Sep 10 '20 at 16:02
  • $\begingroup$ thanks, i think i understand one can get a posterior for some specified probabilistic model. Here, I am more curious about what people do with their obtained posterior. As I mentioned in the question description, I see numerous examples of usage of posterior mean. And as you have mentioned, one can use the posterior interval to perform something like hypothesis testing. Are their other examples on the usage of posterior? $\endgroup$ – david Sep 10 '20 at 16:06
  • $\begingroup$ We must have orthogonal interests/backgrounds in Bayesian statistics. I dont recall reading a recent application where only the posterior mean is used. // I feel sort of like I'm being asked for references that the CLT is used in frequentist inference $\endgroup$ – BruceET Sep 10 '20 at 16:13
  • 1
    $\begingroup$ The mean can be useful but the credibility interval of the posterior predictive distribution tells you how useful the mean really is for prediction. It's analogous to how the prediction interval is used in frequentest analyses. Here is just one example of it's use (see pgs. 15-16): epa.gov/sites/production/files/2020-05/documents/… $\endgroup$ – Michael Webb Sep 10 '20 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.