6
$\begingroup$

The widely used MCMCglmm package for R, which implements Bayesian Markov chain Monte Carlo estimation for generalized linear mixed models, computes a "pMCMC" value $$ p_{\textrm{MCMC}}(\beta) = 2 \times \min\left( \textrm{Prob}(\beta>0), \textrm{Prob}(\beta<0) \right) $$ which is a crude analogue of the frequentist 2-tailed $p$-value. The package author (Jarrod Hadfield) has commented that

[It's] not a p-value as such, and better ways of obtaining Bayesian p-values exist.

In one of Hadfield's papers (Hadfield et al. 2013) he defines it as

twice the posterior probability that the estimate is negative or positive (whichever probability is smallest)

(Unfortunately, some tutorials on MCMCglmm, and papers using it, have defined $p_{\textrm{MCMC}}$ as the "probability that the estimate does not differ from zero" (!!))

$p_{\textrm{MCMC}}$ could also be defined as $2 (1-p_D)$ where $p_D$ is the probability of direction, "the proportion of the posterior distribution that is of the median’s sign" (Makowski et al. 2019).

I am headed toward presenting $p_{\textrm{MCMC}}$ values in a paper, and am curious

  • whether anyone has seen this approach used elsewhere/more formally defined
  • suggestions for other useful Bayesian $p$-value analogues that will be reasonably interpretable by non-statisticians (I am not interested in model comparison methods, or in Bayesian p-values for diagnosing model adequacy)

Hadfield, Jarrod D., Elizabeth A. Heap, Florian Bayer, Elizabeth A. Mittell, and Nicholas M. A. Crouch. “Intraclutch Differences in Egg Characteristics Mitigate the Consequences of Age-Related Hierarchies in a Wild Passerine.” Evolution 67, no. 9 (2013): 2688–2700.

Makowski, Dominique, Mattan S. Ben-Shachar, S. H. Annabel Chen, and Daniel Lüdecke. “Indices of Effect Existence and Significance in the Bayesian Framework.” Frontiers in Psychology 10 (2019). https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02767.

$\endgroup$
2
  • $\begingroup$ @statmerkur, would you like to post this as an answer? $\endgroup$
    – Ben Bolker
    Jul 27, 2022 at 15:35
  • $\begingroup$ Elaboration would be nice but just posting what you've got in the comment would certainly suffice. $\endgroup$
    – Ben Bolker
    Jul 27, 2022 at 19:52

1 Answer 1

3
$\begingroup$

You might find Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests by Shi and Yin (2021) useful.

They show interesting connections between the two-sided posterior probability $\mathrm{{PoP}_2}$ and the (frequentist) p-value if flat or "non-informative" priors are used. In particular, $\mathrm{{PoP}_2}$ for two independent samples with normally or Bernoulli distributed outcomes is defined as
$$ \mathrm{{PoP}_2}=2[1-\max{\{\mathbb{P}(\beta<0|\mathrm{data}),\mathbb{P}(\beta>0|\mathrm{data})\}}], $$ where $\beta$ is the difference of the two population means.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.