For a Bayesian model $M(\theta_1, \theta_2)$ I simulated 100 datasets with fixed $\theta_1$ and $\theta_2$ and then ran posterior sampling on each dataset using model $M(\theta_1, \theta_2)$. Now, my question is, what is the best way to present results for this experiment? My idea would be to report for both parameters the average posterior mean, 2.5% and 97.5% credible intervals extremes accross simulations and compare them with the true values, would this be a valid way of representing the experiment? Thanks

  • $\begingroup$ I think what you mentioned are a good way of comparing posterior results with the true values $\endgroup$
    – Fiodor1234
    Jul 7 at 11:33
  • $\begingroup$ The question is what do you want to achieve with? If you want to see if the inference is correct, it would be more common to have variable parameters, see e.g. rdrr.io/cran/BayesianTools/man/calibrationTest.html $\endgroup$ Jul 7 at 11:50
  • $\begingroup$ @FlorianHartig, thanks for the reference, I am aware of that approach, however I am more interested in overall calibration of posterior estimates, which I already compute. I was just looking for a good way to report some info about point and interval estimates. Additionally, I use vague priors in the MCMC, so I would have to use different priors for simulation and posterior sampling, otherwise I'd generate absurd datasets $\endgroup$
    – Mkg
    Jul 7 at 12:05
  • $\begingroup$ @Mkg - OK, I see ... I guess if you want to check the inference, then it's fine for me, although I would probably prefer median posterior or MAP over the mean. And of course there is the issue that in general, neither of these values will be unbiased / calibrated (due to the prior dependency in Bayesian stats), but in practice, the approach should still tell you if you get back the "true" parameters with a reasonable coverage. $\endgroup$ Jul 7 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.