Can you give a practical numerical example illustrating how bayesian p-values should be used for model checking?

  • 2
    $\begingroup$ Please avoid asking multiple questions at once. You seem to be asking someone to write a full tutorial on using Bayesian p-values, rather then asking a specific question. $\endgroup$ – Tim Jun 8 at 19:27
  • 1
    $\begingroup$ Great, that's much more clear question! Please consider however that previously you seem to have asked completely different question, that already got answered. The unwritten etiquette of this site is that we do not change the questions that already got answered, because people already made the effort trying to answer them. This basically discourages people from answering your questions and questions in general. I would suggest rather making the current version of the question a new question and keeping this the "How do I judge whether a "Bayesian p-value" is 'small enough'?" question. $\endgroup$ – Tim Jun 8 at 19:39
  • $\begingroup$ @Tim I already asked the other question on this: stats.stackexchange.com/questions/471085/… maybe Demetri can just move his answer to that one? Although this answer is not really answering anything, as I said, he is just saying "it depends". $\endgroup$ – user272422 Jun 8 at 19:40
  • $\begingroup$ Just to further clarify that Demetri's answer was not answering the original question: the original question asked "How do I judge"; Demetri answered with "it depends." Clearly, "it depends" does not explain "how." $\endgroup$ – user272422 Jun 8 at 19:53

I've seen you post some questions pretty frequently looking for guidelines (e.g. how do I know when this is good enough). That's great and an excellent attitude, but I think you're going to be frustrated with a lot of Bayesian Stats because the answer will more often than not be "it depends".

And this answer is no different. The posterior probability that you observe data at least as extreme as some value is not a hard and fast rule as it is in Frequentism. In my own experience, I don't use this quantity in isolation. I instead loop in subject matter experts and ask them targeted questions. For example, I might ask "how many patients out of 100 might have a max concentration: larger than $C$? The subject matter expert will give me some approximation which I can then use to check my models. But the important part here is that I could not know that number a priori, and so there is no "small enough" for every situation.

It heavily depends on what you're doing and what is important to you.

| cite | improve this answer | |
  • $\begingroup$ Thanks Demetri, but I'm not asking for a number that a priori will serve for all purposes. If you want to illustrate with context how you decided 5% was small, please provide the contextual information you used. $\endgroup$ – user272422 Jun 8 at 19:09
  • $\begingroup$ The problem of this type of answer is that you can answer anything like this... "Is this model for COVID-19 good? Well it depends on what you want to do with it." Yes, everyone know it depends. But what does it depend on? Can you show how you would use Bayesian p-values in a real (or at least in a toy) problem? $\endgroup$ – user272422 Jun 8 at 19:10
  • 1
    $\begingroup$ @BayesianNewbie please avoid lengthy discussions in comments. Comments are not meant for discussions. $\endgroup$ – Tim Jun 8 at 19:25
  • 3
    $\begingroup$ @BayesianNewbie under many answers to your questions you seem to be asking many follow-up questions in comments, this may suggest that you are not making your question specific enough, what makes it hard for people to give you the kind of answers that you'd expect. $\endgroup$ – Tim Jun 8 at 19:29
  • 2
    $\begingroup$ @BayesianNewbie I'm not really sue what you want out of an answer. "Show me how you do it" is not really specific. I'm happy to edit my answer and invest my time when you edit your question to unambiguously say what you would expect and what you would accept as an answer. $\endgroup$ – Demetri Pananos Jun 8 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy