2
$\begingroup$

In this article from D. Berry https://www.jstor.org/stable/2684222?seq=1#page_scan_tab_contents

the author uses an example to introduce some limits of p-values in frequentists analyses.

He takes an example where two different sampling schemes lead to different inferences even if the same data are collected:

  1. A sampling of 10 Bernoulli, resulting in 8 sucesses and 2 failures
  2. A sampling of Bernoulli until 8 sucesses are obtained, resulting in 10 draws with 8 sucesses and 2 failures

Letting $\theta$ be the probability of success the likelihood are the same for both experiments : $$\theta^8(1-\theta)^2$$

When testing $\theta = \frac{1}{2}$ versus $\theta > \frac{1}{2}$ those sampling schemes lead to different p-values. Since p-values can be stated as $P(T(X) \ \text{"more extreme than"} \ T(X_{obs}) \mid H_0)$ this lead to different definitions of "more extreme observations" base on the sampling scheme used and to different p-values.

However, even if this shows a limit of usual frenquentist analyses regarding the Likelihood principle my question is: are Bayesian analyses still relevant under the second sampling scheme? It seems to me that those observations are not exchangeable in this case since observing 8 sucesses in a row will prevent from observing the 2 remaining values.

$\endgroup$
1
  • 2
    $\begingroup$ In the second case, the number of failures is the random variable, so you can observe $0, 1, 2, \dots$. If you observe $2$, this is what goes into the likelihood function that is used in the Bayesian analysis. If you observe $0$, this is what goes into the likelihood function that is used in the Bayesian analysis. In either case, or any of the others, you can still do the Bayesian analysis. $\endgroup$ – jbowman Mar 23 '19 at 3:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.