In Kruschke's book, in chapter 11, he gives an example of testing whether a coin is biased. He shows how, if one conditions on $N$ (the number of flips), $z$ (the number of heads), or on the duration ($N$ is then given a Poisson distribution), then we get different sampling distributions, and consequently, different decisions, confidence intervals, etc. The reason given for why it's OK in a classical frequentist context to do this type of conditioning is because,

... will result in exactly 5% false alarms in the long-run when the null hypothesis is true.

Why is it allowed? What's the 'math' behind this statement?


The Bayesian interpretation of data does not depend on the covert sampling and testing intentions of the data collector. ...The likelihood function captures everything we assume to influence data.

Hence, why can't we condition in a similar way the Bayesian analysis and check for different posterior distributions, while using the exact same priors? Why isn't the likelihood altered by different 'generating sample procedures', but the sampling distribution is?


  • 2
    $\begingroup$ It's not clear to me what you're asking, especially your first question. If you condition on $N$, your random variable is going to be $z$, and if you condition on $z$, your random variable is either going to be $N$ or $N-z$. Why would you think the sampling distributions etc. would be the same when the r.vs themselves are clearly different? As for your second question, try writing out the probability distributions for the cases you mention above and convert them to likelihood functions. You will see they are indeed the same. $\endgroup$ – jbowman Jul 26 '17 at 19:43
  • $\begingroup$ @jbowman But isn't the $l(\theta|D)=p(D|\theta)$? Why with a frequentist perspective would you be allowed to condition on some variables, but when doing a Bayesian analysis, the same conditioning would not influence $l(\theta|D)$? For example, when I condition on N (with a discrete dist.), then $p(D|\theta)=\sum_N p(D|N,\theta) p(N|\theta)$ which is a mixture of binomials (coin ->bernoulli)... $\endgroup$ – An old man in the sea. Jul 26 '17 at 20:27
  • 2
    $\begingroup$ I'm not sure exactly what you're asking either, but here are some thoughts: Both frequentist and Bayesian use the same likelihood function (in whatever model is assumed). The likelihood function is the probability of single observations, not samples of multiple data. But frequentist approaches additionally generate a probability distribution of a descriptive statistic that summarizes a sample of data, and that sampling distribution is created by assumption about how samples of data are generated. Are samples created with fixed N? With fixed z? With fixed duration? Something else? $\endgroup$ – John K. Kruschke Jul 26 '17 at 21:15
  • 1
    $\begingroup$ The likelihood function is not the same as the probability distribution (an oft-misunderstood point) as the former is a function of the parameter conditional upon the data and the latter is a function of the data conditional upon the parameter. In the latter case, how you generate that data matters, but in the former, since we're conditioning upon the data, how it was generated doesn't matter any more (except insofar as the likelihood function itself might change.) How it was generated is on the other side of the "wall of conditioning", so to speak, and is "invisible" in some sense. $\endgroup$ – jbowman Jul 26 '17 at 22:51
  • $\begingroup$ @JohnK.Kruschke My first question is for a more complete justification of why it's allowed to condition, from a frequentist perspective. My 2nd question is why the data generating process influences the sampling distribution, but not the likelihood function. I've edited the question. I hope it's clearer. Thanks ;) $\endgroup$ – An old man in the sea. Jul 27 '17 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.