A question on Bayesian credible interval vs frequentist confidence interval

Question

The difference of Bayesian credible interval (BCI) and the frequentist confidence interval (FCI) is well explained with a nice example in this answer. Here is my own summary of the situation in the example. There are four kinds of jars. All jars contain the same number of cookies. Each cookie contains some chocolate chips. Each kind of jar has a fixed distribution of the number of chips in a cookie but different kinds of jars have different distributions. Now I have a jar on my desk delivered by my friend but I don't know what kind of jar it is. I draw a cookie from the jar and count the number of chips in it. Now I want to infer what kind of jar I have. I want the inference to be right in 70% of the case. BCI and FCI are different in this example as explained in the answer.

Let's modify the situation. Now I don't have a jar on my desk currently. Tomorrow my friend will pick up a jar and bring it to me. So the kind of jar is not an unknown fixed value. It is not fixed yet. To Bayesians, the modified situation is actually equivalent to the original situation. We just have the same information on the jar. However, for frequentists the two situations are different each other. In the original example, I think that the ensemble of virtual experiments of a frequetist is a repetition of the drwaing a cookie from the jar on the desk. In the modified example, I think that the virtual experiment must also contain my friend's picking up a jar. So in this case, the frequentist must "assume" a model of the pick up procedure and this amounts to a Bayesian's prior in the original example. So finally, in the modified example, BCI and FCI are the same and it is identical with the BCI of the original example.

Am I right? Or did I misunderstand something?

Answer 1

I think that the crucial difference between BCI and FCI is that Bayesians calculate the probability conditional on the observed number of chips, i.e., they calculate the posterior probability and find the interval while frequentists don't. Let's repeat your modified experiment many times. For a frequentist the 70% is the ratio of the number of right answer she made over the number of all repetitions. For a Bayesian, the 70% is the ratio of the number of right answer he made over the observed number of each possible number of chips.

Let your friend choose a jar randomly with the uniform probability. In the modified case, a frequentist reads the third table given in the answer you linked. Then she makes some numbers bold so that the ratio of the sum of the bold numbers over the sum of all numbers is to be 0.7 (while in the original case, she reads each columns independently as explained in the answer). A Bayesian with uniform prior (or the uniform pick up process is known) reads each row of the third table independently. Then, for each row, he makes some numbers bold so that the ratio of the sum of the bold numbers over the sum of the numbers in the row is to be 0.7.

Then the FCI of the modified ensemble of experiments is still the same one as the original case. The relative frequency of each possible event (jar type, number of chips) is the same with the original one and so nothing changes.