2
$\begingroup$

For example for this problem:

You have a coin that when flipped ends up head with probability p and ends up tail with probability 1−p. (The value of p is unknown.)

Trying to estimate p, you flip the coin 14 times. It ends up head 10 times.

Then you have to decide on the following event: "In the next two tosses we will get two heads in a row."

Would you bet that the event will happen or that it will not happen?

(http://www.behind-the-enemy-lines.com/2008/01/are-you-bayesian-or-frequentist-or.html)

Could one flip a bunch of different coins a million times and see what happens? (Or write a program to do so?)

EDIT: Some people were asking, I think, how probability theory could possibly be relevant when empirical data was available. For the above problem, I wrote a script which would flip a coin with a 1% chance of landing heads, a 2% coin, a 3% coin, etc. fourteen times; and if it came up heads ten out of those fourteen times, then it would flip the coin twice more and see whether it came up heads both times. This "experiment" is repeated a bunch of times and the percent of the time you get two heads, given that you've gotten ten out of fourteen heads, is around 48%, which is exactly the Bayesian answer given on the website above.

I tried not to make any assumptions about the nature of probability and just run the experiment the way it would happen "in the wild", if it were possible to run a million trials of this kind in real life, but I'm slightly worried that I made some inherently Bayesian assumption in the writing of the code.

Thoughts?

Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ if we knew what reality was there would be no need for a model, frequentist or bayesian. $\endgroup$
    – bdeonovic
    May 18, 2015 at 17:25
  • 9
    $\begingroup$ Why would an empirical test be needed? All statistical procedures can be evaluated with perfect accuracy using mathematical reasoning. Any empirical test would only be a test of whether various underlying assumptions may be true, not of the statistical procedures. $\endgroup$
    – whuber ♦
    May 18, 2015 at 19:22
  • $\begingroup$ Here is an author who wrote Python scripts to do simulations, in order to show the difference between frequentist and bayesian methods. $\endgroup$
    – Pieter Hogendoorn
    Feb 3, 2017 at 13:24

3 Answers 3

Reset to default
3
$\begingroup$

In Bayesian approach as the sample size grows the influence of the prior diminishes. With an infinite sample you'll get the same results in frequentist and Bayesian approaches. With a finite sample, or small sample, there is no way to tell which approach is more accurate.

Improve this answer
$\endgroup$
12
  • 5
    $\begingroup$ Counterexample: the parameter of interest $y$ is a mean. The prior is uniform on the positive reals but the true value of $y$ is negative. The Bayesian solution will never coincide with the frequentist estimate. $\endgroup$
    – Sycorax ♦
    May 18, 2015 at 17:37
  • 1
    $\begingroup$ @user777, should we can start a thread on dangers of improper priors? :) $\endgroup$
    – Aksakal
    May 18, 2015 at 18:16
  • 2
    $\begingroup$ @user777 - not a fair comparison. The parameter space for the frequentist estimator should be $(0, \infty)$ if the investigator is absolutely certain that the true parameter is $ > 0$, and of course both frequentist and Bayesian estimates would go to 0 from above as the sample size increases. After all, we don't treat the mean of the Poisson as if negative values were possible in frequentist analysis (see any reporting of confidence intervals on same.) $\endgroup$
    – jbowman
    May 18, 2015 at 19:57
  • 1
    $\begingroup$ @Aksakal This isn't an issue with improper priors. Some gamma prior has the same "problem," but is itself proper and strictly positive. I point this out to illustrate that there are some assumptions underlying your post which are not explicit. $\endgroup$
    – Sycorax ♦
    May 18, 2015 at 20:22
  • 1
    $\begingroup$ Isn't the Frequentist "approach" just the Bayesian approach with an unstated prior? Why present them as two different approaches? Why not just say the Bayesianism makes the prior explicit? $\endgroup$
    – Neil G
    Jun 6, 2015 at 8:30
3
$\begingroup$

In comparing Bayesian and frequentist methods note that the frequentist approach is indirect and doesn't really give an "answer" in the same sense as a direct probability model such as a Bayesian model. Notes above have concentrated on a point estimate of the population mean, which is fine; also look at inferential quantities.

When the person viewing the results is using the prior that was used in the calculations, the Bayesian results have to be right.

See http://www.fharrell.com/2017/01/p-values-and-type-i-errors-are-not.html for more about problems caused by the indirectness of the frequentist approach.

Improve this answer
$\endgroup$
0
$\begingroup$

sometimes it is just imposible to do that much experiments, unlike the example of flipping a coin , it could be some medical issues, which is unaffordable to be repeated for many times, in this case frequencist based methods not work, the only choice is Bayesian

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.