8
$\begingroup$

Can you help me understand the frequentist point of view in the bayesian vs frequentist debate? I have read a lot and all the sources I found are either filled with complex equations or written from a bayesian point of view, or both. I have not found a single sample problem where the frequentist approach would produce more useful output than the bayesian approach. I feel like I only understand one side of this debate and I would like to understand the other side as well. I do not have any background in statistics, so I would appreciate simple examples of cases where frequentist methods produce more value than bayesian methods.

A nice example would be a betting scenario where a frequentist and bayesian bet against each other about some future outcome and the frequentist has positive expected value.

$\endgroup$
11
  • 3
    $\begingroup$ Surely you could find a few tens of thousands of such examples just by browsing through this site. In light of this, just what kind of answers are you looking for? $\endgroup$
    – whuber
    Jan 26, 2016 at 17:31
  • 7
    $\begingroup$ After 2 hours of googling I found 0 examples where the frequentist approach is more useful than the Bayesian. If you have 10 000 examples, can you provide 1 of them? Thanks. $\endgroup$
    – bkoodaa
    Jan 26, 2016 at 17:35
  • 2
    $\begingroup$ I do not know if this is at thew level you would like, but you can find a relevant discussion in L. Wasserman's book which is avaliable online too. read.pudn.com/downloads158/ebook/702714/… . If you go to page 216, you will find an example concerning confidence intervals where the frequentist approach outperforms the Bayesian. $\endgroup$
    – JohnK
    Jan 26, 2016 at 18:05
  • 2
    $\begingroup$ @whuber: I don't believe your definition of "useful" will differ from mine in a way where it's meaningful to discuss about it. I am not here to infer that bayesian > frequentist. I have very recently learned about these subjects and I feel like I only understand one side of the debate. I would like to understand the other side as well. I find it easiest to grasp new concepts via practical examples; in this case an example where frequentism provides something of value (where bayesian methods fall short) $\endgroup$
    – bkoodaa
    Jan 26, 2016 at 18:44
  • 3
    $\begingroup$ I vote to reopen. @whuber, the fact that 20k+ people came here to ask a question about frequentist techniques and got a useful answer does not imply that frequentist techniques were more appropriate than Bayesian ones in those specific cases; it just means that they are widespread. $\endgroup$
    – amoeba
    Jan 26, 2016 at 22:03

1 Answer 1

6
$\begingroup$

A nice example would be a betting scenario where a frequentist and Bayesian bet against each other about some future outcome and the frequentist has positive expected value.

I will not give you this example because such an example would favor a Bayesian approach unless the Bayesian chooses a bad prior which is a cop-out example not really worth writing about.

The frequentest approach is not designed to obtain the highest expected value in betting scenarios (luckily the world of statistics and probability is much more broad than just that). Rather, frequentist techniques are designed to guarantee certain desirable frequency properties, particularly that of coverage. These properties are important for parameter estimation and inference in the context of scientific research and inquiry.

I encourage you to check out this link here to a blog post by Dr. Larry Wasserman. In it he talks about frequency guarantees in more depth (see the examples he gives).

Suppose we had some data $Y$ and we think it is distributed according to some conditional distribution $Y \sim f(Y|\theta^*)$ (if you like you can imagine that $Y$ is normally distributed and $\theta^*$ is the mean and\or variance). We do not know the value of $\theta^*$ , so we have to estimate it. We can use either a frequentist or Bayesian approach to do so.

In the frequentist approach we would obtain a point estimate $\hat \theta$ and a confidence interval for that estimate. Assuming $\theta^*$ exists and the model is valid and well behaved, the frequentist $(1-\alpha)$ confidence interval is guaranteed to contain $\theta^*$ $(1-\alpha)$% of the time regardless of what $\theta^*$ actually is. $\theta^*$ could be 0, it could be 1,000,000, it could be -53.2, it doesn't matter, the above statement holds true.

However, the above does not hold true for Bayesian confidence intervals otherwise known as credible intervals. This is because,in a Bayesian setting, we have to specify a prior $\theta \sim \pi(\theta)$ and simulate from the posterior, $\pi(\theta|Y) \propto f(Y|\theta)\pi(\theta)$. We can form $(1-\alpha)$% credible intervals using the resulting sample, but the probability that these intervals will contain $\theta^*$ depends upon how probable $\theta^*$ is under our prior.

In a betting scenario, we may believe that certain values are less likely to be $\theta^*$ then others, and we can assign a prior to reflect these beliefs. If our beliefs are accurate the probability of containing $\theta^*$ in the credible interval is higher. This is why smart people using Bayesian techniques in betting scenarios beat frequentist.

But consider a different scenario, like a study where you are testing the effect of education on wages, call it $\beta$, in a regression model. A lot of researchers would prefer the confidence interval of $\beta$ to have the frequency property of coverage rather than reflect their own degrees of belief regarding the effect education on wages.

From a pragmatic standpoint, it should also be noted that in my earlier example, as the sample size approaches infinity, both the frequentist $\hat \theta$ and Bayesian posterior $\pi(\theta|Y)$ converge onto $\theta^*$. So as you obtain more and more data, the difference between the Bayesian and frequentist approach becomes negligible. Since Bayesian estimation is often (not always) more computationally and mathematically rigorous than frequentist estimation, practitioners often opt for frequentist techniques when they have "large" data sets. This is true even when the primary goal is prediction as opposed to parameter estimation/inference.

$\endgroup$
7
  • $\begingroup$ +1 but regarding your regression example (testing the effect of education on wages), while I agree that "a lot of researches" (myself included!) do prefer to use frequentist procedures, there are many people, statisticians included, saying that this whole approach is misguided and does not work properly or even as intended. This is not a place to debate it, but it should be mentioned that this point of view exists too. $\endgroup$
    – amoeba
    Jan 26, 2016 at 21:56
  • $\begingroup$ @amoeba, pretty much all of those arguments are not about properly used frequentist approaches per se but about overuse, misuse, and misunderstanding of them. $\endgroup$
    – John
    Feb 11, 2016 at 3:50
  • $\begingroup$ Zachary, as this thread is closed, would you mind or perhaps prefer if your answer were moved in the stats.stackexchange.com/questions/194035 ? This can be done if this thread is "merged" into that one (i.e. closed as duplicate and all answers are moved). I think this could be helpful. $\endgroup$
    – amoeba
    Feb 11, 2016 at 10:31
  • $\begingroup$ @amoeba sure, if you think that would be helpful. $\endgroup$ Feb 11, 2016 at 10:47
  • 1
    $\begingroup$ "I will not give you this example because such an example would favor a Bayesian approach unless the Bayesian chooses a bad prior which is a cop-out example not really worth writing about." I strongly disagree with this. This is the fundamental reason for considering frequentist statistics in the first place: good priors are hard to come by. Bayesian results are trivially better with a good prior, but a obtaining a good prior is very non-trivial. $\endgroup$
    – Cliff AB
    Oct 20, 2016 at 21:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.