Why should I be Bayesian when my dataset is large?

Question

From "Why should I be Bayesian when my model is wrong?", one of the key benefits of Bayesian inference to be able to inject exogenous domain knowledge into the model, in the form of a prior. This is especially useful when you don't have enough observed data to make good predictions.

However, the prior's influence diminishes (to zero?) as the dataset grows larger. So if you have enough data, the prior provides very little value.

What's the benefit of using Bayesian analysis in this case?

Maybe that we still get a posterior distribution over parameter values? (But for large enough data, wouldn't the posterior just collapse to the MLE?)

Answer 1

1. Being Bayesian is not only about information fed through the prior. But even then: Where the prior is zero, no amount of data will turn that over.

2. Having a full Bayesian posterior distribution to draw from opens loads and loads of ways to make inference from.

3. It is easy to explain a credible interval to any audience whilst you know that most audiences have a very vague understanding of what a confidence interval is.

4. Andrew Gelman said in one of his youtube videos, that $p$ is always slightly lower then $0.05$ because if it wasn't smaller then we would not read about it and if it was much smaller they'd examine subgroups. While that is not an absolute truth, indeed when you have large data you will be tempted to investigate defined subgroups ("is it still true when we only investigate caucasian single women under 30?") and that tends to shrink even large data quite a lot.

5. $p$-values tend to get worthless with large data as in real life no null hypthesis holds true in large data sets. It is part of the tradition about $p$ values that we keep the acceptable alpha error at $.05$ even in huge datasets where there is absolutely no need for such a large margin of error. Baysian analysis is not limited to point hyptheses and can find that the data is in a region of practical equivalence to a null hypotheses, a Baysian factor can grow your believe in some sort of null hypothesis equivalent where a $p$ value can only accumulate evidence against it. Could you find ways to emulate that via confidence intervals and other Frequentist methods? Probably yes, but Bayes comes with that approach as the standard.

6. "But for large enough data, wouldn't the posterior just collapse to the MLE" - what if a posterior was bimodal or if two predictors are correlated so you could have different combinations of e.g. $\beta_8$ and $\beta_9$ - a posterior can represent these different combinations, an MLE point estimator does not.

Answer 2

The other answers address what's probably your actual question. But just to add a more concrete viewpoint: if you're already a Bayesian (for small/medium datasets) and you get a large data, why not use the methodology you're familiar with? It will be relatively slow but you are familiar with the steps so you're less likely to make mistakes and you're more likely to spot problems. And a Bayesian workflow includes things like posterior predictive checks, etc, which are useful for understanding your model.

Answer 3

One place where Bayesian approach meets large datasets is Bayesian deep learning. When using Bayesian approach to neural networks people usually use rather simplistic priors (Gaussians, centered at zero), this is mostly for computational reasons, but also because there is not much prior knowledge (neural network parameters are black-boxish). The reason why Bayesian approach is used, is because out-of-the-box it gives us uncertainty estimates.

Answer 4

I'd like to echo some of the points in the other answer with slightly different emphasis.

To me the most important issue is that the Bayesian view of uncertainty/probability/randomness is the one that directly answers the questions we probably care about, whereas the Frequentist view of uncertainty directly answers other questions that are often somewhat besides the point. Bayesian inferences try to tell us what we (or an algorithm, machine, etc.) should believe given the data we have seen, or in other words "what can I learn about the world from this data?" Frequentist inferences try to tell us how different our results would be if the data that we actually saw were "re-generated" or "repeatedly sampled" an infinite number of times. Personally I sometimes find Frequentist questions interesting, but I can't think of a scenario where the Bayesian questions aren't what matter most (since at the end of the day I want to make a decision about what to believe or do now that I've seen new data). It's worth noting that often people (statisticians included) incorrectly interpret Frequentist analyses as answering Bayesian questions, probably betraying their actual interests. And while people get worried about the subjectivity inherent in Bayesian methods, I think of the Tukey line, "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise." For what it's worth, Frequentist methods are also subjective, and arguably in ways that are less obvious and convenient to critique.

Getting off my Bayesian high horse, you're right that answers to Frequentist questions (especially MLE) sometimes coincide closely (and in rare cases, exactly) with answers to Bayesian questions.

However, large data is a vague notion in a few senses that can make Bayesian and Frequentist (MLE) answers remain different:

1. Most results about large data are asymptotic as the sample size goes to infinity, meaning that they don't tell us when our sample size is actually large enough for the asymptotic result to be accurate enough (up to some known level of error). If you go through the trouble to do both Bayesian and Frequentist analyses of your data and find they're numerically similar then it doesn't matter so much.
2. Often with "large" data (e.g. many observations) we also have a large number of questions or parameters of interest. This is basically Bernhard's point #4.
3. A lot of large data sets are not perfectly designed and relate to our interests indirectly because of issues like measurement error or sampling bias. Treated honestly, these complications may not go away even asymptotically, meaning that the models that realistically relate the data to what we care about have non-identifiable sensitivity parameters that are most natural to deal with using priors and the Bayesian machinery.

Of course, the flip-side of this question is "Why should I be Frequentist when my dataset is large?"