When (if ever) is a frequentist approach substantively better than a Bayesian?

Question

Background: I do not have an formal training in Bayesian statistics (though I am very interested in learning more), but I know enough--I think--to get the gist of why many feel as though they are preferable to Frequentist statistics. Even the undergraduates in the introductory statistics (in social sciences) class I am teaching find the Bayesian approach appealing--"Why are we interested in calculating the probability of the data, given the null? Why can't we just quantify the probability of the null hypothesis? Or the alternative hypothesis? And I've also read threads like these, which attest to the empirical benefits of Bayesian statistics as well. But then I came across this quote by Blasco (2001; emphasis added):

If the animal breeder is not interested in the philosophical problems associated with induction, but in tools to solve problems, both Bayesian and frequentist schools of inference are well established and it is not necessary to justify why one or the other school is preferred. Neither of them now has operational difficulties, with the exception of some complex cases...To choose one school or the other should be related to whether there are solutions in one school that the other does not offer, to how easily the problems are solved, and to how comfortable the scientist feels with the particular way of expression results.

The Question: The Blasco quote seems to suggest that there might be times when a Frequentist approach is actually preferable to a Bayesian one. And so I am curious: when would a frequentist approach be preferable over a Bayesian approach? I'm interested in answers that tackle the question both conceptually (i.e., when is knowing the probability of the data conditioned on the null hypothesis especially useful?) and empirically (i.e., under what conditions do Frequentist methods excel vs. Bayesian?).

It would also be preferable if answers were conveyed as accessibly as possible--it would be nice to take some responses back to my class to share with my students (though I understand some level of technicality is required).

Finally, despite being a regular user of Frequentist statistics, I am actually open to the possibility that Bayesian just wins across the board.

Answer 1

Here's five reasons why frequentists methods may be preferred:

• Faster. Given that Bayesian statistics often give nearly identical answers to frequentist answers (and when they don't, it's not 100% clear that Bayesian is always the way to go), the fact that frequentist statistics can be obtained often several orders of magnitude faster is a strong argument. Likewise, frequentist methods do not require as much memory to store the results. While these things may seem somewhat trivial, especially with smaller datasets, the fact that Bayesian and Frequentist typically agree in results (especially if you have lots of informative data) means that if you are going to care, you may start caring about the less important things. And of course, if you live in the big data world, these are not trivial at all.

• Non-parametric statistics. I recognize that Bayesian statistics does have non-parametric statistics, but I would argue that the frequentist side of the field has some truly undeniably practical tools, such as the Empirical Distribution Function. No method in the world will ever replace the EDF, nor the Kaplan Meier curves, etc. (although clearly that's not to say those methods are the end of an analysis).

• Less diagnostics. MCMC methods, the most common method for fitting Bayesian models, typically require more work by the user than their frequentist counter part. Usually, the diagnostic for an MLE estimate is so simple that any good algorithm implementation will do it automatically (although that's not to say every available implementation is good...). As such, frequentist algorithmic diagnostics is typically "make sure there's no red text when fitting the model". Given that all statisticians have limited bandwidth, this frees up more time to ask questions like "is my data really approximately normal?" or "are these hazards really proportional?", etc.

• Valid inference under model misspecification. We've all heard that "All models are wrong but some are useful", but different areas of research take this more or less seriously. The Frequentist literature is full of methods for fixing up inference when the model is misspecified: bootstrap estimator, cross-validation, sandwich estimator (link also discusses general MLE inference under model misspecification), generalized estimation equations (GEE's), quasi-likelihood methods, etc. As far as I know, there is very little in the Bayesian literature about inference under model misspecification (although there's a lot of discussion of model checking, i.e., posterior predictive checks). I don't think this just by chance: evaluating how an estimator behaves over repeated trials does not require the estimator to be based on a "true" model, but using Bayes theorem does!

• Freedom from the prior (this is probably the most common reason for why people don't use Bayesian methods for everything). The strength of the Bayesian standpoint is often touted as the use of priors. However, in all of the applied fields I have worked in, the idea of an informative prior in the analysis is not considered. Reading literature on how to elicit priors from non-statistical experts gives good reasoning for this; I've read papers that say things like (cruel straw-man like paraphrasing my own) "Ask the researcher who hired you because they have trouble understanding statistics to give a range that they are 90% certain the effect size they have trouble imagining will be in. This range will typically be too narrow, so arbitrarily try to get them to widen it a little. Ask them if their belief looks like a gamma distribution. You will probably have to draw a gamma distribution for them, and show how it can have heavy tails if the shape parameter is small. This will also involve explaining what a PDF is to them."(note: I don't think even statisticians are really able to accurately say a priori whether they are 90% or 95% certain whether the effect size lies in a range, and this difference can have a substantial effect on the analysis!). Truth be told, I'm being quite unkind and there may be situations where eliciting a prior may be a little more straightforward. But you can see how this is a can of worms. Even if you switch to non-informative priors, it can still be a problem; when transforming parameters, what are easily mistaken for non-informative priors suddenly can be seen as very informative! Another example of this is that I've talked with several researchers who adamantly do not want to hear what another expert's interpretation of the data is because empirically, the other experts tend to be over confident. They'd rather just know what can be inferred from the other expert's data and then come to their own conclusion. I can't recall where I heard it, but somewhere I read the phrase "if you're a Bayesian, you want everyone to be a Frequentist". I interpret that to mean that theoretically, if you're a Bayesian and someone describes their analysis results, you should first try to remove the influence of their prior and then figure out what the impact would be if you had used your own. This little exercise would be simplified if they had given you a confidence interval rather than a credible interval!

Of course, if you abandon informative priors, there is still utility in Bayesian analyses. Personally, this where I believe their highest utility lies; there are some problems that are extremely hard to get any answer from in using MLE methods but can be solved quite easily with MCMC. But my view on this being Bayesian's highest utility is due to strong priors on my part, so take it with a grain of salt.

Answer 2

A few concrete advantages of frequentist statistics:

• There are often closed-form solutions to frequentist problems whereas you would need a conjugate prior to have a closed form solution in the Bayesian analogue. This is useful for a number of reasons - one of which is computation time.
• A reason that'll, hopefully, eventually go away: laymen are taught frequentists statistics. If you want to be understood by many, you need to speak frequentist.
• An "Innocent until proven guilty" Null Hypothesis Significance Testing (NHST) approach is useful when the goal is to prove someone wrong (I'm going to assume your right and show the data overwhelming suggests you're wrong). Yes, there are NHST analogues in Bayesian but I find the frequentists versions much more straight-forward and interpretable.
• There is no such thing as a truly uninformative prior which makes some people uncomfortable.

Answer 3

The most important reason to use Frequentist approaches, which has surprisingly not yet been mentioned, is error control. Very often, research leads to dichotomous interpretations (should I do a study building on this, or not? Should be implement an intervention, or not?). Frequentist approaches allow you to strictly control your Type 1 error rate. Bayesian approaches don't (although some inherit the universal bound from likelihood approaches, but even then, error rates can be quite high in small samples and with relatively low thresholds of evidence (e.g., BF > 3). You can examine Frequentist properties of Bayes factors (see for example http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2604513) but that's still a Frequentist approach. I think very often, researchers care more about error control than about quantifying evidence per se (relative to some specific hypothesis), and I think at the very least, everyone cares about error control to some extent, and thus the two approaches should be used complementarily.

Answer 4

I think one of the biggest questions, as a statistican, you have to ask yourself is whether or not you believe in, or want to adhere to, the likelihood principle. If you don't believe in the likelihood principle then I think the frequentist paradigm to statistics can be extremely powerful, however, if you do believe in the likelihood principle, then (I believe) you most certainly have to espouse the Bayesian paradigm in or to not violate it.

---

In case you are unfamiliar with it, what the likelihood principle tells us is the following:

The Likelihood Principle: In making inferences or decisions about $\theta$ after some data $\mathbf{x}$ is observed, all relevant experimental information is contained in the likelihood function: $$\ell(\theta;\mathbf{x})=p(\mathbf{x}|\theta)$$ where $\mathbf{x}$ corresponds to the data observed and is thus fixed.

Furthermore, if $\mathbf{x}$ and $\mathbf{y}$ are two sample points such that $\ell(\theta;\mathbf{x})$ is proportional to $\ell(\theta;\mathbf{y})$, that is, there exists a constant $C(\mathbf{x},\mathbf{y})$ such that

$$\ell(\theta;\mathbf{x})=C(\mathbf{x},\mathbf{y})\ell(\theta;\mathbf{y})\hspace{.1in}\text{for all }\theta,$$

then the conclusions drawn from $\mathbf{x}$ and $\mathbf{y}$ should be identical.\

Note that the constant $C(\mathbf{x},\mathbf{y})$ above may be different for different $(\mathbf{x},\mathbf{y})$ pairs but $C(\mathbf{x},\mathbf{y})$ does not depend on $\theta$.

In the special case of $C(\mathbf{x},\mathbf{y})=1$, the Likelihood Principle states that if two sample points result in the same likelihood function, then they contain the same information about $\theta$. But the Likelihood Principle goes further. It states that even if two sample points have only proportional likelihoods, then they contain equivalent information about $\theta$.

---

Now, one of the draws of Bayesian statistics is that, under proper priors, the Bayesian paradigm never violates the likelihood principle. However, there are very simple scenarios where the frequentist paradigm will violate the likelihood principle.

Here is a very simple example based on hypothesis testing. Consider the following:

Consider an experiment where 12 Bernoulli trials were run and 3 successes were observed. Depending on the stopping rule we could characterize the data as the following:

• Binomial Distribution: $X|\theta\sim\text{Bin}(n=12,\theta)$ and Data: $x=3$
• Negative Binomial Distribution: $Y|\theta\sim\text{NegBin}(k=3,\theta)$ and Data: $y=12$

And thus we would obtain the following likelihood functions: \begin{align} \ell_1(\theta;x=3)&=\binom{12}{3}\theta^3(1-\theta)^9\\ \ell_2(\theta;y=12)&=\binom{11}{2}\theta^3(1-\theta)^9\\ \end{align} which implies that $$\ell_1(\theta;x)=C(x,y)\ell_2(\theta,y)$$ and thus, by the Likelihood Principle, we should obtain the same inferences about $\theta$ from either likelihood.

Now, imagine testing the following hypotheses from the frequentist paradigm $$H_o:\theta\geq\frac{1}{2}\hspace{.2in}\text{versus}\hspace{.2in}H_a:\theta<\frac{1}{2}$$

For the Binomial model we have the following: \begin{align} \text{p-value}&=P\left(X\leq 3|\theta=\frac{1}{2}\right)\\ &=\binom{12}{0}\left(\frac{1}{2}\right)^{12}+\binom{12}{1} \left(\frac{1}{2}\right)^{12}+ \binom{12}{2}\left(\frac{1}{2}\right)^{12}+\binom{12}{3}\left(\frac{1}{2}\right)^{12}=0.0723 \end{align}

Notice that $\binom{12}{3}\left(\frac{1}{2}\right)^{12}=\ell_1(\frac{1}{2};x=3)$ but the other terms do not satisfy the likelihood principle.

For the Negative Binomial model we have the following: \begin{align} \text{p-value}&=P\left(Y\geq 12|\theta\frac{1}{2}\right)\\ &=\binom{11}{2}\left(\frac{1}{2}\right)^{12}+\binom{12}{2}\left(\frac{1}{2}\right)^{12}+ \binom{13}{2}\left(\frac{1}{2}\right)^{12}+...=0.0375 \end{align}

From the above p-value calculations we see that in the Binomial model we would fail to reject $H_o$ but using the Negative Binomial model we would reject $H_o$. Thus, even though $\ell_1(\theta;x)\propto\ell_2(\theta;y)$ there p-values, and decisions based on these p-values, do not coincide. This p-value argument is one often used by Bayesians against the use of Frequentist p-values.

Now consider again testing the following hypotheses, but from the Bayesian paradigm $$H_o:\theta\geq\frac{1}{2}\hspace{.2in}\text{versus}\hspace{.2in}H_a:\theta<\frac{1}{2}$$

For the Binomial model we have the following: \begin{align} P\left(\theta\geq\frac{1}{2}|x\right)=\int_{1/2}^1\pi(\theta|x)dx=\int_{1/2}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \bigg/\int_{0}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \end{align}

Similarly, for the Negative Binomial model we have the following: \begin{align} P\left(\theta\geq\frac{1}{2}|y\right)=\int_{1/2}^1\pi(\theta|x)dx=\int_{1/2}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \bigg/\int_{0}^1\theta^3(1-\theta)^9\pi(\theta)d\theta \end{align}

Now using Bayesian decision rules, pick $H_o$ if $P(\theta\geq\frac{1}{2}|x)>\frac{1}{2}$ (or some other threshold) and repeat similarly for $y$.

However, $P\left(\theta\geq\frac{1}{2}|x\right)=P\left(\theta\geq\frac{1}{2}|y\right)$ and so we arrive at the same conclusion and thus this approach satisfies the likelihood Principle.

---

And so to conclude my ramblings, if you don't care about the likelihood principle then being frequentist is great! (If you can't tell, I'm a Bayesian :) )

Answer 5

Personally I'm having difficulty thinking of a situation where the frequentist answer would be preferred over a Bayesian one. My thinking is detailed here and in other blog articles on fharrell.com about problems with p-values and null hypothesis testing. Frequentists tend to ignore a few fundamental problems. Here is just a sample:

• Outside of the Gaussian linear model with constant variance and a few other cases, the p-values that are computed are of unknown accuracy for your dataset and model
• When the experiment is sequential or adaptive, it is often the case that a p-value can't even be computed and one can only set an overall $\alpha$ level to achieve
• Frequentists seem happy to not let the type I error go below, say, 0.05 no matter now the sample size grows
• There is no frequentist prescription for how multiplicity corrections are formed, leading to an ad hoc hodge-podge of methods

Regarding the first point, one commonly used model is the binary logistic model. Its log likelihood is very non-quadratic, and the vast majority of confidence limits and p-values computed for such models are not very accurate. Contrast that with the Bayesian logistic model, which provides exact inference.

Others have mentioned error control as a reason for using frequentist inference. I do not think this is logical, because the error to which they refer is the long-run error, envisioning a process in which thousands of statistical tests are run. A judge who said "the long run false conviction probability in my courtroom is only 0.03" should be disbarred. She is charged with having the highest probability of making the correct decision for the current defendent. On the other hand one minus the posterior probability of an effect is the probabiity of zero or backwards effect and is the error probability we actually need.

Answer 6

You and I are both scientists, and as scientists, are chiefly interested in questions of evidence. For that reason, I think Bayesian approaches, when feasible, are preferable.

Bayesian approaches answer our question: What is the strength of evidence for one hypothesis over another? Frequentist approaches, on the other hand, do not: They report only whether the data are weird given one hypothesis.

That said, Andrew Gelman, notable Bayesian, seems to espouse the use of p-values (or p-value-like graphical checks) as a check for errors in model specification. You can see an allusion to this approach in this blog post.

His approach, as I understand it, is something like a two-step process: First, he asks the Bayesian question of what is the evidence for one model over the other. Second, he asks the Frequentist question of whether the preferred model actually looks at all plausible given the data. It seems like a reasonable hybrid approach to me.

Answer 7

Many people do not seem aware of a third philosophical school: likelihoodism. AWF Edwards's book, Likelihood, is probably the best place to read up on it. Here is a short article he wrote.
Likelihoodism eschews p-values, like Bayesianism, but also eschews the Bayesian's often dubious prior. There is an intro treatment here as well.

Answer 8

One of the biggest disadvantages of frequentist approaches to model building has always been, as TrynnaDoStats notes in his first point, the challenges involved with inverting big closed-form solutions. Closed-form matrix inversion requires that the entire matrix be resident in RAM, a significant limitation on single CPU platforms with either large amounts of data or massively categorical features. Bayesian methods have been able to work around this challenge by simulating random draws from a specified prior. This has always been one of the biggest selling points of Bayesian solutions, although answers are obtained only at a significant cost in CPU.

Andrew Ainslie and Ken Train, in a paper from about 10 years ago that I have lost the reference to, compared finite mixture (which are frequentist or closed form) with Bayesian approaches to model-building and found that across a wide range of functional forms and performance metrics, the two methods delivered essentially equivalent results. Where Bayesian solutions had an edge or possessed greater flexibility were in those instances where the information was both sparse and very high-dimensional.

However, that paper was written before "divide and conquer" algorithms were developed that leverage massively parallel platforms, e.g., see Chen and Minge's paper for more about this http://dimacs.rutgers.edu/TechnicalReports/TechReports/2012/2012-01.pdf

The advent of D&C approaches has meant that, even for the hairiest, sparsest, most high dimensional problems, Bayesian approaches no longer have an advantage over frequentist methods. The two methods are at parity.

This relatively recent development is worth noting in any debate about the practical advantages or limitations of either method.

Answer 9

Several comments:

• The fundamental difference between the bayesian and frequentist statistician is that the bayesian is willing to extend the tools of probability to situations where the frequentist wouldn't.

  • More specifically, the bayesian is willing to use probability to model the uncertainty in her own mind over various parameters. To the frequentist, these parameters are scalars (albeit scalars where the statistician does not know the true value). To the Bayesian, various parameters are represented as random variables! This is extremely different. The Bayesian's uncertainty over parameters valeus is represented by a prior.

• In Bayesian statistics, the hope is that after observing data, the posterior overwhelms the prior, that the prior doesn't matter. But this often isn't the case: results may be sensitive to the choice of prior! Different Bayesians with different priors need not agree on the posterior.

A key point to keep in mind is that statements of the frequentist statistician are statements that any two Bayesians can agree on, regardless of their prior beliefs!

The frequentist does not comment on priors or posteriors, merely the likelihood.

The statements of the frequentist statistician in some sense are less ambitious, but the bolder statements of the Bayesian can significantly rely on the assignment of a prior. In situations where priors matter and where there is disagreement on priors, the more limited, conditional statements of frequentist statistics may stand on firmer ground.

Answer 10

Frequentist tests focus on falsifying the null hypothesis. However, Null Hypothesis Significance Testing (NHST) can also be done from a Bayesian perspective, because in all cases NHST is simply a calculation of P( Observed Effect | Effect = 0 ). So, it's hard to identify a time when it would be necessary to conduct NHST from a frequentist perspective.

That being said, the best argument for conducting NHST using a frequentist approach is ease and accessibility. People are taught frequentist statistics. So, it's easier to run a frequentist NHST, because there are many more statistical packages that make it simple to do this. Similarly, it is easier to communicate the results of a frequentist NHST, because people are familiar with this form of NHST. So, I see that as the best argument for frequentist approaches: accessibility to stats programs that will run them and ease of communication of results to colleagues. This is just cultural, though, so this argument could change if frequentist approaches lose their hegemony.

Answer 11

The goal of much research is not to reach a final conclusion, but just to obtain a little more evidence to incrementally push the community's sense of a question in one direction.

Bayesian statistics are indispensable when what you need is to evaluate a decision or conclusion in light of the available evidence. Quality control would be impossible without Bayesian statistics. Any procedure where you need to take some data and then act on it (robotics, machine learning, business decision making) benefits from Bayesian statistics.

But a lot of researchers are not doing that. They are running some experiments, collecting some data, and then saying "The data points this way", without really worrying too much about whether that's the best conclusion given all the evidence others have gathered so far. Science can be a slow process, and a statement like "The probability that this model is correct is 72%!" is often premature or unnecessary.

This is appropriate in a simple mathematical way, too, because frequentist statistics often turn out to be mathematically the same as the update-step of a Bayesian statistic. In other words, while Bayesian statistics is (Prior Model, Evidence) → New Model, frequentist statistics is just Evidence, and leaves it to others to fill in the other two parts.

Answer 12

The actual execution of a Bayesian method is more technical than that of a Frequentist. By "more technical" I mean things like: 1) choosing priors, 2) programming your model in a BUGS/JAGS/STAN, and 3) thinking about sampling and convergence.

Obviously, #1 is pretty much not optional, by definition of Bayesian. Though with some problems and procedures, there can be reasonable defaults, somewhat hiding the issue from the user. (Though this can also cause problems!)

Whether #2 is an issue depends on the software you use. Bayesian statistics has a bent towards more general solutions than frequentist statistical methods, and tools like BUGS, JAGS, and STAN are a natural expression of this. However, there are Bayesian functions in various software packages that appear to work like the typical frequentist procedure, so this is not always an issue. (And recent solutions like the R packages rstanarm and brms are bridging this gap.) Still, using these tools is very similar to programming in a new language.

Item #3 is usually applicable, since the majority of real-world Bayesian applications are going to use MCMC sampling. (On the other hand, frequentist MLE-based procedures use optimization which may converge to a local minima or not converge at all, and I wonder how many users should be checking this and don't?)

As I said in a comment, I'm not sure that freedom from priors is actually a scientific benefit. It's certainly convenient in several ways and at several points in the publication process, but I'm not sure it actually makes for better science. (And in the big picture, we all have to be aware of our priors as scientists, or we'll suffer from all kinds of biases in our investigations, regardless of what statistical methods we use.)

Answer 13

One type of problem in which a particular Frequentist based approach has essentially dominated any Bayesian is that of prediction in the M-open case.

What does M-open mean?

M-open implies that the true model that generates the data does not appear in the set of models we are considering. For example, if the true mean of $y$ is quadratic as a function of $x$, yet we only consider models with the mean a linear function of $x$, we are in the M-open case. In other words, model miss-specification results in an M-open case.

In most cases, this is a huge problem for Bayesian analyses; pretty much all theory that I know about relies on the model being correctly specified. Of course, as critical statisticians, we should think that our model is always misspecified. This is quite an issue; most of our theory is based on the model being correct, yet we know it never is. Basically, we're just crossing our fingers hoping that our model is not too incorrect.

Why do Frequentist methods handle this better?

Not all do. For example, if we use standard MLE tools for creating the standard errors or building prediction intervals, we're not better off than using Bayesian methods.

However, there is one particular Frequentist tool that is very specifically intended for exactly this purpose: cross validation. Here, in order to estimate how well our model will predict on new data, we simply leave of some of the data when fitting the model and measure how well our model predicts the unseen data.

Note that this method is completely ambivalent to model miss-specification, it merely provides a method for us to estimate for how well a model will predict on new data, regardless of whether the model is "correct" or not.

I don't think it's too hard to argue that this really changes the approach to predictive modeling that's hard to justify from a Bayesian perspective (prior is supposed to represent prior knowledge before seeing data, likelihood function is the model, etc.) to one that's very easy to justify from a Frequentist perspective (we chose the model + regularization parameters that, over repeated sampling, leads to the best out of sample errors).

This has completely revolutionized how predictive inference is done. I don't think any statistician would (or at least, should) seriously consider a predictive model that wasn't built or checked with cross-validation, when it's available (i.e., we can reasonable assume observations are independent, not trying to account for sampling bias, etc.).

Answer 14

Conceptually: I don't know. I believe Bayesian statistics is the most logical way to think but I coudn't justify why.

The advantage of frequentist is that it is easier for most people at elementary level. But for me it was strange. It took years until I could really clarify intellectually what a confidence interval is. But when I started facing practical situations, frequentist ideas appeared to be simple and highly relevant.

Empirically

The most important question I try to focus on nowadays is more about practical efficiency: personal work time, precision, and computation speed.

Personal work time: For basic questions, I actually almost never use Bayesian methods: I use basic frequentist tools and will always prefer a t-test over a Bayesian equivalent that would just give me a headache. When I want to know if I'm significantly better at tictactoe than my girlfriend, I do a chi-squared :-). Actually, even in serious work as a computer scientist, frequentist basic tools are just invaluable to investigate problems and avoid false conclusions due to random.

Precision: In machine learning where prediction matters more than analysis, there is not an absolute boundary between Bayesian and frequentist. MLE is a frequentist approcah: just an estimator. But regularized MLE (MAP) is a partially Bayesian approach: you find the mode of the posterior and you don't care for the rest of the posterior. I don't know of a frequentist justification of why use regularization. Practically, regularization is sometimes just inevitable because the raw MLE estimate is so overfitted that 0 would be a better predictor. If regularization is agreed to be a truly Bayesian method, then this alone justifies that Bayes can learn with less data.

Computation speed: frequentist methods are most often computationally faster and simpler to implement. And somehow regularization provides a cheap way to introduce a bit of Bayes in them. It might be because Bayesian methods are still not as optimized as they could. For example, some LDA implementations are fast nowadays. But they required very hard work. For entropy estimations, the first advanced methods were Bayesian. They worked great but soon frequentist methods were discovered and take much less computation time... For computation time frequentist methods are generally clearly superior. It is not absurd, if your are a Bayesian, to think of frequentist methods as approximations of Bayesian methods.