Bayesian analysis used merely as a computational tool?

Question

I have sometimes seen some statisticians used bayesian analysis and related techniques such as MCMC simply as a tool when a frequentist approach is not satisfying, typically for example when the maximum likelihood estimator is hard to find or takes too much time to compute.

In these case they focus only on the definition of the model and estimation (by MCMC for example) and barely on the choice prior distributions (from what I've seen). I guess they use flat distributions by default but the particular choice of a prior is never discussed, maybe because of the large number of parameters the model may have and the difficulty to assign a meaningful prior distribution to each one of them.

Is it something usual to switch to a bayesian analysis and use it merely a as computational counterpart of a frequentist approach when the latter performs poorly?
My feeling is that bayesian and frequentist are very distinct statistical framework (at least that's what my teachers taught me!) and using one or another shouldn't be justified simply by computational purposes and moreover the choice of the prior distributions should at least be carefully made.

I know this is not really a statistical question but I just would like to know what pure bayesians think about this kind of use of bayesian analysis. Sorry in advance if it does not fit this site.

Answer 1

The set of methods called "frequentist" statistics is quite broad. It allows you to specify any proposed estimator you want and then investigate its long-run properties conditional on the true values of the parameters. This method only counts an estimator out completely if it is "inadmissible", meaning that it is dominated by another available estimator (i.e., it gives equal/higher risk over every possible value of the parameter and higher risk over at least some parameter values).

Now, there is a famous theorem that says that, under wide conditions, Bayesian estimators are admissible --- i.e., they are not dominated by other estimators. Bayesian estimators tend to be biased (since they incorporate prior information) but they are also consistent under fairly wide conditions. This means that they are estimators that will tend to perform well in terms of the frequentist criteria. Consequently, frequentists usually consider these estimators as one option that can be used in their analysis.

By definition, "pure Bayesians" are going to adopt the Bayesian methodology in all cases. Most pure Bayesians are going to have adopted this methodology by being convinced of its underlying philosophical and mathematical superiority. However, part of the motivation for adoption of Bayesian methods may be the knowledge that even under the frequentist paradigm, these methods tend to perform well according to frequentist criteria. As to what a pure Bayesian would think of a frequentist using a Bayesian estimator, I suppose it is somewhat like what a priest would think of an atheist who decides one day to pray for spiritual guidance (e.g., on the basis that it can't do any harm even under their own philosophy). They would likely see this as a desirable change in behaviour, improperly motivated, but also possibly a useful entry-point to try to convince them that the general philosophy underpinning that activity is coherent and desirable.

Answer 2

In a 2002 paper with Arnaud Doucet and Simon Godsill, Marginal maximum a posteriori estimation using Markov chain Monte Carlo, we use an MCMC approach to derive the maximum likelihood estimator in latent variable models where the observed likelihood is not available. By repeating the number of repetitions of said latent variables in a simulated annealing spirit. Similar proposals were proposed subsequently by

• Gaetan and Yao (2003) under the name of multiple imputation Metropolis-EM
• Lele et al. (2007) under the name of data cloning
• Jacquier, Johannes and Polson (2007) under the name of MCMC maximum likelihood

Answer 3

Comment: Here are a few reasons why a frequentist statistician might use a Bayesian approach.

Computational convenience, as @Fiodor1234 says, may not be high on the list. One such example might be use of the Jeffreys posterior probability interval as a confidence interval for a binomial proportion. For example, if you have $x = 42$ successes in $n=100$ trials, the asymptotic Wald interval is not the best choice because of the small sample size. The Agresti-Coull interval is easy to compute and comes close to more accurate intervals that are somewhat intricate to compute. The Jeffreys interval, based on the noninformative Bayesian prior $\mathsf{Beta}(.5, .5)$, is easy to compute in R and has good frequentist properties.

p.hat = 42/100
CI.Wald = p.hat + qnorm(c(.025,.975))*sqrt(p.hat*(1-p.hat)/100)
round(CI.Wald,4)
[1] 0.3233 0.5167

p.est = (42+2)/(100+4)
CI.Agr = p.est + qnorm(c(.025,.975))*sqrt(p.est*(1-p.est)/104)
round(CI.Agr,4)
[1] 0.3281 0.5180

CI.Jeff = qbeta(c(.025,.975), 42+.5, 100-42+.5)
round(CI.Jeff,4)
[1] 0.3267 0.5179

Proper support of distribution. In an attempt to find the prevalence of a disease from screening test data, traditional methods can give an interval for prevalence that extends beyond $(0,1).$ By using a Gibbs sampler with a beta prior distribution, it is possible to get a useful interval estimate for prevalence. (Since the beta prior has the unit interval as support, then the posterior distribution will also.) See example..

'Simulate' latent data. Sometimes one wants to test or to give a parameter estimate for latent data, which can be reliably reconstructed using a Gibbs sampler. One simple example is to know the variability of groups in a one-way random-effects ANOVA. Observed values from the groups are available, but the components of variance due to the various groups (separate from overall variance) are typically latent.

Answer 4

Other reasons to use Bayesian approaches include

• getting more accurate inference when the log-likelihood is very non-Gaussian. For example, in binary logistic regression standard p-values and confidence intervals may be inaccurate whereas Bayesian quantities are exact.
• getting accurate uncertainty intervals for complex derived parameters. In the frequentist world we frequently have to resort to the delta method to get approximate confidence intervals. Not only is this labor intensive, but the result is often very unsatisfactory because such intervals are forced to be symmetric when they should have been asymmetric in order to have accurate coverage with respect to both tails. One example is state occupancy probabilities in a state transition model, which involve recursive matrix multiplications are a mess to deal with in the frequentist domain. With MCMC (let's say you have 4000 posterior draws from the multivariate distribution of all parameters together) you just compute the complex derived parameter 4000 times and estimate the highest posterior density interval from those 4000 numbers.