18
$\begingroup$

My understanding of the bayesian vs frequentist debate is that frequentist statistics:

  • is (or claims to be) objective
  • or at least unbiased
  • so different researchers, using different assumptions can still get quantitatively comparable results

while bayesian statistics

  • claims to make "better" predictions (i.e. lower expected loss), because it can use prior knowledge (among other reasons)
  • needs fewer "ad hoc" choices, replacing them by prior/model choices that (at least in principle) have a real-world interpretation.

Given that, I would have expected that bayesian statistics would be very popular in SPC: If I were a factory owner trying to control my process quality, I would primarily care about expected loss; If I could reduce that, because I have more/better prior knowledge than my competitors, even better.

But practically everything I have read about SPC seems to be firmly frequentist (i.e. no prior distributions, point estimates of all parameters, many ad-hoc choices about sample size, p-values etc.)

Why is that? I can see why frequentist statistics were a better choice in the 1960's, when SPC was done using pen and paper. But why hasn't anyone tried different methods since then?

$\endgroup$
  • 2
    $\begingroup$ I think Bayesian statistics as my Digital SLP camera while frequentist as iPhone camera. I bought both of them sometime but I use DSLR less 5% of photos while phone rest 95%. Because it is easy, handy and in pocket and many time provides as per quality (based my DSLR skills). Just like incorporating the priors and running chains in hayes I need to find optimum balance of aperture opening duration, length and other parameters. Iphone end of popular. $\endgroup$ – Ram Sharma Jun 26 '14 at 18:50
  • $\begingroup$ @RamSharma you should post that as an answer! I like it better than my chef knife analogy. $\endgroup$ – shadowtalker Jun 26 '14 at 19:38
17
$\begingroup$

WARNING I wrote this answer a long time ago with very little idea what I was talking about. I can't delete it because it's been accepted, but I can't stand behind most of the content.


This is a very long answer and I hope it'll be helpful in some way. SPC isn't my area, but I think these comments are general enough that they apply here.

I'd argue that the most-oft-cited advantage -- the ability to incorporate prior beliefs -- is a weak advantage applied/empirical fields. That's because you need to quantify your prior. Even if I can say "well, level z is definitely implausible," I can't for the life of me tell you what should happen below z. Unless authors start publishing their raw data in droves, my best guesses for priors are conditional moments taken from previous work that may or may not have been fitted under similar conditions to the ones you're facing.

Basically, Bayesian techniques (at least on a conceptual level) are excellent for when you have a strong assumption/idea/model and want to take it to data, then see how wrong or not wrong you turn out to be. But often you are not looking to see whether you're right about one particular model for your business process; more likely you have no model, and are looking to see what your process is going to do. You do not want to push your conclusions around, you want your data to push your conclusions. If you have enough data, that's what will happen anyway, but in that case why bother with the prior? Perhaps that's overly skeptical and risk-averse, but I've never heard of an optimistic businessman that was also successful. There is no way to quantify your uncertainty about your own beliefs, and you would rather not run the risk of being overconfident in the wrong thing. So you set an uninformative prior and the advantage disappears.

This is interesting in the SPC case because unlike in, say, digital marketing, your business processes aren't forever in an unpredictable state of flux. My impression is that business processes tend to change deliberately and incrementally. That is, you have a long time to build up good, safe priors. But recall that priors are all about propagating uncertainty. Subjectivity aside, Bayesianism has the advantage that it objectively propagates uncertainty across deeply-nested data generating processes. That, to me, is really what Bayesian statistics is good for. And if you're looking for reliability of your process well beyond the 1-in-20 "significance" cutoff, it seems like you would want to account for as much uncertainty as possible.

So where are the Bayesian models? First off, they're hard to implement. To put it bluntly, I can teach OLS to a mechanical engineer in 15 minutes and have him cranking out regressions and t-tests in Matlab in another 5. To use Bayes, I first need to decide what kind of model I'm fitting, and then see if there's a ready-made library for it in a language someone at my company knows. If not, I have to use BUGS or Stan. And then I have to run simulations to get even a basic answer, and that takes about 15 minutes on an 8-core i7 machine. So much for rapid prototyping. And second off, by the time you get an answer, you've spent two hours of coding and waiting, only to get the same result as you could have with frequentist random effects with clustered standard errors. Maybe this is all presumptuous and wrongheaded and I don't understand SPC at all. But I see it in academia and in for-profit social science constantly, and I'd be surprised if things were different in other fields.

I liken Bayesianism to a very high-quality chef knife, a stockpot, and a sautee pan; frequentism is like a kitchen full of As-Seen-On-TV tools like banana slicers and pasta pots with holes in the lid for easy draining. If you're a practiced cook with lots of experience in the kitchen--indeed, in your own kitchen of substantive knowledge, which is clean and organized and you know where everything is located--you can do amazing things with your small selection of elegant, high-quality tools. Or, you can use a bunch of different little ad-hoc* tools, that require zero skill to use, to make a meal that's simple, really not half bad, and has a couple basic flavors that get the point across. You just got home from the data mines and you're hungry for results; which cook are you?

*Bayes is just as ad-hoc, but less transparently so. How much wine goes in your coq au vin? No idea, you eyeball it because you're a pro. Or, you can't tell the difference between a Pinot Grigio and a Pinot Noir but the first recipe on Epicurious said to use 2 cups of the red one so that's what you're going to do. Which one is more "ad-hoc?"

$\endgroup$
  • 1
    $\begingroup$ +1, great answer. I'm curious: Could you add a paragraph about small/adaptive sample sizes? In SPC, samples sizes of 3-5 seem to be common. And if the SPC software could tell the technician after 2 samples whether it really needed 3 more samples or not, that would be a great feature. With a bayesian model, that's almost a no-brainer: Define a cost for measurements, false positives and -negatives, then estimate the expected cost of taking another measurement vs. stopping. In frequentist statistics, you'd have to deal with weird stopping rule effects (Can you teach those to an ME in 15 mins?) $\endgroup$ – nikie Jun 26 '14 at 20:17
  • 1
    $\begingroup$ As for sample size, the problem, and I would have mentioned this if I had known the samples were that small, is that with very few observations your estimates will be very sensitive to your choice of prior. You can't get blood from a stone, so it's a trade-off: either you grossly overfit with a frequentist estimator, but do so with few assumptions, or you incorporate your own knowledge (or lack thereof) into a sufficiently vague prior and essentially fit both to the data you have in front of you and the "data" you have in your head. You are allowed to have a uniform prior in your head. $\endgroup$ – shadowtalker Jun 26 '14 at 21:06
  • 1
    $\begingroup$ Basically, Bayes puts more burden on the analyst to use her brain at the outset. I personally think being averse to the idea of setting priors is a sign that you either a) are too lazy to , or b) don't really understand how statistics works (it takes one to know one, etc). I said it was hard to quantify priors in my answer; I actually don't agree with that in practice. One thing you can always do is draw a bell curve on a page, and ask yourself "would I expect my data to look like that?" If not, start tweaking the curve. And if you can't decide where to stick the mode, use a hyperprior. $\endgroup$ – shadowtalker Jun 26 '14 at 21:15
  • 2
    $\begingroup$ One question (not meant to be bratty): you do know there's a literature on (quantitatively) soliciting prior beliefs, right? Including published beliefs, interviewed expert and non-expert beliefs, and self-beliefs. The reason I ask, is that I have heard this complaint before, but the authors of such complaints thought that their objection was the end of the discussion, rather than the beginning of an inquiry. $\endgroup$ – Alexis Jun 29 '14 at 15:49
  • 1
    $\begingroup$ @CliffAB Interesting… I have not read that literature deeply (Bernardo, Kaas, Garthwaite... from several decades back)… but that's value-laden science for you: different prior beliefs inform whether one prefers frequentist or Bayesian methods. ;) $\endgroup$ – Alexis Jan 23 '16 at 1:50
5
$\begingroup$

In my humble opinion, Bayesian statistics suffers from some drawbacks that conflict with its widespread use (in SPC but in other research sectors as well):

  1. It is more difficult to get estimates vs. its frequentist counterpart (the widest part of classes on statistics adopt the frequentist approach. By the way, it would be interesting to investigate if this is the cause or the effect of the limited popularity of Bayesian statistics).

  2. Very often Bayesian statistics imposes choices about different ways of dealing with the same problem (e.g., which is the best prior?), not just click-and-see (anyway, this approach should not be encouraged under the frequentist framework, either).

  3. Bayesian statistics has some topics which are difficult to manage by less than very experienced statisticians (e.g., improper priors);

  4. It requires sensitivity analyses (usually avoided under the frequentist framework), and exceptions made for some topics, such as missing data analysis.

  5. It has only one (laudably, free downloadable) software available for calculation.

  6. It takes more time to be an autonomous researcher with Bayesian than with frequentist tools.

$\endgroup$
  • 6
    $\begingroup$ Good answer but I disagree with point 5: I can think of many different (free) software for Bayesian analysis: WinBUGS, OpenBUGS, JAGS, Stan, PyMC ... and I'm sure there are more. What I would say is that all of these software have a steep learning curve and require a decent amount of programming and statistical knowledge. $\endgroup$ – COOLSerdash Jun 26 '14 at 18:58
  • $\begingroup$ COOLSerdash is right and I welcome both clarification and comment. My lack of comprehensiveness in listing Bayesian analysis softwares was probably driven by my (loose) familiarity with WinBugs only. $\endgroup$ – Carlo Lazzaro Jun 29 '14 at 17:36
  • $\begingroup$ @CarloLazzaro I agree with COOLSerdash's point about #5, also: As of version 14, the private licensed yet mainstream stats package Stata now incorporates Bayesian models and estimation in the vanilla package. I think Bayesian computational availability will only grow. But your other points are important, and should help inform agenda for Bayesian proponents. $\endgroup$ – Alexis Jan 23 '16 at 1:52
  • $\begingroup$ @Alexis: being a Stata user I'm happy with its quite recent Bayesian flavour. As a more general thought, I would vouch learning both frequentist and Bayesian approaches during statistical classes at the university (probably likelihoodists start grumbling!!). $\endgroup$ – Carlo Lazzaro Feb 22 '16 at 18:56
3
$\begingroup$

One reason is that Bayesian statistics was frozen out of the mainstream until around 1990. When I was studying statistics in the 1970s it was almost heresy (not everywhere, but in most graduate programs). It didn't help that most of the interesting problems were intractable. As a result, nearly everyone who is teaching statistics today (and reviewing articles for journals, and designing curricula) is trained as a frequentist. Things started to change around 1990 with the popularization of Markov Chain Monte Carlo (MCMC) methods which are gradually finding their way into packages like SAS and Stata. Personally I think they will be much more common in 10 years though in specialized applications (SPC) they may not have much of an advantage.

One group that is woking make Bayesian analysis more widely available is the group developing the STAN package (mc-stan.org).

$\endgroup$
  • $\begingroup$ Welcome to our site! Just a note that it's "Stata" rather than "STATA" - I have been on the wrong end of Stata users when I have capitalised it myself! (I thought it was like SAS, SPSS etc, but apparently not...) $\endgroup$ – Silverfish Apr 9 '16 at 12:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.