The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [bayesian]

Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.

Filter by
Sorted by
Tagged with
342
votes
15answers
202k views

Bayesian and frequentist reasoning in plain English

How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
240
votes
11answers
166k views

How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?

Maybe the concept, why it's used, and an example.
230
votes
9answers
119k views

What's the difference between a confidence interval and a credible interval?

Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the the difference between confidence intervals and credible intervals were the correct ones. How you would ...
192
votes
36answers
110k views

What is the best introductory Bayesian statistics textbook?

Which is the best introductory textbook for Bayesian statistics? One book per answer, please.
125
votes
3answers
166k views

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
113
votes
13answers
22k views

What's wrong with XKCD's Frequentists vs. Bayesians comic?

This xkcd comic (Frequentists vs. Bayesians) makes fun of a frequentist statistician who derives an obviously wrong result. However it seems to me that his reasoning is actually correct in the sense ...
100
votes
8answers
5k views

ASA discusses limitations of $p$-values - what are the alternatives?

We already have multiple threads tagged as p-values that reveal lots of misunderstandings about them. Ten months ago we had a thread about psychological journal that "banned" $p$-values, now American ...
92
votes
12answers
10k views

Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
81
votes
2answers
22k views

XKCD's modified Bayes theorem: actually kinda reasonable?

I know this is from a comic famous for taking advantage of certain analytical tendencies, but it actually looks kind of reasonable after a few minutes of staring. Can anyone outline for me what this "...
81
votes
6answers
9k views

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic: Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
73
votes
4answers
21k views

What is an “uninformative prior”? Can we ever have one with truly no information?

Inspired by a comment from this question: What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior? I generally see the prior in ...
72
votes
14answers
11k views

When (if ever) is a frequentist approach substantively better than a Bayesian?

Background: I do not have an formal training in Bayesian statistics (though I am very interested in learning more), but I know enough--I think--to get the gist of why many feel as though they are ...
68
votes
11answers
8k views

Why should I be Bayesian when my model is wrong?

Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad. I, a fairly devout ...
67
votes
10answers
8k views

Is there any *mathematical* basis for the Bayesian vs frequentist debate?

It says on Wikipedia that: the mathematics [of probability] is largely independent of any interpretation of probability. Question: Then if we want to be mathematically correct, shouldn't we ...
66
votes
7answers
4k views

How much to pay? A practical problem

This is not a home work question but real problem faced by our company. Very recently (2 days ago) we ordered for manufacturing of 10000 product labels to a dealer. Dealer is independent person. He ...
64
votes
8answers
7k views

What is a good, convincing example in which p-values are useful?

My question in the title is self explanatory, but I would like to give it some context. The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
63
votes
9answers
4k views

List of situations where a Bayesian approach is simpler, more practical, or more convenient

There have been many debates within statistics between Bayesians and frequentists. I generally find these rather off-putting (although I think it has died down). On the other hand, I've met several ...
62
votes
8answers
5k views

Bayesians: slaves of the likelihood function?

In his book "All of Statistics", Prof. Larry Wasserman presents the following Example (11.10, page 188). Suppose that we have a density $f$ such that $f(x)=c\,g(x)$, where $g$ is a known (nonnegative, ...
61
votes
5answers
17k views

Why is the Jeffreys prior useful?

I understand that the Jeffreys prior is invariant under re-parameterization. However, what I don't understand is why this property is desired. Why wouldn't you want the prior to change under a change ...
59
votes
6answers
6k views

Where did the frequentist-Bayesian debate go?

The world of statistics was divided between frequentists and Bayesians. These days it seems everyone does a bit of both. How can this be? If the different approaches are suitable for different ...
57
votes
2answers
26k views

Bayes regression: how is it done in comparison to standard regression?

I got some questions about the Bayesian regression: Given a standard regression as $y = \beta_0 + \beta_1 x + \varepsilon$. If I want to change this into a Bayesian regression, do I need prior ...
55
votes
10answers
8k views

Who are frequentists?

We already had a thread asking who are Bayesians and one asking if frequentists are Bayesians, but there was no thread asking directly who are frequentists? This is a question that was asked by @...
54
votes
5answers
13k views

Examples of Bayesian and frequentist approach giving different answers

Note: I am aware of philosophical differences between Bayesian and frequentist statistics. For example "what is the probability that the coin on the table is heads" doesn't make sense in frequentist ...
51
votes
2answers
28k views

What is the difference between a particle filter (sequential Monte Carlo) and a Kalman filter?

A particle filter and Kalman filter are both recursive Bayesian estimators. I often encounter Kalman filters in my field, but very rarely see the usage of a particle filter. When would one be used ...
50
votes
3answers
66k views

What is the difference in Bayesian estimate and maximum likelihood estimate?

Please explain to me the difference in Bayesian estimate and Maximum likelihood estimate?
47
votes
6answers
7k views

Bayesian statistics tutorial

I am trying to get upto speed in Bayesian Statistics. I have a little bit of stats background (STAT 101) but not too much - I think I can understand prior, posterior, and likelihood :D. I don't want ...
46
votes
2answers
3k views

Why is a Bayesian not allowed to look at the residuals?

In the article "Discussion: Should Ecologists Become Bayesians?" Brian Dennis gives a surprisingly balanced and positive view of Bayesian statistics when his aim seems to be to warn people about it. ...
46
votes
2answers
24k views

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some great courses, I'd ...
44
votes
8answers
13k views

What are the cons of Bayesian analysis?

What are some practical objections to the use of Bayesian statistical methods in any context? No, I don't mean the usual carping about choice of prior. I'll be delighted if this gets no answers.
44
votes
7answers
5k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
43
votes
3answers
5k views

Is it possible to interpret the bootstrap from a Bayesian perspective?

Ok, this is a question that keeps me up at night. Can the bootstrap procedure be interpreted as approximating some Bayesian procedure (except for the Bayesian bootstrap)? I really like the Bayesian "...
42
votes
5answers
4k views

Eliciting priors from experts

How should I elicit prior distributions from experts when fitting a Bayesian model?
41
votes
1answer
12k views

Can someone explain the concept of 'exchangeability'?

I see the concept of 'exchangeability' being used in different contexts (e.g., bayesian models) but I have never understood the term very well. What does this concept mean? Under what circumstances ...
39
votes
7answers
6k views

Would a Bayesian admit that there is one fixed parameter value?

In Bayesian data analysis, parameters are treated as random variables. This stems from the Bayesian subjective conceptualization of probability. But do Bayesians theoretically acknowledge that there ...
39
votes
4answers
14k views

Bayesian equivalent of two sample t-test?

I'm not looking for a plug and play method like BEST in R but rather a mathematical explanation of what are some Bayesian methods I can use to test the difference between the mean of two samples.
38
votes
6answers
3k views

What is the connection between credible regions and Bayesian hypothesis tests?

In frequentist statistics, there is a close connection between confidence intervals and tests. Using inference about $\mu$ in the $\rm N(\mu,\sigma^2)$ distribution as an example, the $1-\alpha$ ...
37
votes
6answers
26k views

Bayesian vs frequentist Interpretations of Probability

Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability? From what I understand: The frequentists view is that the data is a repeatable ...
37
votes
5answers
2k views

Do working statisticians care about the difference between frequentist and Bayesian inference?

As an outsider, it appears that there are two competing views on how one should perform statistical inference. Are the two different methods both considered valid by working statisticians? Is ...
36
votes
5answers
6k views

Is p-value essentially useless and dangerous to use?

This article "The Odds, Continually Updated" from NY Times happened to catch my attention. To be short, it states that [Bayesian statistics] is proving especially useful in approaching complex ...
35
votes
2answers
31k views

Why is the Dirichlet distribution the prior for the multinomial distribution?

In LDA topic model algorithm, I saw this assumption. But I don't know why chose Dirichlet distribution? I don't know if we can use Uniform distribution over Multinomial as a pair?
35
votes
8answers
14k views

What is Bayes' theorem all about?

What are the main ideas, that is, concepts related to Bayes' theorem? I am not asking for any derivations of complex mathematical notation.
35
votes
5answers
5k views

Think like a bayesian, check like a frequentist: What does that mean?

I am looking at some lecture slides on a data science course which can be found here: https://github.com/cs109/2015/blob/master/Lectures/01-Introduction.pdf I, unfortunately, cannot see the video ...
34
votes
5answers
22k views

What exactly is a Bayesian model?

Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad. So what exactly is a Bayesian model?
33
votes
1answer
6k views

Variational inference versus MCMC: when to choose one over the other?

I think I get the general idea of both VI and MCMC including the various flavors of MCMC like Gibbs sampling, Metropolis Hastings etc. This paper provides a wonderful exposition of both methods. I ...
32
votes
8answers
3k views

Should I teach Bayesian or frequentist statistics first?

I am helping my boys, currently in high school, understanding statistics, and I am considering beginning with some simple examples without disregarding some glimpses to theory. My goal would be to ...
32
votes
6answers
3k views

What would a robust Bayesian model for estimating the scale of a roughly normal distribution be?

There exists a number of robust estimators of scale. A notable example is the median absolute deviation which relates to the standard deviation as $\sigma = \mathrm{MAD}\cdot1.4826$. In a Bayesian ...
31
votes
5answers
5k views

What do confidence intervals say about precision (if anything)?

Morey et al (2015) argue that confidence intervals are misleading and there are multiple bias related to understanding of them. Among others, they describe the precision fallacy as following: The ...
31
votes
1answer
11k views

What are posterior predictive checks and what makes them useful?

I understand what the posterior predictive distribution is, and I have been reading about posterior predictive checks, although it isn't clear to me what it does yet. What exactly is the posterior ...
31
votes
3answers
2k views

Entropy-based refutation of Shalizi's Bayesian backward arrow of time paradox?

In this paper, the talented researcher Cosma Shalizi argues that to fully accept a subjective Bayesian view, one must also accept an unphysical result that the arrow of time (given by the flow of ...