Questions tagged [bayesian]
Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.
8,333
questions
437
votes
14
answers
283k
views
Bayesian and frequentist reasoning in plain English
How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
- 4,479
329
votes
10
answers
197k
views
What's the difference between a confidence interval and a credible interval?
Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
- 6,107
277
votes
12
answers
189k
views
How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?
Maybe the concept, why it's used, and an example.
- 10.1k
248
votes
38
answers
156k
views
What is the best introductory Bayesian statistics textbook?
Which is the best introductory textbook for Bayesian statistics?
One book per answer, please.
156
votes
3
answers
250k
views
Help me understand Bayesian prior and posterior distributions
In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
- 1,561
151
votes
15
answers
81k
views
Amazon interview question—probability of 2nd interview
I got this question during an interview with Amazon:
50% of all people who receive a first interview receive a second interview
95% of your friends that got a second interview felt they had a good ...
- 1,481
136
votes
14
answers
31k
views
What's wrong with XKCD's Frequentists vs. Bayesians comic?
This xkcd comic (Frequentists vs. Bayesians) makes fun of a frequentist statistician who derives an obviously wrong result.
However it seems to me that his reasoning is actually correct in the sense ...
- 1,667
116
votes
10
answers
8k
views
ASA discusses limitations of $p$-values - what are the alternatives?
We already have multiple threads tagged as p-values that reveal lots of misunderstandings about them. Ten months ago we had a thread about psychological journal that "banned" $p$-values, now American ...
- 140k
106
votes
4
answers
45k
views
What is an "uninformative prior"? Can we ever have one with truly no information?
Inspired by a comment from this question:
What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior?
I generally see the prior in ...
- 23.5k
102
votes
9
answers
15k
views
Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals
A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic:
Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
- 55.1k
97
votes
12
answers
12k
views
Who Are The Bayesians?
As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
- 26.7k
91
votes
11
answers
12k
views
Why should I be Bayesian when my model is wrong?
Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad.
I, a fairly devout ...
- 2,851
90
votes
15
answers
24k
views
When (if ever) is a frequentist approach substantively better than a Bayesian?
Background: I do not have an formal training in Bayesian statistics (though I am very interested in learning more), but I know enough--I think--to get the gist of why many feel as though they are ...
86
votes
2
answers
47k
views
Bayes regression: how is it done in comparison to standard regression?
I got some questions about the Bayesian regression:
Given a standard regression as $y = \beta_0 + \beta_1 x + \varepsilon$.
If I want to change this into a Bayesian regression, do I need prior ...
- 1,066
83
votes
2
answers
23k
views
XKCD's modified Bayes theorem: actually kinda reasonable?
I know this is from a comic famous for taking advantage of certain analytical tendencies, but it actually looks kind of reasonable after a few minutes of staring. Can anyone outline for me what this "...
- 5,239
77
votes
11
answers
12k
views
Is there any *mathematical* basis for the Bayesian vs frequentist debate?
It says on Wikipedia that:
the mathematics [of probability] is largely independent of any interpretation of probability.
Question: Then if we want to be mathematically correct, shouldn't we disallow ...
- 6,389
75
votes
6
answers
28k
views
Why is the Jeffreys prior useful?
I understand that the Jeffreys prior is invariant under re-parameterization. However, what I don't understand is why this property is desired.
Why wouldn't you want the prior to change under a change ...
- 1,023
71
votes
4
answers
109k
views
What is the difference in Bayesian estimate and maximum likelihood estimate?
Please explain to me the difference in Bayesian estimate and Maximum likelihood estimate?
- 877
71
votes
2
answers
46k
views
What does the inverse of covariance matrix say about data? (Intuitively)
I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?"
Edit:
Thanks for replies
After taking some great courses, I'd ...
- 973
68
votes
8
answers
8k
views
What is a good, convincing example in which p-values are useful?
My question in the title is self explanatory, but I would like to give it some context.
The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
- 21.8k
67
votes
7
answers
7k
views
Where did the frequentist-Bayesian debate go?
The world of statistics was divided between frequentists and Bayesians. These days it seems everyone does a bit of both. How can this be? If the different approaches are suitable for different ...
67
votes
1
answer
27k
views
Can someone explain the concept of 'exchangeability'?
I see the concept of 'exchangeability' being used in different contexts (e.g., bayesian models) but I have never understood the term very well.
What does this concept mean?
Under what circumstances ...
- 865
66
votes
9
answers
6k
views
List of situations where a Bayesian approach is simpler, more practical, or more convenient
There have been many debates within statistics between Bayesians and frequentists. I generally find these rather off-putting (although I think it has died down). On the other hand, I've met several ...
65
votes
8
answers
6k
views
Are bayesians slaves of the likelihood function?
In his book "All of Statistics", Prof. Larry Wasserman presents the following Example (11.10, page 188). Suppose that we have a density $f$ such that $f(x)=c\,g(x)$, where $g$ is a known (nonnegative, ...
- 24.9k
65
votes
11
answers
19k
views
Examples of Bayesian and frequentist approach giving different answers
Note: I am aware of philosophical differences between Bayesian and frequentist statistics.
For example "what is the probability that the coin on the table is heads" doesn't make sense in ...
65
votes
7
answers
4k
views
How much to pay? A practical problem
This is not a home work question but real problem faced by our company.
Very recently (2 days ago) we ordered for manufacturing of 10000 product labels to a dealer. Dealer is independent person. He ...
- 2,320
61
votes
10
answers
10k
views
Who are frequentists?
We already had a thread asking who are Bayesians and one asking if frequentists are Bayesians, but there was no thread asking directly who are frequentists? This is a question that was asked by @...
- 140k
59
votes
3
answers
13k
views
What kind of information is Fisher information?
Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic ...
- 4,433
59
votes
2
answers
47k
views
What is the difference between a particle filter (sequential Monte Carlo) and a Kalman filter?
A particle filter and Kalman filter are both recursive Bayesian estimators. I often encounter Kalman filters in my field, but very rarely see the usage of a particle filter.
When would one be used ...
- 12.5k
59
votes
1
answer
29k
views
What are posterior predictive checks and what makes them useful?
I understand what the posterior predictive distribution is, and I have been reading about posterior predictive checks, although it isn't clear to me what it does yet.
What exactly is the posterior ...
- 19.5k
56
votes
9
answers
44k
views
Bayesian vs frequentist Interpretations of Probability
Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability?
From what I understand:
The frequentists view is that the data is a repeatable ...
- 1,505
55
votes
6
answers
6k
views
Eliciting priors from experts
How should I elicit prior distributions from experts when fitting a Bayesian model?
- 12.9k
55
votes
5
answers
27k
views
Bayesian equivalent of two sample t-test?
I'm not looking for a plug and play method like BEST in R but rather a mathematical explanation of what are some Bayesian methods I can use to test the difference between the mean of two samples.
- 581
53
votes
7
answers
8k
views
Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?
If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
user10525
53
votes
4
answers
24k
views
What are the factors that cause the posterior distributions to be intractable?
In Bayesian statistics, it is often mentioned that the posterior distribution is intractable and thus approximate inference must be applied. What are the factors that cause this intractability?
- 3,617
53
votes
2
answers
5k
views
Why is a Bayesian not allowed to look at the residuals?
In the article "Discussion: Should Ecologists Become Bayesians?" Brian Dennis gives a surprisingly balanced and positive view of Bayesian statistics when his aim seems to be to warn people about it. ...
- 633
53
votes
1
answer
17k
views
Variational inference versus MCMC: when to choose one over the other?
I think I get the general idea of both VI and MCMC including the various flavors of MCMC like Gibbs sampling, Metropolis Hastings etc. This paper provides a wonderful exposition of both methods.
I ...
- 3,602
52
votes
3
answers
8k
views
Is it possible to interpret the bootstrap from a Bayesian perspective?
Ok, this is a question that keeps me up at night.
Can the bootstrap procedure be interpreted as approximating some Bayesian procedure (except for the Bayesian bootstrap)?
I really like the Bayesian &...
- 6,860
51
votes
2
answers
23k
views
Can somebody explain to me NUTS in english?
My understanding of the algorithm is the following:
No U-Turn Sampler (NUTS) is a Hamiltonian Monte Carlo Method. This means that it is not a Markov Chain method and thus, this algorithm avoids the ...
- 641
50
votes
8
answers
8k
views
Would a Bayesian admit that there is one fixed parameter value?
In Bayesian data analysis, parameters are treated as random variables. This stems from the Bayesian subjective conceptualization of probability. But do Bayesians theoretically acknowledge that there ...
- 1,901
50
votes
8
answers
21k
views
What are the cons of Bayesian analysis?
What are some practical objections to the use of Bayesian statistical methods in any context? No, I don't mean the usual carping about choice of prior. I'll be delighted if this gets no answers.
user6666
49
votes
2
answers
58k
views
What exactly is the alpha in the Dirichlet distribution?
I'm fairly new to Bayesian statistics and I came across a corrected correlation measure, SparCC, that uses the Dirichlet process in the backend of it's algorithm. I have been trying to go through the ...
- 1,482
49
votes
7
answers
23k
views
Combining probabilities/information from different sources
Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the ...
- 611
48
votes
7
answers
8k
views
Bayesian statistics tutorial
I am trying to get upto speed in Bayesian Statistics. I have a little bit of stats background (STAT 101) but not too much - I think I can understand prior, posterior, and likelihood :D.
I don't want ...
47
votes
6
answers
5k
views
How seriously should I think about the different philosophies of statistics?
I've just finished a module where we covered the different approaches to statistical problems – mainly Bayesian vs frequentist. The lecturer also announced that she is a frequentist. We covered some ...
- 423
47
votes
3
answers
13k
views
Do Bayesian priors become irrelevant with large sample size?
When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
- 659
47
votes
5
answers
85k
views
What exactly is a Bayesian model?
Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad.
So what exactly is a Bayesian model?
- 2,629
47
votes
2
answers
29k
views
Why is Laplace prior producing sparse solutions?
I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero.
I know how these priors look ...
- 725
45
votes
2
answers
113k
views
Difference between naive Bayes & multinomial naive Bayes
I've dealt with Naive Bayes classifier before. I've been reading about Multinomial Naive Bayes lately.
Also Posterior Probability = (Prior * Likelihood)/(Evidence).
The only prime difference (while ...
- 2,083