Questions tagged [bayesian]
Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.
1,095
questions
98
votes
4
answers
42k
views
What is an "uninformative prior"? Can we ever have one with truly no information?
Inspired by a comment from this question:
What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior?
I generally see the prior in ...
- 22.3k
321
votes
10
answers
190k
views
What's the difference between a confidence interval and a credible interval?
Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
- 5,937
14
votes
2
answers
24k
views
Bayesian logit model - intuitive explanation?
I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad.
What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
- 2,394
246
votes
38
answers
152k
views
What is the best introductory Bayesian statistics textbook?
Which is the best introductory textbook for Bayesian statistics?
One book per answer, please.
26
votes
5
answers
9k
views
What does "likelihood is only defined up to a multiplicative constant of proportionality" mean in practice?
I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners.
As a likelihood example, they ...
- 557
434
votes
14
answers
280k
views
Bayesian and frequentist reasoning in plain English
How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
- 4,449
46
votes
3
answers
12k
views
Do Bayesian priors become irrelevant with large sample size?
When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
- 619
102
votes
9
answers
15k
views
Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals
A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic:
Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
- 53.3k
85
votes
2
answers
46k
views
Bayes regression: how is it done in comparison to standard regression?
I got some questions about the Bayesian regression:
Given a standard regression as $y = \beta_0 + \beta_1 x + \varepsilon$.
If I want to change this into a Bayesian regression, do I need prior ...
- 1,026
15
votes
2
answers
21k
views
Linear discriminant analysis and Bayes rule: classification
What is the relation between Linear discriminant analysis and Bayes rule? I understand that LDA is used in classification by trying to minimize the ratio of within group variance and between group ...
- 861
14
votes
3
answers
2k
views
How exactly do Bayesians define (or interpret?) probability?
Part of a series of trying to understand Bayesian vs frequentist: 1 2 3 4 5 6 7
I think I get the difference of how Bayesians and frequentists approach choosing between hypotheses, but I'm not quite ...
- 2,394
48
votes
7
answers
22k
views
Combining probabilities/information from different sources
Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the ...
- 601
10
votes
1
answer
460
views
Trying to Estimate Disease Prevalence from Fragmentary Test Results
In response to the spread of COVID-19 disease, all Californians were ordered on 19 March 2020 to stay at home, except for such necessary errands as trips to grocery stores, pharmacies, etc. On 21 ...
- 55.1k
54
votes
9
answers
41k
views
Bayesian vs frequentist Interpretations of Probability
Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability?
From what I understand:
The frequentists view is that the data is a repeatable ...
- 1,485
35
votes
5
answers
3k
views
Wikipedia entry on likelihood seems ambiguous
I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.)
It starts from the Wikipedia page on likelihood. They say ...
- 1,615
154
votes
3
answers
242k
views
Help me understand Bayesian prior and posterior distributions
In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
- 1,541
48
votes
2
answers
56k
views
What exactly is the alpha in the Dirichlet distribution?
I'm fairly new to Bayesian statistics and I came across a corrected correlation measure, SparCC, that uses the Dirichlet process in the backend of it's algorithm. I have been trying to go through the ...
- 1,442
29
votes
4
answers
17k
views
Why is a normalizing factor required in Bayes’ Theorem?
Bayes theorem goes
$$ P(\textrm{model}|\textrm{data}) = \frac{P(\textrm{model}) \times P(\textrm{data}|\textrm{model})}{P(\textrm{data})} $$
This is all fine. But, I've read somewhere:
Basically, ...
64
votes
11
answers
18k
views
Examples of Bayesian and frequentist approach giving different answers
Note: I am aware of philosophical differences between Bayesian and frequentist statistics.
For example "what is the probability that the coin on the table is heads" doesn't make sense in ...
34
votes
3
answers
17k
views
Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?
My understanding is that when using a Bayesian approach to estimate parameter values:
The posterior distribution is the combination of the prior distribution and the likelihood distribution.
We ...
- 2,481
11
votes
5
answers
4k
views
Interpretation of Bayes Theorem applied to positive mammography results
I'm trying to wrap my head around the result of Bayes Theorem applied to the classic mammogram example, with the twist of the mammogram being perfect.
That is,
Incidence of cancer: $.01$
...
- 229
52
votes
3
answers
12k
views
What kind of information is Fisher information?
Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic ...
- 3,983
276
votes
12
answers
186k
views
How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?
Maybe the concept, why it's used, and an example.
- 9,762
25
votes
3
answers
36k
views
Bayesian updating with new data
How do we go about calculating a posterior with a prior N~(a, b) after observing n data points? I assume that we have to calculate the sample mean and variance of the data points and do some sort of ...
- 255
56
votes
1
answer
27k
views
What are posterior predictive checks and what makes them useful?
I understand what the posterior predictive distribution is, and I have been reading about posterior predictive checks, although it isn't clear to me what it does yet.
What exactly is the posterior ...
43
votes
2
answers
6k
views
Why should we use t errors instead of normal errors?
In this blog post by Andrew Gelman, there is the following passage:
The Bayesian models of 50 years ago seem hopelessly simple (except, of
course, for simple problems), and I expect the Bayesian ...
- 1,075
37
votes
1
answer
30k
views
Equivalence between least squares and MLE in Gaussian model
I am new to Machine Learning, and am trying to learn it on my own. Recently I was reading through some lecture notes and had a basic question.
Slide 13 says that "Least Square Estimate is same as ...
- 1,683
52
votes
7
answers
8k
views
Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?
If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
user10525
21
votes
1
answer
15k
views
Choosing between uninformative beta priors
I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
- 211
116
votes
10
answers
8k
views
ASA discusses limitations of $p$-values - what are the alternatives?
We already have multiple threads tagged as p-values that reveal lots of misunderstandings about them. Ten months ago we had a thread about psychological journal that "banned" $p$-values, now American ...
Tim♦
- 136k
23
votes
3
answers
26k
views
Normalizing constant in Bayes theorem
I read that in Bayes rule, the denominator $\Pr(\textrm{data})$ of
$$\Pr(\text{parameters} \mid \text{data}) = \frac{\Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\text{parameters})}{\Pr(\text{...
- 384
46
votes
2
answers
26k
views
Why is Laplace prior producing sparse solutions?
I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero.
I know how these priors look ...
- 715
20
votes
2
answers
11k
views
Effective Sample Size for posterior inference from MCMC sampling
When obtaining MCMC samples to make inference on a particular parameter, what are good guides for the minimum number of effective samples that one should aim for?
And, does this advice change as the ...
- 3,319
13
votes
1
answer
1k
views
Aside from the exponential family, where else can conjugate priors come from?
Do all conjugate priors have to come from the exponential family? If not, what other families are known to have/produce conjugate priors?
- 3,998
136
votes
14
answers
30k
views
What's wrong with XKCD's Frequentists vs. Bayesians comic?
This xkcd comic (Frequentists vs. Bayesians) makes fun of a frequentist statistician who derives an obviously wrong result.
However it seems to me that his reasoning is actually correct in the sense ...
- 1,667
98
votes
12
answers
12k
views
Who Are The Bayesians?
As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
- 25.7k
76
votes
11
answers
12k
views
Is there any *mathematical* basis for the Bayesian vs frequentist debate?
It says on Wikipedia that:
the mathematics [of probability] is largely independent of any interpretation of probability.
Question: Then if we want to be mathematically correct, shouldn't we disallow ...
- 6,169
63
votes
1
answer
25k
views
Can someone explain the concept of 'exchangeability'?
I see the concept of 'exchangeability' being used in different contexts (e.g., bayesian models) but I have never understood the term very well.
What does this concept mean?
Under what circumstances ...
- 825
52
votes
6
answers
6k
views
Eliciting priors from experts
How should I elicit prior distributions from experts when fitting a Bayesian model?
- 12.4k
39
votes
6
answers
7k
views
If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?
I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless:
Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
- 793
6
votes
2
answers
4k
views
Normalizing constant irrelevant in Bayes theorem? [duplicate]
I've been reviewing Bayesian literature in an attempt to utilize Bayesian inference for hypothesis testing when I have very well established priors, but there's one thing I cannot get my head around:
...
- 323
47
votes
5
answers
84k
views
What exactly is a Bayesian model?
Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad.
So what exactly is a Bayesian model?
- 2,588
30
votes
1
answer
6k
views
Computation of the marginal likelihood from MCMC samples
This is a recurring question (see this post, this post and this post), but I have a different spin.
Suppose I have a bunch of samples from a generic MCMC sampler. For each sample $\theta$, I know the ...
- 5,166
14
votes
1
answer
18k
views
Is Bayesian Ridge Regression another name of Bayesian Linear Regression?
I searched about Bayesian Ridge Regression on Internet but most of the result I got is about Bayesian Linear Regression. I wonder if it's both the same things because the formula look quite similar
- 315
11
votes
2
answers
16k
views
How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?
I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
- 229
27
votes
3
answers
2k
views
Having a conjugate prior: Deep property or mathematical accident?
Some distributions have conjugate priors and some do not. Is this distinction just an accident? That is, you do the math, and it works out one way or the other, but it does not really tell you ...
- 3,047
11
votes
1
answer
577
views
Can we think of a probability in both the classical and subjective sense simultaneously?
I'm a statistics student. I am trying to understand the classical and objective definitions of probability and how they are related to frequentist and Bayesian inference. It's not obvious to me why ...
- 354
2
votes
1
answer
759
views
could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?
I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution.
To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
- 1,807
32
votes
4
answers
14k
views
Why don't Bayesian methods require multiple testing corrections?
Andrew Gelman wrote an extensive article on why Bayesian AB testing doesn't require multiple hypothesis correction: Why We (Usually) Don’t Have to Worry
About Multiple Comparisons, 2012.
I don't ...
user46925
25
votes
4
answers
7k
views
Weakly informative prior distributions for scale parameters
I have been using log normal distributions as prior distributions for scale parameters (for normal distributions, t distributions etc.) when I have a rough idea about what the scale should be, but ...
- 4,432