Questions tagged [bayesian]

Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.

Filter by
Sorted by
Tagged with
97 votes
4 answers

What is an "uninformative prior"? Can we ever have one with truly no information?

Inspired by a comment from this question: What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior? I generally see the prior in ...
Fomite's user avatar
  • 22k
318 votes
10 answers

What's the difference between a confidence interval and a credible interval?

Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
Matt Parker's user avatar
  • 5,887
14 votes
2 answers

Bayesian logit model - intuitive explanation?

I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad. What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
BCLC's user avatar
  • 2,394
246 votes
38 answers

What is the best introductory Bayesian statistics textbook?

Which is the best introductory textbook for Bayesian statistics? One book per answer, please.
26 votes
5 answers

What does "likelihood is only defined up to a multiplicative constant of proportionality" mean in practice?

I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners. As a likelihood example, they ...
kmm's user avatar
  • 557
432 votes
14 answers

Bayesian and frequentist reasoning in plain English

How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
Daniel Vassallo's user avatar
46 votes
3 answers

Do Bayesian priors become irrelevant with large sample size?

When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
pixels's user avatar
  • 619
85 votes
2 answers

Bayes regression: how is it done in comparison to standard regression?

I got some questions about the Bayesian regression: Given a standard regression as $y = \beta_0 + \beta_1 x + \varepsilon$. If I want to change this into a Bayesian regression, do I need prior ...
TinglTanglBob's user avatar
14 votes
3 answers

How exactly do Bayesians define (or interpret?) probability?

Part of a series of trying to understand Bayesian vs frequentist: 1 2 3 4 5 6 7 I think I get the difference of how Bayesians and frequentists approach choosing between hypotheses, but I'm not quite ...
BCLC's user avatar
  • 2,394
101 votes
9 answers

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic: Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
Dikran Marsupial's user avatar
48 votes
7 answers

Combining probabilities/information from different sources

Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the ...
Biela Diela's user avatar
14 votes
2 answers

Linear discriminant analysis and Bayes rule: classification

What is the relation between Linear discriminant analysis and Bayes rule? I understand that LDA is used in classification by trying to minimize the ratio of within group variance and between group ...
zca0's user avatar
  • 841
10 votes
1 answer

Trying to Estimate Disease Prevalence from Fragmentary Test Results

In response to the spread of COVID-19 disease, all Californians were ordered on 19 March 2020 to stay at home, except for such necessary errands as trips to grocery stores, pharmacies, etc. On 21 ...
BruceET's user avatar
  • 54.6k
34 votes
5 answers

Wikipedia entry on likelihood seems ambiguous

I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.) It starts from the Wikipedia page on likelihood. They say ...
Creatron's user avatar
  • 1,587
53 votes
9 answers

Bayesian vs frequentist Interpretations of Probability

Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability? From what I understand: The frequentists view is that the data is a repeatable ...
BYS2's user avatar
  • 1,475
154 votes
3 answers

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
Bob's user avatar
  • 1,541
48 votes
2 answers

What exactly is the alpha in the Dirichlet distribution?

I'm fairly new to Bayesian statistics and I came across a corrected correlation measure, SparCC, that uses the Dirichlet process in the backend of it's algorithm. I have been trying to go through the ...
O.rka's user avatar
  • 1,412
29 votes
4 answers

Why is a normalizing factor required in Bayes’ Theorem?

Bayes theorem goes $$ P(\textrm{model}|\textrm{data}) = \frac{P(\textrm{model}) \times P(\textrm{data}|\textrm{model})}{P(\textrm{data})} $$ This is all fine. But, I've read somewhere: Basically, ...
Sreejith Ramakrishnan's user avatar
64 votes
11 answers

Examples of Bayesian and frequentist approach giving different answers

Note: I am aware of philosophical differences between Bayesian and frequentist statistics. For example "what is the probability that the coin on the table is heads" doesn't make sense in ...
33 votes
3 answers

Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

My understanding is that when using a Bayesian approach to estimate parameter values: The posterior distribution is the combination of the prior distribution and the likelihood distribution. We ...
Dave's user avatar
  • 2,441
11 votes
5 answers

Interpretation of Bayes Theorem applied to positive mammography results

I'm trying to wrap my head around the result of Bayes Theorem applied to the classic mammogram example, with the twist of the mammogram being perfect. That is, Incidence of cancer: $.01$ ...
user2666425's user avatar
52 votes
3 answers

What kind of information is Fisher information?

Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic ...
Stan Shunpike's user avatar
276 votes
12 answers

How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?

Maybe the concept, why it's used, and an example.
Neil McGuigan's user avatar
55 votes
1 answer

What are posterior predictive checks and what makes them useful?

I understand what the posterior predictive distribution is, and I have been reading about posterior predictive checks, although it isn't clear to me what it does yet. What exactly is the posterior ...
Amelio Vazquez-Reina's user avatar
37 votes
1 answer

Equivalence between least squares and MLE in Gaussian model

I am new to Machine Learning, and am trying to learn it on my own. Recently I was reading through some lecture notes and had a basic question. Slide 13 says that "Least Square Estimate is same as ...
Andy's user avatar
  • 1,683
25 votes
3 answers

Bayesian updating with new data

How do we go about calculating a posterior with a prior N~(a, b) after observing n data points? I assume that we have to calculate the sample mean and variance of the data points and do some sort of ...
statstudent's user avatar
52 votes
7 answers

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
user avatar
43 votes
2 answers

Why should we use t errors instead of normal errors?

In this blog post by Andrew Gelman, there is the following passage: The Bayesian models of 50 years ago seem hopelessly simple (except, of course, for simple problems), and I expect the Bayesian ...
Potato's user avatar
  • 1,075
21 votes
1 answer

Choosing between uninformative beta priors

I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
Mateus's user avatar
  • 211
116 votes
10 answers

ASA discusses limitations of $p$-values - what are the alternatives?

We already have multiple threads tagged as p-values that reveal lots of misunderstandings about them. Ten months ago we had a thread about psychological journal that "banned" $p$-values, now American ...
Tim's user avatar
  • 134k
23 votes
3 answers

Normalizing constant in Bayes theorem

I read that in Bayes rule, the denominator $\Pr(\textrm{data})$ of $$\Pr(\text{parameters} \mid \text{data}) = \frac{\Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\text{parameters})}{\Pr(\text{...
amateur's user avatar
  • 384
45 votes
2 answers

Why is Laplace prior producing sparse solutions?

I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero. I know how these priors look ...
Dmitry Smirnov's user avatar
20 votes
2 answers

Effective Sample Size for posterior inference from MCMC sampling

When obtaining MCMC samples to make inference on a particular parameter, what are good guides for the minimum number of effective samples that one should aim for? And, does this advice change as the ...
Matt Albrecht's user avatar
13 votes
1 answer

Aside from the exponential family, where else can conjugate priors come from?

Do all conjugate priors have to come from the exponential family? If not, what other families are known to have/produce conjugate priors?
Josh's user avatar
  • 3,908
135 votes
14 answers

What's wrong with XKCD's Frequentists vs. Bayesians comic?

This xkcd comic (Frequentists vs. Bayesians) makes fun of a frequentist statistician who derives an obviously wrong result. However it seems to me that his reasoning is actually correct in the sense ...
repied2's user avatar
  • 1,657
98 votes
12 answers

Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
Antoni Parellada's user avatar
75 votes
11 answers

Is there any *mathematical* basis for the Bayesian vs frequentist debate?

It says on Wikipedia that: the mathematics [of probability] is largely independent of any interpretation of probability. Question: Then if we want to be mathematically correct, shouldn't we disallow ...
Chill2Macht's user avatar
  • 6,099
63 votes
1 answer

Can someone explain the concept of 'exchangeability'?

I see the concept of 'exchangeability' being used in different contexts (e.g., bayesian models) but I have never understood the term very well. What does this concept mean? Under what circumstances ...
sxv's user avatar
  • 825
52 votes
6 answers

Eliciting priors from experts

How should I elicit prior distributions from experts when fitting a Bayesian model?
csgillespie's user avatar
  • 12.1k
39 votes
6 answers

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
pomodoro's user avatar
  • 793
6 votes
2 answers

Normalizing constant irrelevant in Bayes theorem? [duplicate]

I've been reviewing Bayesian literature in an attempt to utilize Bayesian inference for hypothesis testing when I have very well established priors, but there's one thing I cannot get my head around: ...
Tom.Rampley's user avatar
46 votes
5 answers

What exactly is a Bayesian model?

Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad. So what exactly is a Bayesian model?
Sibbs Gambling's user avatar
30 votes
1 answer

Computation of the marginal likelihood from MCMC samples

This is a recurring question (see this post, this post and this post), but I have a different spin. Suppose I have a bunch of samples from a generic MCMC sampler. For each sample $\theta$, I know the ...
lacerbi's user avatar
  • 5,156
14 votes
1 answer

Is Bayesian Ridge Regression another name of Bayesian Linear Regression?

I searched about Bayesian Ridge Regression on Internet but most of the result I got is about Bayesian Linear Regression. I wonder if it's both the same things because the formula look quite similar
Thien's user avatar
  • 315
11 votes
2 answers

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
user1516425's user avatar
32 votes
4 answers

Why don't Bayesian methods require multiple testing corrections?

Andrew Gelman wrote an extensive article on why Bayesian AB testing doesn't require multiple hypothesis correction: Why We (Usually) Don’t Have to Worry About Multiple Comparisons, 2012. I don't ...
user avatar
27 votes
3 answers

Having a conjugate prior: Deep property or mathematical accident?

Some distributions have conjugate priors and some do not. Is this distinction just an accident? That is, you do the math, and it works out one way or the other, but it does not really tell you ...
andrewH's user avatar
  • 3,037
2 votes
1 answer

could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?

I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution. To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
JJJohn's user avatar
  • 1,797
25 votes
4 answers

Weakly informative prior distributions for scale parameters

I have been using log normal distributions as prior distributions for scale parameters (for normal distributions, t distributions etc.) when I have a rough idea about what the scale should be, but ...
John Salvatier's user avatar
16 votes
6 answers

Does standardising independent variables reduce collinearity?

I've come across a very good text on Bayes/MCMC. IT suggests that a standardisation of your independent variables will make an MCMC (Metropolis) algorithm more efficient, but also that it may reduce (...
user avatar

2 3 4 5