Skip to main content

Questions tagged [bayesian]

Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.

Filter by
Sorted by
Tagged with
107 votes
4 answers

What is an "uninformative prior"? Can we ever have one with truly no information?

Inspired by a comment from this question: What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior? I generally see the prior in ...
Fomite's user avatar
  • 23.5k
328 votes
10 answers

What's the difference between a confidence interval and a credible interval?

Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
Matt Parker's user avatar
  • 6,097
14 votes
2 answers

Bayesian logit model - intuitive explanation?

I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad. What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
BCLC's user avatar
  • 2,444
247 votes
38 answers

What is the best introductory Bayesian statistics textbook?

Which is the best introductory textbook for Bayesian statistics? One book per answer, please.
26 votes
5 answers

What does "likelihood is only defined up to a multiplicative constant of proportionality" mean in practice?

I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners. As a likelihood example, they ...
kmm's user avatar
  • 557
436 votes
14 answers

Bayesian and frequentist reasoning in plain English

How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
Daniel Vassallo's user avatar
46 votes
3 answers

Do Bayesian priors become irrelevant with large sample size?

When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
pixels's user avatar
  • 649
15 votes
3 answers

How exactly do Bayesians define (or interpret?) probability?

Part of a series of trying to understand Bayesian vs frequentist: 1 2 3 4 5 6 7 I think I get the difference of how Bayesians and frequentists approach choosing between hypotheses, but I'm not quite ...
BCLC's user avatar
  • 2,444
102 votes
9 answers

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic: Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
Dikran Marsupial's user avatar
86 votes
2 answers

Bayes regression: how is it done in comparison to standard regression?

I got some questions about the Bayesian regression: Given a standard regression as $y = \beta_0 + \beta_1 x + \varepsilon$. If I want to change this into a Bayesian regression, do I need prior ...
TinglTanglBob's user avatar
15 votes
2 answers

Linear discriminant analysis and Bayes rule: classification

What is the relation between Linear discriminant analysis and Bayes rule? I understand that LDA is used in classification by trying to minimize the ratio of within group variance and between group ...
zca0's user avatar
  • 861
49 votes
7 answers

Combining probabilities/information from different sources

Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the ...
Biela Diela's user avatar
10 votes
1 answer

Trying to Estimate Disease Prevalence from Fragmentary Test Results

In response to the spread of COVID-19 disease, all Californians were ordered on 19 March 2020 to stay at home, except for such necessary errands as trips to grocery stores, pharmacies, etc. On 21 ...
BruceET's user avatar
  • 57k
56 votes
9 answers

Bayesian vs frequentist Interpretations of Probability

Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability? From what I understand: The frequentists view is that the data is a repeatable ...
BYS2's user avatar
  • 1,505
36 votes
5 answers

Wikipedia entry on likelihood seems ambiguous

I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.) It starts from the Wikipedia page on likelihood. They say ...
Creatron's user avatar
  • 1,685
155 votes
3 answers

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
Bob's user avatar
  • 1,551
65 votes
11 answers

Examples of Bayesian and frequentist approach giving different answers

Note: I am aware of philosophical differences between Bayesian and frequentist statistics. For example "what is the probability that the coin on the table is heads" doesn't make sense in ...
49 votes
2 answers

What exactly is the alpha in the Dirichlet distribution?

I'm fairly new to Bayesian statistics and I came across a corrected correlation measure, SparCC, that uses the Dirichlet process in the backend of it's algorithm. I have been trying to go through the ...
O.rka's user avatar
  • 1,472
29 votes
4 answers

Why is a normalizing factor required in Bayes’ Theorem?

Bayes theorem goes $$ P(\textrm{model}|\textrm{data}) = \frac{P(\textrm{model}) \times P(\textrm{data}|\textrm{model})}{P(\textrm{data})} $$ This is all fine. But, I've read somewhere: Basically, ...
Sreejith Ramakrishnan's user avatar
59 votes
3 answers

What kind of information is Fisher information?

Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic ...
Stan Shunpike's user avatar
35 votes
3 answers

Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

My understanding is that when using a Bayesian approach to estimate parameter values: The posterior distribution is the combination of the prior distribution and the likelihood distribution. We ...
Dave's user avatar
  • 2,651
11 votes
5 answers

Interpretation of Bayes Theorem applied to positive mammography results

I'm trying to wrap my head around the result of Bayes Theorem applied to the classic mammogram example, with the twist of the mammogram being perfect. That is, Incidence of cancer: $.01$ ...
user2666425's user avatar
277 votes
12 answers

How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?

Maybe the concept, why it's used, and an example.
Neil McGuigan's user avatar
28 votes
3 answers

Bayesian updating with new data

How do we go about calculating a posterior with a prior N~(a, b) after observing n data points? I assume that we have to calculate the sample mean and variance of the data points and do some sort of ...
statstudent's user avatar
59 votes
1 answer

What are posterior predictive checks and what makes them useful?

I understand what the posterior predictive distribution is, and I have been reading about posterior predictive checks, although it isn't clear to me what it does yet. What exactly is the posterior ...
Amelio Vazquez-Reina's user avatar
53 votes
7 answers

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
user avatar
116 votes
10 answers

ASA discusses limitations of $p$-values - what are the alternatives?

We already have multiple threads tagged as p-values that reveal lots of misunderstandings about them. Ten months ago we had a thread about psychological journal that "banned" $p$-values, now American ...
Tim's user avatar
  • 140k
44 votes
2 answers

Why should we use t errors instead of normal errors?

In this blog post by Andrew Gelman, there is the following passage: The Bayesian models of 50 years ago seem hopelessly simple (except, of course, for simple problems), and I expect the Bayesian ...
Potato's user avatar
  • 1,105
37 votes
1 answer

Equivalence between least squares and MLE in Gaussian model

I am new to Machine Learning, and am trying to learn it on my own. Recently I was reading through some lecture notes and had a basic question. Slide 13 says that "Least Square Estimate is same as ...
Andy's user avatar
  • 1,683
21 votes
1 answer

Choosing between uninformative beta priors

I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
Mateus's user avatar
  • 211
136 votes
14 answers

What's wrong with XKCD's Frequentists vs. Bayesians comic?

This xkcd comic (Frequentists vs. Bayesians) makes fun of a frequentist statistician who derives an obviously wrong result. However it seems to me that his reasoning is actually correct in the sense ...
repied2's user avatar
  • 1,667
47 votes
2 answers

Why is Laplace prior producing sparse solutions?

I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero. I know how these priors look ...
Dmitry Smirnov's user avatar
41 votes
6 answers

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
pomodoro's user avatar
  • 813
23 votes
3 answers

Normalizing constant in Bayes theorem

I read that in Bayes rule, the denominator $\Pr(\textrm{data})$ of $$\Pr(\text{parameters} \mid \text{data}) = \frac{\Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\text{parameters})}{\Pr(\text{...
amateur's user avatar
  • 384
14 votes
1 answer

Aside from the exponential family, where else can conjugate priors come from?

Do all conjugate priors have to come from the exponential family? If not, what other families are known to have/produce conjugate priors?
Josh's user avatar
  • 4,518
20 votes
2 answers

Effective Sample Size for posterior inference from MCMC sampling

When obtaining MCMC samples to make inference on a particular parameter, what are good guides for the minimum number of effective samples that one should aim for? And, does this advice change as the ...
Matt Albrecht's user avatar
12 votes
1 answer

Can we think of a probability in both the classical and subjective sense simultaneously?

I'm a statistics student. I am trying to understand the classical and objective definitions of probability and how they are related to frequentist and Bayesian inference. It's not obvious to me why ...
Kareem Carr's user avatar
6 votes
2 answers

Normalizing constant irrelevant in Bayes theorem? [duplicate]

I've been reviewing Bayesian literature in an attempt to utilize Bayesian inference for hypothesis testing when I have very well established priors, but there's one thing I cannot get my head around: ...
Tom.Rampley's user avatar
97 votes
12 answers

Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
Antoni Parellada's user avatar
78 votes
11 answers

Is there any *mathematical* basis for the Bayesian vs frequentist debate?

It says on Wikipedia that: the mathematics [of probability] is largely independent of any interpretation of probability. Question: Then if we want to be mathematically correct, shouldn't we disallow ...
Chill2Macht's user avatar
  • 6,369
67 votes
1 answer

Can someone explain the concept of 'exchangeability'?

I see the concept of 'exchangeability' being used in different contexts (e.g., bayesian models) but I have never understood the term very well. What does this concept mean? Under what circumstances ...
sxv's user avatar
  • 865
55 votes
6 answers

Eliciting priors from experts

How should I elicit prior distributions from experts when fitting a Bayesian model?
csgillespie's user avatar
  • 13.1k
31 votes
1 answer

Computation of the marginal likelihood from MCMC samples

This is a recurring question (see this post, this post and this post), but I have a different spin. Suppose I have a bunch of samples from a generic MCMC sampler. For each sample $\theta$, I know the ...
lacerbi's user avatar
  • 5,226
2 votes
1 answer

could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?

I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution. To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
JJJohn's user avatar
  • 1,995
47 votes
5 answers

What exactly is a Bayesian model?

Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad. So what exactly is a Bayesian model?
Sibbs Gambling's user avatar
24 votes
6 answers

When are Bayesian methods preferable to Frequentist?

I really want to learn about Bayesian techniques, so I have been trying to teach myself a bit. However, I am having a hard time seeing when using Bayesian techniques ever confer an advantage over ...
HFBrowning's user avatar
  • 1,296
14 votes
1 answer

Is Bayesian Ridge Regression another name of Bayesian Linear Regression?

I searched about Bayesian Ridge Regression on Internet but most of the result I got is about Bayesian Linear Regression. I wonder if it's both the same things because the formula look quite similar
Thien's user avatar
  • 315
11 votes
2 answers

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
user1516425's user avatar
27 votes
3 answers

Having a conjugate prior: Deep property or mathematical accident?

Some distributions have conjugate priors and some do not. Is this distinction just an accident? That is, you do the math, and it works out one way or the other, but it does not really tell you ...
andrewH's user avatar
  • 3,157
9 votes
2 answers

What does it mean intuitively to know a pdf "up to a constant"?

I've seen this mentioned numerous times, most recently in motivating the MCMC method and description of the Metropolis-Hastings algorithm. The text (Simulation and the Monte Carlo Method, Second ...
Dingo Kilo's user avatar

2 3 4 5