Questions tagged [prior]

In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior distribution represents a greater degree of information.

Filter by
Sorted by
Tagged with
89 votes
4 answers
37k views

What is an "uninformative prior"? Can we ever have one with truly no information?

Inspired by a comment from this question: What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior? I generally see the prior in ...
  • 21.5k
40 votes
3 answers
10k views

Do Bayesian priors become irrelevant with large sample size?

When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
  • 559
149 votes
3 answers
222k views

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
  • 1,491
21 votes
1 answer
12k views

Choosing between uninformative beta priors

I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
  • 211
50 votes
7 answers
7k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
's user avatar
44 votes
2 answers
23k views

Why is Laplace prior producing sparse solutions?

I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero. I know how these priors look ...
49 votes
6 answers
5k views

Eliciting priors from experts

How should I elicit prior distributions from experts when fitting a Bayesian model?
  • 11.9k
38 votes
6 answers
6k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
  • 723
24 votes
4 answers
6k views

Weakly informative prior distributions for scale parameters

I have been using log normal distributions as prior distributions for scale parameters (for normal distributions, t distributions etc.) when I have a rough idea about what the scale should be, but ...
8 votes
1 answer
5k views

How do I complete the square with normal likelihood and normal prior?

How do I complete the square from the point I have left off at, and is this correct so far? I have a normal prior for $\beta$ of the form $p(\beta|\sigma^2)\sim \mathcal{N}(0,\sigma^2V)$, to get: $...
  • 81
2 votes
2 answers
3k views

How to define prior for beta-binomial A/B test

I would like to run an A/B test using a Bayesian beta-binomial model whereby I would state probabilities such as $P(p_B>p_A)$ in place of using a traditional T-test. I've read that the prior should ...
  • 21
24 votes
3 answers
33k views

How to choose prior in Bayesian parameter estimation

I know 3 methods to do parameter estimation, ML, MAP and Bayes approach. And for MAP and Bayes approach, we need to pick priors for parameters, right? Say I have this model $p(x|\alpha,\beta)$, in ...
  • 3,175
33 votes
2 answers
15k views

Why is Lasso penalty equivalent to the double exponential (Laplace) prior?

I have read in a number of references that the Lasso estimate for the regression parameter vector $B$ is equivalent to the posterior mode of $B$ in which the prior distribution for each $B_i$ is a ...
  • 1,217
71 votes
5 answers
25k views

Why is the Jeffreys prior useful?

I understand that the Jeffreys prior is invariant under re-parameterization. However, what I don't understand is why this property is desired. Why wouldn't you want the prior to change under a change ...
  • 943
2 votes
1 answer
3k views

Bayesian updating with conjugate priors using the closed form expressions

I have one two data sets of scalar values: one large data set (about 700 data points) and one small data set (80 data points). I would like to update the large data set with the small one using the ...
  • 133
31 votes
4 answers
6k views

Why are Jeffreys priors considered noninformative?

Consider a Jeffreys prior where $p(\theta) \propto \sqrt{|i(\theta)|}$, where $i$ is the Fisher information. I keep seeing this prior being mentioned as a uninformative prior, but I never saw an ...
  • 809
25 votes
3 answers
15k views

How can an improper prior lead to a proper posterior distribution?

We know that in the case of a proper prior distribution, $P(\theta \mid X) = \dfrac{P(X \mid \theta)P(\theta)}{P(X)}$ $ \propto P(X \mid \theta)P(\theta)$. The usual justification for this step is ...
  • 3,012
27 votes
3 answers
1k views

History of uninformative prior theory

I am writing a short theoretical essay for a Bayesian Statistics course (in an Economics M.Sc.) on uninformative priors and I am trying to understand which are the steps in the development of this ...
  • 2,520
18 votes
2 answers
765 views

Frequentism and priors

Robby McKilliam says in a comment to this post: It should be pointed out that, from the frequentists point of view, there is no reason that you can't incorporate the prior knowledge into the model. ...
  • 811
8 votes
1 answer
7k views

Why use precision instead of variance in a prior?

I am a newcomer in the field of statistic. I am wondering about using precision instead of variance in a prior.
5 votes
0 answers
833 views

Bayes-Poincaré solution to the Behrens-Fisher problem 2: calculations for Jeffreys’ priors [closed]

In a previous post Bayes-Poincaré solution to k-sample tests for comparison and the Behrens-Fisher problem?, the classical Bayesian and likelihoodist solutions to 2-sample tests for comparison and the ...
's user avatar
5 votes
2 answers
4k views

Haldane's prior Beta(0,0) - Part 1

This article$^1$ on p.16 specifies Haldane's prior as: $$p(\theta) = \frac{1}{θ(1−θ)}$$. However, other$^2$ source on p.6 specifies Haldane's prior as proportional to $\frac{1}{θ(1−θ)}$, i.e. $$p(\...
  • 571
4 votes
0 answers
1k views

Bayes-Poincaré solution to k-sample tests for comparison and the Behrens-Fisher problem?

I’d like to share and submit for (dis)approval and discussion yet another, simple but original (to the best of my knowledge) Bayesian solution to the classical problem of comparing k samples or groups,...
's user avatar
23 votes
2 answers
12k views

Why is a $p(\sigma^2)\sim\text{IG(0.001, 0.001)}$ prior on variance considered weak?

Background One of the most commonly used weak prior on variance is the inverse-gamma with parameters $\alpha =0.001, \beta=0.001$ (Gelman 2006). However, this distribution has a 90%CI of ...
20 votes
4 answers
3k views

How is the bayesian framework better in interpretation when we usually use uninformative or subjective priors?

It is often argued that the bayesian framework has a big advantage in interpretation (over frequentist), because it computes the probability of a parameter given the data - $p(\theta|x)$ instead of $p(...
  • 5,915
15 votes
2 answers
3k views

You observe k heads out of n tosses. Is the coin fair?

I was asked this question with $(n, k) = (400, 220)$ in an interview. Is there a "correct" answer? Assume the tosses are i.i.d. and the probability of heads is $p=0.5$. The distribution of the ...
  • 3,840
8 votes
1 answer
12k views

What exactly does it mean to and why must one update prior?

I'm still trying to understand prior and posterior distributions in Bayesian inference. In this question, one flips a coin. Priors: unfair is 0.1, and being fair is 0.9 Coin is flipped 10x and is ...
  • 2,216
12 votes
3 answers
10k views

Priors for log-normal models

I am trying to determine what the most appropriate non-informative priors are for the two parameters of a log-normal distribution (for an accelerated failure time model). I had been working with a ...
  • 709
11 votes
2 answers
5k views

What is a "Unit Information Prior"?

I've been reading Wagenmakers (2007) A practical solution to the pervasive problem of p values. I'm intrigued by the conversion of BIC values into Bayes factors and probabilities. However, so far I ...
11 votes
1 answer
5k views

Neg Binomial and the Jeffreys' Prior

I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated. Okay, so the situation ...
  • 2,308
11 votes
2 answers
6k views

What exactly is weakly informative prior?

Is there a precise definition of weakly informative prior? How is it different from a subjective prior with broad support?
  • 1,580
5 votes
1 answer
92 views

Numbers of draws on a modified Bernouilli process

Here is the setup: Bob runs an experiment: he flips a coin N times (between 0 and +$\infty$). The coin has a probability p of landing on heads. Bob starts with zero points. For each head, Bob scores a ...
  • 341
0 votes
1 answer
647 views

Definition of weakly informative prior [duplicate]

According to Gelman, a weakly informative prior is defined in the following way: We characterize a prior distribution as weakly informative if it is proper but is set up so that the information ...
  • 2,027
26 votes
2 answers
29k views

Natural interpretation for LDA hyperparameters

Can somebody explain what is the natural interpretation for LDA hyperparameters? ALPHA and BETA are parameters of Dirichlet ...
27 votes
2 answers
8k views

Bayesian batting average prior

I wanted to ask a question inspired by an excellent answer to the query about the intuition for the beta distribution. I wanted to get a better understanding of the derivation for the prior ...
  • 31.6k
20 votes
2 answers
7k views

What is the relationship between sample size and the influence of prior on posterior?

If we have a small sample size, will the prior distribution influence the posterior distribution a lot?
  • 271
11 votes
2 answers
3k views

When does the maximum likelihood correspond to a reference prior?

I have been reading James V. Stone's very nice books "Bayes' Rule" and "Information Theory". I want to know which sections of the books I did not understand and thus need to re-...
  • 5,817
28 votes
5 answers
11k views

Is a vague prior the same as a non-informative prior?

This is a question about terminology. Is a "vague prior" the same as a non-informative prior, or is there some difference between the two? My impression is that they are same (from looking up vague ...
  • 809
21 votes
1 answer
2k views

Is there a Bayesian interpretation of linear regression with simultaneous L1 and L2 regularization (aka elastic net)?

It's well known that linear regression with an $l^2$ penalty is equivalent to finding the MAP estimate given a Gaussian prior on the coefficients. Similarly, using an $l^1$ penalty is equivalent to ...
15 votes
2 answers
9k views

Flat, conjugate, and hyper- priors. What are they?

I am currently reading about Bayesian Methods in Computation Molecular Evolution by Yang. In section 5.2 it talks about priors, and specifically Non-informative/flat/vague/diffuse, conjugate, and ...
  • 862
12 votes
4 answers
3k views

Why I should use Bayesian inference with uninformative prior? [duplicate]

I am a Ph.D. student and currently I am studying Bayesian inference concerning vector autoregressive models. A lot of researchers when talking about uninformative prior, conclude that the results of ...
  • 121
8 votes
4 answers
2k views

How is data generated in the Bayesian framework and what is the nature on the parameter that generates the data?

I was trying to re-learn Bayesian statistics (every time I thought I finally got it, something else pops out that I didn't consider earlier....) but it wasn't clear (to me) what the data generation ...
4 votes
3 answers
588 views

Bayesian prior via cross validation

I have a particular problem where I am using Bayesian techniques to estimate parameters of a distribution of a random variable. I would like to use an external source of data to determine an ...
  • 2,448
16 votes
3 answers
19k views

Jeffreys Prior for normal distribution with unknown mean and variance

I am reading up on prior distributions and I calculated Jeffreys prior for a sample of normally distributed random variables with unknown mean and unknown variance. According to my calculations, the ...
  • 470
6 votes
2 answers
7k views

What does "a distribution over distributions" mean?

I am reading a paper about Dirichlet Processes, and it said "A Dirichlet Process is also a distribution over distributions." What does that mean?
5 votes
6 answers
383 views

What is the point of using a Bayesian prior?

I do struggle with the most basic starting point of Bayesian statistics: why is using a prior useful? It seems to me that if anything they hurt much more than help. Moreover, Bayesians always say ...
  • 773
2 votes
1 answer
1k views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
12 votes
2 answers
658 views

Can a proper prior and exponentiated likelihood lead to an improper posterior?

(This question is inspired by this comment from Xi'an.) It is well known that if the prior distribution $\pi(\theta)$ is proper and the likelihood $L(\theta | x)$ is well-defined, then the posterior ...
  • 2,027
10 votes
1 answer
4k views

Why is uniform prior on log(x) equal to 1/x prior on x?

I'm trying to understand Jeffreys prior. One application is for 'scale' variables like the standard deviation $\sigma$ (or its square, the variance $\sigma^2$) of Gaussian distributions. It is often ...
7 votes
2 answers
3k views

Jeffreys prior for continuous uniform distribution

A nonnegative random variable $x$ has a continuous uniform distribution in the interval $(0,\theta)$. Therefore, the likelihood is given by: $f(x|\theta) = \frac{1}{\theta}I(x\leq\theta)$, where $I$ ...