Questions tagged [prior]

In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior distribution represents a greater degree of information.

Filter by
Sorted by
Tagged with
87
votes
4answers
31k views

What is an “uninformative prior”? Can we ever have one with truly no information?

Inspired by a comment from this question: What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior? I generally see the prior in ...
33
votes
3answers
8k views

Do Bayesian priors become irrelevant with large sample size?

When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
141
votes
3answers
203k views

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
20
votes
1answer
10k views

Choosing between uninformative beta priors

I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
47
votes
7answers
6k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
36
votes
6answers
5k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
49
votes
6answers
5k views

Eliciting priors from experts

How should I elicit prior distributions from experts when fitting a Bayesian model?
34
votes
2answers
18k views

Why is Laplace prior producing sparse solutions?

I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero. I know how these priors look ...
24
votes
4answers
6k views

Weakly informative prior distributions for scale parameters

I have been using log normal distributions as prior distributions for scale parameters (for normal distributions, t distributions etc.) when I have a rough idea about what the scale should be, but ...
2
votes
2answers
2k views

How to define prior for beta-binomial A/B test

I would like to run an A/B test using a Bayesian beta-binomial model whereby I would state probabilities such as $P(p_B>p_A)$ in place of using a traditional T-test. I've read that the prior should ...
8
votes
1answer
4k views

How do I complete the square with normal likelihood and normal prior?

How do I complete the square from the point I have left off at, and is this correct so far? I have a normal prior for $\beta$ of the form $p(\beta|\sigma^2)\sim \mathcal{N}(0,\sigma^2V)$, to get: $...
66
votes
5answers
23k views

Why is the Jeffreys prior useful?

I understand that the Jeffreys prior is invariant under re-parameterization. However, what I don't understand is why this property is desired. Why wouldn't you want the prior to change under a change ...
28
votes
4answers
5k views

Why are Jeffreys priors considered noninformative?

Consider a Jeffreys prior where $p(\theta) \propto \sqrt{|i(\theta)|}$, where $i$ is the Fisher information. I keep seeing this prior being mentioned as a uninformative prior, but I never saw an ...
1
vote
1answer
3k views

Bayesian updating with conjugate priors using the closed form expressions

I have one two data sets of scalar values: one large data set (about 700 data points) and one small data set (80 data points). I would like to update the large data set with the small one using the ...
29
votes
2answers
12k views

Why is Lasso penalty equivalent to the double exponential (Laplace) prior?

I have read in a number of references that the Lasso estimate for the regression parameter vector $B$ is equivalent to the posterior mode of $B$ in which the prior distribution for each $B_i$ is a ...
25
votes
3answers
14k views

How can an improper prior lead to a proper posterior distribution?

We know that in the case of a proper prior distribution, $P(\theta \mid X) = \dfrac{P(X \mid \theta)P(\theta)}{P(X)}$ $ \propto P(X \mid \theta)P(\theta)$. The usual justification for this step is ...
20
votes
3answers
27k views

How to choose prior in Bayesian parameter estimation

I know 3 methods to do parameter estimation, ML, MAP and Bayes approach. And for MAP and Bayes approach, we need to pick priors for parameters, right? Say I have this model $p(x|\alpha,\beta)$, in ...
25
votes
3answers
1k views

History of uninformative prior theory

I am writing a short theoretical essay for a Bayesian Statistics course (in an Economics M.Sc.) on uninformative priors and I am trying to understand which are the steps in the development of this ...
18
votes
2answers
677 views

Frequentism and priors

Robby McKilliam says in a comment to this post: It should be pointed out that, from the frequentists point of view, there is no reason that you can't incorporate the prior knowledge into the model. ...
5
votes
2answers
2k views

Haldane's prior Beta(0,0) - Part 1

This article$^1$ on p.16 specifies Haldane's prior as: $$p(\theta) = \frac{1}{θ(1−θ)}$$. However, other$^2$ source on p.6 specifies Haldane's prior as proportional to $\frac{1}{θ(1−θ)}$, i.e. $$p(\...
5
votes
0answers
790 views

Bayes-Poincaré solution to the Behrens-Fisher problem 2: calculations for Jeffreys’ priors [closed]

In a previous post Bayes-Poincaré solution to k-sample tests for comparison and the Behrens-Fisher problem?, the classical Bayesian and likelihoodist solutions to 2-sample tests for comparison and the ...
4
votes
0answers
987 views

Bayes-Poincaré solution to k-sample tests for comparison and the Behrens-Fisher problem?

I’d like to share and submit for (dis)approval and discussion yet another, simple but original (to the best of my knowledge) Bayesian solution to the classical problem of comparing k samples or groups,...
23
votes
2answers
11k views

Why is a $p(\sigma^2)\sim\text{IG(0.001, 0.001)}$ prior on variance considered weak?

Background One of the most commonly used weak prior on variance is the inverse-gamma with parameters $\alpha =0.001, \beta=0.001$ (Gelman 2006). However, this distribution has a 90%CI of ...
20
votes
4answers
3k views

How is the bayesian framework better in interpretation when we usually use uninformative or subjective priors?

It is often argued that the bayesian framework has a big advantage in interpretation (over frequentist), because it computes the probability of a parameter given the data - $p(\theta|x)$ instead of $p(...
14
votes
2answers
2k views

You observe k heads out of n tosses. Is the coin fair?

I was asked this question with $(n, k) = (400, 220)$ in an interview. Is there a "correct" answer? Assume the tosses are i.i.d. and the probability of heads is $p=0.5$. The distribution of the ...
11
votes
3answers
9k views

Priors for log-normal models

I am trying to determine what the most appropriate non-informative priors are for the two parameters of a log-normal distribution (for an accelerated failure time model). I had been working with a ...
8
votes
1answer
9k views

What exactly does it mean to and why must one update prior?

I'm still trying to understand prior and posterior distributions in Bayesian inference. In this question, one flips a coin. Priors: unfair is 0.1, and being fair is 0.9 Coin is flipped 10x and is ...
7
votes
1answer
6k views

Why use precision instead of variance in a prior?

I am a newcomer in the field of statistic. I am wondering about using precision instead of variance in a prior.
11
votes
2answers
4k views

What is a “Unit Information Prior”?

I've been reading Wagenmakers (2007) A practical solution to the pervasive problem of p values. I'm intrigued by the conversion of BIC values into Bayes factors and probabilities. However, so far I ...
11
votes
1answer
4k views

Neg Binomial and the Jeffreys' Prior

I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated. Okay, so the situation ...
4
votes
2answers
2k views

Specifying conditional probabilities in hybrid Bayesian networks

I am trying to get a deeper understanding of the various types of Bayesian networks. Most of the literature/lectures I've come across use discrete random variables exclusively and only mention ...
10
votes
2answers
5k views

What exactly is weakly informative prior?

Is there a precise definition of weakly informative prior? How is it different from a subjective prior with broad support?
0
votes
1answer
516 views

Definition of weakly informative prior [duplicate]

According to Gelman, a weakly informative prior is defined in the following way: We characterize a prior distribution as weakly informative if it is proper but is set up so that the information ...
23
votes
2answers
27k views

Natural interpretation for LDA hyperparameters

Can somebody explain what is the natural interpretation for LDA hyperparameters? ALPHA and BETA are parameters of Dirichlet ...
26
votes
2answers
7k views

Bayesian batting average prior

I wanted to ask a question inspired by an excellent answer to the query about the intuition for the beta distribution. I wanted to get a better understanding of the derivation for the prior ...
10
votes
1answer
15k views

How does the beta prior affect the posterior under a binomial likelihood

I have two questions, Question 1: How can I show that the posterior distribution is a beta distribution if the likelihood is binomial and the prior is a beta Question 2: How does choices the prior ...
20
votes
2answers
6k views

What is the relationship between sample size and the influence of prior on posterior?

If we have a small sample size, will the prior distribution influence the posterior distribution a lot?
28
votes
5answers
10k views

Is a vague prior the same as a non-informative prior?

This is a question about terminology. Is a "vague prior" the same as a non-informative prior, or is there some difference between the two? My impression is that they are same (from looking up vague ...
20
votes
1answer
2k views

Is there a Bayesian interpretation of linear regression with simultaneous L1 and L2 regularization (aka elastic net)?

It's well known that linear regression with an $l^2$ penalty is equivalent to finding the MAP estimate given a Gaussian prior on the coefficients. Similarly, using an $l^1$ penalty is equivalent to ...
15
votes
2answers
8k views

Flat, conjugate, and hyper- priors. What are they?

I am currently reading about Bayesian Methods in Computation Molecular Evolution by Yang. In section 5.2 it talks about priors, and specifically Non-informative/flat/vague/diffuse, conjugate, and ...
12
votes
4answers
3k views

Why I should use Bayesian inference with uninformative prior? [duplicate]

I am a Ph.D. student and currently I am studying Bayesian inference concerning vector autoregressive models. A lot of researchers when talking about uninformative prior, conclude that the results of ...
8
votes
2answers
2k views

When does the maximum likelihood correspond to a reference prior?

I have been reading James V. Stone's very nice books "Bayes' Rule" and "Information Theory". I want to know which sections of the books I did not understand and thus need to re-...
5
votes
2answers
5k views

What does “a distribution over distributions” mean?

I am reading a paper about Dirichlet Processes, and it said "A Dirichlet Process is also a distribution over distributions." What does that mean?
4
votes
3answers
502 views

Bayesian prior via cross validation

I have a particular problem where I am using Bayesian techniques to estimate parameters of a distribution of a random variable. I would like to use an external source of data to determine an ...
11
votes
2answers
610 views

Can a proper prior and exponentiated likelihood lead to an improper posterior?

(This question is inspired by this comment from Xi'an.) It is well known that if the prior distribution $\pi(\theta)$ is proper and the likelihood $L(\theta | x)$ is well-defined, then the posterior ...
8
votes
1answer
3k views

Why is uniform prior on log(x) equal to 1/x prior on x?

I'm trying to understand Jeffreys prior. One application is for 'scale' variables like the standard deviation $\sigma$ (or its square, the variance $\sigma^2$) of Gaussian distributions. It is often ...
7
votes
2answers
2k views

Jeffreys prior for continuous uniform distribution

A nonnegative random variable $x$ has a continuous uniform distribution in the interval $(0,\theta)$. Therefore, the likelihood is given by: $f(x|\theta) = \frac{1}{\theta}I(x\leq\theta)$, where $I$ ...
5
votes
3answers
618 views

What methods can be used to specify priors from data?

Background I am generally interested in learning appropriate methods of using data to specify priors. A previous question asks how to elicit priors from experts and received some good recommendations....
4
votes
1answer
967 views

Interpreting prior and posterior

I am bit puzzled on how we can interpret the posterior. Assume a coin which is 0.1 probable to be unfair. So our prior probability on the coin being unfair is 0.1, and being fair is 0.9. Also by ...
6
votes
2answers
2k views

Definition of improper priors

If a prior integrates to a finite constant that is not 1, is it still considered proper? Is a prior only improper if it integrates to an nonfinite value?