Questions tagged [prior]

In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior distribution represents a greater degree of information.

Filter by
Sorted by
Tagged with
141
votes
3answers
203k views

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
87
votes
4answers
31k views

What is an “uninformative prior”? Can we ever have one with truly no information?

Inspired by a comment from this question: What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior? I generally see the prior in ...
66
votes
5answers
23k views

Why is the Jeffreys prior useful?

I understand that the Jeffreys prior is invariant under re-parameterization. However, what I don't understand is why this property is desired. Why wouldn't you want the prior to change under a change ...
49
votes
6answers
5k views

Eliciting priors from experts

How should I elicit prior distributions from experts when fitting a Bayesian model?
47
votes
7answers
6k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
36
votes
6answers
5k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
34
votes
2answers
18k views

Why is Laplace prior producing sparse solutions?

I was looking through the literature on regularization, and often see paragraphs that links L2 regulatization with Gaussian prior, and L1 with Laplace centered on zero. I know how these priors look ...
33
votes
3answers
8k views

Do Bayesian priors become irrelevant with large sample size?

When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
32
votes
1answer
20k views

What are the properties of a half Cauchy distribution?

I am currently working on a problem, where I need to develop a Markov chain Monte Carlo (MCMC) algorithm for a state space model. To be able to solve the problem, I have been given the following ...
29
votes
2answers
12k views

Why is Lasso penalty equivalent to the double exponential (Laplace) prior?

I have read in a number of references that the Lasso estimate for the regression parameter vector $B$ is equivalent to the posterior mode of $B$ in which the prior distribution for each $B_i$ is a ...
28
votes
4answers
5k views

Why are Jeffreys priors considered noninformative?

Consider a Jeffreys prior where $p(\theta) \propto \sqrt{|i(\theta)|}$, where $i$ is the Fisher information. I keep seeing this prior being mentioned as a uninformative prior, but I never saw an ...
28
votes
5answers
10k views

Is a vague prior the same as a non-informative prior?

This is a question about terminology. Is a "vague prior" the same as a non-informative prior, or is there some difference between the two? My impression is that they are same (from looking up vague ...
26
votes
2answers
7k views

Bayesian batting average prior

I wanted to ask a question inspired by an excellent answer to the query about the intuition for the beta distribution. I wanted to get a better understanding of the derivation for the prior ...
25
votes
3answers
14k views

How can an improper prior lead to a proper posterior distribution?

We know that in the case of a proper prior distribution, $P(\theta \mid X) = \dfrac{P(X \mid \theta)P(\theta)}{P(X)}$ $ \propto P(X \mid \theta)P(\theta)$. The usual justification for this step is ...
25
votes
3answers
1k views

History of uninformative prior theory

I am writing a short theoretical essay for a Bayesian Statistics course (in an Economics M.Sc.) on uninformative priors and I am trying to understand which are the steps in the development of this ...
24
votes
4answers
6k views

Weakly informative prior distributions for scale parameters

I have been using log normal distributions as prior distributions for scale parameters (for normal distributions, t distributions etc.) when I have a rough idea about what the scale should be, but ...
23
votes
2answers
27k views

Natural interpretation for LDA hyperparameters

Can somebody explain what is the natural interpretation for LDA hyperparameters? ALPHA and BETA are parameters of Dirichlet ...
23
votes
2answers
11k views

Why is a $p(\sigma^2)\sim\text{IG(0.001, 0.001)}$ prior on variance considered weak?

Background One of the most commonly used weak prior on variance is the inverse-gamma with parameters $\alpha =0.001, \beta=0.001$ (Gelman 2006). However, this distribution has a 90%CI of ...
23
votes
2answers
2k views

What is/are the implicit priors in frequentist statistics?

I've heard the notion that Jaynes claims frequentists operate with an "implicit prior". What is or are these implicit priors? Does this mean frequentist models are all special cases of Bayesian ...
22
votes
6answers
4k views

Posterior very different to prior and likelihood

If the prior and the likelihood are very different from each other, then sometimes a situation occurs where the posterior is similar to neither of them. See for example this picture, which uses normal ...
20
votes
2answers
6k views

What is the relationship between sample size and the influence of prior on posterior?

If we have a small sample size, will the prior distribution influence the posterior distribution a lot?
20
votes
1answer
10k views

Choosing between uninformative beta priors

I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
20
votes
4answers
3k views

How is the bayesian framework better in interpretation when we usually use uninformative or subjective priors?

It is often argued that the bayesian framework has a big advantage in interpretation (over frequentist), because it computes the probability of a parameter given the data - $p(\theta|x)$ instead of $p(...
20
votes
3answers
27k views

How to choose prior in Bayesian parameter estimation

I know 3 methods to do parameter estimation, ML, MAP and Bayes approach. And for MAP and Bayes approach, we need to pick priors for parameters, right? Say I have this model $p(x|\alpha,\beta)$, in ...
20
votes
1answer
883 views

What are the best ways to generate Bayesian prior estimates using beliefs of non-statisticians?

I work with a lot of qualitative researchers and designers. Many of whom interact with users and develop strong, often accurate intuitions about how the data should look. I frequently try to quantify ...
20
votes
1answer
2k views

Is there a Bayesian interpretation of linear regression with simultaneous L1 and L2 regularization (aka elastic net)?

It's well known that linear regression with an $l^2$ penalty is equivalent to finding the MAP estimate given a Gaussian prior on the coefficients. Similarly, using an $l^1$ penalty is equivalent to ...
19
votes
2answers
7k views

Parameters without defined priors in Stan

I've just started to learn to use Stan and rstan. Unless I've always been confused about how JAGS/BUGS worked, I thought you always had to define a prior ...
19
votes
1answer
5k views

Why LKJcorr is a good prior for correlation matrix?

I´m reading the chapter 13 "Adventures in Covariance" in the (superb) book Statistical Rethinking by Richard McElreath where he presents the following hierarchical model: (...
18
votes
2answers
2k views

What is the problem with empirical priors?

In literature I sometimes stumple upon the remark, that choosing priors that depend on the data itself (for example Zellners g-prior) can be criticized from a theoretical point of view. Where exactly ...
18
votes
4answers
2k views

How do Bayesian Statistics handle the absence of priors?

This question was inspired by two recent interactions I had, one here in CV, the other over at economics.se. There, I had posted an answer to the well-known "Envelope Paradox" (mind you, not as the "...
18
votes
2answers
677 views

Frequentism and priors

Robby McKilliam says in a comment to this post: It should be pointed out that, from the frequentists point of view, there is no reason that you can't incorporate the prior knowledge into the model. ...
18
votes
2answers
1k views

What is the relation behind Jeffreys Priors and a variance stabilizing transformation?

I was reading about the Jeffreys prior on wikipedia: Jeffreys Prior and saw that after each example, it describes how a variance-stabilizing transformation turns the Jeffreys prior into a uniform ...
18
votes
2answers
7k views

What prior distributions could/should be used for the variance in a hierarchical bayesisan model when the mean variance is of interest?

In his widely cited paper Prior distributions for variance parameters in hierarchical models (916 citation so far on Google Scholar) Gelman proposes that good non-informative prior distributions for ...
16
votes
3answers
15k views

Jeffreys Prior for normal distribution with unknown mean and variance

I am reading up on prior distributions and I calculated Jeffreys prior for a sample of normally distributed random variables with unknown mean and unknown variance. According to my calculations, the ...
16
votes
2answers
3k views

What's a good prior distribution for degrees of freedom in a t distribution?

I want to use a t distribution to model short interval asset returns in a bayesian model. I'd like to estimate both the degrees of freedom (along with other parameters in my model) for the ...
15
votes
2answers
8k views

Flat, conjugate, and hyper- priors. What are they?

I am currently reading about Bayesian Methods in Computation Molecular Evolution by Yang. In section 5.2 it talks about priors, and specifically Non-informative/flat/vague/diffuse, conjugate, and ...
15
votes
2answers
5k views

Ridge regression – Bayesian interpretation

I have heard that ridge regression can be derived as the mean of a posterior distribution, if the prior is adequately chosen. Is the intuition that the constraints as set on the regression ...
15
votes
3answers
4k views

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its ...
14
votes
2answers
2k views

You observe k heads out of n tosses. Is the coin fair?

I was asked this question with $(n, k) = (400, 220)$ in an interview. Is there a "correct" answer? Assume the tosses are i.i.d. and the probability of heads is $p=0.5$. The distribution of the ...
14
votes
1answer
1k views

Jeffreys prior for multiple parameters

In certain cases, the Jeffreys prior for a full multidimensional model is generaly considered as inadequate, this is for example the case in: $$ y_i=\mu + \varepsilon_i \, , $$ (where $\varepsilon \...
13
votes
2answers
418 views

Making a Bayesian prior from a frequentist result

How should one go about turning a frequentist result into a Bayesian prior? Consider the following pretty generic scenario: An experiment was conducted in the past and a result on some parameter $\...
12
votes
4answers
3k views

Why I should use Bayesian inference with uninformative prior? [duplicate]

I am a Ph.D. student and currently I am studying Bayesian inference concerning vector autoregressive models. A lot of researchers when talking about uninformative prior, conclude that the results of ...
12
votes
2answers
1k views

Why are there recommendations against using Jeffreys or entropy based priors for MCMC samplers?

On their wiki page, the developers of Stan state: Some principles we don't like: invariance, Jeffreys, entropy Instead, I see a lot of normal distribution recommendation. So far I used Bayesian ...
12
votes
1answer
515 views

When should I be worried about the Jeffreys-Lindley paradox in Bayesian model choice?

I am considering a large (but finite) space of models of varying complexity which I explore using RJMCMC. The prior on the parameter vector for each model is fairly informative. In what cases (if any)...
11
votes
3answers
9k views

Priors for log-normal models

I am trying to determine what the most appropriate non-informative priors are for the two parameters of a log-normal distribution (for an accelerated failure time model). I had been working with a ...
11
votes
2answers
610 views

Can a proper prior and exponentiated likelihood lead to an improper posterior?

(This question is inspired by this comment from Xi'an.) It is well known that if the prior distribution $\pi(\theta)$ is proper and the likelihood $L(\theta | x)$ is well-defined, then the posterior ...
11
votes
1answer
4k views

Neg Binomial and the Jeffreys' Prior

I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated. Okay, so the situation ...
11
votes
2answers
4k views

What is a “Unit Information Prior”?

I've been reading Wagenmakers (2007) A practical solution to the pervasive problem of p values. I'm intrigued by the conversion of BIC values into Bayes factors and probabilities. However, so far I ...
11
votes
1answer
507 views

Do statisticians use the Jeffreys' prior in actual applied work?

When I learned about the Jeffreys' prior in my graduate statistical inference class my professors made it sound sort of like it was interesting mostly for historical reasons rather than because anyone ...
11
votes
2answers
702 views

What is the mathematical difference between using a un-informative prior and a frequentist approach?

Un-informative priors are preferred in instances where bias is not acceptable (ie. courtrooms, etc.) However, it seems to me that it would just make sense to use a frequentist approach instead. Why ...

1
2 3 4 5
17