Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [uninformative-prior]

A prior that express lack of detailed information or lack of any information at all.

13
votes
1answer
4k views

Choosing between uninformative beta priors

I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey ...
12
votes
1answer
10k views

What is the point of non-informative priors?

Why even have non-informative priors? They don't provide information about $\theta$. So why use them? Why not only use informative priors? For example, suppose $ \theta \in [0,1]$. Then is $\theta \...
11
votes
4answers
3k views

Bayesian uninformative priors vs. frequentist null hypotheses: what's the relationship?

I came across this image in a blog post here. I was disappointed that reading the statement did not elicit the same facial expression for me as it did for this guy. So, what is meant by the ...
10
votes
3answers
2k views

What should an uninformative prior be for the slope when doing linear regression?

When performing bayesian linear regression, one needs to assign a prior for the slope $a$ and intercept $b$. Since $b$ is a location parameter it makes sense to assign an uniform prior; however, it ...
9
votes
3answers
7k views

Estimating the parameter of a uniform distribution: improper prior?

We have N samples, $X_i$, from a uniform distribution $[0,\theta]$ where $\theta$ is unknown. Estimate $\theta$ from the data. So, Bayes' rule... $f(\theta | {X_i}) = \frac{f({X_i}|\theta)f(\theta)}...
8
votes
1answer
1k views

Why is Zellner's g prior “unacceptable”?

I know Zellner's prior is using data in order to set prior information, but actually the whole model depends on the data. Is there any other reason?
8
votes
2answers
373 views

What is the mathematical difference between using a un-informative prior and a frequentist approach?

Un-informative priors are preferred in instances where bias is not acceptable (ie. courtrooms, etc.) However, it seems to me that it would just make sense to use a frequentist approach instead. Why ...
8
votes
1answer
3k views

Deriving the posterior density for a lognormal likelihood and Jeffreys's prior

The likelihood function of a lognormal distribution is: $f(x; \mu, \sigma) \propto \prod_{i_1}^n \frac{1}{\sigma x_i} \exp \left ( - \frac{(\ln{x_i} - \mu)^2}{2 \sigma^2} \right ) $ and Jeffreys's ...
8
votes
1answer
562 views

Choosing non-informative priors

I am working on a model relying on an ugly parametrized function acting as a calibration function on a part of the model. Using a Bayesian setting, I need to get non-informative priors for the ...
6
votes
1answer
2k views

Why is uniform prior on log(x) equal to 1/x prior on x?

I'm trying to understand Jeffreys prior. One application is for 'scale' variables like the standard deviation $\sigma$ (or its square, the variance $\sigma^2$) of Gaussian distributions. It is often ...
6
votes
2answers
521 views

How does one interpret the distribution over parameters in bayesian estimation?

I am new to Bayesian estimation. The assumption that the parameters are random variables seems a little unsettling to me. For example when considering a model for data, what physical interpretation ...
6
votes
2answers
154 views

Properties of MaxEnt posterior distribution for a die with prescribed average

Question: after throwing a die a large number of times and discovering that the average of the outcomes is $4$, what probability distribution one should assign to statements "the next roll will be $i$"...
6
votes
0answers
87 views

How to construct “reference priors”?

I have been reading about noninformative priors. Two of the most popular priors of this kind seem to be the Jeffreys prior and the reference prior. The Jeffreys prior has a clear construction, being ...
5
votes
1answer
2k views

Downsides of inverse Wishart prior in hierarchical models

I am working with a Bayesian hierarchical model that has a number of parameters for each experimental unit (6 parameters). I really do not know all that much about them a-priori, but it is quite ...
5
votes
1answer
357 views

Issues with Importance sampling for flat prior

I am trying to draw Bayesian inference via importance sampling for a parameter $\xi$ attached with an (unbounded) flat prior. This seems problematic as this is clearly not a probability measure but ...
5
votes
2answers
192 views

Non-informative prior for regression model

I'm looking at p. 355 of Gelman's Bayesian Data Analysis (3rd ed.), for which there is no errata, and I see this: In the normal regression model, a convenient non-informative prior distribution ...
5
votes
0answers
236 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, \...
4
votes
1answer
2k views

Define own noninformative prior in stan

In the simple case of normally distributed data with unknown mean and variance, Jeffrey's prior is given by $$p(\mu, \sigma^2)=\frac{1}{\sigma^2}.$$ How can I define such a prior in the Stan language,...
4
votes
1answer
275 views

How can a uniform prior make the posterior mean different from the MLE?

I read the following in Machine Learning: A Probabilistic Perspective: How can a uniform prior move the posterior mean? Isn't a uniform distribution supposed to not bias the result? Are there any ...
4
votes
1answer
522 views

Posterior distribution for Gamma scale parameter under the Jeffreys prior

What is the posterior distribution for parameter $b$ with $X \sim Gamma(a,b)$, under the Jeffreys prior? We can assume that $a$ is known. The Jeffreys prior is the square of the Fisher information ...
4
votes
1answer
102 views

Understanding definition of informative and uninformative prior distribution

When using the "non-informative" prior $\pi(\mu,\sigma)\propto\frac{1}{\sigma^2}$ where $\pi(\mu)\propto1$ and $\pi(\sigma^2)\propto\frac{1}{\sigma^2}$ Where is the no information for the ...
4
votes
1answer
517 views

KL divergence between an uninformative (?) Gaussian and a Gaussian

I have to calculate the KL divergence between a distribution $q$ and a prior distribution $p$, both of which are univariate Gaussians, i.e. $KL(q|p), q \sim \mathcal{N}(\mu, \sigma^2), p \sim \mathcal{...
4
votes
3answers
191 views

Why are weakly informative priors a good idea?

There are many solutions to the problem that typically not enough information is available to fully specify a prior. For all approaches (I know) but weakly informative priors I kind of understand ...
4
votes
0answers
262 views

Prior for $\lambda$ is LASSO prior?

I have a regression model with regression coefficients $\beta_j$, $j=1,...,n$, and I would like to use a LASSO prior for $\beta_j$, this is: $$\beta_j \sim Laplace(0,1/\lambda),$$ where the Laplace ...
3
votes
1answer
1k views

Laplace's law of succession using different priors

Laplace's law of succession is a well-known rule, relying on Bayes' theorem. A possible proof of the rule of succession can be found on Wikipedia. Note that for this proof we use a uniform ...
3
votes
1answer
175 views

How to quantify strength of beta prior?

How would one interpret the strength of prior belief associated with a parameter with a prior beta(10,8) distribution, compared to a beta(0,0) prior, (with data from a binomial distribution)? Some ...
3
votes
1answer
737 views

What is a good example of a non-informative prior for the uniform distribution?

I recently noticed that for non-informative priors, people usually use something like a uniform prior, which works for many different distributions. However, assuming that your likelihood is nothing ...
3
votes
1answer
414 views

Why there is no uniform prior for Box-Cox Power Transformed Normal Models

I am trying to get intuition why uniform prior like below will not work for the box-cox model. Box-cox model: $y^{(\phi)}_i \sim N(\mu, \sigma^{2})$ where $y^{(\phi)}_i = (y^{\phi}_i-1)/\phi $ if ...
3
votes
1answer
739 views

Understanding the Proof for why Jeffreys' prior is invariant

I was reviewing the section of Andrew Gelman's "Bayesian Data Analysis" on uninformative priors, and came across this explanation for why Jeffreys' prior is invariant to parameterization. My question ...
3
votes
0answers
111 views

Understanding my posterior with an uninformative prior with a poisson likelihood. Am I thinking about this correctly?

I have a problem to which I am trying to apply a Bayesian model. My data is generated as follows \begin{align} N_i \mid \mu &\sim \text{Poisson}(\mu) \\ Y_i \mid N_i, \theta_i &\sim \text{...
3
votes
0answers
109 views

Linear regression with prior on $\arctan \beta_1$

Suppose we have $\hat{y} = \beta_1 x + \beta_0$ (I ask only for the univariate case.) A typical Bayesian approach might involve Normal priors on both parameters. I was thinking today about a ...
2
votes
3answers
759 views

Prior distribution importance in Bayesian inference

I am performing a Bayesian multivariate regression, and therefore I have to construct the prior and the subsequent posterior. But the paper that I am using as a reference, uses a "Uninformative prior"...
2
votes
1answer
396 views

Bayesian estimate of the parameter of a truncated Poisson distribution (using R)

Suppose that I am working with an integer-valued variable $x$ and that all I know about my sample is as follows: Mean $\bar{x}$: 4.5 Sample size $n$: 100 I can safely assume that the underlying model ...
2
votes
1answer
29 views

How do Bayesian hierarchical models adaptively learn the prior?

It seems the main difference between a hierarchical and a non hierarchical model is that the hierarchical model learns the prior. That is it adaptively comes up with a regularizing prior to be applied ...
2
votes
1answer
179 views

Fisher information metric for hierarchical Bayesian model is negative-definite?

I'm strugling with the computation of the Fisher information matrix for the hierarchical Bayesian model. For simplicity, consider theta following hierarchical Bayesian model: \begin{align} X|\sigma &...
2
votes
1answer
264 views

What is the bayesian uninformative exponential prior?

I am looking to model a poisson process $X \sim Poisson(\lambda)$ and infer $\lambda$ using bayesian inference from $X$ e.g. $[0,0,0,2,1,0,0,2,1]$ could be my dataset. The data comes from the count of ...
2
votes
1answer
391 views

Why is Inverted Gamma (3,0.0005) a diffuse prior when variance of this random variable is small?

A few times I came across the statement that an Inverted Gamma (3, 0.0005) prior for the variance is quite diffuse but proper. Hence, my understanding is that the prior variance of this random ...
2
votes
1answer
686 views

Priors for Truncated Parameters - RJAGS

I would like to estimate the parameters of a specific model. The model specification is as follows: $p_t = k_t + B_t/(1-B_t) + \eta_t$, where $ \eta_t \sim N(0, \sigma^2)$ $R_{t+1} = R_{t} + R_t (...
2
votes
0answers
29 views

does Informativeness of the prior always decreases similarity of the posterior mean to the data mean? [closed]

I am looking for a proof of the statement "If the variance of the prior distribution is greater, the posterior is more affected by the data". More specifically, if X, X' are priors such that E(X)=E(X'...
2
votes
0answers
22 views

uninformative prior for something dependent on states?

Say I have a model which predicts something say abc over time I define 3 states, S1, S2, S3. I define transition transitional probs. I have another variable say, xyz which has value of 0 to 5. xyz ...
2
votes
0answers
65 views

Understanding the influence of the prior distribution on the original parametrization

Be $y_1,y_2,..y_2$ a simple random sampling from $p(y|\theta)$. Be $\theta$ a parameter with a given prior distribution $p(\theta)$. A way to understand how much informative is $p(\theta)$ is to plot ...
2
votes
0answers
258 views

Is Independent jeffreys prior different from independent reference prior?

I have a model involving two scalar parameters $\theta_1$ and $\theta_2$ and derived the Jeffreys prior for $\theta_1$ and $\theta_2$ independently (so for, e.g. $\pi(\theta_1)$, setting in the ...
2
votes
1answer
37 views

Objective priors for simulator-based models?

I've read a bit about how to derive parametrization-invariant priors for models where we have access to derivatives of the likelihood function and can compute the Fisher Information Matrix: http://www....
2
votes
0answers
158 views

Stochastic Block Model Priors

In the generic stochastic block model (binary edge data, no degree correction, etc.), if an uninformed prior is used for the Bernoulli coefficients i.e. Beta with $(a,b) = (1,1)$, will the model ...
2
votes
0answers
138 views

Convenient posterior distribution for homogeneous bivariate Gaussian model

For the model given by some independent pairs $(x_i,y_i)$ identically generated from a bivariate Gaussian distribution, there is the convenient semi-conjugate family of "Normal-Wishart" prior ...
1
vote
2answers
732 views

Haldane's prior Beta(0,0) - Part 1

This article$^1$ on p.16 specifies Haldane's prior as: $$p(\theta) = \frac{1}{θ(1−θ)}$$. However, other$^2$ source on p.6 specifies Haldane's prior as proportional to $\frac{1}{θ(1−θ)}$, i.e. $$p(\...
1
vote
1answer
284 views

Forecasting with no prior knowledge - Bayesian vs Frequentist

I have a basic question about Bayesian statistics. Lets say that I want to make forecasts of a certain response variable, based on explanatory variables and lagged responses variables, while I have ...
1
vote
1answer
520 views

Prior comparison: Uninformative vs informative

I have a question about prior choice that has arisen from some analysis I have been doing. I don't think the particular details of the model are necessary for this question, but my Bayesian knowledge ...
1
vote
2answers
182 views

Bayesian stats: trick to accept the null?

There's a lot to be said and read about this, but I haven't found a clear answer to this question: Bayesian statistics are said to 'penalize' vague hypotheses with weak priors, by giving more support ...
1
vote
0answers
17 views

How to set a Bayesian prior on a set with a large but unknown number of elements?

Let us suppose that we are trying to analyze a given starfish. We would like to know which species does the starfish belong to. We have a list of 1000 starfish species, but we know that there is an ...