Skip to main content

Questions tagged [improper-prior]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
9 votes
2 answers
426 views

How is data generated when using an improper prior

Let $X$ be an $\mathcal{X}$ valued random variable. We are doing Bayesian statistics. Suppose that $\theta$ is a $\Theta$ valued random variable with known prior distribution $\Pi$ and that the ...
温泽海's user avatar
  • 456
1 vote
0 answers
15 views

Unconstrained Biases and Neural Network Regularization

In Bishop's PRML on page 259 he discusses a L2 regularizer for each layer of a 2-layer neural network, given by $$ \begin{equation} \frac{\lambda_1}{2}\sum_{w\in W_1}w^2 + \frac{\lambda_2}{2}\...
olives's user avatar
  • 31
7 votes
2 answers
238 views

Is the class of models for which the MLE exists also the one for which flat priors are permissible?

By "permissible" (for lack of a better term) I mean models which despite of a "flat" (improper) prior (i.e., $\int_{\Theta} p(\theta) d \theta = + \infty$) nevertheless produce a ...
Durden's user avatar
  • 1,352
2 votes
0 answers
28 views

Distribution families whose likelihoods integrate to $+\infty$ for some sample values

I've recently started learning about Bayesian statistics, and I came across this very nice answer by Xi'an https://stats.stackexchange.com/a/129908/268693, which [in my slight paraphrasing] says the ...
Leonidas's user avatar
  • 121
2 votes
0 answers
108 views

Improper Prior in Logit and Probit Models: Proper Posterior Conditions

Let $y_i \vert p_i \sim \mathrm{Bernoulli}(p_i)$, $p_i = F_h(X_i^\prime \beta) \ \ , \ \ h = 1,2 \ ,\ \ X , \beta \in \mathbb R^p$, where $F_1(x) = (2\pi)^{-1/2}\int_{-\infty}^x \exp(-t^2/2) \ dt \ $ ...
paoletinho's user avatar
2 votes
1 answer
209 views

For multivariate normal posterior with improper prior, why posterior is proper only if $n\geq d$

This is related to Gelman's BDA chapter 3 section 5's noninformative prior density for $\mu$. Let $\Sigma$ be fixed positive definite symmetric matrix of size $d$ by $d$. Let $y_1,\dots, y_n$ be iid ...
user45765's user avatar
  • 1,445
1 vote
1 answer
107 views

Is there any strong argument about objective/non-informative improper prior?

Decades ago improper objective priors - e.g. $\pi(\sigma) \propto \sigma^{-1}, \sigma > 0,$ for a scale parameter - were considered problematic because some authors thought they were leading to the ...
Celi's user avatar
  • 51
0 votes
1 answer
42 views

Bayesian statistics

Assuming I have that $Y_i\mid \mu$ is an iid ~ $N(\mu,\sigma^2)$, for $i \in (1,\dotsc,n)$ with $\sigma_i$ known and improper prior $\pi(\mu)=1$ for all $\mu$. i. How can I derive a formula for the ...
user354604's user avatar
1 vote
1 answer
317 views

Prior predictive distribution with an improper prior for a Poisson likelihood

I have recently started exploring some bayesian statistics and I cannot seem to understand something about improper priors. In particular, the set up consists of a Poisson likelihood $ p(X|\theta) = \...
BackgroundType2's user avatar
0 votes
1 answer
102 views

Showing that a posterior is Normal given improper prior

I am having difficulty showing the following problem and I suspect it has something to do with my lack of understanding of the question. The question is this: Suppose we have an improper prior ...
CharlieCornell's user avatar
1 vote
0 answers
183 views

How to obtain a generalized bayes estimator when we have random sample from the uniform distribution with a Pareto prior and a improper hyperprior?

Let $\boldsymbol{X}=\left(X_{1}, \ldots, X_{n}\right)$ be a random sample from the uniform distribution on $(0, \theta),$ where $\theta>0$ is unknown. Let $$ \pi(\theta)=b a^{b} \theta^{-(b+1)}, a&...
JoZ's user avatar
  • 699
3 votes
2 answers
2k views

What is a non-informative choice of parameters for a Dirichlet distribution?

Dirichlet distribution is a conjugate prior for multinomial distribution. I want to impose a non-informative prior over sampling weights $\pi$ for a draw $x=(x_1,…,x_N)$ from a multinomial ...
Blade's user avatar
  • 655
2 votes
1 answer
2k views

Why is this an example of a noninformative prior?

From Bayesian Data Analysis 3rd Edition [Gelman et. al], they give this as an example when introducing non-informative priors: "We return to the problem of estimating the mean θ of a normal ...
Jake Daly's user avatar
3 votes
2 answers
223 views

Can an improper prior distribution be informative?

I have just worked through an example where, with an improper prior, the bayesian estimator equals the maximum likelihood estimator, leading me to believe that improper priors are uninformative. But ...
David's user avatar
  • 1,276
1 vote
1 answer
1k views

Finding the posterior distribution given an improper prior

Let $X \sim N(\theta, \sigma^2)$ where $\sigma^2$ is known. Let the prior density $\pi(\theta) =1, \theta \in \mathbb{R}$ to be the improper uniform density over the real line. Find the posterior ...
zerxee's user avatar
  • 51
3 votes
1 answer
185 views

Integrating out parameter with improper prior

I got this problem while I was reading the book "Machine Learning: A Probabilistic Perspective" by Kevin Murphy. It is in section 7.6.1 of the book. Assume the likelihood is given by $$ \begin{...
zwcikyf's user avatar
  • 33
8 votes
3 answers
1k views

verifying a posterior is proper

There's a homework problem in a textbook that asks to verify propriety of a certain posterior distribution, and I'm having a little trouble with it. The setup is you have a logistic regression model ...
Taylor's user avatar
  • 21.2k
3 votes
1 answer
511 views

Calculating Integral Using MCMC

Consider the integral $\int_{\Theta}f(\theta|\mathbf{x}) \Pi(\theta)d\theta$,where $\theta$ is a univariate parameter and $\Theta$ is the support of $\Pi(\theta)$. I need to evaluate the value of this ...
Aayush Agrawal's user avatar
4 votes
1 answer
212 views

Objective Bayesianism: Jeffreys priors vs reference priors vs principle of transformation groups

According to this answer, José Bernardo has produced an original theory of reference priors where he chooses the prior in order to maximise the information brought by the data by maximising the ...
user76284's user avatar
  • 993
4 votes
2 answers
1k views

Gaussian mixture model - does an improper uniform prior give a proper posterior?

We draw $n$ i.i.d. points $x_1 , x_2 , ..., x_n$ from the following Gaussian mixture: $$p(x|\mu_1,\mu_2) = \frac{1}{2} \text{N} (x|\mu_1,1) + \frac{1}{2} \text{N} (x|\mu_2,1).$$ The prior is the ...
NerdHardAtWork's user avatar
5 votes
2 answers
5k views

Haldane's prior Beta(0,0) - Part 1

This article$^1$ on p.16 specifies Haldane's prior as: $$p(\theta) = \frac{1}{θ(1−θ)}$$. However, other$^2$ source on p.6 specifies Haldane's prior as proportional to $\frac{1}{θ(1−θ)}$, i.e. $$p(\...
AlexMe's user avatar
  • 571
2 votes
1 answer
304 views

Compute $\pi(H_0|x)$ with Jeffreys prior for a family $N(\theta,1)$

Given a random sample $x = (x_1,\ldots,x_n)$ taken from a family $\{N(x|\theta,1):\theta \in \mathbb{R}\}$. And consider the hypothesis test: $H_0: \theta = 0 $ vs $H_1: \theta \in \mathbb{R}$ (this ...
user1868607's user avatar
2 votes
2 answers
258 views

How are does software compute posterior distributions from improper (flat) priors?

In Bayesian statistics, how do software packages compute the posterior distribution when the prior is improper (flat)? If I understand correctly, this can't be done analytically so how is it done ...
Michael Webb's user avatar
  • 2,176
1 vote
1 answer
1k views

Bayesian Biased Prior Formula

I know that for a Bayesian uniform/flat prior, the formula is 1/n (and n=1), as each value has an equal chance of being chosen. However, is there an equation for when the prior is biased/informative? ...
Kirsten Morehouse's user avatar
10 votes
2 answers
4k views

Difference between non-informative and improper Priors

I wonder what is the difference between these two kind of priors: Non-informative Improper
Bram's user avatar
  • 243
2 votes
0 answers
260 views

Understanding my posterior with an uninformative prior with a poisson likelihood. Am I thinking about this correctly?

I have a problem to which I am trying to apply a Bayesian model. My data is generated as follows \begin{align} N_i \mid \mu &\sim \text{Poisson}(\mu) \\ Y_i \mid N_i, \theta_i &\sim \text{...
Count Zero's user avatar
  • 1,029
5 votes
1 answer
82 views

Bayesian hierarchical linear model with uninformative priors

I have the model $y_{i,t}=x_{i,t}'\beta_{i} + \epsilon_{i,t}$ where $x_{i,t}$ is a k-dimensional vector of explanatory variables and $\beta_{i}$ is a k-dimensional parameter vector, where $\beta_{i}=\...
quant's user avatar
  • 521
17 votes
2 answers
3k views

Sampling from an Improper Distribution (using MCMC and otherwise)

My basic question is: how would you sample from an improper distribution? Does it even make sense to sample from an improper distribution? Xi'an's comment here kind of addresses the question, but I ...
Greenparker's user avatar
  • 15.8k
5 votes
2 answers
2k views

Does "improper" posterior or prior refer to a density function that does not integrate to 1 or to one that does not integrate to a finite value?

I am a bit confused about improper priors and posteriors. I have seen references that classify a prior or posterior probability density function as "improper" if the integral over infinite support ...
user1398057's user avatar
  • 2,415
2 votes
2 answers
368 views

How to choose a importance density for Jeffreys prior?

I want to draw Bayesian inference via importance sampling and I do not come up with a good idea of an importance density for $$p(\sigma)\sim\frac{1}{\sigma}.$$ Is there a way to sample from this ...
Stefan Voigt's user avatar
  • 1,330
12 votes
1 answer
638 views

When should I be worried about the Jeffreys-Lindley paradox in Bayesian model choice?

I am considering a large (but finite) space of models of varying complexity which I explore using RJMCMC. The prior on the parameter vector for each model is fairly informative. In what cases (if any)...
Jeff's user avatar
  • 1,057
13 votes
1 answer
5k views

Bayes factors with improper priors

I have a question regarding model comparison using Bayes factors. In many cases, statisticians are interested on using a Bayesian approach with improper priors (for example some Jeffreys priors and ...
Jeffrey's user avatar
  • 131