Questions tagged [conjugate-prior]
A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.
341
questions
3
votes
0
answers
72
views
Generating Samples from the Unnormalized Gamma Conjugate Prior
Wikipedia lists the following as an unnormalized conjugate prior of the gamma distribution in the case where both parameters $\alpha,\beta$ are unknown:
$$
\frac{p^{\alpha-1} e^{-\beta q}}{\Gamma(\...
- 66
5
votes
1
answer
150
views
Uniform prior and poisson likelihood, what posterior distribution will be produced?
If i have a uniform distribution over a fixed specified and a finite range, and a Poisson likelihood distribution, what posterior will be produced?
The likelihood has this form $$P(\pmb{X}|
\pmb{\...
- 75
1
vote
0
answers
31
views
Gaussian linear model marginal likelihood under g-prior
Consider a Gaussian linear model with an $ n \times 1 $ outcome vector $ y $ and an $ n \times p $ matrix of centered predictors $ X $:
$ y = \iota\alpha + X\beta + \varepsilon \quad \quad \varepsilon ...
- 720
1
vote
1
answer
35
views
known variance in conjugate normal
$Posterior\ mean=\frac{1}{\frac{1}{\sigma_{0}^{2}} + \frac{n}{\sigma^{2}}}\left( \frac{\mu_{0}}{\sigma_{0}^{2}} + \frac{\sum_{i=1}^{n} x_i}{\sigma^2} \right)$
Using this updating equation with known ...
- 63
3
votes
1
answer
100
views
For a separable covariance in a Gaussian process, is an inverse Wishart prior conjugate?
Suppose we have a GP for the vector $\mathbf{y}\sim\text{GP}(\boldsymbol{0},\Sigma_y)$, where $\Sigma_y=\Sigma_r\otimes\Sigma_f$ is a separable covariance matrix. Assume $\Sigma_f$ is fixed and an ...
- 2,043
0
votes
1
answer
38
views
Bayes rule and terms with expectation
I am reading the following paper in economics;
link
On page 495, authors give an expression with Bayes rule. As an example, say that there is a random variable $\beta$ which can be either $\beta_L$ or ...
- 111
1
vote
0
answers
42
views
Non-informative prior in Bayesian Linear Regression
It's known that in Bayesian Linear Regression with $\text{Inv-}\Gamma(a_0, b_0)$ prior on variance parameter $\sigma^2$, the posterior distribution after $n$ observations $(X, Y)$ is $f(\beta, \sigma^...
- 11
4
votes
1
answer
161
views
Conjugate Prior for Student T distribution with known degrees of freedom
Somebody asked a question about a conjugate prior distribution for Student-t distribution with unknown degrees of freedom. It was answered that there are no conjugate prior distribution in that case.
...
- 43
3
votes
0
answers
100
views
Form of conjugate prior on $(\mu,\sigma^2)$ for Normal $N(\mu,\sigma^2)$ distribution
I have seen many postings related to finding a conjugate prior for $(\mu,\sigma^2)$ for the normal distribution $N(\mu,\sigma^2)$. I am trying to derived an expression that has a rather simple ...
- 201
0
votes
0
answers
24
views
Posterior distribution for multivariate Gamma-Normal model
Let $\theta \in \mathbb{R}_{>0}^n$ be a random variable with prior distribution $p(\theta)$:
\begin{equation}
p(\theta) = \prod_{i=1}^n \text{Ga}(\alpha_i, \beta_i)(\theta_i),
\end{equation}
where $...
0
votes
0
answers
28
views
Conjugate or VB update for normal likelihood with shifted variance
Consider following model with normal likelihood at time $t$.
\begin{align*}
y_t | \mu_t, \sigma_t^2 \sim \mathcal{N}(\mu_t, \sigma_t^2 + P_t)
\end{align*}
we would like to find some priors $\mathcal{F}...
- 21
0
votes
0
answers
32
views
Conditionally conjugate prior in heteroskedastic model
I am researching a linear model where the noise is a function of the slope parameter as follows
$$y_i = \beta_0 + \beta_1x_i + \beta_1\epsilon_i$$
$$\epsilon_i \sim N(0, \sigma^2 g)$$
where $g$ is ...
- 351
5
votes
2
answers
436
views
How good is the Beta distribution as a conjugate for Binomial distribution?
I understand that the Beta Distribution is a 'natural conjugate' of the Binomial distribution, in sense that the Posterior Distribution is proportional to the multiplication of both.
$$ Posterior(\...
- 304
0
votes
0
answers
44
views
Posterior of Inverse Wishart distribution with a subset of data observed
Suppose:
\begin{equation}
x_1\in \mathbb{R}^{p_1}\\
x_2\in \mathbb{R}^{p_2}
\end{equation}
such that
\begin{equation}
x \sim \mathcal{N}(
\begin{bmatrix}
x_1\\
x_2
\end{bmatrix};
\begin{bmatrix}
\...
- 331
2
votes
0
answers
46
views
Mean of normal follows a T distribution
Suppose:
$x \sim \mathcal{N}(x; \mu, \Sigma) \;\;\;$ st. $\;\;\; \mu \sim T_{v}(\mu; k, M)$
Where $T$ is the $t$-distribution with v degrees of freedom, location $k$, and shape $M$.
Then, is there a ...
- 331
1
vote
1
answer
149
views
Picking parameters for beta prior
I have some data that I believe come from a binomially distributed population. A beta prior seems like an appropriate choice, but I don't have any very strong prior beliefs. I could use a less ...
- 193
2
votes
2
answers
368
views
What is the posterior probability for flipping a coin, assuming a beta distribution as conjugate prior
Suppose, I toss a fair coin n = 10 times and get 7 heads and 3 tails. The probability of fair coin is p = 0.5. Now, that the beta distribution is a conjugate prior of the binomial likelihood. I used ...
2
votes
1
answer
23
views
Show that the multinomial distribution with $k$ categories and Dirichlet distribution are conjugate prior
Problem: Show that the following distributions are conjugate priors for the corresponding densities..
The multinomial distribution with $k$ categories and
$$ p_{X|\theta_1 , \dots, \theta_k} (x_1, \...
- 265
0
votes
0
answers
105
views
posterior predictive of a normal distribution with normal prior over mean and Gamma prior over precision
What is the posterior predictive of a normal distribution with normal prior over mean and Gamma prior over precision. Thus, what is the distribution of x given:
\begin{equation}
x \sim \mathcal{N}(x; \...
- 331
0
votes
0
answers
39
views
Conjugate prior for univariate normal with same mean and unknown sum of two variances
I have a Bayesian inference problem where the likelihood function is conditioned on two unknown variances.
$$\log\mathcal{L}(d\mid \sigma_n,\sigma_s) = -\frac{1}{2} \log (\sigma_n^2 + \sigma_s^2) -\...
0
votes
0
answers
117
views
Posterior Distribution using a Normal Likelihood and Laplace Prior
I have the working out below but is this correct. I just want the posterior distribution of when mu=0 given x.
What I have tried is setting mu=0 after rewriting the first pdf with the summation ...
2
votes
0
answers
95
views
A hierarchical model with conjugate hyperprior
I have a modeling problem that I am trying to formulate in a Bayesian manner to do inference.
Basically, I have a prior where the variance is unknown, and we want to treat it as uncertain (though with ...
1
vote
0
answers
37
views
Bayesian conjugate updating when the likelihood can be approximated by a finite mixture of normals?
I'm facing a situation where I'd like to do Bayesian conjugate updating, but both the prior and the likelihood (a Student-t) can only be approximated by a finite mixture of normals. I know that a ...
- 34.2k
4
votes
1
answer
234
views
Is Inverse-Wishart a conjugate prior for Wishart likelihood?
Suppose I have a noisy observation $Z$ of a covariance matrix $F$, given a prior on $F: p(F)$, I would like to find the posterior of $p(F|Z)$, does the following specification forms conjugacy?:
$$ F \...
- 41
1
vote
0
answers
18
views
To derive Posteriors from Conjugate Priors, do we just multiply the terms in the PDFs with the parameters of interest?
Consider the beta-binomial model (beta prior, binomial likelihood). So we have$$
\begin{align}
P(\theta)&\sim \text{Beta}(\theta|\hat a,\hat b)
\propto \theta^{\hat a-1}(1-\theta)^{\hat b-1}\\
P(Y|...
- 55
1
vote
0
answers
165
views
Multivariate Normal Bayesian Updating with Conjugate Priors but Non-Standard Likelihood
I am trying to solve for the posterior of two parameters $\theta_1$ and $\theta_2$. I have priors $N(\mu_1, \sigma_1^2)$ and $N(\mu_2, \sigma_2^2)$ for $\theta_1$ and $\theta_2$ respectively, where $\...
- 11
2
votes
0
answers
316
views
Posterior predictive distribution for Bernoulli (and categorical)
I'm trying to confirm something I've tried to figure out about the posterior predictive distribution for Bernoulli vs. Binomial (and categorical vs. multinomial) random variables after a Bayesian ...
- 34.2k
3
votes
1
answer
830
views
Bayesian estimation of iid sample from Uniform$[0,\theta]$ and a Pareto$(\alpha,\beta)$ prior for $\theta$
I am working on Bayesian estimation: suppose that $X_1,\dots, X_n$ is an iid sample from Uniform$[0,\theta]$. Assume a Pareto prior for $\theta\sim Pareto(\alpha,\beta)$, i.e.
$$
f(\theta)=\frac{\...
- 737
0
votes
1
answer
105
views
Equivalent of Conjugate Priors for Marginal Probability Distributions
In probability, there are nice “conjugate prior” distributions that enable closed-form Bayesian updating – e.g. if you have a Normal likelihood and Normal prior (on the mean parameter), you get a ...
- 3
0
votes
0
answers
38
views
Is updating gamma conjugate distributions always increasing?
In Bayesian inference, the gamma distribution is the conjugate prior to many likelihood distributions: Poisson, exponential, normal (with known mean), etc.
The update rule seems to always add to $\...
- 547
0
votes
0
answers
305
views
Posterior distribution of a $\text{Gamma}(\alpha,\beta)$ random variable given a Gamma prior for $\beta$
Let $Y$ be a $\text{Gamma}(\alpha,\beta)$ random variable with known shape parameter $\alpha$ and unknown scale parameter $\beta$. Suppose we assign a $\Gamma(\alpha_0,\beta_0)$ prior to $\beta$. I am ...
- 121
3
votes
1
answer
458
views
What is the conjugate prior for the Von Mises distribution's precision
Does the Von Mises distribution have a conjugate prior for its precision/variance?
Update:
The concentration parameter $\kappa$ (Kappa) seems to control the variance of the Von Mises distribution. If $...
- 61
0
votes
1
answer
3k
views
How to calculate the posterior distribution with a normal likelihood function and a prior that involves sigma
In the problem, the data X follows a normal distribution, or $f(x|\mu,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp(-\frac{1}{2}(\frac{x-\mu}{\sigma})^2)$. Let's say I know the value of $\sigma^2$ and ...
- 297
0
votes
1
answer
58
views
Eliciting a Gamma informative prior in a Gamma–Poisson Bayesian problem
I employ the Gamma–Poisson conjugate family for my statistical model.
I want to use an informative prior.
From theory, I know that the values of the Gamma-distributed random variable lie within the ...
- 37
4
votes
0
answers
1k
views
Pareto distribution with Gamma prior on parameter $\theta$
I want to calculate the posterior distribution of Pareto distribution with known parameter $X_m$ and unknown parameter $\theta$, with conjugate prior on $\theta$ the Gamma distribution:
My effort is ...
0
votes
0
answers
192
views
Necessity of Metropolis Hastings algorithm for given posterior distribution
Let's say that we have calculated the posterior distribution of a parameter of interest given the data of a binomial experiment $N=70,x=34$ which the probability of event occurrence $\theta$ follows ...
0
votes
1
answer
87
views
Should the updated posterior for a Poisson distribution be discretized if based on the Gamma distribution as the prior?
I know that the Gamma distribution is the conjugate prior of the Poisson distribution, such that given $\alpha$ and $\beta$ that describe the prior distribution, the posterior distribution is $Gamma(\...
- 101
1
vote
1
answer
205
views
Does the beta negative binomial (BNB) distribution have a conjugate prior?
BNB distribution is constructed using negative binomial and beta distributions, which are both exponential family, so my guess would be yes, there shoudl exist a conjugate prior in theory. But what is ...
- 193
0
votes
1
answer
584
views
Use the gamma prior to obtain the gamma posterior values
I have the following information for the ages of individuals:
Sample size = 5.
Data: $$ x_i = (10, 12, 15, 16, 14) $$
The population mean previously accurately estimated is 12.
Prior information ...
user339518
1
vote
1
answer
685
views
Find a conjugate prior for the Weibull distribution under reparametrization
Consider the Weibull sampling model for $X_1,\ldots,X_n$ iid, where
$$p(x|\lambda,k)=k\lambda^kx^{k-1}e^{-\lambda^kx^k}$$
for $x>0$. Assume $k$ is known and $\lambda$ is unknown. First, if I adopt ...
2
votes
0
answers
177
views
the reason for using InverseGamma and LogNormal as prior for covariance matrix or variance
In the Bayesian analysis, sometimes we can see that InverseGamma and LogNormal distributions are used as prior for variance or covariance matrix respectively. What are the logic or explanations of ...
- 5,222
0
votes
1
answer
225
views
Proportional to Gamma means the posterior is gamma
I'm reading through these lecture notes on posteriors and conjugate priors.
https://web.stanford.edu/class/stats200/Lecture20.pdf
In particular, it asserts that: "This is proportional to the PDF ...
1
vote
0
answers
68
views
Dirichlet Process posterior with partially observed data
Suppose I dipose of a set of independant observed couples $(x_1,y_1),...,(x_N, y_N)$ from a joint distribution $P(x,y)$. Furthermore, I suppose that the random distribution $P$ as a Dirichlet prior
$P\...
- 11
6
votes
3
answers
1k
views
Why not use the same distribution for the prior in Bayesian statistics?
I am wondering why introductory books on statistics use a conjugate distribution family for the prior instead of using the same pdf of the one we are trying to infer the parameters?
For example, the ...
- 777
2
votes
0
answers
38
views
In what ways do conjugate priors compose?
A lot of conjugate priors are known for a lot of likelihood distributions (mostly the exponential family). But most Bayesian models in practice don't just consist of one distribution. Usually, you ...
- 210
0
votes
0
answers
63
views
How should I deduce the conjugate prior and corresponding posterior for a geometric distribution
The given pmf is for a geometric distribution and is $f(x_i|\theta) = (1-\theta)^{x_i - 1}\theta; ~x_i = 1, 2 ,\cdots, $ and the 1-parameter exponential family I have obtained is; $$f(x|\theta) = \exp ...
user365863
0
votes
1
answer
74
views
Posterior distribution when the domain of the likelihood depends on the parameter
I am trying to calculate a posterior density given distribution and a prior. And I am a bit confused about how I should act as the domain of the distribution depends on the parameter.
I am talking ...
1
vote
2
answers
910
views
Sequential Bayesian updating of mean and variance of normal distribution
I am trying to write some code to learn the parameters of a normal distribution. I am new to this, and I have patched together the equations from various sources, which may be part of the problem. In ...
- 23
0
votes
0
answers
22
views
Posterior density problem
I am preparing for an exam and I have stumbled upon this exercise, which I am not certain of.
a)
You are given i.i.d. data $x_1, \dots, x_n$ from a continuous distribution with density $\frac{\alpha ...
- 1
0
votes
0
answers
88
views
Identifying the value of parameters of the prior distribution. Arbitrarily?
Referring to this Question, let's not use Jeffrey's prior for $\theta$ but use $Gamma(\alpha,\beta)$ as the conjugate prior for $\theta$. Under quadratic loss function, the bayes estimator for $\theta$...
- 362