Questions tagged [conjugate-prior]

A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Sequential update of Posterior distribution of a Correlated Gaussian distribution

Consider the following problem of sequentially estimating ( in a Bayesian setting ) the means of the two Gaussian distributions $\mathcal{N}(\mu_1,\sigma_0^2)$ and $\mathcal{N}(\mu_2,\sigma_0^2)$ ...
0
votes
0answers
45 views

Find Bayesian estimator for $e^{\theta}$

Given $\{Y_i\}_n\sim U(\theta-1,\theta+1)$ and prior distribution $\theta\sim U(a,b),1\leq a<b$ is the posterior distribution conjugate? Find the absolute error estimator for $e^{\theta}$ and ...
4
votes
1answer
80 views

Posterior derivation of normal model

Working through the book Bayesian Essentials with R by Jean-Michel Marin & Christian Robert I am trying to work out the posterior for the model given on page 29 when the data is from a normal with ...
0
votes
0answers
10 views

Tractable predictive distributions

I have a parametric model $p\left(x\mid\theta\right)$, the marginal distribution this induces is $$p\left(x\right)=\int p\left(x\mid\theta\right)p\left(\theta\right)d\theta$$ I understand the concept ...
0
votes
0answers
17 views

Exponential likelihood with scale parameter + Exponential prior with rate parameter

We have an exponential model with a scale parameter $\lambda$: $$ f(x|\lambda) = \frac{1}{\lambda}e^{-\frac{x}{\lambda}}1_{x>0} $$ We posit a prior on $\lambda$ with exponential distribution and ...
0
votes
0answers
19 views

HMM with emission probability being hidden state

I've been working on this problem for a while but cannot think of any solution. So here's the problem explained in the context of HMM. The hidden state is a probability that is updated given ...
0
votes
0answers
22 views

Recommended/Commonly Used Likelihoods for TF-IDF Observations in Mixture Models?

What are recommended or commonly used likelihoods for TF-IDF observations in mixture models? The below related questions ask about whether a Multinomial likelihood can be used (and if I understand ...
0
votes
0answers
24 views

Posterior predictive of normal normal-mean conjugacy

I want to compute: $$p(x | X) = \int p(x | \mu , \Sigma) p(\mu | X) = \int \mathcal{N}(x | \mu , \Sigma) \mathcal{N}(\mu | \mu_N , \Sigma_N ) d\mu$$ Actually, this is the posterior predictive of the ...
8
votes
1answer
268 views

Does a sufficient statistic imply the existence of a conjugate prior?

In the comments on this answer, user Scortchi asks: So iff there's a sufficient statistic of constant dimension, there's a conjugate prior? As far as I know this didn't get a complete answer, so I'm ...
1
vote
1answer
51 views

Gamma family as conjugate prior of Inverse Gaussian with known $\mu$

I want to show that, when $\mu=\mu_0$, then gamma family $\Gamma(a,b)$ is a conjugate prior to inverse Gaussian with density $f(x,\mu,\lambda)=\sqrt{\frac{\lambda}{2\pi x^2}}exp[-\frac{\lambda(x-\mu)^...
1
vote
1answer
56 views

Bayesian Estimation of CDF

i'm getting pretty confused by the following problem, hope anyone can clarify my mind: Using a bayesian approach obtain a posteriori and interval estimations for $\mathbf{F}_{X}(x)$ using a Uniform(0,...
3
votes
1answer
225 views

proof for posterior predictive of normal-gamma conjugacy

Giving the following equations $$ \mu_n = \frac{\kappa_0 \mu_0 + n \overline{x}}{\kappa_0 + n}, \\ \kappa_n = \kappa_0 + n, \\ \alpha_n = \alpha_0 + n/2, \\ \beta_n = \beta_0 + \frac{1}{2} \sum\...
0
votes
1answer
63 views

Beta-binomial vs updating a prior beta distribution

Bear with me, as I've just recently been learning about conjugate priors, prior and posterior distributions, and such material. My understanding of the beta-binomial distribution is that it basically ...
0
votes
0answers
17 views

Interpretation of Gamma parameters in conjugate prior

So I'm modeling a problem using a Poisson process with rate $\theta$ and this $\theta$ follows a Gamma distribution with parameters $\alpha$ and $\beta$. Say $\theta$ captures the arrival of customers ...
0
votes
0answers
20 views

Normal Conjugate Normal Inverse-gamma Updating

I am attempting to code a multi-arm bandit where there are multiple variants that can be served to customers with the objective of learning the best one based on an outcome modeled with a normal ...
0
votes
0answers
26 views

Closed form posterior for a mixtures of two univariate Gaussians

Giving a univariate Gaussian mixture model $$\pi_1N(x|\mu_1,\sigma_1)+(1-\pi_1)N(x|\mu_2,\sigma_2),$$ are there any priors for $\pi_1$, $\mu_1$, $\sigma_1$, $\mu_2$, $\sigma_2$ which gives a closed ...
6
votes
1answer
78 views

Are there any conjugate likelihood distributions for a Categorical Prior?

I have the following generative process: $$\begin{align} z &\sim Cat(\pi)\\ o | z &\sim p(o|z) \end{align}$$ I'd like to infer a posterior over $z$ i.e. $p(z|o)$. Thankfully, I have complete ...
0
votes
0answers
11 views

What are the hyperparameters and base measure in the conjugate prior for the exponential family?

Setup Suppose we have an exponential family model $\{P_{\theta} : \theta \in \Theta\}$. Let the density function of a random variable $X$ and the prior on $\theta$ have following forms: $$ \begin{...
1
vote
0answers
34 views

Conjugate Prior Distribution with a Normally distributed marginal distribution?

The Normal-Inverse-Gamma distribution $(X,\mu,\sigma^2)\sim NIG(\mu_0, \nu, \alpha, \beta)$ is the conjugate prior for the Normal distribution. However, this would correspond to the marginal ...
0
votes
1answer
220 views

Non-informative prior for Exponential

I am working with a Bayesian model: $T \sim exp(\theta)$ for survival data, I have chosen a gamma distribution as a prior since its conjugate by an exponential distribution. I'd like to choose a $\...
2
votes
0answers
45 views

Does a conjugate prior always exists? [duplicate]

Are there distributions where no conjugate prior exists? Is there a necessary and/or sufficient condition which guarantees the existence of a conjugate prior? Edit: Why has this question been closed? ...
0
votes
0answers
26 views

Bayesian Inference Intuition: Beta and Binomial vs Gamma and Poisson

When the data is assumed to be binomial distributed, and the prior probability is assumed to be a beta distribution, the posterior follows the distribution $Beta (\alpha - 1 + k , \beta - 1 + n- k $). ...
1
vote
0answers
33 views

Compound beta-binomial and beta distribution

I have a process that is modelled by a beta-binomial, parametrised by mean $\mu$ and correlation $\rho = 1/(\alpha+\beta+1)$ (as per dbetabinom in the R VGAM package). I know $\rho$, but the mean $\...
0
votes
0answers
12 views

Are there conjugate priors used in nonparametric density estimation methods?

I am still quite new to Bayesian inference and non-parametric methods. I was wondering if conjugate priors feature in non-parametrix density estimation. I understand that in non-parametric density ...
1
vote
0answers
22 views

Alternating between Hamiltonian Monte Carlo and sampling from conjugate posterior in large models

For Bayesian models with a large number of parameters and pieces that have conditionally closed form solutions for drawing samples, when is it worth it to use these closed form updates instead of a ...
0
votes
0answers
91 views

Normal Conjugate Prior, Known Mean and Unknown Variance?

For Normal distribution, with know mean and unknown variance. When $\tau = 1/\sigma^2$ ~ Gamma(). In such has posterior of $\tau$ has the following distribution: $p(\tau|\alpha, \beta, x) \sim G(\...
0
votes
0answers
47 views

Bayesian Regression Estimates

Hi I am new to Bayesian Regression, I wanted to understand why would the Bayesian regression give exactly the same results as the priors supplied? I tried running a bayesian model on 10% of the data ...
1
vote
1answer
102 views

Find the prior distribution for the natural parameter of an exponential family

Show that for the binomial likelihood $y$ ~$Bin(n, \theta)$, $p(\theta) \propto \theta^{-1} (1-\theta)^{-1}$ is the uniform prior distribution for the natural parameter of the exponential family. I am ...
5
votes
1answer
87 views

Random number generation for conjugate distribution of beta distribution

I try to generate random numbers from the conjugate distribution of beta distribution. It is as follows $$ p(α,β∣a,b,d)∝ \frac{e^{-a \alpha} e^{-b \beta}}{(\beta(\alpha,\beta))^d} \:\:\:\:,\:\:\: \...
0
votes
0answers
18 views

Evaluating the integral using conjugate distributions

Hello I just want to verify that I am evaluating this integral correctly. When I implement it in code values seem to be incorrect. could be my implementation though. Thank you for the second pair of ...
2
votes
1answer
95 views

Help with the prior distribution

The question is as follows: Consider an SDOF mass-spring system. The value of the mass is known and is equal to 1 kg. The value of the spring stiffness is unknow and based on the experience and ...
4
votes
1answer
86 views

Is there a limiting distribution if you keep taking conjugate priors within the exponential family?

I'm assuming this is pretty commonly discussed but haven't found it yet. For example, if you start with a binary (0-1 binomial), conjugate prior is beta distribution. Beta distribution has some un-...
0
votes
0answers
46 views

Are $\alpha, \beta$ of Beta distribution positive integer inn Thompson Sampling

In wikipedia on beta distribution, they say that domain of hyperparameter $\alpha, \beta$ are positive real numbers. However, according to my reasons, the domain of $\alpha, \beta$ should be limited ...
0
votes
1answer
166 views

What family is the posterior predictive distribution in when the likelihood is a Bernoulli and the prior (and posterior) is Gaussian?

I have a problem where the hidden random variables I want to know about are continuous (i.e. my parameters can take any real number), and so I have modeled them as having a Gaussian p.d.f. (because ...
1
vote
1answer
26 views

Bayesian estimation difference between inferences is conjugate prior and non informative prior [duplicate]

What are the differences in the inferences using non-informative (specifically Jeffrys) priors as compared to the conjugate prior?
3
votes
1answer
204 views

Conjugate priors outside exponential family

The usual exception I have come across regarding non-existence of conjugate prior outside the exponential family is the uniform distribution on $(0,\theta)$ (i.e. $U(0,\theta)$) where $\theta$ has a ...
1
vote
0answers
26 views

Can a non-conjugate prior lead to a closed form or analytically tractable posterior? [duplicate]

I understand conjugate priors to the likelihood lead to a nice closed form posterior, and all members of the exponential family have conjugate priors. But is this if and only if? ie. Can a non-...
15
votes
2answers
468 views

What is the origin of the name “conjugate prior”?

I know what a conjugate prior is. But I'm confused by the name itself. Why is it called "conjugate"? A complex conjugate $z^\ast$ has a reciprocal relationship with $z$, i.e., ${z^\ast}^\ast ...
0
votes
0answers
16 views

Use inference on the whole data as a prior to regularize estimates in a subgroup

Suppose that the probability of a certain condition in the data $D$ is $\theta$ which can be estimated by updating a Beta prior as $Pr(\theta|s, N) = Beta(\alpha + s, \beta + N - s)$ with $s$ being ...
1
vote
0answers
277 views

How to use scipy stats gamma pdf to update the posterior distribution?

I'm trying to "get my bearings" performing bayesian analysis, specifically I'm exploring the Gamma-Poisson conjugate prior. The definition of the PDF is below If the prior takes the form of ...
1
vote
1answer
265 views

Beta-Binomial conjugate proof

Can someone explain this proof to me? I get stuck on the transition from the third line to the last line. Namely: Is the integral being evaluated or not? How does the entire expression reduce to a ...
0
votes
0answers
22 views

Direct relationship between prior and posterior parameter

I've been trying to learn JAGS and Bayesian modeling more generally and I'm running into something I can't quite explain. I've noticed that when fitting simple mean and variances to normally ...
1
vote
1answer
123 views

Bayesian priors and probability distributions

Book "Bayesian Statistics the Fun Way: Understanding Statistics and Probability with Star Wars, Lego, and Rubber Ducks", chapter 9 "Bayesian priors and working with probability ...
3
votes
2answers
370 views

Why is the inverse chi-squared distribution a natural prior and posterior for an unknown variance of a normal distribution?

Wikipedia says [the inverse-chi-squared distribution] arises in Bayesian inference, where it can be used as the prior and posterior distribution for an unknown variance of the normal distribution. ...
0
votes
2answers
81 views

The prior in MAP and Bayesian interference

We can use a Normal distribution as a prior when handling a Normal distribution as likelihood in Bayesian inference However if we want to do MAP given a Bernoulli as likelihood can we use Normal ...
0
votes
0answers
22 views

What is the conjugate prior for the hypoexponential distribution?

Can't find it anywhere. I know Gamma is the conjugate prior for the exponential distribution (one parameter) but for the sum of exponential distributions (the hypoexponential distribution), I can't ...
1
vote
1answer
40 views

Help understanding bayesian update for exponential distribution

I'm following these slides. I have some lamps, which I expect to die at a time $t$ where $p(t) = \lambda e ^{-\lambda t}$ for some $\lambda$. My prior for $\lambda$ is given by $p(\lambda) = \Gamma(\...
1
vote
0answers
36 views

How to define different priors for $\mu$ parameters using a Normal-Inverse-Wishart prior?

Suppose, $y_i \sim N_2 (\mu, \Sigma)$ I want to define a Normal-Inverse-Wishart prior on $\mu$ and $\Sigma$. Such that, $(\mu, \Sigma) \sim NIW(\mu_0,\lambda,\Psi,v)$ My prior knowledge about $\mu ...
2
votes
0answers
77 views

Normal distribution with known mean and unknown variance (product of two variables)

Assume there is a data point $x$ sampled from a Normal distribution: $$\begin{align} x \sim \mathcal{N}(\mu,\frac{1}{yz}) \propto (yz)^{1/2} \exp [-\frac{1}{2} (x-\mu)^2yz] \end{align}$$ where $\mu$ ...
1
vote
1answer
260 views

Justification for use of non-conjugate priors?

Google searches gives no results to this question and there is the opposite question in this site, which makes me think this has an intuitive response I am missing. In most course notes and responses ...

1
2 3 4 5 6