Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [conjugate-prior]

A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.

0
votes
0answers
10 views

Using gaussian as prior probability distribution

I have seen in several places that people use a Gaussian distribution as the prior probability distribution. But when I am assigning probabilities, does using Gaussian a good idea, given that we can ...
0
votes
0answers
20 views

Uniform conjugate prior for a Beta distribution

Given $$ \pi \sim \text{Beta}(\alpha, \beta) $$ I'd like to place a prior on $\alpha$ and $\beta$. The "trick" mentioned in this post and this post seems to be to recognize that since $$ \begin{...
0
votes
0answers
9 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
3
votes
1answer
47 views

Compute conjugate prior from the sample distribution

I feel like this question might be marked as duplicate because I see many similar incurring in that fate but I'll try anyway. I would say I did not find anything similar. I have been thought a ...
1
vote
1answer
49 views

Bayesian inference on binarized Poisson distribution

I have a variable that is Poisson distributed. Let's say I have a number of boxes each with a number of balls inside according to a Poisson distribution, with $\lambda=0.4$, (the average number of ...
2
votes
1answer
36 views

How does L2 penalize large weights

The L2 regularization term is useful because it penalizes large weights over smaller weights which is good to prevent overfitting. I'm having a hard time understanding how exactly it does this. This ...
3
votes
1answer
33 views

Joint AR(1) posterior distribution explicit under conjugate prior

I have encountered a problem in my textbook 'The Bayesian Choice' by Christian P. Robert. It goes something like this: $"$For a particular case of AR(1) model, $(x_t)_{1\leq t\leq T}$. Where $x_t = \...
1
vote
1answer
60 views

Mean of the posterior distribution in bayesian linear regression with infinitely broad prior

Currently reading from Christopher Bishop's Pattern Recognition and Machine Learning book about parameter distribution under a bayesian linear regression. On page 153, the author deduces that the ...
0
votes
0answers
32 views

Does Levy distribution has a conjugate prior?

I have searched a lot but can not find the conjugate prior of Levy distribution.
1
vote
0answers
43 views

Proving Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model

How does one prove that the Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model? This prior is a generalization of the Normal-Inverse-Wishart Distribution. By Matrix-...
5
votes
1answer
133 views

Truncated Gamma Distribution

The Gamma distribution is the conjugate prior of Poisson distribution. What about the Truncated Gamma distribution? Is it still the conjugate prior of Poisson distribution?
1
vote
1answer
70 views

Conjugate Prior for Student T distribution

Does the Student T distribution have a conjugate prior distribution? If so, what is it and what are the parameters?
0
votes
0answers
23 views

Is the posterior distribution for the model described in this question Gaussian?

I was in the middle of writing a long answer to Uncertainty estimation in high-dimensional inference problems without sampling? but I was suddenly struck by doubt: since the model assumes a Gaussian ...
2
votes
1answer
35 views

Computing posterior density of Normal with unknown $\mu$ and $\sigma^2$

(From Bayesian Essentials with R by Marin & Robert page 31) We are given an iid sample $\mathfrak{D}_n = (x_1, \dots, x_n)$ from the normal distribution $\mathcal{N}(0, \sigma^2)$ and $\theta=(\...
1
vote
1answer
127 views

How to use the Dirichlet prior for estimating the multinomial parameters? [closed]

I know that the multinomial distribution gives the likelihood of some vector D of occurrences to happen given a probability vector (parameters) P' i.e. P(D|P'). Now with a Dirichlet prior we are ...
1
vote
1answer
51 views

Maximum A Posteriori Estimate

The formula for calculating the MAP estimate of a particular parameter, $p$, is given by the following: $p^{MAP} =$ argmax $P(p)P(p|x)$. Now I am trying to do a question where I am told the prior ...
1
vote
0answers
14 views

Eigenfunctions heuristics for self-conjugate priors [duplicate]

Previously asked on math.stackexchange. I am looking for a citable reference (books, research papers, PhD theses, not websites, internal reports, etc.) about the heuristic interpretation of self-...
1
vote
1answer
103 views

Interpretation of the rate parameter of a Gamma distribution

I am toying with mixture models, especially in a bayesian context and the Gamma (or the inverse Gamma) distribution appears quite often. For example, inverse Gamma is used as a conjugate prior for the ...
0
votes
1answer
24 views

Bernoulli distribution/ SOME probability/conjugate prior

I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it? EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
0
votes
2answers
30 views

Beta distribution and normalization [duplicate]

Here in the 4 pictures in the last answer, is the vertical axe the probability? I.e. it seems to me that it is somewhat unnormalized : it has the value 2 in the 2nd picture and 3 in the 3rd picture. ...
2
votes
1answer
163 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
1
vote
0answers
95 views

Dirichlet distribution as conjugate prior to Multinomial distribution [closed]

I stumbled upon the following exercise: Show that Dirichlet distribution with parameter $\alpha$ is a conjugate prior to the Multinomial distribution as likelihood. Derive the parameters to the ...
1
vote
1answer
32 views

Basic question on proportionality in Bayesian Inference for Normal distribution

I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that: $\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$ $Y | \theta \...
4
votes
1answer
499 views

If both prior and likelihood are Gaussian what can we say about the posterior? [closed]

If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior? As posterior is proporional to prior*likelihood which are Gaussians, the posterior ...
5
votes
2answers
227 views

The meaning of Bayesian update

I'm new in Bayesian inference and I can't found the answer to this: In real life scenario people use MCMC to compute the posterior distribution given the likelihood and the prior. Analytical ...
1
vote
1answer
42 views

Normal - Inv chi squared posterior calculation

Given that for a known mean $\mu$ and unknown variance $\sigma^2$ the normal distribution is $$X_i|\sigma^2 \sim \mathcal{N}(\mu, \sigma^2) = \frac{1}{\displaystyle\sigma\sqrt{2\pi}}\exp\left[-\...
1
vote
1answer
64 views

Gamma-gamma conjugacy for rate parameter of Gamma distribution

the question is as follows. Assume the shape $r$ is a known constant. For $x \sim$ Gamma(shape = $r$, rate = $v$), the p.d.f is: $$p(x|r,v) = x^{r-1}e^{-vx}v^r/\Gamma(r)$$ a) Show that the $\theta \...
0
votes
1answer
132 views

Survival time problem exponential with gamma prior

The survival times, in days, of patients diagnosed with a severe form of a terminal illness are thought to be well modelled by an exponential($\theta$) distribution. We observe the survival times ...
1
vote
1answer
51 views

Obtaining posteriors for multivariate Normal mixture models

So I want to fit a mixture model $$f(y) = \pi_1 f_1 (y) + \pi_2 f_2 (y)$$ where $\pi_k = P(S = k)$ and $S_i$ is a latent unobserved variable. I assume that, conditional on $S=k$, we have the model ...
0
votes
3answers
185 views

Conjugate prior, unclear definition

Consider the following definition: A family $\cal F$ of probability distributions on $\Theta$ is said to be conjugate (or closed under sampling) for a likelihood function $f(x|\theta)$ if for every $\...
3
votes
0answers
318 views

Predictive Posterior Distribution of Normal Distribution with Unknown Mean and Variance

Suppose that $x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$ for $i = 1,...n$. Assume that the assigned prior distributions are $\mu$ ~ $\mathcal{N}$($\mu_{0}$, $\sigma^{2}_{0}$) and $\tau \...
0
votes
0answers
126 views

Coverage probability for the Bayesian credible interval for Normal distribution

Bayesian Inference for the Normal Distribution, I use the following r code to obtain the posterior distribution. Let's say the data, $X\sim N(\mu, \sigma^{2})$ and $\mu \sim N(0,10)$ and $\sigma \sim \...
3
votes
0answers
40 views

Can two families of distributions be conjugate priors for one another?

Is it possible to find two families of distributions $\mathcal F$ and $\mathcal G$ such that: $\mathcal F$ is a conjugate prior for $\mathcal G$ $\mathcal G$ is a conjugate prior for $\mathcal F$ $\...
2
votes
1answer
43 views

Prior For Gaussian R.V.s with Common Variance

Suppose we observe $n$ independent random variables $X_1, \dots, X_n$. Suppose also that the mean and variance of each $X_i$ is unknown. If $$X_i \sim \mathrm{N}(\mu_i,\sigma^2_i)$$ then a conjugate ...
4
votes
2answers
85 views

MCMC combined with numerical integration towards more efficient Bayesian inference

I am quite new to Bayesian statistics so the question can be a bit naive. My question is on how to deal with a model with individual coefficients. Simple versions of a task and a model I deal with is ...
1
vote
1answer
1k views

Gamma Conjugate Prior & Poisson Process

I am analyzing daily data transaction data. I am assuming that The number of transactions in every day of length t has the Poisson distribution with mean λt The number of transactions in evert ...
2
votes
2answers
107 views

Should updating one data point at a time or all change the posterior of a normal-inverse-gamma?

I have implemented the normal inverse gamma distribution per section 3 of https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf in some code. However, I've noticed ...
4
votes
1answer
103 views

How many natural parameters are really in the exponential family conjugate prior?

The exponential family with natural parameter $\theta$ can be written $$ p(x|\theta)=h_\ell(x)\exp(\theta^Tt(x)-a_\ell(\theta)) $$ with conjugate prior $$ p(\theta|\lambda)=h_c(\theta)\exp(\lambda_1^T\...
0
votes
0answers
30 views

Hybrid Bayesian Network - Conjugute Prior for Bernoulli Variable with Discrete and Continuous Parents

I have a discrete variable $x_a$ for which the likelihood function is Bernoulli. Within the Hybrid Bayesian Network, $Parents(x_a)$ includes both continuous and discrete variables. Is there a ...
1
vote
1answer
354 views

Intuitive explanation of Inverse Wishart prior for covariance estimation

I am trying to understand what is going on in the use of an Inverse Wishart prior for (Gaussian) covariance, and what is the motivation for it. I am seeing this posed as a solution for when the ...
0
votes
0answers
8 views

Combining AEVB with conjugate exponential family observations

I'm trying to build a probabilistic model that is a combination of a neural network and a graphical model; namely it uses a MLP as an encoder network, and the "decoder" is an exponential family ...
4
votes
1answer
1k views

Why do we use inverse Gamma as prior on variance, when empirical variance is Gamma (chi square)

Let $$X_i\sim \mathcal{N}(0,\sigma^2)$$ than we know that $$\sum_{i=1}^N\frac{X_i^2}{N}\sim\Gamma(\frac{N}{2},\frac{2\sigma^2}{N})$$ that the empirical variance follows a Gamma distribution. How do ...
0
votes
1answer
49 views

Prior for mean unknown parameter

We know that there are two versions of exponential $\big(1\big)$ $exp(\lambda)$ with pdf $$f(x;\lambda)=\lambda e^{-\lambda x}$$ ,$\lambda>0,x>0$ and $\mathbb{E}[x]=\frac{1}{\lambda}$ for ...
0
votes
0answers
78 views

how to get posterior distribution of beta with gamma prior

I have $X_1, ..., X_n \sim beta(\theta,1)$ and $\theta \sim gamma(r, \lambda)$ and wish to compute the posterior distribution. Since $f(\textbf{X} | \theta) = \theta^nx^{n(\theta-1)}$ and $\pi(\...
2
votes
0answers
96 views

Bayesian inference - conjugate priors and MAP

I came across a exercise on Bayesian inference with conjugate priors and MAP estimation: When you pet your cat, she might scratch you with probability $p$ or start purring with probability $1-p$. You ...
4
votes
0answers
118 views

Bayesian Multivariate Normal-Normal Model when covariance depends on the mean

Let $y$ denote a D-dimensional multivariate normal random variable with density $$y \sim N(\mu, \Sigma(\mu)) $$ such that the covariance $\Sigma$ is a deterministic non-linear function of $\mu$. ...
4
votes
2answers
164 views

Conjugate prior distribution for a specific (normal) curved exponential family

Is there a conjugate prior distribution in the model $\text{N}(\theta, \sigma^2=\theta^2)$, that is a normal distribution where the mean and standard deviation are equal ($\theta>0$)? (This ...
1
vote
0answers
24 views

Computing the posterior for the splitted observation in a collapsed Gibbs sampler using conjugacy for the Gaussian distribution

It has shown in the Doshi 2009 that we can attain the speed of uncollapsed Gibbs sampler using collapsed Gibbs sampler by splitting the observations into two parts. In Equation (8), the posterior of $...
2
votes
2answers
82 views

Marginalize Gaussian Random Variable completing the square [duplicate]

I have $$P(x | \lambda_1, \lambda_2) = \mathcal{N}(\mu + L_1 \lambda_1 + L_2 \lambda_2, \Sigma)$$ and a Gaussian prior for $$P(\lambda_1) = P(\lambda_2) = \mathcal{N}(0, I)$$. My task is to compute $...
3
votes
1answer
175 views

How to quantify strength of beta prior?

How would one interpret the strength of prior belief associated with a parameter with a prior beta(10,8) distribution, compared to a beta(0,0) prior, (with data from a binomial distribution)? Some ...