Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [conjugate-prior]

A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.

1
vote
1answer
26 views

Interpretation of the rate parameter of a Gamma distribution

I am toying with mixture models, especially in a bayesian context and the Gamma (or the inverse Gamma) distribution appears quite often. For example, inverse Gamma is used as a conjugate prior for the ...
0
votes
1answer
23 views

Bernoulli distribution/ SOME probability/conjugate prior

I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it? EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
0
votes
2answers
22 views

Beta distribution and normalization [duplicate]

Here in the 4 pictures in the last answer, is the vertical axe the probability? I.e. it seems to me that it is somewhat unnormalized : it has the value 2 in the 2nd picture and 3 in the 3rd picture. ...
2
votes
1answer
136 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
1
vote
0answers
29 views

Dirichlet distribution as conjugate prior to Multinomial distribution [closed]

I stumbled upon the following exercise: Show that Dirichlet distribution with parameter $\alpha$ is a conjugate prior to the Multinomial distribution as likelihood. Derive the parameters to the ...
-1
votes
0answers
11 views

Posterior predictive for the beta distribution? [duplicate]

I'm looking to do Bayesian inference using the beta distribution. Are there any sources for a derivation of the posterior predictive? It's an exponential family, so it should have a conjugate prior, ...
1
vote
1answer
26 views

Basic question on proportionality in Bayesian Inference for Normal distribution

I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that: $\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$ $Y | \theta \...
0
votes
0answers
17 views

What is the significance of conjugate joint prior for normal distribution?

For example, for a normal distribution N(μ,σ2), it is possible to place, say, a separate normal prior on μ and another separate IG prior on σ2, but what is the prrof for conjugate joint prior ...
4
votes
1answer
435 views

If both prior and likelihood are Gaussian what can we say about the posterior? [closed]

If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior? As posterior is proporional to prior*likelihood which are Gaussians, the posterior ...
5
votes
2answers
201 views

The meaning of Bayesian update

I'm new in Bayesian inference and I can't found the answer to this: In real life scenario people use MCMC to compute the posterior distribution given the likelihood and the prior. Analytical ...
1
vote
1answer
19 views

Normal - Inv chi squared posterior calculation

Given that for a known mean $\mu$ and unknown variance $\sigma^2$ the normal distribution is $$X_i|\sigma^2 \sim \mathcal{N}(\mu, \sigma^2) = \frac{1}{\displaystyle\sigma\sqrt{2\pi}}\exp\left[-\...
1
vote
1answer
33 views

Gamma-gamma conjugacy for rate parameter of Gamma distribution

the question is as follows. Assume the shape $r$ is a known constant. For $x \sim$ Gamma(shape = $r$, rate = $v$), the p.d.f is: $$p(x|r,v) = x^{r-1}e^{-vx}v^r/\Gamma(r)$$ a) Show that the $\theta \...
0
votes
1answer
58 views

Survival time problem exponential with gamma prior

The survival times, in days, of patients diagnosed with a severe form of a terminal illness are thought to be well modelled by an exponential($\theta$) distribution. We observe the survival times ...
1
vote
1answer
47 views

Obtaining posteriors for multivariate Normal mixture models

So I want to fit a mixture model $$f(y) = \pi_1 f_1 (y) + \pi_2 f_2 (y)$$ where $\pi_k = P(S = k)$ and $S_i$ is a latent unobserved variable. I assume that, conditional on $S=k$, we have the model ...
0
votes
3answers
161 views

Conjugate prior, unclear definition

Consider the following definition: A family $\cal F$ of probability distributions on $\Theta$ is said to be conjugate (or closed under sampling) for a likelihood function $f(x|\theta)$ if for every $\...
3
votes
0answers
186 views

Predictive Posterior Distribution of Normal Distribution with Unknown Mean and Variance

Suppose that $x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$ for $i = 1,...n$. Assume that the assigned prior distributions are $\mu$ ~ $\mathcal{N}$($\mu_{0}$, $\sigma^{2}_{0}$) and $\tau \...
0
votes
0answers
50 views

Coverage probability for the Bayesian credible interval for Normal distribution

Bayesian Inference for the Normal Distribution, I use the following r code to obtain the posterior distribution. Let's say the data, $X\sim N(\mu, \sigma^{2})$ and $\mu \sim N(0,10)$ and $\sigma \sim \...
3
votes
0answers
32 views

Can two families of distributions be conjugate priors for one another?

Is it possible to find two families of distributions $\mathcal F$ and $\mathcal G$ such that: $\mathcal F$ is a conjugate prior for $\mathcal G$ $\mathcal G$ is a conjugate prior for $\mathcal F$ $\...
2
votes
1answer
31 views

Prior For Gaussian R.V.s with Common Variance

Suppose we observe $n$ independent random variables $X_1, \dots, X_n$. Suppose also that the mean and variance of each $X_i$ is unknown. If $$X_i \sim \mathrm{N}(\mu_i,\sigma^2_i)$$ then a conjugate ...
4
votes
2answers
70 views

MCMC combined with numerical integration towards more efficient Bayesian inference

I am quite new to Bayesian statistics so the question can be a bit naive. My question is on how to deal with a model with individual coefficients. Simple versions of a task and a model I deal with is ...
1
vote
1answer
196 views

Gamma Conjugate Prior & Poisson Process

I am analyzing daily data transaction data. I am assuming that The number of transactions in every day of length t has the Poisson distribution with mean λt The number of transactions in evert ...
2
votes
2answers
86 views

Should updating one data point at a time or all change the posterior of a normal-inverse-gamma?

I have implemented the normal inverse gamma distribution per section 3 of https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf in some code. However, I've noticed ...
4
votes
1answer
55 views

How many natural parameters are really in the exponential family conjugate prior?

The exponential family with natural parameter $\theta$ can be written $$ p(x|\theta)=h_\ell(x)\exp(\theta^Tt(x)-a_\ell(\theta)) $$ with conjugate prior $$ p(\theta|\lambda)=h_c(\theta)\exp(\lambda_1^T\...
0
votes
0answers
29 views

Hybrid Bayesian Network - Conjugute Prior for Bernoulli Variable with Discrete and Continuous Parents

I have a discrete variable $x_a$ for which the likelihood function is Bernoulli. Within the Hybrid Bayesian Network, $Parents(x_a)$ includes both continuous and discrete variables. Is there a ...
0
votes
0answers
20 views

P-values Calculation as significance (Pseudo-counts and Hypergeometric)

I am looking for a way to solve this problem I have run k-means to obtain a set of clusters with elements, some of this clusters have 1 or 2 elements in them. I use the hypergeometric function to ...
0
votes
1answer
155 views

Intuitive explanation of Inverse Wishart prior for covariance estimation

I am trying to understand what is going on in the use of an Inverse Wishart prior for (Gaussian) covariance, and what is the motivation for it. I am seeing this posed as a solution for when the ...
0
votes
0answers
7 views

Combining AEVB with conjugate exponential family observations

I'm trying to build a probabilistic model that is a combination of a neural network and a graphical model; namely it uses a MLP as an encoder network, and the "decoder" is an exponential family ...
0
votes
0answers
73 views

Are conjugate priors unique in the case of normal distributions?

Suppose I have the following hierarchical normal model: $X \sim N(\theta, \tau^2)$ $ X | \mu \sim N(\mu, \sigma^2)$ I suppose $\sigma^2$ and $\tau^2$ are known and fixed. Then does $\mu$ ...
4
votes
1answer
627 views

Why do we use inverse Gamma as prior on variance, when empirical variance is Gamma (chi square)

Let $$X_i\sim \mathcal{N}(0,\sigma^2)$$ than we know that $$\sum_{i=1}^N\frac{X_i^2}{N}\sim\Gamma(\frac{N}{2},\frac{2\sigma^2}{N})$$ that the empirical variance follows a Gamma distribution. How do ...
0
votes
1answer
44 views

Prior for mean unknown parameter

We know that there are two versions of exponential $\big(1\big)$ $exp(\lambda)$ with pdf $$f(x;\lambda)=\lambda e^{-\lambda x}$$ ,$\lambda>0,x>0$ and $\mathbb{E}[x]=\frac{1}{\lambda}$ for ...
0
votes
0answers
36 views

how to get posterior distribution of beta with gamma prior

I have $X_1, ..., X_n \sim beta(\theta,1)$ and $\theta \sim gamma(r, \lambda)$ and wish to compute the posterior distribution. Since $f(\textbf{X} | \theta) = \theta^nx^{n(\theta-1)}$ and $\pi(\...
2
votes
0answers
57 views

Bayesian inference - conjugate priors and MAP

I came across a exercise on Bayesian inference with conjugate priors and MAP estimation: When you pet your cat, she might scratch you with probability $p$ or start purring with probability $1-p$. You ...
4
votes
0answers
61 views

Bayesian Multivariate Normal-Normal Model when covariance depends on the mean

Let $y$ denote a D-dimensional multivariate normal random variable with density $$y \sim N(\mu, \Sigma(\mu)) $$ such that the covariance $\Sigma$ is a deterministic non-linear function of $\mu$. ...
3
votes
2answers
125 views

Conjugate prior distribution for a specific (normal) curved exponential family

Is there a conjugate prior distribution in the model $\text{N}(\theta, \sigma^2=\theta^2)$, that is a normal distribution where the mean and standard deviation are equal ($\theta>0$)? (This ...
1
vote
0answers
24 views

Computing the posterior for the splitted observation in a collapsed Gibbs sampler using conjugacy for the Gaussian distribution

It has shown in the Doshi 2009 that we can attain the speed of uncollapsed Gibbs sampler using collapsed Gibbs sampler by splitting the observations into two parts. In Equation (8), the posterior of $...
2
votes
2answers
62 views

Marginalize Gaussian Random Variable completing the square [duplicate]

I have $$P(x | \lambda_1, \lambda_2) = \mathcal{N}(\mu + L_1 \lambda_1 + L_2 \lambda_2, \Sigma)$$ and a Gaussian prior for $$P(\lambda_1) = P(\lambda_2) = \mathcal{N}(0, I)$$. My task is to compute $...
3
votes
1answer
120 views

How to quantify strength of beta prior?

How would one interpret the strength of prior belief associated with a parameter with a prior beta(10,8) distribution, compared to a beta(0,0) prior, (with data from a binomial distribution)? Some ...
11
votes
2answers
350 views

Bayes estimator are immune to selection Bias

Are Bayes estimators immune to selection bias? Most papers that discuss estimation in high dimension, e.g., whole genome sequence data, will often raise the issue of selection bias. Selection bias ...
0
votes
1answer
88 views

Inverse-Wishart prior in multivariate linear regression

The Bayesian Multivariate Linear Regression wikipedia article states that posterior parameters of a multivariate linear regression model with a Inverse-Wishart prior can be obtained from the following ...
1
vote
2answers
355 views

Haldane's prior Beta(0,0) - Part 1

This article$^1$ on p.16 specifies Haldane's prior as: $$p(\theta) = \frac{1}{θ(1−θ)}$$. However, other$^2$ source on p.6 specifies Haldane's prior as proportional to $\frac{1}{θ(1−θ)}$, i.e. $$p(\...
2
votes
0answers
57 views

conjugate prior (and posterior) for Matrix Variate Distributions

I was wondering what is the conjugate prior for Matrix Variate Distributions (e.g., unknown mean, known variance matrices), and what's the corresponding posterior? Is there analytical solution?
1
vote
1answer
176 views

Binomial Likelihood for bayesian statistics

We know that Beta distribution is conjugate prior for binomial likelihood. In a course, I am doing on coursera, instructor used beta distribution with alpha as k + 1 and beta as n-k+1 as likelihood ...
0
votes
1answer
52 views

Conjugate normal distribution

Given a random variable $X$ which follows a normal distribution $X \sim (\mu,\sigma^2)$ with unknown mean and known variance. Say we have $n$ observations $y_i$ where $1\le i \le n$ and each $y_i$ is ...
2
votes
0answers
44 views

Parameters' value for weakly informative normal inverse wishart prior

What should one take as parameters for a normal inverse Wishart prior to be weakly informative? Is there a standard? Thank you
3
votes
2answers
358 views

beta-binomial as conjugate to hypergeometric

According to the table of conjugate distributions on Wikipedia, the hypergeometric distribution has as conjugate prior a beta-binomial distribution, where the parameter of interest is "$M$, the number ...
2
votes
1answer
256 views

Predictive Density for Dirichlet Multinomial

I am wondering what the predictive distribution of a Dirichlet-Multinomial distribution is. In this tutorial (page 24), the predictive density is simple and something like "pseudo samples." However, ...
1
vote
1answer
198 views

Why does MCMCpack use normal priors when running Poisson regression?

I thought that since the conjugate prior of Poisson distribution is gamma, we needed to use that when assigning prior distributions to the beta coefficient. MCMCpack and rstanarm both specify a ...
6
votes
2answers
181 views

Bayesian inference on the correlation parameter of a bivariate normal

Suppose that the data $y_i$'s are from the following bivariate normal $$y_i\sim \mathcal{N}\bigg(\mu,\left[ {\begin{array}{cc} \sigma_{11} & \sqrt{\sigma_{11}\sigma_{22}}\rho \\ \sqrt{\...
5
votes
1answer
505 views

Help with Bayesian derivation of normal model with conjugate prior

I'm reading Marin's and Robert's Bayesian Essentials with R to work on the foundations of my understanding of Bayesian analysis. In the first chapter, about normal models, I'm having some trouble ...
0
votes
1answer
866 views

Conjugate prior for inverse Gamma with known scale parameter

Suppose $Y \sim \text{Inverse Gamma}(\alpha, \beta)$ with scale parameter $\beta$ known, and $\alpha$ unknown, and the pdf is given by $$f(y) = \frac{\exp(-1/\beta y)}{\Gamma(\alpha) \beta^\alpha y^...