Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [conjugate-prior]

A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.

1
vote
0answers
20 views

Proving Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model

How does one prove that the Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model? This prior is a generalization of the Normal-Inverse-Wishart Distribution. By Matrix-...
5
votes
1answer
95 views

Truncated Gamma Distribution

The Gamma distribution is the conjugate prior of Poisson distribution. What about the Truncated Gamma distribution? Is it still the conjugate prior of Poisson distribution?
1
vote
1answer
35 views

Conjugate Prior for Student T distribution

Does the Student T distribution have a conjugate prior distribution? If so, what is it and what are the parameters?
0
votes
0answers
18 views

Is the posterior distribution for the model described in this question Gaussian?

I was in the middle of writing a long answer to Uncertainty estimation in high-dimensional inference problems without sampling? but I was suddenly struck by doubt: since the model assumes a Gaussian ...
2
votes
1answer
30 views

Computing posterior density of Normal with unknown $\mu$ and $\sigma^2$

(From Bayesian Essentials with R by Marin & Robert page 31) We are given an iid sample $\mathfrak{D}_n = (x_1, \dots, x_n)$ from the normal distribution $\mathcal{N}(0, \sigma^2)$ and $\theta=(\...
0
votes
0answers
10 views

Concrete example of a generative process using the joint distribution of (Dirichlet prior and Multinomial Distribution)?

I am not a statistician as it is not my field. Please bear with me and correct me when I am wrong. Your help is much appreciated. I know that when having a Dirichlet Prior and Multinomial ...
1
vote
1answer
51 views

What is the point of Dirichlet prior? [closed]

I understand the multinomial distribution and that it can calculate the likelihood of some vector D to happen given the probability vector P' i.e. P(D|P'). Now with Dirichlet prior we are introducing ...
1
vote
1answer
46 views

Maximum A Posteriori Estimate

The formula for calculating the MAP estimate of a particular parameter, $p$, is given by the following: $p^{MAP} =$ argmax $P(p)P(p|x)$. Now I am trying to do a question where I am told the prior ...
1
vote
0answers
14 views

Eigenfunctions heuristics for self-conjugate priors [duplicate]

Previously asked on math.stackexchange. I am looking for a citable reference (books, research papers, PhD theses, not websites, internal reports, etc.) about the heuristic interpretation of self-...
1
vote
1answer
92 views

Interpretation of the rate parameter of a Gamma distribution

I am toying with mixture models, especially in a bayesian context and the Gamma (or the inverse Gamma) distribution appears quite often. For example, inverse Gamma is used as a conjugate prior for the ...
0
votes
1answer
24 views

Bernoulli distribution/ SOME probability/conjugate prior

I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it? EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
0
votes
2answers
27 views

Beta distribution and normalization [duplicate]

Here in the 4 pictures in the last answer, is the vertical axe the probability? I.e. it seems to me that it is somewhat unnormalized : it has the value 2 in the 2nd picture and 3 in the 3rd picture. ...
2
votes
1answer
145 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
1
vote
0answers
53 views

Dirichlet distribution as conjugate prior to Multinomial distribution [closed]

I stumbled upon the following exercise: Show that Dirichlet distribution with parameter $\alpha$ is a conjugate prior to the Multinomial distribution as likelihood. Derive the parameters to the ...
1
vote
1answer
28 views

Basic question on proportionality in Bayesian Inference for Normal distribution

I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that: $\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$ $Y | \theta \...
4
votes
1answer
471 views

If both prior and likelihood are Gaussian what can we say about the posterior? [closed]

If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior? As posterior is proporional to prior*likelihood which are Gaussians, the posterior ...
5
votes
2answers
214 views

The meaning of Bayesian update

I'm new in Bayesian inference and I can't found the answer to this: In real life scenario people use MCMC to compute the posterior distribution given the likelihood and the prior. Analytical ...
1
vote
1answer
21 views

Normal - Inv chi squared posterior calculation

Given that for a known mean $\mu$ and unknown variance $\sigma^2$ the normal distribution is $$X_i|\sigma^2 \sim \mathcal{N}(\mu, \sigma^2) = \frac{1}{\displaystyle\sigma\sqrt{2\pi}}\exp\left[-\...
1
vote
1answer
45 views

Gamma-gamma conjugacy for rate parameter of Gamma distribution

the question is as follows. Assume the shape $r$ is a known constant. For $x \sim$ Gamma(shape = $r$, rate = $v$), the p.d.f is: $$p(x|r,v) = x^{r-1}e^{-vx}v^r/\Gamma(r)$$ a) Show that the $\theta \...
0
votes
1answer
98 views

Survival time problem exponential with gamma prior

The survival times, in days, of patients diagnosed with a severe form of a terminal illness are thought to be well modelled by an exponential($\theta$) distribution. We observe the survival times ...
1
vote
1answer
49 views

Obtaining posteriors for multivariate Normal mixture models

So I want to fit a mixture model $$f(y) = \pi_1 f_1 (y) + \pi_2 f_2 (y)$$ where $\pi_k = P(S = k)$ and $S_i$ is a latent unobserved variable. I assume that, conditional on $S=k$, we have the model ...
0
votes
3answers
174 views

Conjugate prior, unclear definition

Consider the following definition: A family $\cal F$ of probability distributions on $\Theta$ is said to be conjugate (or closed under sampling) for a likelihood function $f(x|\theta)$ if for every $\...
3
votes
0answers
265 views

Predictive Posterior Distribution of Normal Distribution with Unknown Mean and Variance

Suppose that $x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$ for $i = 1,...n$. Assume that the assigned prior distributions are $\mu$ ~ $\mathcal{N}$($\mu_{0}$, $\sigma^{2}_{0}$) and $\tau \...
0
votes
0answers
101 views

Coverage probability for the Bayesian credible interval for Normal distribution

Bayesian Inference for the Normal Distribution, I use the following r code to obtain the posterior distribution. Let's say the data, $X\sim N(\mu, \sigma^{2})$ and $\mu \sim N(0,10)$ and $\sigma \sim \...
3
votes
0answers
36 views

Can two families of distributions be conjugate priors for one another?

Is it possible to find two families of distributions $\mathcal F$ and $\mathcal G$ such that: $\mathcal F$ is a conjugate prior for $\mathcal G$ $\mathcal G$ is a conjugate prior for $\mathcal F$ $\...
2
votes
1answer
37 views

Prior For Gaussian R.V.s with Common Variance

Suppose we observe $n$ independent random variables $X_1, \dots, X_n$. Suppose also that the mean and variance of each $X_i$ is unknown. If $$X_i \sim \mathrm{N}(\mu_i,\sigma^2_i)$$ then a conjugate ...
4
votes
2answers
76 views

MCMC combined with numerical integration towards more efficient Bayesian inference

I am quite new to Bayesian statistics so the question can be a bit naive. My question is on how to deal with a model with individual coefficients. Simple versions of a task and a model I deal with is ...
1
vote
1answer
655 views

Gamma Conjugate Prior & Poisson Process

I am analyzing daily data transaction data. I am assuming that The number of transactions in every day of length t has the Poisson distribution with mean λt The number of transactions in evert ...
2
votes
2answers
101 views

Should updating one data point at a time or all change the posterior of a normal-inverse-gamma?

I have implemented the normal inverse gamma distribution per section 3 of https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf in some code. However, I've noticed ...
4
votes
1answer
82 views

How many natural parameters are really in the exponential family conjugate prior?

The exponential family with natural parameter $\theta$ can be written $$ p(x|\theta)=h_\ell(x)\exp(\theta^Tt(x)-a_\ell(\theta)) $$ with conjugate prior $$ p(\theta|\lambda)=h_c(\theta)\exp(\lambda_1^T\...
0
votes
0answers
29 views

Hybrid Bayesian Network - Conjugute Prior for Bernoulli Variable with Discrete and Continuous Parents

I have a discrete variable $x_a$ for which the likelihood function is Bernoulli. Within the Hybrid Bayesian Network, $Parents(x_a)$ includes both continuous and discrete variables. Is there a ...
0
votes
0answers
20 views

P-values Calculation as significance (Pseudo-counts and Hypergeometric)

I am looking for a way to solve this problem I have run k-means to obtain a set of clusters with elements, some of this clusters have 1 or 2 elements in them. I use the hypergeometric function to ...
1
vote
1answer
252 views

Intuitive explanation of Inverse Wishart prior for covariance estimation

I am trying to understand what is going on in the use of an Inverse Wishart prior for (Gaussian) covariance, and what is the motivation for it. I am seeing this posed as a solution for when the ...
0
votes
0answers
7 views

Combining AEVB with conjugate exponential family observations

I'm trying to build a probabilistic model that is a combination of a neural network and a graphical model; namely it uses a MLP as an encoder network, and the "decoder" is an exponential family ...
4
votes
1answer
966 views

Why do we use inverse Gamma as prior on variance, when empirical variance is Gamma (chi square)

Let $$X_i\sim \mathcal{N}(0,\sigma^2)$$ than we know that $$\sum_{i=1}^N\frac{X_i^2}{N}\sim\Gamma(\frac{N}{2},\frac{2\sigma^2}{N})$$ that the empirical variance follows a Gamma distribution. How do ...
0
votes
1answer
46 views

Prior for mean unknown parameter

We know that there are two versions of exponential $\big(1\big)$ $exp(\lambda)$ with pdf $$f(x;\lambda)=\lambda e^{-\lambda x}$$ ,$\lambda>0,x>0$ and $\mathbb{E}[x]=\frac{1}{\lambda}$ for ...
0
votes
0answers
47 views

how to get posterior distribution of beta with gamma prior

I have $X_1, ..., X_n \sim beta(\theta,1)$ and $\theta \sim gamma(r, \lambda)$ and wish to compute the posterior distribution. Since $f(\textbf{X} | \theta) = \theta^nx^{n(\theta-1)}$ and $\pi(\...
2
votes
0answers
82 views

Bayesian inference - conjugate priors and MAP

I came across a exercise on Bayesian inference with conjugate priors and MAP estimation: When you pet your cat, she might scratch you with probability $p$ or start purring with probability $1-p$. You ...
4
votes
0answers
94 views

Bayesian Multivariate Normal-Normal Model when covariance depends on the mean

Let $y$ denote a D-dimensional multivariate normal random variable with density $$y \sim N(\mu, \Sigma(\mu)) $$ such that the covariance $\Sigma$ is a deterministic non-linear function of $\mu$. ...
3
votes
2answers
141 views

Conjugate prior distribution for a specific (normal) curved exponential family

Is there a conjugate prior distribution in the model $\text{N}(\theta, \sigma^2=\theta^2)$, that is a normal distribution where the mean and standard deviation are equal ($\theta>0$)? (This ...
1
vote
0answers
24 views

Computing the posterior for the splitted observation in a collapsed Gibbs sampler using conjugacy for the Gaussian distribution

It has shown in the Doshi 2009 that we can attain the speed of uncollapsed Gibbs sampler using collapsed Gibbs sampler by splitting the observations into two parts. In Equation (8), the posterior of $...
2
votes
2answers
78 views

Marginalize Gaussian Random Variable completing the square [duplicate]

I have $$P(x | \lambda_1, \lambda_2) = \mathcal{N}(\mu + L_1 \lambda_1 + L_2 \lambda_2, \Sigma)$$ and a Gaussian prior for $$P(\lambda_1) = P(\lambda_2) = \mathcal{N}(0, I)$$. My task is to compute $...
3
votes
1answer
145 views

How to quantify strength of beta prior?

How would one interpret the strength of prior belief associated with a parameter with a prior beta(10,8) distribution, compared to a beta(0,0) prior, (with data from a binomial distribution)? Some ...
11
votes
2answers
394 views

Bayes estimator are immune to selection Bias

Are Bayes estimators immune to selection bias? Most papers that discuss estimation in high dimension, e.g., whole genome sequence data, will often raise the issue of selection bias. Selection bias ...
0
votes
1answer
107 views

Inverse-Wishart prior in multivariate linear regression

The Bayesian Multivariate Linear Regression wikipedia article states that posterior parameters of a multivariate linear regression model with a Inverse-Wishart prior can be obtained from the following ...
1
vote
2answers
552 views

Haldane's prior Beta(0,0) - Part 1

This article$^1$ on p.16 specifies Haldane's prior as: $$p(\theta) = \frac{1}{θ(1−θ)}$$. However, other$^2$ source on p.6 specifies Haldane's prior as proportional to $\frac{1}{θ(1−θ)}$, i.e. $$p(\...
2
votes
0answers
65 views

conjugate prior (and posterior) for Matrix Variate Distributions

I was wondering what is the conjugate prior for Matrix Variate Distributions (e.g., unknown mean, known variance matrices), and what's the corresponding posterior? Is there analytical solution?
1
vote
1answer
204 views

Binomial Likelihood for bayesian statistics

We know that Beta distribution is conjugate prior for binomial likelihood. In a course, I am doing on coursera, instructor used beta distribution with alpha as k + 1 and beta as n-k+1 as likelihood ...
0
votes
1answer
58 views

Conjugate normal distribution

Given a random variable $X$ which follows a normal distribution $X \sim (\mu,\sigma^2)$ with unknown mean and known variance. Say we have $n$ observations $y_i$ where $1\le i \le n$ and each $y_i$ is ...
2
votes
0answers
46 views

Parameters' value for weakly informative normal inverse wishart prior

What should one take as parameters for a normal inverse Wishart prior to be weakly informative? Is there a standard? Thank you