Questions tagged [conjugate-prior]
A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.
318
questions
1
vote
1
answer
25
views
(un)conditional density from other densties in continuous beta model
This question relates to my other question, where I actually noticed a mistake. Suppose I have a random variable $X\in[0,1]$ and a signal $S\in[0,1]$ bearing some info about $X$. I know $f_X(x)$ and $...
- 123
1
vote
0
answers
87
views
How to define (natural) conjugate prior for this linear model?
I have the general linear regression model, $\ y = Xβ+e $ , where $\ e ∼ N$ $\ (0, σ^2 * I_T),
$\ X $ is a (T × K) fixed non-stochastic matrix of regressors, $\ y $ is a (T × 1) vector of ...
- 285
1
vote
0
answers
380
views
Variance of multinomial and Dirichlet-multinomial distributions
I have an application where I would like to sample from a multinomial distribution, but I am concerned that the variance will be too low. As an alternative, I am considering the Dirichlet-multinomial ...
- 411
0
votes
2
answers
178
views
How do I determine my Bayesian sampling size when comparing two proportions?
I am currently working on writing a simulation in R to compare the results of Frequentist vs Bayesian when it comes to two-proportion hypothesis testing. For the Bayesian side, I am simply using a ...
- 235
2
votes
2
answers
527
views
Updating conjugate priors with noisy observations
I'm considering a problem that has been partially addressed elsewhere:
Bayesian updating with conjugate priors using the closed form expressions
but now I have an added twist. My samples are drawn ...
- 123
0
votes
0
answers
140
views
why use MH for Gaussian prior when its the conjugate prior
As I understand, gaussian likelihood function has a conjugate prior for μ which is also a gaussian. In that case, the posterior can be derived in closed form. Why do some many papers use metropolis ...
- 547
4
votes
1
answer
767
views
Bayes Factor and likelihood for two sample from different distributions?
I'd like to calculate Bayes Factor for two-sample t-test $H_0: \mu_1=\mu_2$ (model $M_0$) against $H_1: \mu_1\not=\mu_2$ (model $M_1$)
My data are: $x_1,x_2,\ldots, x_{n_1}\sim N(\mu_1, \sigma)$ ...
- 1,458
0
votes
1
answer
2k
views
Calculating the parameters of a Normal distribution using alpha and beta from Inverse-gamma (conjugate prior)
How is it possible to calculate the variance $\sigma^2$ for the Normal distribution if only $\alpha$ and $\beta$ (based on data) from the Inverse-gamma distribution are available? I followed the ...
- 35
5
votes
1
answer
1k
views
Conjugate beta / interpretation of the "continuous binomial" signal
Note: this question has significantly evolved, thanks to inspiring comments by Tim.
Assume there is some "truth" $x\in[0,1]=Beta(1,1)$ that is signaled with some precision. I assume that the ...
- 123
5
votes
1
answer
71
views
Posterior probability when data consists of $k$ largest of $N$ samples
Given an underlying unknown distribution, I sample $N$ numbers. From those $N$ numbers I take the highest $k$ numbers. How do I model the posterior probability from those $k$ numbers.
I know I can ...
- 203
3
votes
1
answer
1k
views
Prior elicitation with Inverse Gamma and parametrization issue
This is a homework problem. I am very new to Bayesian conjugate analysis, so hang on with me.
I have a sample $(x_1...x_n)$ of $n=20$ observations from an experiment. These observations are breakdown ...
- 694
2
votes
0
answers
47
views
How to determine whether a given likelihood function has a conjugate prior? [duplicate]
Does there exists a certain formal procedure one has to go through before being able to claim that no conjugate prior $p(\theta)$ exists for the given likelihood function $p(X | \theta)$?
In other ...
- 446
3
votes
1
answer
686
views
Existence of conditional conjugate prior for the dispersion parameter of a negative binomial model
Is there a conditional conjugate prior distribution for the dispersion parameter of the negative binomial distribution given the mean rate parameter, if I parameterize it in this manner
$$f(y_i; \mu, \...
- 28.2k
2
votes
0
answers
179
views
Is there a reasonable way to impose a prior within a likelihood-based model?
I have been using GAM (mgcv's gam()) to perform a fairly complex and computationally intensive analysis - I have millions of observations and dozens of terms ...
- 10.6k
0
votes
1
answer
573
views
Bayesian updating with conjugate prior (specific example)
This question deals with Bayesian updating with conjugate prior.Suppose we have a prior distribution of N~(5, 3) and then we observe 5 data points (8, 9, 10, 8, 7) (assumed to be taken randomly from a ...
- 255
25
votes
3
answers
33k
views
Bayesian updating with new data
How do we go about calculating a posterior with a prior N~(a, b) after observing n data points? I assume that we have to calculate the sample mean and variance of the data points and do some sort of ...
- 255
2
votes
0
answers
164
views
Recursive Bayesian estimation for the coefficients of a convex combination
I'm given a sequential measurements of vectors $\vec{v}_t\in R^{K+1}$ such that $v_{t,0}$ is a convex combination of $\{v_{t,k}\}_{k\ge 1}$ (i.e. $v_{t.0}=\sum_{k\ge 1}{w_{t,k}v_{t,k}}$ for some ...
- 510
2
votes
1
answer
4k
views
Bayesian updating with conjugate priors using the closed form expressions
I have one two data sets of scalar values: one large data set (about 700 data points) and one small data set (80 data points). I would like to update the large data set with the small one using the ...
- 133
0
votes
1
answer
236
views
Bayesian inference with conjugate priors - triplot
I'm trying to make a "triplot" to illustrate Bayesian inference (so I'd like to have prior, likelihood and posterior in the same picture). For likelihood I'm using
\begin{equation}\label{eq:lik}
f(y|\...
- 278
4
votes
0
answers
1k
views
A mixture of conjugate priors is conjugate
I want to prove that a mixture of conjugate priors is itself conjugate. It does not look difficult, but I'm still a bit unsure when manipulating probabilities, especially in a Bayesian context. Is ...
- 2,211
1
vote
1
answer
2k
views
Update gamma prior with new rate parameter instead of observations
From wikipedia:
In Bayesian inference, the conjugate prior for the rate parameter $λ$
of the Poisson distribution is the gamma distribution. Let
$$\lambda \sim \mathrm{Gamma}(\alpha, \beta)$$...
- 4,020
4
votes
0
answers
2k
views
Posterior distribution under Cauchy prior?
I have a (I hope) simple question! If I had a linear regression,
$Y_t = \alpha + \beta X_t + \epsilon_t$
with $\epsilon_t \sim N(0,\sigma^2)$
and I assume a Cauchy prior for $\sigma$, is it ...
- 153
1
vote
0
answers
817
views
Python Bayesian invgamma.rvs - joint posterior of normal distribution sampling
(My question is inspired by this blog post: The Bayesian analysis of normal distributions with Python. If you read it, you will get a good background on what I am asking.)
I am trying to model the ...
- 11
4
votes
1
answer
1k
views
Gibbs sampling and Conjugate Priors
Are conjugate priors required when performing Gibbs sampling?
- 53
4
votes
0
answers
111
views
what is this property? $\int p(x,\pi)d\pi=p(x|E[\pi])$?
Sorry if the title does not make sense, from the answer of this question Mistake in derivation about categorical distribution and Dirichlet distribution? it can be shown that
say $p(x|\pi)$ follows ...
- 15.3k
0
votes
1
answer
172
views
Conjugate prior for parameter W when the likelihood is normal with mean and variance both functions of W
Suppose that $x$ is an observable scalar variable and
$$
x \thicksim N(W\mu_0,W^2\sigma_0^2)
$$
Where $W$ is a parameter that must be estimated from data, and $\mu_0$ , $\sigma_0$ are known constants....
- 58
1
vote
1
answer
345
views
Mistake in derivation about categorical distribution and Dirichlet distribution?
$p(x|\pi)$ follows the categorical distribution (the multinomial with one observation), where $\sum\pi_i=1$ and $x$ is a one-hot vector, and $p(\pi|\alpha)$ follows the Dirichlet distribution.
$p(x|\...
- 15.3k
6
votes
1
answer
808
views
Overestimation of the noise precision in Bayesian linear regression when $n\gtrsim p$
I would like to infer the regression coefficients and the noise precision of a standard linear regression problem defined by
$$
y=X\theta + \epsilon,
$$
where $X$ is a $n\times p$ design matrix, $\...
- 1,017
4
votes
1
answer
2k
views
conjugate prior for (multivariate) Gaussian mixtures (with known mean and covariance)?
Say I have a (multivariate) Gaussian mixture model $$p(x)=\sum_k\pi_iN(\mu_i,\Sigma_i),$$
of which the $\boldsymbol\mu$ and $\boldsymbol\Sigma$ are known, so the likelihood function of the ...
- 15.3k
2
votes
1
answer
1k
views
Marginal prior $p(\mu)$ of mean of a normal distribution when both mean and variance are unknown
I read that if the data is normally distributed with mean $\mu$ and variance $\sigma^2$ (both unknown) then to have the joint posterior distribution $p(\mu, \sigma^2 | y)$ in closed form, one has to ...
- 1,013
1
vote
1
answer
176
views
Are there analytically derivable posteriors that save from doing MCMC other than conjugate priors? [duplicate]
Posteriors for conjugate priors can be analytically derived and save us from doing MCMC.
Conjugate priors simply have a posterior in the same family as the prior distribution. Are there other ...
user46925
5
votes
1
answer
344
views
Do conjugate priors just lead to a posterior that is a modification of the parameters of the prior?
We know, for example, that the conjugate relationship between the classic beta-binomial is as follows:
\begin{align}
y &∼ \mathcal{Bin}(n,\ θ) \\
θ &∼ \mathcal{Beta}(α,\ β) \\
θ|y &∼ \...
user46925
2
votes
1
answer
598
views
Better understanding of Posterior Hyper Parameters for Normal Likelihood with known Variance
I must be missing something in terms of the way the posterior distribution is parameterized when a conjugate (normal) prior is applied to normal data.
From https://en.wikipedia.org/wiki/...
- 31
9
votes
3
answers
1k
views
Gaussian likelihood + which prior = Gaussian Marginal?
Given a Gaussian likelihood for a sample $y$ like $$p(y|\theta) = \mathcal{N}(y;\mu(\theta),\Sigma(\theta))$$ with $\Theta$ being the parameter space and $\mu(\theta)$, $\Sigma(\theta)$ arbitrary ...
- 1,810
2
votes
1
answer
4k
views
How to determine posterior distribution of the parameter in a binomial
Assuming that I performed n iid tests, and the total number of test is n which is a fixed value, and the observaton of 1 which corresponding to successful results is X observations yeild with ...
- 41
13
votes
1
answer
1k
views
Aside from the exponential family, where else can conjugate priors come from?
Do all conjugate priors have to come from the exponential family? If not, what other families are known to have/produce conjugate priors?
- 3,818
8
votes
3
answers
1k
views
Dirichlet conjugate update derivation
I am attempting to derive the update equations for the conjugate to the Dirichlet distribution, as outlined here: https://mathoverflow.net/questions/20399/conjugate-prior-of-the-dirichlet-distribution
...
- 89
7
votes
1
answer
2k
views
Dirichlet Prior for Multinomial
The Dirichlet function is the conjugate prior of the multinomial.
So the posterior is also Dirichlet given some observations.
If e.g. I observe the counts $X=(10,3,4)$ from 17 trials (10 for class 1, ...
- 662
13
votes
3
answers
4k
views
Understanding the Beta conjugate prior in Bayesian inference about a frequency
Following is an excerpt from Bolstad's Introduction to Bayesian Statistics.
For all you experts out there, this might be trivial but I don't understand how the author concludes that we don't have to ...
- 899
2
votes
2
answers
3k
views
How to define prior for beta-binomial A/B test
I would like to run an A/B test using a Bayesian beta-binomial model whereby I would state probabilities such as $P(p_B>p_A)$ in place of using a traditional T-test. I've read that the prior should ...
- 21
3
votes
0
answers
366
views
Conjugate Gamma Prior
If I have a normal distributed variable $N(\mu,\sigma^2)$ so with fixed $\mu$ the conjugate prior for $\lambda:=\frac{1}{\sigma^2}$ is given by the gamma distribution $\propto \lambda^{\alpha-1}exp{-\...
- 125
4
votes
2
answers
472
views
FInding the high density region for a $\chi^2$ (Chi-Squared) distribution?
So I am trying to figure out this problem.
My approach so far has been to consider $\frac{S_o}{\sigma^2} \sim \chi^{2}_k$ as the prior, thus making the posterior $\frac{( S + S_o)}{\sigma^2}\sim\chi^...
- 899
1
vote
0
answers
90
views
conjugate prior for my model parameter
This question is related to the another thread that I posted: Help with Variational Bayes on a weighted linear regression model
To reiterate, I have the model as follows:
$$
y_i \sim \mathcal{N}(T(...
- 4,530
2
votes
1
answer
750
views
Bayesian analysis and Lindley's paradox?
I have this problem I am trying to wrap my head around that my friend's professor created. Can anyone give me some hints on how to get started, particularly in parts b and c?
I have an understanding ...
- 899
35
votes
4
answers
8k
views
Can anyone explain conjugate priors in simplest possible terms?
I have been trying to understand the idea of conjugate priors in Bayesian statistics for a while but I simply don't get it. Can anyone explain the idea in the simplest possible terms, perhaps using ...
- 899
2
votes
1
answer
68
views
From a Bayesian practitioner's standpoint, what are the specific conditions to decide on whether we have conjugacy or not?
I've been learning Bayesian statistical analysis on my spare time using textbooks, videos on YT, etc. I'm slowly going up that mountain. Please correct me if my wording below is poor or ask for ...
- 385
1
vote
1
answer
169
views
The purpose of scaling the normal variance in NIG-distribution
The Normal-Inverse-Gamma distribution is often written as
$N(\phi | \mu, \sigma^2 \Sigma) IG(\sigma^2 | \alpha, \beta),$
and used as a conjugate prior for a linear model given observations
$y_t \...
- 11
3
votes
0
answers
327
views
Finding the Complicated Posterior Probability Distribution of $θ$
Suppose, we are given a likelihood function, $f(x|θ)$ corresponding to a shifted-exponential distribution and the prior distribution on the parameter $θ$ is a standard Cauchy distribution.
Now I am ...
- 579
2
votes
1
answer
752
views
what is "Minimum Length Least Square"
I am in the process of implementing Bayesian Lasso with Normal-Gamma prior; In section 3.3 mention
The prior for the scale parameter $\gamma$ conditional on $\lambda$ is given by $v_\beta = 2 \lambda ...
- 1,128
1
vote
1
answer
187
views
Distribution of partially observable binominal parameter
I suspect this is a textbook question but I don't seem to have the right textbook.
Anyway I am trying to estimate probability of coin landing on heads, p, by repeatedly flipping it N times, i.e., ...
- 13