Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [conjugate-prior]

A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.

Filter by
Sorted by
Tagged with
2
votes
1answer
35 views

Bayesian A/B test analysis for non-binary actions

When analyzing experiments with binary action (for example, will a user convert or not), then Beta-Binomial distribution is apt for that. But I am not able to figure out what distributions to use ...
2
votes
0answers
44 views

Conjuage priors for linear combination

My knwoledge of statistics in general and Bayesian statistics in particular is limited. With that in mind, I would sincerely appreciate if somebody could help me with the following problem that I have ...
1
vote
1answer
80 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
1
vote
1answer
30 views

Bayesian Linear Regression, trouble with posterior. Variance equal identity

I am trying to solve the following problem. If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function ...
1
vote
1answer
31 views

Literature on Bayesian stuff with Normal Distribution? [on hold]

I am writing something on Bayesian Analysis involving the normal distribution. I know that the conjugate prior is the so-called normalized Gamma inverse distribution, I know the update rule for the ...
0
votes
1answer
24 views

Gibbs sampling: Conjugate prior for a two component known-unknown variance?

If I have a normally distributed variable $\mathcal{N}(\mu,\frac{1}{\tau})$ with fixed $\mu$ then the conjugate prior for an unknown $\tau$ is then $\mathcal{Ga}(\frac{n}{2}+\alpha, \beta + \sum_i \...
1
vote
0answers
21 views

Posterior (conjugate) prior of two parameter Gamma likelihood

This question is related to a previous question on this site. Assume some data is generated from Gamma distribution $p(x\mid\alpha,\beta) \sim \operatorname{Gamma} (\alpha,\beta)$, and both parameters ...
2
votes
0answers
46 views

Posterior distribution for Weibull scale parameter with censored data

I am modeling a survival problem using a Weibull distribution with known shape parameter $k$ and unknown scale parameter $\theta$. The PDF and CDF are given by, \begin{align} f(t|k,\theta)=\frac{k}{\...
0
votes
1answer
24 views

what is the step by step procedure of updating a posterior probability with new data coming in

I have a question about how to update posterior probability sequentially when new data comes in sequentially by say $x_1$, then $x_2$, then $x_3$,.... I understand this form when the first data $x_1$ ...
1
vote
0answers
31 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
4
votes
2answers
47 views

What is the relation between “conjugate priors” and the approximate inference?

I know that "conjugate prior" is to help us calculate the the denominator of the Bayes formula(to make the calculations easier). And I just learnt to approximate the inference by mean field ...
0
votes
0answers
44 views

If $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ conjugate to $p(\theta)$?

If exponential family $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ for $r>0$ conjugate to $p(\theta)$? If not, what can we do about it in terms of sampling to make use of ...
0
votes
0answers
30 views

Is there a conjugate prior for normal location family N(x|u,1) such that the mean is always positive?

The conjugate prior to normal location family is usually a normal distribution. However, I want to constrain the mean to be positive. Is there a conjugate prior to the normal location family $x\sim N(...
0
votes
0answers
16 views

Shape of parameters marginal posterior in hierarchical Bayes model

Consider a generic hierarchical Bayes model with data $y_i\sim p(y|\theta_i)$, dependent of parameters $\theta_i\sim p(\theta|\phi)$ and hyperparameters $\phi\sim p(\phi)$. Furthermore, assume that $\...
0
votes
0answers
9 views

Tuning the “strength” of updates to a posterior distribution for conjugates

I'm asking this question as a sanity check- I am not a statistician or research scientist, and just am doing a gut check on a model I am building. I want to quantify uncertainty of a specific metric ...
0
votes
0answers
40 views

Metropolis-within-Gibbs for parametric inference of a regressive model

I have a regressive model of this form \begin{equation} y=f(\theta)+\varepsilon \end{equation} to describe observations $y$, with noise $\varepsilon$ and a parametric function $f$ with parameters $\...
0
votes
0answers
24 views

Conjugate prior for DPGMMs using Gibbs sampling

I am using Gibbs sampling to infer DPGMMs. The prior for multivariate Gaussians is Normal-inverse Wishart. But it turns out that the covariances are not estimated accurately. Here is codes and results....
0
votes
0answers
13 views

In bayesian updating, how to keep precision matrix suceptible to data?

In bayesian updating, I want precision matrix, or standard errors to keep susceptible, if much data comes. I am considering bayesian updating of simple regresion, using conjugate prior of Normal-...
0
votes
1answer
22 views

Determine hyper-prior for gaussian distribution from existing data [closed]

Not sure how to determine hyper-prior for prior distributions, specifically using historical data. First what I am doing: I want to estimate parameters for a normal likelihood function using Bayesian ...
2
votes
1answer
34 views

Understanding of conjugation relationship in Bishop book

Referring to Pattern Recognition and Machine Learning by Bishop(Page 367, Section 8.1): Such models have particularly nice properties if we choose the relationship between each parent-child pair in ...
0
votes
0answers
24 views

Uniform conjugate prior for a Beta distribution

Given $$ \pi \sim \text{Beta}(\alpha, \beta) $$ I'd like to place a prior on $\alpha$ and $\beta$. The "trick" mentioned in this post and this post seems to be to recognize that since $$ \begin{...
0
votes
0answers
22 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
2
votes
1answer
55 views

Compute conjugate prior from the sample distribution

I feel like this question might be marked as duplicate because I see many similar incurring in that fate but I'll try anyway. I would say I did not find anything similar. I have been thought a ...
1
vote
1answer
59 views

Bayesian inference on binarized Poisson distribution

I have a variable that is Poisson distributed. Let's say I have a number of boxes each with a number of balls inside according to a Poisson distribution, with $\lambda=0.4$, (the average number of ...
2
votes
1answer
46 views

How does L2 penalize large weights

The L2 regularization term is useful because it penalizes large weights over smaller weights which is good to prevent overfitting. I'm having a hard time understanding how exactly it does this. This ...
3
votes
1answer
44 views

Joint AR(1) posterior distribution explicit under conjugate prior

I have encountered a problem in my textbook 'The Bayesian Choice' by Christian P. Robert. It goes something like this: $"$For a particular case of AR(1) model, $(x_t)_{1\leq t\leq T}$. Where $x_t = \...
1
vote
1answer
91 views

Mean of the posterior distribution in bayesian linear regression with infinitely broad prior

Currently reading from Christopher Bishop's Pattern Recognition and Machine Learning book about parameter distribution under a bayesian linear regression. On page 153, the author deduces that the ...
0
votes
0answers
35 views

Does Levy distribution has a conjugate prior?

I have searched a lot but can not find the conjugate prior of Levy distribution.
1
vote
0answers
104 views

Proving Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model

How does one prove that the Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model? This prior is a generalization of the Normal-Inverse-Wishart Distribution. By Matrix-...
5
votes
1answer
150 views

Truncated Gamma Distribution

The Gamma distribution is the conjugate prior of Poisson distribution. What about the Truncated Gamma distribution? Is it still the conjugate prior of Poisson distribution?
1
vote
1answer
254 views

Conjugate Prior for Student T distribution

Does the Student T distribution have a conjugate prior distribution? If so, what is it and what are the parameters?
0
votes
0answers
26 views

Is the posterior distribution for the model described in this question Gaussian?

I was in the middle of writing a long answer to Uncertainty estimation in high-dimensional inference problems without sampling? but I was suddenly struck by doubt: since the model assumes a Gaussian ...
2
votes
1answer
37 views

Computing posterior density of Normal with unknown $\mu$ and $\sigma^2$

(From Bayesian Essentials with R by Marin & Robert page 31) We are given an iid sample $\mathfrak{D}_n = (x_1, \dots, x_n)$ from the normal distribution $\mathcal{N}(0, \sigma^2)$ and $\theta=(\...
1
vote
1answer
370 views

How to use the Dirichlet prior for estimating the multinomial parameters? [closed]

I know that the multinomial distribution gives the likelihood of some vector D of occurrences to happen given a probability vector (parameters) P' i.e. P(D|P'). Now with a Dirichlet prior we are ...
1
vote
1answer
81 views

Maximum A Posteriori Estimate

The formula for calculating the MAP estimate of a particular parameter, $p$, is given by the following: $p^{MAP} =$ argmax $P(p)P(p|x)$. Now I am trying to do a question where I am told the prior ...
1
vote
0answers
15 views

Eigenfunctions heuristics for self-conjugate priors [duplicate]

Previously asked on math.stackexchange. I am looking for a citable reference (books, research papers, PhD theses, not websites, internal reports, etc.) about the heuristic interpretation of self-...
1
vote
1answer
325 views

Interpretation of the rate parameter of a Gamma distribution

I am toying with mixture models, especially in a bayesian context and the Gamma (or the inverse Gamma) distribution appears quite often. For example, inverse Gamma is used as a conjugate prior for the ...
0
votes
1answer
25 views

Bernoulli distribution/ SOME probability/conjugate prior

I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it? EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
0
votes
2answers
37 views

Beta distribution and normalization [duplicate]

Here in the 4 pictures in the last answer, is the vertical axe the probability? I.e. it seems to me that it is somewhat unnormalized : it has the value 2 in the 2nd picture and 3 in the 3rd picture. ...
2
votes
1answer
167 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
1
vote
1answer
39 views

Basic question on proportionality in Bayesian Inference for Normal distribution

I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that: $\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$ $Y | \theta \...
4
votes
1answer
548 views

If both prior and likelihood are Gaussian what can we say about the posterior? [closed]

If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior? As posterior is proporional to prior*likelihood which are Gaussians, the posterior ...
5
votes
2answers
266 views

The meaning of Bayesian update

I'm new in Bayesian inference and I can't found the answer to this: In real life scenario people use MCMC to compute the posterior distribution given the likelihood and the prior. Analytical ...
1
vote
1answer
85 views

Normal - Inv chi squared posterior calculation

Given that for a known mean $\mu$ and unknown variance $\sigma^2$ the normal distribution is $$X_i|\sigma^2 \sim \mathcal{N}(\mu, \sigma^2) = \frac{1}{\displaystyle\sigma\sqrt{2\pi}}\exp\left[-\...
1
vote
1answer
128 views

Gamma-gamma conjugacy for rate parameter of Gamma distribution

the question is as follows. Assume the shape $r$ is a known constant. For $x \sim$ Gamma(shape = $r$, rate = $v$), the p.d.f is: $$p(x|r,v) = x^{r-1}e^{-vx}v^r/\Gamma(r)$$ a) Show that the $\theta \...
0
votes
1answer
234 views

Survival time problem exponential with gamma prior

The survival times, in days, of patients diagnosed with a severe form of a terminal illness are thought to be well modelled by an exponential($\theta$) distribution. We observe the survival times ...
1
vote
1answer
54 views

Obtaining posteriors for multivariate Normal mixture models

So I want to fit a mixture model $$f(y) = \pi_1 f_1 (y) + \pi_2 f_2 (y)$$ where $\pi_k = P(S = k)$ and $S_i$ is a latent unobserved variable. I assume that, conditional on $S=k$, we have the model ...
0
votes
3answers
203 views

Conjugate prior, unclear definition

Consider the following definition: A family $\cal F$ of probability distributions on $\Theta$ is said to be conjugate (or closed under sampling) for a likelihood function $f(x|\theta)$ if for every $\...
3
votes
0answers
401 views

Predictive Posterior Distribution of Normal Distribution with Unknown Mean and Variance

Suppose that $x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$ for $i = 1,...n$. Assume that the assigned prior distributions are $\mu$ ~ $\mathcal{N}$($\mu_{0}$, $\sigma^{2}_{0}$) and $\tau \...
0
votes
0answers
188 views

Coverage probability for the Bayesian credible interval for Normal distribution

Bayesian Inference for the Normal Distribution, I use the following r code to obtain the posterior distribution. Let's say the data, $X\sim N(\mu, \sigma^{2})$ and $\mu \sim N(0,10)$ and $\sigma \sim \...