Questions tagged [conjugate-prior]

A prior distribution in Bayesian statistics that is such that, when combined with the likelihood, the resulting posterior is from the same family of distributions.

Filter by
Sorted by
Tagged with
0
votes
1answer
15 views

What is the intuitive meaning of these statements in the context Bayesian prior and posterior?

I am now familiar with the Bayesian thinking process of using a prior and then getting the posterior once we observe data using the prior. I read the following statements which I am trying to get my ...
0
votes
1answer
17 views

Specific step in the proof of conjugate prior for normal distribution with unknown mean and variance

I'm struggling to follow a specific step in the proof that $$ \tau \sim \text{Gamma}(\alpha, \beta), \quad \mu | \tau \sim \mathcal{N}(\nu, \frac{1}{k\tau}) $$ is a conjugate prior distribution for ...
2
votes
1answer
14 views

Deriving the binomial-beta conjugate model

I think I am correctly deriving the binomial-beta conjugate model, but my answer differs slightly from what's on Wikipedia's page on conjugacy. My solution Assume that $$ X_t \sim \text{Binomial}(m,...
0
votes
0answers
18 views

Bayesian factor analysis: Help with posterior derivation

I am trying to derive the Bayesian factor analysis model described on page 10 of this paper. In brief, consider the model (simplified from the paper) $y_i \sim N(M \boldsymbol{f}_i, S), \quad i=1,\...
1
vote
1answer
30 views

Estimate covariance given noisy data and the mean

Basically, I'm inferring the parameters of a Gaussian, $\mu$ and $\Sigma$ given observed data $y_i$ that have uncertainties $\sigma^2_i$ associated with them. 1D example: Prior I intend to use the ...
0
votes
0answers
10 views

Finding prior conjugate for reparametrized model

Let $X_i$ be iid Bernoulli$(\pi)$ for $i=1,...,n$. My task is to find the prior conjugate for $\theta$, where $\theta$ is the natural parameter of the sampling model. The sampling model can be ...
1
vote
1answer
56 views

Poisson-Gamma conjunction - calculating posterior [duplicate]

How to calculate posterior distribution step-by-step while given: some observed numbers of customers from the last days that number of clients is distributed by Poisson($\lambda$) ($\lambda$ is not ...
0
votes
0answers
23 views

What distributions are conjugate to themselves, besides the normal?

I know the normal distribution is conjugate to itself; are there others? Is there some sort of intuition behind why a given distribution would be conjugate to itself?
4
votes
2answers
216 views

MLE vs Binomial - Beta conjugate pairs

I'm trying to understand how to use the conjugate relationship between the Binomial and Beta distributions to update the parameters of the Beta distribution. Specifically, I am imagining a series of ...
2
votes
1answer
83 views

Bayesian A/B test analysis for non-binary actions

When analyzing experiments with binary action (for example, will a user convert or not), then Beta-Binomial distribution is apt for that. For example, to model Click Through Rate (CTR): ...
2
votes
0answers
83 views

Conjuage priors for linear combination

My knwoledge of statistics in general and Bayesian statistics in particular is limited. With that in mind, I would sincerely appreciate if somebody could help me with the following problem that I have ...
3
votes
1answer
214 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
1
vote
1answer
51 views

Bayesian Linear Regression, trouble with posterior. Variance equal identity

I am trying to solve the following problem. If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function ...
1
vote
1answer
34 views

Literature on Bayesian stuff with Normal Distribution? [closed]

I am writing something on Bayesian Analysis involving the normal distribution. I know that the conjugate prior is the so-called normalized Gamma inverse distribution, I know the update rule for the ...
0
votes
1answer
39 views

Gibbs sampling: Conjugate prior for a two component known-unknown variance?

If I have a normally distributed variable $\mathcal{N}(\mu,\frac{1}{\tau})$ with fixed $\mu$ then the conjugate prior for an unknown $\tau$ is then $\mathcal{Ga}(\frac{n}{2}+\alpha, \beta + \sum_i \...
1
vote
0answers
30 views

Posterior (conjugate) prior of two parameter Gamma likelihood

This question is related to a previous question on this site. Assume some data is generated from Gamma distribution $p(x\mid\alpha,\beta) \sim \operatorname{Gamma} (\alpha,\beta)$, and both parameters ...
2
votes
0answers
85 views

Posterior distribution for Weibull scale parameter with censored data

I am modeling a survival problem using a Weibull distribution with known shape parameter $k$ and unknown scale parameter $\theta$. The PDF and CDF are given by, \begin{align} f(t|k,\theta)=\frac{k}{\...
0
votes
1answer
25 views

what is the step by step procedure of updating a posterior probability with new data coming in

I have a question about how to update posterior probability sequentially when new data comes in sequentially by say $x_1$, then $x_2$, then $x_3$,.... I understand this form when the first data $x_1$ ...
1
vote
0answers
60 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
5
votes
2answers
51 views

What is the relation between “conjugate priors” and the approximate inference?

I know that "conjugate prior" is to help us calculate the the denominator of the Bayes formula(to make the calculations easier). And I just learnt to approximate the inference by mean field ...
0
votes
0answers
47 views

If $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ conjugate to $p(\theta)$?

If exponential family $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ for $r>0$ conjugate to $p(\theta)$? If not, what can we do about it in terms of sampling to make use of ...
0
votes
0answers
37 views

Is there a conjugate prior for normal location family N(x|u,1) such that the mean is always positive?

The conjugate prior to normal location family is usually a normal distribution. However, I want to constrain the mean to be positive. Is there a conjugate prior to the normal location family $x\sim N(...
0
votes
0answers
28 views

Shape of parameters marginal posterior in hierarchical Bayes model

Consider a generic hierarchical Bayes model with data $y_i\sim p(y|\theta_i)$, dependent of parameters $\theta_i\sim p(\theta|\phi)$ and hyperparameters $\phi\sim p(\phi)$. Furthermore, assume that $\...
0
votes
0answers
9 views

Tuning the “strength” of updates to a posterior distribution for conjugates

I'm asking this question as a sanity check- I am not a statistician or research scientist, and just am doing a gut check on a model I am building. I want to quantify uncertainty of a specific metric ...
0
votes
0answers
55 views

Metropolis-within-Gibbs for parametric inference of a regressive model

I have a regressive model of this form \begin{equation} y=f(\theta)+\varepsilon \end{equation} to describe observations $y$, with noise $\varepsilon$ and a parametric function $f$ with parameters $\...
0
votes
0answers
25 views

Conjugate prior for DPGMMs using Gibbs sampling

I am using Gibbs sampling to infer DPGMMs. The prior for multivariate Gaussians is Normal-inverse Wishart. But it turns out that the covariances are not estimated accurately. Here is codes and results....
0
votes
0answers
13 views

In bayesian updating, how to keep precision matrix suceptible to data?

In bayesian updating, I want precision matrix, or standard errors to keep susceptible, if much data comes. I am considering bayesian updating of simple regresion, using conjugate prior of Normal-...
0
votes
1answer
27 views

Determine hyper-prior for gaussian distribution from existing data [closed]

Not sure how to determine hyper-prior for prior distributions, specifically using historical data. First what I am doing: I want to estimate parameters for a normal likelihood function using Bayesian ...
2
votes
1answer
41 views

Understanding of conjugation relationship in Bishop book

Referring to Pattern Recognition and Machine Learning by Bishop(Page 367, Section 8.1): Such models have particularly nice properties if we choose the relationship between each parent-child pair in ...
0
votes
0answers
38 views

Uniform conjugate prior for a Beta distribution

Given $$ \pi \sim \text{Beta}(\alpha, \beta) $$ I'd like to place a prior on $\alpha$ and $\beta$. The "trick" mentioned in this post and this post seems to be to recognize that since $$ \begin{...
0
votes
0answers
31 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
2
votes
1answer
64 views

Compute conjugate prior from the sample distribution

I feel like this question might be marked as duplicate because I see many similar incurring in that fate but I'll try anyway. I would say I did not find anything similar. I have been thought a ...
1
vote
1answer
61 views

Bayesian inference on binarized Poisson distribution

I have a variable that is Poisson distributed. Let's say I have a number of boxes each with a number of balls inside according to a Poisson distribution, with $\lambda=0.4$, (the average number of ...
2
votes
1answer
64 views

How does L2 penalize large weights

The L2 regularization term is useful because it penalizes large weights over smaller weights which is good to prevent overfitting. I'm having a hard time understanding how exactly it does this. This ...
3
votes
1answer
70 views

Joint AR(1) posterior distribution explicit under conjugate prior

I have encountered a problem in my textbook 'The Bayesian Choice' by Christian P. Robert. It goes something like this: $"$For a particular case of AR(1) model, $(x_t)_{1\leq t\leq T}$. Where $x_t = \...
1
vote
1answer
124 views

Mean of the posterior distribution in bayesian linear regression with infinitely broad prior

Currently reading from Christopher Bishop's Pattern Recognition and Machine Learning book about parameter distribution under a bayesian linear regression. On page 153, the author deduces that the ...
0
votes
0answers
35 views

Does Levy distribution has a conjugate prior?

I have searched a lot but can not find the conjugate prior of Levy distribution.
2
votes
0answers
142 views

Proving Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model

How does one prove that the Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model? This prior is a generalization of the Normal-Inverse-Wishart Distribution. By Matrix-...
5
votes
1answer
171 views

Truncated Gamma Distribution

The Gamma distribution is the conjugate prior of Poisson distribution. What about the Truncated Gamma distribution? Is it still the conjugate prior of Poisson distribution?
1
vote
1answer
451 views

Conjugate Prior for Student T distribution

Does the Student T distribution have a conjugate prior distribution? If so, what is it and what are the parameters?
0
votes
0answers
27 views

Is the posterior distribution for the model described in this question Gaussian?

I was in the middle of writing a long answer to Uncertainty estimation in high-dimensional inference problems without sampling? but I was suddenly struck by doubt: since the model assumes a Gaussian ...
2
votes
1answer
42 views

Computing posterior density of Normal with unknown $\mu$ and $\sigma^2$

(From Bayesian Essentials with R by Marin & Robert page 31) We are given an iid sample $\mathfrak{D}_n = (x_1, \dots, x_n)$ from the normal distribution $\mathcal{N}(0, \sigma^2)$ and $\theta=(\...
1
vote
1answer
621 views

How to use the Dirichlet prior for estimating the multinomial parameters? [closed]

I know that the multinomial distribution gives the likelihood of some vector D of occurrences to happen given a probability vector (parameters) P' i.e. P(D|P'). Now with a Dirichlet prior we are ...
1
vote
1answer
103 views

Maximum A Posteriori Estimate

The formula for calculating the MAP estimate of a particular parameter, $p$, is given by the following: $p^{MAP} =$ argmax $P(p)P(p|x)$. Now I am trying to do a question where I am told the prior ...
1
vote
0answers
15 views

Eigenfunctions heuristics for self-conjugate priors [duplicate]

Previously asked on math.stackexchange. I am looking for a citable reference (books, research papers, PhD theses, not websites, internal reports, etc.) about the heuristic interpretation of self-...
1
vote
1answer
679 views

Interpretation of the rate parameter of a Gamma distribution

I am toying with mixture models, especially in a bayesian context and the Gamma (or the inverse Gamma) distribution appears quite often. For example, inverse Gamma is used as a conjugate prior for the ...
0
votes
1answer
26 views

Bernoulli distribution/ SOME probability/conjugate prior

I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it? EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
0
votes
2answers
70 views

Beta distribution and normalization [duplicate]

Here in the 4 pictures in the last answer, is the vertical axe the probability? I.e. it seems to me that it is somewhat unnormalized : it has the value 2 in the 2nd picture and 3 in the 3rd picture. ...
2
votes
1answer
172 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
1
vote
1answer
47 views

Basic question on proportionality in Bayesian Inference for Normal distribution

I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that: $\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$ $Y | \theta \...

1 2 3 4 5