I have got a set $\{Y_t\}$ of observations consisting of two subsets $\{Y_{t,1}\}$ and $\{Y_{t,2}\} \subset \{Y_t\}$ with $\{Y_{t,1}\} \sim \mathcal{N}(\mu_1,\sigma^2)$ and $\{Y_{t,2}\} \sim \mathcal{N}(\mu_2,\sigma^2)$ i.e. different means but the same variance (resulting from a regime switching model).

I know the means and want to draw a sample of $\sigma^2$ in a step of a MCMC estimation.

In the case of $\mu_1 = \mu_2$ I would have used the conjugate prior of the Inverse Gamma distribution (see [1], "Normal with known mean").

Can I use a conjugate prior in the case of $\mu_1 \neq \mu_2$ as well? For example by setting $\beta = \beta_0 + \frac{1}{2}\sum_{i=0}^n(Y_i - \mu_{I_i})^2 $ ($I_i$ being the correct indices according to the observations)?

Or will I have to use Metropolis-Hastings to get my sample of $\sigma^2$?

Best, Matt

[1] http://en.wikipedia.org/wiki/Conjugate_prior#Continuous_distributions

  • $\begingroup$ This is a Gaussian linear model and there is a conjugate family of priors. The Jeffreys prior belongs to this family and this is the one we commonly use (and this is not the proposal of @DavidR) $\endgroup$ – Stéphane Laurent Feb 10 '14 at 13:41
  • $\begingroup$ Sorry for ignoring your answer, Stéphane, I unfortunately didn't see it until now. How would you apply the Jeffreys prior to my problem and in which ways would it be preferrable to the Inverse Gamma prior? $\endgroup$ – Matt Mar 3 '14 at 11:53

Centering your set of ${Y_t}$ (i.e. ${Y_t^\ast} = {Y_t} - \mu_{I_i}$) has no influence on the variance. Therefore using centred data and a Inverse Gamma prior $IG(\alpha_0, \beta_0)$ results in an Inverse Gamma posterior with parameters $\alpha_0 + n/2$ and $\beta_0 + \frac{1}{2} \sum_{i=0}^n (Y_i - \mu_{I_i})^2$, which is equivalent to your idea (apart from the $^2$ you probably just forgot).

Aside from this, there is no need to use MCMC estimation when having a closed-form expression for the posterior. You can just draw from the above mentioned distribution - for example in R by using

alpha <- alpha0 + length(y_centred)/2
beta <- beta0 + sum((y_centred)^2)
sampleOfVar <- 1/rgamma(100,alpha,beta)
| cite | improve this answer | |
  • $\begingroup$ Thank you, David. I am using MCMC as there is a second set of observations that is linked to the $Y_t$ by a function of $\sigma^2$. Therefore the sample from the conjugate-Inverse Gamma prior is used as an input for a Metropolis-Hastings step connecting this second set of observations. $\endgroup$ – Matt Feb 10 '14 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.