From a course problem:

Consider the regression model, $$y_i=\alpha_i+\beta x_i+\epsilon, \hspace{1em} i=1,\dots,n,$$ with $\epsilon_i\sim\text{N}(0,1/\lambda)$ i.i.d. and prior structure \begin{align}\alpha_i&\sim\text{N}(\mu,1/p) \; \text{independent for }i=1,\dots,n,\\\mu&\sim\text{N}(a,1/r),\\\beta&\sim\text{N}(b,1/q),\\\lambda&\sim\text{Ga}(c,d).\end{align} Show that the full conditional of each of the intercepts $\alpha_i$ is Gaussian and provide explicit expression for the parameters.

I'd like to begin by writing $$\pi(\alpha_i|\text{some variables})\propto\pi(\alpha_i)f(\text{some variables}|\alpha_i)$$ but I'm not sure what those variables should be. From given solutions I can deduce that $$f(\text{some variables}|\alpha_i)\propto\exp\left(-\frac{\lambda}{2}(y_i-\alpha_i-\beta x_i)^2\right);$$ does that mean $f$ should be $$f(y_i,\beta,x_i,\lambda|\alpha_i)?$$


1 Answer 1


In this problem the variables $(a,b,c,d,p,q,r)$ are your hyperparameters and you will observe the data $(\mathbf{x},\mathbf{y})$ from your model. Your posterior distribution is conditional on the conjunction of the hyperparameters and data. In the present case, you can write the required posterior kernel using Bayes' rule as:

$$\begin{align} \pi(\boldsymbol{\alpha}|\mathbf{x},\mathbf{y},a,b,c,d,p,q,r) &\overset{\boldsymbol{\alpha}}{\propto} \pi(\boldsymbol{\alpha},\beta,\lambda|\mathbf{x},\mathbf{y},a,b,c,d,p,q,r) \\[6pt] &\propto L_{\mathbf{x},\mathbf{y}}(\boldsymbol{\alpha},\beta,\lambda) \cdot \pi(\boldsymbol{\alpha}|a,p,r) \cdot \pi(\beta|b,q) \cdot \pi(\lambda|c,d). \\[6pt] \end{align}$$

The likelihood function is given by the proportionality requirement:

$$\begin{align} L_{\mathbf{x},\mathbf{y}}(\boldsymbol{\alpha},\beta,\lambda) &\propto \prod_{i=1}^n \text{N} \bigg( y_i \bigg| \alpha_i + \beta x_i, \frac{1}{\lambda} \bigg) \\[6pt] &= \prod_{i=1}^n \sqrt{\frac{\lambda}{2 \pi}} \exp \bigg( \frac{\lambda}{2} (y_i - \alpha_i - \beta x_i)^2 \bigg) \\[6pt] &\propto \lambda^{n/2} \exp \bigg( \frac{\lambda}{2} \sum_{i=1}^n (y_i - \alpha_i - \beta x_i)^2 \bigg). \\[6pt] \end{align}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.