I have a simple linear model: $y_{i}=\mu+e_{i}$ for $i=1,...,n$, where $P(e_{i})=w\mathcal{N}(0,\sigma^2) + (1-w)\mathcal{N}(0,k^2\sigma^2)$ with $w=0.9$, $k=10$ and $\sigma=0.1$. It can be understood that I am trying to model outliers from within the noise model.

Now, I need to derive an expression for the posterior distribution of $\mu|J,y,\sigma^2$, where the indicator variable $J_{i}=1$ if $e_{i} \sim \mathcal{N}(0,\sigma^2)$ and $J_{i}=2$ if $e_{i} \sim \mathcal{N}(0,k^2 \sigma^2)$. I wrote the following:

$p(\mu|J,y,\sigma^2) \propto p(y|J,\mu,\sigma^2) p(\mu) = \prod_{i=1}^{n} p(y_i|J_i,\sigma^2) = \prod_{i:J_i=1} \frac{\text{exp}(-\frac{1}{2 \sigma^2}(y_{i}-\mu)^{2})}{\sqrt{2 \pi \sigma^2}} \times \prod_{i:J_i=2} \frac{\text{exp}(-\frac{1}{2 k^2 \sigma^2}(y_{i}-\mu)^{2})}{\sqrt{2 \pi k^2 \sigma^2}} = \prod_{i=1}^{n} \frac{\text{exp}-\frac{1}{2 K(i) \sigma^2} (y_i-\mu)^2}{\sqrt{2 \pi K(i) \sigma^2}}$

where $K(i) = 1$ when $J_i=1$ and $K(i) = k^2$ when $J_i=2$.

It can be understood that, (a) I have chosen the prior over $\mu$ - $P(\mu)$ - to be a constant, and (b) the indicator variable $J_{i}$ takes two values - 1 and 2 - which will in turn determine the posterior distribution.

I understand that the posterior over $\mu$ will be a normal with some mean and variance, i.e., $\mu \sim \mathcal{N}(\mu_{p},\Sigma_{p})$. Can anyone help me derive expressions for the posterior mean and variance?

  • $\begingroup$ Your likelihood is likely wrong, since $w$ does not make an appearance. $\endgroup$ – Greenparker Mar 27 '16 at 12:45
  • $\begingroup$ OK I figured it out. The expression is quadratic in $\mu$ so I had to complete the square and find the coefficient of $\mu$. $\endgroup$ – PaulC Mar 29 '16 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.