MMSE estimator with dirac delta prior pdf

Question

The question is as follows, it's mainly part 3 that I was having problem with.

A discrete-valued parameter with the prior pdf $$p(x) = > \sum_{i=1}^2p_i\delta(x-i)$$ is measured with the additive noise $w ~ N(0, \sigma^2)$. The measurement z is given by: $$z = x + w$$

1. Find the posterior pdf of the parameter.
2. Find the MAP estimate and the associated MSE conditioned on z.
3. Find its MMSE estimate and the associated variance.

For part 1, I used Bayes formula and the total probability theorem to get $$p(x|z) = \frac{f(z)}{2\pi\sigma^2}*e^{\frac{(z-x)^2}{2\sigma^2}}*\sum_{i=1}^2p_i\delta(x-i)$$ where $f(z) = e^{-\frac{(z-1)^2}{2\sigma^2}} + e^{-\frac{(z-2)^2}{2\sigma^2}}$

For part 2, I solved $$\frac{d}{dx}[ln(p(x|z)]= 0$$ by considering the dirac delta function as a constant and setting z-x =0, giving $\hat x^{MAP}= z$ and $MSE(\hat x^{MAP})= 0$

However, when it comes to part 3, I know that $\hat x^{MMSE} = E(x|z)$. I also noticed $p(x|z)$ can be divided into three parts, a function of z, a normal distribution of with mean x and variance $\sigma$, and p(x) which involves the dirac delta function. But I'm not sure how to calculate the expected value.

I thought about expanding the summation, and since $\delta(x-i) = 0$ everywhere except for when x = 1 & x = 2, I can just take x = 1 & x = 2 and have $p(x|z)$ as a function of z, which is esentially a constant. But it also seems a little sketchy.

I would really appreciate it if someone could give me a hint and point me in the right direction to solve this problem. Thank you in advance.

Answer 1

Ok, let's start with fixing up part 2, then go back to part 1.

Suppose the prior probabilities are equal. The prior is supported only on the set of points $\{1,2\}$, so the posterior will also be supported only on those points. The MAP estimator must also take values in $\{1,2\}$. The prior probability is equal on those two points. The loglikelihood for an observation $z$ given $x$ is proportional to $(x-z)^2$, so the posterior probability will be higher on whichever of 1 and 2 is closer to $z$. That gives you the MAP estimator.

To get question 1 (and to handle the non-equal case of question 2), you'd need to quantify how much more likely the closer point is. Bayes' Theorem says $p(x|z)\propto p(x)f(z|x)$, so the probabilities on $\{1,2\}$ are proportional to the $N(0,\sigma^2)$ densities at $z-1$ and $z-2$ respectively.

Now you have the posterior probabilities over the two points you should be able to calculate the posterior mean