# MMSE estimator with dirac delta prior pdf

The question is as follows, it's mainly part 3 that I was having problem with.

A discrete-valued parameter with the prior pdf $$p(x) = > \sum_{i=1}^2p_i\delta(x-i)$$ is measured with the additive noise $$w ~ N(0, \sigma^2)$$. The measurement z is given by: $$z = x + w$$

1. Find the posterior pdf of the parameter.
2. Find the MAP estimate and the associated MSE conditioned on z.
3. Find its MMSE estimate and the associated variance.

For part 1, I used Bayes formula and the total probability theorem to get $$p(x|z) = \frac{f(z)}{2\pi\sigma^2}*e^{\frac{(z-x)^2}{2\sigma^2}}*\sum_{i=1}^2p_i\delta(x-i)$$ where $$f(z) = e^{-\frac{(z-1)^2}{2\sigma^2}} + e^{-\frac{(z-2)^2}{2\sigma^2}}$$

For part 2, I solved $$\frac{d}{dx}[ln(p(x|z)]= 0$$ by considering the dirac delta function as a constant and setting z-x =0, giving $$\hat x^{MAP}= z$$ and $$MSE(\hat x^{MAP})= 0$$

However, when it comes to part 3, I know that $$\hat x^{MMSE} = E(x|z)$$. I also noticed $$p(x|z)$$ can be divided into three parts, a function of z, a normal distribution of with mean x and variance $$\sigma$$, and p(x) which involves the dirac delta function. But I'm not sure how to calculate the expected value.

I thought about expanding the summation, and since $$\delta(x-i) = 0$$ everywhere except for when x = 1 & x = 2, I can just take x = 1 & x = 2 and have $$p(x|z)$$ as a function of z, which is esentially a constant. But it also seems a little sketchy.

I would really appreciate it if someone could give me a hint and point me in the right direction to solve this problem. Thank you in advance.

• There are many issues here such as a sign error in what should be $e^{-\frac{(z-x)^2}{\sigma^2}}$ and the implausibility of the MAP being $z$ when that is almost never $1$ or $2$ Commented Sep 26, 2021 at 1:41
• Sorry fixed the typo. Commented Sep 26, 2021 at 2:25
• The posterior is incorrect for at least two reasons, you cannot take derivatives when $x\in\{1,2\}$, the posterior mean is the sum of two terms and is not the MMSE when $x\in\{1,2\}$. Commented Sep 26, 2021 at 6:08

Suppose the prior probabilities are equal. The prior is supported only on the set of points $$\{1,2\}$$, so the posterior will also be supported only on those points. The MAP estimator must also take values in $$\{1,2\}$$. The prior probability is equal on those two points. The loglikelihood for an observation $$z$$ given $$x$$ is proportional to $$(x-z)^2$$, so the posterior probability will be higher on whichever of 1 and 2 is closer to $$z$$. That gives you the MAP estimator.
To get question 1 (and to handle the non-equal case of question 2), you'd need to quantify how much more likely the closer point is. Bayes' Theorem says $$p(x|z)\propto p(x)f(z|x)$$, so the probabilities on $$\{1,2\}$$ are proportional to the $$N(0,\sigma^2)$$ densities at $$z-1$$ and $$z-2$$ respectively.