Computing a posterior distribution

Question

I need to compute the posterior distribution of a parameter $\theta$ conditionally on signal $t$. $\theta$ is uniformly distributed in $[0,1]$, while $t=\theta+\eta$ where $\eta$ represents a noise, distributed uniformly in $[-0.5,0.5]$. I know Bayes' rule but I don't know how to apply it when distributions, instead of single probabilities, are involved. Can someone show me the actual computations for this?

Answer 1

I'll try to answer it because I gave it some thought but I'm not 100% sure so I would like some to correct me if possible.

If I understand well, you have the following process, you have some underlying true signal $\theta$ and you observed a corrupted signal $t$, the corruption is $\eta$. So, if we repeadeadly receive a signal the noise each time going to be different so, we will end with some data that look like $t_{1}, t_{2},..., t_{n}$ with noises $\eta_{1},\eta_{2},...,\eta_{n}$. In your case you have only one observation $t$.

In order to find the posterior of $\theta$ you use the Bayes theorem

$p(\theta|t,\eta) = \frac{p(t|\theta,\eta)p(\theta,\eta)}{p(t,\eta)}$

I assume that $\eta\perp t,\theta $

so we can rewrite the Bayes theorem as

$p(\theta|t,\eta) = \frac{p(t|\theta,\eta)p(\theta)p(\eta)}{p(t)p(\eta)}\propto p(t|\theta,\eta)p(\theta)$

since we are only interested in terms of having $\theta$.

Know I've seen that the marginal of $t$, $p(t)$ can be calculated with the following convolution since it is the sum of independet random variables $t=\theta +\eta$.

$p_{t}(t) = \int p_{\theta}(\theta)p_{\eta}(t-\theta)d\theta$

Next we want to obtain the conditional distribution $p_{t}(t|\theta,\eta)$ since we have the marginilization of through the convolution, we just have to take a step back and get rid of the integral? (I'm not 100% about this). So, I guess the conditional will be

$p_{t}(t|\theta,\eta) = p_{\theta}(\theta)p_{\eta}(t-\theta) = \mathbb{I}_{\theta\in(0,1)}\mathbb{I}_{(t-\theta)\in(-0.5,0.5)} = \mathbb{I}_{\theta \in (t-0.5,t+0.5)}$

Now getting back to the posterior

$p_{\theta}(\theta|t,\eta)\propto p_{t}(\theta,\eta)p_{\theta}(\theta) = \mathbb{I}_{\theta \in (t-0.5,t+0.5)}\mathbb{I}_{\theta\in(0,1)}=\mathbb{I}_{\theta \in (t-0.5,t+0.5)}$

So, the posterior distribution of $\theta$ is a uniform over the interval $(t-0.5,t+0.5)$.

Now, because I wasn't sure about my approach I did a numerical example in R and it seems that it is correct.

eta = runif(10000,-0.5,0.5)
theta = runif(1,0,1)
t = theta+eta
plot(t)
abline(h=theta,col='red')
mean(t)
mean(t-0.5) #this should be approx equal to min(t)
mean(t+0.5) #this should be approx equal to max(t)
min(t)
max(t)

```

Answer 2

The prior probability distribution of $\theta$ is $$ f_\theta(u) \, du = \begin{cases} 1\, du & \text{if } 0<u<1, \\ 0 \, du & \text{otherwise.} \end{cases} $$ The likelihood function is $$ L_{\theta\,\mid\,t}(u) = \begin{cases} 2 & \text{if } t-\tfrac12<u<t+\tfrac12, \\ 0 & \text{otherwise.} \end{cases} $$ (Note that I wrote $\text{“}du\text{”}$ for the prior and not for the likelihood. That is worth understanding.)

Therefore the posterior probability distribution is \begin{align} & \text{constant} \times L_{\theta\,\mid\,t}(u) f_\theta(u)\, du \\[6pt]= {} & \begin{cases} \text{positive constant}\cdot du & \text{if } \max\{0,t-\tfrac12\} <u < \min\{ 1, t + \tfrac 12 \}, \\ 0\,du & \text{otherwise.} \end{cases} \end{align} That positive constant must be the reciprocal of the length of the interval, so that the integral will be $1.$

Thus the posterior distribution is uniform on the interval $$ \left( \max\{0,t-\tfrac12\}, \, \min\{1,t+\tfrac12\} \right). $$