# Computing a posterior distribution

I need to compute the posterior distribution of a parameter $$\theta$$ conditionally on signal $$t$$. $$\theta$$ is uniformly distributed in $$[0,1]$$, while $$t=\theta+\eta$$ where $$\eta$$ represents a noise, distributed uniformly in $$[-0.5,0.5]$$. I know Bayes' rule but I don't know how to apply it when distributions, instead of single probabilities, are involved. Can someone show me the actual computations for this?

• $p(\theta| t,\eta)=\frac{p(t|\theta,\eta)p(\theta)p(\eta)}{p(t)p(\eta)}$ fill in the according distributions. I assumed that $\eta$ with $\theta$ and $t$ are independent. Commented May 29, 2023 at 14:38
• Yes, they are independent. So, just to check: $p(\eta)$ simplifies since it is both at num and den, how can I deduce $p(t)$ and $p(t|\theta, \eta)$ since $t$ is the sum of two uniformly distributed random variables? Commented May 29, 2023 at 14:45
• This link will help you math.stackexchange.com/questions/895075/…, on what distribution the sum of two uniform distributions is Commented May 29, 2023 at 14:52
• Thanks, by following this I obtain $p(t)$ I guess, and for the other one? Commented May 29, 2023 at 14:57
• This will help you obtain $p(t)$ and $p(t|\theta,\eta)$ Commented May 29, 2023 at 15:05

I'll try to answer it because I gave it some thought but I'm not 100% sure so I would like some to correct me if possible.

If I understand well, you have the following process, you have some underlying true signal $$\theta$$ and you observed a corrupted signal $$t$$, the corruption is $$\eta$$. So, if we repeadeadly receive a signal the noise each time going to be different so, we will end with some data that look like $$t_{1}, t_{2},..., t_{n}$$ with noises $$\eta_{1},\eta_{2},...,\eta_{n}$$. In your case you have only one observation $$t$$.

In order to find the posterior of $$\theta$$ you use the Bayes theorem

$$p(\theta|t,\eta) = \frac{p(t|\theta,\eta)p(\theta,\eta)}{p(t,\eta)}$$

I assume that $$\eta\perp t,\theta$$

so we can rewrite the Bayes theorem as

$$p(\theta|t,\eta) = \frac{p(t|\theta,\eta)p(\theta)p(\eta)}{p(t)p(\eta)}\propto p(t|\theta,\eta)p(\theta)$$

since we are only interested in terms of having $$\theta$$.

Know I've seen that the marginal of $$t$$, $$p(t)$$ can be calculated with the following convolution since it is the sum of independet random variables $$t=\theta +\eta$$.

$$p_{t}(t) = \int p_{\theta}(\theta)p_{\eta}(t-\theta)d\theta$$

Next we want to obtain the conditional distribution $$p_{t}(t|\theta,\eta)$$ since we have the marginilization of through the convolution, we just have to take a step back and get rid of the integral? (I'm not 100% about this). So, I guess the conditional will be

$$p_{t}(t|\theta,\eta) = p_{\theta}(\theta)p_{\eta}(t-\theta) = \mathbb{I}_{\theta\in(0,1)}\mathbb{I}_{(t-\theta)\in(-0.5,0.5)} = \mathbb{I}_{\theta \in (t-0.5,t+0.5)}$$

Now getting back to the posterior

$$p_{\theta}(\theta|t,\eta)\propto p_{t}(\theta,\eta)p_{\theta}(\theta) = \mathbb{I}_{\theta \in (t-0.5,t+0.5)}\mathbb{I}_{\theta\in(0,1)}=\mathbb{I}_{\theta \in (t-0.5,t+0.5)}$$

So, the posterior distribution of $$\theta$$ is a uniform over the interval $$(t-0.5,t+0.5)$$.

Now, because I wasn't sure about my approach I did a numerical example in R and it seems that it is correct.

eta = runif(10000,-0.5,0.5)
theta = runif(1,0,1)
t = theta+eta
plot(t)
abline(h=theta,col='red')
mean(t)
mean(t-0.5) #this should be approx equal to min(t)
mean(t+0.5) #this should be approx equal to max(t)
min(t)
max(t)

$$$$
`

The prior probability distribution of $$\theta$$ is $$f_\theta(u) \, du = \begin{cases} 1\, du & \text{if } 0 The likelihood function is $$L_{\theta\,\mid\,t}(u) = \begin{cases} 2 & \text{if } t-\tfrac12 (Note that I wrote $$\text{“}du\text{”}$$ for the prior and not for the likelihood. That is worth understanding.)

Therefore the posterior probability distribution is \begin{align} & \text{constant} \times L_{\theta\,\mid\,t}(u) f_\theta(u)\, du \\[6pt]= {} & \begin{cases} \text{positive constant}\cdot du & \text{if } \max\{0,t-\tfrac12\} That positive constant must be the reciprocal of the length of the interval, so that the integral will be $$1.$$

Thus the posterior distribution is uniform on the interval $$\left( \max\{0,t-\tfrac12\}, \, \min\{1,t+\tfrac12\} \right).$$

• Can you elaborate a little bit more how you found the likelihood because I get confused. Commented May 29, 2023 at 17:32
• @Fiodor1234 : The likelihood function is $$L_{\theta\,\mid\,t}(u) = f_{\text{data}\,\mid\, \theta\,=\,u} (t)$$ where the latter is the conditional probability density function of the observed data given that the value of $\theta$ is the value $u$, evaluated at the quantity $t$ that was actually observed. $\qquad$ Commented May 29, 2023 at 19:14
• @Fiodor1234 : I suspect it would be less confusing if you were working with density functions that are not constant. Commented May 29, 2023 at 19:14