I need to compute the posterior distribution of a parameter $\theta$ conditionally on signal $t$. $\theta$ is uniformly distributed in $[0,1]$, while $t=\theta+\eta$ where $\eta$ represents a noise, distributed uniformly in $[-0.5,0.5]$. I know Bayes' rule but I don't know how to apply it when distributions, instead of single probabilities, are involved. Can someone show me the actual computations for this?

  • 1
    $\begingroup$ $p(\theta| t,\eta)=\frac{p(t|\theta,\eta)p(\theta)p(\eta)}{p(t)p(\eta)}$ fill in the according distributions. I assumed that $\eta$ with $\theta$ and $t$ are independent. $\endgroup$
    – Fiodor1234
    May 29 at 14:38
  • $\begingroup$ Yes, they are independent. So, just to check: $p(\eta)$ simplifies since it is both at num and den, how can I deduce $p(t)$ and $p(t|\theta, \eta)$ since $t$ is the sum of two uniformly distributed random variables? $\endgroup$
    – user389081
    May 29 at 14:45
  • $\begingroup$ This link will help you math.stackexchange.com/questions/895075/…, on what distribution the sum of two uniform distributions is $\endgroup$
    – Fiodor1234
    May 29 at 14:52
  • $\begingroup$ Thanks, by following this I obtain $p(t)$ I guess, and for the other one? $\endgroup$
    – user389081
    May 29 at 14:57
  • $\begingroup$ This will help you obtain $p(t)$ and $p(t|\theta,\eta)$ $\endgroup$
    – Fiodor1234
    May 29 at 15:05

2 Answers 2


I'll try to answer it because I gave it some thought but I'm not 100% sure so I would like some to correct me if possible.

If I understand well, you have the following process, you have some underlying true signal $\theta$ and you observed a corrupted signal $t$, the corruption is $\eta$. So, if we repeadeadly receive a signal the noise each time going to be different so, we will end with some data that look like $t_{1}, t_{2},..., t_{n}$ with noises $\eta_{1},\eta_{2},...,\eta_{n}$. In your case you have only one observation $t$.

In order to find the posterior of $\theta$ you use the Bayes theorem

$p(\theta|t,\eta) = \frac{p(t|\theta,\eta)p(\theta,\eta)}{p(t,\eta)}$

I assume that $\eta\perp t,\theta $

so we can rewrite the Bayes theorem as

$p(\theta|t,\eta) = \frac{p(t|\theta,\eta)p(\theta)p(\eta)}{p(t)p(\eta)}\propto p(t|\theta,\eta)p(\theta)$

since we are only interested in terms of having $\theta$.

Know I've seen that the marginal of $t$, $p(t)$ can be calculated with the following convolution since it is the sum of independet random variables $t=\theta +\eta$.

$p_{t}(t) = \int p_{\theta}(\theta)p_{\eta}(t-\theta)d\theta$

Next we want to obtain the conditional distribution $p_{t}(t|\theta,\eta)$ since we have the marginilization of through the convolution, we just have to take a step back and get rid of the integral? (I'm not 100% about this). So, I guess the conditional will be

$p_{t}(t|\theta,\eta) = p_{\theta}(\theta)p_{\eta}(t-\theta) = \mathbb{I}_{\theta\in(0,1)}\mathbb{I}_{(t-\theta)\in(-0.5,0.5)} = \mathbb{I}_{\theta \in (t-0.5,t+0.5)}$

Now getting back to the posterior

$p_{\theta}(\theta|t,\eta)\propto p_{t}(\theta,\eta)p_{\theta}(\theta) = \mathbb{I}_{\theta \in (t-0.5,t+0.5)}\mathbb{I}_{\theta\in(0,1)}=\mathbb{I}_{\theta \in (t-0.5,t+0.5)}$

So, the posterior distribution of $\theta$ is a uniform over the interval $(t-0.5,t+0.5)$.

Now, because I wasn't sure about my approach I did a numerical example in R and it seems that it is correct.

eta = runif(10000,-0.5,0.5)
theta = runif(1,0,1)
t = theta+eta
mean(t-0.5) #this should be approx equal to min(t)
mean(t+0.5) #this should be approx equal to max(t)


The prior probability distribution of $\theta$ is $$ f_\theta(u) \, du = \begin{cases} 1\, du & \text{if } 0<u<1, \\ 0 \, du & \text{otherwise.} \end{cases} $$ The likelihood function is $$ L_{\theta\,\mid\,t}(u) = \begin{cases} 2 & \text{if } t-\tfrac12<u<t+\tfrac12, \\ 0 & \text{otherwise.} \end{cases} $$ (Note that I wrote $\text{“}du\text{”}$ for the prior and not for the likelihood. That is worth understanding.)

Therefore the posterior probability distribution is \begin{align} & \text{constant} \times L_{\theta\,\mid\,t}(u) f_\theta(u)\, du \\[6pt]= {} & \begin{cases} \text{positive constant}\cdot du & \text{if } \max\{0,t-\tfrac12\} <u < \min\{ 1, t + \tfrac 12 \}, \\ 0\,du & \text{otherwise.} \end{cases} \end{align} That positive constant must be the reciprocal of the length of the interval, so that the integral will be $1.$

Thus the posterior distribution is uniform on the interval $$ \left( \max\{0,t-\tfrac12\}, \, \min\{1,t+\tfrac12\} \right). $$

  • $\begingroup$ Can you elaborate a little bit more how you found the likelihood because I get confused. $\endgroup$
    – Fiodor1234
    May 29 at 17:32
  • $\begingroup$ @Fiodor1234 : The likelihood function is $$L_{\theta\,\mid\,t}(u) = f_{\text{data}\,\mid\, \theta\,=\,u} (t)$$ where the latter is the conditional probability density function of the observed data given that the value of $\theta$ is the value $u$, evaluated at the quantity $t$ that was actually observed. $\qquad$ $\endgroup$ May 29 at 19:14
  • $\begingroup$ @Fiodor1234 : I suspect it would be less confusing if you were working with density functions that are not constant. $\endgroup$ May 29 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.