Find bayesian credible Interval

Question

I am taking this example from here. They have given the steps but i cannot understand them so a little dumbing down of answer is necessary or you can just explain the answer given there. its on page 5

consider a device that will operate without failure for a time θ because of a protective chemical inhibitor injected into it; but at time θ the supply of the chemical is exhausted, and failures then commence, following the exponential failure law. It is not feasible to observe the depletion of this inhibitor directly; one can observe only the resulting failures. From data on actual failure times, estimate the time θ of guaranteed safe operation. Find 95% uncertainty bounds

Failure times observed : $(10, 12, 15)$. use uninformative prior

Answer 1

I am not surprised that you are puzzled. The author skips several non-trivial steps. Although for many purposes it is convenient to omit the constants of proportionality when working with Bayes's theorem, that can make arguments difficult to follow.

As an equation, therefore, Bayes's theorem states that $$ p(\theta|D) = \frac{p(D|\theta)p(\theta)}{p(D)}. $$ So we want to find the lhs, the distribution of $\theta$ given the data, so that we can work out the credible range. To do this we evaluate each term on the rhs.

Just consider the case in which we have a single observation, $x_0$, say. Then $p(\theta|D)$, the likelihood, is given by $$ p(D|\theta) \equiv p(x_0 | \theta) = \exp(\theta - x_0) = e^{\theta}e^{-x_0}\quad x_0 >\theta$$ and zero otherwise. By assumption $p(\theta)=1$.

So all we need to complete the calculation is $p(D)$. That is given by $$ \int_0^{\infty}p(D|\theta)d\theta = \int_0^{x_0}\exp(\theta - x_0)d\theta\\ =e^{-x_0}(e^{x_0} - 1). $$ Note that the lower bound of the integration is zero rather than $-\infty$. That is because $\theta$ must be non-negative.

Putting the three components of the rhs together we get $$ p(\theta|D) = e^{\theta}e^{-x_0}.e^{x_0}/(e^{x_0} - 1)\\ = \exp(\theta - x_0)/(1- e^{-x_0}) $$ In my equation, author's proportionality $p(\theta|D) \propto N\exp[N(\theta - \min(D))]$ becomes $$\exp[\theta - \min(D)]/(1 - e^{-\min(D)})$$.

Now, if we move to the actual case, in which there are $N$ observations, the calculation is very similar but with one adjustment, not mentioned by the author. He states that the likelihood is proportrional to the product of the probabilities of each observation,$\Pi_{i = 1}^{N}\exp(\theta - x_i)$. So it is, but if you want to work with Bayes's theorem as an equation not a proportionality you need the joint likelihood of the N observations taking into account that one of them, the minimum, needs to be identified explicitly. That requires a factor of $N^{-1}$ to be applied.

If you work through it, you will find that the author's proportionality becomes $$N\exp[N(\theta - \min(D))]/(1 - e^{-\min(D)})$$.

The counter-intuitive result of all this is that the posterior distribution of $\theta$ does not depend on the actual values of the observations that are higher than the minimum observation.