0
$\begingroup$

I am taking this example from here. They have given the steps but i cannot understand them so a little dumbing down of answer is necessary or you can just explain the answer given there. its on page 5

consider a device that will operate without failure for a time θ because of a protective chemical inhibitor injected into it; but at time θ the supply of the chemical is exhausted, and failures then commence, following the exponential failure law. It is not feasible to observe the depletion of this inhibitor directly; one can observe only the resulting failures. From data on actual failure times, estimate the time θ of guaranteed safe operation. Find 95% uncertainty bounds

Failure times observed : $(10, 12, 15)$. use uninformative prior

$\endgroup$
  • $\begingroup$ What section of the paper in the link is relevant to this problem? What do you know about choices of uninformative priors? If failure times are 10, 12, 15, what is your estimate of $\theta.$ If you can answer those questions, it would be easier to give useful help. $\endgroup$ – BruceET Nov 5 '18 at 17:34
  • $\begingroup$ @BruceET - "What section of the paper in the link is relevant to this problem?" - The page would be numbered 95 and under the heading "Truncated exponential: A bayesian approach". "What do you know about choices of uninformative priors?" - That they make all choices of $\theta$ equally likely hence, $P(\theta) \propto 1$. "If failure times are 10, 12, 15, what is your estimate of θ." -$ (10+12+15)/3 - 1$. But this is from frequentist perspective. Looking forward to your help $\endgroup$ – MiloMinderbinder Nov 6 '18 at 8:25
  • $\begingroup$ All I would add to @JeremyC's Answer (+1) is that the frequentist point estimate of $\theta$ is the minimum observation, which contains all of the info in the data relevant for estimating $\theta.$ // Explanations and derivations in published articles often leave out steps that might seem obvious to someone familiar with the general topic. The point of view is to use journal space for new results and insights. Sometimes it helps to read relevant references supplied by the author. $\endgroup$ – BruceET Nov 6 '18 at 19:24
  • $\begingroup$ I did not intend any criticism of the author, whose account of this case is the clearest I have read, and much clearer than Jaynes's own. $\endgroup$ – JeremyC Nov 6 '18 at 22:02
2
$\begingroup$

I am not surprised that you are puzzled. The author skips several non-trivial steps. Although for many purposes it is convenient to omit the constants of proportionality when working with Bayes's theorem, that can make arguments difficult to follow.

As an equation, therefore, Bayes's theorem states that $$ p(\theta|D) = \frac{p(D|\theta)p(\theta)}{p(D)}. $$ So we want to find the lhs, the distribution of $\theta$ given the data, so that we can work out the credible range. To do this we evaluate each term on the rhs.

Just consider the case in which we have a single observation, $x_0$, say. Then $p(\theta|D)$, the likelihood, is given by $$ p(D|\theta) \equiv p(x_0 | \theta) = \exp(\theta - x_0) = e^{\theta}e^{-x_0}\quad x_0 >\theta$$ and zero otherwise. By assumption $p(\theta)=1$.

So all we need to complete the calculation is $p(D)$. That is given by $$ \int_0^{\infty}p(D|\theta)d\theta = \int_0^{x_0}\exp(\theta - x_0)d\theta\\ =e^{-x_0}(e^{x_0} - 1). $$ Note that the lower bound of the integration is zero rather than $-\infty$. That is because $\theta$ must be non-negative.

Putting the three components of the rhs together we get $$ p(\theta|D) = e^{\theta}e^{-x_0}.e^{x_0}/(e^{x_0} - 1)\\ = \exp(\theta - x_0)/(1- e^{-x_0}) $$ In my equation, author's proportionality $p(\theta|D) \propto N\exp[N(\theta - \min(D))]$ becomes $$\exp[\theta - \min(D)]/(1 - e^{-\min(D)})$$.

Now, if we move to the actual case, in which there are $N$ observations, the calculation is very similar but with one adjustment, not mentioned by the author. He states that the likelihood is proportrional to the product of the probabilities of each observation,$\Pi_{i = 1}^{N}\exp(\theta - x_i)$. So it is, but if you want to work with Bayes's theorem as an equation not a proportionality you need the joint likelihood of the N observations taking into account that one of them, the minimum, needs to be identified explicitly. That requires a factor of $N^{-1}$ to be applied.

If you work through it, you will find that the author's proportionality becomes $$N\exp[N(\theta - \min(D))]/(1 - e^{-\min(D)})$$.

The counter-intuitive result of all this is that the posterior distribution of $\theta$ does not depend on the actual values of the observations that are higher than the minimum observation.

$\endgroup$
  • $\begingroup$ "but if you want to work with Bayes's theorem as an equation not a proportionality you need the joint likelihood of the N observations taking into account that one of them, the minimum, needs to be identified explicitly. That requires a factor of $N^{−1}$ to be applied." could you explain this. The data are assumed to be iids so why is joint probability not equal to a simple product of all the exponentially decaying probabilities with decay starting at $min(D)$. What is up with the factor or $N^{-1}$ $\endgroup$ – MiloMinderbinder Nov 8 '18 at 10:17
  • $\begingroup$ Let $\{X_i; i = 1,\dots,N\}$ be a sequence of iid RVs with common density $f(x)$, and suppose WLOG that $min(X_i) = X_0$, then the joint density of the$\ {X_i; i = 1,\dots,N\}$ is not $f^N(x)$ but N^{-1}f^N(x)$. It is a special case of the theorem on the density of order statistics. $\endgroup$ – JeremyC Nov 8 '18 at 11:20
  • $\begingroup$ ohhh yes.. I confused probability density with probability. Thank you so much for all the help. $\endgroup$ – MiloMinderbinder Nov 8 '18 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.