A nonnegative random variable $x$ has a continuous uniform distribution in the interval $(0,\theta)$. Therefore, the likelihood is given by: $f(x|\theta) = \frac{1}{\theta}I(x\leq\theta)$, where $I$ is an indicator function.

But what is the Jeffreys prior for the parameter $\theta$? In particular, how to handle the indicator function when calculating the Fisher information?

  • $\begingroup$ It might be of use to review these measure theory concepts. $\endgroup$ – AdamO Dec 11 '13 at 1:23
  • 1
    $\begingroup$ Note that this model has a conjugate prior: the Pareto distribution. $\endgroup$ – Zen Dec 11 '13 at 2:46

I think the other answer is wrong, so will give a detailed development here. First, let $X_1, \dotsc, X_n$ be iid uniform on the interval $(0,\theta)$. Then the likelihood function can be written as $$ L(\theta)= \theta^{-n} \cdot \mathbb{1}(\theta \ge T) $$ where $T=\max(X_1, \dotsc, X_n)$ is the sufficient statistic for $\theta$. The log likelihood then can be written $$ l(\theta)=\log L(\theta)= -n \log \theta+ \begin{cases} 0 &,\theta\ge T \\ -\infty &, \theta<T \end{cases} $$ and its first derivative (where it exists) can be written $$ -n/\theta $$ with expectation equal to $ -n/\theta \not= 0$, so **we cannot calculate the Fisher information via the expectation of the second derivative, since that equality depends on equality to zero above. If we nevertheless do that, we end up with Fisher information as $-n/\theta^2$, which is negative so of course impossible.

Then, using directly the definition of Fisher information, see Wikipedia: Fisher information, we get $$\DeclareMathOperator{\E}{\mathbb{E}} I(\theta)=\E_{\theta}\left\{ \left[\frac{\partial}{\partial\theta}\log f(x;\theta)\right]^2\right\} = \int_0^\theta [-n/\theta]^2 (1/\theta)\; dx = (n/\theta)^2 $$ Then the Jeffrey's uninformative prior is proportional to its squareroot, that is, $$ \pi(\theta) \propto 1/\theta, \quad \theta>0 $$ which is an improper prior. But when $n\ge 2$ we get a proper posterior, which is Pareto with parameters of scale $x_m = L$ and shape $\alpha = n$. The density is given by $$ \pi(\theta \,|\, T) = \frac{n}{\theta} \left( \frac{T}{\theta}\right)^{n} $$ for $\theta \ge T$ and $n\ge 2$.

Note that the Jeffrey's prior can be seen as a degenerate Pareto conjugate prior with parameters $x_m = 0, \, \alpha = 0$.

  • $\begingroup$ Great answer (+1). Is degenerate really the correct word in the last sentence? Don't you just mean "improper" again? $\endgroup$ – knrumsey Nov 18 '19 at 19:10

The Jeffreys prior for $\theta$ doesn't depend upon the indicator function, although of course the posterior will. The square root of the second derivative of the log likelihood function is all you need:

$p(\theta) = \left(-\frac{\text{d}^2(\log \theta)}{\text{d}\theta^2}\right)^{1/2}$

When moving on to the posterior, you'll have to remember that indicator function; if $x \leq \theta$ for all $x$, the data says something important about the values that $\theta$ can take. But it's perfectly OK to have a prior that covers a range of values part of which is ruled out once you observe the data.

  • 1
    $\begingroup$ But what is the justification for just ignoring the indicator function when calculating the second derivative? After all, it is a function of $\theta$. $\endgroup$ – user36000 Dec 11 '13 at 14:57
  • $\begingroup$ Well, you could include it in the log likelihood... $\log 1 = 0$, and logs are additive, and $\text{d}0/\text{d}\theta = 0$, so including it won't change anything. Note that from a "forming the posterior" perspective we don't care about the 2nd derivative in the range $\theta < x$, as we already know the posterior probability is... (what?) for $\theta < x$, so the value of the 2nd derivative is irrelevant. $\endgroup$ – jbowman Dec 11 '13 at 15:32
  • 1
    $\begingroup$ Actually it is the square root of the expectation of the second derivative of the loglikelihood with respect to $\theta$. That is $-E_{\theta}\left(\frac{\partial^2}{\partial \theta^2}logf(x|\theta)\right)$. Thus when you are going to calculate the expectation you need the indicator. It indicates the support of the distribution. Are you trying to see if this distribution belongs to the exponential family? In this case, something is wrong, because the support of the distribution must not depend on $\theta$. $\endgroup$ – Christos Dec 11 '13 at 17:40
  • $\begingroup$ But isn't it $\log(I(x \leq \theta))$ instead of $\log(1)$? Maybe AdamO's or Zen's earlier comment is the answer, but I can't connect the dots. $\endgroup$ – user36000 Dec 11 '13 at 21:38
  • $\begingroup$ $I(x \leq \theta) = 1$ when $x \leq \theta$, $0$ otherwise. Since $x \leq \theta$ all the time (the pdf = 0 otherwise), it's always equal to 1 for any observable values of $x$. (More generally, $I(x \in A) = 1$ if $x \in A$, $0$ otherwise.) $\endgroup$ – jbowman Dec 11 '13 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.