8
$\begingroup$

A nonnegative random variable $x$ has a continuous uniform distribution in the interval $(0,\theta)$. Therefore, the likelihood is given by: $f(x|\theta) = \frac{1}{\theta}I(x\leq\theta)$, where $I$ is an indicator function.

But what is the Jeffreys prior for the parameter $\theta$? In particular, how to handle the indicator function when calculating the Fisher information?

Improve this question
$\endgroup$
2
  • $\begingroup$ It might be of use to review these measure theory concepts. $\endgroup$
    – AdamO
    Commented Dec 11, 2013 at 1:23
  • 1
    $\begingroup$ Note that this model has a conjugate prior: the Pareto distribution. $\endgroup$
    – Zen
    Commented Dec 11, 2013 at 2:46

2 Answers 2

Reset to default
6
$\begingroup$

I think the other answer is wrong, so will give a detailed development here. First, let $X_1, \dotsc, X_n$ be iid uniform on the interval $(0,\theta)$. Then the likelihood function can be written as $$ L(\theta)= \theta^{-n} \cdot \mathbb{1}(\theta \ge T) $$ where $T=\max(X_1, \dotsc, X_n)$ is the sufficient statistic for $\theta$. The log likelihood then can be written $$ l(\theta)=\log L(\theta)= -n \log \theta+ \begin{cases} 0 &,\theta\ge T \\ -\infty &, \theta<T \end{cases} $$ and its first derivative (where it exists) can be written $$ -n/\theta $$ with expectation equal to $ -n/\theta \not= 0$, so **we cannot calculate the Fisher information via the expectation of the second derivative, since that equality depends on equality to zero above. If we nevertheless do that, we end up with Fisher information as $-n/\theta^2$, which is negative so of course impossible.

Then, using directly the definition of Fisher information, see Wikipedia: Fisher information, we get $$\DeclareMathOperator{\E}{\mathbb{E}} I(\theta)=\E_{\theta}\left\{ \left[\frac{\partial}{\partial\theta}\log f(x;\theta)\right]^2\right\} = \int_0^\theta [-n/\theta]^2 (1/\theta)\; dx = (n/\theta)^2 $$ Then the Jeffrey's uninformative prior is proportional to its squareroot, that is, $$ \pi(\theta) \propto 1/\theta, \quad \theta>0 $$ which is an improper prior. But when $n\ge 2$ we get a proper posterior, which is Pareto with parameters of scale $x_m = L$ and shape $\alpha = n$. The density is given by $$ \pi(\theta \,|\, T) = \frac{n}{\theta} \left( \frac{T}{\theta}\right)^{n} $$ for $\theta \ge T$ and $n\ge 2$.

Note that the Jeffrey's prior can be seen as a degenerate Pareto conjugate prior with parameters $x_m = 0, \, \alpha = 0$.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Great answer (+1). Is degenerate really the correct word in the last sentence? Don't you just mean "improper" again? $\endgroup$
    – knrumsey
    Commented Nov 18, 2019 at 19:10
  • $\begingroup$ While there is nothing wrong mathematically, I think you would agree that $I(\theta)$ obtained here is not usual in the sense highlighted by Xi'an in this post. I looked at Lehmann's book on large sample theory where he justifies restricting the measure $I(\theta)$ to so called 'regular' cases while discussing the $U(0,\theta)$ case and why Fisher information is not meaningful here. $\endgroup$
    – StubbornAtom
    Commented Apr 24, 2020 at 21:42
0
$\begingroup$

The Jeffreys prior for $\theta$ doesn't depend upon the indicator function, although of course the posterior will. The square root of the second derivative of the log likelihood function is all you need:

$p(\theta) = \left(-\frac{\text{d}^2(\log \theta)}{\text{d}\theta^2}\right)^{1/2}$

When moving on to the posterior, you'll have to remember that indicator function; if $x \leq \theta$ for all $x$, the data says something important about the values that $\theta$ can take. But it's perfectly OK to have a prior that covers a range of values part of which is ruled out once you observe the data.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ But what is the justification for just ignoring the indicator function when calculating the second derivative? After all, it is a function of $\theta$. $\endgroup$
    – user36000
    Commented Dec 11, 2013 at 14:57
  • $\begingroup$ Well, you could include it in the log likelihood... $\log 1 = 0$, and logs are additive, and $\text{d}0/\text{d}\theta = 0$, so including it won't change anything. Note that from a "forming the posterior" perspective we don't care about the 2nd derivative in the range $\theta < x$, as we already know the posterior probability is... (what?) for $\theta < x$, so the value of the 2nd derivative is irrelevant. $\endgroup$
    – jbowman
    Commented Dec 11, 2013 at 15:32
  • 1
    $\begingroup$ Actually it is the square root of the expectation of the second derivative of the loglikelihood with respect to $\theta$. That is $-E_{\theta}\left(\frac{\partial^2}{\partial \theta^2}logf(x|\theta)\right)$. Thus when you are going to calculate the expectation you need the indicator. It indicates the support of the distribution. Are you trying to see if this distribution belongs to the exponential family? In this case, something is wrong, because the support of the distribution must not depend on $\theta$. $\endgroup$
    – Christos
    Commented Dec 11, 2013 at 17:40
  • $\begingroup$ But isn't it $\log(I(x \leq \theta))$ instead of $\log(1)$? Maybe AdamO's or Zen's earlier comment is the answer, but I can't connect the dots. $\endgroup$
    – user36000
    Commented Dec 11, 2013 at 21:38
  • $\begingroup$ $I(x \leq \theta) = 1$ when $x \leq \theta$, $0$ otherwise. Since $x \leq \theta$ all the time (the pdf = 0 otherwise), it's always equal to 1 for any observable values of $x$. (More generally, $I(x \in A) = 1$ if $x \in A$, $0$ otherwise.) $\endgroup$
    – jbowman
    Commented Dec 11, 2013 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.