I recently noticed that for non-informative priors, people usually use something like a uniform prior, which works for many different distributions. However, assuming that your likelihood is nothing more than a uniform $\frac{1}{\theta}$ for a parameter theta, what is a non-informative prior that works here? The first thing coming to mind is the Jeffrey's prior, but that yields $-\frac{1}{\theta^2}$, which when trying to calculate the posterior mean / variance leads to a divergent series when trying to sum across the product of the prior and likelihood. Does anyone have any ideas of what is a good non-informative prior?

  • 1
    $\begingroup$ To be clear, this prior is a Pareto(1, 1) distribution. The posterior form is known and has finite moments: Pareto(max(1, data), 1 + sample_size). The fact that the prior does not have a finite first moment shouldn't scare us as long as it is a distribution (e.g., using a Cauchy distribution as a prior). So I'm a little confused as to what the problem is exactly? $\endgroup$
    – user44764
    Oct 1 '15 at 0:20
  • 1
    $\begingroup$ Are you talking about a Uniform$(0,\theta)$?. (Note that the prior is for the parameter, rather than a distribution.) $\endgroup$
    – Glen_b
    Oct 1 '15 at 0:52
  • $\begingroup$ @Matt What I mean is that if I want a prior where the expectation and variance exist, what should I choose? $\endgroup$
    – user123276
    Oct 1 '15 at 2:10
  • 2
    $\begingroup$ Warning: Your likelihood is more than $1/\theta$ in that there is an indicator at play: $\mathbb{I}_{0\le x\le \theta}$. $\endgroup$
    – Xi'an
    Oct 23 '15 at 14:14
  • $\begingroup$ A similar question with answers: stats.stackexchange.com/questions/69383/… $\endgroup$ Feb 11 '19 at 11:27

I will assume your model is a uniform distribution on the interval $(0, \theta)$. So let $X_1, \dotsc, X_n$ iid with that distribution, with $\theta>0$. Then the likelihood function can be written $$ L(\theta) = \theta^{-n} \cdot \mathbb{1}(\theta \ge T) $$ where $T=\max_{i=1}^n X_i$. The first idea is the Jeffrey' prior, and your statement of that cannot be right (it is negative!). What I get is $n/\theta$ for $\theta \ge T$. That may look strange, first, it depends on the data through $T$. But in this case that isn't a problem, since the likelihood is zero for $\theta < T$, so the prior on that interval is unimportant, it will always be multiplied with zero when calculating the posterior. Second, it is improper, but, as long as $n\ge 2$ it leads to a proper posterior (which do have the pareto form). A detailed development is here.

The posterior is $$ f(\theta | T) = \frac{n-1}{\theta}\cdot\left(\frac{T}{\theta}\right)^{n-1} $$ for $\theta \ge T$.

For more information see this paper. A more thorough paper is this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.