I'm trying to estimate the parameters of a Pareto distribution (actually the paretian tail of a generic distribution) via Metropolis-Hastings.

The problem is that the loglikelihood,

$$ l(\alpha, x_m) = n\log(\alpha) + n\alpha\log(x_m) - (1 + \alpha)\sum\log(x),$$

is monotonically increasing with $x_m$, so that the greater the value of $x_m$, the greater the value of the likelihood function.

Thus, the chain for the parameter $x_m$ never converges. Any idea about overcoming such a problem?

EDIT: the following picture illustrates the kind of distribution I'm dealing with. In this case, $x_m = 260$, and $\alpha = 2.5$.

enter image description here

The following pictures shows the loglikelihood (fixed $\alpha = 2.5$) for increasing values of $x_m$. enter image description here

  • 1
    $\begingroup$ If you put a prior on $x_m$ that decreases faster than $\log(x_m)$ after some point, you should be OK. How to select such a prior is beyond the scope of this comment (sorry about that!) $\endgroup$ – jbowman Aug 15 '15 at 14:31
  • $\begingroup$ How can you have data less than $x_m$ when $x_m$ is the lower bound of the support for the data? $\endgroup$ – jaradniemi Aug 16 '15 at 15:06
  • $\begingroup$ Because $x_m$ is the lower bound of the support for the tail of the distribution. $\endgroup$ – stochazesthai Aug 16 '15 at 15:07
  • $\begingroup$ No, it's the lower bound for the support of the data, i.e. no data can be smaller than $x_m$. $\endgroup$ – jaradniemi Aug 16 '15 at 15:08

The problem is not with Metropolis Hasting, but rather that you've created a degenerate distribution.

If your posterior distribution monotonically increases with $x_m$, and $x_m$ does not have an upper bound, we get

$P(x_m > \gamma | x, \alpha) \propto \int_\gamma^\infty l(\alpha, x_m) dx_m = \infty$

$\forall \gamma \in \mathbb{R}$

Therefore, your MH algorithm should diverge.

| cite | improve this answer | |
  • $\begingroup$ For simplicity's sake, I'm assuming $\alpha$ is fixed. $\endgroup$ – Cliff AB Aug 16 '15 at 15:35
  • $\begingroup$ Even though @stochazesthai has selected this as the answer. $x_m$ does not have a degenerate distribution. It has a completely proper distribution with support $(0,\min(x))$ (assuming the prior had support in this region). Using your notation, $P(x_m>\min(x)|x,\alpha) = 0$. $\endgroup$ – jaradniemi Aug 16 '15 at 17:18
  • $\begingroup$ @jaradniemi: I think there's some confusion (but maybe it's on my end?): I don't believe stochazesthai is using a Pareto distribution, but rather a distribution where the tail is based on a Pareto distribution. However, this novel distribution leads to a degenerate log likelihood function. As he has defined this distribution, I do not see any declaration that the support is restricted to (0, min(x) ) $\endgroup$ – Cliff AB Aug 16 '15 at 17:24
  • $\begingroup$ And perhaps it is on my part, but the log-likelihood that is written down is exactly the log-likelihood of the Pareto distribution except that he is missing the indicator. $\endgroup$ – jaradniemi Aug 16 '15 at 18:05

Data arising from a Pareto distribution have support $[x_m,\infty)$. Thus the maximum value $x_m$ can have is the minimum of your data, i.e. the posterior distribution and full conditional distribution have support bounded above by the minimum of your data.

| cite | improve this answer | |
  • $\begingroup$ This is not useful at all, since when I have to estimate the paretian tail of a distribution, the minimum of data is not $x_m$. Otherwise, I could set $x_m = \min(x)$ without trying to estimate it. $\endgroup$ – stochazesthai Aug 16 '15 at 4:54
  • $\begingroup$ I think you are missing the point. As a function of $x_m$, the likelihood function increases monotonically up to $\min(x)$. For all values of $x_m$ above $\min(x)$, the likelihood function is zero, i.e. the log-likelihood function is negative infinity. $\endgroup$ – jaradniemi Aug 16 '15 at 14:58
  • $\begingroup$ I updated the question with the ccdf of the distribution i'm dealing with. As you can see, the likelihood does exist also for values smaller than the true $x_m$. $\endgroup$ – stochazesthai Aug 16 '15 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.