# Metropolis-Hastings fails when the loglikelihood is monotonically increasing with a parameter

I'm trying to estimate the parameters of a Pareto distribution (actually the paretian tail of a generic distribution) via Metropolis-Hastings.

The problem is that the loglikelihood,

$$l(\alpha, x_m) = n\log(\alpha) + n\alpha\log(x_m) - (1 + \alpha)\sum\log(x),$$

is monotonically increasing with $x_m$, so that the greater the value of $x_m$, the greater the value of the likelihood function.

Thus, the chain for the parameter $x_m$ never converges. Any idea about overcoming such a problem?

EDIT: the following picture illustrates the kind of distribution I'm dealing with. In this case, $x_m = 260$, and $\alpha = 2.5$.

The following pictures shows the loglikelihood (fixed $\alpha = 2.5$) for increasing values of $x_m$.

• If you put a prior on $x_m$ that decreases faster than $\log(x_m)$ after some point, you should be OK. How to select such a prior is beyond the scope of this comment (sorry about that!) – jbowman Aug 15 '15 at 14:31
• How can you have data less than $x_m$ when $x_m$ is the lower bound of the support for the data? – jaradniemi Aug 16 '15 at 15:06
• Because $x_m$ is the lower bound of the support for the tail of the distribution. – stochazesthai Aug 16 '15 at 15:07
• No, it's the lower bound for the support of the data, i.e. no data can be smaller than $x_m$. – jaradniemi Aug 16 '15 at 15:08

The problem is not with Metropolis Hasting, but rather that you've created a degenerate distribution.

If your posterior distribution monotonically increases with $x_m$, and $x_m$ does not have an upper bound, we get

$P(x_m > \gamma | x, \alpha) \propto \int_\gamma^\infty l(\alpha, x_m) dx_m = \infty$

$\forall \gamma \in \mathbb{R}$

Therefore, your MH algorithm should diverge.

• For simplicity's sake, I'm assuming $\alpha$ is fixed. – Cliff AB Aug 16 '15 at 15:35
• Even though @stochazesthai has selected this as the answer. $x_m$ does not have a degenerate distribution. It has a completely proper distribution with support $(0,\min(x))$ (assuming the prior had support in this region). Using your notation, $P(x_m>\min(x)|x,\alpha) = 0$. – jaradniemi Aug 16 '15 at 17:18
• @jaradniemi: I think there's some confusion (but maybe it's on my end?): I don't believe stochazesthai is using a Pareto distribution, but rather a distribution where the tail is based on a Pareto distribution. However, this novel distribution leads to a degenerate log likelihood function. As he has defined this distribution, I do not see any declaration that the support is restricted to (0, min(x) ) – Cliff AB Aug 16 '15 at 17:24
• And perhaps it is on my part, but the log-likelihood that is written down is exactly the log-likelihood of the Pareto distribution except that he is missing the indicator. – jaradniemi Aug 16 '15 at 18:05

Data arising from a Pareto distribution have support $[x_m,\infty)$. Thus the maximum value $x_m$ can have is the minimum of your data, i.e. the posterior distribution and full conditional distribution have support bounded above by the minimum of your data.

• This is not useful at all, since when I have to estimate the paretian tail of a distribution, the minimum of data is not $x_m$. Otherwise, I could set $x_m = \min(x)$ without trying to estimate it. – stochazesthai Aug 16 '15 at 4:54
• I think you are missing the point. As a function of $x_m$, the likelihood function increases monotonically up to $\min(x)$. For all values of $x_m$ above $\min(x)$, the likelihood function is zero, i.e. the log-likelihood function is negative infinity. – jaradniemi Aug 16 '15 at 14:58
• I updated the question with the ccdf of the distribution i'm dealing with. As you can see, the likelihood does exist also for values smaller than the true $x_m$. – stochazesthai Aug 16 '15 at 15:04