# Estimating the MLE where the parameter is also the constraint

Independent random variables $$X_1,X_2,\ldots,X_n \sim f_X$$ are modeled with a common density

$$f_X(x) = \frac{\alpha(x/\beta)^{\alpha-1}}{\beta} \quad \quad \quad \text{for all } 0 \le x \le \beta.$$

Then I've calculated the log-likelihood function as $$l(\alpha,\beta)=n\log(\alpha)-n\alpha\log(\beta)+(\alpha-1)\sum_{i=1}^n\log(x_i).$$

And I've found an estimate for $$\alpha$$ by taking the derivative $$\hat\alpha=\frac{n}{\log(\beta)-\sum_{i=1}^n\log(x_i)},$$

for $$\log(\beta)\ne\sum_{i=1}^n\log(x_i)$$. But how can I find the MLE of $$\beta$$, when it is also a constraint on where the function is defined?

• I just realised that since the score function is decreasing in $\beta$ then it should be minimized which is gotten through order statistics, so $$\hat\beta=X^{(n)}$$ Oct 30, 2018 at 9:35
• Is this is part of an assignment or homework, consider adding the self-study tag. Also read the tag wiki. Oct 30, 2018 at 9:53
• Forgot to add the divide by $\beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as. Oct 30, 2018 at 10:08

$$p_X(x|\alpha,\beta) = \frac{\alpha}{\beta} \Big( \frac{x}{\beta} \Big)^{\alpha-1} \quad \quad \quad \text{for } 0 \leqslant x \leqslant \beta.$$

$$\ell_\mathbf{x}(\alpha, \beta) = n \ln \alpha - n \alpha \ln \beta + (\alpha-1) \sum_{i=1}^n \ln x_i \quad \quad \quad \text{for } 0 \leqslant x_{(1)} \leqslant x_{(n)} \leqslant \beta.$$

The score function and Hessian matrix are given respectively by:

\begin{aligned} \nabla \ell_\mathbf{x}(\alpha, \beta) &= \begin{bmatrix} n/\alpha - n \ln \beta + \sum_{i=1}^n \ln x_i \\[6pt] n \alpha/\beta \\[6pt] \end{bmatrix}, \\[10pt] \nabla^2 \ell_\mathbf{x}(\alpha, \beta) &= \begin{bmatrix} -n/\alpha^2 & n/\beta \\[6pt] n/\beta & - n \alpha/\beta^2 \\[6pt] \end{bmatrix}. \end{aligned}

The function is strictly increasing with respect to $$\beta$$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $$\beta$$ occurs at the boundary point, and the MLE of $$\alpha$$ occurs at the unique critical point. We have the estimators:

$$\hat{\alpha} = \frac{n}{\sum_{i=1}^n (\ln x_{(n)} - \ln x_i)} \quad \quad \quad \hat{\beta} = x_{(n)}.$$

As you can see, when your parameter enters the density as a bound on the range of $$x$$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.

• Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite? Oct 30, 2018 at 10:45
• Since the MLE for $\beta$ occurs at a boundary point, you only need to check SOC for the MLE of $\alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
Oct 30, 2018 at 10:48
• I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $\frac{\partial \ell_{\mathbf x}}{\partial \beta}$ does not exist at $\beta=x_{(n)}$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems. Oct 30, 2018 at 11:01
• You can still check SOC by looking at the "profile log-likelihood" that results from substituting $\beta = x_{(n)}$.
– Ben
Oct 30, 2018 at 11:03
• And what is SOC? Oct 30, 2018 at 11:05

Looks like both $$\alpha$$ and $$\beta$$ are unknown here. So our parameter is $$\theta=(\alpha,\beta)$$.

The population pdf is $$f_{\theta}(x)=\frac{\alpha}{\beta^{\alpha}}x^{\alpha-1}\mathbf1_{00$$

So, given the sample $$(x_1,x_2,\ldots,x_n)$$, likelihood function of $$\theta$$ is

\begin{align} L(\theta)&=\prod_{i=1}^n f_{\theta}(x_i) \\&=\left(\frac{\alpha}{\beta^{\alpha}}\right)^n\left(\prod_{i=1}^n x_i\right)^{\alpha-1}\mathbf1_{00 \end{align}

, where $$x_{(n)}=\max_{1\le i\le n} x_i$$ is the largest order statistic.

The log-likelihood is therefore $$\ell(\theta)=n(\ln\alpha-\alpha\ln\beta)+(\alpha-1)\sum_{i=1}^n \ln x_i+\ln(\mathbf1_{0

Observe that, given the sample, the parameter space has become $$\Theta=\{\theta:\alpha>0,\beta>x_{(n)}\}$$

Keeping $$\alpha$$ fixed, justify that $$\ell(\theta)$$ is maximized for the minimum value of $$\beta$$ subject to the constraint $$\beta\in(x_{(n)},\infty)$$. In other words, as you say, $$\ell(\theta)$$ is a decreasing function of $$\beta$$ for fixed $$\alpha$$. Hence conclude that MLE of $$\beta$$ as you guessed is $$\hat\beta=X_{(n)}$$

It is now valid to derive the MLE of $$\alpha$$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $$\beta$$.

Indeed,

\begin{align} \frac{\partial\ell}{\partial\alpha}&=\frac{n}{\alpha}-n\ln\beta+\sum_{i=1}^n \ln x_i \end{align}

, which vanishes if and only if $$\alpha=\frac{n}{n\ln\beta-\sum_{i=1}^n\ln x_i}$$

(Since $$x_i<\beta\implies \ln x_i<\ln\beta\implies \sum \ln x_i, the above expression is defined.)

So our possible candidate for MLE of $$\alpha$$ is $$\hat\alpha=\frac{n}{n\ln\hat\beta-\sum_{i=1}^n\ln x_i}$$

At this point, you can finish your argument saying that MLE of $$\theta=(\alpha,\beta)$$ is $$\hat\theta=(\hat\alpha,\hat\beta)$$.

But since this is a maximization problem in two variables $$(\alpha,\beta)$$, you could perhaps verify that $$\ell(\hat\theta)\ge \ell (\theta)$$ holds for every $$\theta$$. This would be a bit more rigorous I think.