5
$\begingroup$

Independent random variables $X_1,X_2,\ldots,X_n \sim f_X$ are modeled with a common density

$$f_X(x) = \frac{\alpha(x/\beta)^{\alpha-1}}{\beta} \quad \quad \quad \text{for all } 0 \le x \le \beta.$$

Then I've calculated the log-likelihood function as $$ l(\alpha,\beta)=n\log(\alpha)-n\alpha\log(\beta)+(\alpha-1)\sum_{i=1}^n\log(x_i).$$

And I've found an estimate for $\alpha$ by taking the derivative $$\hat\alpha=\frac{n}{\log(\beta)-\sum_{i=1}^n\log(x_i)},$$

for $\log(\beta)\ne\sum_{i=1}^n\log(x_i)$. But how can I find the MLE of $\beta$, when it is also a constraint on where the function is defined?

$\endgroup$
3
  • $\begingroup$ I just realised that since the score function is decreasing in $\beta$ then it should be minimized which is gotten through order statistics, so $$\hat\beta=X^{(n)} $$ $\endgroup$
    – Nic Nic
    Oct 30, 2018 at 9:35
  • $\begingroup$ Is this is part of an assignment or homework, consider adding the self-study tag. Also read the tag wiki. $\endgroup$ Oct 30, 2018 at 9:53
  • 1
    $\begingroup$ Forgot to add the divide by $\beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as. $\endgroup$
    – Nic Nic
    Oct 30, 2018 at 10:08

2 Answers 2

4
$\begingroup$

Your density function is:

$$p_X(x|\alpha,\beta) = \frac{\alpha}{\beta} \Big( \frac{x}{\beta} \Big)^{\alpha-1} \quad \quad \quad \text{for } 0 \leqslant x \leqslant \beta.$$

Hence, your log-likelihood function is:

$$\ell_\mathbf{x}(\alpha, \beta) = n \ln \alpha - n \alpha \ln \beta + (\alpha-1) \sum_{i=1}^n \ln x_i \quad \quad \quad \text{for } 0 \leqslant x_{(1)} \leqslant x_{(n)} \leqslant \beta.$$

The score function and Hessian matrix are given respectively by:

$$\begin{equation} \begin{aligned} \nabla \ell_\mathbf{x}(\alpha, \beta) &= \begin{bmatrix} n/\alpha - n \ln \beta + \sum_{i=1}^n \ln x_i \\[6pt] n \alpha/\beta \\[6pt] \end{bmatrix}, \\[10pt] \nabla^2 \ell_\mathbf{x}(\alpha, \beta) &= \begin{bmatrix} -n/\alpha^2 & n/\beta \\[6pt] n/\beta & - n \alpha/\beta^2 \\[6pt] \end{bmatrix}. \end{aligned} \end{equation}$$

The function is strictly increasing with respect to $\beta$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $\beta$ occurs at the boundary point, and the MLE of $\alpha$ occurs at the unique critical point. We have the estimators:

$$\hat{\alpha} = \frac{n}{\sum_{i=1}^n (\ln x_{(n)} - \ln x_i)} \quad \quad \quad \hat{\beta} = x_{(n)}.$$

As you can see, when your parameter enters the density as a bound on the range of $x$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.

$\endgroup$
6
  • $\begingroup$ Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite? $\endgroup$ Oct 30, 2018 at 10:45
  • $\begingroup$ Since the MLE for $\beta$ occurs at a boundary point, you only need to check SOC for the MLE of $\alpha$. This confirms that there is a unique critical point value that is a local maximum. $\endgroup$
    – Ben
    Oct 30, 2018 at 10:48
  • $\begingroup$ I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $\frac{\partial \ell_{\mathbf x}}{\partial \beta}$ does not exist at $\beta=x_{(n)}$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems. $\endgroup$ Oct 30, 2018 at 11:01
  • $\begingroup$ You can still check SOC by looking at the "profile log-likelihood" that results from substituting $\beta = x_{(n)}$. $\endgroup$
    – Ben
    Oct 30, 2018 at 11:03
  • $\begingroup$ And what is SOC? $\endgroup$ Oct 30, 2018 at 11:05
2
$\begingroup$

Looks like both $\alpha$ and $\beta$ are unknown here. So our parameter is $\theta=(\alpha,\beta)$.

The population pdf is $$f_{\theta}(x)=\frac{\alpha}{\beta^{\alpha}}x^{\alpha-1}\mathbf1_{0<x<\beta}\quad,\,\alpha>0$$

So, given the sample $(x_1,x_2,\ldots,x_n)$, likelihood function of $\theta$ is

\begin{align} L(\theta)&=\prod_{i=1}^n f_{\theta}(x_i) \\&=\left(\frac{\alpha}{\beta^{\alpha}}\right)^n\left(\prod_{i=1}^n x_i\right)^{\alpha-1}\mathbf1_{0<x_1,x_2,\ldots,x_n<\beta} \\&=\left(\frac{\alpha}{\beta^{\alpha}}\right)^n\left(\prod_{i=1}^n x_i\right)^{\alpha-1}\mathbf1_{0<x_{(n)}<\beta}\quad,\,\alpha>0 \end{align}

, where $x_{(n)}=\max_{1\le i\le n} x_i$ is the largest order statistic.

The log-likelihood is therefore $$\ell(\theta)=n(\ln\alpha-\alpha\ln\beta)+(\alpha-1)\sum_{i=1}^n \ln x_i+\ln(\mathbf1_{0<x_{(n)}<\beta})$$

Observe that, given the sample, the parameter space has become $$\Theta=\{\theta:\alpha>0,\beta>x_{(n)}\}$$

Keeping $\alpha$ fixed, justify that $\ell(\theta)$ is maximized for the minimum value of $\beta$ subject to the constraint $\beta\in(x_{(n)},\infty)$. In other words, as you say, $\ell(\theta)$ is a decreasing function of $\beta$ for fixed $\alpha$. Hence conclude that MLE of $\beta$ as you guessed is $$\hat\beta=X_{(n)}$$

It is now valid to derive the MLE of $\alpha$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $\beta$.

Indeed,

\begin{align} \frac{\partial\ell}{\partial\alpha}&=\frac{n}{\alpha}-n\ln\beta+\sum_{i=1}^n \ln x_i \end{align}

, which vanishes if and only if $$\alpha=\frac{n}{n\ln\beta-\sum_{i=1}^n\ln x_i}$$

(Since $x_i<\beta\implies \ln x_i<\ln\beta\implies \sum \ln x_i<n\ln\beta$, the above expression is defined.)

So our possible candidate for MLE of $\alpha$ is $$\hat\alpha=\frac{n}{n\ln\hat\beta-\sum_{i=1}^n\ln x_i}$$

At this point, you can finish your argument saying that MLE of $\theta=(\alpha,\beta)$ is $\hat\theta=(\hat\alpha,\hat\beta)$.

But since this is a maximization problem in two variables $(\alpha,\beta)$, you could perhaps verify that $$\ell(\hat\theta)\ge \ell (\theta)$$ holds for every $\theta$. This would be a bit more rigorous I think.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.