# Prove that the MLE exists almost surely and is consistent

I need to show that given an i.i.d sample $$X_1,\dots X_n$$ arising from the model:

$$\{f(x,\theta)=\theta x^{\theta-1}exp\{-x^{\theta}\},x>0,\theta\in (0,\infty)\}$$

that the MLE exists with probability one and is consistent.

I have been instructed to use the following lemma(which I have proved):

1. Let $$S_n$$ be a sequence of random real-valued continuous functions defined on $$\Theta$$ such that, as $$n \xrightarrow{}\infty, S_n(\theta) \xrightarrow{P}S(\theta) \forall\theta \in \Theta$$ where $$S:\Theta \xrightarrow{} R$$ is nonrandom. Suppose for some $$\theta_0$$ in the interior of $$\Theta$$ and every $$\epsilon >0$$ small enough we have $$S(\theta_0 \pm \epsilon)<0 and that $$S_n$$ has exactly one zero $$\hat{\theta}_n$$ for every natural number n. Then we must have $$\hat{\theta}_n\xrightarrow{P} \theta_0$$.

2. I have also been permitted to interchange differentiation $$\frac{d}{d\theta}$$ and dx-integration without justification.

I am not entirely sure how to begin to prove the existence of the MLE I have tried taking log-likelihoods and manipulating the derivative but to no avail.

Assuming its existence I thought of letting $$S_n(\theta)=\frac{1}{n}\sum_{i=1}^n \frac{d}{d\theta}(log(f(X_i,\theta))$$ so that we can use the law of large numbers but this has not proved successful. I can see that if I can find $$S_n$$ and $$S$$ to match the conditions in 1. then consistency would be immediate.

Any help would be appreciated

• You need to study the behavior of the derivative function $\theta\to\sum_i \log(x_i)\left(1-x_i^\theta\right)+n/\theta.$ Start by showing that it monotonically decreases from a positive to a negative value on $(0,\infty)$ and therefore, because it is continuous, has a unique zero.
– whuber
Oct 2, 2020 at 15:38
• @whuber oh dear... I calculated the derivative wrong, thank you for that, any help for the second part? Oct 2, 2020 at 15:58
• @user3184807 For consistency, taking expectation of $S_n$ gives you $S$. Use the fact/assumption that expectation and derivative commute to show $S$ has the properties stated in your fact 1. Then by LLN, $S_n$ converges to $S$ pointwise in probability. Oct 4, 2020 at 11:58

As @whuber says, the score $$S_n(\theta) = \frac1n\sum_{i=1}^n \frac{\partial}{\partial \theta} \log f(x_i, \theta) = -\frac{1}{\theta^2} - \frac1n\sum_{i=1}^n x_i^{\theta} \log x_i + \frac1n\sum_{i=1}^n \log x_i$$ is a monotonically decreasing function (compute its derivative) such that $$\lim_{\theta \rightarrow 0^+} S_n(\theta) = \infty \mbox{ and } \lim_{\theta \rightarrow \infty} S_n(\theta) < 0.$$ This tells you $$S_n(\theta)$$ has unique zero, almost surely.
Define $$S(\theta) = E_{\theta_0}[\frac{\partial}{\partial \theta} \log f(x, \theta)]$$. Then by LLN, $$S_n(\theta) \stackrel{p}{\rightarrow} S(\theta)$$.
Also, using your fact/assumption (2) that differentiation and expectation commute, $$E_{\theta_0}[\frac{\partial}{\partial \theta_0} \log f(x, \theta_0)] = \int \frac{\partial}{\partial \theta_0} f(x, \theta_0) dx = \frac{d}{d \theta_0} \int f(x, \theta_0) dx = 0.$$ So $$S$$ has a zero at $$\theta_0$$. In fact this zero is unique. (I believe you need this, the continuity condition in your fact/assumption (1) is necessary but not sufficient. $$S$$ is continuous by the Dominated Convergence Theorem.)
So $$S_n$$ and $$S$$ fall under your fact/assumption (1) and consistency follows.