Prove that the MLE exists almost surely and is consistent

Question

I need to show that given an i.i.d sample $X_1,\dots X_n$ arising from the model:

$$\{f(x,\theta)=\theta x^{\theta-1}exp\{-x^{\theta}\},x>0,\theta\in (0,\infty)\}$$

that the MLE exists with probability one and is consistent.

I have been instructed to use the following lemma(which I have proved):

1. Let $S_n$ be a sequence of random real-valued continuous functions defined on $\Theta$ such that, as $n \xrightarrow{}\infty, S_n(\theta) \xrightarrow{P}S(\theta) \forall\theta \in \Theta$ where $S:\Theta \xrightarrow{} R$ is nonrandom. Suppose for some $\theta_0$ in the interior of $\Theta$ and every $\epsilon >0$ small enough we have $S(\theta_0 \pm \epsilon)<0<S(\theta_0 \mp \epsilon)$ and that $S_n$ has exactly one zero $\hat{\theta}_n$ for every natural number n. Then we must have $\hat{\theta}_n\xrightarrow{P} \theta_0$.

2. I have also been permitted to interchange differentiation $\frac{d}{d\theta}$ and dx-integration without justification.

I am not entirely sure how to begin to prove the existence of the MLE I have tried taking log-likelihoods and manipulating the derivative but to no avail.

Assuming its existence I thought of letting $S_n(\theta)=\frac{1}{n}\sum_{i=1}^n \frac{d}{d\theta}(log(f(X_i,\theta))$ so that we can use the law of large numbers but this has not proved successful. I can see that if I can find $S_n$ and $S$ to match the conditions in 1. then consistency would be immediate.

Any help would be appreciated

Answer 1

Existence a.s.

As @whuber says, the score $$ S_n(\theta) = \frac1n\sum_{i=1}^n \frac{\partial}{\partial \theta} \log f(x_i, \theta) = -\frac{1}{\theta^2} - \frac1n\sum_{i=1}^n x_i^{\theta} \log x_i + \frac1n\sum_{i=1}^n \log x_i $$ is a monotonically decreasing function (compute its derivative) such that $$ \lim_{\theta \rightarrow 0^+} S_n(\theta) = \infty \mbox{ and } \lim_{\theta \rightarrow \infty} S_n(\theta) < 0. $$ This tells you $S_n(\theta)$ has unique zero, almost surely.

Consistency

Define $S(\theta) = E_{\theta_0}[\frac{\partial}{\partial \theta} \log f(x, \theta)]$. Then by LLN, $S_n(\theta) \stackrel{p}{\rightarrow} S(\theta)$.

Also, using your fact/assumption (2) that differentiation and expectation commute, $$ E_{\theta_0}[\frac{\partial}{\partial \theta_0} \log f(x, \theta_0)] = \int \frac{\partial}{\partial \theta_0} f(x, \theta_0) dx = \frac{d}{d \theta_0} \int f(x, \theta_0) dx = 0. $$ So $S$ has a zero at $\theta_0$. In fact this zero is unique. (I believe you need this, the continuity condition in your fact/assumption (1) is necessary but not sufficient. $S$ is continuous by the Dominated Convergence Theorem.)

So $S_n$ and $S$ fall under your fact/assumption (1) and consistency follows.