5
$\begingroup$

I need to show that given an i.i.d sample $X_1,\dots X_n$ arising from the model:

$$\{f(x,\theta)=\theta x^{\theta-1}exp\{-x^{\theta}\},x>0,\theta\in (0,\infty)\}$$

that the MLE exists with probability one and is consistent.

I have been instructed to use the following lemma(which I have proved):

  1. Let $S_n$ be a sequence of random real-valued continuous functions defined on $\Theta$ such that, as $n \xrightarrow{}\infty, S_n(\theta) \xrightarrow{P}S(\theta) \forall\theta \in \Theta$ where $S:\Theta \xrightarrow{} R$ is nonrandom. Suppose for some $\theta_0$ in the interior of $\Theta$ and every $\epsilon >0$ small enough we have $S(\theta_0 \pm \epsilon)<0<S(\theta_0 \mp \epsilon)$ and that $S_n$ has exactly one zero $\hat{\theta}_n$ for every natural number n. Then we must have $\hat{\theta}_n\xrightarrow{P} \theta_0$.

  2. I have also been permitted to interchange differentiation $\frac{d}{d\theta}$ and dx-integration without justification.

I am not entirely sure how to begin to prove the existence of the MLE I have tried taking log-likelihoods and manipulating the derivative but to no avail.

Assuming its existence I thought of letting $S_n(\theta)=\frac{1}{n}\sum_{i=1}^n \frac{d}{d\theta}(log(f(X_i,\theta))$ so that we can use the law of large numbers but this has not proved successful. I can see that if I can find $S_n$ and $S$ to match the conditions in 1. then consistency would be immediate.

Any help would be appreciated

$\endgroup$
3
  • $\begingroup$ You need to study the behavior of the derivative function $\theta\to\sum_i \log(x_i)\left(1-x_i^\theta\right)+n/\theta.$ Start by showing that it monotonically decreases from a positive to a negative value on $(0,\infty)$ and therefore, because it is continuous, has a unique zero. $\endgroup$
    – whuber
    Commented Oct 2, 2020 at 15:38
  • $\begingroup$ @whuber oh dear... I calculated the derivative wrong, thank you for that, any help for the second part? $\endgroup$ Commented Oct 2, 2020 at 15:58
  • $\begingroup$ @user3184807 For consistency, taking expectation of $S_n$ gives you $S$. Use the fact/assumption that expectation and derivative commute to show $S$ has the properties stated in your fact 1. Then by LLN, $S_n$ converges to $S$ pointwise in probability. $\endgroup$
    – Michael
    Commented Oct 4, 2020 at 11:58

1 Answer 1

3
$\begingroup$

Existence a.s.

As @whuber says, the score $$ S_n(\theta) = \frac1n\sum_{i=1}^n \frac{\partial}{\partial \theta} \log f(x_i, \theta) = -\frac{1}{\theta^2} - \frac1n\sum_{i=1}^n x_i^{\theta} \log x_i + \frac1n\sum_{i=1}^n \log x_i $$ is a monotonically decreasing function (compute its derivative) such that $$ \lim_{\theta \rightarrow 0^+} S_n(\theta) = \infty \mbox{ and } \lim_{\theta \rightarrow \infty} S_n(\theta) < 0. $$ This tells you $S_n(\theta)$ has unique zero, almost surely.

Consistency

Define $S(\theta) = E_{\theta_0}[\frac{\partial}{\partial \theta} \log f(x, \theta)]$. Then by LLN, $S_n(\theta) \stackrel{p}{\rightarrow} S(\theta)$.

Also, using your fact/assumption (2) that differentiation and expectation commute, $$ E_{\theta_0}[\frac{\partial}{\partial \theta_0} \log f(x, \theta_0)] = \int \frac{\partial}{\partial \theta_0} f(x, \theta_0) dx = \frac{d}{d \theta_0} \int f(x, \theta_0) dx = 0. $$ So $S$ has a zero at $\theta_0$. In fact this zero is unique. (I believe you need this, the continuity condition in your fact/assumption (1) is necessary but not sufficient. $S$ is continuous by the Dominated Convergence Theorem.)

So $S_n$ and $S$ fall under your fact/assumption (1) and consistency follows.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.