19
$\begingroup$

Could anyone please tell me what the regularity conditions are for the asymptotic distribution of Likelihood Ratio test?

Everywhere I look, it is written 'Under the regularity conditions' or 'under the probabilistic regularities'. What are the conditions exactly? That the first and second log-likelihood derivatives exist and Information matrix not be zero? Or something else entirely?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
26
$\begingroup$

The required regularity conditions are listed in most intermediate textbooks and are not different than those of the mle. The following ones concern the one parameter case yet their extension to the multi-parameter one is straightforward.

Condition 1: The pdfs are distinct, i.e. $\theta \neq \theta ^{\prime} \Rightarrow f(x_i;\theta)\neq f(x_i;\theta ^{\prime}) $

Note that this condition essentially states that the parameter identifies the pdf.

Condition 2: The pdfs have common support for all $\theta$

What this implies is that the support does not depend on $\theta$

Condition 3:The point $\theta_0$, the real parameter that is, is an interior point in some set $\Omega$

The last one concerns the possibility that $\theta$ appears in the endpoints of an interval.

These three together guarantee that the likelihood is maximised at the true parameter $\theta_0$ and then that the mle $\hat{\theta}$ that solves the equation

$$\frac{\partial l(\theta)} {\partial \theta}=0$$

is consistent.

Condition 4: The pdf $f(x;\theta)$ is twice differentiable as a function of $\theta$

Condition 5: The integral $\int_{-\infty}^{\infty} f(x;\theta)\ \mathrm dx$ can be differentiated twice under the integral sign as a function of $\theta$

We need the last two to derive the Fisher Information which plays a central role in the theory of convergence of the mle.

For some authors these suffice but if we are to be thorough we additionally need a final condition that ensures the asymptotic normality of the mle.

Condition 6:The pdf $f(x;\theta)$ is three times differentiable as a function of $\theta$. Further for all $\theta \in \Omega$, there exist a constant $c$ and a function $M(x)$ such that

$$\left| \frac{\partial^3 \log f(x;\theta)}{\partial \theta^3} \right| \leq M(x)$$

with $E_{\theta_0} \left[M(X)\right] <\infty$ for all $|\theta-\theta_0|<c$ and all $x$ in the support of $X$

Essentially the last condition allows us to conclude that the remainder of a second order Taylor expansion about $\theta_0$ is bounded in probability and thus poses no problem asymptotically.

Is that what you had in mind?

Improve this answer
$\endgroup$
9
  • $\begingroup$ Thanks. But you sure that the regularity conditions related to the proof that -2log(lambda) follows Chi square with df 1 are the same? $\endgroup$
    – Kingstat
    Commented Jun 7, 2014 at 15:06
  • 1
    $\begingroup$ @Kingstat Yes. These conditions come from Hogg and Craig "Introduction to Mathematical Statistics" and ensure that under $H_0$,that is $\theta=\theta_0$, $-2\log\Lambda \to^D \chi^2 (1)$ $\endgroup$
    – JohnK
    Commented Jun 7, 2014 at 15:08
  • $\begingroup$ could you please also tell me how for a N(θ,1) density, Rao's Score test is equivalent to the UMPU test? $\endgroup$
    – Kingstat
    Commented Jun 7, 2014 at 15:14
  • $\begingroup$ @Kingstat What does UMPU stand for? $\endgroup$
    – JohnK
    Commented Jun 7, 2014 at 15:20
  • 1
    $\begingroup$ @ABC -- I think you might be confusing the parameter ($\theta$) and the variable itself ($x$). Condition 2 says that no matter the value of $\theta$, the same values of $x$ have non-zero probability. $\endgroup$
    – rcorty
    Commented Apr 8, 2017 at 20:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.