Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Could anyone please tell me what the regularity conditions are for the asymptotic distribution of Likelihood Ratio test?

Everywhere I look, it is written 'Under the regularity conditions' or 'under the probabilistic regularities'. What are the conditions exactly? That the first and second log-likelihood derivatives exist and Information matrix not be zero? Or something else entirely?

share|improve this question

1 Answer 1

up vote 5 down vote accepted

The required regularity conditions are listed in most intermediate textbooks and are not different than those of the mle. The following ones concern the one parameter case yet their extension to the multiparameter one is straightforward.

Condition 1: The pdfs are distinct, i.e. $\theta \neq \theta \prime \Rightarrow f(x_i;\theta)\neq f(x_i;\theta \prime) $

Note that this condition essentially states that the parameter identifies the pdf.

Condition 2: The pdfs have common support for all $\theta$

What this implies is that the support does not depend on $\theta$

Condition 3:The point $\theta_0$, the real parameter that is, is an interior point in some set $\Omega$

The last one concerns the possibility that $\theta$ appears in the endpoints of an interval.

These three together guarantee that the likelihood is maximised at the true parameter $\theta_0$ and then that the mle $\hat{\theta}$ that solves the equation

$$\frac{\partial l(\theta)} {\partial \theta}=0$$

is consistent.

Condition 4: The pdf $f(x;\theta)$ is twice differentiable as a function of $\theta$

Condition 5: The integral $\int_{-\infty}^{\infty} f(x;\theta)\ \mathrm dx$ can be differentiated twice under the integral sign as a function of $\theta$

We need the last two to derive the Fisher Information which plays a central role in the theory of convergence of the mle.

For some authors these suffice but if we are to be thorough we additionally need a final condition that ensures the asymptotic normality of the mle.

Condition 6:The pdf $f(x;\theta)$ is three times differentiable as a function of $\theta$. Further for all $\theta \in \Omega$, there exist a constant $c$ and a function $M(x)$ such that

$$\left| \frac{\partial^3 log f(x;\theta)}{\partial \theta^3} \right| \leq M(x)$$

with $E_{\theta_0} \left[M(X)\right] <\infty$ for all $|\theta-\theta_0|<c$ and all $x$ in the support of $X$

Essentially the last condition allows us to conclude that the remainder of a second order Taylor expansion about $\theta_0$ is bounded in Probability and thus poses no problem asymptotically.

Is that what you had in mind?

share|improve this answer
    
Thanks. But you sure that the regularity conditions related to the proof that -2log(lambda) follows Chi square with df 1 are the same? –  Kingstat Jun 7 at 15:06
    
@Kingstat Yes. These conditions come from Hogg and Craig "Introduction to Mathematical Statistics" and ensure that under $H_0$,that is $\theta=\theta_0$, $-2\log\Lambda \to^D \chi^2 (1)$ –  JohnK Jun 7 at 15:08
1  
Thank you so much! –  Kingstat Jun 7 at 15:10
    
could you please also tell me how for a N(θ,1) density, Rao's Score test is equivalent to the UMPU test? –  Kingstat Jun 7 at 15:14
    
@Kingstat What does UMPU stand for? –  JohnK Jun 7 at 15:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.