Skip to main content
deleted 2 characters in body
Source Link
Alecos Papadopoulos
  • 60.7k
  • 8
  • 154
  • 278

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And they are is not, because thethese terms used areinvolve the expected values of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And they are is not, because the terms used are the expected values of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And they are not, because these terms involve the expected values of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

typos
Source Link
Alecos Papadopoulos
  • 60.7k
  • 8
  • 154
  • 278

ADDENDUM
Responding to the valid remark of the OP in the comments (sometimes, indeed, questions become a springboard for sharing a more general result, and themselves may be neglected in the process), here is how Wilks's proof proceeds: Wilks starts with the joint normal distribution of the MLE, and proceeds to derive the functional expression of the Likelihood Ratio. Up to anand including his eq. $[9]$, the proof can move forward even if we assume that we have a distributional misspecification: as the OP notes, the terms of the variance covariance matrix will be different in the misspecification scenario, but all Wilks does is take derivatives, and identify asymptotically negligible terms. And so he arrives at eq. $[9]$ where we see that the likelihood ratio statistic, if the specification is correct, is just the sum of $h-m$ squared standard normal random variables, and so they are distributed as one chi-square with $h-m$ degrees of freedom: (generic notation)

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And itthey are is not, because the termterms used isare the expected valuevalues of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

ADDENDUM
Responding to the valid remark of the OP in the comments (sometimes, indeed, questions become a springboard for sharing a more general result, and themselves may be neglected in the process), here is how Wilks's proof proceeds: Wilks starts with the joint normal distribution of the MLE, and proceeds to derive the functional expression of the Likelihood Ratio. Up to an including his eq. $[9]$, the proof can move forward even if we assume that we have a distributional misspecification: as the OP notes, the terms of the variance covariance matrix will be different in the misspecification scenario, but all Wilks does is take derivatives, and identify asymptotically negligible terms. And so he arrives at eq. $[9]$ where we see that the likelihood ratio statistic, if the specification is correct, is just the sum of $h-m$ squared standard normal random variables, and so they are distributed as one chi-square with $h-m$ degrees of freedom: (generic notation)

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And it is not, because the term used is the expected value of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

ADDENDUM
Responding to the valid remark of the OP in the comments (sometimes, indeed, questions become a springboard for sharing a more general result, and themselves may be neglected in the process), here is how Wilks's proof proceeds: Wilks starts with the joint normal distribution of the MLE, and proceeds to derive the functional expression of the Likelihood Ratio. Up to and including his eq. $[9]$, the proof can move forward even if we assume that we have a distributional misspecification: as the OP notes, the terms of the variance covariance matrix will be different in the misspecification scenario, but all Wilks does is take derivatives, and identify asymptotically negligible terms. And so he arrives at eq. $[9]$ where we see that the likelihood ratio statistic, if the specification is correct, is just the sum of $h-m$ squared standard normal random variables, and so they are distributed as one chi-square with $h-m$ degrees of freedom: (generic notation)

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And they are is not, because the terms used are the expected values of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

Added the answer to the question
Source Link
Alecos Papadopoulos
  • 60.7k
  • 8
  • 154
  • 278

ADDENDUM
Responding to the valid remark of the OP in the comments (sometimes, indeed, questions become a springboard for sharing a more general result, and themselves may be neglected in the process), here is how Wilks's proof proceeds: Wilks starts with the joint normal distribution of the MLE, and proceeds to derive the functional expression of the Likelihood Ratio. Up to an including his eq. $[9]$, the proof can move forward even if we assume that we have a distributional misspecification: as the OP notes, the terms of the variance covariance matrix will be different in the misspecification scenario, but all Wilks does is take derivatives, and identify asymptotically negligible terms. And so he arrives at eq. $[9]$ where we see that the likelihood ratio statistic, if the specification is correct, is just the sum of $h-m$ squared standard normal random variables, and so they are distributed as one chi-square with $h-m$ degrees of freedom: (generic notation)

$$-2\ln \lambda = \sum_{i=1}^{h-m}\left(\sqrt n\frac{\hat \theta_i - \theta_i}{\sigma_i}\right)^2 \xrightarrow{d} \mathcal \chi^2_{h-m}$$

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And it is not, because the term used is the expected value of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

So under misspecification we have something like $$-2\ln \lambda = \sum_{i=1}^{h-m}\left(\sqrt n\frac{\hat \theta_i - \theta_i}{a_i}\right)^2$$ and the best we can do is to manipulate it into

$$-2\ln \lambda = \sum_{i=1}^{h-m}\frac {\sigma_i^2}{a_i^2}\left(\sqrt n\frac{\hat \theta_i - \theta_i}{\sigma_i}\right)^2 = \sum_{i=1}^{h-m}\frac {\sigma_i^2}{a_i^2}\mathcal \chi^2_1$$

which is a sum of scaled chi-square r.v.'s, no longer distributed as one chi-square r.v. with $h-m$ degrees of freedom. The reference provided by the OP is indeed a very clear exposition of this more general case that includes Wilks' result as a special case.

ADDENDUM
Responding to the valid remark of the OP in the comments (sometimes, indeed, questions become a springboard for sharing a more general result, and themselves may be neglected in the process), here is how Wilks's proof proceeds: Wilks starts with the joint normal distribution of the MLE, and proceeds to derive the functional expression of the Likelihood Ratio. Up to an including his eq. $[9]$, the proof can move forward even if we assume that we have a distributional misspecification: as the OP notes, the terms of the variance covariance matrix will be different in the misspecification scenario, but all Wilks does is take derivatives, and identify asymptotically negligible terms. And so he arrives at eq. $[9]$ where we see that the likelihood ratio statistic, if the specification is correct, is just the sum of $h-m$ squared standard normal random variables, and so they are distributed as one chi-square with $h-m$ degrees of freedom: (generic notation)

$$-2\ln \lambda = \sum_{i=1}^{h-m}\left(\sqrt n\frac{\hat \theta_i - \theta_i}{\sigma_i}\right)^2 \xrightarrow{d} \mathcal \chi^2_{h-m}$$

But if we have misspecification, then the terms that are used in order to scale the centered and magnified MLE $\sqrt n(\hat \theta -\theta)$ are no longer the terms that will make the variances of each element equal to unity, and so transform each term into a standard normal r.v and the sum into a chi-square.
And it is not, because the term used is the expected value of the second derivatives of the log-likelihood... but the expected value can only be taken with respect to the true distribution, since the MLE is a function of the data and the data follows the true distribution, while the second derivatives of the log-likelihood are calculated based on the wrong density assumption.

So under misspecification we have something like $$-2\ln \lambda = \sum_{i=1}^{h-m}\left(\sqrt n\frac{\hat \theta_i - \theta_i}{a_i}\right)^2$$ and the best we can do is to manipulate it into

$$-2\ln \lambda = \sum_{i=1}^{h-m}\frac {\sigma_i^2}{a_i^2}\left(\sqrt n\frac{\hat \theta_i - \theta_i}{\sigma_i}\right)^2 = \sum_{i=1}^{h-m}\frac {\sigma_i^2}{a_i^2}\mathcal \chi^2_1$$

which is a sum of scaled chi-square r.v.'s, no longer distributed as one chi-square r.v. with $h-m$ degrees of freedom. The reference provided by the OP is indeed a very clear exposition of this more general case that includes Wilks' result as a special case.

Source Link
Alecos Papadopoulos
  • 60.7k
  • 8
  • 154
  • 278
Loading