2
$\begingroup$

In linear regression, the Ramsey RESET test can be used to test if the model is misspecified. The Gauss-Markov theorem allows us to understand the consequences of misspecification in linear regression.

However, what will happen if a logistic regression is misspecified? The principle of Ramsey test may also be applied to logistic regression as suggested in the paper: A regression error specification test (RESET) for generalized linear models

Suppose the test result indicates the logistic regression is misspecified, what would be the impact or consequences?

$\endgroup$
3
$\begingroup$

To answer this question you must first understand that logistic regression is simply a type of maximum-likelihood estimator (which itself is a type of M-estimator). If you need a refresher on the assumptions made when implementing MLE, I suggest you read this previous answer of mine.

Next, we must specify what you mean by misspecification. In econometrics, we typically take this to mean that the structure of the model is incorrect. This could be the omission of a variable (omitted variable bias) or it could be that the true model is not a logit in the case of logistic regression (perhaps instead it is probit).

It turns out what happens in MLE under misspecification is that our estimator, $\hat \theta$ will be consistent for $\theta^*$ where $\theta^*$ is the unique minimizer of the Kullback-Leibler divergence between the true distribution, given by density $g$ under appropriate conditions, and the model distribution, estimated with density $f$. This result is made rigorous in Theorem 2.2 of White 1982.

We can interpret $f(x|\theta^*)$ as the projection of $g$ onto the model $\{f(x|\theta):\theta\in\Theta\}$ in terms of KL divergence. Such that, we estimate the closest "version" of $f$ to $g$ in KL-divergence. Note this result holds for omitted variable bias with independent regressors since we can take $g = f(x'|\theta)$ for a different set of $x'$. Unfortunately, there are some drawbacks here as this could mean that our parameter is not consistent for some possible parameters of interest.

This is a pretty good and intuitive consistency result, but can we get asymptotic normality for our estimator of $\theta^*$? It turns out that yes we can! But, we have to use a different covariance estimator. In particular, we get the result,

$\sqrt{n}(\hat\theta - \theta^*)\overset{d}{\to}N(0,[\mathbb{E}[\frac{\partial}{\partial\theta}\log f(x|\theta)]]^{-1}[Var_g[\frac{\partial}{\partial\theta}\log f(x|\theta)][\mathbb{E}[\frac{\partial}{\partial\theta}\log f(x|\theta)]]^{-1})$

Which has the common "sandwich" format we typically find in robust variance estimators (Theorem 3.2 of White 1982).

Finally, the White 1982 paper also provides results for hypothesis tests of this estimator and it provides a test for misspecification based on the information matrix equality. You can find this in section 4 of this paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.