Idea and intuition behind quasi maximum likelihood estimation (QMLE)

Question

Question(s): What is the idea and intuition behind quasi maximum likelihood estimation (QMLE; also known as pseudo maximum likelihood estimation, PMLE)? What makes the estimator work when the actual error distribution does not match the assumed error distribution?

The Wikipedia site for QMLE is fine (brief, intuitive, to the point), but I could use some more intuition and detail, perhaps also an illustration. Other references are most welcome. (I remember going over quite a few econometrics textbooks looking for material on QMLE, and to my surprise, QMLE was only covered in one or two of them, e.g. Wooldridge "Econometric Analysis of Cross Section and Panel Data" (2010), Chapter 13 Section 11, pp. 502-517.)

Answer 1

"What makes the estimator work when the actual error distribution does not match the assumed error distribution?"

In principle the QMPLE does not "work", in the sense of being a "good" estimator. The theory developed around the QMLE is useful because it has led to misspecification tests.

What the QMLE certainly does is to consistently estimate the parameter vector which minimizes the Kullback-Leiber Divergence between the true distribution and the one specified. This sounds good, but minimizing this distance does not mean that the minimized distance won't be enormous.

Still, we read that there are many situations that the QMLE is a consistent estimator for the true parameter vector. This has to be assessed case-by-case, but let me give one very general situation, which shows that there is nothing inherent in the QMLE that makes it consistent for the true vector...

... Rather it is the fact that it coincides with another estimator that is always consistent (maintaining the ergodic-stationary sample assumption) : the old-fashioned, Method of Moments estimator.

In other words, when in doubt about the distribution, a strategy to consider is "always specify a distribution for which the Maximum Likelihood estimator for the parameters of interest coincides with the Method of Moments estimator": in this way no matter how off the mark is your distributional assumption, the estimator will at least be consistent.

You can take this strategy to ridiculous extremes: assume that you have a very large i.i.d. sample from a random variable, where all values are positive. Go on and assume that the random variable is normally distributed and apply maximum likelihood for the mean and variance: your QMLE will be consistent for the true values.

Of course this begs the question, why pretending to apply MLE since what we are essentially doing is relying and hiding behind the strengths of Method of Moments (which also guarantees asymptotic normality)?

In other more refined cases, QMLE may be shown to be consistent for the parameters of interest if we can say that we have specified correctly the conditional mean function but not the distribution (this is for example the case for Pooled Poisson QMLE - see Wooldridge).

Answer 2

The originating paper from Wedderburn in 74 is an excellent read regarding the subject of quasilikelihood. In particular he observed that for regular exponential families, the solutions to likelihood equations were obtained by solving a general score equation of the form: $$ 0 = \sum_{i=1}^n \mathbf{S}(\beta, X_i, Y_i) = \mathbf{D}^{T} W \left( Y - g^{-1} (\mathbf{X}^T \beta)\right) $$ Where $\mathbf{D} = \frac{\partial}{\partial \beta} g^{-1} ( \mathbf{X}^T \beta)$ and $W = \mathbf{V}^{-1}$. This notation originates in the work of McCullogh and Nelder in the originating text, "Generalized Linear Models". M&N describe solving these types of functions using the Gauss Newton type algorithm.

Interestingly, however, this formulation hearkened to a method-of-moments type estimator where one could simply sort of "set the thing they want to estimate" in the RHS of the parenthesized expression, and trust that the expression would converge to "that interesting thing". It was a proto form of estimating equations.

Estimating equations were no new concept. In fact, attempts as far back as 1870s and early 1900s to present EEs correctly derived limit theorems from EEs using Taylor expansions, but a lack of connection to a probabilistic model was a cause of contention among critical reviewers.

Wedderburn showed a few very important results: that using the first display in a general framework where the score equation $S$ can be replaced by a quasiscore, not corresponding to any probabilistic model, but instead answering a question of interest, yielded statistically cogent estimates. Reverse transforming a general score resulted in a general qMLE which comes from a likelihood that is correct up to a proportional constant. That proportional constant is called the "dispersion". A useful result from Wedderburn is that strong departures from probabilistic assumptions can result in large or small dispersions.

However, in contrast to the answer above, quasilikelihood has been used extensively. One very nice discussion in McCullogh and Nelder deals with population modeling of horseshoe crabs. Not unlike humans, their mating habits are simply bizarre: where many males may flock to a single female in unmeasured "clusters". From an ecologist perspective, actually observing these clusters is far beyond the scope of their work, but nonetheless arriving at predictions of population size from catch-and-release posed a significant challenge. It turns out that the this mating pattern results in a Poisson model with significant under-dispersion, that is to say the variance is proportional, but not equal to the mean.

Dispersions are considered nuisance parameters in the sense that we generally do not base inference about their value, and jointly estimating them in a single likelihood results in highly irregular likelihoods. Quasilikelihood is a very useful area of statistics, especially in light of the later work on generalized estimating equations.

Answer 3

I had a similar question as the original one posted here from Richard Hardy. My confusion was that the parameters estimated from quasi-ML may not exist in the unknown "true" distribution. In this case, what does "consistency" exactly mean? What do the estimated parameters converge to?

After checking some references (White (1982) should be one of the original articles but is gated. A helpful exposition I found is http://homepage.ntu.edu.tw/~ckuan/pdf/et01/ch9.pdf), my thoughts in plain English are as follows: after admitting that the distribution we assume is just an approximation to the unknown true one, the practical thing we can do is to find the parameter value to minimize their distance (Kullback-Leibler distance to be precise). The beauty of the theory is that, without the need to know the true distribution, the estimated parameters from quasi-ML converge to this distance-minimizing parameter (of course, there are other useful results from the theory such as asymptotic distribution of the estimated parameters etc. but they are not the focus of my question here).

Just as Alecos Papadopolous mentioned in his reply above, the minimized distance could still be large. So the distribution we assume could be a poor approximation to the true one. All that quasi-ML can do is making our assumed distribution as close to the unknown true one as possible. Hope my experience shared here might be helpful for others having similar confusions.