Deriving the common LIML estimator from first principles

David Hendry (1976) comments that deriving the LIML estimator is hard. I tend to agree. Guido Imbens has a nice expression here which reads \begin{eqnarray} \hat{\beta}_{LIML} = (X'(I - \lambda M_Z) X)^{-1} (X'(I-\lambda M_Z) Y), \end{eqnarray} which comes from a model \begin{eqnarray} y = X \beta + \epsilon \\ X = Z \Pi + V \end{eqnarray} Now, sadly he doesn't give a detailed derivation. I got as far as having a matrix \begin{eqnarray} \begin{pmatrix} y' P_Z z & z' P_Z X \\ X' P_Z Y & X' P_Z Z \end{pmatrix} \end{eqnarray} whose choice of eigenvector $b = \begin{pmatrix} 1 \\ -\beta \end{pmatrix}$ that corresponds to the smallest eigenvalue of said matrix should correspond to the LIML estimate that Imbens proposes.

Unfortunately, I am stuck in seeing the connection between the two and have trouble arriving at the above expression.

Can someone point me to a step-by-step derivation or suggest a way in which I can derive an estimator of $\beta$ that equals the LIML estimator above?

The LIML has somehow disapeared from recent econometrics textbooks. You will need to look in older textbooks, such as Amemiya (1985) or Davidson and Mackinnon (1993), the latter providing the best coverage of the derivation of the LIML (as far as I know).

Given that Davidson and Mackinnon's derivation of the LIML takes not less than 5 pages, it makes little sense to reproduce these here, just be ready to derive a (concentrated) maximum likelihood, use inverse of partitioned matrices and minimise quadratic forms :-)

EDIT The older you go, the better it becomes: Theil (1971)'s textbook contains also a detailed treatment of the derivation of the LIML (p. 679-686).

References:

• Davidson, Russell; MacKinnon, James G. (1993). Estimation and inference in econometrics. Oxford University Press.
• Amemiya, Takeshi (1985). Advanced econometrics. Cambridge, Massachusetts: Harvard University Press.
• Theil, H. (1971), Principles of Econometrics. Vol. 1. New York: Wiley.
• Thanks so much @Mat Theil is the best indeed and Davidson and Mackinnon has a good treatment as well. By the way numerically, they are the same because the estimator b that corresponds to the eigenvector simply does not reject a KS test at five per cent when compared to the conventional expression. Sadly, the new Davidson and MacKinnon is a lot less useful than the old. Indeed, Econometricians are ignoring LIML at their peril for it is the only estimation procedure robust to weak identification and consistent (in most cases) when errors are not normal. LIML is part of many robust tests. Feb 24 '15 at 10:21

At the end of my paper with James MacKinnon and others, you'll find a derivation of a generalization of LIML that I figured out: restricted LIML, meaning LIML with linear constraints on coefficients. The derivation is itself derived from Davidson and MacKinnon (1993), which several respondents have already mentioned. But it is online and free, and might help you understand better. RLIML of course includes regular LIML as a special case.