# Literature review on non-linear regression

Does anyone know of a good review article for the statistical literature on non-linear regression? I am primarily interested in consistency results and asymptotics.

Of particular interest is the model

$$y_{it} = m(x_{it},\theta) + \epsilon_{it},$$

for panel data.

Of less interest are non-parametric methods.

Suggestions for journals to look in are also very welcome.

At the moment I'm reading Amemiya (1983) in the Handbook of Econometrics, but I was hoping to get something perhaps more up to date.

Wooldridge, J.M. (1996) "Estimating systems of equations with different instruments for different equations" in Journal of Econometrics is an example of a contribution later than the above review, hence not included.

The book "Nonlinear Regression Analysis and Its Applications" (2007) by Bates & Watts springs to mind as an immediate suggestion. It is co-authored by one of the masters of regression algorithm design (D. Bates). Note that is is not exactly fresh; the edition I link is published on 2007 but most of the material is from the 1989 edition. That being said, it is definitely authoritative and has aged very well. I used it as a reference book at times and it was very good. Especially when it came to computational aspects it was indispensable. It couples well with the "Mixed-Effects Models in S and S-PLUS" (2000) by Pinheiro & Bates, which is closer to a panel data paradigm of the problem.

Secondary suggestions: Ruppert et al. "Semiparametric Regression" (2003) has less computational focus that B&W but I think it has a broader scope too. Depending on how we define non-linear regression, looking at Generalised Additive Models can be very insightful and to that extent Wood's "Generalized Additive Models: An Introduction with R" (2017; 2nd Ed.) is probably the most up-to-date reference out there, it is a great read. Similarly, if we care more for Local Regression Models, checking Fan & Gijbels "Local Polynomial Modelling and Its Applications" (1996) is definitely a classic too. (I appreciate that these secondary suggestions are moving even further away from the panel data paradigm but I need them to make my next point.)

Comment: One could note that there are fewer non-parametric regression books coming out recently; that's not entirely a coincidence: Machine Learning happened. Putting aside best-in-class general books like: "Elements of Statistical Learning" (2009) by Hastie et al. and "Machine Learning: a Probabilistic Perspective" (2013) by Murphy, looking into Devroye et al. "A Probabilistic Theory of Pattern Recognition" (1997) covers consistency results, bounds, error rates, convergence, etc. in great detail. Therefore there are some review articles on the intersection of Machine Learning and Econometrics like: "Machine Learning: An Applied Econometric Approach" (2017) by Mullainathan & Spiess or "Big Data: New Tricks for Econometrics" (2014) by Varian. They give an OK overview but they do not offer a rigorous mathematical treatment of the matter, they should though offer a reasonable list of references.

• Thx, for the answer. You include a lot of good references on how to apply different types of non-linear models. However I wouldn't exactly call any of them "review articles" they are all books and they seem to focus more on introducing the topic than surveying existing litterature. I'm merely pointing this out for the benefit of future readers. Also perhaps you could confirm that this is also the case for "Nonlinear Regression Analysis and Its Applications", because while this is the most interesting reference you give I have not been able to produce an example. I will accept your answer. – Jesper Hybel Oct 9 at 16:21

Non-linear regression is a mature and broad topic, that's why I doubt that there are many recent review papers. The only papers that I can think of are:

Motulsky HJ, Ransnas LA: "Fitting Curves to Data Using Nonlinear Regression: A Practical and Nonmathematical Review." The FASEB Journal, 1(5), 365-374 <- As the name says, a nonmathematical review so not a good place to look for stuff about consistency and asymptotics.

A. R. Gallant: "Nonlinear Regression" The American Statistician Vol. 29, No. 2 (May, 1975), pp. 73-81 <- Older than the paper you mentioned in the question.

You might find a good overview in some statistics handbooks. For instance in "Handbook of regression methods" by Young or in "Modern regression methods" by Ryan you can find a good chapter about nonlinear regression.

About consistency and asymptotics I can recommend chapter 2 of the book "Statistical tools for nonlinear regression" by Huet et al.

Last but not least, the two classics in the English speaking literature are Bates & Watts as mentioned above and "Nonlinear Regression" from Seber and Wild. Another very good bok is "Nonlinear Statistical Models" by Gallant