# Quick test of quality of an econometrics textbook

When encountering an econometrics textbook for the first time, you might wish to assess its quality. What is the first thing you check? Is there a certain topic or a definition you examine, knowing that poorer textbooks tend to get it wrong?

I mostly check if it has a 21th century approach or a 20th century approach. Some indications include

• does it pay inordinate attention to topics that had their day, like the Durbin-Watson test, simultaneous equations etc.
• further red flags include too much attention being paid to fixed regressors, exact finite-sample results in toy settings, no attention to things that are routine nowadays like Eicker-White standard errors
• somewhat similarly, does it mechanically go through "violations of the classical assumptions" in the sense of "if you find heteroskedasticity, do GLS", "if you find serial correlation, do Cochrane-Orcutt" etc. - there is nothing wrong with any of these techniques, but it is rare in practice to have that these issues occur in isolation. (To be fair, it is much easier to explain what not to do than what to do.)
• does it pay attention to numerical implementation, i.e., how to actually carry out an empirical analysis using software, preferably one that allows for easy reproducibility, such as R or Stata?
• especially in undergraduate level texts, does it include interesting empirical applications rather than artificial samples which can be fed into a calculator to illustrate computation of some statistic by hand
• last but not least, is it typeset properly?
• Thank you for a helpful answer! Could you expand just a little bit on mechanically go through "violations of the classical assumptions" and pay attention to numerical implementation? Nov 28, 2019 at 12:19
• I made an edit! Nov 28, 2019 at 12:24
• Thank you! Another example could be how the authors treat causation and whether they use causal and noncausal language appropriately (Chen & Pearl "Regression and Causation: A Critical Examination of Six Econometrics Textbooks" (2013) suggests hardly any of them do). In addition to modern vs. outdated ideas, there is also quality of execution. Typesetting is one facet of that. Another is whether the definitions are correct and precise or sloppy. Do you look at something like that? Nov 28, 2019 at 12:40
• Indeed, that is of course a crucial aspect, that some authors get much better than others. I indirectly had something like that in mind with fixed regressors, where asking questions about correct specification of the type $E(u|X)=0$ is almost vacuous. As for Pearl and coauthors, well, my very humble and irrelevant personal impression is that they are more transparent when it comes to criticizing others than at clarifying what they themselves precisely and concretely put forward. Nov 28, 2019 at 13:17
• Surely DAGs are informative visualizations, but I do not understand (with maybe the exception of the front door criterion) where an analysis of the type advocated (to the extent that ever becomes transparent, at least to amateurs like me) would ultimately really amount to something else than, say, an IV analysis. To the extent that is correct, if econometric types of causal analyses (of which my impression is we sometimes rather see caricatures in such references) are faulty, then wouldn't DGAs be so, too? Moreover, how he thinks of inference (tests, CIs) is unclear to me. Nov 28, 2019 at 13:18

Interesting question.

And I agree with every point of Christoph's answer.

• a good textbook should have a clear audience. I personally like, for example, the textbook Econometric Analysis of Cross Section and Panel Data by Jeffrey Wooldridge. However, I use it to refresh something I already knew, or to really understand certain topics (e.g. as preparation for second or third year PhD exams) but I would not recommend this textbook beginner-level or intermediate econometrics courses.

• I like it if authors use a chapter to describe a sub-field they are familiar with even if that is not discussed everywhere (I will add an example here, currently thinking). If the manage not to get lost in details, these experts can often give intriguing insights which you do not find for every day topics.

• A good textbook should use definitions which are up-to-date and widely used. As Christoph wrote, please do not use fixed regressors and confuse students why you later suddenly change the "world" (by moving from some "fixed" concepts to random ones). Also please do not use yet another definition of what is a fixed vs a random-effect.

• Since much of the literature has become much more applied in recent decades, I think it is important to differentiate between theoretical approximations (asymptotics) and real-world behavior of estimators. Mostly Harmess Econometrics by Joshua Angrist and Jörn-Steffen Pischke could be here a good example which wrote, for example, that the difference between logit/probit and OLS often hardly matters in real-life (I hope I remember this correctly). This is an experience I also have made which can save you a lot of time! At the same time, please do not ignore theoretical aspects at all (see next point).

• Since I left academia, I nowadays read more machine learning related textbooks (if I find the time to still look into textbooks). Here, my small sample impression is that the quality of the top textbooks is extremely high. And often you can read the textbooks online for free, this is amazing (some econometric textbooks are also available on the web but I never know whether legally or not so I have not linked them here)! Nobody has to make his or her book available for free, but students love free books. Make them aware of free good literature! Examples are Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, An Introduction to Statistical Learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani or The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman. Although I personally think that these books should cover more theory (perhaps take a look at econometrics?!), there is much to learn from this field with respect to visualization, writing style, quality of coding etc.!

• The interestingness of the examples given in the textbook should be high! I personally do not like toy examples, or (outdated) datasets I already have seen again and again. It is much more interesting for the students if you have data which is still relevant.

• Good textbooks should take time to explain difficult concepts such as the incidental parameter problem (and what it means for linear and non-linear models) or avoid such terms.

• Last but not least, answer the question students ask again and again. A few examples

Some other thoughts:

• Give some historical context, be critical about what is hyped currently. Tell students, for example, why a nobel laureates such as Angus Deaton or James Heckman are critical about the reduced-form approaches. While of course also saying that are the benefits/alternatives.

• If you do not want to talk about the history. Then perhaps briefly mention more "novel" topics relevant for econometrics such as version-control systems and reproducible codes or how to deal with big data for which classical econometric estimators are often not suited?

• As Christoph wrote: some topics are at one point in fashion (such as the mentioned simultaneous equations) but play hardly any role anymore a decade later. However some authors should still aim at giving a broad overview about the field (Econometric Analyses by William Greene or Microeconometrics: Methods and Applications by Colin Cameron and Pravin Trivedi would be two examples). There is a lot of confusion about the need to use simultaneous equations models (with respect to consistency, bias and efficiency). Or why the Heckman correction (or other early "quasi causal" models) are not so popular anymore. So sometimes I find it nice to mention such topics on 1-2 pages and perhaps telling the reader why this topic is not so important anymore.

• Thanks for your answer! It seems you focus more on what a good textbook should be like but not so much on how to quickly check its quality. I am looking for a quick test, something that would allow me dismiss a poor textbook without spending too much time evaluating it. Dec 19, 2019 at 8:23