7
$\begingroup$

Anyone who reads the questions here on a regular basis quickly recognizes some questions as being from textbooks or exams. Such questions need the self-study tag and one reason for closing a question is not having that tag when it is needed.

That leads to my question: It is often easy to recognize these questions. They are questions that would never come up in the "real world". So, why do books ask them?

$\endgroup$
10
  • 6
    $\begingroup$ One former colleague offered the thought that real datasets often are complicated or compromised in ways beyond the immediate point of the exercise. I was never clear whether he was joking or deadly serious. I've been deadly serious in calling out such poor practice, e.g. in a review that identified 3 real data examples in a 600=page text. onlinelibrary.wiley.com/doi/epdf/10.1111/… $\endgroup$
    – Nick Cox
    Feb 18 at 18:58
  • 4
    $\begingroup$ On the one hand, this can likely only be answered by statistics instructors. On the other hand, @NickCox' first comment seems spot on. If you use a "real" dataset that you spend 80% of your time cleaning, you only have 20% of your time left over for the "actual" analysis. (And yes, people should be taught the importance of data cleaning.) Then again, this only pertains to questions about data analysis. $\endgroup$ Feb 18 at 19:04
  • 5
    $\begingroup$ I think everyone who has ever read a statistics textbook has something to say.... $\endgroup$
    – Nick Cox
    Feb 18 at 21:02
  • 2
    $\begingroup$ To the close voters, yes, this is going to be an opinionated thread. But I disagree to close this for it's a relevant discussion that day in and day out we either tend to snub and overlook or take these for granted. Maybe community wiki, I guess. $\endgroup$ Feb 19 at 4:29
  • 3
    $\begingroup$ Instead of closing, this should be a community question $\endgroup$ Feb 19 at 8:49

4 Answers 4

26
$\begingroup$

Textbooks exist to teach complex topics to people who are unfamiliar with them. To help someone grasp a difficult concept, it helps to first introduce it in a simple context. So a question from a textbook will often focus on the one concept it is trying to teach and simplify everything else. This is how good pedagogy works, regardless of discipline. To teach someone physics we start by pretending that things like friction and air resistance don't exist, and then return to those concepts later once the student understands the more basic ideas.

This may seem obvious, but in my opinion this is something we should keep in mind when helping people learn stats (either on this site or in classrooms). Often I see a student (or OP) ask a very basic question, which makes it clear that the person only have a very limited grasp of statistics. Sometimes the answers that are provided (and upvoted) are extremely technical, with lots of equations and discussions of very deep concepts that the person asking the question is obviously going to have trouble with. These answers - even when they are 100% correct - may actually be less helpful to the person asking the question than an answer that is technically "wrong" in some highly technical way, but which gets the basic point across in a way that the person asking the question will have a much easier time with. Then at the end of the answer, we can say "now, what I just said was technically an oversimplification, see here for more."

I've seen a lot of people who quit stats because it was taught to them badly. They were having trouble with the basics but their professor was allergic to any form of simplification that might help them develop an intuition for what was going on. This is something I always try to keep in mind when teaching, and it's relevant here as well.

$\endgroup$
2
  • 2
    $\begingroup$ “All models are wrong but some are useful” - George Cox. “Know the rules like a professional so you can break them like an artist” - Pablo Picasso… Pros know which assumptions can be violated and when. But to beginners it’s all incoherent dogma. $\endgroup$
    – jbuddy_13
    Feb 19 at 0:02
  • 6
    $\begingroup$ @jbuddy_13 - correct attribution is George Box $\endgroup$
    – Mari153
    Feb 19 at 3:36
11
$\begingroup$

I don't think this is unique to statistics.

It's common in a variety of areas that make use of mathematical models. One may well learn to solve a problem that begins with "Consider a spherical cow ..." some time in advance of learning when that might be close enough for present purposes. Statistics adds that modelling the sources of uncertainty can be a major part of the problem, and the realistic solution may require quite a lot of knowledge; you want people to be able to practice the more basic skills along the way.

However many people haven't read any text that uses applied mathematical models (which may be rough approximations) other than in a stats book since they were in their early teens, if ever. Simplifying assumptions are often used on setting up mathematical models to yield tractable problems where a more realistic problem may rely on having a very wide range of knowledge.

The skills required to choose a suitable model in a real-world situation require broader and to some extent deeper knowledge than the skills required to apply a model that is already chosen. You have to understand (in the context of the sorts of answers being sought) what things may be abstracted and which are more essential. There's often an interplay of different considerations, beyond the relatively simple plug and play of an artificial problem.

As a result, when teaching the simpler skills, the models are pre-chosen and the ignored aspects are "pre-ignored" even when the situation is based in a real one, often leading to obvious artifice in the phrasing.

However, some books don't include any real problems at all, and I think that is partly due to the considerable extra effort involved in including them. I think many books could try to include some actually-real problems and help guide students through the steps involved. There are some books that attempt to do this.

I think artifice is particularly common with probability problems, and to an extent that may make more sebse; in some sense it's more axiomatic - pure mathematics - than the statistical work that employs it, which is nearer to applied epistemology and where artifice may be less reasonable in a real 'product' of analysis

$\endgroup$
8
$\begingroup$

I generally agree with what has already been written in the other answers and comments, so instead of echoing what has been said, here is my hot take (which is too long to be a comment).

I recall from writing Actually Modelling the Data I came across this lovely quote:

Quotation (as cited in Sundberg 1994) Most real-life statistical problems have one or more nonstandard features. There are no routine statistical questions, only questionable statistical routines.

This quote is attributed to J.M. Hammersley in a footnote of Sundberg 1994. It is also attributed to D.R. Cox here where I found the same link to Sundberg’s paper.

What I've learned from doing data analysis myself, learning from McElreath's Statistical Rethinking lectures, and reading Bayesian Workflow, is that quote is correct. Many novices are not aware of how much iteration can be involved in a seemingly-simple modelling task. It is routine for professionals to have to learn more about their problem as they attempt to solve it, so transferable processes and principles are often more valuable than a flow chart of simple recipes. Recipes can show what analyses can look like, but to me this is just a staging area for a more principled approach.

I also find that many of the results I see proven in textbooks are not directly applicable. Try doing statistical forecasts of non-stationary processes that feed into client-customized discrete event simulations doing both simulation optimization and counterfactual causal inference and you're likely to see what I mean. The mathematical formulations of such problems often lead to extremely difficult change of variables. And yet I find that continuing to learn the math, even if not directly-applicable, teaches me lessons about the cases I cannot directly deal with in pure mathematics.

Statistics, like other complex skill sets, needs to cover some fundamentals. For me it was studying more mathematics and learning Monte Carlo methods (believe me, the stuff can drive you NUTS!) that gave me the confidence to go beyond the simple recipes.

The best textbooks build up our repertoire of fundamentals and tools. The worst teach us to mindlessly follow flow charts leading to erroneous conclusions.

$\endgroup$
3
  • 2
    $\begingroup$ Coincidentally, I have started reading Statistical Rethinking's The Golem of Prague and it rightfully articulates the present plight reflected in the words of Peter in the post. The classical textbooks, consciously or unconsciously, try to discern themselves from data analysis pov and as a result we get a mathematical book and not an "engineering" one, as McElreath lamented. $\endgroup$ Feb 19 at 4:37
  • $\begingroup$ @User1865345 It is an interesting perspective. I believe McElreath got the idea from The Golem: What You Should Know About Science, although I have not read it yet. $\endgroup$
    – Galen
    Feb 19 at 4:44
  • $\begingroup$ I believe I show such a flow chart, which I got from McElreath's materials, in Wilcoxon's Heuristic. He's not a big fan of them either. $\endgroup$
    – Galen
    Feb 19 at 4:50
6
$\begingroup$

A good friend of mine calls this tendency of textbooks to have unrealistic examples 'frictionless statistics'. The phrase comes from high school physics books that start every problem with something like "Assuming that there is no friction...". (@Glen_b's spherical cow is equivalent and maybe more amusing.)

The answer to the titular question seems obvious: it makes things easy for the textbook author and for the instructors who might choose the book as a course text.

I agree strongly with @GrahamWright's second paragraph. I think that one reason for that is that the most fruitful application of statistics to real-world problems requires more understanding of the particulars of the problem, the inferential objectives, loss functions relating to incorrect inferences, and probably many other contextual factors. The questions are mostly posted without any of those details —perhaps because those details are routinely ignored in textbooks— but there is nothing to prevent the answerers from asking for them, or writing about their possible consequences.

A textbook that included all of the context and considerations needed to form scientific inferences about the real world assisted by statistical analyses would be a rich and interesting read. However, it would not have space to cover the many topics that are considered to be essential for an introductory statistics course. In other words, such a book would fail to check a sufficient number of recipe checkboxes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.