The way some people learn statistics feels (in some sense to me) overly method oriented.
The problem is not the teaching material itself. The teaching material usually contains good illustrative examples of when to apply a method and what can go wrong (often overlooked by students). I take this from the feedback that I get when discussing statistics with people that have taken a basic stats course or maybe something more advanced.
People that are going through a basics stats course usually go over quite a number of new methods and techniques that may be very foreign to them. The way they learn/consume this new information is often similar to the way I feel mathematics at a lower level is learnt by many people. The students emphasize a specific type of problem and learn how exactly to apply a method to it. When the problem changes slightly, they often have no idea on how to attack it, although the method they are using can generalize to this change trivially.
This way of thinking that some people tend to have is likely not because the way things are taught in the course, but rather the mindset that the students are in with regards to solving mathematical problems.
Maybe the underlying issue is more fundamental than what I thought to begin with.
This all leads to the misconception that I have observed too often:
My model doesn't work well, so I need to try another model.
What the person should often in fact be doing is taking a step back and think more in depth about their data and what is underlying their problem. E.g. ask themselves the following questions/or do the following:
- How do my residuals look? Do they contain a trend?
- Plot the residuals.
- Plot the variables against each other and the response.
- Do I need all these variables? Are some of them redundant?
So my point is that often I feel that students are too focused on learning how to solve specific problems and they do not learn well to generalize, or question what is fundamentally wrong when they do their analysis.
I want to find examples to motivate people further to question what is fundamentally the problem, rather than trying another model/method. I am especially thinking about predictive models of continuous variables, especially linear models.
One example that I feel students should be exposed to when learning about linear regression is Anscombe's quartet. Which clearly shows the importance of plotting your data.
Another typical example is to generate points from a parabola with noise and fit a linear model to it. Then ask the students to observe the trend in the residuals.
What more examples could create common pitfalls that students should fall into and learn to deal with? These should be questions that most students get wrong and they should learn by failure.