Statistical intuition/data sense

Question

I am a second-year undergraduate student, studying Math, and I've been talking to one of my professors a good amount about the difference between mathematical ability and statistical ability. One of the key differences he brought up was "data sense" which he explained as a combination of technical ability while operating within a set of what I'll informally call "common sense restraints" i.e. not losing sight of the reality of the problem amidst a lot of theory. This is an example of what I was talking about, which appeared on Gowers's blog:

In several parts of the UK the police gathered statistics on where road accidents took place, identified accident blackspots, put speed cameras there, and gathered more statistics. There was a definite tendency for the number of accidents at these blackspots to go down after the speed cameras had been installed. Does this show conclusively that speed cameras improve road safety?

The same person who argued for the randomized strategy in the negotiation game basically knew the answer to this question already. He said no, since if you pick out the extreme cases then you would expect them to be less extreme if you run the experiment again. I decided to move on quickly from this question since there wasn’t a lot more to say. But I told people about a plan I had had, which was to do a bogus telepathy experiment. I would get them to guess the outcomes of 20 coin tosses, which I would attempt to beam to them telepathically. I would then pick the three best performers and the three worst, and would toss the coins again, this time asking the best ones to help me beam the answers to the worst ones. People could see easily that the performances would be expected to improve and that it would have nothing to do with telepathy.

What I'm asking is how to learn more about this "data sense", through any publications on the subject, if they exist, or through what other users have found to be helpful in developing this skill. I'm sorry if this question needs clarifying; if so, please post your questions! Thanks.

Answer 1

I would first say that we shouldn't slight mathematics. It is an important tool in the development of statistical theory and statistical methods are justified by theory. Theory also tells you what is wrong and what techniuqes might be better (e.g. more efficient). So I think mathematical knowledge and thinking is important (almost necessary) to be a good statistician. But it is definitely not sufficient. I think the books referenced in comments are good. Let me give some others.

Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining

Making Sense of Data II: A Practical Guide to Data Visualization, Advanced Data Mining Methods, and Applications

Statistical Thinking: Improving Business Performance

The Role of Statistics in Business and Industry

A Career in Statistics: Beyond the Numbers

The books by Hahn and Snee are particularly valuable and interesting because these are famous industrial statisticians with the mathematical skills and the practical experience.

Answer 2

In the example you mention, the core issue is causal inference. A good place to start for causal inference is this triple-book-review by Andrew Gelman, and the books reviewed therein. In addition to learning about causal inference, you should learn about the value of exploratory data analysis, description, and prediction.

I've learned an incredible amount by hearing social scientists criticize each other's research in published work, blogs, seminars, and in personal conversations - there are lots of ways to learn. Follow this site, and Andrew Gelman's blog.

Of course, if you want data-sense, you need practice working with real data. There are general data-sense skills, but there is also data-sense which is specific to a problem area, or even more specifically, data-sense specific to a particular dataset.

Answer 3

A nice, free resource is the Chance News Wiki. It has many examples pulled from real examples along with discussion of good and bad points in how people interpret data and statistics. Often there are discussion questions as well (part of the motivation of the sight is to give teachers of statistics real world examples to discuss with students).

Answer 4

+1 for a great question! (And +1 to all the answerers thus far.)

I think there very much is such a thing as data sense, but I don't think there's anything mystical to it. The analogy I would use is to driving. When you are driving down the road, you just know what is going on with the other cars. For example, you know that the guy in front of you to the side is looking for the street sign where he's supposed to turn, even though he isn't using his turn-signal. You automatically identify the slow, over-cautious driver and anticipate how they'll react in different situations. You can spot the teenager who just wants to race as fast as he can go. You have a recognition-based sense of what all the cars are doing. This is exactly the same as data sense. It comes from experience, lots of experience. If you know enough of the theory, you just need to start playing with real datasets. You might be interested in exploring a site like DASL. One condition though, is that you shouldn't just get experience at loading a dataset, running a test, and getting a p-value. You will need to explore the data, probably plot it different ways, fit some models, and think about what's going on. (Notice that EDA has been a common thread here.)

One possibly non-obvious fact about this process, is that data sense can be localized to a given topical area. For example, you could get a lot of experience working with experimental data and ANOVA's, but not necessarily have a good feel for what's going on when you look at time-series data or survival data.

Let me add one more strategy that I've found enormously helpful: I think it's worth your time to learn a little (statistical) programming. You don't have to be terribly good at it (I'm known for writing "comically inefficient" code). However, once you can write some basic procedural code (say in R), you can simulate. It would be hard for me to overemphasize how much being able to conduct even very simple simulations can help. One thing you can use this for is, when in the course of your studies, you read about some property you can explore it. For instance, if you know (abstractly) that it is difficult to empirically determine whether a logit or a probit model is better for a dataset, you can code up simple simulations of this and play with them to understand the idea more fully. This will also provide you with experience, but of a slightly different type, and will also help you develop your data sense.