I realize this is subjective, but I thought it would be nice to talk about our favorite datasets and what we think makes them interesting. There is a wealth of data out there, and what with all of the APIs (e.g., Datamob) along with classic datasets (e.g., R data), I think this could have some very interesting responses.

For example, I have always liked datasets like the "Boston Housing" dataset (unfortunate implications notwithstanding) and "mtcars" for their versatility. From a pedagogical standpoint, one can show the merits of a wide variety of statistical techniques using them; and Anderson/Fisher's iris dataset will always have a place in my heart.

Thoughts?

  • 2
    The one being used for my dissertation, because it'll land me a doctorate if I analyze it right >.> – Fomite Jan 22 '12 at 5:37
  • 3
    Welcome to Cross Validated! This is designed as a Q&A site for questions with real answers, not a discussion forum. As such, I do not believe this is the type of question we want on this site. Please see the FAQ. – Michael McGowan Jan 22 '12 at 5:40
  • 4
    I am aware it is designed as a Q&A, but with questions like "What is your favorite statistics cartoon?" being voted up highly, I figured this wouldn't be terribly inappropriate. Especially pedagogically, if someone is trying to learn about data analysis and exploratory techniques, it might be useful to get some feedback on public datasets which offer a rich structure and have a large amount of history and research behind them. – D.A. Jan 22 '12 at 6:03
  • 4
    I'm inclined to let the community decide on whether this has to be closed (as not constructive) or not, although I would add that thorough and argumentative replies might well serve as a support for future questions on particular aspects of data analysis. I'm converting this to CW in the meantime because, obviously, there's no single best answer. – chl Jan 22 '12 at 10:34
  • 2
    This question and their answers are very useful to me. Please do not remove. – dsign Jan 23 '12 at 14:01

The low birth weight study

This is one of the datasets in Hosmer and Lemeshow's textbook on Applied Logistic Regression (2000, Wiley, 2nd ed.). The goal of this prospective study was to identify risk factors associated with giving birth to a low birth weight baby (weighing less than 2,500 grams). Data were collected on 189 women, 59 of which had low birth weight babies and 130 of which had normal birth weight babies. Four variables which were thought to be of importance were age, weight of the subject at her last menstrual period, race, and the number of physician visits during the first trimester of pregnancy.

It is available in R as data(birthwt, package="MASS") or in Stata with webuse lbw. A text version appears here: lowbwt.dat (description). Of note, there are several versions of this dataset because it was extended to a case-control study (1-1 or 1-3, matched on age), as illustrated by Hosmer and Lemeshow in ALR chapter 7.

I used to teach introductory courses based on this dataset for the following reasons:

  • It is interesting from an historical and epidemiological perspective (data were collected in 1986); no prior background in medicine or statistics is required to understand the main ideas and what questions can be asked from that study.
  • Several variables of mixed types (continuous, ordinal, and nominal) are available which makes it easy to present basic association tests (t-test, ANOVA, $\chi^2$-test for two-way tables, odds-ratio, Cochrane and Armitage trend test, etc.). Morever, birth weight is available as a continuous measure as well as a binary indicator (above or below 2.5 kg): We can start building simple linear models, followed by multiple regression (with predictors of interest selected from prior exploratory analysis), and then switch to GLM (logistic regression), possibly discussing the choice of a cutoff.
  • It allows to discuss different modeling perspectives (explanatory or predictive approaches), and the implication of the sampling scheme when developing models (stratification/matched cases).

Other points that can be emphasized, depending on the audience and level of expertise with statistical software, or statistics in general.

  1. As for the dataset available in R, categorical predictors are scored as integers (e.g., for mother's ethnicity we have ‘1’ = white, ‘2’ = black, ‘3’ = other), notwithstanding the fact that natural ordering for some predictors (e.g., number of previous premature labors or number of physician visits) or the use of explicit labels (it is always a good idea to use 'yes'/'no' instead of 1/0 for binary variables, even if that doesn't change anything in the design matrix!) are simply absent. As such, it is easy to discuss what issues may be raised by ignoring levels or units of measurement in data analysis.

  2. Variables of mixed types are interesting when it comes to do some exploratory analysis and discuss what kind of graphical displays are appropriate for summarizing univariate, bivariate or trivariate relationships. Likewise, producing nice summary tables, and more generally reporting, is another interesting aspect of this dataset (but the Hmisc::summary.formula command makes it so easy under R).

  3. Hosmer and Lemeshow reported that actual data were modified to protect subject confidentiality (p. 25). It might be interesting to discuss data confidentiality issues, as was done in one of our earlier Journal Club, but see its transcript. (I must admit I never go into much details with that.)

  4. It is easy to introduce some missing values or erroneous values (which are common issues in real life of a statistician), which lead to discuss (a) their detection through codebook (Hmisc::describe or Stata's codebook) or exploratory graphics (always plot your data first!), and (b) possible remedial (data imputation, listwise deletion or pairwise measure of association, etc.).

  • +1 Thank you for providing an exemplary answer that shows this thread can be useful and provides a standard of exposition that other replies can (and should) aim for. – whuber Jan 24 '12 at 15:42
  • This is fantastic and exactly what I was looking for with asking the question. I thank you for you valued insight. – D.A. Jan 25 '12 at 14:23

Of course, the Anscombe 4 datasets are very good for teaching - they look very different, yet have identical simple statistical properties.

I also suggest KDD Cup datasets http://www.kdd.org/kddcup/ because they have been well studied and there are many solutions, so students can compare their results and see how they rank.

In my data mining course I provided a Microarray dataset competition which can be used by professors http://www.kdnuggets.com/data_mining_course/

  • For other data sets that have been designed for a pedagogical purpose in a similar way to the Anscombe Quartet, see this question. – Silverfish Jan 10 '15 at 11:29

A lot of my Statistical Analysis courses at Cal Poly have used the "Iris" dataset which in already in R. It has categorical variables, and highly correlated variables.

  • Would you mind expanding your last points: How do this dataset help teaching statistics? (AFAICT, the iris dataset only has one categorical variable, namely iris class.) – chl Nov 9 '13 at 22:26
  • Here is a thread concerned entirely with the use of the Iris dataset in teaching. – Silverfish Jan 10 '15 at 11:41

The Titanic dataset used by Harrell in "Regression Modeling Strategies". I use a simplified version of his analysis when explaining logistic regression, explaining survival using sex, class and age.

The Loyn dataset discussed in "Experimental Design and Data Analysis for Biologists" by Gerry Quinn and Mick Keough contains nice problems requiring transformation for multiple linear regression.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.