18
$\begingroup$

I realize this is subjective, but I thought it would be nice to talk about our favorite datasets and what we think makes them interesting. There is a wealth of data out there, and what with all of the APIs (e.g., Datamob) along with classic datasets (e.g., R data), I think this could have some very interesting responses.

For example, I have always liked datasets like the "Boston Housing" dataset (unfortunate implications notwithstanding) and "mtcars" for their versatility. From a pedagogical standpoint, one can show the merits of a wide variety of statistical techniques using them; and Anderson/Fisher's iris dataset will always have a place in my heart.

Thoughts?

$\endgroup$
6
  • 5
    $\begingroup$ I am aware it is designed as a Q&A, but with questions like "What is your favorite statistics cartoon?" being voted up highly, I figured this wouldn't be terribly inappropriate. Especially pedagogically, if someone is trying to learn about data analysis and exploratory techniques, it might be useful to get some feedback on public datasets which offer a rich structure and have a large amount of history and research behind them. $\endgroup$
    – D.A.
    Jan 22, 2012 at 6:03
  • 5
    $\begingroup$ I'm inclined to let the community decide on whether this has to be closed (as not constructive) or not, although I would add that thorough and argumentative replies might well serve as a support for future questions on particular aspects of data analysis. I'm converting this to CW in the meantime because, obviously, there's no single best answer. $\endgroup$
    – chl
    Jan 22, 2012 at 10:34
  • 1
    $\begingroup$ (+1) chl's edit is dead on. This question can serve as an idea shop for presentations. Going even further, the best dataset in my opinion would be one, which could be used as a common theme for assignments in the course of a lecture. $\endgroup$
    – steffen
    Jan 22, 2012 at 10:49
  • 3
    $\begingroup$ This question and their answers are very useful to me. Please do not remove. $\endgroup$
    – dsign
    Jan 23, 2012 at 14:01
  • 1
    $\begingroup$ I have added a comment to the statistics jokes thread explaining why it exists and why it does not serve to justify threads like this one here. I find the present question far too ill-formed and vague to serve a useful role or garner consistently strong replies; I wish it would be modified or removed. However, I agree with the previous comment by @chl and therefore have not voted to close, because my vote as moderator would (unfortunately) be unilaterally binding and therefore not reflect community sentiment. $\endgroup$
    – whuber
    Jan 24, 2012 at 15:40

4 Answers 4

13
$\begingroup$

The low birth weight study

This is one of the datasets in Hosmer and Lemeshow's textbook on Applied Logistic Regression (2000, Wiley, 2nd ed.). The goal of this prospective study was to identify risk factors associated with giving birth to a low birth weight baby (weighing less than 2,500 grams). Data were collected on 189 women, 59 of which had low birth weight babies and 130 of which had normal birth weight babies. Four variables which were thought to be of importance were age, weight of the subject at her last menstrual period, race, and the number of physician visits during the first trimester of pregnancy.

It is available in R as data(birthwt, package="MASS") or in Stata with webuse lbw. A text version appears here: lowbwt.dat (description). Of note, there are several versions of this dataset because it was extended to a case-control study (1-1 or 1-3, matched on age), as illustrated by Hosmer and Lemeshow in ALR chapter 7.

I used to teach introductory courses based on this dataset for the following reasons:

  • It is interesting from an historical and epidemiological perspective (data were collected in 1986); no prior background in medicine or statistics is required to understand the main ideas and what questions can be asked from that study.
  • Several variables of mixed types (continuous, ordinal, and nominal) are available which makes it easy to present basic association tests (t-test, ANOVA, $\chi^2$-test for two-way tables, odds-ratio, Cochrane and Armitage trend test, etc.). Morever, birth weight is available as a continuous measure as well as a binary indicator (above or below 2.5 kg): We can start building simple linear models, followed by multiple regression (with predictors of interest selected from prior exploratory analysis), and then switch to GLM (logistic regression), possibly discussing the choice of a cutoff.
  • It allows to discuss different modeling perspectives (explanatory or predictive approaches), and the implication of the sampling scheme when developing models (stratification/matched cases).

Other points that can be emphasized, depending on the audience and level of expertise with statistical software, or statistics in general.

  1. As for the dataset available in R, categorical predictors are scored as integers (e.g., for mother's ethnicity we have ‘1’ = white, ‘2’ = black, ‘3’ = other), notwithstanding the fact that natural ordering for some predictors (e.g., number of previous premature labors or number of physician visits) or the use of explicit labels (it is always a good idea to use 'yes'/'no' instead of 1/0 for binary variables, even if that doesn't change anything in the design matrix!) are simply absent. As such, it is easy to discuss what issues may be raised by ignoring levels or units of measurement in data analysis.

  2. Variables of mixed types are interesting when it comes to do some exploratory analysis and discuss what kind of graphical displays are appropriate for summarizing univariate, bivariate or trivariate relationships. Likewise, producing nice summary tables, and more generally reporting, is another interesting aspect of this dataset (but the Hmisc::summary.formula command makes it so easy under R).

  3. Hosmer and Lemeshow reported that actual data were modified to protect subject confidentiality (p. 25). It might be interesting to discuss data confidentiality issues, as was done in one of our earlier Journal Club, but see its transcript. (I must admit I never go into much details with that.)

  4. It is easy to introduce some missing values or erroneous values (which are common issues in real life of a statistician), which lead to discuss (a) their detection through codebook (Hmisc::describe or Stata's codebook) or exploratory graphics (always plot your data first!), and (b) possible remedial (data imputation, listwise deletion or pairwise measure of association, etc.).

$\endgroup$
2
  • 1
    $\begingroup$ +1 Thank you for providing an exemplary answer that shows this thread can be useful and provides a standard of exposition that other replies can (and should) aim for. $\endgroup$
    – whuber
    Jan 24, 2012 at 15:42
  • $\begingroup$ This is fantastic and exactly what I was looking for with asking the question. I thank you for you valued insight. $\endgroup$
    – D.A.
    Jan 25, 2012 at 14:23
5
$\begingroup$

Of course, the Anscombe 4 datasets are very good for teaching - they look very different, yet have identical simple statistical properties.

I also suggest KDD Cup datasets http://www.kdd.org/kddcup/ because they have been well studied and there are many solutions, so students can compare their results and see how they rank.

In my data mining course I provided a Microarray dataset competition which can be used by professors http://www.kdnuggets.com/data_mining_course/

$\endgroup$
1
  • $\begingroup$ For other data sets that have been designed for a pedagogical purpose in a similar way to the Anscombe Quartet, see this question. $\endgroup$
    – Silverfish
    Jan 10, 2015 at 11:29
3
$\begingroup$

A lot of my Statistical Analysis courses at Cal Poly have used the "Iris" dataset which in already in R. It has categorical variables, and highly correlated variables.

$\endgroup$
2
  • 1
    $\begingroup$ Would you mind expanding your last points: How do this dataset help teaching statistics? (AFAICT, the iris dataset only has one categorical variable, namely iris class.) $\endgroup$
    – chl
    Nov 9, 2013 at 22:26
  • 1
    $\begingroup$ Here is a thread concerned entirely with the use of the Iris dataset in teaching. $\endgroup$
    – Silverfish
    Jan 10, 2015 at 11:41
3
$\begingroup$

The Titanic dataset used by Harrell in "Regression Modeling Strategies". I use a simplified version of his analysis when explaining logistic regression, explaining survival using sex, class and age.

The Loyn dataset discussed in "Experimental Design and Data Analysis for Biologists" by Gerry Quinn and Mick Keough contains nice problems requiring transformation for multiple linear regression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.