Is there a place where one can get standard example data for various statistics tools to try on? For example, if one is learning about ARIMA models, where would one get data that can be modelled well with an ARIMA(2,1,2)x(1,1,1)12 model? Or data for a (5,1,1)x(0,0,0) model? Or multiple linear regression etc? I'm looking for something similar to this:
The U.S. National Institute of Standards and Technology (NIST) has a database online at https://www.itl.nist.gov/div898/strd/nls/nls_main.shtml with both test data sets and associated non-linear regression equations for use in the design and testing of non-linear regression software. Each has example fitted parameters for verification, and there are different levels of difficulty listed. There are also two sets of initial parameter values for for each equation, a "near" and a "far", which are near and far from the reference fitted values for the purpose of testing convergence from different starting points.
The creators of this online regression testing database are aware that in testing, final fitted parameters will not be exactly equal to the listed values and they state that results to within four or five decimal places of their listed values are sufficient.
I personally found these to be of immense value in the design and testing of of my pyeq3 Python fitting library, which is the core of my zunzun.com curve and surface fitting web site. These test equations and associated data sets are what gave me the (statistical) confidence I needed to put the site on the internet in the first place.
There are several different places you could try, but I'll highlight two.
Help Documentation in Statistical Packages
One potentially great and easy-to-access resource is the help documentation in various vignettes in R packages. Often times the vignettes/documentation come with built in datasets meant to facilitate learning about the procedures contained in the R package.
For example the R package
twang is for the development of boosted propensity scores. If you install the package, open it, and then type the name of a dataset associated with the package inside the "data" function call, you'll be able to see the data:
install.packages('twang') library(twang) data(lalonde) > head(lalonde) treat age educ black hispan married nodegree re74 re75 re78 1 1 37 11 1 0 1 1 0 0 9930.0460 2 1 22 9 0 1 0 1 0 0 3595.8940 3 1 30 12 1 0 0 0 0 0 24909.4500 4 1 27 11 1 0 0 1 0 0 7506.1460 5 1 33 8 1 0 0 1 0 0 289.7899 6 1 22 9 1 0 0 1 0 0 4056.4940
How do you get the name of the dataset (in this case, it's called
lalonde)? If you want to see all the datasets in a package, simply type:
packagename is the name of the package in which you want to look for datasets. So for the
twang example, typing:
try(data(package = "twang") )
opens a prompt that contains the following data sets in package ‘twang’:
Data sets in package ‘twang’: AOD Subset of Alcohol and Other Drug treatment data egsingle US Sustaining Effects study iptwExLong Example data for iptw function (long version) iptwExWide Example data for iptw function (wide version) lalonde Lalonde's National Supported Work Demonstration data lindner Lindner Center data on 996 PCI patients analyzed by Kereiakes et al. (2000) mnIptwExLong Example data for iptw function (long version, more than two treatments). mnIptwExWide Example data for iptw function (wide version, more than two treatments) raceprofiling Traffic stop data
If you wanted to see the datasets relevant for learning the
rpart package, you'd type:
try(data(package = "rpart") )
and you'd get:
Data sets in package ‘rpart’: car.test.frame Automobile Data from 'Consumer Reports' 1990 car90 Automobile Data from 'Consumer Reports' 1990 cu.summary Automobile Data from 'Consumer Reports' 1990 kyphosis Data on Children who have had Corrective Spinal Surgery solder Soldering of Components on Printed-Circuit Boards stagec Stage C Prostate Cancer
If you then want to obtain a more detailed description of the dataset and its contents, you can simply type the name of the dataset inside the
help() function call. So typing:
essentially launches your browser and opens up a description like the one here: https://rdrr.io/cran/cobalt/man/lalonde.html
Another great resource for finding datasets relevant to specific analyses is the UCLA's Institute for Digital Research & Education website (IDRE). The site guides users through different types of analyses and contains hyperlinks directly to datasets relevant to the analyses.
For example, if you wanted to obtain a SAS dataset relevant to Poisson Regression you could click the SAS hyperlink on the IDRE link (see above) and you'll see in the tutorial, a hyperlink to a SAS datset for example 3.
A good way to access data is to actually simulate data for a number of models to learn how to identify latent structure AND then introduce pulses into the data and learn how to identify data like this. Then introduce level/step shifts and/or local time trends and then learn how to identify data like this. Then simulate data where deterministic seasonal pulses are embedded in the data and learn how to identify data like this.
Then simulate data where the model parameters change over time and learn how to identify data like this.
Then simulate data where the error variance changes over time and learn how to identify data like this. Deterministic Error variance change can be detected following TSAY http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html while linkage between the expected value and the error process is remedied by Box_Cox When (and why) should you take the log of a distribution (of numbers)?
Extend this univariate discussion to include exogenous factors that not only are important contemporaneously but with lag effects .
That is what I would do and continuously do to motivate learning by data when no theory is pre-existent and to create robotic solutions to aid analyses.
In all of the above you have the knowledge of how the data was actually constructed and you can learn about the strengths and weaknesses of alternative identification strategies