Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

Does anyone know of a large data set (upwards of $10^7$ rows, but I'll take $10^5$ as well) that would be appropriate for testing kernel logistic regression? Continuous variables, 2 to 50 independent variables? Would prefer business like data as aposed to scientific (weather, chemical, bio ...).

share|improve this question
What is your objective? To validate an implementation? Just to play around and familiarize yourself with the methodology? Or... – jbowman Mar 28 '12 at 19:43
Mainly to validate an implementation and benchmark. – csta Mar 28 '12 at 20:07
4  
In that case, you could always construct an artificial dataset, e.g., in R, and dump it out into a file with the appropriate structure. The advantage that has over a "natural" dataset is that you know the true model, so validation of the code is easy. – jbowman Mar 28 '12 at 21:00

closed as off topic by whuber Oct 3 '12 at 13:48

Questions on Cross Validated are expected to relate to statistics within the scope defined in the FAQ. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about closed questions here.

1 Answer

up vote 1 down vote accepted

Have a look at some of the data sets on the Elements of Statistical Learning textbook website. They might be on the small side (most have between $10^2$ to $10^5$ observations, but it's a faily varied selection of datasets.

Specifically, you might want to look at:

  • Marketing (may be a good business dataset)

    A total of N=9409 questionnaires containg 502 questions were filled out by shopping mall customers in the San Francisco Bay area.

    The dataset income.data is an extract from this survey. It consists of 14 demographic attributes. The dataset is a good mixture of categorical and continuos variables with a lot of missing data. This is characteristic for data mining applications.

    The goal is to predict the Anual Income of Household from the other 13 demographics attributes.

  • Zip Code OCR (sorry, can't post more than two hyperlinks)

    Normalized handwritten digits, automatically scanned from envelopes by the U.S. Postal Service. The original scanned digits are binary and of different sizes and orientations; the images here have been deslanted and size normalized, resulting in 16 x 16 grayscale images (Le Cun et al., 1990).

    The data are in two gzipped files, and each line consists of the digit id (0-9) followed by the 256 grayscale values.

    There are 7291 training observations and 2007 test observations, distributed as follows:

             0    1   2   3   4   5   6   7   8   9 Total
    Train 1194 1005 731 658 652 556 664 645 542 644 7291
    Test   359  264 198 166 200 160 170 147 166 177 2007
    
share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.