# Artificial dataset generator for classification data

I would like to generate some artificial data to evaluate an algorithm for classification (the algorithm induces a model that predicts posterior probabilities).

These are some basic properties of the dataset:

• Features have to be continuous
• Response variable is dichotomous (either 0 or 1)

I would like to test whether the algorithm can cope with:

• Many feature / high dimensional problems
• noise (it can drop features)
• Multi-modality
• ??? (how do I simulate correlation etc.)

I intend to implement the algorithm in R or Matlab. I can sample from multivariate normal distributions and specify a covariance matrix.

I would appreciate any feedback.

• Have you looked into using a real-world pre-existing dataset? There are many advantages to using real-world data instead of synthetic data. I can immediately think of at least one type of problem that follows your criteria: vision classification. For example, in the MNIST dataset the features are pixel values and the response variable is a handwritten digit ('0'-'9'). edit: Just saw that you want binary classification, in the case of MNIST you could subset the data, for example try to classify zeros vs non-zeros. Jan 23, 2012 at 15:05
• Sorry I disaggree. As I produce probabilities an artifical generator would also allow me to test the calibration. This would (IMHO) be impossible using real-world datasets. It would also be hard to see under which conditions (e.g. above 10 higly correlated features) the algorithm starts to fail ... etc. Jan 23, 2012 at 15:20
You can generate a set like this starting with mlbench.xor (or mlbench.hypercube, might be easier) form mlbench package, then you combine classes it generated into two groups to make the dichotomous response and add new attributes to increase dimensionality -- some being random linear combinations of the original ones, some being just random noise.