Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

As context: When working with a very large data set, I am sometimes asked if we can create a synthetic data set where we "know" the relationship between predictors and the response variable, or relationships among predictors.

Over the years, I seem to encounter either one-off synthetic data sets, which look like they were cooked up in an ad hoc manner, or more structured data sets that seem especially favorable for the researcher's proposed modeling method.

I believe that I'm over looking standard methods for creating synthetic data sets. Although bootstrap resampling is one common method for creating synthetic data set, it doesn't satisfy the condition that we know the structure a priori. Moreover, exchanging bootstrap samples with others essentially requires the exchange of data, rather than of a data generating method.

If we can fit a parametric distribution to the data, or find a sufficiently close parametrized model, then this is one example where we can generate synthetic data sets.

What other methods exist? I am especially interested in high dimensional data, sparse data, and time series data. For high dimensional data, I'd look for methods that can generate structures (e.g. covariance structure, linear models, trees, etc.) of interest. For time series data, from distributions over FFTs, AR models, or various other filtering or forecasting models seems like a start. For sparse data, reproducing a sparsity pattern seems useful.

I believe these only scratch the surface - these are heuristic, not formal practices. Are there references or resources for generating synthetic data that should be known to practitioners?


Note 1: I realize that this question addresses the literature on how one may generate data like a particular time series model. The distinction here is on practices, especially in order to indicate a known structure (my question), versus similarity / fidelity to an existing data set. It's not necessary in my case to have similarity, as much as known structure, though similarity is greatly preferred to dissimilarity. An exotic synthetic data set for which a model shows promise is less preferred than a realistic simulation.

Note 2: The Wikipedia entry for synthetic data points out that luminaries such as Rubin and Fienberg have addressed this issue, though I have found no references on best practices. It would be interesting to know what would pass muster with, say, the Annals of Applied Statistics (or the AOS), or in review works in these or other journals. In simple and whimsical terms, one may ask where does the threshold between "(acceptably) cooked up" and "too cooked up" exist?

Note 3: Although it doesn't affect the question, the usage scenario is in modeling of vary large, high dimensional data sets, where the research agenda is to learn (both by human and machine ;-)) the structure of the data. Unlike univariate, bivariate, and other low dimensional scenarios, the structure isn't readily inferred. As we step toward a better understanding of the structure, being able to generate data sets with similar properties is of interest in order to see how a modeling method interacts with the data (e.g. to examine parameter stability). Nonetheless, older guides on low dimensional synthetic data can be a starting point that may be extended or adapted for higher dimensional data sets.

share|improve this question
add comment

2 Answers

I'm not sure there are standard practices for generating synthetic data - it's used so heavily in so many different aspects of research that purpose-built data seems to be a more common and arguably more reasonable approach.

For me, my best standard practice is not to make the data set so it will work well with the model. That's part of the research stage, not part of the data generation stage. Instead, the data should be designed to reflect the data generation process. For example, for simulation studies in Epidemiology, I always start from a large hypothetical population with a known distribution, and then simulate study sampling from that population, rather than generating "the study population" directly.

For example, based on our discussion below, two examples of simulated data I've made:

  • Somewhat similar to your SIR-model example below, I once used a mathematical model of the spread of disease over a network to show myself via simulation that a particular constant parameter didn't necessarily imply a constant hazard if you treated the results as the outcome of a cohort study. It was a useful proof of concept while I went digging for an analytical solution.
  • I wanted to explore the impact of a certain sampling scheme for a case-control study. Rather than trying to generate the study outright, I walked through each step of the process. A population of 1,000,000 people, with a given known prevalence of disease and a known covariate pattern. Then from that simulating the sampling process - in this case, how cases and controls were drawn from the population. Only then did I throw an actual statistical model at the collected "simulated studies".

Simulations like the latter are very common when examining the impact of study recruitment methods, statistical approaches to controlling for covariates, etc.

share|improve this answer
    
Thanks for answering. However, based on what I know about epidemiology, a lot more has been done in developing fairly useful stochastic models, most notably SIR models. This isn't true in a lot of other domains, though being able to develop acceptable stochastic data models quickly is part of my intent. –  Iterator Oct 16 '11 at 1:34
    
@Iterator That depends hugely on where you are, and what aspect of epidemiology you're talking about. A cancer epidemiologist, for example, would be shocked to hear that "a lot more" has been done with SIR models - they've likely never encountered one in their work. While SIR-type models are an aspect of a particular part of epidemiology (Infectious disease epi, and even then, not even a large subset of ID Epi), Epidemiology as a field makes tremendous use of statistical models, particularly general linear models, survival analysis and time series. –  Fomite Oct 16 '11 at 1:38
    
Whoa. No offense intended, I only stated that SIR models are a very good example of a standard data generating model. Of course I'm aware that there are other areas of epi that use a whole spectrum of modeling methods. If you have some pointers or references on other data generating methods used in epidemiology, I'm quite open to them. –  Iterator Oct 16 '11 at 1:42
    
@Iterator Sorry if I gave the impression I was offended. Very much not ;). It's just something I get because I sit square in the intersection between mathematical epi and observational epidemiology, and people in one area are flabbergasted the other area exists. I'll edit my answer with a non-SIR type example. –  Fomite Oct 16 '11 at 1:45
add comment

The R statistical package has a simulate function that will simulate data based on a model fit to existing data. This uses the fitted model as the "known" population relationship, then simulates new data based on that model. There is a method for this function in the lme4 package. These fitted objects can take into account random and fixed effects and correlation (including autocorrelation for time series).

This may work do what you want.

share|improve this answer
    
Thanks for the suggestion. This function is certainly useful, though my interest is more in terms of statistical practices and methodology rather than code for a particular solution. This is akin to asking about sampling methods or reproducible analyses, rather than particular packages that implement said methods and analyses. Nonetheless, good methods should become packages. :) –  Iterator Oct 15 '11 at 15:05
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.