As context: When working with a very large data set, I am sometimes asked if we can create a synthetic data set where we "know" the relationship between predictors and the response variable, or relationships among predictors.
Over the years, I seem to encounter either one-off synthetic data sets, which look like they were cooked up in an ad hoc manner, or more structured data sets that seem especially favorable for the researcher's proposed modeling method.
I believe that I'm over looking standard methods for creating synthetic data sets. Although bootstrap resampling is one common method for creating synthetic data set, it doesn't satisfy the condition that we know the structure a priori. Moreover, exchanging bootstrap samples with others essentially requires the exchange of data, rather than of a data generating method.
If we can fit a parametric distribution to the data, or find a sufficiently close parametrized model, then this is one example where we can generate synthetic data sets.
What other methods exist? I am especially interested in high dimensional data, sparse data, and time series data. For high dimensional data, I'd look for methods that can generate structures (e.g. covariance structure, linear models, trees, etc.) of interest. For time series data, from distributions over FFTs, AR models, or various other filtering or forecasting models seems like a start. For sparse data, reproducing a sparsity pattern seems useful.
I believe these only scratch the surface - these are heuristic, not formal practices. Are there references or resources for generating synthetic data that should be known to practitioners?
Note 1: I realize that this question addresses the literature on how one may generate data like a particular time series model. The distinction here is on practices, especially in order to indicate a known structure (my question), versus similarity / fidelity to an existing data set. It's not necessary in my case to have similarity, as much as known structure, though similarity is greatly preferred to dissimilarity. An exotic synthetic data set for which a model shows promise is less preferred than a realistic simulation.
Note 2: The Wikipedia entry for synthetic data points out that luminaries such as Rubin and Fienberg have addressed this issue, though I have found no references on best practices. It would be interesting to know what would pass muster with, say, the Annals of Applied Statistics (or the AOS), or in review works in these or other journals. In simple and whimsical terms, one may ask where does the threshold between "(acceptably) cooked up" and "too cooked up" exist?
Note 3: Although it doesn't affect the question, the usage scenario is in modeling of vary large, high dimensional data sets, where the research agenda is to learn (both by human and machine ;-)) the structure of the data. Unlike univariate, bivariate, and other low dimensional scenarios, the structure isn't readily inferred. As we step toward a better understanding of the structure, being able to generate data sets with similar properties is of interest in order to see how a modeling method interacts with the data (e.g. to examine parameter stability). Nonetheless, older guides on low dimensional synthetic data can be a starting point that may be extended or adapted for higher dimensional data sets.
