Generate synthetic data to match sample data If I have a sample data set of 5000 points with many features and I have to generate a dataset with say 1 million data points using the sample data. It is like oversampling the sample data to generate many synthetic out-of-sample data points. The out-of-sample data must reflect the distributions satisfied by the sample data. The data here is of telecom type where we have various usage data from users. Is there any techniques available for this? Can SMOTE be applied for this problem?
 A: I found this R package named synthpop that was developed for public release of confidential data for modeling. Supersampling with it seems reasonable.
  synthpop: Bespoke Creation of Synthetic Data in R
A: You could also look at MUNGE. It generates synthetic datasets from a nonparametric estimate of the joint distribution. The idea is similar to SMOTE (perturb original data points using information about their nearest neighbors), but the implementation is different, as well as its original purpose. Whereas SMOTE was proposed for balancing imbalanced classes, MUNGE was proposed as part of a 'model compression' strategy. The goal is to replace a large, accurate model with a smaller, efficient model that's trained to mimic its behavior. There are many details you can ignore if you're just interested in the sampling procedure. The paper compares MUNGE to some simpler schemes for generating synthetic data.
Basic idea:

  
*
  
*Generate a synthetic point as a copy of original data point $e$
  
*Let $e'$ be be the nearest neighbor
  
*For each attribute $a$:
  
  
*
  
*If $a$ is discrete: With probability $p$, replace the synthetic point's attribute $a$ with $e'_a$.
  
*If $a$ is continuous: With probability $p$, replace the synthetic point's attribute $a$ with a value drawn from a normal distribution with mean $e'_a$ and standard deviation $\left | e_a - e'_a \right | / s$
  
*$p$ and $s$ are parameters
  
  

The paper:

Bucila et al. (2006). Model compression.

Regarding the stats/plots you showed, it would be good to check some measure of the joint distribution too, since it's possible to destroy the joint distribution while preserving the marginals.
A: I am developing a Python package, PySynth, aimed at data synthesis that should do what you need: https://pypi.org/project/pysynth/ The IPF method used there now does not work well for datasets with many columns, but it should be sufficient for the needs you mention here.
