How to simulate more data for machine learning? I am attempting to analyze a small dataset using machine learning (SVM, binary problem). There are $103$ samples and $215$ variables (all variables are real numbers). Some of the variables (around half) are moderately/highly correlated with each other.
Knowing the empirical distribution of each variable, would it be somehow possible to generate more data by a simulation? I guess I should do it only for the training data. I am using nested cross-validation for picking parameters / estimating performance if it matters.
 A: You can ignore some of the correlated attributes and use Cross validation, instead of "generating" more data. Cross validation is a common approach in situations where you have more attributes than observations.
A: Similar to what stan0 said, I would use cross validation, but more specifically repeated k-fold Cross Validation. For a 'smaller' data set of 103 by 215, I would perhaps use 10x10 (10 repeats of 10 fold cross validation, so 100 model fits in total). If you're using R, I would highly suggest the caret package which is able to handle this for you using the trainControl() function.
Good resources for the underlying funciton calls and algorithum:
https://topepo.github.io/caret/training.html
https://topepo.github.io/caret/splitting.html
A: 
Knowing the empirical distribution of each variable, would it be
  somehow possible to generate more data by a simulation?

Yes, you can always generate more data, the question is of its quality! If you know the empirical distributions of the individual variables, this is not enough to generate more data. If you generated data on each of the variables independently, then you would end up with dataset where the features are unrelated to themselves and your target variable. So if your purpose is to check how does your model behaves with the data where the variables are unrelated to your target variables (i.e. how much is it prone to produce false positives), then this would help. Otherwise, such approach does not make sense. As others said, $k$-fold cross-validation and similar methods may help if size of your sample is limited (see also questions tagged as small-sample for other suggestions).
However, there are situations where you can produce artificial data for your model. It can be done, and often is done, in image processing where you can start with small dataset and then add noise to it (rotate images, add white noise, mix them with other pictures, change the backgrounds, make them blurry, change their sizes, resolution etc.). Unfortunately, this does not easily translate to other kinds of data.
