My question is simple: I have a dataset with multiple numerical features (let's say 1500 data points with 7 features). The question is the following: Can you generate 10k rows of new data from the same distribution?

Now, I'm having a little trouble approaching this problem. How would you do that? And which hypotheses would you take?

I was trying to look at each individual feature and plot pdf for each one of them, see what they look like, try to make some assumptions about the distributions they come from, but not really sure about that.

For info, I'm doing this on Python, I don't know if you an idea of which packages I could use.

Thanks for your help !

  • 1
    $\begingroup$ What for do you need those samples? $\endgroup$ – Tim Jul 24 at 18:41
  • $\begingroup$ @Tim it's just an assignment that I have, no real application so far $\endgroup$ – Derbs Jul 24 at 18:46
  • 1
    $\begingroup$ Note that generating from the individual marginal distributions will in general be insufficient; typically you need to concern yourself with the joint distribution. Further note that KDEs don't quite generate "from the same distribution" since they will inflate the variance slightly (though often that isn't a bad thing for this sort of exercise) $\endgroup$ – Glen_b -Reinstate Monica Jul 25 at 3:00
  • $\begingroup$ @Glen_b so you’re saying that I should model this using a multivariate scenario instead of considering each feature individually ? $\endgroup$ – Derbs Jul 25 at 6:59
  • $\begingroup$ Yes, unless you have reason to believe the variables to be independent. $\endgroup$ – Glen_b -Reinstate Monica Jul 25 at 12:48

What you need is a Generative Model of you data.

This will let you model and learn how to sample from the hidden/latent distribution that "generated" your data (or in some cases find exactly the hidden/latent distribution according to your model).

Some examples of the SoA techniques that you could use are:

  • GANs
  • VAEs
  • Wasserstein Autoencoders

You can implement all of them with many Python Frameworks like:

  • TensorFlow
  • Keras
  • PyTorch

If you want to take this as a didactic task or if the data is simple enough, you can also explore the basic techniques like trying to fit your data to a model with things like: Maximum Likelihood, MAP, GMMs ecc...

This is probably the approach you were going to use if I understand correctly from your question.

  • $\begingroup$ Thanks for your answer ! But as I'm not very familiar with deep learning frameworks, I was considering the other approaches you suggested. Could you explain them a little more ? Also, I was looking at KDE. Would that be a good option to create a generative model? @luca $\endgroup$ – Derbs Jul 24 at 18:42
  • $\begingroup$ @Derbs Yes, of course KDE is fine. Any estimar or estimation technique is the right way to approach this. Obviously then, depending on the data and their distribution, some techniques will perform better that others. Moreover you might want to consider the trade-offs between the simplicity, computational costs and performances of the estimators. Wothout having a good look at the data it's hard to say which technique will perform better. $\endgroup$ – Luca Angioloni Jul 24 at 18:51
  • $\begingroup$ @Derbs Anyway the idea behind estimators, using the example of the ML estimator, is this (I am going to oversimplify everything): You choose a model for your data, let's say a Gaussian; now you need to find the parameters of this model (in this case mean and variance) so that they will maximize the likelihood of your data; this means that you want to find the best Gaussian distribution you can find i.e. the most similar Gaussian to the distribution of your data. Once you have this you can use this specific Gaussian as your data model, as your hidden/latent data distribution and sample from it. $\endgroup$ – Luca Angioloni Jul 24 at 19:01
  • $\begingroup$ Thank you very much, this was very helpful ! $\endgroup$ – Derbs Jul 24 at 21:10
  • $\begingroup$ I used KDE for now (I'll check GMM and others to compare after) to generate new data. But how would would check if the generated distribution is satisfactory ? I've been plotting KDE for both old data and new data (for each feature) to compare, but is there an analytical way ? @luca $\endgroup$ – Derbs Jul 26 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.