regarding generating synthetic data simulating the real data [closed]

Question

We are trying to develop some predictive models. The current scenario is that we have to rely on synthetic data at first since the real data set will not be available quite soon. It is understandable that the model developed based on synthetic data may be totally wrong for the real data set. We are just trying to ensure the generated synthetic data can simulate the real data in an acceptable way.

Are there any systematic approaches (from both statistical point of view and data mining perspective), to create a synthetic data set that are as similar to real data set as possible. Besides, are there any statistical measure to evaluate/estimate the similarity between synthetic data and real data?

I understand that it is kind of difficult to answer this question without knowing the concrete problem itself. I just would like to know the generic statistical/data mining approaches for this problem. Thanks.

Answer 1

Since this question doesn't have any answers so far, I'm going to jump in and propose something that might help. Take a look at this paper. If you have some decent amount of data available, then this approach should allow you to learn the properties of that data and generate more. The downside of this is that you can't actually do anything until you get at least some data. Hope this helps!

Answer 2

You need to have some kind of information on the distribution of your real dataset. In other words, can you answer if your dataset has a Normal, Poisson, Exponential, etc, distribution. If you can answer that, then you can simply use some programming tool to generate your sample dataset according to the suspected distribution.

When you get your real dataset, you can perform a statistical test to see if the sample distribution matches your real distribution.

Answer 3

In a similar case, I have used the following approach. I had some ideas about the characteristics of the synthetic data I wanted to generate.

I started searching for these characteristics in public data by designing some data filters. I applied these filters to several public data sets, unrelated to my problem but good enough to be potentially useful.

I ended up with a data set from public companies' financial statements that fitted my requirements. Although my problem had nothing to do with financial data, their numbers were adequate to developing further tests.

Answer 4

If you had some real data, then you could leverage GMMs or Generative AI models to create synthetic data to mimic your data. If you don't have any real data, then could go for declarative approaches, but it would be difficult to achieve real data value from it. If you have some idea of the type of data and properties (e.g., range, distributions, etc) you want to generate, take a look at the Synner project, it looks promising. If you eventually get some real data (or another real dataset that somewhat matches your requirements -- even if it is from another domain), then have a look at the ydata-synthetic project to create synthetic data.