0
$\begingroup$

I am using the bank marketing dataset from the UCI ML repo to build an example of a big data storage system along with ETL workflows and Machine Learning models. I would like to create more data so I can feed it to the storage solution pretending it is new "fresh" data from different time periods.

I know there are techniques to create more data by adding noise while at the same time maintaining the same underlying structure. Could anybody suggest some that would apply to this dataset and add a motivation?

I am dealing with just numeric features of type categorical, continuous and binary (no image or text data). I don't think this matters but in case it does, this is a binary classification problem.

Thanks for all your inputs!

$\endgroup$
  • $\begingroup$ You could use SMOTE to oversample your data. $\endgroup$ – user2974951 Dec 21 '18 at 9:05
  • $\begingroup$ Could you provide a link to an example? I have been researching SMOTE but found very few sites where it is being used for data augmentation $\endgroup$ – Odisseo Dec 22 '18 at 21:33
  • $\begingroup$ I don't have such a source, I don't know whether other people use it for this purpose. However SMOTE is used for under / oversampling, so in your case you could oversample your data to generate more data that is similar to your existing data (and maintain your structure). If you are looking for something more complex (that has more variability) then consider permutations. For each class copy all the observations, but permute all the values for each variable. I also don't have a source for this, but this is a common strategy used for assessing variable importances. $\endgroup$ – user2974951 Dec 23 '18 at 20:10
0
$\begingroup$

When the variables are all continuous, I've seen this be done by adding a vector of values from some normal distribution with mean 0. The problem comes with the categorical data, where you have to change some of the categorical values intelligently enough that the new data point is different but realistic. This could be done by switching each category with a small probability. If that's not reasonable, you could also just add random noise at the continuous variables and leave the rest of them alone.

$\endgroup$
  • $\begingroup$ Could you please describe in more detail how you would go about augmenting categorical and binary variables? $\endgroup$ – Odisseo Dec 22 '18 at 22:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.