0
$\begingroup$

Wondering if they could be two ways to create data, when more data is required for machine learning (conventional algorithm for binary classification).

The goal is to create new rows of data, by having varied values from the originals.

Taking South African Hearth Disease Data as an example.

Idea #1: Based on the existing data - each row of the original data, is used to create a new row of data. That is, each cell value of the row, is multiplied by a random number between a range, for example, +/-20%, i.e. 0.8 ~ 1.2.

Trying to illustrate the idea as:

enter image description here

Idea #2: First, to find out the minimum and maximum of each input variable, of each target variable.

Each cell value of the new row is a random number in the correspondent (min, max). By doing this, new rows of the target "1" and "0" will be created.

Trying to illustrate the idea as:

enter image description here

Except to use the created data to build models and apply to validation data, to examine the effect of the created data, what would be the theory and practice to verify if above ideas are valid? Thank you.

$\endgroup$
6
  • $\begingroup$ Have you checked SMOTE? arxiv.org/abs/1106.1813 $\endgroup$
    – Jon Nordby
    Dec 8, 2020 at 15:23
  • $\begingroup$ @jonnor, thank you for the comment. The question and purpose is to create new rows for a bigger dataset, from the original one. SMOTE doesn't create new rows of data, I am afraid. :) $\endgroup$
    – Mark K
    Dec 8, 2020 at 15:26
  • 1
    $\begingroup$ Sure it does? It creates synthetic examples for the minority class. At leas the Python implementaton imbalanced-learn.readthedocs.io/en/stable/generated/… machinelearningmastery.com/… $\endgroup$
    – Jon Nordby
    Dec 8, 2020 at 17:10
  • 1
    $\begingroup$ @jonnor, thank you for pointing out SMOTE as an option. Would you consider to post your comments as an answer so that we can close this question? $\endgroup$
    – Mark K
    Dec 31, 2020 at 0:52
  • 1
    $\begingroup$ I added an answer now with the above comments, and a bit more. $\endgroup$
    – Jon Nordby
    Jan 1, 2021 at 22:12

1 Answer 1

2
$\begingroup$

Your ideas are similar to some already explored in the literature. The most important for success is that the modified samples are faithful to the underlying data distribution. If there is correlation between variables, randomizing individual variables may violate patterns in the data.

One of the most commonly used methods is SMOTE: Synthetic Minority Over-sampling Technique. There is a Python implementation in imblearn.over_sampling, including some variations.

One can use any general generative method to synthesize new samples. For numeric data one can consider a mixture model, or a Gaussian Process model, or a Probabilistic Graphical Model. These can handle (some) relationships between features, such as correlation. Note that generative models can be used to do inference as well, so make sure to compare its predictive performance as well - it can sometimes outperform a discriminative classifier.

Synthesizing new samples in this this manner can be seen as a Data Augmentation strategy. Verifying the performance of applying (old or new) data augmentation techniques is the same as for most other (potential) modelling changes. Split the dataset, applying the techniques on the training set, and evaluate the performance (gains) on a held-out testing set. Care must be taken to avoid over-fitting on the test set. If hyper-parameters are to be tuned, that should be done on a validation set.

$\endgroup$
2
  • $\begingroup$ superb, Jonnor! wish you a very happy new year! $\endgroup$
    – Mark K
    Jan 2, 2021 at 0:16
  • 1
    $\begingroup$ Thank you Mark! Wish you all the best in the new year! $\endgroup$
    – Jon Nordby
    Jan 2, 2021 at 0:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.