25
$\begingroup$

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, mirroring, adjusting contrast, etc. it is possible to generate additional images from the original ones.

In the case of images, the data augmentation is relatively straightforward. However, suppose (for example) that one has a training set of $100$ samples and few hundred continuous variables that represent different things. The data augmentation does not anymore seem so intuitive. What could be done in such case?

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ I think some methods like PCA or AE is still intuitive for data augmentation. The first methods is apply PCA and keep first k eigenvalues and set k-n eigenvalues randomly from a distriution, gaussian etc. If Another methods use Auto-encoders to construct data itself. If the number of hidden units are close to visible units, it can construction itself pretty good. Reconstructed data can be used to as augmented part of the data. $\endgroup$
    – yasin.yazici
    May 26, 2015 at 12:55
  • $\begingroup$ @mmh, did this answer your question? $\endgroup$
    – shf8888
    Jun 20, 2015 at 22:14
  • $\begingroup$ @yasin.yazici Hi. could you explain the bit about data augmentation using pca ? Suppose I have a data $100x50$ with $50$ being the feature dimension. Now I do PCA and find that the first $30$ top eigenvectors are sufficient. What should I do in the next $20$ eigenvectors and how should I introduce the randomness ? $\endgroup$
    – roni
    Aug 31, 2016 at 12:11
  • $\begingroup$ See my masters thesis, page 80 for an overview of data augmentation techinques for images. $\endgroup$
    – Martin Thoma
    Aug 1, 2017 at 6:36
  • $\begingroup$ Data augmentation makes a lot of sense for images, eg. A rotated image of the object is still an image of the object, and you need the model to be exposed to that, but you presumably don't necessarily have such an image in your dataset. What is the need for / point of data augmentation in this case? It isn't clear to me that you should be doing this. $\endgroup$
    – gung - Reinstate Monica
    Aug 1, 2019 at 1:04

2 Answers 2

Reset to default
19
+50
$\begingroup$

I understand this question as involving both feature construction and dealing with the wealth of features you already have + will construct, relative to your observations (N << P).

Feature Construction

Expanding upon @yasin.yazici's comment, some possible ways to augment the data would be:

  • PCA
  • Auto-encoding
  • Transform's such as log, powers, etc.
  • Binning continuous variables into discrete categories (i.e., continuous variable is 1 SD above mean, 1 below mean, etc.)
  • Composite variables (for example, see here)

I'm sure there are many more I'm missing.

Feature Selection / Dimensionality reduction

You may reduce dimensionality with techniques such as PCA (although perhaps not after augmenting your data with PCA variables). Alternatively, you may use algorithms that perform feature selection for you, such as lasso, random forest, etc.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Can you please tell how auto encoding can be used for feature construction ? $\endgroup$
    – roni
    Aug 25, 2016 at 12:16
  • 1
    $\begingroup$ @roni Successful training of an autoencoder yields some representation of the data at a higher level of abstraction. Hopefully a more useful representation that you can use in a classifier. $\endgroup$
    – chris
    Jun 8, 2017 at 20:23
  • $\begingroup$ Hi, doesn't adding samples using transformations (e.g. log, etc.) significantly change the original distribution? Also, is there any specific set of desirable property of the augmented data (for example, has to have same standard deviation as the original)? Do you think these are applicable to spectrum data? $\endgroup$
    – Darren Christopher
    Mar 4, 2020 at 12:27
0
$\begingroup$

I faced a similar problem where in I wanted to augment unlabelled numeric data. I augmented data in the following way: (Say I have a data set of size 100*10.)

  1. Create a list by randomly sampling values from {0,1}, such that the number of zeros are less than the number of 1s,say the proportion of 0s is 20% in this case. So one will have a list of 0s and 1s of length 100.
  2. Use this list as a dependent variable and passed it into smote to generate more data points. (Here smote would generate points on the edges connecting the data points which correspond to 0s in the list generated).
  3. Repeat this process until data set of required size is obtained.
Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Please do not post identical answers to multiple threads. If you really believe that the same answer as you've posted elsewhere fully answers another question, flag that question as a duplicate of the first. $\endgroup$
    – gung - Reinstate Monica
    Aug 1, 2019 at 0:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.