21
$\begingroup$

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, mirroring, adjusting contrast, etc. it is possible to generate additional images from the original ones.

In the case of images, the data augmentation is relatively straightforward. However, suppose (for example) that one has a training set of $100$ samples and few hundred continuous variables that represent different things. The data augmentation does not anymore seem so intuitive. What could be done in such case?

$\endgroup$
  • 2
    $\begingroup$ I think some methods like PCA or AE is still intuitive for data augmentation. The first methods is apply PCA and keep first k eigenvalues and set k-n eigenvalues randomly from a distriution, gaussian etc. If Another methods use Auto-encoders to construct data itself. If the number of hidden units are close to visible units, it can construction itself pretty good. Reconstructed data can be used to as augmented part of the data. $\endgroup$ – yasin.yazici May 26 '15 at 12:55
  • $\begingroup$ @mmh, did this answer your question? $\endgroup$ – shf8888 Jun 20 '15 at 22:14
  • $\begingroup$ @yasin.yazici Hi. could you explain the bit about data augmentation using pca ? Suppose I have a data $100x50$ with $50$ being the feature dimension. Now I do PCA and find that the first $30$ top eigenvectors are sufficient. What should I do in the next $20$ eigenvectors and how should I introduce the randomness ? $\endgroup$ – roni Aug 31 '16 at 12:11
  • $\begingroup$ See my masters thesis, page 80 for an overview of data augmentation techinques for images. $\endgroup$ – Martin Thoma Aug 1 '17 at 6:36
  • $\begingroup$ Data augmentation makes a lot of sense for images, eg. A rotated image of the object is still an image of the object, and you need the model to be exposed to that, but you presumably don't necessarily have such an image in your dataset. What is the need for / point of data augmentation in this case? It isn't clear to me that you should be doing this. $\endgroup$ – gung - Reinstate Monica Aug 1 '19 at 1:04
17
+50
$\begingroup$

I understand this question as involving both feature construction and dealing with the wealth of features you already have + will construct, relative to your observations (N << P).

Feature Construction

Expanding upon @yasin.yazici's comment, some possible ways to augment the data would be:

  • PCA
  • Auto-encoding
  • Transform's such as log, powers, etc.
  • Binning continuous variables into discrete categories (i.e., continuous variable is 1 SD above mean, 1 below mean, etc.)
  • Composite variables (for example, see here)

I'm sure there are many more I'm missing.

Feature Selection / Dimensionality reduction

You may reduce dimensionality with techniques such as PCA (although perhaps not after augmenting your data with PCA variables). Alternatively, you may use algorithms that perform feature selection for you, such as lasso, random forest, etc.

$\endgroup$
  • 2
    $\begingroup$ Can you please tell how auto encoding can be used for feature construction ? $\endgroup$ – roni Aug 25 '16 at 12:16
  • 1
    $\begingroup$ @roni Successful training of an autoencoder yields some representation of the data at a higher level of abstraction. Hopefully a more useful representation that you can use in a classifier. $\endgroup$ – Chris Anderson Jun 8 '17 at 20:23
0
$\begingroup$

I faced a similar problem where in I wanted to augment unlabelled numeric data. I augmented data in the following way: (Say I have a data set of size 100*10.)

  1. Create a list by randomly sampling values from {0,1}, such that the number of zeros are less than the number of 1s,say the proportion of 0s is 20% in this case. So one will have a list of 0s and 1s of length 100.
  2. Use this list as a dependent variable and passed it into smote to generate more data points. (Here smote would generate points on the edges connecting the data points which correspond to 0s in the list generated).
  3. Repeat this process until data set of required size is obtained.
$\endgroup$
  • 1
    $\begingroup$ Please do not post identical answers to multiple threads. If you really believe that the same answer as you've posted elsewhere fully answers another question, flag that question as a duplicate of the first. $\endgroup$ – gung - Reinstate Monica Aug 1 '19 at 0:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.