I have problems understanding data augmentation. I currently have low-dimension features, each data point only has 3 features. A total of 20k non-linear data with only 3 features. I have generated about 120 additional features from statistical values (e.g. mean, var, std). Is generating statistical values called data augmentation? Is it possible for the augmented data to be oversampled like SMOTE? What would be the validity?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Generating additional features is considered feature engineering rather than data augmentation. For example, assuming our data point to a JPEG image, calculating the average entropy per pixel is an additional feature so it is "feature engineering"; rotating an image by $x^\text{o}$ degrees and changing its contrast thought is a "new image" so it counts as "data augmentation". As a basic rule of thumb, if we add rows (i.e. items in our dataset) we call it data augmentation; if we add columns (i.e. features in our dataset) we call it feature engineering.
If we want to create additional data points, it makes sense to sample then on their original space. I would thus suggest avoiding oversampling the derived statistics themselves. To that extent, I would also suggest not using SMOTE to create new data-points, we recently had a good CV.SE question on this here: Is SMOTE any good at creating new points? where a number of potential shortcomings are discussed. Particular to the case presented as we expect "non-linear data", SMOTE's linear interpolation paradigm is likely too simplistic. Instead of SMOTE, using a generative model for your data hand (i.e. a model that explicitly aims to capture the joint probability between the features) is more natural and given the low dimensionality of the original features, most likely not very hard to train. A small VAE or a standard Probabilistic PCA, should be enough at first instance. After generating these new 3D points, we could perform our intended featuring engineering steps as before.