Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

  7. current community

    your communities

    more stack exchange communities

    company blog
Cross Validated

Questions tagged [data-augmentation]

Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.

45 questions
info Newest Frequent Votes Active Unanswered
1
vote
1answer
24 views

Does online data augmentation make sense?

Data augmentation is popularly done online as that is how it is typically implemented and suggested in neural network frameworks like Keras and TensorFlow. I have also seen it described in e.g. the ...
0
votes
0answers
22 views

How to choose data augmenation technique?

How can I evaluate the performances of a data augmentation technique? Shall I use the improvement in accuracy? Can a data augmentation technique improve performances of a model, for instance a ...
0
votes
0answers
9 views

augment / map / forecast a time delayed time-series

I am not sure what the proper title for my question is but I'll try to describe what I want to do. I have a time-series where the value changes over time, meaning a portion get's added time-delayed ...
1
vote
0answers
35 views

Subsampling as a method for time series train/validation splits

I have a question concerning train-test splits for time series data: Background I have a dataset of sensor data points for 13 month with datapoints measured every 5 minutes which I downsample to ...
2
votes
0answers
45 views

What are some techniques to augment tabular data?

As we know we can perform data augmentation to "image dataset". We can apply random rotation, shifts, shear and flips over images. Are there techniques to augment tabular small dataset? I know the ...
0
votes
1answer
17 views

Image Augmentation or incrementing dataset by flipping/mirroring?

My task is a regression task, where an input image results in another, transformed image. So far so good, works quite well. As my data set is fairly small, I want to take some actions. Here I wanted ...
0
votes
1answer
31 views

When doing data augmentation, should you train with the original data as well or just the augmented data?

When doing data augmentation in computer vision problems, should you train with the original (un-augmented) data as well or just the augmented data? Are there pros and cons to the two strategies or ...
0
votes
1answer
33 views

Adding Bad Features to Decrease Model Performance

I have a dataset on which some researchers have already performed some definitive data analysis and feature selection. Fitting a model to this dataset returns pretty good accuracy. In order to ...
0
votes
1answer
31 views

Data augmentation methods for Raman Spectra

I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try? I have found this paper which suggests ...
0
votes
1answer
85 views

Data Augmentation Techniques for Cat/Binary/Continuous Numerical Dataset

I am using the bank marketing dataset from the UCI ML repo to build an example of a big data storage system along with ETL workflows and Machine Learning models. I would like to create more data so I ...
1
vote
1answer
35 views

How to paraphrase and augment training data for a question answering ML model?

I have only 50 question, answer pairs in my training data, where each question represent a unique intent. However, the training data is too small to build any meaningful ML model. What are the ...
0
votes
0answers
11 views

Normalization of augmented data?

I just came across this question which is about the order of augmentation and normalization. It seems that it does not make any difference if I first do the augmentation and afterwards the ...
0
votes
0answers
39 views

Is bootstrapping a viable method for augmenting time series data?

I have recently learnt about the bootstrapping method and I am using it in my model tuning phase of my current project. I am working with time series data and therefore have decided to use a ...
0
votes
0answers
122 views

Should I augment validation set?

I am doing image recognition with neural nets and applying image augmentation to extend train set and lower overfitting. Should I apply augmentation to validation set too? Currently I don't and ...
0
votes
1answer
40 views

Imputing nested time series data with R

Does anyone know what is the superior algorithm to impute data in time series? I had strong dropouts over time because it was free to participants how many times to participate in my study (otherwise ...
1
vote
0answers
27 views

Active learning for object detection- Batch Selection

I have a small dataset of about 220 images for three classes. I am using YOLO (you only look once) network for an object detection. I am trying to use Active learning in order to reduce the number of ...
2
votes
0answers
53 views

Using bootstrap for robust estimation

I am hoping to understand the process of bootstrapping outlier-contaminated data, and the effects on (simple) OLS estimators. In particular, we have a DGP, $$Y_t = \beta X_t + \epsilon_t$$ where $\...
0
votes
0answers
33 views

Should you reshuffle your dataset after you use five or ten crop data augmentation in general machine learning?

My data was shuffled randomly first then I applied a five crop data augmentation. Now my batch went from [8, 3, 256, 256] to ...
0
votes
1answer
36 views

How do you link multiple X input images to a single ground truth image in machine learning?

What I'm asking is basically manual data augmentation. I have some very specific data augmentation for my inputs so I have to create them first in another software instead of doing it on the fly. I ...
1
vote
0answers
31 views

How far can I go with data-augmentation [closed]

I just started with machine learning. Actually I am creating my fist cnn with a own dataset. I've crawled 1200 images from google. hammers, screws, pliers and saws - so 4 classes. I know this data set ...
0
votes
0answers
101 views

How to use data augmentation on uneven multiclass dataset?

I have 12 classes(images) and uneven distributed data in each of these classes. They are as follows(all images): X1 = 16 X2 = 203 X3 = 192 X4 = 220 X5 = 172 X6 = 143 X7 = 22 X8 = 89 X9 = ...
0
votes
0answers
16 views

Choosing test images from augmented training set

I'm training a convolutional neural network to identify land forms. My training set contains 2300 images from each of the following classes: rivers, plains, trees and cities. I've augmented the ...
0
votes
0answers
173 views

Performing data augmentation on validation split

Imagine a dataset with the standard split of train, valid, and test. You train your model on the training set, evaluate it and pick the hyperparameters based on the validation set and -once picked- ...
5
votes
1answer
3k views

Can a GAN be used for data augmentation? [closed]

Can a generative adversarial network (GAN) be used for data augmentation (i.e. to generate synthetic examples that are added to a dataset)? Would it have any impact on the performance of a model ...
0
votes
1answer
148 views

Why data augmentation techniques are applied stochastically

I am using object detection in order to detect objects from drones. I have noticed that using data augmentation can create some images of object as if they were recorded from a different position of ...
2
votes
0answers
62 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
1
vote
2answers
518 views

Data Augmentation and Balancing Dataset in a context of Object Detection

I have a dataset of object detection (bounding box + class) with 2 classes (excluding "background" class). I am worried about two things : First, my dataset counts only 196 samples (I am not too ...
8
votes
3answers
3k views

Data Augmentation strategies for Time Series Forecasting

I'm considering two strategies to do "data augmentation" on time-series forecasting. First, a little bit of background. A predictor $P$ to forecast the next step of a time-series $\lbrace A_i\rbrace$ ...
7
votes
3answers
3k views

Data augmentation on training set only?

Is it common practice to apply data augmentation to training set only, or to both training and test sets?
1
vote
0answers
156 views

Data augmentation and effective class imbalance

Let's say I have a binary classification dataset skewed towards negative samples. Let's say it's 1000 positives and 100000 negatives. Let's say it's image data. I'm training a classifier and I'm ...
3
votes
2answers
55 views

How to augment imaging data for deep learning if I have subject meta information about the original images?

I'd like to try deep learning on about 200 CTs of cancer patients and I gave a lot of meta information about these people that I'd like to make use of in the classification (age, body mass index, ...
2
votes
1answer
238 views

ConvNet data augmentation, full pass or random samples?

I'm adding data augmentation on a FCN model, right now I'm doing simple flips, random zoom and random rotations. At the moment for each sample I do all the four transforms (vertical flip, horizontal ...
0
votes
1answer
37 views

Artificial Datasets; useful for Ai purposes, what about Ai applications in other fields of science?

In Artificial Intelligence, it's common to create sample 'fake' datasets and use them for the purpose of making more efficient algorithms from classification to regression. Datasets with data points ...
0
votes
0answers
138 views

A Bayesian model using random-walk Metropolis method for data augmentation

I have following model, z=beta1+X1*beta2+e; e~N(0,sigma2) prob=exp(z)/(1+exp(z)); and y= 1 with prob, 0 with (1-prob). I have the following prior: beta~N(betahat,A^(-1)); betahat=c(0,0) and A=0....
5
votes
2answers
2k views

Why is data augmentation classified as a type of regularization?

In deep learning papers, data augmentation is often presented as a type of regularization. For example, this is explored in Chiyan Zhang and coauthor's presentation at ICLR17, Understanding deep ...
0
votes
1answer
403 views

Image classification: Using image augmentation to resolve class imbalance

I am working on an image based classification task with some significant class imbalance in the training database of images (largest class: 4967 images, smallest class: 61 images). I will be ...
1
vote
1answer
448 views

Convolution Neural Network Data Augmentation After Normalization Works Much Better [closed]

I am training a Convolution Neural Network similar to LeNet5 to detect road signs in the German Traffic Signs Dataset. With about 35,000 training samples I get to 95% validation accuracy. To improve ...
1
vote
1answer
828 views

Data augmentation

In many papers on CNN,I have read that data augmentation is carried out on a per epoch basis. My thoughts regarding this were that data augmentation is carried out prior to starting the training ...
1
vote
1answer
1k views

Why does the Ciphar 10 tutorial on TensorFlow crop the images to be 24x24?

I was going over the cifar 10 tutorial in tensorflow and was trying to understand why the guys in tensorflow/google decided to crop the images. The only reason I could justify it to myself is because ...
3
votes
1answer
861 views

How does data augmentation reduce overfitting?

I'm trying to understant the benefit apported by the step of data augmentation in a classification algorithm. I have a vector of hexadecimal strings and a column vector containing the label ...
6
votes
3answers
1k views

Data augmentation step in Krizhevsky et al. paper

In the paper Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012., ...
11
votes
3answers
4k views

How to do data augmentation and train-validate split?

I am doing image classification using machine learning. Suppose I have some training data (images) and will split the data into training and validation sets. And I also want to augment the data (...
19
votes
1answer
16k views

Data augmentation techniques for general datasets?

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, ...
8
votes
1answer
1k views

MCMC and data augmentation

I have been looking at an MCMC data augmentation question; the general form of the question is as follows: Suppose data gathered on a process suggests $X_{i} \sim \text{Pois}(\lambda)$ and a prior ...
2
votes
1answer
2k views

Data Augmentation Examples

I am looking for applied references to data augmentation (preferably with some written code). Either online references are books would be great. I found this book online: http://www.amazon.com/...