Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

  7. current community

    your communities

    more stack exchange communities

    company blog
Cross Validated

Questions tagged [data-augmentation]

Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.

38 questions
info newest frequent votes active unanswered
0
votes
1answer
15 views

Adding Bad Features to Decrease Model Performance

I have a dataset on which some researchers have already performed some definitive data analysis and feature selection. Fitting a model to this dataset returns pretty good accuracy. In order to ...
0
votes
1answer
15 views

Data augmentation methods for Raman Spectra

I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try? I have found this paper which suggests ...
0
votes
0answers
11 views

Using Data Augmentation Drops the accuracy by 40% datagen.flow [on hold]

I working on a project with a group, we are using different pre-trained models with imagenet weights and we added 3 dense layers where we freeze the model layers , we are using MURA dataset but a ...
0
votes
1answer
26 views

Data Augmentation Techniques for Cat/Binary/Continuous Numerical Dataset

I am using the bank marketing dataset from the UCI ML repo to build an example of a big data storage system along with ETL workflows and Machine Learning models. I would like to create more data so I ...
1
vote
0answers
12 views

How to paraphrase and augment training data for a question answering ML model?

I have only 50 question, answer pairs in my training data, where each question represent a unique intent. However, the training data is too small to build any meaningful ML model. What are the ...
0
votes
0answers
7 views

Normalization of augmented data?

I just came across this question which is about the order of augmentation and normalization. It seems that it does not make any difference if I first do the augmentation and afterwards the ...
0
votes
0answers
31 views

Is bootstrapping a viable method for augmenting time series data?

I have recently learnt about the bootstrapping method and I am using it in my model tuning phase of my current project. I am working with time series data and therefore have decided to use a ...
0
votes
0answers
61 views

Should I augment validation set?

I am doing image recognition with neural nets and applying image augmentation to extend train set and lower overfitting. Should I apply augmentation to validation set too? Currently I don't and ...
0
votes
1answer
30 views

Imputing nested time series data with R

Does anyone know what is the superior algorithm to impute data in time series? I had strong dropouts over time because it was free to participants how many times to participate in my study (otherwise ...
2
votes
0answers
45 views

Using bootstrap for robust estimation

I am hoping to understand the process of bootstrapping outlier-contaminated data, and the effects on (simple) OLS estimators. In particular, we have a DGP, $$Y_t = \beta X_t + \epsilon_t$$ where $\...
0
votes
0answers
27 views

Should you reshuffle your dataset after you use five or ten crop data augmentation in general machine learning?

My data was shuffled randomly first then I applied a five crop data augmentation. Now my batch went from [8, 3, 256, 256] to ...
0
votes
1answer
29 views

How do you link multiple X input images to a single ground truth image in machine learning?

What I'm asking is basically manual data augmentation. I have some very specific data augmentation for my inputs so I have to create them first in another software instead of doing it on the fly. I ...
1
vote
0answers
28 views

How far can I go with data-augmentation [closed]

I just started with machine learning. Actually I am creating my fist cnn with a own dataset. I've crawled 1200 images from google. hammers, screws, pliers and saws - so 4 classes. I know this data set ...
0
votes
0answers
93 views

How to use data augmentation on uneven multiclass dataset?

I have 12 classes(images) and uneven distributed data in each of these classes. They are as follows(all images): X1 = 16 X2 = 203 X3 = 192 X4 = 220 X5 = 172 X6 = 143 X7 = 22 X8 = 89 X9 = ...
0
votes
0answers
16 views

Choosing test images from augmented training set

I'm training a convolutional neural network to identify land forms. My training set contains 2300 images from each of the following classes: rivers, plains, trees and cities. I've augmented the ...
0
votes
0answers
146 views

Performing data augmentation on validation split

Imagine a dataset with the standard split of train, valid, and test. You train your model on the training set, evaluate it and pick the hyperparameters based on the validation set and -once picked- ...
4
votes
1answer
2k views

Can a GAN be used for data augmentation? [closed]

Can a generative adversarial network (GAN) be used for data augmentation (i.e. to generate synthetic examples that are added to a dataset)? Would it have any impact on the performance of a model ...
0
votes
1answer
134 views

Why data augmentation techniques are applied stochastically

I am using object detection in order to detect objects from drones. I have noticed that using data augmentation can create some images of object as if they were recorded from a different position of ...
2
votes
0answers
57 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
1
vote
2answers
483 views

Data Augmentation and Balancing Dataset in a context of Object Detection

I have a dataset of object detection (bounding box + class) with 2 classes (excluding "background" class). I am worried about two things : First, my dataset counts only 196 samples (I am not too ...
7
votes
3answers
2k views

Data Augmentation strategies for Time Series Forecasting

I'm considering two strategies to do "data augmentation" on time-series forecasting. First, a little bit of background. A predictor $P$ to forecast the next step of a time-series $\lbrace A_i\rbrace$ ...
7
votes
2answers
2k views

Data augmentation on training set only?

Is it common practice to apply data augmentation to training set only, or to both training and test sets?
1
vote
0answers
142 views

Data augmentation and effective class imbalance

Let's say I have a binary classification dataset skewed towards negative samples. Let's say it's 1000 positives and 100000 negatives. Let's say it's image data. I'm training a classifier and I'm ...
3
votes
2answers
53 views

How to augment imaging data for deep learning if I have subject meta information about the original images?

I'd like to try deep learning on about 200 CTs of cancer patients and I gave a lot of meta information about these people that I'd like to make use of in the classification (age, body mass index, ...
2
votes
1answer
213 views

ConvNet data augmentation, full pass or random samples?

I'm adding data augmentation on a FCN model, right now I'm doing simple flips, random zoom and random rotations. At the moment for each sample I do all the four transforms (vertical flip, horizontal ...
0
votes
1answer
37 views

Artificial Datasets; useful for Ai purposes, what about Ai applications in other fields of science?

In Artificial Intelligence, it's common to create sample 'fake' datasets and use them for the purpose of making more efficient algorithms from classification to regression. Datasets with data points ...
0
votes
0answers
120 views

A Bayesian model using random-walk Metropolis method for data augmentation

I have following model, z=beta1+X1*beta2+e; e~N(0,sigma2) prob=exp(z)/(1+exp(z)); and y= 1 with prob, 0 with (1-prob). I have the following prior: beta~N(betahat,A^(-1)); betahat=c(0,0) and A=0....
4
votes
2answers
2k views

Why is data augmentation classified as a type of regularization?

In deep learning papers, data augmentation is often presented as a type of regularization. For example, this is explored in Chiyan Zhang and coauthor's presentation at ICLR17, Understanding deep ...
0
votes
1answer
386 views

Image classification: Using image augmentation to resolve class imbalance

I am working on an image based classification task with some significant class imbalance in the training database of images (largest class: 4967 images, smallest class: 61 images). I will be ...
1
vote
1answer
410 views

Convolution Neural Network Data Augmentation After Normalization Works Much Better [closed]

I am training a Convolution Neural Network similar to LeNet5 to detect road signs in the German Traffic Signs Dataset. With about 35,000 training samples I get to 95% validation accuracy. To improve ...
1
vote
1answer
772 views

Data augmentation

In many papers on CNN,I have read that data augmentation is carried out on a per epoch basis. My thoughts regarding this were that data augmentation is carried out prior to starting the training ...
1
vote
1answer
1k views

Why does the Ciphar 10 tutorial on TensorFlow crop the images to be 24x24?

I was going over the cifar 10 tutorial in tensorflow and was trying to understand why the guys in tensorflow/google decided to crop the images. The only reason I could justify it to myself is because ...
3
votes
1answer
826 views

How does data augmentation reduce overfitting?

I'm trying to understant the benefit apported by the step of data augmentation in a classification algorithm. I have a vector of hexadecimal strings and a column vector containing the label ...
5
votes
3answers
988 views

Data augmentation step in Krizhevsky et al. paper

In the paper Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012., ...
11
votes
3answers
4k views

How to do data augmentation and train-validate split?

I am doing image classification using machine learning. Suppose I have some training data (images) and will split the data into training and validation sets. And I also want to augment the data (...
19
votes
1answer
16k views

Data augmentation techniques for general datasets?

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, ...
8
votes
1answer
1k views

MCMC and data augmentation

I have been looking at an MCMC data augmentation question; the general form of the question is as follows: Suppose data gathered on a process suggests $X_{i} \sim \text{Pois}(\lambda)$ and a prior ...
2
votes
1answer
2k views

Data Augmentation Examples

I am looking for applied references to data augmentation (preferably with some written code). Either online references are books would be great. I found this book online: http://www.amazon.com/...