Questions tagged [data-augmentation]

Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

trying to create a cache dataset in monai: LinAlgError: SVD did not converge [closed]

This is the error that I am getting: ...
Macdro's user avatar
  • 1
0 votes
1 answer
11 views

How useful is data augmentation techique?

I have a very small sample size in one group and cannot perform statistical analysis. Is there any other way of performing analysis by artificially generating data and then presenting the results of ...
MG Lewis's user avatar
0 votes
0 answers
10 views

Computing coordinates of points of an image after elastic deformation

My task is: given an image and set of points of interest, elastically and randomly deform the image and save it with the modified aforementioned points. example: (blue points are the points of ...
FirePapaya's user avatar
1 vote
0 answers
27 views

How does the order of implementation of Generative Data Augmentation and Generative Audio Super-Resoln matter? Which one should be implemented first?

I have a low quality audio dataset that I will use for classification. My goal is both to increase the quality of this dataset to make it easier to label it with supervised methods (super resolution) ...
Yalçın Cenik's user avatar
1 vote
0 answers
62 views

Regression ML model: Data Augmentation [closed]

I'm currently working on data augmentation to my regression problem, and a (possible) solution that came to my mind was to add a perturbed dataset to the original dataset, and hence double the ...
Pedro Italo's user avatar
0 votes
0 answers
27 views

When is the case that a classifier's accuracy is better when no data augmentation has been applied?

When is the case that a classifier's accuracy is better when no data augmentation has been applied (Compared to the dataset with the augmentations applied)? I'd like to know especially how the data is ...
Cork's user avatar
  • 3
0 votes
0 answers
18 views

Data augmentation to equilibrate class occurences on image set with multiple different objects in each image

I am using this (reduced) dataset to run tests for detection of defects in solid wood using YOLOV5. Initial tests are promising. As can be seen in Fig. 1, my class occurences are quite imbalanced - ...
iamnotgoodwithcomputer's user avatar
1 vote
0 answers
25 views

Investigating the Impact of Additive Gaussian Noise on EEG Signal Classification: Analyzing the Relationship between Augmented and Original Data

Definition: I have conducted research on EEG signal classification, specifically focusing on distinguishing between two different classes using raw EEG signals. Data availability poses a significant ...
Armin Amini's user avatar
1 vote
1 answer
29 views

small dataset for a regression task

I am trying to train a NN model in MATLAB to predict the amount of overflow for flooded junctions in an urban runoff system and I have 45 samples and 15 features. The issue is, I don't think 45 ...
Ari's user avatar
  • 11
1 vote
0 answers
18 views

Data augmentation specific per class

I have a database of defects on plastic films. Defects are burns, holes, and similar things. Some defects are direction specific. A vertical sign on the material represents a scratch done on the ...
Jonny_92's user avatar
2 votes
1 answer
55 views

Augmenting Image Data on Small Dataset [duplicate]

I want to train image segmentation model using U-Net with pretrained ResNet34 as encoder. My dataset is really small, i separate it with Train data : 57 images Validation data : 16 images Test data : ...
anastasia's user avatar
0 votes
0 answers
33 views

How to perform MCMC posterior sampling of $X+Y$ random variable?

Let $X$ follow normal distribution with mean $\mu_X=10$ and variance $V_X=20$. Let $Y$ follow truncated normal distribution with mean $\mu_Y=-10,V_Y=8$, where $Y\leq 0$ by truncation. Let $Z=X+Y$. ...
user45765's user avatar
  • 1,365
1 vote
1 answer
25 views

Manually adding edge-cases to a text classification model

Suppose I want to get training data for a model that deals with sentiment analysis for text that indicates an affirmative (yes) or negative (no) response, such as ...
multiheadedattention's user avatar
1 vote
0 answers
29 views

Were missing values in data augmentation procedure parameters?

Suppose I have a data set with $(Y,X)$ where $Y$ has some missing entries(say 20% missing). From $p(Y,X|\lambda)$ and prior $p(\lambda)$, I can conduct MCMC by data augmentation to impute missing $Y$'...
user45765's user avatar
  • 1,365
4 votes
1 answer
375 views

Monte Carlo Options for Data Augmentation

In the seminal paper by Tanner and Wong (1987) on data augmentation, they describe a method for obtaining the posterior distribution $p(\theta|y)$ by data augmentation. Let $Z$ be a latent variable (...
Winston's user avatar
  • 215
0 votes
0 answers
67 views

Can the same training data be used for MLM and fine-tuning of a transformer model?

Can I use the same training and validation data to perform MLM and train the weights of a classification head? Here is the background of my specific problem: The problem is a binary classification ...
crabnebul's user avatar
1 vote
1 answer
34 views

What is the correct method for training NLP models with augmented data?

I have a very small dataset (~50 rows) for a text classification problem. I found some open source data that's similar to the problem I'm trying to solve. Should I... Train the (BERT) model on the ...
krisjuna's user avatar
1 vote
0 answers
245 views

Does image data augmentation make sense when fine-tuning a transformer-based encoder-decoder model (Donut) on a small dataset (~100 samples)?

I am trying to fine-tune Donut model on ~100 (training) labelled data samples (pairs of images and json files). (Donut is a transformer-based encoder-decoder model, the encoder encodes images and the ...
nim.py's user avatar
  • 250
1 vote
1 answer
45 views

Synthetic vs Data augmentation for low dimensionality data

I have problems understanding data augmentation. I currently have low-dimension features, each data point only has 3 features. A total of 20k non-linear data with only 3 features. I have generated ...
kasyful's user avatar
  • 23
1 vote
0 answers
13 views

Augumentation methods for audio clips (RAVDESS dataset, only audio)

I'm trying to augment the data of the RAVDESS dataset. I still performed many techniques over the data such as: pitch tuning stretching shifting noise adding do you know any other way to augment the ...
Damiano Imola's user avatar
0 votes
0 answers
35 views

Classifier as loss function for image generation, "tricked" too easily

I have a StyleGAN model for which I want to manipulate some properties of the generated image, e.g. eyes being opened/closed [1]. I finetuned a classifier (using pretrained ConvNeXt) on subset of 500 ...
Astralite Heart's user avatar
1 vote
1 answer
291 views

Calculating Shannon Information of Data Augmentation Strategies [closed]

I recently caught Andrew Ng's 2021 talk on MLOps (MLOps: From Model-centric to Data-centric AI). At 26:40, he talks about calculating the effectiveness of cleaning your data (training examples) vs. ...
Nicolai B. Thomsen's user avatar
3 votes
1 answer
61 views

Decide on the number of adversarial samples to include during training

A sample is considered adversarial if it drastically changes classifier's confidence $\in [0,1]$ when given as input. For example, if a spam/ham binary classifier considers some input $X$ to be 0.9 ...
Alexandru Dinu's user avatar
3 votes
4 answers
2k views

Does it make sense to use data augmentation on the Validation set? (note, this is not the same as asking to augment the test set)

Curious, do people use data augmentation on the validation set? I am aware there is a debate for the test set -- but the validation set is usually a split form the train set, so wouldn't it make sense ...
Charlie Parker's user avatar
2 votes
2 answers
945 views

How to perform data augmentation with traditional machine learning algorithms?

I am currently working on a multi-class image classification project, in which I have to use traditional machine learning and feature extraction methods (no convolutional neural networks). I know data ...
brio97's user avatar
  • 21
0 votes
1 answer
435 views

How do we perform hyperparameter tuning on parameter of data augmentation?

I was wondering how do we perform hyperparameter tuning on parameters of data augmentation. Suppose I have to select the best pair of (alpha, alpha) values of beta distribution that works best on data ...
hamza -'s user avatar
  • 23
1 vote
0 answers
76 views

Is it "cheating" to augment data by adding noise to the label?

I have a data set that I'd like to augment by adding noise to the label. I've seen people on Kaggle duplicate each row twice and add 1 and subtract 1 to the label Whenever I do this and then do a ...
NateRob's user avatar
  • 11
0 votes
0 answers
88 views

ROSE acceptable dispersion/shrinkage

To solve imbalanced data, I used oversampling strategy using ROSE algorithm in Python. As you may know, ROSE is a smoothed bootstrapping method and we can control the dispersion of the augmented data. ...
Darren Christopher's user avatar
2 votes
1 answer
2k views

Does data augmentation with white noise improve accuracy of deep learning models?

I was reading Aurélien Géron's Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow. There, on the 14th chapter I read something on data augmentation which I could not be sure of its ...
Kaan Güven's user avatar
0 votes
0 answers
19 views

What is the equivalent of image augmentation in time series forecasting? I'm in need for more data [duplicate]

I think it would be fair for me to explain a little bit on a background into what I am doing so that my question would make more sense. I am currently working at a company where I need to develop a ...
Joseph Anderson's user avatar
2 votes
0 answers
171 views

Overfitting, generalization, data augmentation, regularization, how do they relate to each other? How to measure?

Recent work such as Deep Double Descent shows that overfitting is not really a problem with large models, even without any data augmentation or regularization (L2 weight norm, dropout or so). Edit: Ok,...
Albert's user avatar
  • 1,205
0 votes
0 answers
179 views

Using data augmentation for balancing dataset

Hey there, I have a question about a topic that's been discussed many times before, but to which I could find a satisfying answer. I'm working with a self generated dataset that is comprised of only ...
TheoBoveri's user avatar
1 vote
1 answer
218 views

When augmenting data, shall the dataset keep a balanced ratio

When training a model it is more and more common to augment data. posts indicate that only the training set shall be augmented. On the other hand it is common to split dataset in a fashion following ...
tn3m3lc's user avatar
  • 11
0 votes
0 answers
298 views

Based on Data Augmentation in numerical dataset

I want to develop a machine learning model for a dataset like 5 inputs and 1 output. I stuck in stage with not sufficient dataset, I am aware about data augmentation technique in image and text type ...
Arjun's user avatar
  • 1
1 vote
1 answer
203 views

Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling

This question is relate to Differenciate between two distributions using gibbs sampling . For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl} \...
Abdoul Haki's user avatar
1 vote
1 answer
365 views

Why Accuracy increase only 1% after data augmentation NLP?

i have small dataset 4840 samples (60% negative ,28% positive,12% negative) i use data augmentation on training set (70%train 30% test) and i have about 2000 samples for each class while test is ...
Flavio Torre's user avatar
1 vote
0 answers
48 views

From 1 to 5-shot learning with data augmentation

I'm currently exploring k-shot classification and I would like to start a first experiment on logo classification. The problem is that for some logos I can only find one image, while for others I ...
JulienD's user avatar
  • 11
1 vote
0 answers
337 views

Mask for image padding in semantic segmentation

I'm using data augmentation for a semantic segmentation task, where some images are cropped or rotated. As a result, some padding is added to ensure that the image is always the same size. These ...
gmedina-v's user avatar
6 votes
1 answer
691 views

Does oversampling lead to more overfitting than classweights for really small classes?

Assume I have a couple of thousand hens that I want to classify into those that never lay an egg and those that will at some point in their life lay an egg. Assume that already works perfectly. Now ...
BigBadWolf's user avatar
0 votes
0 answers
82 views

Data augmentation for traditional machine learning algorithms

Data augmentation suffices multiple purposes, I would list a few here: Increasing dataset size: The data is just fragment/stand-in trying to represent reality, having more data should thus result in ...
Imago's user avatar
  • 139
1 vote
0 answers
16 views

Learning Distribution of Data [duplicate]

Sometimes it's important to generate data due to data imbalance issues. I heard that data augmentation by leaning distribution of data is a hot topic now. Could you please give me some resources and ...
Avv's user avatar
  • 249
1 vote
3 answers
186 views

How to generate synthetic data from a balanced dataset?

Let say I have a balanced dataset that has a small training sample size (lack of data). How do I increase the training sample size by generating synthetic data based on the original data? I believe ...
Aqee's user avatar
  • 39
1 vote
1 answer
93 views

What is the best practice to overcome small insufficient data

I have a small number of images (i.e. 108), and I wish to train a deep convolutional neural network with it. As I know - you need to have a large number of samples to be able to train a neural network,...
Melinda Nolls's user avatar
5 votes
1 answer
83 views

In a parametric model, if I do not have enough data, can I estimate the parameter, and simulate data from the estimated model and estimate again?

Suppose I have a logistic regression model $Y_i=\mathbf{1}(X_i\beta>\epsilon_i)$ to estimate, where the distribution of $\epsilon_i$ is known, $X_i$ follows distribution $F_{\theta}$ with an ...
ExcitedSnail's user avatar
  • 2,526
3 votes
1 answer
55 views

Expert Knowledge Acquisition and Machine learning

Having data sets regarding symptoms and diseases such that I use to observe the conditional distributions P(Disease X | Symptom A , Symptom H , Age >20 ) which I use for classification and ...
Latent's user avatar
  • 350
0 votes
0 answers
2k views

Data augmentation by adding noise in python regression model

I am building a regression model for a target variable which is heavy tailed. I want to augment data so that the model gets enough training samples in the region where it's a long tail. Accuracy of ...
user291255's user avatar
1 vote
1 answer
237 views

Train test validation splits and augmentation

I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
please help aahhh's user avatar
1 vote
0 answers
95 views

Data Augmentation causing test and validation sets to be smaller

I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
user291074's user avatar
2 votes
1 answer
24 views

Augmenting training data with cases that won't be in future data

Background: I am working on coding survey responses, where the respondent writes in a description of their job. So the person might write in "McDonald's Employee" and get coded to something like 1002 ...
astel's user avatar
  • 1,498
1 vote
0 answers
169 views

Test Time Augmentation on Validation set?

In the traditional usage of data augmentations, we augment only the train set examples, in order to keep the distribution of the validation and test set equal. In the TTA method, we apply ...
Jjang's user avatar
  • 241