Questions tagged [data-augmentation]
Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.
80
questions
-1
votes
0answers
18 views
Minimal dataset for the classifier [closed]
I have an older dataset of 43 participants who watched emotionally charged footage. There were three phases, initial rest, video and the rest phase after the video. We recorded electrophysiological ...
1
vote
1answer
38 views
Why Accuracy increase only 1% after data augmentation NLP?
i have small dataset 4840 samples (60% negative ,28% positive,12% negative) i use data augmentation on training set (70%train 30% test) and i have about 2000 samples for each class while test is ...
1
vote
0answers
15 views
From 1 to 5-shot learning with data augmentation
I'm currently exploring k-shot classification and I would like to start a first experiment on logo classification.
The problem is that for some logos I can only find one image, while for others I ...
0
votes
0answers
18 views
How to deal with data augmentation for training neural networks?
I'm trying to apply some data augmentation techniques for training my neural network model.
I know that I need to avoid including synthetic data generated from test data in the training data. Also, I'...
0
votes
0answers
25 views
Mask for image padding in semantic segmentation
I'm using data augmentation for a semantic segmentation task, where some images are cropped or rotated. As a result, some padding is added to ensure that the image is always the same size. These ...
6
votes
1answer
59 views
Does oversampling lead to more overfitting than classweights for really small classes?
Assume I have a couple of thousand hens that I want to classify into those that never lay an egg and those that will at some point in their life lay an egg. Assume that already works perfectly.
Now ...
0
votes
0answers
8 views
The labels using MixUp data augmentation in Kernel SVM (dual form)
As we know that mixup data augmentation do convex combination on the label
$\lambda y_i + (1-\lambda)y_j$, $\lambda\sim Beta(\alpha,\alpha)$, assume that in binary classification, our label can only ...
0
votes
0answers
31 views
Data augmentation for traditional machine learning algorithms
Data augmentation suffices multiple purposes, I would list a few here:
Increasing dataset size: The data is just fragment/stand-in trying to represent reality, having more data should thus result in ...
0
votes
0answers
12 views
Show that (binary) cross-entropy is linear w.r.t labels
I am reading up on mixup augmentation. The original idea is that labels are one-hot encoded and then the loss function is just $\ell(\hat{x}, \hat{y}) $where
$$\hat{x} = \lambda x_1 + (1 − \lambda)...
0
votes
0answers
13 views
Using GAN for image data augmentation (unbalanced dataset)
Lets say I have a image classification problem of 5 classes who are very similar to each other, the only difference is their length, and one class is under-represented. How can I use a GAN to create ...
1
vote
0answers
13 views
Learning Distribution of Data [duplicate]
Sometimes it's important to generate data due to data imbalance issues. I heard that data augmentation by leaning distribution of data is a hot topic now. Could you please give me some resources and ...
0
votes
2answers
31 views
How to generate synthetic data from a balanced dataset?
Let say I have a balanced dataset that has a small training sample size (lack of data).
How do I increase the training sample size by generating synthetic data based on the original data?
I believe ...
0
votes
0answers
16 views
How does RandAugment theoretically improves generalization/robustness of the model?
Recently I have been experimenting with autoaugmentation methods. I have started from RandAugment ( https://arxiv.org/pdf/1909.13719.pdf).
In the paper they also show that magnitude of transformation ...
0
votes
0answers
15 views
Fitting delay distribution to time series data using MCMC
My objective is to estimate the parameters for a delay distribution linking two time series. Suppose X(t) and Y(t) are two sets of daily counts, where the same individuals are counted in both time ...
0
votes
0answers
11 views
Data augmentation for regression problem - Only a part of the instances has surely correct target variable
Context: I'm addressing a regression problem where I have a total of around 2200 instances. I'm not using deep learning for now, but tree-based models, linear regressions and support vector machines. ...
1
vote
1answer
52 views
What is the best practice to overcome small insufficient data
I have a small number of images (i.e. 108), and I wish to train a deep convolutional neural network with it.
As I know - you need to have a large number of samples to be able to train a neural network,...
0
votes
0answers
11 views
Is data augmentation necessary for authentication application models?
I'm working on creating an ECG Biometric Authentication system based CNN whereby the data contains multiple signals from different ECG nodes for each person. And my concern is since augmentation ...
0
votes
0answers
97 views
Proper way to augment tabular data for the regression task?
Existing conditions:
I have 30 gasoline samples:
The spectrum of each sample was measured three times in the near-infrared range. These are 2000 values (features) for one sample, which are used to ...
0
votes
0answers
19 views
Mean and standard deviation in conditioning augmentation
Consider the following statement
we randomly sample the latent variables $\hat{c}$ from an independent
Gaussian distribution $N(\mu(\phi(t)),\sum(\phi(t)))$, where the mean
$\mu(\phi(t))$ and ...
0
votes
0answers
17 views
Markov Decision Process augmented with latent/hidden variables but does not use a belief state distribution (what do we call this?)
I have a Markov Decision Process where packets arrive to a queue which services them. It has a high cost fast setting and a low cost slow setting. Usually the arrival rates are assumed to follow some ...
0
votes
0answers
14 views
Does data augmentation on the incompleted dataset really improves robustness?
Lots of research use data augmentation(DA) during the training phase, and the result shows that expanding the dataset is feasible for model generalization. The goal of DA is to generalize data ...
5
votes
1answer
67 views
In a parametric model, if I do not have enough data, can I estimate the parameter, and simulate data from the estimated model and estimate again?
Suppose I have a logistic regression model $Y_i=\mathbf{1}(X_i\beta>\epsilon_i)$ to estimate, where the distribution of $\epsilon_i$ is known, $X_i$ follows distribution $F_{\theta}$ with an ...
3
votes
1answer
43 views
Expert Knowledge Acquisition and Machine learning
Having data sets regarding symptoms and diseases such that I use to observe the conditional distributions P(Disease X | Symptom A , Symptom H , Age >20 ) which I use for classification and ...
0
votes
0answers
578 views
Data augmentation by adding noise in python regression model
I am building a regression model for a target variable which is heavy tailed. I want to augment data so that the model gets enough training samples in the region where it's a long tail. Accuracy of ...
1
vote
1answer
105 views
Train test validation splits and augmentation
I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
1
vote
0answers
37 views
Data Augmentation causing test and validation sets to be smaller
I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
2
votes
1answer
18 views
Augmenting training data with cases that won't be in future data
Background: I am working on coding survey responses, where the respondent writes in a description of their job. So the person might write in "McDonald's Employee" and get coded to something like 1002 ...
1
vote
0answers
108 views
Test Time Augmentation on Validation set?
In the traditional usage of data augmentations, we augment only the train set examples, in order to keep the distribution of the validation and test set equal.
In the TTA method, we apply ...
0
votes
0answers
85 views
How to train a neural network with an incomplete dataset?
I am currently training a neural network with a dataset containing approximately 10 features and 1000 entries.
The problem is that 70% of the entries contain at least one missing value for at least ...
20
votes
3answers
2k views
What are the mathematically rigorous data augmentation techniques?
Imagine you have a dataset of 1000 observations. To keep things intuitive imagine they are (x,y) coordinates. They are temporary independent, so that makes it easier.
You wish you had about a million ...
2
votes
0answers
67 views
MCMC converges to MAP and stays at same value - what may go wrong?
I am working on a Gibbs sampler which is complex and I would like to avoid giving all the details here. I will focus on the most necessary details.
The Gibbs sampler involves parameters and latent ...
5
votes
0answers
103 views
Does EM algorithm require us to know the joint (predictive) distribution of the latent variables $Z$ when $Z$ is two-dimensional?
In its general form the E-step of the EM algorithm finds the expectation
$$ Q(\theta|\theta') =\int \log[ p(Y,Z | \theta)] p(Z|Y,\theta') d Z$$
where $Y$ the data, $Z$ the latent variables, $\theta'$...
0
votes
0answers
92 views
Generate more data for a small dataset
I have been working on a dataset which has 14 attributes and 303 rows(instances) along with the binary labels.
I want to generate more data so that I could train my neural networks so that I could ...
0
votes
1answer
403 views
is this way of applying data augmentation correct [closed]
I'm training a CNN and want to apply some data augmentation to my input images. I combined some code from tensorflow tutorials and have the following workflow:
I have a dataset containing all ...
8
votes
1answer
120 views
Structure of Generative Adversarial Networks (GAN) for mapping a simulation model
There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy.
The simulation model receives real-valued ...
0
votes
0answers
68 views
Is GAN effective enough to replace data augmentation and manual annotation?
We all know that GAN can be used to augment and expand our dataset Can a GAN be used for data augmentation?. But my question is, is it effective and fast enough?
For example I have done experiment ...
2
votes
0answers
232 views
Can GANs be used for timeseries data augmentation? (2019)
Timeseries, in particular signal timeseries, are distinct in many respects - so GANs working on images may not work for timeseries. Since other questions asking on data augmentation, GANs have ...
1
vote
0answers
31 views
Needing 4th dimension for shape [closed]
I was working on a transfer learning solution to categorize between diseases in the eye. I was using the Xception model built into Keras and it uses a data set that I was able to accumulate. However ...
2
votes
0answers
876 views
GANs for non image data
I'm looking to narrow down the subject for my bachelor thesis:
I am currently working on a project, that only offers a small dataset and there will be no more data incoming for now.
What I'm trying to ...
1
vote
0answers
11 views
Variation in accuracy of data splitting before and after data augmentation
How much accuracy of the system varied/changes between two cases
Data augmentation before splitting
Data augmentation after splitting, only on training data
Is there any literature published?
0
votes
1answer
378 views
Data augmentation on entire dataset before splitting
If I apply rotation of 5 different angles and randomly cropp 10 different images from each rotated image and than divided into training testing and validation. Will it be totally incorrect evaluation ...
1
vote
0answers
642 views
Data augmentation techniques for numeric datasets? [duplicate]
I'm writing a paper about Data Augmentation and I'm looking for some way of increasing the size of a dataset. I'm already aware of the techniques used for images (transformations, PCA, blurs, etc.) ...
1
vote
1answer
283 views
Why is using keras ImageDataGenerator for data augmentation relevent?
I have used keras ImageDataGenerator to generate more data in my neural networks as I have had really small datasets and it has proven itself. As far as I ...
2
votes
1answer
749 views
Can upsampling be used as time-series data augmentation?
I have a timeseries with a sample every 5 minutes. I want to forecast the timeseries multiple step ahead (e.g., 60 minutes, which is 12 samples ahead) using its past values. Unfortunately my model ...
0
votes
1answer
514 views
Augmented data in test/val set
I am about to build CNN for image classification. I have a rather small dataset and have done some data augmentation to make it bigger. While doing so, I got a little confused whether what I am doing ...
4
votes
1answer
859 views
Does online data augmentation make sense?
Data augmentation is popularly done online as that is how it is typically implemented and suggested in neural network frameworks like Keras and TensorFlow. I have also seen it described in e.g. the ...
1
vote
0answers
237 views
Subsampling as a method for time series train/validation splits
I have a question concerning train-test splits for time series data:
Background
I have a dataset of sensor data points for 13 month with datapoints measured every 5 minutes which I downsample to ...
6
votes
1answer
2k views
What are some techniques to augment tabular data?
As we know we can perform data augmentation to "image dataset". We can apply random rotation, shifts, shear and flips over images.
Are there techniques to augment tabular small dataset?
I know the ...
0
votes
1answer
99 views
Image Augmentation or incrementing dataset by flipping/mirroring?
My task is a regression task, where an input image results in another, transformed image. So far so good, works quite well.
As my data set is fairly small, I want to take some actions. Here I wanted ...
0
votes
1answer
351 views
When doing data augmentation, should you train with the original data as well or just the augmented data?
When doing data augmentation in computer vision problems, should you train with the original (un-augmented) data as well or just the augmented data? Are there pros and cons to the two strategies or ...
