Questions tagged [data-augmentation]
Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.
73
questions
0
votes
0answers
9 views
StyleGAN as conditional GAN? [closed]
Can you modify the StyleGAN, so that you can generate images only from one specific class?
0
votes
0answers
7 views
Using GAN for image data augmentation (unbalanced dataset)
Lets say I have a image classification problem of 5 classes who are very similar to each other, the only difference is their length, and one class is under-represented. How can I use a GAN to create ...
1
vote
0answers
10 views
Learning Distribution of Data [duplicate]
Sometimes it's important to generate data due to data imbalance issues. I heard that data augmentation by leaning distribution of data is a hot topic now. Could you please give me some resources and ...
0
votes
2answers
19 views
How to generate synthetic data from a balanced dataset?
Let say I have a balanced dataset that has a small training sample size (lack of data).
How do I increase the training sample size by generating synthetic data based on the original data?
I believe ...
0
votes
0answers
10 views
How does RandAugment theoretically improves generalization/robustness of the model?
Recently I have been experimenting with autoaugmentation methods. I have started from RandAugment ( https://arxiv.org/pdf/1909.13719.pdf).
In the paper they also show that magnitude of transformation ...
0
votes
0answers
14 views
Fitting delay distribution to time series data using MCMC
My objective is to estimate the parameters for a delay distribution linking two time series. Suppose X(t) and Y(t) are two sets of daily counts, where the same individuals are counted in both time ...
0
votes
0answers
8 views
Data augmentation for regression problem - Only a part of the instances has surely correct target variable
Context: I'm addressing a regression problem where I have a total of around 2200 instances. I'm not using deep learning for now, but tree-based models, linear regressions and support vector machines. ...
1
vote
1answer
48 views
What is the best practice to overcome small insufficient data
I have a small number of images (i.e. 108), and I wish to train a deep convolutional neural network with it.
As I know - you need to have a large number of samples to be able to train a neural network,...
0
votes
0answers
11 views
Is data augmentation necessary for authentication application models?
I'm working on creating an ECG Biometric Authentication system based CNN whereby the data contains multiple signals from different ECG nodes for each person. And my concern is since augmentation ...
0
votes
0answers
61 views
Proper way to augment tabular data for the regression task?
Existing conditions:
I have 30 gasoline samples:
The spectrum of each sample was measured three times in the near-infrared range. These are 2000 values (features) for one sample, which are used to ...
0
votes
0answers
11 views
Mean and standard deviation in conditioning augmentation
Consider the following statement
we randomly sample the latent variables $\hat{c}$ from an independent
Gaussian distribution $N(\mu(\phi(t)),\sum(\phi(t)))$, where the mean
$\mu(\phi(t))$ and ...
0
votes
0answers
14 views
Markov Decision Process augmented with latent/hidden variables but does not use a belief state distribution (what do we call this?)
I have a Markov Decision Process where packets arrive to a queue which services them. It has a high cost fast setting and a low cost slow setting. Usually the arrival rates are assumed to follow some ...
0
votes
0answers
13 views
Does data augmentation on the incompleted dataset really improves robustness?
Lots of research use data augmentation(DA) during the training phase, and the result shows that expanding the dataset is feasible for model generalization. The goal of DA is to generalize data ...
5
votes
1answer
66 views
In a parametric model, if I do not have enough data, can I estimate the parameter, and simulate data from the estimated model and estimate again?
Suppose I have a logistic regression model $Y_i=\mathbf{1}(X_i\beta>\epsilon_i)$ to estimate, where the distribution of $\epsilon_i$ is known, $X_i$ follows distribution $F_{\theta}$ with an ...
3
votes
1answer
40 views
Expert Knowledge Acquisition and Machine learning
Having data sets regarding symptoms and diseases such that I use to observe the conditional distributions P(Disease X | Symptom A , Symptom H , Age >20 ) which I use for classification and ...
0
votes
0answers
319 views
Data augmentation by adding noise in python regression model
I am building a regression model for a target variable which is heavy tailed. I want to augment data so that the model gets enough training samples in the region where it's a long tail. Accuracy of ...
1
vote
1answer
64 views
Train test validation splits and augmentation
I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
1
vote
0answers
22 views
Data Augmentation causing test and validation sets to be smaller
I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
2
votes
1answer
18 views
Augmenting training data with cases that won't be in future data
Background: I am working on coding survey responses, where the respondent writes in a description of their job. So the person might write in "McDonald's Employee" and get coded to something like 1002 ...
1
vote
0answers
88 views
Test Time Augmentation on Validation set?
In the traditional usage of data augmentations, we augment only the train set examples, in order to keep the distribution of the validation and test set equal.
In the TTA method, we apply ...
0
votes
0answers
56 views
How to train a neural network with an incomplete dataset?
I am currently training a neural network with a dataset containing approximately 10 features and 1000 entries.
The problem is that 70% of the entries contain at least one missing value for at least ...
0
votes
0answers
21 views
Daily timeseries decomposition into seasonal, trend, remainder?
I have time series for each day, that captures the time(s) of the day where a certain event $E$ happens (or alternatively, when it certainly isn't happening). This looks like the following:
...
20
votes
3answers
2k views
What are the mathematically rigorous data augmentation techniques?
Imagine you have a dataset of 1000 observations. To keep things intuitive imagine they are (x,y) coordinates. They are temporary independent, so that makes it easier.
You wish you had about a million ...
2
votes
0answers
45 views
MCMC converges to MAP and stays at same value - what may go wrong?
I am working on a Gibbs sampler which is complex and I would like to avoid giving all the details here. I will focus on the most necessary details.
The Gibbs sampler involves parameters and latent ...
5
votes
0answers
101 views
Does EM algorithm require us to know the joint (predictive) distribution of the latent variables $Z$ when $Z$ is two-dimensional?
In its general form the E-step of the EM algorithm finds the expectation
$$ Q(\theta|\theta') =\int \log[ p(Y,Z | \theta)] p(Z|Y,\theta') d Z$$
where $Y$ the data, $Z$ the latent variables, $\theta'$...
0
votes
0answers
69 views
Generate more data for a small dataset
I have been working on a dataset which has 14 attributes and 303 rows(instances) along with the binary labels.
I want to generate more data so that I could train my neural networks so that I could ...
0
votes
1answer
314 views
is this way of applying data augmentation correct [closed]
I'm training a CNN and want to apply some data augmentation to my input images. I combined some code from tensorflow tutorials and have the following workflow:
I have a dataset containing all ...
8
votes
1answer
116 views
Structure of Generative Adversarial Networks (GAN) for mapping a simulation model
There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy.
The simulation model receives real-valued ...
0
votes
0answers
52 views
Is GAN effective enough to replace data augmentation and manual annotation?
We all know that GAN can be used to augment and expand our dataset Can a GAN be used for data augmentation?. But my question is, is it effective and fast enough?
For example I have done experiment ...
2
votes
0answers
184 views
Can GANs be used for timeseries data augmentation? (2019)
Timeseries, in particular signal timeseries, are distinct in many respects - so GANs working on images may not work for timeseries. Since other questions asking on data augmentation, GANs have ...
1
vote
0answers
28 views
Needing 4th dimension for shape [closed]
I was working on a transfer learning solution to categorize between diseases in the eye. I was using the Xception model built into Keras and it uses a data set that I was able to accumulate. However ...
2
votes
0answers
690 views
GANs for non image data
I'm looking to narrow down the subject for my bachelor thesis:
I am currently working on a project, that only offers a small dataset and there will be no more data incoming for now.
What I'm trying to ...
1
vote
0answers
11 views
Variation in accuracy of data splitting before and after data augmentation
How much accuracy of the system varied/changes between two cases
Data augmentation before splitting
Data augmentation after splitting, only on training data
Is there any literature published?
0
votes
1answer
292 views
Data augmentation on entire dataset before splitting
If I apply rotation of 5 different angles and randomly cropp 10 different images from each rotated image and than divided into training testing and validation. Will it be totally incorrect evaluation ...
1
vote
0answers
275 views
Data augmentation techniques for numeric datasets? [duplicate]
I'm writing a paper about Data Augmentation and I'm looking for some way of increasing the size of a dataset. I'm already aware of the techniques used for images (transformations, PCA, blurs, etc.) ...
1
vote
1answer
266 views
Why is using keras ImageDataGenerator for data augmentation relevent?
I have used keras ImageDataGenerator to generate more data in my neural networks as I have had really small datasets and it has proven itself. As far as I ...
2
votes
1answer
652 views
Can upsampling be used as time-series data augmentation?
I have a timeseries with a sample every 5 minutes. I want to forecast the timeseries multiple step ahead (e.g., 60 minutes, which is 12 samples ahead) using its past values. Unfortunately my model ...
0
votes
1answer
447 views
Augmented data in test/val set
I am about to build CNN for image classification. I have a rather small dataset and have done some data augmentation to make it bigger. While doing so, I got a little confused whether what I am doing ...
2
votes
1answer
693 views
Does online data augmentation make sense?
Data augmentation is popularly done online as that is how it is typically implemented and suggested in neural network frameworks like Keras and TensorFlow. I have also seen it described in e.g. the ...
1
vote
0answers
187 views
Subsampling as a method for time series train/validation splits
I have a question concerning train-test splits for time series data:
Background
I have a dataset of sensor data points for 13 month with datapoints measured every 5 minutes which I downsample to ...
6
votes
1answer
2k views
What are some techniques to augment tabular data?
As we know we can perform data augmentation to "image dataset". We can apply random rotation, shifts, shear and flips over images.
Are there techniques to augment tabular small dataset?
I know the ...
0
votes
1answer
81 views
Image Augmentation or incrementing dataset by flipping/mirroring?
My task is a regression task, where an input image results in another, transformed image. So far so good, works quite well.
As my data set is fairly small, I want to take some actions. Here I wanted ...
0
votes
1answer
303 views
When doing data augmentation, should you train with the original data as well or just the augmented data?
When doing data augmentation in computer vision problems, should you train with the original (un-augmented) data as well or just the augmented data? Are there pros and cons to the two strategies or ...
0
votes
1answer
144 views
Adding Bad Features to Decrease Model Performance
I have a dataset on which some researchers have already performed some definitive data analysis and feature selection. Fitting a model to this dataset returns pretty good accuracy.
In order to ...
2
votes
1answer
360 views
Data augmentation methods for Raman Spectra
I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try?
I have found this paper which suggests ...
0
votes
2answers
946 views
Data Augmentation Techniques for Cat/Binary/Continuous Numerical Dataset
I am using the bank marketing dataset from the UCI ML repo to build an example of a big data storage system along with ETL workflows and Machine Learning models. I would like to create more data so I ...
2
votes
1answer
681 views
How to paraphrase and augment training data for a question answering ML model?
I have only 50 question, answer pairs in my training data, where each question represent a unique intent. However, the training data is too small to build any meaningful ML model.
What are the ...
0
votes
1answer
106 views
Imputing nested time series data with R
Does anyone know what is the superior algorithm to impute data in time series?
I had strong dropouts over time because it was free to participants how many times to participate in my study (otherwise ...
1
vote
0answers
132 views
Active learning for object detection- Batch Selection
I have a small dataset of about 220 images for three classes. I am using YOLO (you only look once) network for an object detection. I am trying to use Active learning in order to reduce the number of ...
2
votes
0answers
154 views
Using bootstrap for robust estimation
I am hoping to understand the process of bootstrapping outlier-contaminated data, and the effects on (simple) OLS estimators.
In particular, we have a DGP,
$$Y_t = \beta X_t + \epsilon_t$$
where $\...
