Questions tagged [data-augmentation]

Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

StyleGAN as conditional GAN? [closed]

Can you modify the StyleGAN, so that you can generate images only from one specific class?
0
votes
0answers
7 views

Using GAN for image data augmentation (unbalanced dataset)

Lets say I have a image classification problem of 5 classes who are very similar to each other, the only difference is their length, and one class is under-represented. How can I use a GAN to create ...
1
vote
0answers
10 views

Learning Distribution of Data [duplicate]

Sometimes it's important to generate data due to data imbalance issues. I heard that data augmentation by leaning distribution of data is a hot topic now. Could you please give me some resources and ...
0
votes
2answers
19 views

How to generate synthetic data from a balanced dataset?

Let say I have a balanced dataset that has a small training sample size (lack of data). How do I increase the training sample size by generating synthetic data based on the original data? I believe ...
0
votes
0answers
10 views

How does RandAugment theoretically improves generalization/robustness of the model?

Recently I have been experimenting with autoaugmentation methods. I have started from RandAugment ( https://arxiv.org/pdf/1909.13719.pdf). In the paper they also show that magnitude of transformation ...
0
votes
0answers
14 views

Fitting delay distribution to time series data using MCMC

My objective is to estimate the parameters for a delay distribution linking two time series. Suppose X(t) and Y(t) are two sets of daily counts, where the same individuals are counted in both time ...
0
votes
0answers
8 views

Data augmentation for regression problem - Only a part of the instances has surely correct target variable

Context: I'm addressing a regression problem where I have a total of around 2200 instances. I'm not using deep learning for now, but tree-based models, linear regressions and support vector machines. ...
1
vote
1answer
48 views

What is the best practice to overcome small insufficient data

I have a small number of images (i.e. 108), and I wish to train a deep convolutional neural network with it. As I know - you need to have a large number of samples to be able to train a neural network,...
0
votes
0answers
11 views

Is data augmentation necessary for authentication application models?

I'm working on creating an ECG Biometric Authentication system based CNN whereby the data contains multiple signals from different ECG nodes for each person. And my concern is since augmentation ...
0
votes
0answers
61 views

Proper way to augment tabular data for the regression task?

Existing conditions: I have 30 gasoline samples: The spectrum of each sample was measured three times in the near-infrared range. These are 2000 values (features) for one sample, which are used to ...
0
votes
0answers
11 views

Mean and standard deviation in conditioning augmentation

Consider the following statement we randomly sample the latent variables $\hat{c}$ from an independent Gaussian distribution $N(\mu(\phi(t)),\sum(\phi(t)))$, where the mean $\mu(\phi(t))$ and ...
0
votes
0answers
14 views

Markov Decision Process augmented with latent/hidden variables but does not use a belief state distribution (what do we call this?)

I have a Markov Decision Process where packets arrive to a queue which services them. It has a high cost fast setting and a low cost slow setting. Usually the arrival rates are assumed to follow some ...
0
votes
0answers
13 views

Does data augmentation on the incompleted dataset really improves robustness?

Lots of research use data augmentation(DA) during the training phase, and the result shows that expanding the dataset is feasible for model generalization. The goal of DA is to generalize data ...
5
votes
1answer
66 views

In a parametric model, if I do not have enough data, can I estimate the parameter, and simulate data from the estimated model and estimate again?

Suppose I have a logistic regression model $Y_i=\mathbf{1}(X_i\beta>\epsilon_i)$ to estimate, where the distribution of $\epsilon_i$ is known, $X_i$ follows distribution $F_{\theta}$ with an ...
3
votes
1answer
40 views

Expert Knowledge Acquisition and Machine learning

Having data sets regarding symptoms and diseases such that I use to observe the conditional distributions P(Disease X | Symptom A , Symptom H , Age >20 ) which I use for classification and ...
0
votes
0answers
319 views

Data augmentation by adding noise in python regression model

I am building a regression model for a target variable which is heavy tailed. I want to augment data so that the model gets enough training samples in the region where it's a long tail. Accuracy of ...
1
vote
1answer
64 views

Train test validation splits and augmentation

I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
1
vote
0answers
22 views

Data Augmentation causing test and validation sets to be smaller

I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
2
votes
1answer
18 views

Augmenting training data with cases that won't be in future data

Background: I am working on coding survey responses, where the respondent writes in a description of their job. So the person might write in "McDonald's Employee" and get coded to something like 1002 ...
1
vote
0answers
88 views

Test Time Augmentation on Validation set?

In the traditional usage of data augmentations, we augment only the train set examples, in order to keep the distribution of the validation and test set equal. In the TTA method, we apply ...
0
votes
0answers
56 views

How to train a neural network with an incomplete dataset?

I am currently training a neural network with a dataset containing approximately 10 features and 1000 entries. The problem is that 70% of the entries contain at least one missing value for at least ...
0
votes
0answers
21 views

Daily timeseries decomposition into seasonal, trend, remainder?

I have time series for each day, that captures the time(s) of the day where a certain event $E$ happens (or alternatively, when it certainly isn't happening). This looks like the following: ...
20
votes
3answers
2k views

What are the mathematically rigorous data augmentation techniques?

Imagine you have a dataset of 1000 observations. To keep things intuitive imagine they are (x,y) coordinates. They are temporary independent, so that makes it easier. You wish you had about a million ...
2
votes
0answers
45 views

MCMC converges to MAP and stays at same value - what may go wrong?

I am working on a Gibbs sampler which is complex and I would like to avoid giving all the details here. I will focus on the most necessary details. The Gibbs sampler involves parameters and latent ...
5
votes
0answers
101 views

Does EM algorithm require us to know the joint (predictive) distribution of the latent variables $Z$ when $Z$ is two-dimensional?

In its general form the E-step of the EM algorithm finds the expectation $$ Q(\theta|\theta') =\int \log[ p(Y,Z | \theta)] p(Z|Y,\theta') d Z$$ where $Y$ the data, $Z$ the latent variables, $\theta'$...
0
votes
0answers
69 views

Generate more data for a small dataset

I have been working on a dataset which has 14 attributes and 303 rows(instances) along with the binary labels. I want to generate more data so that I could train my neural networks so that I could ...
0
votes
1answer
314 views

is this way of applying data augmentation correct [closed]

I'm training a CNN and want to apply some data augmentation to my input images. I combined some code from tensorflow tutorials and have the following workflow: I have a dataset containing all ...
8
votes
1answer
116 views

Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy. The simulation model receives real-valued ...
0
votes
0answers
52 views

Is GAN effective enough to replace data augmentation and manual annotation?

We all know that GAN can be used to augment and expand our dataset Can a GAN be used for data augmentation?. But my question is, is it effective and fast enough? For example I have done experiment ...
2
votes
0answers
184 views

Can GANs be used for timeseries data augmentation? (2019)

Timeseries, in particular signal timeseries, are distinct in many respects - so GANs working on images may not work for timeseries. Since other questions asking on data augmentation, GANs have ...
1
vote
0answers
28 views

Needing 4th dimension for shape [closed]

I was working on a transfer learning solution to categorize between diseases in the eye. I was using the Xception model built into Keras and it uses a data set that I was able to accumulate. However ...
2
votes
0answers
690 views

GANs for non image data

I'm looking to narrow down the subject for my bachelor thesis: I am currently working on a project, that only offers a small dataset and there will be no more data incoming for now. What I'm trying to ...
1
vote
0answers
11 views

Variation in accuracy of data splitting before and after data augmentation

How much accuracy of the system varied/changes between two cases Data augmentation before splitting Data augmentation after splitting, only on training data Is there any literature published?
0
votes
1answer
292 views

Data augmentation on entire dataset before splitting

If I apply rotation of 5 different angles and randomly cropp 10 different images from each rotated image and than divided into training testing and validation. Will it be totally incorrect evaluation ...
1
vote
0answers
275 views

Data augmentation techniques for numeric datasets? [duplicate]

I'm writing a paper about Data Augmentation and I'm looking for some way of increasing the size of a dataset. I'm already aware of the techniques used for images (transformations, PCA, blurs, etc.) ...
1
vote
1answer
266 views

Why is using keras ImageDataGenerator for data augmentation relevent?

I have used keras ImageDataGenerator to generate more data in my neural networks as I have had really small datasets and it has proven itself. As far as I ...
2
votes
1answer
652 views

Can upsampling be used as time-series data augmentation?

I have a timeseries with a sample every 5 minutes. I want to forecast the timeseries multiple step ahead (e.g., 60 minutes, which is 12 samples ahead) using its past values. Unfortunately my model ...
0
votes
1answer
447 views

Augmented data in test/val set

I am about to build CNN for image classification. I have a rather small dataset and have done some data augmentation to make it bigger. While doing so, I got a little confused whether what I am doing ...
2
votes
1answer
693 views

Does online data augmentation make sense?

Data augmentation is popularly done online as that is how it is typically implemented and suggested in neural network frameworks like Keras and TensorFlow. I have also seen it described in e.g. the ...
1
vote
0answers
187 views

Subsampling as a method for time series train/validation splits

I have a question concerning train-test splits for time series data: Background I have a dataset of sensor data points for 13 month with datapoints measured every 5 minutes which I downsample to ...
6
votes
1answer
2k views

What are some techniques to augment tabular data?

As we know we can perform data augmentation to "image dataset". We can apply random rotation, shifts, shear and flips over images. Are there techniques to augment tabular small dataset? I know the ...
0
votes
1answer
81 views

Image Augmentation or incrementing dataset by flipping/mirroring?

My task is a regression task, where an input image results in another, transformed image. So far so good, works quite well. As my data set is fairly small, I want to take some actions. Here I wanted ...
0
votes
1answer
303 views

When doing data augmentation, should you train with the original data as well or just the augmented data?

When doing data augmentation in computer vision problems, should you train with the original (un-augmented) data as well or just the augmented data? Are there pros and cons to the two strategies or ...
0
votes
1answer
144 views

Adding Bad Features to Decrease Model Performance

I have a dataset on which some researchers have already performed some definitive data analysis and feature selection. Fitting a model to this dataset returns pretty good accuracy. In order to ...
2
votes
1answer
360 views

Data augmentation methods for Raman Spectra

I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try? I have found this paper which suggests ...
0
votes
2answers
946 views

Data Augmentation Techniques for Cat/Binary/Continuous Numerical Dataset

I am using the bank marketing dataset from the UCI ML repo to build an example of a big data storage system along with ETL workflows and Machine Learning models. I would like to create more data so I ...
2
votes
1answer
681 views

How to paraphrase and augment training data for a question answering ML model?

I have only 50 question, answer pairs in my training data, where each question represent a unique intent. However, the training data is too small to build any meaningful ML model. What are the ...
0
votes
1answer
106 views

Imputing nested time series data with R

Does anyone know what is the superior algorithm to impute data in time series? I had strong dropouts over time because it was free to participants how many times to participate in my study (otherwise ...
1
vote
0answers
132 views

Active learning for object detection- Batch Selection

I have a small dataset of about 220 images for three classes. I am using YOLO (you only look once) network for an object detection. I am trying to use Active learning in order to reduce the number of ...
2
votes
0answers
154 views

Using bootstrap for robust estimation

I am hoping to understand the process of bootstrapping outlier-contaminated data, and the effects on (simple) OLS estimators. In particular, we have a DGP, $$Y_t = \beta X_t + \epsilon_t$$ where $\...