Questions tagged [data-augmentation]
Data augmentation is the practice of making slight modifications to the observed data with the goal of making models trained on that data more robust.
71
questions
0
votes
0answers
13 views
Markov Decision Process augmented with latent/hidden variables but does not use a belief state distribution (what do we call this?)
I have a Markov Decision Process where packets arrive to a queue which services them. It has a high cost fast setting and a low cost slow setting. Usually the arrival rates are assumed to follow some ...
0
votes
0answers
12 views
Does data augmentation on the incompleted dataset really improves robustness?
Lots of research use data augmentation(DA) during the training phase, and the result shows that expanding the dataset is feasible for model generalization. The goal of DA is to generalize data ...
0
votes
0answers
9 views
How can I obtain a full-conditional distribution resulting from a transformation that has a dimension reduction?
Problem statement
Suppose we have a function $h(\mathbf{s})=1-[\exp(e^{\beta_0+\beta_1(||\mathbf{s}-\mathbf{x}||)^2})]^{-1}=y$, where both s and x are $1\times2$ vectors, $y$ is a scalar, $\mathbf{s}=(...
4
votes
1answer
63 views
In a parametric model, if I do not have enough data, can I estimate the parameter, and simulate data from the estimated model and estimate again?
Suppose I have a logistic regression model $Y_i=\mathbf{1}(X_i\beta>\epsilon_i)$ to estimate, where the distribution of $\epsilon_i$ is known, $X_i$ follows distribution $F_{\theta}$ with an ...
2
votes
1answer
37 views
Expert Knowledge Acquisition and Machine learning
Having data sets regarding symptoms and diseases such that I use to observe the conditional distributions P(Disease X | Symptom A , Symptom H , Age >20 ) which I use for classification and ...
0
votes
0answers
45 views
Data augmentation by adding noise in python regression model
I am building a regression model for a target variable which is heavy tailed. I want to augment data so that the model gets enough training samples in the region where it's a long tail. Accuracy of ...
0
votes
1answer
16 views
Train test validation splits and augmentation
I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
1
vote
0answers
20 views
Data Augmentation causing test and validation sets to be smaller
I am dealing with an image dataset of 400, and split it into 70% train, 15%test, 15%validation. I would like to do some data augmentation (rotations/flips) to increase the amount of train data I have ...
2
votes
1answer
14 views
Augmenting training data with cases that won't be in future data
Background: I am working on coding survey responses, where the respondent writes in a description of their job. So the person might write in "McDonald's Employee" and get coded to something like 1002 ...
1
vote
0answers
46 views
Test Time Augmentation on Validation set?
In the traditional usage of data augmentations, we augment only the train set examples, in order to keep the distribution of the validation and test set equal.
In the TTA method, we apply ...
0
votes
0answers
34 views
How to train a neural network with an incomplete dataset?
I am currently training a neural network with a dataset containing approximately 10 features and 1000 entries.
The problem is that 70% of the entries contain at least one missing value for at least ...
0
votes
0answers
17 views
Daily timeseries decomposition into seasonal, trend, remainder?
I have time series for each day, that captures the time(s) of the day where a certain event $E$ happens (or alternatively, when it certainly isn't happening). This looks like the following:
...
0
votes
0answers
13 views
Naive data augmentation using a fixed constant
I have a dataset $\underline{\underline{D}}$ (collected at a specific value of an environmental parameter) and I am thinking of augmenting it by $\underline{\underline{D}} + C$ (where $C$ is a fixed ...
21
votes
3answers
2k views
What are the mathematically rigorous data augmentation techniques?
Imagine you have a dataset of 1000 observations. To keep things intuitive imagine they are (x,y) coordinates. They are temporary independent, so that makes it easier.
You wish you had about a million ...
2
votes
0answers
27 views
MCMC converges to MAP and stays at same value - what may go wrong?
I am working on a Gibbs sampler which is complex and I would like to avoid giving all the details here. I will focus on the most necessary details.
The Gibbs sampler involves parameters and latent ...
0
votes
0answers
11 views
Spectra data augmentation
I am working on spectra dataset (magnetic resonance) and would like to perform data augmentation on top of that. I found this paper which doesn't seem to be clear (at least, to me) on what the exact ...
0
votes
0answers
13 views
Image Augmentation and Online learning
I have to question. I can't find any answers online therefore I'm going to ask them here.
Is Image Augmentation in the context of Object detection always meaningful? I have 100 images of a object ...
5
votes
0answers
99 views
Does EM algorithm require us to know the joint (predictive) distribution of the latent variables $Z$ when $Z$ is two-dimensional?
In its general form the E-step of the EM algorithm finds the expectation
$$ Q(\theta|\theta') =\int \log[ p(Y,Z | \theta)] p(Z|Y,\theta') d Z$$
where $Y$ the data, $Z$ the latent variables, $\theta'$...
0
votes
0answers
12 views
Neural ODEs, augmentation and subspace “projection”
The answer to the neural ODE question, the Augmented neural ODEs paper is mentioned. There, the following process happens:
2D data is augmented by padding with 1 zero
3D data is augmented once again ...
0
votes
0answers
44 views
Is it possible to increase dataset size for regression model training?
I have a dataset that can be used for regression model training. However, the problem of this dataset is the size which is very small. So, is it ethical to increase the size of this dataset? If yes, ...
0
votes
0answers
41 views
Generate more data for a small dataset
I have been working on a dataset which has 14 attributes and 303 rows(instances) along with the binary labels.
I want to generate more data so that I could train my neural networks so that I could ...
0
votes
0answers
67 views
Should training set images change during epochs?
I'm training a convolutional neural networks for image segmentation.
In training data preprocessing i'm applying some data augmentation to change luminosity of images.
I'm using tensorflow ...
0
votes
1answer
174 views
is this way of applying data augmentation correct [closed]
I'm training a CNN and want to apply some data augmentation to my input images. I combined some code from tensorflow tutorials and have the following workflow:
I have a dataset containing all ...
8
votes
1answer
111 views
Structure of Generative Adversarial Networks (GAN) for mapping a simulation model
There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy.
The simulation model receives real-valued ...
0
votes
0answers
34 views
Is GAN effective enough to replace data augmentation and manual annotation?
We all know that GAN can be used to augment and expand our dataset Can a GAN be used for data augmentation?. But my question is, is it effective and fast enough?
For example I have done experiment ...
0
votes
0answers
25 views
How does Data Augmentation work for supervised learning models?
I've ran into a few Kaggle competitions where the winning solution used data augmentation, and a new ML platform, which claimed to help with Data Augmentation. Use cases were imbalanced classes and ...
0
votes
0answers
120 views
Can GANs be used for timeseries data augmentation? (2019)
Timeseries, in particular signal timeseries, are distinct in many respects - so GANs working on images may not work for timeseries. Since other questions asking on data augmentation, GANs have ...
0
votes
0answers
23 views
Domain-Agnostic Data Augmentation Techniques?
I was wondering if there are any accurate and widely accepted domain-agnostic data augmentation techniques? Usually they are highly contingent on the data input type (text, images, time-series, ...
1
vote
0answers
21 views
Needing 4th dimension for shape [closed]
I was working on a transfer learning solution to categorize between diseases in the eye. I was using the Xception model built into Keras and it uses a data set that I was able to accumulate. However ...
2
votes
0answers
437 views
GANs for non image data
I'm looking to narrow down the subject for my bachelor thesis:
I am currently working on a project, that only offers a small dataset and there will be no more data incoming for now.
What I'm trying to ...
1
vote
0answers
9 views
Variation in accuracy of data splitting before and after data augmentation
How much accuracy of the system varied/changes between two cases
Data augmentation before splitting
Data augmentation after splitting, only on training data
Is there any literature published?
0
votes
1answer
175 views
Data augmentation on entire dataset before splitting
If I apply rotation of 5 different angles and randomly cropp 10 different images from each rotated image and than divided into training testing and validation. Will it be totally incorrect evaluation ...
1
vote
0answers
138 views
Data augmentation techniques for numeric datasets? [duplicate]
I'm writing a paper about Data Augmentation and I'm looking for some way of increasing the size of a dataset. I'm already aware of the techniques used for images (transformations, PCA, blurs, etc.) ...
1
vote
1answer
243 views
Why is using keras ImageDataGenerator for data augmentation relevent?
I have used keras ImageDataGenerator to generate more data in my neural networks as I have had really small datasets and it has proven itself. As far as I ...
2
votes
1answer
462 views
Can upsampling be used as time-series data augmentation?
I have a timeseries with a sample every 5 minutes. I want to forecast the timeseries multiple step ahead (e.g., 60 minutes, which is 12 samples ahead) using its past values. Unfortunately my model ...
0
votes
1answer
303 views
Augmented data in test/val set
I am about to build CNN for image classification. I have a rather small dataset and have done some data augmentation to make it bigger. While doing so, I got a little confused whether what I am doing ...
2
votes
1answer
544 views
Does online data augmentation make sense?
Data augmentation is popularly done online as that is how it is typically implemented and suggested in neural network frameworks like Keras and TensorFlow. I have also seen it described in e.g. the ...
1
vote
0answers
121 views
Subsampling as a method for time series train/validation splits
I have a question concerning train-test splits for time series data:
Background
I have a dataset of sensor data points for 13 month with datapoints measured every 5 minutes which I downsample to ...
6
votes
1answer
1k views
What are some techniques to augment tabular data?
As we know we can perform data augmentation to "image dataset". We can apply random rotation, shifts, shear and flips over images.
Are there techniques to augment tabular small dataset?
I know the ...
0
votes
1answer
70 views
Image Augmentation or incrementing dataset by flipping/mirroring?
My task is a regression task, where an input image results in another, transformed image. So far so good, works quite well.
As my data set is fairly small, I want to take some actions. Here I wanted ...
0
votes
1answer
247 views
When doing data augmentation, should you train with the original data as well or just the augmented data?
When doing data augmentation in computer vision problems, should you train with the original (un-augmented) data as well or just the augmented data? Are there pros and cons to the two strategies or ...
0
votes
1answer
116 views
Adding Bad Features to Decrease Model Performance
I have a dataset on which some researchers have already performed some definitive data analysis and feature selection. Fitting a model to this dataset returns pretty good accuracy.
In order to ...
2
votes
1answer
289 views
Data augmentation methods for Raman Spectra
I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try?
I have found this paper which suggests ...
0
votes
2answers
683 views
Data Augmentation Techniques for Cat/Binary/Continuous Numerical Dataset
I am using the bank marketing dataset from the UCI ML repo to build an example of a big data storage system along with ETL workflows and Machine Learning models. I would like to create more data so I ...
2
votes
1answer
503 views
How to paraphrase and augment training data for a question answering ML model?
I have only 50 question, answer pairs in my training data, where each question represent a unique intent. However, the training data is too small to build any meaningful ML model.
What are the ...
0
votes
1answer
87 views
Imputing nested time series data with R
Does anyone know what is the superior algorithm to impute data in time series?
I had strong dropouts over time because it was free to participants how many times to participate in my study (otherwise ...
1
vote
0answers
105 views
Active learning for object detection- Batch Selection
I have a small dataset of about 220 images for three classes. I am using YOLO (you only look once) network for an object detection. I am trying to use Active learning in order to reduce the number of ...
2
votes
0answers
120 views
Using bootstrap for robust estimation
I am hoping to understand the process of bootstrapping outlier-contaminated data, and the effects on (simple) OLS estimators.
In particular, we have a DGP,
$$Y_t = \beta X_t + \epsilon_t$$
where $\...
0
votes
1answer
55 views
How do you link multiple X input images to a single ground truth image in machine learning?
What I'm asking is basically manual data augmentation. I have some very specific data augmentation for my inputs so I have to create them first in another software instead of doing it on the fly. I ...
11
votes
4answers
6k views
Can a GAN be used for data augmentation?
Can a generative adversarial network (GAN) be used for data augmentation (i.e. to generate synthetic examples that are added to a dataset)? Would it have any impact on the performance of a model ...
