Questions tagged [synthetic-data]
The synthetic-data tag has no usage guidance.
1
vote
0answers
13 views
Creating synthetic data from existing data
I have data with 8 features and 900 rows each. Those features are dependent on each other in some way. I want to conduct machine learning, but the data size is small.
I was suggested to generate ...
2
votes
0answers
21 views
Generating Correlated Random Variables following Weibull Distribution
I am working on analyzing power generation from geographically distributed wind farms. I have wind speed data that have been collected at multiple locations, but the measurement periods for different ...
0
votes
0answers
35 views
Bayesian inference on multiple datasets
I have a data-set of synthetic observations that contain an x, y and z component.
The model parameters used to make it is known, including the noise level added.
I am using a probabilistic ...
0
votes
1answer
16 views
Sample whole number from distribution with average less than 1
I am trying to create some simulated data, where for each participant, the average number of events that occur in a given week can be any positive number, but usually in the range of 0 to 5. Trouble ...
0
votes
0answers
39 views
Handling imbalanced data for classification [duplicate]
What are the best ways to deal with imbalanced datasets for classifying whether or not individuals pay their tuition? The data is 75% positive class (paid) and 25% negative (unpaid). Some approaches I ...
0
votes
0answers
234 views
“Add White Gaussian Noise with SNR” vs. “Add 5% Gaussian Noise”
I have a noise-free dataset which is a vector of numbers, $\mathbf{d}$, with length $N$.
I want to "add noise" to this data. My understanding is that there are two ways to do this.
1) Add some ...
7
votes
1answer
39 views
Creating an Imbalanced Dataset
I would like to have my trained model tested on an imbalanced dataset. Is there any algorithms available to generate synthetic data from a balanced labelled dataset (spam/non-spam)?
0
votes
0answers
33 views
Constructing a synthetic treatment group using Synth
Synth is a package for R that enables one to construct a synthetic control group.
It constructs a synthetic control group based both on predictors, and on the dependent variable.
https://www....
1
vote
0answers
40 views
Methods for generating regression data whose target follows an arbitrary distribution
I'd like to generate a regression dataset (independent vars that combine somehow into a target or dependent variable). Generating such a dataset is, in general, easy: we can e.g. pick some $x_i$ at ...
3
votes
0answers
39 views
Is it possible to use a time series to construct a synthetic control variable?
Imagine I want to reconstruct a counterfactual euro-dollar exchange rate in 2010 using a synthetic control variable to assess the impact of some policy. Could I use, for example, the exchange rates ...
0
votes
0answers
131 views
Generalizing “Causal Impact” synthetic controls, to multiple outcomes
Does anybody know a way to generalize the use of the Causal Impact google R package to multiple outcome time series?
Say I ran a time series experiment and was able to set up multiple test outcome ...
1
vote
1answer
111 views
Generating synthetic data for robust regression?
I was running some test to see the benefits of using robust regression over ols. For that purposes, I created synthetic data in the following manner, A is the ...
0
votes
2answers
56 views
Generating a synthetic dataset from a latent class model
In latent class analysis, the estimated model paramters are class sizes and item-response probablities in each class.
Based on these basic parameters, is it possible to generate a synthetic sample ...
1
vote
0answers
42 views
How to evaluate whether the mean of the synthetic time series is statistically equal to the historical mean
I am testing a stochastic model that generates synthetic hydrologic time series (five-year series, 60(month) data points). I want to evaluate whether these synthetic time series have the same ...
1
vote
0answers
39 views
Using fake datasets to optimize a model
I have a machine learning application I'm working on that will make predictions based on data about retail stores. Problem is, any data collected by BI regarding the performance of a given retail ...
1
vote
1answer
25 views
Generating a matrix so that correlations amongst columns are close to pre-defined values
I want to generate a matrix of values such that the correlations (e.g., Pearsons correlations) are close to a pre-defined set of values (e.g., ...
0
votes
1answer
46 views
Over-fitting because of Artificial Synthesis of Data
In the process of artificial data synthesis in Machine Learning applications, when we introduce noise to generate new data, is it possible that overfitting can occur because we are essentially ...
0
votes
0answers
257 views
How do I best go about generating synthetic data with anomalies?
I have sensor data for which I wish to build an anomaly detector. This data, unfortunately, does not contain any anomalies. Consequently I would like to generate synthetic data that contains anomalies,...
3
votes
0answers
63 views
Is it possible to determine if a dataset is real or randomly generated?
I've been tasked with developing regression and classification models for time series data. For each observation I have a continuous target for regression and a discrete target for classification.
I'...
0
votes
0answers
97 views
When to use synthetic data and when to use regularization parameters to avoid the over fitting and which is better?
Can anyone explain me when to consider generating the synthetic data or when to consider regularization parameters to reduce the error so the machine learning model will not overfit
0
votes
1answer
29 views
Synthetic Control when leaving treatment
I'm trying to do a comparative analysis using synthetic control to compare a country that went from a dictatorship to democracy to it's dictatorial synthetic twin.
As I understand it, synthetic ...
0
votes
0answers
23 views
Validating decomposition of Synthetic Tensor generated from Unevenly Sampled Tensor
I have a 3-way tensor generated from 7 experiments, with each experiment being matricized and becoming a frontal slice of the tensor (thus mode-3 is of length 7). The data is generated from ...
1
vote
1answer
56 views
How to build Predictive models with insufficient historical/performance data
I'm building a auto loan probability of default model where the loan term could be 3 to 7 years and hence default can happen anytime in that interval. But we are a start-up and have only 3 years of ...
3
votes
1answer
494 views
Create synthetic data with a given intraclass correlation coefficient (ICC)?
I want to generate some synthetic data with $I$ observations across $J$ clusters. Additionally, I want the intraclass correlation coefficient ($ICC$) to be an input of my data generation process. So, ...
0
votes
2answers
190 views
Generate non-random data that follows a given correlation matrix
Problem: I have to generate in R a synthetic dataset which contains three different variables that didn't follow any known distribution.
What I did until now: Grouping historical values of each ...
3
votes
0answers
319 views
How to generate synthetic data with a given $R_{x,y}^2$
I would like to generate some data with the following relationships:
$ y = x\beta + T\delta + \varepsilon $
$ R_{x,y}^2 = a $, where $a$ is a number that I can choose when generating the data
$ \...
0
votes
1answer
261 views
How to evaluate the quality of a synthetic dataset?
I'm still playing with the data related to the year 2008 of the "Household power consumption" dataset (free to download at UCI Machine Learning Repository). I was able to generate some synthetic data ...
3
votes
1answer
314 views
Synthetic data behaviour different from real data
I'm studying in R a way to generate synthetic data starting from real world data and, in particular, I'm focusing on the year 2008 of the "Household power consumption" dataset (free to download at UCI ...
0
votes
0answers
180 views
Generating synthetic GPS trajectories based on a sample of real data
I have some moving object trajectories for vehicles (with vehicle ID numbers labelling each lat/lon co-ordinate), and I would like to generate some more data that is synthetic, but realistic. Ideally, ...
2
votes
0answers
2k views
SMOTE using unbalanced package in R fails on simple simulated data
SMOTE is a popular method to generate synthetic examples of the minority class in an unbalanced-class data set.
I am trying out SMOTE in the "unbalanced" package in R. I am generating a simple ...
0
votes
1answer
193 views
regarding generating synthetic data simulating the real data
We are trying to develop some predictive models. The current scenario is that we have to rely on synthetic data at first since the real data set will not be available quite soon. It is understandable ...
0
votes
0answers
351 views
Monte Carlo Simulations: Can I Use Real Data as Universe?
In Monte Carlo simulations, it is a commonly used procedure to generate synthetic data based on a large survey (e.g. a microcensus) first. These synthetic data is then used as universe/population for ...
0
votes
1answer
200 views
Synthetic datasets for concept drifting data
Is there any synthetic / artificial datasets for concept drifting data? I want to visualize the performance of some clustering algorithm when data experiences concept drift and changes over time.
0
votes
1answer
69 views
How to create synthetic mortality data set?
Mortality data for certain countries is available for at the Human Mortality Database (www.mortality.org). However, I would like to develop a multipopulation model that first estimates a mortality ...
1
vote
1answer
2k views
Placebo tests in Synthetic Control Method
I am trying to make graphs for the placebo tests under synthetic control, however I seem to be doing something wrong as I am not being able to merge the files. I am using the dataset of Abadie(2010) ...
1
vote
0answers
53 views
Relation between bootstrap mean and parameter value estimated via maximum likelihood
I have an observed data set $O$ and a synthetic model $S(\theta)$ which attempts to describe it.
By fixing $\theta$ to different values I can generate $M$ synthetic
realizations of the model:
$$S_k =...
7
votes
2answers
267 views
Approaches for generating synthetic survey data with dependent answers?
I would like to produce synthetic survey data. At the moment I produce independent answers between questions according to an arbitrary discrete distribution as in this question.
I want to generate ...
0
votes
0answers
83 views
Create a continuous variable not related to binary target
I would like to create a continuous variable in such a way that it has least predictive information with respect to binary target variable.
That continuous variable needs to have good relationship ...
5
votes
2answers
685 views
Synthetic Control Method
I came across this journal http://www.hks.harvard.edu/fs/aabadie/ccsp.pdf which basically uses Synthetic Control Method (SCM) to estimate the difference between the impact on a variable when an event ...
0
votes
0answers
51 views
Simulate microarray technical error
I need to simulate some microarray experiment datasets. I have the levels of expression of a set of synthetic genes in different experimental conditions. For simplicity of the method this levels are ...
0
votes
0answers
392 views
Generating artificial experimental data to test model fitting software
I wish to test some model fitting software and I would like to generate synthetic datasets for this test. The synthetic data is supposed to originate from a experiment that measures the change in ...
3
votes
1answer
583 views
Generating non-homogeneous spatial Gaussian data
I want to generate a spatial data following multivariate Gaussian distribution.
However, I don't want it to be homogeneous, meaning I don't want the correlation/covariance to be homogeneous. I want ...
1
vote
0answers
207 views
Generate synthetic data from a graph (e.g., bipartite)
Does anyone know of literature describing methods for generating synthetic data from a graphical structure that has been learned from observations of real data? The graph, depending on the type of ...
1
vote
0answers
525 views
Simulated Microarray data [closed]
I would like to generate synthetic microarray data sets for simulation purposes.
The web and literature search so far returned the following:
Microarray Simulator - It provides Matlab skripts to ...
3
votes
1answer
1k views
Synthetic time series generation
I have a linear model (with seasonal dummy variables) that produces monthly
forecasts. I'm using R together with the 'forecast' package:
...
0
votes
0answers
175 views
Null distribution on synthetic data
I need a sort of review of all (or most of) available methods to create null distributions (as reference) to use to compare a result. For ex: if I want to validate a network I can create a set of 1000 ...
3
votes
0answers
1k views
Why do I get 100% error rate in unsupervised random forest, and how do unsupervised patterns work in “randomForest” R package
I tried to use random forest to classify microarray data. Basing on research of L.Breiman and Tao Shi, I constructed a synthetic data base using bootstrap methods (Assuming it is a matrix with samples ...
2
votes
0answers
170 views
Generating likely populations given a subsample and control totals
Context
In transportation planning, agent-based microsimulation is a method to deal with the complexity of the problem. Instead of computing aggregate flows (as in the classical four-step model), ...
24
votes
2answers
14k views
What are some standard practices for creating synthetic data sets?
As context: When working with a very large data set, I am sometimes asked if we can create a synthetic data set where we "know" the relationship between predictors and the response variable, or ...
