Questions tagged [data-imputation]

Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

Filter by
Sorted by
Tagged with
1
vote
0answers
53 views

Missing values for different dependent variables

I did an experiment across four weeks to collect data on different dependent variables to answer diverse sub-questions. Since on each dependent variable, different participants did not show up and ...
2
votes
0answers
141 views

Imputation in normalized signals

I'm currently analyzing a variety of signals. The problem I have is that I have several "missing" values. These "missing" values represent the absence of signal, they are not errors in sampling or ...
8
votes
2answers
16k views

How to combine multiple imputed datasets?

I need a single imputed dataset (e.g. to create a country group dummy from the imputed country per capita income data). R offers packages package for creating multiple imputed data (e.g. Amelia) and ...
2
votes
0answers
222 views

left-censored dependent variables and prediction

I'm coding up a monte-carlo analysis; I've got a deterministic model that depends on parameters that are uncertain. One of those uncertain parameters is a partially-observed vector of prices by ...
16
votes
5answers
46k views

KNN imputation R packages

I am looking for a KNN imputation package. I have been looking at imputation package (http://cran.r-project.org/web/packages/imputation/imputation.pdf) but for some reason the KNN impute function (...
2
votes
0answers
212 views

Confusion related to conditional Gaussian distribution

I have a certain confusion. I refer to this paper. Let's say I have $p$ variables $x_1, x_2, \dots, x_p$ which follow a multivariate Gaussian distribution. Now suppose I have $N$ examples or samples ...
0
votes
2answers
201 views

Handling Missing Values During Test Phase

I was searching for methods for handling missing values in case of Regression task. There are already few threads but I couldn't find what I was looking for. Suppose I have 4 independent categorical ...
1
vote
0answers
47 views

Fitting missing points to a dataset

I've been assigned to fit some missing values into a large dataset, and I've come across a problem. I've finally got a function describing my data (for simplicity say y=2x^2). Now, for every value of ...
0
votes
1answer
703 views

Time Series Analysis and Forecasting

I am looking at ways to forecast monthly time series data over a larger geographic region. I have time series weather data (e.g., temperature, precipitation) from multiple stations, and the stations ...
2
votes
0answers
79 views

performing logistic regression with imputed variables

I am trying to to run a logistic regression (case-control) and the variable of interest is categorical, taking the values 0 to 6. For a subset of individuals, I do not have the exact value (0, 1 .. or ...
7
votes
2answers
607 views

Imputing a missing variable based on common variables with another data set

I have 2 data sets: $A$ and $B$. The variables are common to both data sets with the exception of two, which are both missing in A. Let's call those two additional variables: $b_1$ and $b_2$. We ...
10
votes
3answers
1k views

What is the advantage of imputation over building multiple models in regression?

I wonder if someone could provide some insight into if an why imputation for missing data is better than simply building different models for cases with missing data. Especially in the case of [...
10
votes
2answers
17k views

Imputation with Random Forests

I have two questions on using random forest (specifically randomForest in R) for missing value imputation (in the predictor space). 1) How does the imputation algorithm work - specifically how and ...
1
vote
1answer
506 views

Multiple imputations via MICE package: is there an upper limit for number of imputations?

When performing mulitple imputations with the MICE package in R, I found some resources (http://www.statisticalhorizons.com/more-imputations) that recommend around 10 imputations. I was wondering if ...
5
votes
1answer
3k views

Hot deck imputation: validity of double imputation and selection of deck variables for a regression

Background: I had a data set containing 212 observations with a lots of missing values. Most of the IVs and DVs are categorical (DVs are ordinal) in nature. There are 3 DVs and about 30 IVs. My ...
6
votes
1answer
162 views

Creating fuzzy values for binary data

I am doing a logistic regression where my dependent variable is whether or not a person owns a particular product. Among the variables in the model is an indicator of marital status, 1 for married, 0 ...
2
votes
1answer
274 views

Confusion related to calculation of conditional distribution

I have this confusion related to the calculation of a conditional distribution suppose $y_n = N(0,w)$ $p(o_n|y_n) = N(D.y_n,\phi)$ How do I calculate $p(y_n|o_n)$ Actually I was reading this ...
9
votes
2answers
5k views

Using multiple imputation for Cox proportional hazards, then validating with rms package?

I've been researching the mice package, and I haven't yet discovered a way to use the multiple imputations to make a Cox model, then validate that model with the rms package's ...
6
votes
2answers
4k views

Should I use missing value using imputation or listwise or pairwise deletion methods?

I have 60,000 data and around 45% of them is missing and the missing values are random. Can I simply use listwise or pairwise deletion or do I have to use imputation? If imputation is recommended ...
1
vote
0answers
100 views

Constructing a model from multiple non-independent and unreliable predictors?

I have an interesting modelling problem in which I am trying to forecast the occurrence of a type of weather event using an empirical model driven by measurements of a number of different physical ...
27
votes
5answers
54k views

Imputation of missing values for PCA

I used the prcomp() function to perform a PCA (principal component analysis) in R. However, there's a bug in that function such that the ...
2
votes
0answers
140 views

Strategies for Recovering Missing Data

I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it: I have a group of unordered, non-unique ...
1
vote
1answer
4k views

Adjustment for missing values of the categorical variables in a data set

I Have a data set containing about 40 categorical variables. I am trying to factor analyze them. But each categorical variable contains a good number of missing values. Some of them are simply because ...
4
votes
2answers
4k views

How to impute an ordinal variable with MICE but prevent it from taking one value?

I have an ordinal variable, overall_tumor_grade, that can take on values of 1, 2, ...
10
votes
1answer
1k views

Imputation of a censored variable

I have a medical dataset with approx 200 variables. One of the variables is a bio-marker (concentration of a particular enzyme). It's distribution is right skew, and the problem is that values above a ...
4
votes
1answer
2k views

How do I predict performance for individuals who haven't taken any courses yet?

I'm trying to do a logistic regression on some data. Here's a simplified version of the situation: I'm trying to predict student success based on their history, etc. One of my predictors is the ...
4
votes
0answers
485 views

Cross validation for multivariate imputation

I am currently using the mice: Multivariate Imputation by Chained Equations in R (JSS 2011 45(3)) package. Consider the following example. I am using Sites B to Z and mice() to help infill missing ...
2
votes
1answer
185 views

Why does a model perform worse after reintroducing observations with missing data imputed?

My dataset contains about 12% missing data, and much of the missing data is grouped along observations (not randomly scattered or along columns). I optimized a regression method after removing the ...
9
votes
1answer
13k views

How does the mice imputation function work?

I was wondering if anyone had experience using the mice function, as described in mice: Multivariate Imputation by Chained Equations in R (JSS 2011 45(3))? I have a dataset with a number of variables, ...
6
votes
2answers
1k views

Multiple Imputation for Mixed Effects models

I stumbled accross this related question from 2010, and I wonder if there has been any progress on using multiple imputation for mixed effects models ? I prefer to use R, though Stata is also ...
1
vote
2answers
2k views

Clustering variables with outliers

I am performing a cluster analysis in SAS and some of the variables that I am trying to cluster contain outliers. I've tried to transform the data (log and/or standardize them) but didn't quite work ...
2
votes
0answers
130 views

Clustering trajectory patterns with lot of missing values

Is there a problem looking for clusters of trajectory patterns in longitudinal data, when much of the longitudinal values were imputed from the baseline data?
9
votes
2answers
222 views

Imputation to account for systematic error in survey responses

I have a large survey in which students were asked, among other things, their mother's level of education. Some skipped it, and some answered wrongly. I know this, because there a sub-sample of the ...
3
votes
0answers
389 views

How to impute data without missing at random?

Recently I got a global longitudinal data from several countries, and each county has one outcome variable and two predictors from 1995 to 2008. I found one of the predictors is always missing in each ...
5
votes
1answer
530 views

MCAR assumption is plausible should I do MI?

My data consists of measurements on patients with cancer and the variables are some indicators regarding the cancer as well as the stage and the grade of the cancer and some personal info about the ...
8
votes
2answers
4k views

Missing rates and multiple imputation

Is there a limit which is the least acceptable when using multiple imputation (MI)? For example can I use MI if the missing values in a variable are the 20% of the cases while and other variables ...
2
votes
0answers
1k views

Dealing with R type of variables when doing multiple imputation with the mi package [closed]

Volume 45 of the Journal of Statistical Software contains articles about packages that deal with imputation of missing data, one of which is the mi package. In the PDF article regarding that package ...
0
votes
1answer
424 views

Imputation with R and MICE

A short mice-related question as follows: when running simple mice imputation the function goes through well. But complete() ...
3
votes
1answer
2k views

Ordered logistic multilevel regression in imputed dataset

Is there any way to calculate an ordered logistic multilevel regression in an imputed dataset? I have tried Stata’s GLLAMM (Generalized Linear and Latent Mixed ...
3
votes
2answers
4k views

Add a column to a dataframe based on a probability distribution

Suppose I have two dependent categorical attributes A, B. I have a dataframe X that lists probabilities (or expected counts) for all combinations of all categories of A and B. Let's assume two ...
2
votes
1answer
391 views

Imputing/instrumenting for missing variables in a case-control study

I'm combining two surveys in a case-control design. Survey B is drawn from the "case" population, and includes all the variables I need for analysis, plus some extras. Survey A samples a general "...
10
votes
3answers
4k views

Combining two time-series by averaging the data points

I would like to combine the forecasted and backcasted (viz. the predicted past values) of a time-series data set into one time-series by minimizing the Mean Squared Prediction Error. Say I have time ...
3
votes
1answer
373 views

Imputation with panel data exhibiting dependence structure

Let's say that we have longitudinal panel data. Rows are unique by date and individual. Columns consist of characteristics of the individuals on the given date as well as a dependent variable. My ...
5
votes
4answers
3k views

Using information on both sides of a 'gap' in time series data for imputation

As with my previous question, I'm looking at ways to impute missing data in a hierarchical time series data. With al my other procedures, including the experimentation of imputation packages (...
2
votes
2answers
2k views

R function to use for multiple imputation and determining if data is MAR or MCAR

Can anyone tell me which R function to use for multiple imputation? Also, what should I do to determine if the missing data are MAR or MCAR or not?
10
votes
1answer
7k views

Multiple imputation for missing count data in a time series from a panel study

I am trying to tackle a problem which deals with the imputation of missing data from a panel data study(Not sure if I am using 'panel data study' correctly - as I learned it today.) I have total death ...
5
votes
3answers
5k views

Multiple imputation on single subscale item or subscale scores?

Recently I am conducting a research on the relationship between motivation/attitude variables (Gardner's model) and English language proficiency in the Philippines. I encountered a problem: missing ...
2
votes
2answers
740 views

Best imputation method for stochastic noisy data?

What is the best imputation method for a dataset consisting of stochastic data? For example, let's say you have a table of security returns. In some cases the missings are random, in other cases are ...
2
votes
1answer
368 views

Multiple imputation using SPSS

I am working with a database with missing data. I have done "Roderick J. A. Little’s chi-square statistic" and knew that my data are not MCAR. However, I know don't have can I determine if there are ...
2
votes
2answers
1k views

Imputing missing values in time series using SAS

If I have missing values in a time series that has 40 quarters (ten cycles or ten years) of data, what is the best SAS procedure to use to impute the missing values? Part 2: I have 390 series (40 ...

1
8 9 10
11
12