Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

learn more… | top users | synonyms (1)

1
vote
1answer
23 views

Choosing a regression model based on missing values

I'm trying to predict weight change with an intervention from baseline variables. Literature search yields suggests several predictors. Univariate analyses with weight change as dependent and baseline ...
2
votes
0answers
56 views

Imputation with mice: recode variables before or after imputation?

I am using mice in R, a chained equations (sequential regression) algorithm, to impute a series of polytomous variables (e.g. ...
0
votes
0answers
13 views

What are the advantages of using a Neural Network to impute data?

What are the advantages and drawbacks of using neural network methods to impute data? Is the bias and total error any higher than other methods (e.g., median or mean method, nearest neighborhood, or ...
3
votes
1answer
70 views

How to deal with invalid data values such as with age (e.g., -99, 0, F1)?

I have a data set that consists of 15 age values. I want to clean the data before doing anything further. I have a few questions about data cleaning and data integrity. What is the best treatment if ...
0
votes
0answers
15 views

Imputing categorical variables before binarization

I wish to replace the missing values with mode of that categorical variable. In scikit-learn, we can something like Imputer(strategy="most_frequent", axis=0) but ...
0
votes
0answers
41 views

Using MICE in R: is it possible to impute only sub-sections of the data?

When using the mice library in R to impute data I encounter the following problem. I have a data matrix with missing information ...
2
votes
0answers
36 views

Does it make sense to impute missing covariate data when the imputed value is a function of other covariates in the regression model?

We are building a model that adjusts for standard covariates (e.g., age, gender) and for the outcome at baseline. It would be ideal to adjust for each subject's baseline value like so: $$ Y = ...
0
votes
0answers
29 views

Adjusting time series for methodological change during the time period

The issue is that the national bureau of statistics changed permanently the variable calculation methodology during the overall period. Now I have two official time series for the same variable. One ...
1
vote
0answers
29 views

Confidence estimation for data points in a recommender system

I have a 100 by 100 matrix. Each cell is either 0, 1, or missing (denoted NA). Rows denote 'users' and columns denote 'items'. My goal is to impute the missing values, and provide a confidence level ...
0
votes
0answers
18 views

What is the difference between hot deck and single stochastic imputation?

From what I understand hot deck imputation involves 2 parts 1) Choosing the donor pool: This based on variables related to the missing variable. Maybe using a regression technique to test for ...
0
votes
0answers
62 views

How does knnimpute of the preprocess function work?

I am new to R and I use a script I do not completely understand. It preprocesses a dataset for data mining. At one point, the data (stored in fil) should be ...
2
votes
0answers
23 views

Calculating boxplots with imputed data

I have imputed a dataset which has over 200 variables, and 20 observations. In worst case, 80% of the data is missing, in the best case 100% is available. 5 out of the 20 participants provided data ...
0
votes
0answers
34 views

Multiple Imputation and two-level data

I have a question on multiple imputation where one variable is the sum of several sub-groups. I have about 5 variables with a significant level of missingness. However I have a sixth variable which ...
0
votes
0answers
42 views

Multiple imputation on new data in R

I am looking for a R package that can do multiple imputation on 2 sets of data in the same fashion. That is, I would like to multiply impute the training set and then impute the test set in the same ...
2
votes
0answers
36 views

What are some of the ways to deal with missing data when measuring extreme poverty?

The UNDP have reported that the millennium goal of halving the percentage of people living below 1 USD (PPP) a day has been met (compared to 1990). I was looking at the data for that indicator and ...
2
votes
1answer
162 views

How to run chi-squared test on imputed data

I have a survey data set with missing values and I generated 10 multiple imputations in which the missing values were imputed. There are several categorical variables in the data sets and I'd like to ...
0
votes
0answers
20 views

Mean imputation for control variable with small number missing values

I have a dataset of survey data where ~4% of responses for one of my demographic control variables (AGE) are missing. For the dependent and independent variables that I am interested in, the number of ...
0
votes
0answers
52 views

Impute values with Amelia in R from factor variable

I've a dataset of individuals from which I would like to impute the missing values for 'Age'. Althought the set has several columns, I noticed the most relevant one in respect to Age is the column ...
0
votes
0answers
34 views

Imputing missing values in a count time series with variable effort with the goal of trend estimation

I have a time series monitoring data set that looks like below: The response is a count. ...
0
votes
0answers
26 views

Creating a bound for response variable using softImpute?

I'm working on the Netflix challenge in R and was I'm curious if there is a way to create a bound for the possible responses that the softImpute algorithm can predict. It doesn't look like there is a ...
1
vote
2answers
61 views

Missing value treatment

I have a data set with 18% of AGE variable missing which is an important variable for analysis. Should I try regression imputation or should I drop those observations? Does even regression ...
4
votes
1answer
54 views

Whether to transform non-normal variables prior to performing EM imputation?

I recently received the following email: I have a sample of 100 and approximately 6-7% missing data on each independent variable of interest, and non-normally distributed IVs. I have square root ...
3
votes
1answer
54 views

How to compare different sensitivity thresholds and detection limits?

I have observations taken with different sensitivity thresholds and minimum detection levels, i.e. Lab A is less sensitive and has a minimum detection level of .2 and Lab B is more sensitive and has a ...
4
votes
1answer
79 views

Missing data at random

How does one tell if a dataset is missing data at random? I've been reading up on how to impute missing values, and was wondering what techniques can be used to tell if data is really missing at ...
1
vote
1answer
37 views

How to impute or predict a characteristic when one of the IVs in the prediction is other household members having that same characteristic?

In some data set A, we have: household id, person id, age, sex, and then a simple binary likes donuts / does not like donuts variable. In some other data set B, we ...
1
vote
0answers
52 views

Pool results of 5 Amelia imputed datasets after NLME growth modelling

I have a longitudinal data set and I used Amelia to create 5 imputed sets. Then I use the NLME package in R to do growth ...
0
votes
0answers
17 views

Imputing values using additional data w/o knowing underlying structure [duplicate]

I have a dataset "A" containing ~1.5m product purchases. For ~200k of these products, I have additional product content information in a separate dataset "B". A product number serves as key variable ...
0
votes
0answers
174 views

Combining multiple imputation results for hierarchical regression in SPSS

I'm running a hierarchical regression model in SPSS. I used multiple imputation to handle missing data (14 imputations) and then ran the regression. The regression is: Step 1: 3 dummy coded ...
1
vote
1answer
102 views

What statistical methods are best suited for missing data problems?

The data I'm analyzing has been collected and measured from about 2'000 animals. For every animals, I have a number of discrete values, like species, gender, colour etc., as well as some continuous ...
0
votes
0answers
15 views

Is there a way to replicate imputed data in Amelia?

Every time missing data imputation is done in Amelia package in R, new sets of imputed data are generated. Is there a way to replicate the same imputed data, much like setting seed in random number ...
2
votes
1answer
44 views

Two-way imputation: A Bayesian method for estimating missing scores in tests and questionnaires, and an accurate approximation

I read the article "Two-way imputation: A Bayesian method for estimating missing scores in tests and questionnaires, and an accurate approximation" by Van Ginkel et al. (available here) about a ...
1
vote
0answers
34 views

Fuzzy Record Linkage of Spatial Datasets

I have two datasets describing real-estate properties Dataset 1 describes building characteristics; it includes the location of the entrance to the building along with building descriptions and ...
3
votes
1answer
421 views

Python packages for numerical data imputation [closed]

I am working with multivariate numerical data with a lot of missing values (so dropping all entries or columns with missing data is not an option). Is there a Python package for data imputation? I ...
2
votes
1answer
81 views

Outlier detection/imputation - discussion

Introduction: I'm working with heart rate data. IBI (InterBeat Interval) is defined as the time period between any two consecutive heart beat and is usually measured in millisecond. I have followed a ...
2
votes
0answers
28 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
1
vote
0answers
57 views

Deriving the ordinary kriging equations with noisy data

I am trying to derive the ordinary kriging equations when I want to estimate the underlying value at $T(x_0)$ from the noisy observations at its neighbors $Z(x_1),....Z(x_n)$. $Z(x_0) = T(x_0) + ...
3
votes
4answers
97 views

Alternative for mi impute intreg

Is there a free alternative available for the Stata procedure mi impute intreg (Impute using interval regression)? For example as an R package. I have not found any yet.
1
vote
0answers
25 views

Missing values for different dependent variables

I did an experiment across four weeks to collect data on different dependent variables to answer diverse sub-questions. Since on each dependent variable, different participants did not show up and ...
0
votes
0answers
192 views

Imputation and proximity on new data using randomForest

I am trying to develop a random forest model in R 2.13.1 on Linux x64 using package randomForest 4.6-7, and am having trouble accomplishing a couple of things I was hoping to be able to do. 1) How to ...
2
votes
0answers
61 views

Imputation in normalized signals

I'm currently analyzing a variety of signals. The problem I have is that I have several "missing" values. These "missing" values represent the absence of signal, they are not errors in sampling or ...
4
votes
1answer
396 views

How to combine multiple imputed datasets?

I need a single imputed dataset (e.g. to create a country group dummy from the imputed country per capita income data). R offers packages package for creating multiple imputed data (e.g. Amelia) and ...
1
vote
2answers
125 views

Imputation of missing parameter value by regressing the other parameters

I have a question on imputation of missing values. A support vector machine does not work with missing values, and in practice i thus hence na.omit the rows / cases with missing values on any of the ...
2
votes
0answers
64 views

left-censored dependent variables and prediction

I'm coding up a monte-carlo analysis; I've got a deterministic model that depends on parameters that are uncertain. One of those uncertain parameters is a partially-observed vector of prices by ...
4
votes
2answers
2k views

KNN imputation R packages

I am looking for a KNN imputation package. I have been looking at imputation package (http://cran.r-project.org/web/packages/imputation/imputation.pdf) but for some reason the KNN impute function ...
2
votes
0answers
80 views

Confusion related to conditional Gaussian distribution

I have a certain confusion. I refer to this paper. Let's say I have $p$ variables $x_1, x_2, \dots, x_p$ which follow a multivariate Gaussian distribution. Now suppose I have $N$ examples or samples ...
0
votes
0answers
42 views

Handling Missing Values During Test Phase

I was searching for methods for handling missing values in case of Regression task. There are already few threads but I couldn't find what I was looking for. Suppose I have 4 independent categorical ...
1
vote
0answers
30 views

Fitting missing points to a dataset

I've been assigned to fit some missing values into a large dataset, and I've come across a problem. I've finally got a function describing my data (for simplicity say y=2x^2). Now, for every value of ...
0
votes
1answer
176 views

Time Series Analysis and Forecasting

I am looking at ways to forecast monthly time series data over a larger geographic region. I have time series weather data (e.g., temperature, precipitation) from multiple stations, and the stations ...
2
votes
0answers
43 views

performing logistic regression with imputed variables

I am trying to to run a logistic regression (case-control) and the variable of interest is categorical, taking the values 0 to 6. For a subset of individuals, I do not have the exact value (0, 1 .. or ...
6
votes
2answers
177 views

Imputing a missing variable based on common variables with another data set

I have 2 data sets: $A$ and $B$. The variables are common to both data sets with the exception of two, which are both missing in A. Let's call those two additional variables: $b_1$ and $b_2$. We ...