Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

learn more… | top users | synonyms (1)

0
votes
0answers
18 views

Predicting missing values in data [on hold]

I have a dataset that has some missing values. How can I use basic statistics or mathematics to predict the missing values? Right now, I am considering the value before the missing value and the ...
0
votes
0answers
14 views

Missing Value in Commodities Prices

I am trying to model the prices of four energetic commodities with ARIMA models in R. Unfortunately the price series is not regular, as for some days, like Christmas, no price is given. My series is ...
0
votes
1answer
17 views

Overfitting on Missing Value Imputations

When performing Missing Value imputations, should we be concerned about overfitting the data? Why or why not? For example: If I impute a variables missing value using a CART regression tree, should ...
1
vote
0answers
10 views

J48 Handling Missing Value with Tree based Imputation

Aloha, currently i have some trouble and question zu implement some kind of special missing value handling in WEKA J48 algorithm using WEKA JAVA API. I want to test the performance of SHAPIRO ...
1
vote
0answers
20 views

Predict/impute one cell of matrix using all other cells

The question: I want to predict/impute one missing cell of a matrix using the contents of all other cells. Anyone have ideas on how to do this? The context: The matrix is n people's responses to m ...
-1
votes
1answer
33 views

Imputation and Distributions

Suppose you impute a variable using a normal distribution with mean 10 and sd 5. Is it better to draw 1000 random samples from this normal distribution, take the average, and then use this to impute ...
3
votes
1answer
28 views

How to get only positive values when imputing data?

Suppose age is normally distributed with mean 20 and standard deviation 5. How do you ensure that you get only positive values when you sample age from this distribution? I am trying to impute ...
0
votes
1answer
38 views

using cluster information in multiple imputation

i need to impute a dataset all categorical variables before doing analysis. I can just simply impute with mode of all data or a variable. I belief that better option will be to classify the subjects ...
0
votes
3answers
125 views

using random forest for missing data imputation in categorical variables ( in R)

I have following type of associated data. The following example step to generate associated variable. p number of variables and n is number of observations. ...
1
vote
1answer
60 views

Individual and overall RMSE for multivariate data

I have a dataset which contains missing values, and I'm using imputation packages (Rs mi and ...
4
votes
3answers
244 views

How to handle with missing values in order to prepare data for feature selection with LASSO?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
0
votes
1answer
27 views

How do you suppress imputed data and only display results for the pooled data?

I've done 15 imputations and am now running various tests on the new data, but all I'm really interested in is the pooled results at the end. The long list of imputations (especially in a test like an ...
1
vote
2answers
44 views

Imputing missing observation in multivariate time series

Suppose I have a dataframe consisting of six time series. In this dataframe, some observations are missing, meaning at some timepoints all time series contain a NA-value. In R, one possible imputation ...
0
votes
1answer
26 views

Why is collinearity a problem when imputing missing values?

I'm imputing missing values using R's mice package. My data has three numeric variables and a class variable so I am using a ...
0
votes
0answers
45 views

Validation - correctly compare and validated imputation models

I've seen a lot of interesting questions here about multiple imputation and also great answers that helped me a lot to impute my data. I've used Predictive Mean Matching, EMB and I would like to use ...
0
votes
0answers
48 views

Imputation by regression in R

Say I have below example data, where rows are observations and columns are variables, and NAs stand for missing values. ...
0
votes
0answers
27 views

Adding values for missing dates in time-series data using SAS

I am having trouble figuring out how to create entries for missing dates in my time-series data. I'm trying to use proc expand and I'm able to get entries for dates between the already existing dates ...
0
votes
1answer
16 views

svm manual implementation to find missing values

how svm algorithm used to impute the missing data in a dataset. I need a manual implementation of svm algorithm to find missing values with example
11
votes
2answers
107 views

using neighbor information in imputing data or find off-data (in R)

I have dataset with assumption that nearest neighbors are best predictors. Just a perfect example of two-way gradient visualized- Suppose we have case where few values are missing, we can easily ...
7
votes
5answers
235 views

How to perform imputation of values in very large number of data points?

I have a very large dataset and about 5% random values are missing. These variables are correlated with each other. The following example R dataset is just a toy example with dummy correlated data. ...
1
vote
1answer
67 views

optimal scaling / CATREG for imputed data

I have a data set with 5 different kinds of nutrient statuses and I want to see whether they are associated with categorical / ordinal grades at school. I have multiple covariates which I will ...
1
vote
1answer
55 views

Fast missing data imputation in R for big data that is more sophisticated than simply imputing the means?

I need a package for missing data imputation in R. But since I am dealing with big data, the number of missing data entries can also be high. The packages which impute using mean or median are of ...
0
votes
2answers
61 views

Simultaneous imputation of multiple binary variables in R

I have a dataset with multiple correlated binary variables (0/1). Can anyone point me toward a solution for imputing completely random missing values based on information in the other variables? ...
3
votes
0answers
80 views

Is the following procedure to measure the quality of an imputation ok?

I'd like to compare different kinds of imputation techniques, i.e. methods which allow to fill missing data fields in a data frame. For now, I'm only using the R package ...
1
vote
0answers
19 views

Should I use missing data imputation with a model that already allows incomplete data?

I'm just starting to learn about missing data imputation methods, and I'm confused. In every introduction I've read, the author starts by describing listwise deletion and says that it's a bad idea ...
1
vote
2answers
58 views

Missing data due to absent parent

I am using the following regression: $$\text{Test score} = \beta_0+\beta_1\text{Mother's employment}+\beta_2\text{Mother's education}$$ where "Mother's employment" is a set of dummy variables ...
2
votes
0answers
35 views

What to do when exploratory factor analysis results are different for complete-cases and imputed data?

I have a hundred items that I'm performing EFA on, with around 370 complete cases. Using parallel analysis to determine the number of factors to extract, EFA gave 9 factors, all of which make ...
2
votes
1answer
57 views

Does it make sense to impute year of birth?

This is data cleaning and preparation stage question for me. I apologize if the question is basic, but I am a beginner. I have a dataset of a bit less than 4500 records. This is a survey and ...
1
vote
1answer
36 views

Choosing a regression model based on missing values

I'm trying to predict weight change with an intervention from baseline variables. Literature search yields suggests several predictors. Univariate analyses with weight change as dependent and baseline ...
2
votes
0answers
73 views

Imputation with mice: recode variables before or after imputation?

I am using mice in R, a chained equations (sequential regression) algorithm, to impute a series of polytomous variables (e.g. ...
0
votes
0answers
19 views

What are the advantages of using a Neural Network to impute data?

What are the advantages and drawbacks of using neural network methods to impute data? Is the bias and total error any higher than other methods (e.g., median or mean method, nearest neighborhood, or ...
3
votes
1answer
97 views

How to deal with invalid data values such as with age (e.g., -99, 0, F1)?

I have a data set that consists of 15 age values. I want to clean the data before doing anything further. I have a few questions about data cleaning and data integrity. What is the best treatment if ...
0
votes
0answers
32 views

Imputing categorical variables before binarization

I wish to replace the missing values with mode of that categorical variable. In scikit-learn, we can something like Imputer(strategy="most_frequent", axis=0) but ...
0
votes
0answers
70 views

Using MICE in R: is it possible to impute only sub-sections of the data?

When using the mice library in R to impute data I encounter the following problem. I have a data matrix with missing information ...
2
votes
0answers
45 views

Does it make sense to impute missing covariate data when the imputed value is a function of other covariates in the regression model?

We are building a model that adjusts for standard covariates (e.g., age, gender) and for the outcome at baseline. It would be ideal to adjust for each subject's baseline value like so: $$ Y = ...
0
votes
0answers
32 views

Adjusting time series for methodological change during the time period

The issue is that the national bureau of statistics changed permanently the variable calculation methodology during the overall period. Now I have two official time series for the same variable. One ...
1
vote
0answers
36 views

Confidence estimation for data points in a recommender system

I have a 100 by 100 matrix. Each cell is either 0, 1, or missing (denoted NA). Rows denote 'users' and columns denote 'items'. My goal is to impute the missing values, and provide a confidence level ...
0
votes
0answers
29 views

What is the difference between hot deck and single stochastic imputation?

From what I understand hot deck imputation involves 2 parts 1) Choosing the donor pool: This based on variables related to the missing variable. Maybe using a regression technique to test for ...
0
votes
1answer
158 views

How does knnimpute of the preprocess function work?

I am new to R and I use a script I do not completely understand. It preprocesses a dataset for data mining. At one point, the data (stored in fil) should be ...
2
votes
0answers
27 views

Calculating boxplots with imputed data

I have imputed a dataset which has over 200 variables, and 20 observations. In worst case, 80% of the data is missing, in the best case 100% is available. 5 out of the 20 participants provided data ...
0
votes
0answers
43 views

Multiple Imputation and two-level data

I have a question on multiple imputation where one variable is the sum of several sub-groups. I have about 5 variables with a significant level of missingness. However I have a sixth variable which ...
0
votes
0answers
68 views

Multiple imputation on new data in R

I am looking for a R package that can do multiple imputation on 2 sets of data in the same fashion. That is, I would like to multiply impute the training set and then impute the test set in the same ...
2
votes
0answers
41 views

What are some of the ways to deal with missing data when measuring extreme poverty?

The UNDP have reported that the millennium goal of halving the percentage of people living below 1 USD (PPP) a day has been met (compared to 1990). I was looking at the data for that indicator and ...
3
votes
1answer
299 views

How to run chi-squared test on imputed data

I have a survey data set with missing values and I generated 10 multiple imputations in which the missing values were imputed. There are several categorical variables in the data sets and I'd like to ...
0
votes
0answers
26 views

Mean imputation for control variable with small number missing values

I have a dataset of survey data where ~4% of responses for one of my demographic control variables (AGE) are missing. For the dependent and independent variables that I am interested in, the number of ...
0
votes
0answers
85 views

Impute values with Amelia in R from factor variable

I've a dataset of individuals from which I would like to impute the missing values for 'Age'. Althought the set has several columns, I noticed the most relevant one in respect to Age is the column ...
0
votes
0answers
46 views

Imputing missing values in a count time series with variable effort with the goal of trend estimation

I have a time series monitoring data set that looks like below: The response is a count. ...
0
votes
0answers
31 views

Creating a bound for response variable using softImpute?

I'm working on the Netflix challenge in R and was I'm curious if there is a way to create a bound for the possible responses that the softImpute algorithm can predict. It doesn't look like there is a ...
1
vote
2answers
88 views

Missing value treatment

I have a data set with 18% of AGE variable missing which is an important variable for analysis. Should I try regression imputation or should I drop those observations? Does even regression ...
4
votes
1answer
67 views

Whether to transform non-normal variables prior to performing EM imputation?

I recently received the following email: I have a sample of 100 and approximately 6-7% missing data on each independent variable of interest, and non-normally distributed IVs. I have square root ...