Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")
3
votes
2answers
65 views
KNN imputation R packages
I am looking for a KNN imputation package. I have been looking at imputation package (http://cran.r-project.org/web/packages/imputation/imputation.pdf) but for some reason the KNN impute function ...
1
vote
1answer
25 views
Confusion related to conditional gaussian distribution
I have a certain confusion guys. I was referring to this paper
Lets say I have p variables $x_1, x_2 ... x_p$ which follow a multivariate gaussian distribution. Now suppose I have N examples or ...
0
votes
0answers
34 views
Handling Missing Values During Test Phase
I was searching for methods for handling missing values in case of Regression task. There are already few threads but I couldn't find what I was looking for. Suppose I have 4 independent categorical ...
1
vote
0answers
27 views
Fitting missing points to a dataset
I've been assigned to fit some missing values into a large dataset, and I've come across a problem. I've finally got a function describing my data (for simplicity say y=2x^2). Now, for every value of ...
0
votes
0answers
22 views
Strategy to model, then predict / impute with very sparse variable?
Please excuse vague title. I am currently using an unsupervised SOM clustering approach to try to determine values for a variable that is mostly missing. I have ~8000 observations of 10 variables, the ...
0
votes
2answers
103 views
Time Series Analysis and Forecasting
I am looking at ways to forecast monthly time series data over a larger geographic region. I have time series weather data (e.g., temperature, precipitation) from multiple stations, and the stations ...
3
votes
2answers
118 views
Imputing a missing variable based on common variables with another data set
I have 2 data sets: $A$ and $B$. The variables are common to both data sets with the exception of two, which are both missing in A. Let's call those two additional variables: $b_1$ and $b_2$. We ...
-1
votes
1answer
65 views
Estimating values with which to replace NA [closed]
I have a data set that consists of several timeseries for which I want to calculate estimates within those timeseries to replace NA values. I am familiar with ddply() and lm() but I am having a bit ...
3
votes
1answer
293 views
Imputation with Random Forests
I have two questions on using random forest (specifically randomForest in R) for missing value imputation (in the predictor space).
1) How does the imputation algorithm work - specifically how and ...
1
vote
1answer
80 views
Multiple imputations via MICE package: is there an upper limit for number of imputations?
When performing mulitple imputations with the MICE package in R, I found some resources (http://www.statisticalhorizons.com/more-imputations) that recommend around 10 imputations. I was wondering if ...
2
votes
1answer
130 views
Hot deck imputation: validity of double imputation and selection of deck variables for a regression
Background:
I had a data set containing 212 observations with a lots of missing values. Most of the IVs and DVs are categorical (DVs are ordinal) in nature. There are 3 DVs and about 30 IVs. My ...
6
votes
1answer
63 views
Creating fuzzy values for binary data
I am a little bit rusty at this sort of thing, so I will not be offended if you tell me this is a stupid question. I am doing a logistic regression where my dependent variable is whether or not a ...
1
vote
1answer
88 views
Confusion related to calculation of conditional distribution
I have this confusion related to the calculation of a conditional distribution
suppose $y_n = N(0,w)$
$p(o_n|y_n) = N(D.y_n,\phi)$
How do I calculate
$p(y_n|o_n)$
Actually I was reading this ...
7
votes
1answer
220 views
Using multiple imputation for Cox proportional hazards, then validating with rms package?
I've been researching the mice package, and I haven't yet discovered a way to use the multiple imputations to make a Cox model, then validate that model with the rms package's ...
3
votes
2answers
225 views
Should I use missing value using imputation or listwise or pairwise deletion methods?
I have 60,000 data and around 45% of them is missing and the missing values are random. Can I simply use listwise or pairwise deletion or do I have to use imputation? If imputation is recommended ...
1
vote
0answers
27 views
Constructing a model from multiple non-independent and unreliable predictors?
I have an interesting modelling problem in which I am trying to forecast the occurrence of a type of weather event using an empirical model driven by measurements of a number of different physical ...
2
votes
0answers
83 views
Strategies for Recovering Missing Data
I'm working on the following missing data problem to learn more about stats, probability, and machine learning, but I'm not really making progress solving it:
I have a group of unordered, non-unique ...
1
vote
1answer
304 views
Adjustment for missing values of the categorical variables in a data set
I Have a data set containing about 40 categorical variables. I am trying to factor analyze them. But each categorical variable contains a good number of missing values. Some of them are simply because ...
2
votes
2answers
228 views
How to impute an ordinal variable with MICE but prevent it from taking one value?
I have an ordinal variable, overall_tumor_grade, that can take on values of 1, 2, 3, or X if the measurement is indeterminable. There are some NAs that I want to impute using the mice package in R, ...
5
votes
1answer
153 views
Imputation of a censored variable
I have a medical dataset with approx 200 variables. One of the variables is a bio-marker (concentration of a particular enzyme). It's distribution is right skew, and the problem is that values above a ...
2
votes
0answers
162 views
How do I predict performance for individuals who haven't taken any courses yet?
I'm trying to do a logistic regression on some data.
Here's a simplified version of the situation:
I'm trying to predict student success based on their history, etc. One of my predictors is the ...
2
votes
0answers
124 views
Cross validation for multivariate imputation
I am currently using the mice: Multivariate Imputation by Chained Equations in R (JSS 2011 45(3)) package.
Consider the following example. I am using Sites B to Z and mice() to help infill missing ...
1
vote
1answer
109 views
Why does a model perform worse after reintroducing observations with missing data imputed?
My dataset contains about 12% missing data, and much of the missing data is grouped along observations (not randomly scattered or along columns). I optimized a regression method after removing the ...
0
votes
1answer
394 views
How does the mice imputation function work?
I was wondering if anyone had experience using the mice function, as described in mice: Multivariate Imputation by Chained Equations in R (JSS 2011 45(3))? I have a dataset with a number of variables, ...
3
votes
2answers
228 views
Multiple Imputation for Mixed Effects models
I stumbled accross this related question from 2010, and I wonder if there has been any progress on using multiple imputation for mixed effects models ? I prefer to use R, though Stata is also ...
1
vote
1answer
171 views
Clustering variables with outliers
I am performing a clustering analysis in SAS and some of the variables that I am trying to cluster contain outliers. I've tried to transform the data (log and/or standardize them) but didn't quite ...
2
votes
0answers
52 views
Clustering trajectory patterns with lot of missing values
Is there a problem looking for clusters of trajectory patterns in longitudinal data, when much of the longitudinal values were imputed from the baseline data?
8
votes
2answers
113 views
Imputation to account for systematic error in survey responses
I have a large survey in which students were asked, among other things, their mother's level of education. Some skipped it, and some answered wrongly. I know this, because there a sub-sample of the ...
2
votes
0answers
240 views
How to impute data without missing at random?
Recently I got a global longitudinal data from several countries, and each county has one outcome variable and two predictors from 1995 to 2008. I found one of the predictors is always missing in each ...
4
votes
1answer
157 views
MCAR assumption is plausible should I do MI?
My data consists of measurements on patients with cancer and the variables are some indicators regarding the cancer as well as the stage and the grade of the cancer and some personal info about the ...
6
votes
2answers
374 views
Missing rates and multiple imputation
Is there a limit which is the least acceptable when using multiple imputation (MI)?
For example can I use MI if the missing values in a variable are the 20% of the cases while and other variables ...
2
votes
0answers
166 views
Dealing with R type of variables when doing multiple imputation with the mi package
Volume 45 of the Journal of Statistical Software contains articles about packages that deal with imputation of missing data, one of which is the mi package. In the PDF article regarding that package ...
0
votes
0answers
158 views
Imputation with R and MICE
A short mice-related question as follows: when running simple mice imputation the function goes through well. But complete() ...
3
votes
1answer
677 views
Ordered logistic multilevel regression in imputed dataset
Is there any way to calculate an ordered logistic multilevel regression in an imputed dataset? I have tried Stata’s GLLAMM (Generalized Linear and Latent Mixed ...
3
votes
2answers
304 views
Add a column to a dataframe based on a probability distribution
Suppose I have two dependent categorical attributes A, B. I have a dataframe X that lists probabilities (or expected counts) for all combinations of all categories of A and B. Let's assume two ...
2
votes
1answer
149 views
Imputing/instrumenting for missing variables in a case-control study
I'm combining two surveys in a case-control design. Survey B is drawn from the "case" population, and includes all the variables I need for analysis, plus some extras. Survey A samples a general ...
6
votes
3answers
438 views
Combining two time-series by averaging the data points
I would like to combine the forecasted and backcasted (viz. the predicted past values) of a time-series data set into one time-series by minimizing the Mean Squared Prediction Error.
Say I have time ...
2
votes
1answer
202 views
Imputation with panel data exhibiting dependence structure
Let's say that we have longitudinal panel data. Rows are unique by date and individual. Columns consist of characteristics of the individuals on the given date as well as a dependent variable.
My ...
4
votes
2answers
416 views
Using information on both sides of a 'gap' in time series data for imputation
As with my previous question, I'm looking at ways to impute missing data in a hierarchical time series data.
With al my other procedures, including the experimentation of imputation packages (...
2
votes
2answers
336 views
R function to use for multiple imputation and determining if data is MAR or MCAR
Can anyone tell me which R function to use for multiple imputation?
Also, what should I do to determine if the missing data are MAR or MCAR or not?
2
votes
1answer
635 views
Multiple imputation for missing count data in a time series from a panel study
I am trying to tackle a problem which deals with the imputation of missing data from a panel data study(Not sure if I am using 'panel data study' correctly - as I learned it today.) I have total death ...
4
votes
2answers
1k views
Multiple imputation on single subscale item or subscale scores?
Recently I am conducting a research on the relationship between motivation/attitude variables (Gardner's model) and English language proficiency in the Philippines. I encountered a problem: missing ...
1
vote
2answers
180 views
Best imputation method for stochastic noisy data?
What is the best imputation method for a dataset consisting of stochastic data? For example, let's say you have a table of security returns. In some cases the missings are random, in other cases are ...
2
votes
0answers
260 views
Multiple imputation using SPSS
I am working with a database with missing data. I have done "Roderick J. A. Little’s chi-square statistic" and knew that my data are not MCAR. However, I know don't have can I determine if there are ...
1
vote
2answers
383 views
Imputing missing values in time series using SAS
If I have missing values in a time series that has 40 quarters (ten cycles or ten years) of data, what is the best SAS procedure to use to impute the missing values?
Part 2: I have 390 series (40 ...
5
votes
0answers
218 views
Canonical correlation analysis on a MICE data set
I am looking to do a canonical correlations analysis (CCA) in R, using the CCA package, on a multiply imputed dataset (obtained from the mice package).
I know that ...
6
votes
1answer
89 views
Getting an average measurement based on two raters for cases where data is missing for one rater
Context:
I'm investigating behaviour in a clinical study involving children. I had both parents and teachers completing questionnaires to inform an understanding of the same underlying constructs, ...
5
votes
2answers
249 views
Advice on missing value imputation
I am working on insurance data in which a customer has a field named customer_no_dependent (customer's number of dependent). Its coming out to be a significant ...
2
votes
1answer
587 views
Multiple imputation for clustered data
I have a few questions regarding multiple imputation for nested data.
Context: I have repeated measures (4 times) from a survey and these are clustered in workplaces (205 workplaces). There are about ...
3
votes
1answer
204 views
Means and imputation of log-normal variables
The geometric mean is the appropriate measure of central tendency for log-normally distributed variables. However, the arithmetic mean still has some use in relation to log-normal variables - in ...
