Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

learn more… | top users | synonyms (1)

0
votes
0answers
21 views

Difference between imputation and interpolation?

When dealing with data sets that have missing values, imputation replaces missing values with substituted values while interpolation replaces missing values with calculated values within some range. ...
0
votes
0answers
6 views

SPSS Mixed model with imputed data and model LLC scores

I am supposed to build several models (it is multilevel analysis of students from different schools). I used multiple imputation for missing data after running MVA and pattern analysis. Now I have 5 ...
1
vote
0answers
43 views

Data Imputation with Amelia on large dataset: Taking very long time [closed]

I have 761,592 obs for 31 variables on users behaviours towards online ads. Out of 31 variables, 28 are categorical. Many cat. variables have more than 10 categories. I am using Amelia for missing ...
0
votes
2answers
55 views

Framework of multiple imputation

I read this paper about ("Multiple Imputation For Missing Data: What Is It And How Can I Use It?")(http://www.csos.jhu.edu/contact/staff/jwayman_pub/wayman_multimp_aera2003.pdf) Does any one have ...
0
votes
0answers
10 views

Imputation of predictors missing data for logistic modelling

I never used imputation of missing data and I would like to understand the effect of imputation in a specific scenario. Lets suppose that I have a dataset whit some predictors variable and one binary ...
0
votes
0answers
16 views

Lower Denomination Imputation

Before I start, I will point out that I am very new to imputing data and so any advice would be greatly appreciated. Apologies if there is an obvious answer that I am overlooking. I have a data set ...
0
votes
2answers
57 views

Change mean imputation in MICE package

MICE Steps The chained equation process can be broken down into six general steps: Step 1: A simple imputation, such as imputing the mean, is performed for every missing value in the dataset. These ...
3
votes
0answers
34 views

Financial time series data: Imputing before or after calculating returns?

I've got several time series of daily prices ($(p_t^j)_{t=1,\dots,n}$ ) of different tradable cards $j=1,\dots,k$. I'd like to calculate the time series of the (log)returns $r_t= ...
0
votes
0answers
33 views

compare different Imputation method by RMSE

My original dataset : ...
0
votes
2answers
49 views

How to create a variable that is present in test data set but not in train?

Im try to do a classification but i have a variable production budget which is present in test dataset and not in train. so how do i proceed. could i impute that variable somehow. i dont want to drop ...
3
votes
0answers
19 views

Imputation of a (weird) multivariate outcome

I am working with a dataset in which the outcome of interest is a vector of dates of particular events: (date_1,date_2,date_3,...,date_n). Some of these outcome vectors are completely missing, but I ...
0
votes
1answer
97 views

How to know which imputation is best for impute my dataset from Multiple imputation by using mice

I used mice package to impute the missing value as follows: ...
0
votes
0answers
12 views

Multiple imputation: manual calculation of new values

I need to impute some data but I find myself in the impossibility of using the model I will eventually use for my analysis as a model of imputation. I have reasons to believe that the var to be ...
1
vote
0answers
47 views

How to use cross validation when you have missing data & rare events?

I am trying to use repeated cross validation to test my classifier. Moreover, I want to use imputation due to missing values and downsampling due to unbalanced data (I have 88% of my data in the ...
0
votes
0answers
20 views

SAS HELP: Imputing both continuous and categorical variables in a single dataset

I am trying to impute data for missing observations within a longitudinal dataset following an arbitrary missing data pattern. Both continuous and categorical variables have missing data. I am ...
0
votes
0answers
17 views

Estimating unexplained variance from multivariate probit output (for imputation protocol)

I have a use case in which survey data underreports program participation, and I need to impute new recipients from within the survey data. There are two (exogenously provided) sub-objectives: ...
1
vote
0answers
33 views

python classification without having to impute missing values

I have a dataset that is working nicely in weka. It has a lot of missing values represented by '?'. Using a decision tree, I am able to deal with the missing values. However, on sci-kit learn, I see ...
3
votes
1answer
159 views

ML covariance estimation from Expectation-Maximization with missing data

Assuming a multivariate normal distribution with missing data, is there a straightforward way to find the maximum likelihood estimate for covariance using an Expectation-Maximization algorithm? NOTE: ...
0
votes
0answers
41 views

Missing data in time series

What is the minimum percentage of missing values that is permitted in time series models like ARIMA or Exponential smoothing? What is the best method to impute missing time series data?
1
vote
0answers
14 views

How to combine multiply imputed datasets created with MICE from different cohorts?

I have data from two separate cohorts. If not imputed I would just rbind() the two datasets and analyse. But due to non-random missing imputation was needed for ...
0
votes
1answer
50 views

Analysis of imputed datasets in Stata 14

I am relatively new to multiple imputation (and statistical analysis in general), so I apologize if my question seems naïve to more experienced users. I am dealing with a somewhat large dataset ...
0
votes
1answer
42 views

Imputation during pre-processing in data mining

I'm trying to comprehend the most appropriate way to utilize machine learning imputation (e.g., KNN) as a pre-processing step in data mining, and I've run into a few questions that don't have clear ...
3
votes
1answer
132 views

Uncertainty in random forest imputations from R missForest package

I am in the process of imputing missing values for my data set that contains approximately 20 variables and 3,000 observations. Most of the missing data values are contained in 2 of the variables (one ...
0
votes
1answer
163 views

Data imputation with preProcess in caret returns less observations than expected

I wonder why preProcess function from R's caret package used for imputation of dataset's missing values returns less observations than in original dataset? For example: ...
0
votes
0answers
31 views

Appropriate methodologies for missing value imputation regarding continuous variables in R

I have a data frame in R of 8 continuous variables in the rows and 60 paired observations in the columns. I want to use this data frame in a subsequent analysis along with gene expression data ...
0
votes
0answers
31 views

filling missing data with other than mean values [duplicate]

What all options are available for filling missing data ? One obvious choice is the mean, but if the percentage of missing data is large, it will decrease the accuracy. So how do we deal with ...
1
vote
0answers
93 views

Filling missing values for categorical feature

In case of continuous variable, the missing values can be filled by the mean of other values, but in case of categorical feature how should we fill in the missing values?
5
votes
0answers
78 views

Use Available Pairs Method for Missing Data in OLS

I have renewed interest in handling missing covariate data in OLS using the pairwise covariance matrix estimator, i.e., using all available pairs of variables in computing variances and covariances. ...
0
votes
0answers
12 views

Which of the following methods captures better estimate of population

Consider the hypothetical setup: X denotes counties in a city. We can have X1, X2,..., Xi. Y denotes hospitals in the city. So let's say for county X1, we have Y11, Y12,..., Y1j, and so on for other ...
0
votes
0answers
82 views

Cox regression with almost 50% missing data

I am preparing a paper for a medical journal in which I discuss the results of a Cox regression model based on the data retrieved from a database. There are categorical and continuous variables. I ...
0
votes
1answer
54 views

Removing the missing values vs. Removing the variables

I know this question seems to be stupid , but in fact it is the first time I deal with a real dataset with a large amount of missing values . The most easiest and fastest way to deal with the missing ...
1
vote
0answers
34 views

Data imputation for a dataset where all values are 1 or N/A

I have a dataset which contains relations between jobs and skills required for these jobs. It is a matrix with value 1 if a skill is required for a job, and N/A otherwise. N/A instead of zero because ...
0
votes
0answers
22 views

What is the acceptable response rate for imputation?

The data I'm working on comes from a clustered survey, e.g., consisting responses from 15 locations. I want to do imputation for a variable. I plan to use mean values of each cluster/location as the ...
2
votes
0answers
71 views

Group wise imputations allowed?

What is the correct way of imputing missing values? We are trying to predict the label of persons belonging to group A or B. The alternative hypothesis is that there are differences in parameters for ...
0
votes
0answers
10 views

How do I distribute answers lacking geo information from a poll?

My data table looks like this: Region Answer1 Answer2 NaN 20 40 Region1 15 17 Region2 18 19 ... So is it possible to distribute answers for which region is not ...
1
vote
1answer
51 views

MICE: what does returned df mean?

In MICE, the object returned by pool() has a component, df, which is included in the summary of the pooled analysis. In my analysis I have about 55,000 cases, but the returned df is higher for most ...
0
votes
0answers
14 views

Checking for compatible conditional models

I am trying to implement MICE in R, using the package mice. I keep reading in papers that if the full conditionals we specify are compatible (they factor into a joint), then our inference is valid. ...
0
votes
0answers
26 views

Displaying data characteristics after multiple imputation

I have original data which I run a few commands on to get a feel for the data. For example, I have men and women, and in each group, I have the percent in each cancer type (eg brain, lung). In the ...
1
vote
4answers
221 views

How do we decide on how to fill missing values in data?

I have a data set with NA values in many predictor variables. How do we impute the best values ? I have 302 variables in total. Out of them 236 belong to some abstract category, 37 to other, 9 to ...
0
votes
1answer
84 views

Impute with the Mean or Median? Instrumental Variables

I am using instrumental variables and I have missing data. In r, I don't believe you can use the MICE package with the AER package. Therefore, I am going to impute with either the mean or median ...
0
votes
1answer
113 views

Imputation and linear regression analysis paradox

Missing values, especially in small datasets, can introduce biases into your model. There are several data imputation methods (MICE, Amelia II), which use EM algorithms to "fill in" the missing ...
1
vote
0answers
26 views

Advice on imputation of multiple time series

Background In the first year of the study 60 streams had temperature data loggers installed (temperature measured every 30 seconds). The second year only 30 of these same streams had data loggers. ...
0
votes
0answers
19 views

Gap-filling biophysical sensor time series

I am exploring imputation methods for filling gaps in time series from multiple co-located biophysical sensors. At a given site, we have about 25 sensors measuring things like temperature, humidity, ...
0
votes
1answer
39 views

How to handle data with 2 variables that have same missingness pattern?

I've not had much academic coursework on imputation, and I can't find anything online or in any texts regarding how one could handle missing data where there are two (or more, possibly?) variables ...
0
votes
1answer
50 views

Regression analysis with non-integer event rates

I am working as part of a team on a large dataset which has been subject to imputation analysis. One of my colleagues has pointed out that the when carrying out regressions that can provide odds ...
0
votes
1answer
142 views

Imputation in R: How to impute univariate data in R?

I am trying classification(2 classes) using Random Forest. Classes are - Red, Green. My dataset contains 1 numeric attributes(called X), and 51 binary attributes to classify a document into red and ...
0
votes
0answers
26 views

Is there an online method to perform multiple imputation?

I have a dataset with a lot of missing data and I am using multiple imputations (with Amelia in R) before performing analysis on it. This dataset is used to train a classification model and to ...
0
votes
0answers
21 views

Bootstrapping and classification tables after multiple imputation

I have used the mice code to do my multiple imputation and it gave me gave me an output for my model as well as a new appended dataset using the "long" code. However, I tried to use this new bigger ...
0
votes
0answers
19 views

Tests for imputed time-series dataset

I am currently dealing with serial measurements, changes of some parameter in patients over time. Values are mostly annual, distance between samples for each person is about a year. I want to perform ...
1
vote
1answer
64 views

Independent variable has a known non-causal relationship with the dependent variable; is it still okay to regress?

To further elaborate on my question, assume that I have a time series dataset of Tax X and Tax Y, where in Tax X is paid by 100% of the sample while Tax Y is paid by 75%. Both taxes differ with ...