85
votes
Accepted
Multiple Imputation by Chained Equations (MICE) Explained
MICE is a multiple imputation method used to replace missing data values in a data set under certain assumptions about the data missingness mechanism (e.g., the data are missing at random, the data ...
- 20.9k
13
votes
How much missing data is too much (part 2)? statistical power, effective sample size
It really depends on the situation and what is possible. E.g. telephone survey may only have a 5 to 10% response rate. So, for 90 to 95% of people that were called their answers are missing. Obviously,...
- 35.2k
13
votes
How much missing data is too much (part 2)? statistical power, effective sample size
Something that hasn't come up yet in the answers: It's not the amount of missing data that matters, it's the amount of missing information. If you ask someone's height in inches, and you also ask ...
- 19k
11
votes
Accepted
How to analyze a dichotomous outcome with 50% missing data?
Your description implies that you committed the mortal sin of not pre-specifying the final model in the sense that you tried different models in a way not unlike stepwise variable selection does (...
- 98.5k
10
votes
Accepted
Rubin's rule from scratch for multiple imputations
After multiple imputation of data sets (MI) and analyzing each of the imputed sets separately, Rubin's rules do have you take the mean over those imputations as the point estimate. For inference, ...
- 102k
10
votes
Choosing $m$ value when using multiple imputation (MI)
I believe our current best practice is to use the two-step procedure described in von Hippel (2020) and his Statistical Horizons article, which is to estimate the fraction of missing information (FMI),...
- 36.8k
9
votes
Applying Rubin's rule for combining multiply imputed datasets
You correctly wrote down the pooled estimator:
$$ \bar{U} = \frac{1}{m} \sum_{i=1}^m U_i$$
Where $U_i$ represents the analytic results from the $i$-th imputed dataset. Normally, analytic results ...
- 64.8k
9
votes
Accepted
Multiple imputation of binary endpoint using underlying continuous variable
Rubin's rules work on means and their standard errors, so they are only really valid if a normal approximation is appropriate for your statistic.
PROC FREQ provides ...
- 5,683
9
votes
Accepted
Why is multiple imputation not used more widely in Data Science?
A lot of people in applied fields do not realize that "traditional" methods for handling missing data (e.g., listwise or pairwise deletion) are actually more problematic and rest on stronger ...
- 4,333
8
votes
Accepted
Best way to combine MCMC inference with multiple imputation?
One well-known approach is exactly what you describe (if I understood correctly): i.e. combine the inferences from the analyses of a large number of imputed datasets (each analyzed separately) by just ...
- 35.2k
8
votes
How to analyze a dichotomous outcome with 50% missing data?
If you want to predict what happens at T2 from data at T1, you could run a three classes model with "dropout", "no dropout", and "T2 missing" as the classes.
Note that ...
- 28.3k
7
votes
How to improve running time for R MICE data imputation
You can use quickpred() from mice package using which you can limit the predictors by specifying the mincor (Minimum correlation)...
- 71
7
votes
Accepted
Multiple imputation for missing data in longitudinal study
Multiple imputation is an appropriate approach for your situation but you need to account for the multilevel nature of your data. The observations are nested within participants and this fact needs to ...
- 4,857
7
votes
Accepted
How to use restricted cubic splines with the R mice imputation package
You are right that the imputation model needs to be as rich or richer than the outcome model. The fact that imputation based on full maximum likelihood estimation and imputation done by ...
- 98.5k
7
votes
Choosing $m$ value when using multiple imputation (MI)
While they don't provide a strict criterion in their study, Graham et al., 2007 did a Monte Carlo simulation of different $m$ values and came up with a table of estimates based off that data. Here $\...
- 18.5k
7
votes
Would it be preferable to use statistical imputation instead of a subject matter expert's subjective estimate for missing data?
I would avoid the term "missing data" here.
You are using number of medications as a proxy for disease complexity. For some patients, you can't directly know the number of medications, but ...
- 21.2k
7
votes
How to analyze a dichotomous outcome with 50% missing data?
First, I don't see how variables at time 2 can be sensibly used to predict dropout at time 2. So, I don't think you need to even worry about the missing data. I'm not sure why you collected it. You ...
- 128k
7
votes
Accepted
Assessing model fit in logistic regression with multiple imputation
Section 3.9 of Frank Harrell's Regression Modeling Strategies discusses this matter in some detail, with code.
A suggested approach, based on a paper by Chang and Meng (Statistica Sinica 32: 1489–1514,...
- 102k
7
votes
How much missing data is too much (part 2)? statistical power, effective sample size
Missing data generates two issues:
It potentially generates sample selection bias.
It reduces your effective sample size. In an i.i.d. setting, the effective sample size is just the number of non-...
- 928
6
votes
Accepted
How to combine/pool binomial confidence intervals after multiple imputation?
This is indeed an interesting problem. The issue is that the standard errors that are based on the central limit theorem for proportions are often undesirable because proportions are a computed ...
- 116
6
votes
Accepted
Why do I need to run a model on multiple imputed datasets?
The imputed values on your datasets obtained through multiple imputation are predictions from statistical models themselves, and vary according to probabilistic distributions as any predictions from ...
- 858
6
votes
Accepted
Calculating pooled p-values manually
This is for anyone who is interested, after reading pp. 37-43 in Flexible Imputation of Missing Data by Stef van Buuren. If we call the adjusted degrees of freedom ...
- 2,187
6
votes
Is there an R function that performs LASSO regression on multiple imputed datasets and pools results together?
There is both a technical and a conceptual problem here.
Technically, glm.mids() is designed as part of the mice package to ...
- 102k
6
votes
Accepted
Including dependent variables in multiple imputation model when they have missing values
See Kontopantelis et al. (2017), who describe the proper way to handle this situation. You should definitely retain the DV in the imputation model and use it to impute the predictors. You should use ...
- 36.8k
6
votes
Margins after mice?
The general approach to analysis of missing data using multiple imputation is
create several complete datasets, let's say $m$, using whatever multiple imputation alogorithm you choose
perform the ...
- 65.9k
6
votes
Accepted
Is there a way to impute chi-square data?
mice does not implicitly coerce character variables in your data.frame into categorical ...
- 5,893
6
votes
How much missing data is too much (part 2)? statistical power, effective sample size
The proportion of missingness is simply not a problem. Rather there are two issues to consider:
The power and accuracy of the overall proposed analysis
The validity of the missing data assumptions.
...
- 64.8k
6
votes
How much missing data is too much (part 2)? statistical power, effective sample size
If imputation is what you care about, then what matters is not only the proportion of missing data, the amount of missing information, and the randomness-of-missingness (MCAR vs MAR vs NMAR), but also ...
- 1
5
votes
How to improve running time for R MICE data imputation
I made a wrapper for the mice function that includes one extra argument, droplist, where you can pass a character vector of ...
- 5,447
5
votes
How do the number of imputations & the maximum iterations affect accuracy in multiple imputation?
Until 5 years ago, the most popular rule of thumb was that the number of imputations should be equal to the % of missing information, but it turns out it's not a linear relationship. It's quadratic. ...
Only top scored, non community-wiki answers of a minimum length are eligible