# Smaller standard errors *after* multiple imputation?

I have 1771 observations, with 30% missing data for x1 (Yes:No), and no other missing values from 26 other variables (mix of continuous and factor).

I am using quantile regression in R, with and without imputing values for x1. The parameter estimates for y ~ X1 are similar, but the SEs are actually smaller from the models estimated with the imputed data. This seems true regardless of centile. Have I certainly done something wrong (I am leaning this direction) or can this happen under reasonable circumstances? Happy to provide additional detail. Many thanks.

library(rms)

imputes <- aregImpute(formula, data, n.impute = 100, tlinear = FALSE,  nk = 5)

> qrtest # WITH IMPUTATION

Quantile Regression     tau: 0.5

fit.mult.impute(formula = y ~ x1, fitter = Rq, xtrans = imputes,
data = workDf, tau = 0.5)

Coef      S.E.    t      Pr(>|t|)
Intercept      3560.0000 21.9590 162.12 <0.0001
x1=Yes         -170.0000 29.7172  -5.72 <0.0001

> summary(qrtest2) # NO IMPUTATION

Call: rq(formula = y ~ x1, tau = 0.5, data = workDf)

tau:  0.5

Coefficients:
Value      Std. Error t value    Pr(>|t|)
(Intercept)   3600.00000   27.94167  128.83985    0.00000
x1            -200.00000   36.46074   -5.48535    0.00000


NOTE 1:

Perhaps a clue, from here

fit.mult.impute warns the user that when a fitting routine is not from rms, then the standard errors and significance tests are based only on the last fitted model

Though there were no such warnings since it's using Rq rather than rq as the fitter. Also, calculating SEs as suggested matches up.

NOTE 2: Using rms::ols with imputation leads to larger SEs, as expected, than ols without imputation.

NOTE 3: It is not the result of using different standard errors.

Have I certainly done something wrong ?

No, smaller standard errors are not unusual when using multiple imputation due to the larger sample size compared to the complete cases, as Jonathan Bartlett says in his answer. The extent to which they may be smaller will depend on how many auxiliary variables are used in the imputation model and how strong the associations are between them and the variable(s) being imputed and the number of imputations.

A simple simulation can show this:

require(mice)
require(MASS)

set.seed(1)

# simulate some multivariate normal data
(Sigma <- matrix(c(10,4,0.1,4,6,4,0.1,4,5),3,3))
mu <- c(100,40,30)
N <- 2000
dt <- data.frame(mvrnorm(n=N, mu, Sigma))
names(dt) <- c("Y","X1","X2")

m0 <- summary(lm(Y~X1,data=dt))  # this model represents the "truth"

# make 30% MCAR missingness in X1
dt$X1[sample(1:N,N*0.3,replace=F)] <- NA m1 <- summary(lm(Y~X1,data=dt)) # this model is for complete cases only imp <- mice(dt) fit <- with(imp, lm(Y~X1)) m2 <- summary(pool(fit)) # this model is after imputation with mice defaults  So then we have the following standard errors for X1: Truth: print(m0$coefficients[])
 0.02508949


Complete cases:

print(m1\$coefficients[])
 0.0304495


Imputed:

print(m2[2,2])
 0.02607166


Yes it is certainly possible. If the variables other than Y and X1 are predictive of the X1 variable which you are imputing, multiple imputation will allow you to extract this information and use it to gain information about your target parameters (a regression of Y on X1) from those with X1 missing. e.g. suppose X2 is very highly correlated with X1. Multiple imputation will then be able to impute the missing X1 values with relatively little uncertainty, and your standard errors (relative to complete case analysis, which is based on a smaller sample size) should go down.

• I am using all 27 variables in the MI and most of these are related to each other to some degree. The R2 for x1 is about 0.4, though I'm not sure how useful this is for a dichotomous variable. – D L Dahly Feb 18 '15 at 13:17

I think this could happen, even if you did nothing wrong. If the imputation process is very strong, then the added N will have more effect than the added variation.

If you compare multiple imputation to single imputation, I think the SEs have to be bigger in MI.

(That's just my intuition).

• This is your fault actually. We recently co-reviewed a paper, for which you recomended quantile regression, so into the rabbit hole I went... – D L Dahly Feb 18 '15 at 13:08
• Cool! I do like quantile reg. – Peter Flom Feb 18 '15 at 13:12