# Something wrong with as.mids from mice package in R?

I am running some imputations using the mice package in R. During this process I need to use the as.mids function. However, it seems that as.mids change the values of my subsequent analysis - but I hope I am just doing something wrong.

An example:

Let's say I first impute 5 datasets.

imp1 <- mice(myData, m=5)


To prove my point, I want to duplicate "imp1" using the "complete" function followed by the "as.mids" function

#Creates a long/stacked dataframe from the 5 imputed datasets (+ original)
impData <- complete(imp1, "long", include = TRUE)

#The stacked dataframe is now converted to mids class named "imp2"
imp2 <- as.mids(impData)


Now, as I see it - both imp1 and imp2 should be identical. However, when I run an analysis and pool the data the results turn out to be different.

#Runs both analysis
fit1 <- with(imp, lm(A ~ B + C + D))
fit2 <- with(imp2, lm(A ~ B + C + D))

#Pools data from both
pooled1 <- pool(fit1)
pooled2 <- pool(fit2)

#Get the results
round(summary(pooled1), 3)
round(summary(pooled2), 3)


Here are some examples from my data

#Results from imp1
est    se      t       df     Pr(>|t|)
(Intercept)  7.844  0.316  24.832   8.648    0.000
B           -0.013  0.002  -5.193  10.587    0.000
C            0.024  0.009   2.821  24.997    0.009
D            0.248  0.139   1.785   9.425    0.106

#Results from imp2
est    se      t       df     Pr(>|t|)
(Intercept)  7.756  0.376  20.623   8.320    0.000
B           -0.011  0.005  -2.361   5.800    0.058
C            0.024  0.008   2.780  35.214    0.009
D            0.229  0.140   1.637  11.967    0.128


As you can see, there are some differences. While these difference are not large, it is still unsettling because it indicates that I might have made some mistake or misunderstood something.

Is there someone who can explain why I get these results using as.mids function?

There is indeed a bug somewhere in as.mids.

I used as.mids to convert a stacked dataset of 10 imputations, and then got a list of imputations from it. Weird stuff would happen. Once I randomly got NAs from a value. Other times it started changing values in my final imputation. It also calculated my standard errors wrong.

I then wrote a function:

getImpList <- function(MIdata, impIDColumn){
d <- list()
imps <- max(MIdata[,impIDColumn])
for(i in 1:imps){
d[[i]] <- MIdata[MIdata[,impIDColumn]==i,]
}
return(d)
}


and used:

imputationList(getImpList(MI_data, mi_j_col))


where mi_j_col is the column index of mi imputation ID variable. Everything then worked fine. I double checked R's calculations vs. Stata's using mi estimate and everything checked out.

I also discovered this bug in mice.

My analysis: as.mids assumes that the .imp column is coded as a factor, and therefore subtracts 1 from the max to get the right number of imputations. But if it is not a factor – subtracting 1 will lose the last imputation. Furthermore, 1 is only subtracted the second time the number of imputations are calculated, which is the reason for the random content in the last imputation.

I have edited the original function and commented my changes:

as.mids2 <- function (data, .imp = 1, .id = 2)
{
# paste added to get numerical value if .imp is factor
ini <- mice(data[data[, .imp] == 0, -c(.imp, .id)], m = max(as.numeric(paste(data[, .imp]))), maxit = 0)
names <- names(ini$imp) if (!is.null(.id)) { rownames(ini$data) <- data[data[, .imp] == 0, .id]
}
for (i in 1:length(names)) {
# -1 removed and paste added to get numerical value if .imp is factor
for (m in 1:(max(as.numeric(paste(data[, .imp]))))) {
if (!is.null(ini$imp[[i]])) { indic <- data[, .imp] == m & is.na(data[data[, .imp] == 0, names[i]]) ini$imp[[names[i]]][m] <- data[indic, names[i]]
}
}
}
return(ini)
}

• It appears that this may have been corrected in the most recent release of mice (2.25, 2015-11-09). There is now some code that checks to see if .imp is a factor. – Paul de Barros Apr 4 '16 at 21:08