3
$\begingroup$

I am running some imputations using the mice package in R. During this process I need to use the as.mids function. However, it seems that as.mids change the values of my subsequent analysis - but I hope I am just doing something wrong.

An example:

Let's say I first impute 5 datasets.

imp1 <- mice(myData, m=5)

To prove my point, I want to duplicate "imp1" using the "complete" function followed by the "as.mids" function

#Creates a long/stacked dataframe from the 5 imputed datasets (+ original)
impData <- complete(imp1, "long", include = TRUE)

#The stacked dataframe is now converted to mids class named "imp2"
imp2 <- as.mids(impData)

Now, as I see it - both imp1 and imp2 should be identical. However, when I run an analysis and pool the data the results turn out to be different.

#Runs both analysis 
fit1 <- with(imp, lm(A ~ B + C + D))
fit2 <- with(imp2, lm(A ~ B + C + D))

#Pools data from both
pooled1 <- pool(fit1)
pooled2 <- pool(fit2)

#Get the results
round(summary(pooled1), 3)
round(summary(pooled2), 3)

Here are some examples from my data

#Results from imp1
              est    se      t       df     Pr(>|t|)
(Intercept)  7.844  0.316  24.832   8.648    0.000  
B           -0.013  0.002  -5.193  10.587    0.000 
C            0.024  0.009   2.821  24.997    0.009  
D            0.248  0.139   1.785   9.425    0.106

#Results from imp2
              est    se      t       df     Pr(>|t|)
(Intercept)  7.756  0.376  20.623   8.320    0.000  
B           -0.011  0.005  -2.361   5.800    0.058 
C            0.024  0.008   2.780  35.214    0.009  
D            0.229  0.140   1.637  11.967    0.128

As you can see, there are some differences. While these difference are not large, it is still unsettling because it indicates that I might have made some mistake or misunderstood something.

Is there someone who can explain why I get these results using as.mids function?

$\endgroup$
2
$\begingroup$

There is indeed a bug somewhere in as.mids.

I used as.mids to convert a stacked dataset of 10 imputations, and then got a list of imputations from it. Weird stuff would happen. Once I randomly got NAs from a value. Other times it started changing values in my final imputation. It also calculated my standard errors wrong.

I then wrote a function:

getImpList <- function(MIdata, impIDColumn){
  d <- list()
  imps <- max(MIdata[,impIDColumn])
  for(i in 1:imps){
    d[[i]] <- MIdata[MIdata[,impIDColumn]==i,]
  }
  return(d)
}

and used:

imputationList(getImpList(MI_data, mi_j_col))

where mi_j_col is the column index of mi imputation ID variable. Everything then worked fine. I double checked R's calculations vs. Stata's using mi estimate and everything checked out.

$\endgroup$
1
$\begingroup$

I also discovered this bug in mice.

My analysis: as.mids assumes that the .imp column is coded as a factor, and therefore subtracts 1 from the max to get the right number of imputations. But if it is not a factor – subtracting 1 will lose the last imputation. Furthermore, 1 is only subtracted the second time the number of imputations are calculated, which is the reason for the random content in the last imputation.

I have edited the original function and commented my changes:

as.mids2 <- function (data, .imp = 1, .id = 2) 
    {
      # paste added to get numerical value if .imp is factor
      ini <- mice(data[data[, .imp] == 0, -c(.imp, .id)], m = max(as.numeric(paste(data[, .imp]))), maxit = 0)
      names <- names(ini$imp)
          if (!is.null(.id)) {
            rownames(ini$data) <- data[data[, .imp] == 0, .id]
          }
          for (i in 1:length(names)) {
            # -1 removed and paste added to get numerical value if .imp is factor
            for (m in 1:(max(as.numeric(paste(data[, .imp]))))) {
              if (!is.null(ini$imp[[i]])) {
                indic <- data[, .imp] == m & is.na(data[data[, .imp] == 0, names[i]])
                ini$imp[[names[i]]][m] <- data[indic, names[i]]
          }
        }
      }
      return(ini)
    }
$\endgroup$
  • $\begingroup$ It appears that this may have been corrected in the most recent release of mice (2.25, 2015-11-09). There is now some code that checks to see if .imp is a factor. $\endgroup$ – Paul de Barros Apr 4 '16 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.