Converting R data frame to mids object with as.mids returns - error ini$imp[[i]] : subscript out of bounds [closed]

Question

I am currently working on a project addressing individuals' sexual partnering behavior. There are missing data - which I impute in R using mice. The imputations go off without issue.

The trouble begins when I need to ensure that a few impossible values are not present after the imputations. Specifically, in same-sex sexual partnering encounters males cannot have a positive response on the variable 'occurrence of vaginal intercourse.'

For a number of reasons the analyses for the project are divided by sex from the outset. So here I am only dealing with males (as in the starting data frame has already been subset by sex, for which there were no missing values).

Following the imputations, this anatomical impossibility shows up in a few places. My thoughts were to simply edit the values since there are very few (a couple across 5 imputed data sets with several thousand cases). I convert the mids object to a data frame, edit the values manually, and then attempt to convert back to a mids object using as.mids. At which point I get the message:

Error in ini$imp[[i]] : subscript out of bounds

The original data frame I'm dealing with at this point ("data" consisting of only male respondents) contains the following variables (plus a lot of others that aren't relevant here) -

respondentID, age, race, samesexENC, mutualmast, oralsex, vaginalsex, analsex, enjoy

After using mice() to do the imputations, the mids object is dataIMP. At which point I do the conversion to a data frame using complete(). I then subset down to identify the problematic rows and correct vaginalsex to "No".

# convert to a data frame in order to do edits
EDITthis <- complete(dataIMP,action="long",include=TRUE)
# 
# subset to identify samesexENC that included vaginalsex
q <- subset(EDITthis, vaginalsex=="Yes")
q <- subset(q, samesexENC=="Yes")
# 
# edit the cells in the frame directly so 
# that these equal "No" on vaginal (which is column 29)
EDITthis[5955, 29]="No"  
EDITthis[19656, 29]="No"

We're all clear at this point. Pretty simple stuff even for my beginner level R fluency. Then I want to convert back to mids and do the analyses, so I use as.mids() -

# convert back to mids object
dataFINAL <- as.mids(EDITthis)

This is where my full-steam-ahead day of data analysis came to a halt yesterday with the previously mentioned error message. I first went directly to the mice documentation. Looking at the arguments to as.mids() there doesn't seem to be any issues. The defaults for the .id and .imp arguments correspond to the correct columns in EDITthis. I went back to the documentation for complete() as well and cannot find any issues there. Finally I called to as.mids to eyeball the code behind the function. I believe I understand what's going on there, but nothing tipped me off to any potential issues in my code. Finally, as a safety measure, I re-installed the package all together, walked back through the entire process from the beginning, and the results were exactly the same - everything is fine up to the point where I want to turn the EDITthis data frame back into a mids object.

Probably a comma or quotes off somewhere, but I can't ferret it out. Thanks in advance!

Answer 1

It seems that names <- ls(ini$imp) sorts the naming of the list ini$imp alphabetically. Changing the names vector to names <- names(ini$imp) should fix this. We will fix this in the next version of mice. Thanks.

The function below should fix the issue, in the interim:

as.mids2 <- function(data, .imp=1, .id=2 ){
ini <- mice(data[data[, .imp] == 0, -c(.imp, .id)], maxit=0)
names  <- names(ini$imp)
    if (!is.null(.id)){
    rownames(ini$data) <- data[data[, .imp] == 0, .id]
    }
    for (i in 1:length(names)){
    for(m in 1:(max(as.numeric(data[, .imp])) - 1)){
      if(!is.null(ini$imp[[i]])){
        indic <- data[, .imp] == m & is.na(data[data[, .imp]==0, names[i]])
        ini$imp[[names[i]]][m] <- data[indic, names[i]]
  }
}
}
return(ini)
}

Answer 2

Here's a simplified version which should be similar enough:

### minimal reproducible example
library(mice)
set.seed(1)
df1 <- data.frame(age=20*abs(runif(10)),
                  vs=sample(c("y","n",NA), 10, replace=TRUE),
                  os=sample(c("y","n",NA), 10, replace=TRUE)
                  )

Now surely the easiest thing to do would be to replace the values at this stage before performing the imputation:

df1$vs[df1$vs=="y"] <- "n"

However if we skip this step and proceed with:

m1 <- mice(df1)
c1 <- complete(m1, action="long", include=TRUE)
c1$vs[c1$vs=="y"] <- "n"
as.mids(c1)

- it doesn't work as expected as we've manually altered c1. Using undebugonce(as.mids), you can see it throws a warning when indexing on vs (i=3).

Looking at the source for as.mids() it appears you're coming across a similar error, occurring here:

if(!is.null(ini$imp[[i]])))

The logical expression can't be evaluated because

names <- ls(ini$imp) 

appears to be longer than the no. entries in ini$imp (a list). (ini (an object of class "mids") is the initial object generated by as.mids).